LLM.265: Video Codecs Are Secretly Tensor Codecs

Video codecs have been refined for decades to compress visual data efficiently. This paper revisits them as general-purpose tensor codecs for large language models. LLM training and inference are increasingly limited by memory footprint and communication bandwidth, while existing tensor-compression methods are often specialized for weights, activations, gradients, or KV cache. We show that standard video codecs can compress these tensors effectively because tensor layouts expose statistical structure that codec stages such as prediction, transform coding, quantization, and entropy coding can exploit. Building on this observation, LLM.265 repurposes existing GPU video engines to support compression during both inference and distributed training. We further show that video-codec hardware can be customized and enhanced to improve tensor encoding and decoding throughput without substantial cost, making it an effective path for large-scale model deployment without major changes to GPU architecture.

Duke Scholars

Author Matthew Lentz Computer Science

Author Lisa Wills Computer Science