Frame Skipping Architecture for Video-Language Model Acceleration
Video-Language Models (VLMs) have achieved strong performance in video understanding, enabling a wide range of applications. However, the high computational and memory costs of Multimodal Large Language Models (MLLMs), combined with the rapid growth in sequence length introduced by video inputs, hinder efficient deployment, particularly on resource-constrained edge platforms. To address these challenges, we propose a modular extension for VLM accelerators that incorporates a frame-skipping unit prior to MLLM processing. This module dynamically selects informative frames and suppresses redundant ones, effectively reducing unnecessary computation while preserving task-relevant information. The proposed design operates in conjunction with existing VLM accelerator backbones, providing an efficient and edge-friendly solution for video-language tasks. Experimental results show that our method achieves up to 2.42 × speedup and 2.46 × improvement in energy efficiency compared to the baseline, while incurring only a small accuracy degradation, demonstrating a favorable trade-off between efficiency and performance.
