InstructPrune

Adaptive visual-token pruning that trims a vision-language model's compute to match instruction difficulty

Problem

Vision-language models spend the same visual-token budget on a simple caption request and a fine-grained counting question, wasting compute on easy instructions.

Solution

Training-free framework that estimates instruction complexity up front and adapts the visual-token pruning ratio accordingly, evaluated on VQAv2, MMStar, and SEED-Bench for accuracy, latency, GPU memory, and attention-computation savings.

Stack: PyTorchHugging Face TransformersspaCyLLaVA-1.5-7B
Last updated: June 2026