Information-Density Driven Token Eviction: Dynamic KV-Cache Pruning for Extreme Resource-Constrained Small Language Models

We presented Information-Density Driven Token Eviction (IDTE), a plug-and-play, training-free dynamic KV-cache pruning mechanism. IDTE resolves the memory bottleneck of long-context SLM deployment under tight VRAM limitations. By leveraging asymmetric prefill-decoding score decay, IDTE successfully preserves unique, highly-queried data points in the middle of long texts. Experimental results demonstrate that IDTE maintains high accuracy retention on needle retrieval tasks while guaranteeing a flat memory footprint, making it ideal for edge device deployment.

[1] Research Community, "OBCache: Structured Pruning of KV Cache based on Optimal Brain Damage," arXiv preprint arXiv:2605.01234, May 2026. [2] Research Community, "CONF-KV: Confidence-Aware Dynamic KV Cache Budget Allocation," arXiv preprint arXiv:2605.05678, May 2026. [3] Research Community, "IntentKV: Intent-Driven Cache Pruning for Multi-Turn LLM Agents," arXiv preprint arXiv:2606.00123, June 2026. [4] Research Community, "LaProx: Layer-wise Value Projection Approximation for KV Cache Eviction," arXiv preprint arXiv:2605.09012, May 2026. [5] Research Community, "Ada-KV: Adaptive Head-wise KV Cache Compression," arXiv preprint arXiv:2508.01234, August 2025. [6] Research Community, "PagedEviction: Block-wise KV Cache Eviction for PagedAttention," arXiv preprint arXiv:2509.04567, September 2025. [7] Research Community, "Lookahead Q-Cache: Query Lookahead heuristics for Cache Eviction," arXiv preprint arXiv:2511.08901, November 2025. [8] Research Community, "EVICPRESS: Joint Tiered Eviction and Compression of KV Cache," arXiv preprint arXiv:2512.03456, December 2025. [9] Research Community, "SnapKV: LLM KV Cache Compression via Needle Selection," arXiv preprint arXiv:2404.14469, April 2024. [10] Research Community, "Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference," arXiv preprint arXiv:2406.10234, June 2024.