Artificial Intelligence Review · 2026
From model miniaturization to edge deployment
On-device large language models
A unified survey of model compression and system optimization for private, responsive, and resource-aware LLM intelligence at the edge.
Why this survey
On-device intelligence is a full-stack problem.
A smaller model is not automatically a deployable model. Useful edge intelligence emerges when compression, runtime, memory, hardware, and application constraints are optimized together.
01
Model-side efficiency
Reduce parameters, precision, and computation while preserving useful capability.
- Quantization
- Pruning
- Knowledge distillation
- Low-rank factorization
- Hybrid compression
02
System-side execution
Turn compact models into responsive applications under real device constraints.
- Compiler optimization
- Inference frameworks
- Memory & KV cache
- Hardware support
- Edge-cloud collaboration
The research map
A field moving from isolated techniques to co-design.
The literature has grown from individual compression methods into a dense ecosystem of algorithms, systems, and deployment targets. Click each original paper figure to inspect it in full resolution.
Living collection
The paper is a snapshot. The repository keeps moving.
The companion Awesome List extends the published survey with new papers, code links, venues, and community contributions.
Explore the complete taxonomy →ALEM
Measure what deployment actually costs.
The survey proposes a shared evaluation lens for comparing methods across devices and workloads. No single number captures whether an on-device LLM is genuinely useful.
Learn about the protocol →ALEM
Deployment takeaways
Three ideas to carry into the next system.
Start with the bottleneck.
Choose compression and runtime techniques only after profiling memory bandwidth, compute, latency, and energy on the target device.
Treat KV cache as a subsystem.
Long-context deployment shifts pressure from model weights to cache capacity, bandwidth, paging, and scheduling.
Co-design beats local optimum.
The best model in isolation may lose once kernels, accelerators, thermal limits, and application quality are considered together.
Use the work
Read, browse, cite, and contribute.
The journal article provides the synthesis. The repository keeps the map current.
BibTeX
Cite the survey
@article{chen2026ondevice,
title = {On-device large language models: a survey of model compression and system optimization},
author = {Chen, Wanyi and Wang, Junhao and Zhang, Yiwei and Shi, Yufan and Jiang, Tianyi and Zhou, Shengxian and Wu, Chenxu and Zhang, Andi and Zhou, Chenyue and Wang, Minxuan and Liu, Xinyu and Hao, Xiaoshuai and Wu, Yinan and Li, Yichen and Hu, Yuwei and Cao, Zhao and Lu, Yang and Li, Mengke and Ma, Yanbiao and Lu, Zhiwu and Han, Jungong and Guo, Yike},
journal = {Artificial Intelligence Review},
year = {2026},
month = may,
doi = {10.1007/s10462-026-11538-1}
}