Efficient AI Systems
On-device LLM inference through adaptive sparsity
We study how to make large language models practical on resource-constrained devices by exploiting activation sparsity—the computations that a model does not need for a particular input. Our work spans the full inference stack: predicting sparsity before expensive operations begin, translating those predictions into skipped memory accesses and computation, and adapting the amount of sparsity at runtime so that efficiency gains do not come at an uncontrolled cost to model quality.
Students: Ethan Lin · Suri Wang
Collaborators: Youngmin Yi · Joon-Sung Yang · Youngpyo Joo · Chae Eun Rhee
Selected publications
Adaptive sparsity control
ACAS treats sparsity as a runtime control problem. It monitors a quality signal and adjusts sparse execution online, allowing an on-device system to respond to changing inputs and operating conditions instead of relying on one fixed threshold.
Structured FFNs for fast, accurate inference
oFFN jointly organizes activation outliers and hot and cold neurons within the FFN. This structure supports accurate sparsity prediction and selects efficient computation paths as batch size changes, translating sparsity into practical end-to-end acceleration.
Efficient activation-sparsity prediction
SparseInfer uses a lightweight, training-free predictor to expose activation sparsity before computation. Grasp extends this direction with group-based, magnitude-aware prediction, improving the balance between prediction overhead and retained model quality.
Hoeseok Yang
Associate Professor of Electrical and Computer Engineering
My research interests include hardware/software co-design of AI systems, system security, and embedded system design methodology.