Vibeleaderboard
← All Intel
Intel / article

Beyond FLOPs: Energy-Aware Knowledge Distillation for Sustainable LLMs on Code-Related Task

Source
arxiv.org
Author
Enrique Barba Roque, Lu\'is Cruz, Annibale Panichella
Date

Context

Counting arithmetic operations can mislead developers trying to reduce a code model's electricity use. This study compares that proxy with models that estimate CPU and GPU energy directly while selecting smaller models trained from larger ones. The experiments cover code-clone detection, vulnerability prediction and code summarization.

The authors report -energy reductions of up to 90% and memory reductions of 86%, with modest accuracy trade-offs. These are task-specific results, but they support a concrete design choice: optimize for estimated energy consumption when energy is the objective, rather than assuming fewer mathematical operations always produce the lowest-energy model.

Terms in this piece · Glossary
  • inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
  • distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
Recommended reads
Comments

Checking sign-in…

Loading comments…