Counting arithmetic operations can mislead developers trying to reduce a code model's electricity use. This study compares that proxy with models that estimate CPU and GPU energy directly while selecting smaller models trained from larger ones. The experiments cover code-clone detection, vulnerability prediction and code summarization.
The authors report inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition →-energy reductions of up to 90% and memory reductions of 86%, with modest accuracy trade-offs. These are task-specific results, but they support a concrete design choice: optimize for estimated energy consumption when energy is the objective, rather than assuming fewer mathematical operations always produce the lowest-energy model.
Terms in this piece · Glossary
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.