Energy as a first-class software design metric
We define software performance engineering (SPE) as “making software run fast or otherwise consume few resources such as time, storage, energy, network bandwidth, etc.” I’m focusing on energy today.
AI and Energy
In response to the massive carbon footprint of AI, we owe it to ourselves (and our children) to use SPE. But, when we optimize performance in the traditional way — for “time consumption” — we can adversely affect energy consumption. For example, see this paper on understanding and optimizing energy consumption.
Parallelism and Time
The time-energy tradeoff reminds me of parallelism and runtime. I learned years ago that if I want my algorithm to run faster, then parallelizing it might help — or it might add overhead and make everything slower! To put it bluntly: “the whole point of parallelism is performance” (for me, anyway), and so I have to be sure that I only implement parallelism when I’m helping performance.
Performance and Energy
Similarly, if my whole reason for optimizing performance is conserving energy, then I have to be sure that I only optimize performance in ways that help conserve energy. But Fastcode traditionally focuses on resources for measuring and improving time consumption. What about energy? We need principles, metrics, and tools that are specifically designed for optimizing energy.
Toward Energy-Optimal AI
If you’re curious about optimizing energy, please join me at the free virtual Fastcode Seminar on May 6. Mosharaf Chowdhury will introduce the ML.ENERGY Initiative, his team’s effort to understand and curtail AI’s runaway energy demands on three fronts:
Where energy goes: I will present tools to precisely measure AI energy consumption and findings from benchmarking open-weight models across hardware and serving configurations via the ML.ENERGY Leaderboard.
Optimizing energy use: I will describe how identifying computations on and off the critical path in distributed training enables precise GPU frequency control, saving energy on non-critical work without slowing down training.
Exposing tradeoffs: I will present how co-optimizing static and dynamic energy through better kernel scheduling reveals the Pareto frontier between energy and performance, enabling practitioners to make informed deployment decisions under diverse constraints.
Come to the Fastcode Seminar
“Toward Energy-Optimal AI” by Mosharaf Chowdhury, 11AM EDT, Wednesday, May 6, 2026.