AUE: A Normalized Energy Efficiency Metric for AI Servers Under LLM Workloads

Under the rapid advancement of large model-driven artificial intelligence, the surging energy consumption of AI training and inference tasks has created an urgent need for precise and comparable energy efficiency metrics to guide the design and deployment of green computing systems. While existing metrics such as PUE and Green500 metrics focus on infrastructure or traditional numerical computations, they cannot reflect the characteristics of AI workloads. Although applicationoriented metrics like J/response and J/token are designed for LLMs, they remain susceptible to biases induced by model scale and output strategies, lacking cross-model comparability. This paper proposes a novel AI energy efficiency metric, AUE, defined as the energy consumed per thousand tokens per billion activated parameters. By normalizing model size effects, AUE accurately reflects the energy efficiency of underlying computational resources. We theoretically justify the validity of AUE and conduct experiments on a server equipped with 4 × Ascend NPU 910C accelerators, evaluating dense Transformer and MoE architectures across both training and inference workloads. Experimental results demonstrate that traditional J/token metrics disproportionately favor smaller models, whereas AUE reveals true energy utilization efficiency. For instance, while Qwen3 0.6B shows superior J/token values compared to Qwen3 14B, the 14B model achieves a significantly better AUE of 12.88 J/(KToken GParam) versus 24.35 J/(KToken GParam) for the 0.6 B model, consistent with measured FLOPs where the 14B model outperforms its smaller counterpart. With advantages including simple measurement procedures and compatibility across platforms and model architectures, AUE provides a viable pathway toward establishing a normalized and standardized AI energy efficiency evaluation framework.