Improving the Energy Efficiency of AI Clusters Through Variability-Aware Frequency Scaling and Task Allocation

The rapid development of AI technologies has significantly intensified the energy consumption challenge faced by AI clusters. Dynamic Voltage and Frequency Scaling (DVFS) stands as a crucial approach for managing power consumption in GPUs and NPUs. However, conventional DVFS strategies often assume uniform performance across computational units, overlooking the heterogeneity in performance and power caused by chip manufacturing variability. This variability leads to imbalanced task execution times across different processors, consequently reducing overall energy efficiency. Traditional task allocation strategies also fall short in leveraging the energy efficiency characteristics of processors, limiting the optimization potential of AI clusters. To tackle these challenges, we propose the VAFSA framework. VAFSA employs a variability-aware DVFS strategy that enables GPU/NPUs to adjust their operating frequencies to ensure synchronized computation times. VAFSA also features a task allocation strategy that prioritizes selecting processors with higher energy efficiency to further reduce energy consumption. Our simulation based on Ascend NPUs and NVIDIA GPUs demonstrates that VAFSA can reduce energy consumption of AI workloads by 3.5% to 44% in diverse scenarios.