Real-time control for power cost efficient deep learning processing with renewable generation

The explosive increase in deep learning (DL) deployment has led GPU power usage to become a major factor in operational cost of modern HPC clusters. The complex mixture of DL processing, fluctuated renewable generation, and dynamic electricity price impedes the elaborate GPU power control, so as to lead an undesirable cost. However, most previous studies have been concerned only with the design of power management method using DL, and have not care about the cost caused by GPU power consumption for DL processing itself. This paper, as the opposite direction of these trends, proposes a real-time power controller called DeepPow-CTR for cost efficient DL processing in GPU based clusters. We design the GPU frequency scaling algorithm based on model predictive control (MPC), to delicately tune the DL power consumption in response to dynamic renewable generation and electricity price. At the same time, we avoid the unacceptable DL performance degradation by regulating memory-access / feed-forward / back-propagation (MFB) time per each minibatch data in deep neural network (DNN) model training. To solve the designed nonlinear MPC problem rapidly and accurately, we apply the damped Broyden-Fletcher-Goldfarb-Shanno (BFGS) based sequential quadratic programming (SQP) method to our DeepPow-CTR. Our experimental results on lab-scale testbed using real trace data of renewable generation and electricity price, demonstrate that the proposed DeepPow-CTR has superiority and practicality in terms of DL processing power cost and performance, compared to existing methods.