Dive Brief:
- Nvidia has unveiled that some of its rack-scale systems will now feature a new power supply unit designed to manage energy more efficiently, claiming it could cut peak grid demand by up to 30%.
- The company announced this in a blog post, stating that this new power supply, along with necessary hardware and software, will be part of the new GB300 NVL72 platform and GB200 NVL72 systems. Nvidia did not share specific release dates or further details.
- Santiago Grijalva, a professor at Georgia Tech, described the advancement as notable, considering Nvidia’s significant influence in the industry. However, he pointed out that this solution is largely applicable to Nvidia’s high-end systems and competes with technologies from companies like Tesla and Meta, offering improvements but not groundbreaking changes.
Dive Insight:
AI-focused data centers face unique challenges due to their fluctuating power needs, which have been likened to those of a steel mill. This poses difficulties for grid operators and utilities striving to meet the high energy demands.
Nvidia believes its new system can help manage these demands with a strategy that covers three key operational phases: ramping up, maintaining a steady state, and ramping down. The system includes a power cap at the beginning of workloads that gradually increases. Once stable operation is reached, energy storage helps regulate short-term power spikes. For the ramp-down phase, a special power mode for GPUs ensures a smooth decline in power usage instead of a sudden drop.
It’s important to note that the energy storage component primarily optimizes the load profile for the grid and does not return energy to the utility.
Nvidia emphasized that previously, facilities needed to be prepared for maximum power consumption. Now, with effective energy storage, they can be designed closer to average power needs. This means more racks can fit within the same power budget or allow for lower overall power allocations.
Grijalva acknowledged the importance of managing power use for AI, stating that advancements in energy storage and management are beneficial. However, he added that Nvidia’s system does not tackle transmission challenges, which are critical for supplying large amounts of power to data centers.
He also noted that as AI models evolve, their power requirements will shift, requiring adaptable and forward-thinking solutions.