NVIDIA is using full-stack codesign to optimise memory use across its GPUs, CPUs, and networking systems. The company claims its Vera Rubin NVL72 platform delivers up to 30 times higher throughput per dollar than the previous GB300 system. This approach allows the firm to better manage memory constraints within its accelerated computing stack.
The new platform introduces HBM capacity per GPU that is roughly 3 percent higher than previous generations. Despite this modest increase in capacity, the throughput per dollar has risen significantly. On the SemiAnalysis AgentX workload, the GB300 NVL72 provides up to 10 times higher throughput per dollar than the H200 platform. The company was the first to adopt HBM3e and HBM4 technologies into its accelerated computing stack.
NVIDIA is shrinking model weights and offloading inactive KV cache to lower-cost storage tiers to relieve pressure on HBM. This strategy helps maintain performance while managing memory costs. Developers and data centre operators should pay attention to these efficiency gains. You can expect more cost-effective AI deployments as these memory optimisation techniques become standard across the industry.
