NVIDIA Achieves Leading Agentic Coding Performance on First Agentic AI Benchmark

NVIDIA Sets Bar on First Agentic AI Benchmark

NVIDIA has established a significant performance lead on the industry's first benchmark for agentic AI workloads, a critical and rapidly growing segment of the market. According to results from the newly launched Artificial Analysis AgentPerf (AA-AgentPerf) benchmark, the NVIDIA GB300 NVL72 system delivers up to 20 times more concurrent agent throughput per megawatt compared to the prior generation H200. The benchmark's introduction is significant because it provides the first standardized method for measuring hardware performance on complex, multi-step AI tasks, moving beyond simple inference to evaluate how systems handle realistic coding agent trajectories.

How AA-AgentPerf Measures Real-World Performance

AA-AgentPerf measures the number of concurrent AI agents an inference system can support while meeting specific Service Level Objectives (SLOs) for token speed and response time. Its methodology is notable for using private, prerecorded agent trajectories that capture the non-deterministic nature of AI agents, including reasoning steps and simulated CPU-side tool calls. This approach prevents benchmark-specific optimization and provides a more accurate reflection of real-world data center workloads. The results for the NVIDIA GB300 NVL72 highlight a substantial leap in efficiency, crucial for large-scale deployments.

• Concurrent agents per megawatt: 61.4K for the GB300 NVL72 vs. 2.6K for the H200.
• Concurrent agents per GPU: 57.5 for the GB300 NVL72 vs. 1.4 for the H200.
• Key Software Optimizations: The performance gains are supported by optimizations in runtimes like TensorRT-LLM, including WideEP/DeepEP for Mixture-of-Experts models and DeepGEMM for improved compute.
• Key Hardware Features: The high-bandwidth NVLink fabric connecting 72 GPUs is critical for coordinating execution across thousands of agent sessions.

The Impact on AI Infrastructure Planning

The benchmark results provide data center operators and cloud providers with a clear, power-normalized metric for capacity planning as they prepare for the rise of agentic AI applications. By demonstrating a strong performance-per-watt advantage, NVIDIA is positioning its Blackwell architecture as the foundation for economically viable, large-scale agent deployments. The company also signaled its future roadmap, projecting that the upcoming NVIDIA Vera Rubin platform will further extend these gains by accelerating LLM tool calls and leveraging 50 PFLOPs of NVFP4 compute, indicating a sustained focus on optimizing for this complex workload.

The release of the AA-AgentPerf benchmark and NVIDIA's immediate dominance isn't just a performance update; it's a strategic move to define the key performance indicators for the next generation of AI infrastructure, centering the conversation around power efficiency and concurrent agent capacity at scale.