What makes the STAC-AI benchmark different from other LLM performance tests?

The STAC-AI LANG6 benchmark is tailored specifically for financial workloads. It uses custom datasets derived from real SEC 10-K filings to simulate tasks like financial analysis and summarization. It also requires on-the-fly chat template application and tokenization during inference and measures both high-throughput batch processing and real-world interactive scenarios, reflecting how these models are deployed in trading and investment advisory systems.

NVIDIA Blackwell Sets STAC-AI Record for LLM Inference in Finance

NVIDIA's latest Blackwell architecture has established a new performance record on the STAC-AI LANG6 benchmark, a standard specifically designed to measure large language model inference for the financial services industry. The results, using Meta's Llama 3.1 models, point to a significant leap in processing capability for computationally intensive tasks such as analyzing and summarizing complex regulatory filings, which are crucial for generating timely trading insights.

The benchmarks evaluated both high-throughput batch processing and interactive, real-time scenarios using custom datasets derived from EDGAR 10-K financial reports. A cloud-based node of an NVIDIA GB200 NVL72 system, leveraging NVFP4 quantization via the TensorRT LLM framework, was benchmarked against prior-generation Hopper systems. According to the data, a single Blackwell GPU delivered up to 3.2 times the throughput of a single Hopper GPU. Furthermore, the Blackwell system sustained better interactivity, achieving lower reaction times and inter-word latency even at higher throughput levels.

This performance improvement is directly relevant for financial institutions adopting AI for market analysis, algorithmic trading, and risk assessment. The ability to run larger, more sophisticated models with greater speed and efficiency can enable more complex automated strategies and faster extraction of actionable intelligence from unstructured data. The results establish a new hardware baseline for firms seeking to maintain a competitive advantage through advanced AI, potentially accelerating the upgrade cycle for specialized compute infrastructure within the sector.

The STAC-AI results demonstrate that the value of new AI hardware like Blackwell isn't just in raw throughput, but in its ability to maintain low-latency, interactive performance under heavy loads. For financial firms, this translates to a tangible advantage where the speed of processing and reacting to new information is paramount.