**Know Who Spoke When: Build Real-Time, Multi-Speaker AI with NVIDIA Nemotron 3 Diarization**

NVIDIA Releases Nemotron 3 Diarization Model

NVIDIA has released Nemotron 3 Diarization, an open-weight model designed to identify who spoke when in complex, real-time conversations. The model addresses a critical need for applications that rely on understanding conversational dynamics, such as meeting summaries, customer call analytics, and voice agents. By accurately attributing speech to participants, even with overlapping dialogue, it provides the foundational data required for higher-level analysis. The model has already demonstrated its capability by ranking first in Voice Arena's initial Diarization-Bench results, showing a 14.72% Diarization Error Rate (DER) and outperforming the next-best system by a relative 24%.

Technical Details and Performance Gains

The 100-million-parameter model builds on NVIDIA's Sortformer architecture, which orders speakers based on their first appearance to maintain stable speaker labels throughout a conversation. This is particularly effective for streaming applications where audio is processed in chunks. A key advantage is its ability to operate at multiple latency points, from an offline-style 30.4 seconds down to a recommended 0.32 seconds, allowing developers to balance accuracy with responsiveness. Training included licensed, real-world conversational data from David AI, which contributed to a significant reduction in error rates. Compared to NVIDIA's prior four-speaker model, Nemotron 3 Diarization shows an average 41% relative DER reduction across eight public benchmarks at a 1.04-second latency.

• Model Size: 100M parameters
• Speaker Support: Up to eight concurrent speakers
• Architecture: 31-layer Transformer with Rotary Positional Embeddings (RoPE)
• Input: 16 kHz, single-channel audio
• Key Feature: Handles overlapping speech and offers configurable streaming latency
• Benchmark: #1 on Diarization-Bench with 14.72% DER (0ms collar)

Impact on the AI Application Ecosystem

By releasing Nemotron 3 Diarization as an open-weight model, NVIDIA provides developers with a powerful tool to build more sophisticated voice-based AI systems. The model's explicit separation from automatic speech recognition (ASR) allows for modular pipeline construction, where developers can combine best-in-class components for diarization and transcription. This improved accuracy and support for up to eight speakers enables more reliable applications in multi-participant environments like enterprise meetings and group calls, where older systems often struggled. The accessibility of the model is likely to accelerate development of next-generation conversational AI that can track commitments, objections, and action items with greater precision.

NVIDIA's release of a specialized, high-performance, open-weight model like Nemotron 3 Diarization underscores a strategy to provide critical, foundational software components across the entire AI application stack, encouraging developers to build more deeply within its ecosystem beyond the hardware layer.