Building an Adaptive Agentic Cybersecurity System with NVIDIA Nemotron

NVIDIA and CrowdStrike Automate Cyber Defense with Agentic AI

NVIDIA and CrowdStrike have detailed a joint evaluation of an adaptive agentic cybersecurity system that fully automates offense-defense testing in a continuous, closed loop. The system leverages specialized NVIDIA Nemotron open models to operate at machine speed, aiming to bridge the gap between identifying security vulnerabilities and deploying reliable coverage. This approach moves beyond conventional security operations by creating a dynamic feedback loop where AI-powered red and blue agents repeatedly test, detect, and adapt to threats within a realistic, isolated computing environment, significantly reducing the manual effort and time typically required for such exercises.

A Validation-First Agentic Architecture

The project's technical core is a four-stage workflow built around a defensive harness that integrates multiple validation mechanisms to ensure detections are both effective and robust. In the evaluated setup, Nemotron 3 Ultra was used for high-level defensive orchestration, while a fine-tuned Nemotron 3 Super model served as a bounded expert for generating and repairing detection queries. This specialized harness enforced correctness through a combination of automated checks and reinforcement learning trained with verifiable rewards using NVIDIA NeMo Gym and NVIDIA NeMo RL.

• Schema Knowledge Base: Prevented agents from inventing invalid query fields.
• Telemetry Grounding: Anchored analysis in observed event data from CrowdStrike Falcon sensors.
• Specialized Detection Authoring: Used a customized model specifically for code generation and repair.
• Artifact Linting: Automatically rejected detections hardcoded to specific environment details.
• Detection Replay: Backtested every candidate detection against captured attack telemetry.
• Independent Review: Employed a separate AI judge to evaluate detections for quality and robustness.

Performance Metrics Point to Specialization Advantage

The evaluation demonstrated a quantifiable performance advantage for the specialized open-model approach. In backtesting, the optimized pipeline achieved a 41.9% mean detection rate, a 2.5x improvement over a default configuration using Nemotron 3 Ultra alone. More significantly, in live-fire tests against eight previously unseen attacks, 45% of the detections generated by the open-model system successfully generalized, compared to only 29% from a leading proprietary frontier model. The system produced three 'gold' tier detections that covered all eight attacks, a benchmark the frontier model did not achieve, suggesting that domain-adapted open models can outperform larger, general-purpose systems in highly specialized and critical applications like cybersecurity.

The key takeaway from this evaluation is not just the automation of red and blue teaming, but the demonstrated effectiveness of using smaller, specialized open models that are rigorously fine-tuned and validated for a specific, high-stakes domain. This validation-first approach shows a path to creating more reliable and generalizable AI systems that can outperform larger, more generalized frontier models in targeted enterprise applications.