Welcome NVIDIA Cosmos 3: The First Open Omni-model for Physical AI Reasoning and Action

NVIDIA Releases Cosmos 3 for Physical AI

NVIDIA has released Cosmos 3, an open-source foundation model designed to unify world simulation, physical reasoning, and action generation for physical AI systems. Available on Hugging Face, the model consolidates capabilities that previously required separate tools for tasks in robotics, autonomous vehicle simulation, and smart spaces. This single-model approach is intended to streamline development by eliminating the need to manage multiple inference pipelines, providing a unified framework for developers building systems that need to understand and interact with the physical world.

Built on a Mixture-of-Transformers (MoT) architecture, Cosmos 3 processes text, image, video, and action data within a single forward pass. The model uses a dual-subsequence structure, where an autoregressive component handles reasoning and a diffusion component manages generation. NVIDIA has released two versions to address different operational needs. The 8-billion parameter Cosmos 3 Nano is optimized for workstation-grade hardware like the RTX PRO 6000 GPU, while the 32-billion parameter Cosmos 3 Super is targeted at large-scale synthetic data generation and research on high-end NVIDIA Hopper and Blackwell GPUs.

• Model Architecture: Unified Mixture-of-Transformers (MoT) for reasoning and generation.
• Modalities: Supports text, image, video, audio, and action inputs and outputs.
• Versions: Cosmos 3 Nano (8B parameters) and Cosmos 3 Super (32B parameters).
• Availability: Openly available on Hugging Face with Diffusers integration, post-training scripts, and synthetic datasets.

By open-sourcing Cosmos 3 along with training scripts and synthetic datasets, NVIDIA is providing the AI community with a foundational tool for embodied AI. This release is positioned to accelerate innovation in fields that rely heavily on realistic simulations, such as warehouse automation and autonomous driving. The framework encourages post-training on custom data, enabling companies to adapt the model for specific robots, environments, and tasks, potentially establishing Cosmos 3 as a core component in the enterprise physical AI technology stack.

By releasing an open, end-to-end framework like Cosmos 3, NVIDIA is not just providing a model; it's strategically positioning its architecture and hardware as the default platform for the industrial and enterprise simulation market, aiming to standardize the toolkit for developing the next generation of physical AI.