Gemini Robotics ER 2: powering robotics with video understanding, task orchestration, and multi-robot collaboration

Google Releases Embodied Reasoning Model for Advanced Robotics

Google has released Gemini Robotics ER 2, an embodied reasoning model designed to function as a high-level orchestrator for physical robots. The release focuses on advancing robotic capabilities in dynamic, real-world settings by integrating continuous video understanding for progress monitoring, enabling multi-robot collaboration, and ensuring low-latency decision-making. This move targets core challenges in robotics, shifting the focus from static task execution to adaptive, multi-step workflows where robots can perceive their environment over time, correct mistakes, and interact more safely and effectively.

Technical Capabilities and Performance

Gemini Robotics ER 2 operates as a central 'brain' that processes multimodal data streams—including video, audio, and text—to plan complex tasks. It then delegates motor control to lower-level Vision-Language-Action (VLA) models or other control APIs, such as those used by Boston Dynamics' Spot robot. A key technical feature is its integration with the Gemini Live API, which uses a bidirectional streaming endpoint to reduce latency and eliminate the 'stop-and-think' pauses common in robotic operations. This architecture allows the robot to reason about its next move while simultaneously executing a current action. According to Google, the model shows significant performance improvements over its predecessor, Gemini Robotics ER 1.6, and other frontier models on several key benchmarks.

• Temporal Intelligence: The model achieves 57.4% accuracy on progress classification from continuous video and 91.3% accuracy on moment-finding, which identifies the precise frame a critical event occurs.
• Task Orchestration: Outperforms the previous ER 1.6 model across real, simulated, and human-controlled (tele-op) VLA orchestration tasks.
• Spatial Intelligence: Enhances general instrument reading across 10 different types, including digital and linear scales, and improves success/failure detection by analyzing entire video feeds instead of static images.
• Safety: Demonstrates improved performance on Safety Instruction Following and Human Proximity benchmarks, enabling robots to halt operations when a person is nearby.

The release signals a focus on building more autonomous and collaborative systems that can operate in unstructured environments. By enabling different types of robots, such as Apptronik's Apollo 2 humanoid and the Franka F3 Duo, to coordinate on a single workflow, Google is addressing the practical need for specialized hardware to work in concert. This emphasis on real-time adaptation and multi-agent orchestration could accelerate the deployment of robots in logistics, manufacturing, and research by making them more reliable, versatile, and aware of their surroundings.

Google's focus on temporal video understanding and low-latency orchestration in Gemini Robotics ER 2 indicates the industry is shifting from single-task, single-agent robotics towards more practical, collaborative systems that can self-correct and operate safely in dynamic environments.