Gemma 4 VLA Demo on Jetson Orin Nano Super

Gemma 4 with Vision Runs Locally on NVIDIA Jetson Board

A new demonstration by NVIDIA developer Asier Arranz shows Google's Gemma 4 model operating as a Vision Language Assistant (VLA) entirely on a Jetson Orin Nano Super. The system can listen to a spoken query and autonomously decide whether to activate a webcam to gather visual context before answering. This successful local deployment is notable because it moves complex, multimodal AI capabilities from the cloud to a small, power-efficient edge device, enabling applications that require low latency and data privacy.

The project relies on a carefully integrated stack of open-source components running on the 8 GB Jetson board. User speech is captured and transcribed by the Parakeet STT model, fed to a quantized version of Gemma 4 (gemma-4-E2B-it-Q4_K_M.gguf), and the generated response is vocalized using Kokoro TTS. The core of the vision functionality is Gemma 4's native tool-calling, which is exposed to a single function: `look_and_answer`. The model is served using a custom-built `llama.cpp` with CUDA support to maximize performance on the Orin Nano's GPU, and the tutorial provides detailed steps for managing system memory to run the demanding workload.

This implementation serves as a practical blueprint for developers building on-device agents that can perceive and interact with their environment. By running the entire perception-reasoning-action loop locally, this approach is well-suited for robotics, interactive kiosks, and accessibility tools where real-time performance and offline functionality are critical. The project underscores the viability of using quantized models on specialized hardware to bring sophisticated AI features, previously confined to data centers, into consumer and industrial edge products.

• Model: Gemma 4 (unsloth/gemma-4-E2B-it-GGUF, Q4_K_M quant)
• Hardware: NVIDIA Jetson Orin Nano Super (8 GB)
• Inference Server: `llama.cpp` compiled with CUDA support
• Pipeline: Parakeet STT → Gemma 4 (VLA) → Kokoro TTS
• Key Feature: Autonomous tool use for vision, without hardcoded keyword triggers.

The successful deployment of a vision-capable agent on an 8 GB Jetson device demonstrates that the combination of model quantization, efficient inference engines like `llama.cpp`, and specialized hardware is effectively dissolving the boundary between cloud-only AI capabilities and practical edge computing applications.