Why was the NVIDIA Nemotron 3.5 Bolt E4B model chosen for this project?

The Bolt E4B model was selected because it is an open small language model (SLM) optimized for local conversational agents. Its small footprint, using 4-bit quantization, allowed the entire system to run self-contained on a single workstation, effectively managing the memory and compute budget while providing the necessary conversational depth and enabling personality customization through a LoRA adapter.

nemotron-halloween

NVIDIA Engineer Builds On-Device AI Haunted House with Nemotron Models

An engineer at NVIDIA, Rick Grandy, has created an interactive Halloween installation named MonsterHouse, demonstrating the capabilities of the company’s Nemotron open model stack running entirely on a single local workstation. The project transforms a garage door into a grumpy, interactive character that converses with visitors, showcasing a practical application of a fully integrated, on-device AI agent. This self-contained system highlights the growing trend of moving complex AI workloads from the cloud to the edge for real-time, low-latency applications.

The On-Device AI Stack

MonsterHouse is powered by a coordinated pipeline of NVIDIA technologies, with the workload split across two professional Blackwell-generation GPUs to ensure smooth performance. The project's creator chose the NVIDIA Nemotron 3.5 Bolt E4B 4GB, a small language model (SLM), for its efficiency and suitability for local conversational agents. A custom LoRA adapter trained on over 5,000 conversations was used to imbue the house with its unique, cantankerous pirate personality.

Coordination: NVIDIA Nemotron Voice Agent orchestrates the entire pipeline.
Listening: NVIDIA Nemotron 3 ASR handles speech-to-text conversion.
Thinking: NVIDIA Nemotron 3.5 Bolt E4B 4GB generates conversational replies.
Speaking: NVIDIA Magpie TTS Zeroshot provides the character's voice from a short audio sample.
Animating: NVIDIA Audio2Face-3D drives the mouth animation in Unreal Engine 5.
Hardware: The agent pipeline runs on an NVIDIA RTX PRO 6000 Blackwell Max-Q, while graphics are handled by an NVIDIA RTX PRO 4000 Blackwell.

From Hobby Project to Edge AI Blueprint

While a creative holiday project, MonsterHouse serves as a significant proof-of-concept for the development of sophisticated, stateful AI characters outside of cloud-based environments. The ability to run vision, language, speech, and animation models concurrently on local hardware opens possibilities for industries requiring responsive and private AI, including robotics, interactive entertainment, and character design. The use of an SLM like Bolt E4B, customized with a LoRA, provides a blueprint for creating bespoke AI personalities that are both computationally efficient and highly tailored.

This project demonstrates that a complete, multi-modal AI agent with a distinct personality can be deployed on consumer-grade hardware, effectively lowering the barrier for creating complex, real-time interactive experiences without reliance on external APIs.

>> Verify Original Transmission at Hugging Face