NeoMME: an efficient Multimodal-native and Multilingual Encoder

Hcompany Releases NeoMME, a Multimodal-Native Encoder Challenging Generative VLM Architectures

Hcompany has released NeoMME, a new family of multilingual and multimodal encoders aimed at providing a more efficient alternative to large visual language models (VLMs) for retrieval tasks. By processing images and text through a single, unified Transformer trained from scratch, NeoMME bypasses the common architecture of stitching together separate, pretrained vision and language components. This design choice targets improved performance and throughput for enterprise applications like visual document retrieval, where encoding speed and cost are critical.

Technical Specifications

The NeoMME family, available in 260M and 800M parameter sizes, is built on a modern bidirectional encoder architecture rather than a generative, causal decoder. This approach is specifically tailored for tasks that don't require text generation, thereby reducing parameter and compute overhead. The model is trained from the ground up using a masked discrete-diffusion objective, which forces it to learn strong image-text associations without relying on pretrained components.

• Unified Architecture: A single bidirectional Transformer processes both text tokens and raw 32×32 image patches.
• Long Context: Supports a 16,384-token context length, enough to handle high-resolution document images.
• Dual-Output Retriever: The fine-tuned NeoMME-Retriever generates both compact dense embeddings and high-granularity late-interaction embeddings in a single forward pass.
• Multilingual: Trained on a custom 131k-token BPE vocabulary covering multiple languages, code, and mathematics.
• Open Source: All model checkpoints are available on Hugging Face under an Apache 2.0 license.

On the ViDoRe v3 benchmark, NeoMME-Retriever-260M achieves a top-tier nDCG@10 score of 0.523 for its size class, outperforming ColModernVBERT and challenging models nearly 14 times its size. Beyond retrieval quality, the model emphasizes practical deployment efficiency. On an NVIDIA L40S GPU, the 260M model processes 51 pages per second at 2048×2048 resolution, nearly double the throughput of ColModernVBERT. Furthermore, Hcompany has demonstrated significant index compression, reducing the storage for late-interaction embeddings by up to 255x with minimal impact on retrieval quality, addressing a key bottleneck for deploying high-resolution document search at scale.

Strategic Takeaway

Hcompany's NeoMME bets on architectural efficiency, demonstrating that specialized, from-scratch encoders can outperform adapted generative models on tasks like visual document retrieval. This challenges the 'one-VLM-fits-all' approach and could signal a market shift towards purpose-built models for high-throughput enterprise applications where inference cost and speed are paramount.