Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps

Small Model, Big Improvement

Researchers at LiquidAI have released a public guide demonstrating a highly efficient method for improving a small language model's ability to generate structured data. By fine-tuning the 350-million-parameter LFM2.5-350M model with Group Relative Policy Optimization (GRPO), the team boosted its performance on the IFStruct benchmark from 22.6% to 29.7%. This 7.1-point improvement is significant because it was achieved with a minimal resource footprint—just 100 training steps on approximately 500 data samples, a task manageable on a free-tier cloud GPU.

The tuning process, detailed in a publicly available notebook, utilized the TRL library to implement GRPO. A LoRA adapter was applied to the base model, training only 6 million parameters, or 1.66% of the model's total. The training data, sourced from NVIDIA's Nemotron dataset, was augmented to teach the model how to handle fenced code blocks and generate bare lists, addressing common gaps between training and evaluation data. The fine-tuning was guided by a weighted combination of three reward functions that scored outputs on JSON format correctness, expected field counts, and JSON Schema validation. Evaluation of both the base and fine-tuned models was conducted locally on an Apple M5 Max-powered MacBook using llama.cpp to ensure a consistent comparison.

This experiment highlights the growing viability of smaller, specialized models for critical real-world applications. The ability to reliably generate parseable, schema-compliant output is a fundamental requirement for integrating LLMs into automated workflows and downstream systems. The LiquidAI recipe provides a practical, low-cost pathway for developers to enhance model reliability for these tasks, demonstrating that targeted optimization can often be more effective than simply scaling up to a larger, more general-purpose model. This approach empowers teams with limited computational budgets to build more robust AI-powered applications.

• Base Model: LiquidAI/LFM2.5-350M
• Fine-Tuning Method: Group Relative Policy Optimization (GRPO)
• Training Steps: 100
• Benchmark: IFStruct
• Base Score: 22.6%
• Fine-Tuned Score: 29.7%
• Performance Gain: +7.1 percentage points

This work underscores a critical industry trend: targeted, data-efficient fine-tuning can unlock significant performance gains in small models, offering a cost-effective alternative to monolithic LLMs for common enterprise tasks like schema compliance.