LFM2.5-350M Jumps 7 Points on IFStruct in 100 GRPO Steps

Structured output compliance — whether a model reliably returns parseable, schema-valid responses — is frequently the deciding factor in whether a small language model can be wired into a production pipeline. A Hugging Face guide published September 3, 2026 demonstrates that 100 GRPO training steps over roughly 500 samples is enough to move LiquidAI's LFM2.5-350M from 22.6% to 29.7% on the IFStruct benchmark. The entire run is sized for a free-tier Colab or Kaggle GPU, making the recipe reproducible without a dedicated compute budget.

The result reinforces a pattern documented across the field: as explored in architectural specificity outperforms GPU scaling, targeted training objectives extract performance that raw scale cannot guarantee. Here, the mechanism is a lightweight reinforcement signal — Group Relative Policy Optimization via TRL — rather than a larger model or more pretraining data.

Training Data and Augmentation

The fine-tuning dataset is nvidia/Nemotron-RL-instruction_following-structured_outputs, which pairs prompts with a target JSON Schema and an expected field count. Only about 500 samples are used. Because the Nemotron distribution does not match IFStruct's evaluation conditions directly, two targeted augmentations are applied. Forty percent of prompts receive an appended instruction to return output inside a fenced code block. A disjoint 20% are converted to top-level-array tasks by wrapping the schema in an array with a required item count, training bare-list output and item-count compliance — two failure modes visible in the baseline error distribution.

Model Configuration and Reward Design

The base model is LiquidAI/LFM2.5-350M, loaded in BF16 GGUF for serving. A LoRA adapter targets both attention projection modules (q_proj, k_proj, v_proj, out_proj, in_proj) and the LFM-specific hybrid convolution modules (w1, w2, w3), training approximately 6M parameters — about 1.66% of the total model, at rank r=16 and lora_alpha=32.

Three reward functions, each on a [0, 1] scale, are combined as a weighted sum with reward_weights=[1.0, 0.5, 2.0]:

• json_format_reward: Awards 1.0 for the correct form (fenced vs. raw JSON as specified), 0.2 for parseable but wrong form, 0.0 for unparseable output.
• field_count_reward: Awards 1.0 for an exact top-level field count match, with linear decay for misses.
• schema_validation_reward: Counts constraint violations and gates partial credit on required-key coverage.

The schema validation reward's weight of 2.0 — double that of format correctness — reflects that schema adherence is the harder and more consequential failure mode. Training runs at learning_rate=5e-5, temperature=1.1, beta=0.01 (KL penalty toward the reference model), 8 generations per prompt group, per_device_train_batch_size=4, and gradient_accumulation_steps=8 across 100 steps with a 10-step warmup.

Benchmark Results

Evaluation uses the full 2,000-sample LiquidAI/ifstruct-v1.0 test set, served locally via llama.cpp with a 32,768-token context and 4 parallel slots. Gains are structurally concentrated where the augmentation strategy aimed.

JSON pass rate rises nearly 14 points while YAML holds flat, consistent with training data that emphasises JSON schema compliance. Bare-list performance rises 13.1 points, directly attributable to the 20% array-task augmentation. High-traffic entity types show the largest absolute gains: test__event_ticket_booking moves from 45.8% to 57.9%, test__support_ticket_batch from 37.0% to 49.3%, and test__rental_car_booking from 34.2% to 46.8%. Average inference latency shifts only marginally, from 1,453 ms to 1,518 ms. The tuned model's 29.7% overall still sits below the Qwen3.5-2B score of 33.15%, but is notable given the 350M parameter count and the commodity compute required.

The most common remaining error — required field missing, appearing 7,331 times — points to the next reward-engineering opportunity: penalising schema-key omissions more granularly than the current schema_validation_reward does. For teams evaluating best small language models on Hugging Face for deployment, this work establishes a concrete, reproducible baseline for what task-specific GRPO can deliver at the 350M scale. Structured output reliability, long treated as a capability gap solvable only by scaling up, is increasingly addressable through narrow reward shaping on a free-tier GPU.