A companion arXiv paper, LIFT: Scoped Constraints for Evaluating and Training Instruction Following, argues that conventional benchmarks' constraints can be exploited as RL rewards, causing models to neglect actual task fulfillment.

All tasks in the benchmark are synthetic and drawn from Wikipedia, with constraints guided by an LLM; there are 784 test tasks across ten splits, plus flat and structured variants.

The benchmark uses an autogated judge (google/gemma-4-31B), and scores should be compared only across runs using the same judge, with content constraints weighted 90% and structure constraints 10%.

Evaluations across 17 models show GPT-5.5 leading on structured tasks, while open-weight models Kimi K3 and DeepSeek V4 Pro top overall averages; performance on flat tasks trails structured tasks.

LIFT defines scoped constraints that attach to specific document sections and are scored per section, with two constraint types per section—verifiable (code-checked) and judged (LLM-checked).

LIFT-Bench, LIFT-Train, and the generation pipeline are published on GitHub under Apache-2.0, with splits on Hugging Face Hub; task formats are compatible with PrimeIntellect verifiers for RL evaluation.

LIFT-Train uses the same score as the reinforcement learning reward, and experiments show it maintains substantive answer length and quality better than IFTrain, highlighting the importance of content-aware rewards.

Experiments indicate training with scoped constraints and corresponding rewards can improve alignment between task fulfilment and evaluation, with notable gains when training on constrained tasks versus flat tasks.

AI21 Labs released LIFT-Bench on October 25, 2026, as an instruction-following benchmark that scores task content within document sections to curb reward hacking in RL, alongside LIFT-Train (34,258 tasks) and an open synthesis pipeline.

The training history shows IFTrain rewards led models to optimize for constraint satisfaction rather than meaningful task completion, at times reducing substantive content dramatically.