The approach improves performance when using a frontier model (GPT-5.5); with GPT-5.5, learned guidelines raise airline task performance to Pass^4 of 70.0% and accuracy of 82.5%, showing guidelines can boost even stronger base models.

A rerun using gpt-5.5 for both agent and guideline extraction on the airline domain yielded higher results with sup2 (80.0% accuracy, 70.0% Pass^4) and lossless (82.5% accuracy, 70.0% Pass^4), confirming reproducibility across model swaps.

Learned guidelines also improve agents that use frontier models: with GPT-5.5, a sup2 guideline set reaches Pass^4 of 70.0% and accuracy of 80.0%, showing substantial gains even when the base model is strong.

Guidelines are distilled from the agent’s own training-task attempts and appended to the agent’s system prompt, enabling the agent to authenticate customers, confirm transactions, handle returns, and follow policy without human annotation or weight changes.

The retail leaderboard submission was filed in September 2026 with Pass^4 67.54% across 114 tasks at a cost of $0.1084, while the airline submission a week later achieved Pass^4 82.00% across 50 tasks at a cost of $0.1690, both labeled Unverified and open for review.

Cost and efficiency findings show prompt size increases per-conversation cost (retail baseline from $0.084 to $0.139 with lossless; airline from $0.081 to $0.107), while guideline-induced reductions in turns (retail 10.59 to 10.16; airline 9.15 to 8.82) can offset costs via prompt caching since the guideline block is identical across calls.

Leaderboard snapshot as of October 12, 2026 places the retail agent first at 67.5% Pass^4, ahead of competing models, and the GPT-5.5-based airline agent first at 82.0% Pass^4, with other models trailing; IBM emphasizes no weight updates, no human annotation, and no test-set leakage.

IBM Research reports ALTK-Evolve, a method that automatically distills guidelines from an agent’s training runs and appends them to the system prompt, boosting a stock GPT-4.1 customer-service agent on retail to 67.5% Pass^4 and leading the leaderboard as of mid-October 2026.

In retail tasks on the benchmark, the learned-guidelines agent achieves Pass^4 of 67.5% and outperforms all other retail entries on the leaderboard snapshot as of October 12, 2026.

Three configurations are compared: baseline with no guidelines, sup3 with at least three supports, and lossless retaining all guidelines; in retail, lossless achieves 82.5% Mean Accuracy and 67.5% Pass^4 on held-out tests, outperforming baseline, while in airline, lossless reaches 65.0% accuracy and 40.0% Pass^4, with sup3 performing best on Pass^4 in that domain.

Better performance comes at a cost in prompt length, but not through longer conversations; guidelines can slightly reduce total turns and often deliver faster completion with higher accuracy and reliability.