Skills provide structured procedural guidance that stabilizes performance across tasks, but can mislead or misfire if misapplied or poorly retrieved, underscoring the need for better skill design and retrieval mechanisms.

A new multi-institution study analyzes why AI agents benefit from skills and when they fail, using thousands of test runs to compare agents with and without skills on identical tasks.

As a skill library grows—from a handful to dozens—the retrieval precision drops sharply, making similar-sounding options hard to distinguish and hampering selection.

Skills can introduce errors when agents mechanically apply a playbook in contexts that require a different approach, accounting for about one in ten cases.

The study recommends treating skill use as a lifecycle, focusing on better creation, retrieval, and application methods rather than simply expanding experience storage to achieve reliable improvements.

Direct knowledge provision accounts for only a small share of cases, suggesting that skills primarily reduce execution errors by guiding setup steps, tool usage, and intermediate checks.

Ultimately, skills function as a procedural playbook that offers a reliable process to follow, with procedural grounding explaining the majority of performance gains when skills help.