New Ledger-Based Method Boosts Coding Performance at a Fraction of the Cost

October 25, 2026
New Ledger-Based Method Boosts Coding Performance at a Fraction of the Cost
  • A ledger-based, zero-shot self-orchestration method coordinates multiple model instances to decompose problems and achieve frontier-like coding performance at a fraction of the cost, tested across eleven open and closed-weight models on LiveCodeBench problems.

  • The codebase (v2-current) includes clear environment setup, keys, and uv-based example commands, with results written to runs/results.json and workspaces stored in runs/ws/.

  • Results are not universal: some models show no improvement or degradation, indicating the method depends on specific model capabilities.

  • Artifacts and materials include figures such as gvs5h-mark-humans.png and benchmark_livecodebench.png; paper and run data are under CC BY 4.0, with code under MIT License and details in NOTICE.md.

  • Licensing: Code under MIT; paper and run data under CC BY 4.0, with additional licensing notes in NOTICE.md.

  • Key results show Orchestrated Qwen3.8 Flash Next achieving 93.0% pass@1 versus 90.4% for Fable 5 at about 9% of the cost; locally served Qwen3.8-27B improves from 66.8% to 92.4% pass@1.

  • Across eleven models, gains reach up to 25.6 percentage points on pinned backends, with two configurations hitting frontier-level accuracy.

  • Implications for teams: self-hosted coding assistants could largely bridge the gap with proprietary APIs if base models have sufficient reasoning to benefit from problem decomposition.

  • The work is released under CC BY 4.0, and cost analysis shows the multi-agent orchestration achieves frontier-like accuracy at significantly lower cost on a log-cost scale.

  • Experimental scope spans eleven open and closed-weight models on the latest LiveCodeBench problems, with results framed as passing benchmarks and tied to pinned backends and cost.

  • Five instances of Qwen3.8-27B can match or exceed Claude Fable 5 on LiveCodeBench Hard without fine-tuning or RL, using ledger-based zero-shot self-orchestration.

  • Code is available on GitHub under MIT license, with running requirements including uv, Python 3.12, and an API key for the target model.

Summary based on 2 sources


Get a daily email with more Tech stories

More Stories