Acting from Scratch Is Expensive
Our testbed is Crafter, a 2D open-world survival game — procedurally generated, Minecraft-like — with 22 achievements: collect wood, place a table, make a wood pickaxe, defeat a zombie, and so on up a long tech tree. A run's progression score is the fraction of those 22 it unlocks (the ★ badges elsewhere on this page count them), so reaching deeper achievements takes many correct actions in sequence.
An LLM agent solves each step of a task the way it solves every other: re-derive the situation from the prompt, emit one primitive action. In a long-horizon environment like Crafter that is a to move a single square — hundreds of calls per episode, plus the trial-and-error of rediscovering the same routines on every task. Over a long horizon that trial-and-error also tends to degenerate. Most of the bill goes to re-deriving things the agent already worked out earlier.
Reusable skills are the obvious fix, but most prior work either freezes a library after pre-training or runs skill discovery as a separate offline phase. SpeedRunner learns them online, and represents them where it matters for cost: as code.
Skills as Programs
SpeedRunner treats a skill as a program: a fixed sequence of actions the agent runs deterministically, with no LLM call per step. Between rollouts, a coding agent reads the agent's recent trajectories and edits the library in code — adding a function, rewriting one, factoring out a helper, or removing a skill that is no longer used. It learns from the trajectories alone: no replay of past episodes, no separate validation pass. A skill that works is kept and reused, and the agent stops paying to rediscover it.
Visually, it looks like this: the agent , , and eventually . Try the tab strip at the top of the player — no codebook takes a fumbling 30+ primitives to get there; with a fully trained library the same opener is a single call to skills like find_and_collect() or place_item().
Cheaper and Higher-Scoring Than Every Baseline
Pareto frontier on Crafter (gpt-5.4-mini, zombie-frequency 1, 3 seeds per method). Each method trains for 200 rollouts; we then run its final library on 30 held-out test traces and plot the LLM cost per trace against the test-progression score. SpeedRunner (gold) sits in the upper-left corner — the highest score at the lowest per-trace cost of any method on the chart.
Concretely: with its final library, SpeedRunner runs a Crafter test trace for $0.18. The cheapest baseline, ASI, costs $0.86; the most expensive, OPO, costs $2.01 — a 4.7×–10.9× cost reduction, while scoring higher than all of them. The paper shows the same pattern across three embodied environments, and that it holds under distribution shift and environment randomness.
Skills, Edited Online by a Coding Agent
Drag the slider to scrub through 40 sleep cycles. Functions appear ( new ), get rewritten ( v++ ), or get cut and replaced — every edit was made by the sleep-phase coding agent reading recent rollouts. Hover any card to see its source at that point.
skills
skills
What Does the Sleep Agent Actually Do?
Each card above is one edit to the library, and each edit is the work of the coding agent: it reads the most recent trajectories, inspects the current library, and rewrites it in code. There is no replay of past episodes and no held-out validation step — it commits based on what the trajectories already show. Pick a scenario to replay one cycle's tool-by-tool history; the commentary on the right tracks the play head.
First sleep cycle. The library is empty; the coding agent reads five rollouts, decides which patterns are common, and writes two public utilities plus three private helpers.
Dig In
- Paper — Better, Faster, Stronger: Programmatic Skill Learning Best Reduces Agent Cost.
- Code — the full training and evaluation framework.
- Compare tab — scrub through SpeedRunner, OPO, ASI, Voyager, and ReAct side by side on the same simulated wall-clock, at any of 5 training checkpoints.