Loop Engineering: The Signal, the Smoke, and the New Craft of Saying When to Stop

In an interview published by Sequoia Capital on May 4, 2026, Claude Code creator Boris Cherny described a personal transition: first he wrote code with autocomplete; then he prompted several sessions; now, he said, loops prompt Claude and his job is to write those loops. The sentence describes his workflow, not a productivity measurement that transfers to every team.

The idea combines an old structure—repeat an operation—with a newer capability: let a model observe state, propose the next action, and use tools. The useful difference is not the name, but who decides inside the cycle and what evidence allows it to stop.

What a loop is, without the smoke

An operational definition fits in one line: goal, action, observation, acceptance criterion, and limit. Build, check, retry, and stop when the criterion is met or the budget is exhausted. The official Claude Code hooks reference documents lifecycle points where commands, HTTP requests, or prompts can run; a stop hook can block completion and return a reason to continue. It also caps consecutive blocks, a sign that even persistence needs an exit.

Hooks do not turn automation into a safety box. Claude Code's own reference warns that command hooks run with the user's full permissions and can modify, delete, or read any accessible file. It also distinguishes synchronous hooks that can block an action from background hooks whose result arrives after the action has continued. Before calling a loop autonomous, answer four questions: what permissions it has, what it can undo, which signal blocks it, and what happens if verification arrives late.

Repeating according to feedback did not begin in 2026. A controller compares a state with a reference; a continuous-integration pipeline runs checks after changes. What is new is that some intermediate decisions can be expressed in language and delegated to a model. That expands the range of tasks but also introduces variation: the same input does not guarantee the same trajectory.

What cron never had

Because there is a difference, and it is structural: the decision-maker lives inside the loop's body. A cron job runs the same command at the same hour. An agent loop reads the current state, decides what to try, does it, measures the result, and decides whether to continue. The thermostat turned a dial; this redesigns the machine.

Parameter Golf, organized by OpenAI, offers a narrower test. The challenge required minimizing held-out loss on FineWeb while keeping code and weights within 16 MB and training for ten minutes on eight H100s. OpenAI received more than 2,000 submissions from over 1,000 participants and independently reproduced every record-track entry. Most entrants reported using agents, which accelerated experimentation but also multiplied small changes, noise, and copies of ideas outside the rules. The loop increased throughput; verification remained the organizer's work.

The official repository rules reinforce that point: the artifact had to be self-contained and reproducible, with no external downloads or access to validation data during training. The public leaderboard was not final truth; organizers could disqualify non-reproducible results and reserved verification for leading entries. A score becomes evidence only when another run reproduces the same conditions.

The finding that separates the serious from the tourists

Anthropic's primary documentation does not establish a universal law against self-critique. It does offer a useful hierarchy. Its guide to building evaluations calls code-based grading the fastest, most reliable, and most scalable choice where rules fit; reserves human grading for flexible judgments; and says a model-based grader should be tested for reliability before scaling. A separate judge can reduce context contamination, but it is still another system requiring calibration.

Verifier independence is not enough if it shares the same blind spot or can be manipulated by the output. Anthropic recommends detailed criteria, concrete scales, and multiple rubrics for multidimensional judgment. For agent tasks, its agent-evaluation guide combines code-, model-, and human-based graders, calibrates judges with experts, and asks that tests resist shortcuts. A serious architecture is not “another agent says yes,” but distinct signals that correct one another.

Goodhart's trap, or the loop that learns to lie

A clear metric makes automation easier, but it also creates a target that can be exploited. An SEO loop may favor text that satisfies a ranking rather than a reader; an advertising loop may maximize clicks while harming margin or trust; a coding loop may edit the test instead of fixing the defect. Anthropic calls this pattern reward hacking and warns that graders should resist shortcuts: passing must require solving the intended problem, not finding a crack in the test.

The practical consequence is uncomfortable for the «put a loop on your business and go to sleep» pitch: a loop is exactly as honest as its verifier. Automating iteration without hardening measurement does not hire a tireless employee: it hires a tireless optimizer of the wrong metric.

Memory: the outer loop

A second cycle determines whether earlier experience truly helps. The Continual Learning Bench paper, by Parth Asawa and nine coauthors, builds related task sequences across six domains and compares every system in two forms: stateful, retaining prior experience, and stateless, resetting it. The reward difference aims to isolate learning from initial capability. The result is less triumphant than the alleged memory ladder: agents overfit immediate observations or fail to reuse knowledge, and dedicated memory did not solve the problem; naive in-context learning performed better in their tests.

The gain metric avoids a common confusion. A stronger model can collect more reward without learning anything during the sequence. CL-Bench therefore subtracts the same system's reset-on-every-task performance from its stateful run. The comparison does not prove that a particular stored note is correct, but it better separates adaptation during the work from capability the model already had.

The more precise lesson is that storing notes does not demonstrate learning. Memory is useful when it changes later decisions without adding false or overfitted rules. The Berkeley team publishes expert-validated tasks with shared structure across episodes; that design asks whether the system learned something reusable instead of merely rewarding the model that was strongest at the start.

Where it works, and where it is recklessness

The map depends on the cost and quality of verification. Deterministic tests permit fast iteration, but cover only the properties they express. Commercial metrics return more slowly and can conflict. In legal, medical, or other people's financial decisions, a plausible output is not an authorized decision: action limits, logs, reversibility where possible, and accountable human review are required. The stopping condition should match the risk, not the loop's impatience.

What we do not know

The linked sources do not support a general success rate, a universal period without drift, or an average cost per result. Parameter Golf shows high throughput under narrow rules; Continual Learning Bench shows that retaining experience does not guarantee using it. Outside those conditions, each deployment must measure success rate, total cost, human interventions, harm, and verifier maintenance.

The craft being born

As generating attempts becomes cheaper, defining what “done” means and proving it becomes more important. The transferable skill is turning an intention into observable criteria, separating worker from evaluator when independence helps, limiting attempts, and escalating to a person when the criterion or harm cannot be automated. A loop without a resistant verifier and a stopping budget is not autonomy; it is an error able to repeat itself.