Loop Engineering Shipped
Prior art on propose → verify → feedback → refine loops, organized around ten named gaps. Two files: one where every source was fetched and quoted, one where web access was denied for the whole session.
The question
Ten named gaps in running a long build loop: deterministic oracles, oracle reliability on long diffs, oscillation-breaking, convergence forecasting, escalation ladders, budget management, resumption, disposition vocabulary, cross-repo verification, and the reward-hacking risk in any stop condition.
What the verified half found
- Claude Code's own stop-gating has a numeric backstop — it overrides the hook and ends the turn after 8 consecutive blocks. Unbounded "keep going" gating is not what the platform itself does.
- A reviewer in a fresh subagent context sees only the diff and the criteria, not the reasoning that produced the change — the source of both its independence and its blindness.
- Anthropic's own caution, quoted: a reviewer prompted to find gaps will usually report some, even when the work is sound.
- "Context rot" — degrading recall as token count grows — is named by Anthropic and solved by nobody in any source consulted.
- Token usage alone explains about 80% of the variance in agent performance, and multi-agent systems burn roughly 15× more tokens than chat.
- Escalating the FEEDBACK model, not just the builder, gives substantially larger gains — which is why the loop's escalate-once step escalates the oracle.
- A widely-repeated claim about loop design turned out to be single-sourced to a tweet, and is flagged in the pack as not primary.
What a team of agents costs, in tokens
Anthropic's own figure — and token usage alone explains about 80% of the variance in agent performance. Twenty-three specialists is not a free upgrade over one reviewer; it is a fifteen-fold bill that has to be worth paying on every run, which is why selection and pre-checks are deterministic and happen before anything is dispatched.
The half that failed
The companion file is 1,488 lines and 119 trust-scored claims, and its verifier passed zero of fourteen checks: WebFetch and WebSearch were denied for the entire session, so the specialists fell back to training-data recall and the pack says so on its own front page. It is kept, labelled unverified, because it is a useful record of what a research run looks like when its tools are taken away — and because the verified file later independently confirmed two arXiv identifiers that file's own verifier had flagged as fabrication risks. The run cost about $19 and took under an hour.
What it changed: nothing, and that is the finding
Every concept this pack calls net-new already existed in the codebase. The numeric-backstop mechanism was written five days before the pack, and the generator-is-not-the-oracle framing was in a design document six days before it. The honest conclusion is that this was corroboration rather than input — which is worth knowing, because it is the only pack here where that is true.
Sources (the verified file)
- Anthropic — Building Effective Agents
- Anthropic — Effective context engineering for AI agentsWhere "context rot" is named.
- Anthropic — How we built our multi-agent research systemThe token-variance and 15× figures.
- Anthropic — Claude Code best practices
- Olausson et al. — Is Self-Repair a Silver Bullet for Code Generation?Escalating the feedback model beats escalating the builder.
- Huang et al. — Large Language Models Cannot Self-Correct Reasoning YetThe case for an oracle that is not the generator.
- Denison et al. — Sycophancy to SubterfugeSpecification gaming, and why a stop condition is a reward function.
- Baker et al. — Monitoring Reasoning Models for Misbehavior
- METR — Measuring AI Ability to Complete Long Tasks
- Jimenez et al. — SWE-bench