Lessons for your own loop

Keep watching, keep fixing

The main one: plan to keep iterating, keep watching, and fix the process a piece at a time. The squad wasn’t designed and then switched on. It got here through 96 small releases, nearly every one a response to something a real run did wrong. Budget for that from the start: someone has to read what the loop did, every run, for as long as it runs. 0.5.2 shipped on Oct 8 for a problem found that morning.

What makes that loop of fixes work:

• Make every output traceable to the version that wrote it. A bad post leads straight to the rules that produced it, and each fix can be checked against the next run.
• Write the test first, then the fix. A mistake in a log became a failing test before it became a change, so it couldn’t quietly come back.
• Only a real run proves a fix. The cancelled-build rule passed its tests for two releases and never ran. Test against data captured from the real system.
• Experiment before changing a model. A lower cost always gives something up. But not every job needs the most expensive model, and only a test on real work shows which ones do.

Build these in before the loop starts

What to build in before the loop starts:

• Measure cost per unit of work from day one. I didn’t, and had to rebuild the first week’s baseline afterwards.
• Earn write access in steps. Watch-only first, then one item, then all of them, then a schedule.
• Put decisions in scripts and judgment in the agent. A rule written as prose gets read differently from one run to the next. Write the parts that should always come out the same as tested scripts, then build the team around them.
• Don’t let the agent that writes something also check it. Here the Fact Checker clears every claim, and the posting script refuses anything it didn’t clear.
• Make steering a file, not a release. Briefs, instructions and known breaks are files the squad reads each run, so steering it is writing, not shipping.
• Tell the loop what isn’t its problem. Most wasted rounds came from asking Copilot to fix something it didn’t cause.
• Check scope at the first review. A loop repeats a bad early choice every round.
• Give the human one place to answer. One report, one list of questions, answered in plain text.

Since 0.5.2

This series stops at 0.5.2, on Oct 8. The squad kept shipping while I wrote it. Two of the later changes are worth knowing about:

• It merges, on trial. The squad now squash-merges a PR once it is approved, green, and has no open question for me. A script is the gate: it checks GitHub’s own merge rules and never overrides them. It’s the next step in earning write access, and too new to say it has earned it.
• A cap on review rounds. Across 99 merged Copilot PRs, the median was 3 review rounds, but 10 PRs needed 10 or more. Now a PR is held for me after 10 rounds on review feedback, and I decide whether to keep going, scope it down, split it or close it.

What’s next

• Keep refining it. Keep reading the run logs and fixing what they show, one release at a time.
• A second user, then a second repo. It works one person’s PRs in one repo today. Each shows how much of the squad is configuration and how much is assumption.
• Feed it from upstream. Build a squad that writes the issues, the requirements and the designs, so work reaches this squad already scoped. Today I drive those myself.
• Choose who makes the fix. Decide, for each piece of work, whether Copilot should make it or another squad should, such as a builder squad.

Squad, or something else? bradygaster/squad is one way to build this. The same job could be built on another framework, or as one skill or scheduled prompt that calls the same scripts, since the scripts carry most of the rules. That is worth exploring later. For now I’m going to keep building on squad.

If you’re building something similar, I’d like to compare notes. Find me on LinkedIn.