copilot-pr-squad, part 2 of 6 · October 12, 2026
Watching it before trusting it
The squad wasn’t designed and then switched on. It got good through 96 small releases in eleven days, and it earned write access one step at a time.
Part 2 of 6 in a series on copilot-pr-squad, a small team of AI agents, built on bradygaster/squad, that drives my GitHub Copilot coding-agent pull requests. New here? Start with Part 1.
The loop behind every release
None of this was designed up front. Every improvement came from the same loop: the squad ran, I read the log of what it did, and each mistake became a test that failed, then a fix, then a new release. Every comment the squad posts carries the version that wrote it, so any post traces straight back to the rules that produced it.
I ran that loop 96 times in eleven days, from 0.1.0 to 0.5.2. The squad earned write access one step at a time along the way:
Scoped. Scripts first, then the team.
Team formed, watching only. Every run turns up mistakes.
One PR goes live, then every PR.
PRs that would conflict wait their turn.
A status report I can act on. Findings that end. Runs on an hourly schedule.
Scope creep caught earlier. Cost per PR down.
Cancelled builds handled. The squad keeps PR descriptions.
Most releases were small. Four stories show the kind of thing the loop caught: shadow mode is below, and the other three are Part 3, Part 4 and Part 5. The rest are at the end of this post, grouped by what they fixed.
Earning write access, one step at a time
Before there was a team, there were scripts. An earlier squad I’d tried showed me a squad was strongest when a script made each decision and an agent supplied the judgment. So I wrote those parts first, as scripts with their own tests: sort every PR by its state, gather the facts for a round, and check a comment against those facts before posting. Then I formed the team around them.
The squad started in shadow mode: it did everything except write to GitHub, and recorded each comment it would have posted in a file instead. That let me read its work before any PR saw it. The first shadow runs each found something worth a release:
- A comment rewritten after failing its fact check went out without being checked again. The posting script now refuses any text the Fact Checker didn’t clear word for word.
- A PR was marked as a failed build because a Copilot session had failed, while every real check was green.
- Reviews were recorded where nothing read them, so the next run would have asked for the same review again.
- The Reviewer couldn’t see an existing review file and escalated findings it couldn’t read.
Nothing went live until I’d read those runs. One switch moves the squad between watching and acting. One PR went live on Sep 30 and a second that evening; on Oct 1 every Copilot PR went live, with runs still started by hand.
The hourly schedule came on Oct 6, and it mattered more than I expected. On manual runs, a PR waited a median of 4.2 hours after Copilot finished for the squad’s next step, longer than my own 3.0 before the squad. On scheduled runs it waits 2.1.
What else the loop caught
The other fixes fall into four groups.
Working around other PRs
- Two PRs touching the same file conflict once the first merges. Now the second waits its turn, then is asked to merge main with a note on whose side wins.
- Some issues build on code still in an unmerged PR. Now an issue can wait for a named PR to merge, or for its requirements to be accepted, before the squad hands it to Copilot.
Reading the signals right
- 21 of 27 PRs read as “Copilot working”, some for three days. Moving a PR to ready also starts a Copilot code review, and my script counted that as the coding agent. Now only the coding agent counts.
- One known break excused a whole failing log, hiding a second break on main. Now each known break excuses only the failures it explains.
- A known break’s fix landing asked about 20 passing PRs to merge main, moving them off their reviews. Now only a PR that shows the break is asked.
Asking me the right questions
- Questions said what happened, not what to do. Now every question ends with an ask, and my answer can be an action the next run carries out.
- The status report changed shape every run. One read “You (0)” with six questions underneath. Now a script writes it the same way every time.
- Questions stayed open after their PR moved on. Now each run closes them, and raises a fresh one if the problem still holds.
Knowing when a PR is done
- Findings had no way to end, and a fresh review re-raised ones I’d dropped. Now every finding ends in fix, plan or drop, and my decisions stick.
- Merging main triggered a new review of a ready PR, and 14 new findings drove a redesign of work that had shipped clean. Now a clean review holds unless the PR’s own code changes by 50 lines or more, or adds a code file.
- Copilot rewrites the PR description every session, so the section showing how the PR meets its issue kept disappearing. Now the Reviewer records it with every review, and a script writes it back.
One more change made a whole post possible: from Oct 6, every run logs what it cost. Part 5 is what that showed.