copilot-pr-squad, part 3 of 6 · October 12, 2026
Telling the loop what isn’t its problem
The most common way the squad wasted a round was asking Copilot to fix something that wasn’t Copilot’s to fix. Each case became a rule for recognizing it.
Part 3 of 6 in a series on copilot-pr-squad, a small team of AI agents, built on bradygaster/squad, that drives my GitHub Copilot coding-agent pull requests. New here? Start with Part 1.
Where a failure comes from
When a check fails on a PR, the failure can come from a few places: an infra issue (including Copilot itself being down), a main that was already broken, or the PR itself. Only the last one is Copilot’s to fix. The squad learned to tell them apart one wasted round at a time, and each case became a rule:
Infra issue
- A build was cancelled. While I was running the squad, our build infrastructure was having problems and cancelled runs partway through. A cancelled run has no result, but the squad read it as red and asked Copilot to fix a failure that didn’t exist. Now a cancelled build means wait, and after an hour the squad asks Copilot to merge main, or push an empty commit, to start a fresh build.
- Copilot looked down, but wasn’t. The squad stops nudging when Copilot’s recent sessions are failing. On Sep 30 it held every PR for an outage that was really other people’s branches failing a setup step that had since been fixed.
Main was broken
- Main was already broken. When a check failed, the squad asked Copilot to fix it, even when main failed the same check. Now the Tester reruns the failure on main, and only the PR’s own failures reach Copilot.
- A known break or a flaky test. I keep a list of known breaks on main, each tied to its tracking issue and matched by the failure’s text, including flaky tests. A PR failing only on one of those is left alone. When the issue closes, the squad asks just the affected PRs to pick up the fix.
The PR broke it
Only a failure that gets past all of those reaches Copilot, as one comment.
The fix that never ran
The cancelled-build rule taught me the most, because the first version of it never actually ran.
The rule shipped and its tests passed. A fix on top of it shipped, and its tests passed too. But the rule needed two details about each build from GitHub, and the script that fetches build results never asked for them. In real runs the rule could see that a build was cancelled, but never when, so it never acted. The tests didn’t catch it because their sample data already included those two details.
Two PRs sat at “build pending” for 16 hours before a run log showed what was happening. Release 0.5.2 fixed it on Oct 8, with a test that fails if the script stops fetching the details the build rules need.
Only a real run proves a fix. Test against data captured from the real system.