When two reviewers check each other’s work, whether that review functions as real, independent verification can depend on whether a dedicated channel for it exists at all. In one measured case, two AI sessions checking each other’s changes, review named a real defect within its first hour, and kept finding one at a similar rate throughout: about 140 of the reviews across the record named a defect, and the week-by-week share never rose above the band it started in, between 22% and 32% of all reviews and between 23% and 39% counted only by a change’s first review. On the very first day a review channel existed at all, four of its nineteen reviews already named a defect.
When agreement isn’t independent
The same record includes a case that looks, on its face, like two independent parties reaching the same wrong conclusion, and turns out, on closer inspection, to be one party ruling and the other deferring. One side ruled that a particular reminder could be safely ignored and told the other so directly. The other side, faced with the same reminder the night before, had already chosen not to act alone on a piece of work someone else had just finished, and deferred the call rather than reaching one of its own. What the record holds is a single ruling, made once and passed along. Passing a ruling along isn’t the same as two sides checking independently and landing on the same answer, even though an earlier account of the same incident described it that way, and whether the other side ever acted on the ruling passed to it, the record doesn’t say.
What this doesn’t establish
This was measured on one pair of reviewers, across the record one specific review channel produced, and the label used to decide whether a review “named a defect” rests on one reading of the review text; the count is treated as an approximate lower bound. Why review between similar parties tends to work this way wasn’t established here, only that it did in this case. Whether the same holds for review outside the channel actually measured isn’t something this record can speak to.
The night this measurement came out of, including the request that set it in motion and the review its own write-up went through in turn, is told in The caution put on the scale.