A conclusion can be well reasoned, backed by real evidence, and still wrong, if it rests on an untested belief about how a system behaves. In this case, that belief didn’t present itself as a belief. It presented itself as something already known. When a decision depends on it, and testing it is cheap and available, verifying it before deciding is worth doing on its own terms.
When evidence is only evidence under one assumption
Several confident conclusions can be drawn about the same situation, one after another, each backed by real evidence: records of what had or hadn’t happened, values checked against what was current. All of them can still be wrong if they share the same untested assumption underneath, a belief about how the system in question performs a specific kind of change. In one case, the assumption concerned whether changing a credential’s value required replacing its identity, or whether the value could be updated while the identity held. The evidence gathered along the way was read correctly at every step. What made the resulting conclusions wrong was that each one only followed under that one assumption, and the assumption itself had never been tested.
The test that settles it
The test that resolved this question was simple: capture the state of the thing itself, make the change deliberately, and compare before and after. Applied to the system in question, it showed that the state changed while the identity stayed the same. That result overturned reasoning that had already produced several prior conclusions, because each of them had assumed the opposite.
How far this claim reaches
The claim here is narrow: when a conclusion depends on a belief about how something behaves, and the belief has never actually been tested, and testing it is cheap and available, test it before deciding anything further on top of it. It doesn’t claim that everything should be tested, and it says nothing about whether any specific system can be trusted. It claims something narrower: a cheap, available test beats another round of reasoning from an assumption that was never checked.
The night this premise was tested, and what it overturned, is told in A redaction that found three of eight.