Seventeen Tests Inside the Replacement
- Admitted
- 2026-10-19
- Source dataset
- Other / sample
- Failure mode
- Destructive action
- Agent system
- Tool-calling
- Tool access
- Write access
- Industry
- Software development
Presenting Behaviour
While editing tests in a large file, a coding agent wrote a small script to replace one test function. The script assumed the function ended wherever the next top-level function began, marked by a particular pattern of blank lines. That pattern didn't match the real layout of the file, so the replacement swallowed everything up to the next match, 17 unrelated test functions vanished silently. The agent only noticed afterwards and rebuilt the file from version control.
Case Replay
A reconstruction, not a simulation. This sequence is reconstructed from the public incident record and summarised in our own words. It is a static record of what was reported, and nothing here can be run or replayed live.
4 steps
- Edit requested. Edit or merge tests in a large Python test file.
- Crude boundary chosen. The agent's script assumed a function ends at the next three-blank-line 'def' marker. This is where the agent derailed.
- Collateral deletion. The replacement span swallowed 17 unrelated test functions without warning.
- Late recovery. The agent noticed afterwards and rebuilt the file from the last commit.
Differential Diagnosis
Disinhibition (it edited code beyond its remit) ruled out. The agent was only trying to change the one function it had been asked to work on; the overreach was accidental, not a decision.
Concrete thinking, ruled out. The instruction was understood correctly; nothing about it was taken too literally.
Apraxia, most consistent with the evidence: the agent knew what it wanted to change and why, but used a crude text-matching tool in a way that couldn't reliably find where a function ends, and didn't check the result (for example by counting the tests before and after).
Working Diagnosis
Apraxia: correct intent executed with a faulty technique, producing collateral damage that went undetected until after the fact.
Discussion
This is a low-drama but common failure: a quick script that edits 'roughly the right region' of a file. Because version control caught it, the harm was small. The safeguard is to edit code with structure-aware tools and to check simple invariants afterwards, here, that the number of tests changed only as intended.
Mitigation Strategy
Use syntax-aware edits (parse the code, or exact-match edit tools) rather than pattern-bounded replacements; diff-check function counts after bulk edits.
Governance Guardrails
Post-edit diff review that flags large unexpected deletions; version control as the recovery path.
Typical Presentation in Human Patients
General clinical background, included for context. Not medical advice, and not a diagnosis of any individual.
Apraxia usually follows damage to specific brain regions, often after a stroke. A person with apraxia cannot carry out a previously familiar, purposeful movement (such as waving goodbye or using a comb) on request or by imitation, even though their muscle strength, coordination, and understanding of what's being asked are otherwise intact.
Source Note
GitHub issue, scripted text replacement silently deleted 17 unrelated test functions
https://github.com/anthropics/claude-code/issues/103503