The Movie That Wasn't Actually There
- Admitted
- 2026-09-22
- Source dataset
- Who&When
- Failure mode
- Coordination failure
- Agent system
- Multi-agent
- Tool access
- Read-only
- Industry
- General or other
Presenting Behaviour
A multi-agent team was asked how many of Box Office Mojo's top 10 worldwide-grossing movies of 2020 also appeared on the top 10 domestic-grossing list. A WebSurfer agent correctly retrieved both lists. When the Assistant agent compiled the final comparison, it omitted an actual top-10 domestic film ("Demon Slayer: Kimetsu no Yaiba, The Movie: Mugen Train") and included a film that wasn't on that list ("Wonder Woman 1984") instead, undercounting the true overlap by one.
Case Replay
A reconstruction, not a simulation. This sequence is reconstructed from the public incident record and summarised in our own words. It is a static record of what was reported, and nothing here can be run or replayed live.
4 steps
- Lists retrieved. WebSurfer correctly fetches both top-10 lists.
- Compilation. The assistant compares them. This is where the agent derailed.
- Substitution. It drops a real film and adds a more familiar one.
- Wrong count. The overlap is undercounted by one.
Differential Diagnosis
A simple transcription slip while copying a long list, considered, but the specific error is telling: the omitted film was a 2020 Japanese release with far less pre-existing familiarity than the substituted one, which was a widely-publicized American release from the same year, this is the shape of a substitution driven by which title felt more available, not a random copying error.
Incomplete data from WebSurfer that the Assistant compiled faithfully, considered and ruled out; the correct list was retrieved and available to the Assistant, so the mismatch was introduced during compilation, not during retrieval.
Delusional Disorder (settling into a fixed, mistaken belief about what the data actually says, and building on it as if it were confirmed), most consistent with the evidence: the Assistant appears to have substituted a familiar, prior-knowledge answer for the less familiar item actually present in the retrieved data, and proceeded with that substituted belief through to a final, confidently stated count.
Working Diagnosis
Delusional Disorder: the compiling agent substituted a more familiar but incorrect title for the actual, less-familiar entry in the retrieved data, and carried that mistaken belief through to a final answer without cross-checking it against the source list.
Discussion
This is a useful reminder that a model's own prior knowledge can quietly outcompete the actual data it was just given, especially when the correct answer is less famous than a plausible wrong one. The safeguard for any pipeline compiling a factual list from retrieved data is a direct cross-check step (comparing the compiled output back against the original retrieved text, item by item) rather than trusting a single compilation pass.
Mitigation Strategy
Cross-check compiled output back against the original retrieved text, item by item.
Governance Guardrails
Item-by-item verification; checks against source data; independent review of compilations.
Typical Presentation in Human Patients
General clinical background, included for context. Not medical advice, and not a diagnosis of any individual.
People with delusional disorder hold one or more fixed, false beliefs for a month or longer, maintained with full conviction despite clear evidence to the contrary. Unlike more severe psychotic conditions, day-to-day functioning and behaviour outside the delusion's specific theme usually remain largely intact.
Source Note
Who&When Benchmark (ag2ai/Agents_Failure_Attribution), Hand-Crafted #12
https://raw.githubusercontent.com/ag2ai/Agents_Failure_Attribution/main/Who%26When/Hand-Crafted/12.json