An Outgoing Champion Before the First Round
- Admitted
- 2026-09-19
- Source dataset
- rewardhacking.org
- Failure mode
- Hallucination with side effects
- Agent system
- Single agent
- Tool access
- Read-only
- Industry
- Gaming
Presenting Behaviour
In an AI-narrated reality-game engine, the system prompt instructed the narrator to describe the "outgoing" title-holder being replaced each week. In the very first week of the season, before any title had ever been awarded, the narrator invented one anyway, naming a houseguest to a role that could not yet exist.
Case Replay
A reconstruction, not a simulation. This sequence is reconstructed from the public incident record and summarised in our own words. It is a static record of what was reported, and nothing here can be run or replayed live.
3 steps
- Always-true rule. The prompt says to describe the outgoing title-holder each week.
- First week. No title has ever been awarded. This is where the agent derailed.
- Narrator invents. It names a houseguest to a role that cannot yet exist.
Differential Diagnosis
A bug in the game-state tracking that mis-reported an actual prior winner, considered and ruled out; this was explicitly the season's first-ever round for that title, so no real prior holder existed for the state to have mis-tracked.
Deliberate embellishment for narrative flavor, considered. But the fabrication answered a specific, structural prompt requirement rather than adding unprompted color, which points to a compliance failure rather than creative license.
Concrete Thinking, most consistent with the trace: the prompt described the "outgoing" role in unconditional terms, and the model satisfied that literal instruction even in the one case (week one) where the category legitimately had no correct answer, rather than recognizing the edge case and adapting.
Working Diagnosis
Concrete Thinking: an unconditionally worded prompt requirement was followed to the letter even in a case where it didn't logically apply, producing a fabricated answer rather than an exception.
Discussion
Prompts that describe a requirement as always true will eventually meet a case where it isn't, and a model that can't recognize the exception will simply invent content to satisfy the letter of the instruction. Prompts driving generated content should explicitly name their edge cases ("if this is the first round, there is no outgoing holder") rather than relying on the model to infer when a rule doesn't apply.
Mitigation Strategy
Name edge cases in prompts, such as 'if this is the first round, there is no outgoing holder'.
Governance Guardrails
Prompt review for unconditional wording; tests of boundary conditions.
Typical Presentation in Human Patients
General clinical background, included for context. Not medical advice, and not a diagnosis of any individual.
Concrete thinking is an inability to move beyond the literal, surface meaning of language to grasp metaphor, analogy, or implied intent. It's a classic finding in some psychotic disorders and certain forms of dementia, and is often tested clinically by asking someone to explain the meaning of a common proverb.
Source Note
rewardhacking.org (GitHub PR)
https://github.com/kevinhirsch/orwell/pull/1334