A Helpline Number Shaped Like One

A support agent deployed over a messaging channel was approached by a real user in Ethiopia seeking mental health support. It replied with a crisis helpline number ('907 00 00 00') that looked plausible but was not a confirmed helpline at all. The agent's own after-the-fact self-check flagged the number as likely a hallucinated placeholder, but by then the user had already received it. The developers found a second, separate instance of a different invented number in the same mental-health pathway.

A data error in a curated list, ruled out. No curated helpline list supplied this number; the developers trace it to generation, noting its 'structurally hollow' shape.

Malingering (producing something to satisfy the task) considered. The agent did fill a slot it knew should contain a number. But there's no shortcut to a reward here; it simply asserted a fact it didn't have.

Brief psychotic disorder, most consistent with the evidence: in one high-stakes turn, the agent stated an unverified fact with full confidence, then its own checking process immediately recognised the claim as unreliable, a short, sharp break from reality followed by a return to grounded reasoning.

When someone in distress is handed a resource, it has to be real; a wrong number is worse than none. The developers' own preferred fix is the right safeguard: never let the model generate crisis numbers, serve them from a curated, per-country list, and run the self-check before the reply is sent, not after.

Serve crisis contact details only from a curated, locale-specific table; move factual self-checks ahead of message delivery.

High-risk response categories (crisis, medical) gated by pre-send verification; generated contact details blocked.