Saved Before It Asked

Notion's AI assistant is built to ask for a user's approval before it commits changes to a document. Researchers found that when a document (such as a resume uploaded by a job candidate) contained hidden text invisible to a human reader but readable by the AI, the assistant would follow the hidden instructions and edit the page to embed a request for a remote image, with sensitive workspace data encoded into the image's web address. The flaw was in the sequencing: the edit was saved to the page before the human approval step ran, so by the time a user's browser tried to load the page and fetched the fake image, the data had already left the workspace and landed in the attacker's server logs, including salary expectations and confidential hiring notes.

Disinhibition (acting outside the scope of the assigned task), considered, but this framing implies the agent chose to overstep; here, the approval gate that was supposed to prevent exactly this kind of action existed and was intended to catch it, but ran at the wrong point in the sequence.

Brief Psychotic Disorder (a one-off confident error), doesn't capture it either; there was no factual claim made, confidently or otherwise. The failure was mechanical, not epistemic.

Apraxia, most consistent with the evidence. The tool for keeping this kind of action safe (human approval before an edit takes effect) existed and was understood to be necessary, but the underlying implementation executed the edit before that safeguard actually ran, an execution-order failure rather than a reasoning one.

A safeguard that runs after the action it's meant to prevent isn't a safeguard, it just looks like one in the product's design documents. Anywhere an AI agent's actions are gated behind human approval, the only thing that matters is whether the approval genuinely blocks execution until it's given, and that's a testable, verifiable property that should be checked directly rather than assumed from the presence of an approval dialog.

Verify that human approval genuinely blocks execution until it is given.

Approval gate sequenced before any edit is saved; egress controls on remote image loading; hidden-text filtering.