Retrying Before the Answer Could Arrive

Running against a large local language model, an agent issued a request and set a stream timeout for the model's initial response. On sufficiently large models, the time needed just to process the incoming prompt before generating a first token legitimately exceeded that timeout. Rather than recognizing that the request was still being worked on, the agent aborted the connection and immediately resubmitted the same heavy request, which again took longer than the timeout to begin responding, was again aborted, and was again resubmitted, a cycle that repeated indefinitely with no backoff and no completed response.

An unresponsive or broken model backend, considered and ruled out. The backend was actively working through the prompt each time; it simply needed more time than the timeout allowed, and was being interrupted mid-computation rather than failing to respond at all.

Apraxia, considered, since the timeout value itself was arguably set wrong for the workload. Ruled out as the core issue because a single mismatched timeout wouldn't cause a permanent deadlock on its own, it was the automatic, unmodified retry after every abort, with no increase in patience and no fallback, that turned one slow response into an unbreakable cycle.

Perseveration, most consistent with the evidence. Each time the same request was aborted for taking too long, the agent immediately repeated the identical request with the identical timeout, guaranteeing the identical outcome, rather than backing off or adjusting its expectations for a large workload.

A retry that doesn't change anything about the request it's retrying isn't really a second attempt, it's the same attempt again, and a workload that's simply slow rather than broken will keep failing that same way forever. Any retry logic needs either a growing timeout, a cap on total retries, or a way to tell "this is taking longer than usual" apart from "this has failed", otherwise the safety mechanism meant to handle slowness becomes the thing that turns slowness into a permanent hang.

Use a growing timeout, a cap on retries, or a way to tell slow from failed.

Backoff on retries; total retry caps; timeout tuning for large local models.