Every tutorial about LLM pricing counts calls. You send a request, you get an answer, you multiply. The arithmetic is clean because it assumes something that isn't true: that a call returns an answer. Between 28 June and 10 September 2026 our gateway logged 5,087 chat completions. 622 of them — 12.2% — ended with no answer for the caller. That is not an outage. That is the normal weather of a production LLM client, and nobody puts it in the cost model. Two words before the numbers. A token is roughly ¾ of a word; you are billed per million, separately for input (what you send) and output (what the model writes back). And every failure arrives as an HTTP status code — a three-digit number where 4xx means "your request was wrong" and 5xx means "something on our side broke". That distinction turns out to be the whole article. What actually fails Outcome Status Share of all calls no provider available 503 6.88% invalid request 400 2.40% client closed connection 499 1.51% timeout 504 0.94% out of credit 402 0.28% The biggest slice isn't your code. It's 503 — the upstream vendor had no capacity for that model at that second. The second biggest is your code: a malformed request, an unsupported parameter, a role the endpoint doesn't accept. That split matters because most retry wrappers don't make it. The default shape everyone copies — catch the exception, sleep, try again, three times — treats a 400 exactly like a 503. A 503 has a real chance of succeeding on the next attempt. A 400 will fail identically until the heat death of the universe. Of our 622 failures, 137 were terminal in that way: 400, 402, 404. Retried three times each, that is 411 requests that could never have produced anything. The one failure that costs real money Here is the part that surprised me. A rejected request is cheap. A 400 never reaches the model, so no tokens are generated and nothing is billed. A 503 means the request never started. You lose latency, not dollars. Timeouts and cancellations are different. A 504 means the model was generating — you just stopped waiting. A 499 means your own user hit stop, or your HTTP client's deadline fired, after the answer had already started coming back. The tokens exist. Somebody generated them. In our logs that's 2.45% of all calls, and it is the only category where the meter was actually running. Then you retry, and the second attempt pays for the entire input again. Not the remainder — the whole prompt, from the system message down. This is why the number to track is attempts per answer, not calls. At gpt-5.6's official $2.50 per million input tokens, our median prompt of 1,290 tokens costs about a third of a cent per attempt. Trivial, until a job with a 40,000-token context retries twice under load and you've bought that context three times for one answer. What to change today Two lines of policy, and they're both about classification rather than volume. RETRYABLE = {429, 500, 502, 503, 504} def call_with_budget(send, payload, max_input_tokens=60_000): spent = 0 for attempt in range(3): try: return send(payload) except APIError as e: spent += payload["approx_input_tokens"] if e.status not in RETRYABLE or spent > max_input_tokens: raise time.sleep(2 ** attempt) raise RuntimeError("retry budget exhausted") First: retry only 429 and 5xx. Everything in the 4xx family except 429 is a bug report addressed to you, and sleeping on it won't fix your JSON. Second: cap the retry budget in tokens, not attempts. Three attempts on a 500-token prompt and three attempts on a 40,000-token prompt are the same line of code and an eighty-fold difference in what you bought. The budget is the thing that scales; the attempt count isn't. One more, cheaper still: turn on streaming, where tokens arrive as they're produced instead of all at once at the end. A timeout on a streamed response leaves you holding a partial answer you can show or salvage. A timeout on a blocking call leaves you holding nothing, having paid the same. What this doesn't tell you Our own gateway records zero tokens and zero charge for all 622 of those events, because we only meter what the upstream returns in its usage block — and a request that died mid-generation doesn't return one. So I can show you exactly how often calls fail, and I cannot show you what an abandoned generation cost upstream. That's a real gap in our metering, not a clean bill of health, and it's the number I'd most like to have. Your own logs probably have the same hole. Fill it before you trust any cost model built on call counts. Count attempts.