$0.084 instead of $0.052: the timeout receipt

01 / A $0.052 request can become an $0.084 receipt

A cheap token rate does not protect you from paying for repeated work. If your client abandons a healthy stream and starts the request again, the second attempt can generate another bill. The first question is not simply how long a request takes. It is which timer expired, and whether the original request actually stopped.

Here is a worked example using illustrative rates, not CheapRelay prices or measured performance. Assume input costs $1 per million tokens and output costs $4 per million tokens, with no cache activity.

• ■ Planned input: 20,000 tokens × $1 per million = $0.020.
• ■ Planned output: 8,000 tokens × $4 per million = $0.032.
• ■ One completed attempt: $0.052.
• ■ Interrupted attempt, assuming billing stops after 3,000 output tokens: $0.020 input + $0.012 output = $0.032.
• ■ Full restart after that interruption: another $0.052.
• ■ Combined receipt: $0.084, or $0.032 more than one successful attempt.

That is about 62% above the original $0.052 budget. This receipt assumes the first attempt really stopped at 3,000 output tokens. A local timeout alone cannot prove that; upstream work might continue.

02 / Give connection, silence and total duration separate limits

For an interactive client that expects long responses, a reasonable starting configuration is a 10-second connection timeout, a 30-second write timeout, a 120-second read timeout and a separate 900-second application deadline. These are engineering starting points, not service guarantees. Adjust them using your own workload.

• ■ Connection timeout, 10 seconds: bounds connection establishment rather than generation time.
• ■ Write timeout, 30 seconds: limits stalled request uploads; large attachments may need more.
• ■ Read timeout, 120 seconds: allows a substantial quiet interval while awaiting response data.
• ■ Application deadline, 900 seconds: caps the entire operation, even if data keeps arriving.

With HTTPX-backed clients, a read timeout generally limits waiting for incoming data, not the total lifetime of a stream. A response delivering data regularly can therefore run longer than 120 seconds. Conversely, a long silence before the first response data can trigger that timeout. A separate wall-clock deadline prevents an active stream from occupying resources indefinitely.

03 / Configure the SDK and every hop carrying the stream

For CheapRelay's OpenAI-compatible routes, set the SDK base URL to https://api.cheapapirelay.com/v1. In the Python OpenAI SDK, supply an httpx.Timeout with connect=10.0, write=30.0, read=120.0 and pool=10.0. The pool setting limits waiting for an available connection; it is not another generation allowance.

Also set max_retries=0 while diagnosing timeout behavior. Check the installed SDK's documented retry behavior before enabling retries in production. Once part of a stream has reached your application, do not treat a restart as a transparent continuation.

Your SDK is only one hop. Check reverse proxies, load balancers, application servers and job workers for shorter idle or execution limits. Disable response buffering where appropriate so streamed data can reach the caller promptly. A generous client timeout cannot override a proxy that closes the connection earlier.

04 / Make retries a visible receipt line

Record a logical job ID plus a distinct attempt ID, elapsed time, time to first data, last-data time, termination reason and any available usage. Never infer zero cost merely because the client received an exception.

Before retrying, classify the failure. A connection failure before submission differs from a disconnect after several thousand generated tokens. For partial output, preserve what arrived and decide whether to resume through a new prompt, restart deliberately or ask the user. Resuming can also incur new input charges.

05 / Budget real tokens, including cache activity

CheapRelay bills per real token, including cache reads and writes, with no monthly fee or subscription. Use the applicable model and cache rates when replacing this example with your own receipt; do not assume cached input is free.

Sign up for CheapRelay when you are ready to test your client with a small, controlled budget. First verify one long stream without automatic retries. Then test deliberate interruptions and reconcile usage. The goal is not an infinite timeout: it is a bounded operation whose failures do not quietly purchase the same work twice.