Async Python in production: timeouts, cancellation and backpressure

Async does not make blocking work disappear

An async endpoint that calls a synchronous HTTP client, performs heavy CPU work or uses a blocking database driver can stop the event loop from serving unrelated requests. The function signature says nothing about whether the work inside it yields control safely.

Inventory every I/O boundary and library used on the request path. Confirm that network and database operations use compatible async clients. Move unavoidable blocking work to a bounded worker or thread strategy instead of letting it occupy the shared event loop.

Timeouts are part of the contract

Without a timeout, a dependency call can wait until infrastructure closes it, consuming a connection and request slot throughout. A single missing timeout becomes a capacity incident when the dependency slows down across many concurrent requests.

Set timeouts according to the caller's total deadline. Connection, read and pool acquisition may need separate budgets. Leave enough time for the application to handle failure and respond; a downstream timeout equal to the entire inbound deadline provides no recovery margin.

• Retry only operations that are safe to repeat and failures likely to be transient.
• Use jittered backoff and a strict retry budget.
• Record timeout and retry outcomes as metrics, not only logs.

Cancellation must reach the work

When a client disconnects or a deadline expires, continuing expensive work may waste capacity or create an outcome nobody is waiting to receive. Cancellation should propagate through awaitable operations where stopping is safe.

Cancellation is not rollback. If durable state has already changed, the operation needs an explicit completion or compensation path. Shielding critical cleanup can be correct, but shielding entire workflows hides overload and defeats the caller's deadline.

Backpressure prevents cascading failure

Unlimited concurrency moves the queue into memory and the connection pool. When downstream capacity is lower than incoming demand, latency rises, timeouts trigger retries and the additional load makes recovery less likely.

Use bounded queues, semaphores and pool sizes to make capacity explicit. Reject, defer or shed work once those limits are reached. A controlled 503 with a retry hint is often safer than accepting work that cannot finish before its deadline.