How to stop FastAPI, Flask and worker processes without dropping in-flight work, leaking tasks or making deployments depend on timing luck.
Stop accepting new work first, give existing work a bounded drain window and make unfinished work recoverable when the window expires.
Separate readiness from liveness
When shutdown begins, readiness should fail before the process exits so load balancers stop sending new traffic. Liveness should not immediately fail, or the orchestrator may kill the process before it can drain.
The sequence and timing need to match the platform. Document the termination grace period, pre-stop hook and application deadline instead of assuming defaults are safe.
Bound the drain
Track in-flight requests, active background tasks and open consumers. Stop accepting new work, then wait only as long as the service can safely wait. A bounded timeout prevents one stuck dependency from blocking every deployment.
For work that cannot finish, persist a recoverable state or leave the message unacknowledged so another worker can retry. Cancellation must not silently lose the business operation.
Test termination as a workflow
Send SIGTERM during a slow request, a database transaction, a queue delivery and an outbound call. Verify responses, acknowledgements, connection closure and telemetry.
A deployment check should confirm that the new instance becomes ready only after dependencies are usable and that the old instance stops receiving new work before it exits.
Evidence to take into review
- Readiness fails before termination begins.
- In-flight work has an explicit drain deadline.
- Unfinished jobs remain recoverable and observable.
- Dependency calls respect the shutdown deadline.
- Termination scenarios are covered in integration or operational tests.
Explore all engineering notes.
Use PRODUCTION-7 to connect this concern with the other dimensions of a trustworthy backend.
View all articles Get the checklist