Graceful Shutdown: In‑flight requests during zero‑downtime migration

Direct answers

Requests exceeding the timeout abort after the timeout, not immediately. Ktor's stop(gracePeriodMillis, timeoutMillis) (and the equivalent engine configuration) stops accepting new connections, waits up to the grace period for in-flight requests to finish, and then forcibly terminates whatever is still running once the timeout elapses. A handler that sleeps past the limit is cancelled mid-flight — the client sees a reset or truncated connection.

The 503 for new connections is effectively not customizable. That response is engine behavior (CIO/Netty/Jetty), not a documented application hook. There is no supported API to change the status code during shutdown, so treat 503 as fixed at the server level. If you need a different signal, put it in front of Ktor (load balancer or proxy), not inside it.

There is no built-in guarantee that all in-flight requests complete. Ktor only waits up to the configured timeout. For non-idempotent operations, the timeout is a backstop, not a correctness mechanism — a terminated handler can leave partial writes or half-committed transactions.

What to do for your migration

1. Drain at the infrastructure layer. Run old and new instances behind a load balancer. Remove the old instance from rotation, wait for active connections to drain, then stop it. This is the standard zero-downtime pattern and makes Ktor's timeout a safety net rather than the primary mechanism.
2. Track in-flight work in the application. Keep a counter or registry of active non-idempotent operations (or supervise them with a parent coroutine scope). Make handlers responsive to cancellation — check isActive or use structured concurrency so a cancelled request rolls back cleanly instead of committing partial state.
3. Prefer idempotent or resumable endpoints. Idempotency keys or upsert semantics make an aborted request safe to retry, which removes the correctness problem entirely.
4. Size the timeout from real data. Set the grace/timeout above your p99 handler latency. If your longest legitimate request is 45 s, a 30 s timeout guarantees aborts.

Version assumptions and uncertainty

Exact configuration keys differ across Ktor 1.x/2.x/3.x and across engines — newer versions expose shutdown grace/timeout in the connector configuration, while older code calls stop(gracePeriod, timeout) directly. Confirm the parameter names and the stated 30 s default against the Ktor version you actually run; treat that default as unverified until checked.

Verification before you rely on it

• Add a test handler that sleeps longer than the configured timeout, trigger shutdown mid-request, and confirm the client connection is reset only after the timeout fires.
• Send a request after shutdown begins and record the actual status code your engine/version returns (expected 503).
• During a staged rolling restart, log completion of tracked in-flight requests and confirm the instance fully drains before termination. Also confirm your load balancer's deregistration delay exceeds your drain time, or new traffic will still reach a shutting-down instance.

One detail that would change this recommendation: whether your long-running handlers are idempotent. If they are, infrastructure-level draining plus a sane timeout is enough; if they are not, application-level tracking and cancellation-aware rollback are mandatory, not optional.