Old TorchServe workers remain active after new model version registration

Goal: Ensure that when a new model version is registered in TorchServe, workers serving the previous version terminate automatically after all in‑flight requests finish, enabling a seamless cutover without manual intervention.

Constraints: The version_policy feature does not automatically drain old workers; they continue to handle existing requests until completion or timeout. The interaction of max_idle_time and scale_down with ongoing traffic is not explicitly documented, leaving uncertainty about whether occasional requests can reset idle timers and keep old workers alive indefinitely. There is no built‑in API to confirm when all in‑flight requests for a version have drained, so operators must rely on external monitoring or custom tracking to know when it is safe to decommission the old version.

Questions: How can operators guarantee that old‑version workers terminate after all in‑flight requests complete given the current version_policy behavior? What is the precise interaction between max_idle_time and scale_down when a new version is registered and occasional requests continue to hit old workers? Is there a recommended method to detect when no in‑flight requests remain for a version without implementing custom request tracking?