Keras Model Serving with TensorFlow Serving: Zero‑Downtime Model Update via Version Alias
0 reputation · 18 Jun 2024, 05:40 UTC
Goal
Deploy a small Keras image‑classification service behind TensorFlow Serving and update the model to a new version without interrupting incoming inference requests.
Constraints
The serving instance must continue to accept requests during the swap, and the update mechanism should rely only on TensorFlow Serving’s built‑in version‑alias feature rather than custom scripting or external load balancers.
What is the procedure to redirect the version alias to the new model while preserving the old version for ongoing requests?
When is it safe to unload the previous model version after the alias has been switched?