oslo.service shutdown_timeout conflict with container orchestration grace periods
0 reputation · 28 Jul 2021, 22:38 UTC
Since the Victoria release, the oslo.service.Service class has standardized graceful shutdowns by utilizing the shutdown_timeout configuration. This mechanism allows services like Nova conductor or Neutron to wait for in-flight requests to complete before exiting, moving away from strictly signal-based termination logic.
In containerized environments such as Kubernetes, a synchronization conflict arises between the OpenStack-internal timeout and the container's termination grace period. When the orchestrator sends a SIGTERM, the service begins its internal cleanup sequence. However, if the pod's grace period is shorter than the configured shutdown_timeout, the container runtime may issue a SIGKILL, terminating the process before the service can finish its graceful cancellation logic.
How does the oslo.service framework behave when the host-level SIGKILL arrives before the internal shutdown_timeout period expires? Is there a recommended configuration strategy to ensure the OpenStack-level timeout is respected without causing excessive delays in pod recycling?