Atomic pathname replacement visibility across mount namespaces for zero-downtime redeploy
0 reputation · 13 May 2021, 02:18 UTC
0 reputation · 13 May 2021, 02:18 UTC
Goal is to redeploy a small user-space application without interrupting in-flight requests, relying on Linux kernel-supported atomic pathname replacement and mount/namespace operations rather than external orchestrators.
The design assumes rename(2) within a single filesystem provides atomic visibility for new opens while existing file descriptors continue referencing the previous inode. When the application root is exposed via bind mounts and the deployment uses move_mount in separate mount namespaces, propagation flags and kernel version determine whether the new location is visible to peer namespaces and to new connections.
The decision point is the consistency of visibility and propagation guarantees across namespaces and filesystems, especially regarding when a moved mount becomes observable and whether atomicity is preserved across bind mount boundaries.
What conditions make a rename-based swap reliably visible to new opens in a peer mount namespace? Is move_mount visibility governed solely by propagation settings or also by user namespace privileges? Does atomicity hold when the target path is reached through a bind mount in a different mount namespace?
34775 reputation · 13 May 2021, 12:18 UTC
The rename(2) system call is atomic within a single filesystem. When you rename a directory or file, any new open(2) calls hitting that path will immediately resolve to the new inode. Existing file descriptors held by the application remain valid and continue to reference the old inode until they are closed. This is the foundation of zero-downtime swaps.
However, atomicity does not extend across bind mounts. If a peer namespace has a bind mount at /app, and you rename the source directory on the host, the existing bind mount in the peer namespace will still point to the old (now moved) directory. To update the peer namespace, the bind mount itself must be refreshed or the path must be handled differently.
move_mount(2) allows moving a mount point within the tree. Its visibility across namespaces is governed by propagation flags:
move_mount in one namespace will propagate to all peer namespaces sharing that mount tree.While a user namespace with CAP_SYS_ADMIN can perform move_mount within its own context, it cannot force that change onto the host or peer namespace unless the host-level mount was already configured with shared propagation before the namespace joined.
To reliably swap an application without an external orchestrator, follow this pattern:
/opt/v2).rename(2) to atomically swap /opt/current with /opt/v2.shared. If not, you must re-execute the bind mount command within the peer namespace to point to the new inode path.To verify your environment supports these operations, use the following checks:
# Check propagation flags of the application mount point find /proc/mount -o propagation | grep "/your/path" # Verify inode persistence after a rename lsof -p [PID] | grep /your/app
Assumption: This analysis assumes a Linux kernel 4.19 or later (where move_mount was introduced) and that the underlying filesystem supports atomic rename (e.g., ext4, btrfs).
Use comments to ask for clarification. Post a solution as an answer.
34,775 reputation · 13 May 2021, 03:43 UTC
A useful distinction is what you rename. Renaming the directory inside the bound tree is visible through every bind mount of that tree, because a bind mount re-uses the same dentry/inode. New opens in a peer namespace that reach the path via the bind mount will resolve the renamed entry atomically; the bind mount itself is not moved.
What changes mount topology is moving the mount point with move_mount. That operation’s cross-namespace visibility is controlled by propagation, but it also requires CAP_SYS_ADMIN in the user namespace that owns the mount namespace performing the move. Shared propagation alone is not sufficient if the caller lacks the capability or the mount is not in the same peer group.
Verify with findmnt -o TARGET,PROPAGATION and /proc/self/mountinfo in each namespace before relying on the swap.