Atomic File Writes with Rename-After-Write: A Practical Architecture Note

Your process writes a config file, a cache snapshot, or a small data file. Somewhere between "open" and "close" the machine loses power, the process gets SIGKILLed, or another process reads the file at the wrong moment. The result is a truncated file, a half-written JSON document, or a reader that crashes on corrupt state. The fix is a small, well-understood pattern: write to a temporary file, then rename it over the target. This note covers when that pattern is sufficient, where its guarantees come from, and where it breaks.

Requirements: what you're actually guaranteeing

The pattern solves a specific problem: a reader must never observe a partially written file. It should see either the complete old version or the complete new version, with nothing in between. This matters for:

• Configuration files read by long-running daemons that reload on change.
• State snapshots (queue offsets, checkpoints, small embedded databases) rewritten periodically.
• Cache files shared between processes where a torn read would cause a crash or bad behavior.

What it does not guarantee: durability across power loss by itself (that requires flushing data to disk, covered below), or coordination between multiple concurrent writers (rename gives you last-writer-wins, not merging).

The smallest suitable design

The pattern has four steps, all performed by the writer:

1. Open a temporary file in the same directory as the target. This is essential: the atomicity of rename only holds within a single filesystem, and the same directory guarantees the same filesystem.
2. Write the full new content to the temporary file.
3. Flush the data to stable storage: call fsync on the file descriptor before closing it.
4. Rename the temporary file over the target path with a single rename(2) call.

A minimal POSIX shell illustration (run as the user that owns the target file, in the directory containing it):

#!/bin/sh
# Usage: atomic-write.sh /path/to/target
target="$1"
tmp="${target}.tmp.$$"   # PID suffix avoids collisions between concurrent writers

cat > "$tmp"              # write new content from stdin
sync "$tmp" 2>/dev/null || true   # best-effort flush; see fsync note below
mv -T "$tmp" "$target"   # rename(2) on Linux; atomic replacement

In a real program, use your language's equivalents: os.replace() in Python, Files.move() with ATOMIC_MOVE in Java, std::filesystem::rename in C++. Python example:

import os, tempfile

def atomic_write(path, data: bytes):
    dir_ = os.path.dirname(os.path.abspath(path))
    fd, tmp = tempfile.mkstemp(dir=dir_, prefix=".tmp-")
    try:
        with os.fdopen(fd, "wb") as f:
            f.write(data)
            f.flush()
            os.fsync(f.fileno())
        os.replace(tmp, path)   # atomic on POSIX and Windows
    except BaseException:
        os.unlink(tmp)          # don't leave orphans on failure
        raise

Run this as the user that owns the target file, with write permission on the containing directory (rename requires directory write permission, not just file write permission). A meaningful check afterward: read the file back and validate it parses (e.g., json.load for JSON state), and confirm no .tmp-* files remain in the directory.

Trust and data boundaries

The atomicity guarantee comes from the operating system's rename syscall, not from your code. On POSIX systems, rename(2) is atomic even when the target already exists: at every instant, the target path resolves to either the old inode or the new one. Readers that open the file before the rename keep reading the old inode; readers that open it after see the new one. No reader ever sees a mixture.

This means the trust boundary sits at two places:

• The filesystem. The guarantee holds on local POSIX filesystems (ext4, XFS, APFS, and similar). It does not reliably hold on many network filesystems — NFS and SMB have historically had edge cases around rename semantics and close-to-open consistency. If your target lives on a network mount, treat the guarantee as unverified until tested on that exact setup.
• The platform. On Windows, the equivalent is MoveFileEx with MOVEFILE_REPLACE_EXISTING (and ideally MOVEFILE_WRITE_THROUGH). Python's os.replace maps to the right call on both platforms, which is one reason to prefer it over os.rename in older code.

Durability is a separate problem

Atomic rename protects readers from torn writes. It does not guarantee the new content survives a power cut. Data can sit in the kernel's page cache and be lost even though the rename succeeded. The fsync before rename pushes the file's data to stable storage. For full crash durability you should also fsync the directory after the rename, so the directory entry itself is persisted:

dir_fd = os.open(dir_, os.O_DIRECTORY)
try:
    os.fsync(dir_fd)
finally:
    os.close(dir_fd)

Whether you need this depends on your failure tolerance. A cache file that can be rebuilt doesn't need directory fsync; a checkpoint that took an hour to compute does.

Operational checks

Before trusting this in production, verify on the actual deployment platform:

• Crash simulation. Kill the writer process (SIGKILL, not SIGTERM) at random points in a write loop, then check the target: it must always parse as either the old or new complete version. A simple way is to write incrementing sequence numbers plus a checksum and validate after each kill.
• Orphan check. After crash tests, look for leftover temporary files. Orphans are harmless to correctness but accumulate; a startup cleanup pass that deletes stale .tmp-* files older than some threshold is a reasonable mitigation.
• Concurrent reader check. Run a reader in a tight loop opening and parsing the file while the writer rewrites it. Any parse error indicates the atomicity assumption doesn't hold on that platform/filesystem combination.

Failure modes

• Temp file on a different filesystem. If the temp file is created in /tmp but the target is on another mount, rename either fails (EXDEV) or silently degrades to copy-and-delete, destroying the guarantee. Always create the temp file in the target's directory.
• Network filesystems. NFS and SMB rename semantics vary by server and mount options. If you must write atomically to a network share, test explicitly, or use a different mechanism (write locally and rsync, or use a small database).
• Multiple writers. Two processes running the pattern concurrently produce last-writer-wins behavior: one writer's complete file replaces the other's. If you need serialized or merged updates, add a lock file (e.g., flock) around the whole write-rename sequence.
• Permissions and metadata. The renamed file has the temp file's permissions, not the old target's. If the target requires specific ownership or mode, set them on the temp file (e.g., os.chmod) before the rename.

When this design is the wrong one

Choose something else when: files are large and rewritten frequently (the full-copy cost dominates — consider append-only logs or an embedded database like SQLite, which has its own crash-recovery machinery); multiple writers need to merge rather than overwrite; or the target must live on a filesystem where rename atomicity can't be verified. For single-writer, whole-file replacement of small-to-moderate files on a local filesystem, rename-after-write remains the simplest correct answer.