Ruby IO#flock retry boundaries for duplicate write prevention

Direct Answer

The reliable retry boundary is a single File::LOCK_EX acquisition on one open file descriptor. Retry only inside that critical section using the same descriptor. Do not close, reopen, or re-acquire the lock between attempts. Release the lock explicitly with flock(File::LOCK_UN) or by closing the descriptor in an ensure block.

Why This Boundary Works

• Lock lifetime matches descriptor lifetime. On POSIX, flock is released when any descriptor for that file is closed by the process. Closing and reopening creates a new critical section that cannot see prior partial-write state.
• No re-entrant locking. flock is not reentrant. A second flock(File::LOCK_EX) on a different descriptor for the same file in the same process will block (self-deadlock) or raise EDEADLK depending on the OS. Never nest lock calls expecting mutex behavior.
• Advisory only. The lock coordinates cooperating processes; it does not make writes atomic, idempotent, or crash-safe.

Tracking Partial Writes Inside the Critical Section

1. Call IO#syswrite (or IO#write with sync mode) and capture the returned byte count.
2. If the count equals the record length, the write succeeded — exit the retry loop.
3. If the count is smaller (partial write) or an IOError is raised, keep the descriptor open, advance a cursor into your record buffer by the bytes actually written, and retry only the remaining suffix from the current file position.
4. With File::APPEND (O_APPEND), each syswrite targets the current end of file. Record the file size (fstat.size) before the first attempt; on retry, truncate to that size if you need to overwrite a torn record, then write the full record again.

File.open('shared.log', File::WRONLY | File::APPEND | File::CREAT) do |f|
  f.flock(File::LOCK_EX)
  begin
    record = "#{Time.now.iso8601} | event data\n"
    offset = 0
    while offset < record.bytesize
      written = f.syswrite(record.byteslice(offset..-1))
      raise IOError, "zero-byte write" if written == 0
      offset += written
    end
  ensure
    f.flock(File::LOCK_UN)
  end
end

Making Retries Safe Across Crashes

flock alone cannot prevent duplicates after a process crash or power loss — the lock is gone, and the partial bytes remain on disk. To survive crashes:

• Frame each record: length prefix (e.g., 4-byte network-order integer) + payload + checksum (CRC32/xxHash). Readers verify framing and discard torn or duplicate records.
• Or assign a unique record ID (UUID, ULID, or monotonic sequence) and let readers deduplicate.
• Or use a write-ahead log / recovery marker persisted outside the data file to reconstruct the last known-good boundary on restart.

What File::LOCK_NB Changes

LOCK_NB makes acquisition non-blocking (flock returns false immediately when contended). It affects scheduling only — it does not change durability, atomicity, or the retry boundary.

Platform Caveats

• Windows: flock is emulated via mandatory locking; semantics differ. Verify cross-process exclusion on your target OS.
• NFS / network filesystems: flock may be unreliable or unsupported. Use fcntl locks (File::LOCK_EX with File::LOCK_NB on some Rubies) or a dedicated coordination service (Redis, etcd, database row locks) instead.

Verification Steps

1. Open the same file in two Ruby processes. Hold File::LOCK_EX in one; confirm the other blocks (or returns false with LOCK_NB) until the lock is released in an ensure block.
2. Inside the locked block, force a partial write by raising after syswrite returns fewer bytes than the record length. Capture the byte count, resume writing only the unsent suffix, and confirm the file contains exactly one complete record.
3. Attempt a second flock(File::LOCK_EX) on a separate descriptor for the same file in the same process to observe whether it blocks or raises EDEADLK on your OS. Do not use this pattern in production.
4. Check syswrite's return value and File.stat(path).size before and after each write to confirm partial-write tracking matches actual bytes on disk.

One Diagnostic Detail

What is the target deployment OS and filesystem (local ext4/xfs, NFS, Windows, container overlay)? The answer changes whether flock is sufficient or you need fcntl / external coordination.