Interrupted saveRDS in RStudio leaves a truncated output file with no rollback
0 reputation · 18 Mar 2024, 10:34 UTC
0 reputation · 18 Mar 2024, 10:34 UTC
When a long-running R session in RStudio is interrupted — for example the R process is killed or the session crashes partway through a large saveRDS or write.csv call — a partially written output file can remain on disk. My understanding is that the RStudio IDE provides no transactional or idempotent write semantics for user code, and no automatic rollback of the incomplete file, so the truncated artifact looks like a valid output unless something downstream detects it.
The goal is a reliable pattern for project scripts so that an interrupted write never silently replaces a previously good output, without relying on IDE behavior that may not exist. Constraints: the pattern should work the same across RStudio Desktop and Server/Workbench sessions, should not depend on the Git pane (which versions source, not large generated outputs), and should be verifiable by deliberately killing R mid-write and inspecting the result.
Is it correct that RStudio itself offers no recovery or cleanup for partially written files, leaving this entirely to user code? Is writing to a temporary file followed by an atomic rename the accepted approach, and does file.rename behave atomically on the network filesystems common in Workbench deployments? Are there documented R functions or packages that already wrap this pattern?
34775 reputation · 18 Mar 2024, 21:10 UTC
Yes — your understanding is correct on all three points. RStudio provides no transactional semantics, rollback, or cleanup for files written by user code; saveRDS() truncates the destination the moment it opens it, so any interruption destroys the previously good copy. The accepted fix is write-to-temp-then-rename, and it is a user-code responsibility, not an IDE feature. One caveat: atomic rename is only guaranteed within the same filesystem, which matters on Workbench deployments with network mounts.
saveRDS() serializes the object and streams bytes sequentially to the target path. The file is truncated at open time, then filled. If the R process is killed, crashes, runs out of memory, or the session dies mid-write, you get a prefix of a valid serialization — which readRDS() rejects with errors like "unknown input format" or a gzip read error. Nothing in R or RStudio checkpoints the old version first. Note this is not RStudio-specific: plain R and Rscript behave identically, so any fix belongs in your scripts, not the IDE.
save_rds_atomic <- function(object, path) {
dir <- dirname(path)
tmp <- tempfile(pattern = ".tmp-", tmpdir = dir) # same filesystem!
on.exit(unlink(tmp), add = TRUE)
saveRDS(object, tmp)
if (!file.rename(tmp, path)) {
stop("Atomic rename failed for ", path)
}
invisible(path)
}Key details:
tempfile(tmpdir = dirname(path)) ensures the temp file sits next to the target. A rename across filesystems silently degrades to copy-then-delete and loses atomicity.file.rename() returns FALSE on failure rather than erroring — this bites on some Windows and network drives.On POSIX local filesystems, rename within a mount is atomic. On NFS it is generally atomic too, but on some SMB/CIFS mounts and certain Windows-backed shares, rename can fail or behave non-atomically, especially if the destination is open elsewhere. This is the one place I'd verify rather than assume: confirm the temp file and target share a device ID (fs::file_info(c(tmp, path))$device_id should match), and run the kill test below on your actual Workbench storage. If rename proves unreliable there, a fallback is copy-then-verify-checksum-then-switch-a-symlink, but test the simple pattern first.
You don't have to hand-roll everything: fs::file_move() gives clearer move semantics, and packages in the targets/pins ecosystem handle versioned outputs, but a small helper like the one above is the common, dependency-free approach and works identically on Desktop and Server/Workbench. The same pattern applies to write.csv() and any other output writer.
save_rds_atomic(x, "out.rds").kill <pid>) or via Session > Interrupt/Terminate in RStudio.out.rds either doesn't exist yet or still loads the previous complete object; confirm a .tmp-* leftover exists and is cleaned up.saveRDS(x, "out.rds") to confirm the truncated file fails readRDS() — proving the test actually exercises the failure mode.If your kill test shows the final path itself getting truncated despite the rename, that points to a cross-filesystem temp file — print tempfile(tmpdir = dirname(path)) and the mount table to check.
Use comments to ask for clarification. Post a solution as an answer.
34,775 reputation · 18 Mar 2024, 21:54 UTC
While the write-to-temp-then-rename pattern is the standard approach, it is important to note a platform-specific nuance regarding file.rename(). On many Windows configurations, file.rename() will return FALSE and fail to replace the destination if the target file already exists.
To ensure the pattern works reliably across both POSIX and Windows environments without relying on external packages, you may need to explicitly remove the target file if the rename fails:
# Ensure the target is replaced on Windows
if (!file.rename(tmp, path)) {
unlink(path)
if (!file.rename(tmp, path)) {
stop("Atomic rename failed after target removal")
}
}
This ensures that the "rollback" is effectively handled by the fact that the original file is only touched after the temporary file has been successfully written to disk. Verification can be performed by attempting to overwrite an existing RDS file and interrupting the process mid-write.