Direct answer
The error fires when R asks the operating system for a single contiguous block of memory for one vector and the request is refused. It is not a measure of total RAM used — the number in the message is only the size of the failed request, which can look small even when the session is already near its limit. When you modify a data.frame, R's copy-on-modify semantics mean the old column (or the whole frame, in the worst case) and the new one coexist briefly, so peak usage during the operation is roughly the working set plus the new allocation.
What the copy actually costs
A data.frame is a list of column vectors. Adding one column allocates one new vector of nrow(df) elements — for a double, 8 bytes per row, so 10 million rows costs about 80 MB for that column alone. The danger is rarely the new column itself; it is that operations like df$col <- transform(df$col) hold old and new copies simultaneously, and patterns like df <- rbind(df, new_row) in a loop reallocate the entire frame on every iteration, which is the most common way this error appears on data that "should fit."
Likely explanation vs. confirmed facts
Confirmed: the message reports bytes requested, not total exhaustion; gc() frees unreachable objects but cannot fix fragmentation or a genuinely oversized working set; and a 32-bit R build has a small address space (around 2–4 GB) regardless of installed RAM. Likely, but unverified for your case: whether you are hitting physical RAM, a per-process limit, or fragmentation. There is no fixed threshold percentage of RAM where copying becomes unsustainable — it depends on contiguous free memory at the moment of the call.
Steps for this case
- Measure before the failing line:
object.size(df) and gc() (or pryr::mem_used()) to see the real footprint.
- Confirm your build:
.Machine$sizeof.pointer returns 8 on 64-bit R. If it is 4, switching to a 64-bit build is the single biggest fix.
- Replace loop
rbind with pre-allocation or data.table::rbindlist(list_of_frames).
- For in-place column changes, use
data.table with setDT(df) then df[, col := new_value], which modifies by reference and avoids the duplicate copy. You still need room for the result.
- Shrink column types before modifying: integer instead of double where values allow, factor instead of repeated character strings.
Verify the diagnosis
Reproduce on a subset, e.g. df[seq_len(1e5), ]. If the small copy succeeds and failure scales with row count, it is genuine capacity, not a bug in the operation. Wrap the failing call with options(error = traceback) to confirm exactly which modification triggers the allocation. On Linux/macOS check ulimit -v and system free memory; on Windows check memory.limit().
One detail that would change the recommendation: the exact operation in the failing line. A loop-grown frame points to restructuring the code; a single column assignment on an already-large frame points to type reduction or data.table.
r