Diagnosing Missing Records When getline Is Used Inside awk Loops

Records disappear after a loop that uses getline

You run a gawk script over a file and the output is shorter than expected. NR (Number of Records) jumps unexpectedly, or the final line of a small test file never appears. The script contains a for or while loop and a call to getline inside it. The useful takeaway is that getline reads from the input stream by default; any read that is not captured or is directed at the wrong descriptor permanently advances the main record pointer. Correcting the read form and record separator handling restores the missing records.

Recognizable condition

The symptom is record loss rather than a script error. The script processes the first few records correctly, then skips one or more subsequent records. This skip is repeatable and typically correlates with the iteration count of the loop. NR and FNR (File Number of Records) no longer match the physical line numbers of the input file.

Cause and diagnostic table

Observed behavior	Likely cause in gawk	Quick diagnostic signal
One record missing per loop iteration	getline called without assignment (e.g., getline) or getline < file without capturing the return value. The read consumes the next record from the main input stream.	Search script for bare getline calls or those without a left-hand variable.
Multiple lines merged, then later lines skipped	RS (Record Separator) is changed inside the loop and not restored before the next implicit read. gawk uses the current RS to delimit the next record.	Print RS before and after the loop, or grep for RS = inside a block.
Skipping persists after fixing getline and RS	Embedded record separators in the input or unclosed file descriptors when processing multiple files.	Check for close() calls on file descriptors opened with getline < fd.

Ordered checks

1. Locate all getline uses. Run a syntax check with lint on a host where gawk is installed and you have read permission for the script:

   gawk --lint script.awk

   Lint flags misuse of getline and uninitialized RS changes.
2. Inspect the getline form. A safe form assigns the result and tests it:

   line = getline < fd

   or

   if ((getline line) > 0) { ... }

   A bare getline or getline without an assignment consumes input and discards it. Note the risk: the consumed record cannot be recovered.
3. Check RS mutation. Add temporary prints of RS and NR around the loop, or run with debug mode:

   gawk -d -v RS="\n" script.awk file.txt

   Observe whether RS changes inside the loop and whether NR increments as expected. Changing RS affects how the next record is split.
4. Build a minimal reproducible case. Create a three-line test file and a script that uses getline without assignment inside a loop. Run the script and compare the number of processed records to the file length by printing NR. This isolates the loss from other logic.
5. Trace the point of loss. Insert print statements before and after each getline to log NR, FNR, and the variable that receives the line. The point where NR jumps indicates the consuming read.

Fixes tied to findings

getline consumes the main stream

Always use the assignment form and check the return value. The return value is 1 for success, 0 for EOF, and -1 for error. Example pattern:

fd = "aux.txt"
while ((getline line < fd) > 0) {
    # process line
}
close(fd)

Never use getline without a left-hand variable unless you intentionally want to discard the next record. If you need to read from the main input stream explicitly, use getline without a file descriptor but still capture the line into a variable.

RS changed and not restored

Save the original separator and restore it after the loop:

origRS = RS
RS = "\r\n"
# perform loop work
RS = origRS

Alternatively, perform RS-specific work in a separate script or use a different variable for parsing instead of changing the global RS.

File descriptor hygiene

When getline reads from a file descriptor, close it when done. Unclosed descriptors can cause later reads to hit the wrong source when processing multiple files.

Escalation criteria

If records still skip after correcting getline usage and RS handling, suspect input data issues. Embedded record separators (e.g., blank lines when RS is empty) can cause multi-line records. Also review scripts that process multiple files with a single loop and getline, as the main input pointer and the auxiliary descriptor can interact.

Escalate for review when: lint reports remain after fixes, debug output shows NR advancing without a corresponding getline call, or the script must run on non-gawk implementations. Behavior of getline and RS differs across mawk, nawk, and gawk.

Limitations and practical verification

This guidance targets GNU awk 5.x and newer. Other awk implementations handle getline and RS differently. Do not assume portability.

Verify the fix by re-running the minimal test case and confirming that NR reaches the expected final value and that all input lines are accounted for in the output. Remove temporary prints after verification to avoid performance overhead.