Stop Reaching for while-read: Bash's readarray Is Often the Better Tool

You need to load a file into a Bash script and do something with each line. Most of us reflexively write while IFS= read -r line; do ... done < file, because that's what every tutorial teaches. But if your file fits comfortably in memory and you need indexed or repeated access to its lines, Bash's built-in readarray (also spelled mapfile) is usually the cleaner, faster choice — and it avoids a whole class of subtle bugs.

The problem with the while-read habit

The classic loop has two recurring traps. First, if you pipe into it (cat file | while read ...), the loop body runs in a subshell, so any variables you set inside vanish when the loop ends. Second, the loop reads one line per iteration, which means per-line I/O overhead — fine for small files, noticeably slower for hundreds of thousands of lines.

readarray sidesteps both: it reads the entire file in bulk, in the current shell, straight into an indexed array.

What readarray actually does

Available since Bash 4.0, readarray loads lines from stdin into an array. The most useful flags:

• -t — strip the trailing delimiter (newline by default) from each element. Almost always what you want.
• -d DELIM — split on a different delimiter instead of newline (handy for NUL-delimited data from find -print0).
• -n N — read at most N lines.
• -s N — skip the first N lines.

Because the result is a normal array, you get random access (${lines[42]}), a line count (${#lines[@]}), and the ability to iterate as many times as you like without re-reading the file.

A worked example

Say you have a file of hostnames and want to count how many respond to ping, keeping the failures for a report. Run this in a terminal under Bash 4.0 or newer — check first with type readarray, which should report it as a shell builtin. No special permissions are needed; the script only reads a file you own.

#!/usr/bin/env bash
# hosts.txt: one hostname per line

readarray -t hosts < hosts.txt

echo "Loaded ${#hosts[@]} hosts"

failed=()
for host in "${hosts[@]}"; do
  [[ -z $host ]] && continue   # skip blank lines
  if ping -c1 -W2 "$host" &>/dev/null; then
    echo "ok: $host"
  else
    failed+=("$host")
  fi
done

printf 'failed: %s\n' "${failed[@]}" > failures.txt
echo "${#failed[@]} hosts unreachable"

Two things to notice: -t means $host never contains a stray newline, so string comparisons and ping arguments behave; and because failed is populated in the current shell (no pipeline, no subshell), it survives the loop with no workarounds like process substitution.

To verify the result, compare wc -l < failures.txt against the count printed on the last line, and spot-check that every line in failures.txt also appears in hosts.txt (e.g. with grep -Fxf failures.txt hosts.txt | wc -l).

Trade-offs you should know

Memory. The whole file lands in RAM. For a few megabytes that's nothing; for multi-gigabyte logs it can exhaust memory or trigger swapping. If you're streaming huge data and only need one pass, the while-read loop (fed via redirection, not a pipe) is the right tool.

Portability. readarray requires Bash 4.0+. Scripts targeting POSIX sh, or ancient systems (notably macOS's default Bash 3.2), will fail. Guard with a version check or use #!/usr/bin/env bash and document the requirement.

CRLF files. Windows-style line endings leave a trailing \r on each element even with -t, which breaks exact string matches. Strip them first (tr -d '\r' < file or dos2unix) if the input might come from Windows.

Whitespace. Unlike read with default IFS, readarray preserves leading and trailing whitespace in each line — usually what you want, but be aware when comparing values.

When to use which

Takeaway

Next time you catch yourself typing while read, ask: do I need streaming, or do I just need the lines? If it's the latter, readarray -t lines < file gives you faster bulk I/O, no subshell surprises, and full array semantics for free. Confirm your Bash version with bash --version, sanity-check the array with declare -p lines | head -c 200, and keep the old loop in your pocket for the genuinely big files.