Avoid Temporary Files With Bash Process Substitution <() and >()

You need to compare two command outputs with diff, but diff requires filenames, not pipes. Writing both outputs to temporary files works, but it adds cleanup, race conditions, and disk I/O for data that only needs to be compared in memory. Process substitution is the Bash-specific way to give file-argument utilities a live pathname to a command's stdout without creating a real file.

The takeaway is simple: <(cmd) makes Bash run cmd concurrently and replace the construct with a pathname like /dev/fd/63 that the caller can read. >(cmd) does the reverse, giving the current command a pathname whose writes are fed to cmd.

How process substitution works in Bash

Bash replaces <(cmd) with a path to a FIFO or a /dev/fd entry. The substituted command runs in a subshell at the same time as the parent command. The consumer opens the pathname and reads the producer's output as it is generated.

This matters because many core utilities are file-argument only. diff, comm, sort -m, and grep -f all expect paths. With process substitution you can pass them live streams:

# run in a Bash interactive shell or script with #!/usr/bin/env bash
diff <(curl -s https://example.com/config.yaml) <(cat /etc/app/config.yaml)

Required permissions are read access to the local file and network access for curl. The risk is that failures inside the substitution are silent by default. If curl exits non-zero, diff still runs and may compare an empty stream. Pair process substitution with strict mode and explicit checks:

set -euo pipefail
remote=$(mktemp)
# avoid temp file with substitution instead
diff <(curl -sSf https://example.com/config.yaml) <(cat /etc/app/config.yaml)

Input substitution is also useful for pattern files:

grep -f <(grep -oE '[a-z]{4}' /usr/share/dict/words | head -n 100) access.log

The command inside <() runs concurrently. Do not nest deeply, and avoid long pipelines with many substitutions because each consumes a file descriptor.

Output substitution for streaming consumers

>(cmd) creates a pathname whose writes are delivered to cmd. This lets you keep streaming behavior while adding a side consumer, such as logging or monitoring, without tee to a file.

#!/usr/bin/env bash
set -euo pipefail
# run as a user with permission to read source and write to log destination
grep ERROR app.log | sort >(gzip > /var/log/errors-$(date +%F).gz) >(wc -l > /tmp/error_count.txt)

Here sort's stdout is duplicated to two background consumers. The pathname given to sort is temporary; Bash manages the FIFO or /dev/fd link. The risk is that the background consumers run asynchronously. If gzip fails, the main pipeline may still exit zero unless you capture statuses. Error output from the substituted command is not captured by the parent unless redirected explicitly.

A safer pattern for critical logging is to redirect stderr of the consumer inside the substitution:

cmd >(logger -t myapp 2>&1)

Portability and safety limits

Process substitution is Bash-specific. It requires /dev/fd support and will fail in POSIX sh, dash, busybox ash, and minimal containers without it. Verify the interpreter before relying on it:

bash --version

Check support by inspecting the file descriptor table after a substitution. In a Bash shell:

bash -c 'cat <(echo hi); ls /proc/self/fd'

You should see a numeric fd entry for the substitution. Running the same script with dash -c will produce a syntax error near <(, confirming lack of support.

Limitations to keep in mind:

• Not portable. Scripts using <() or >() cannot run under /bin/sh. Use a Bash shebang and document the requirement.
• Concurrent execution. The substituted command starts immediately. Failures do not abort the parent unless strict mode and explicit status checks are used.
• File descriptor exhaustion. Each substitution consumes an fd. Excessive nesting or loops can hit the ulimit.
• Error visibility. Stderr from the substituted command goes to the terminal or script stderr unless redirected inside the substitution.

Practical checklist

Use process substitution when you need a filename for a streaming source and you control the Bash environment. Keep substitutions shallow, limit them to a few per command, and prefer explicit error handling for the producer commands. For scripts that must be portable, fall back to named pipes or temporary files with guaranteed cleanup.

Actionable closing: add #!/usr/bin/env bash at the top of scripts that use <() or >(), enable set -euo pipefail, and test the script with dash to confirm it intentionally fails outside Bash. That makes the Bash dependency explicit and avoids silent breakage in environments where process substitution is unavailable.