Set -e + pipefail vs Manual Exit Checks: Which Guarantees Robust Deployment Log Analysis?
0 reputation · 15 May 2025, 21:44 UTC
0 reputation · 15 May 2025, 21:44 UTC
In a CI/CD pipeline, a Bash script parses log output to detect errors. The script must terminate immediately upon any failure that may corrupt downstream steps, yet it must also preserve cleanup and error reporting.
Using set -e alone causes the script to exit on the first failing command, but failures inside pipelines or subshells are often ignored unless pipefail is enabled. Conversely, manual if [[ $? -ne 0 ]] checks provide fine‑grained control but increase boilerplate and can be missed in complex pipelines.
Should all deployment scripts adopt set -e with pipefail, or should they selectively apply these options only to critical sections to balance reliability against readability?
set -e combined with pipefail reliably surface failures in command substitutions that parse logs, or are there still silent errors in Bash 5.0+?33775 reputation · 16 May 2025, 02:15 UTC
For most deployment pipelines, set -e -o pipefail -o errtrace at the top of the script gives the strongest, most readable guarantee that a failure will abort the run. It catches every failing command in a pipeline, any command that exits non‑zero outside of exempted contexts, and propagates errors from functions. However, command substitutions and subshells that capture log output can still hide failures because the outer shell receives only the substitution’s output, not its exit status. In those cases, a single if [[ $? -ne 0 ]]; then …; fi after the substitution is the safest way to surface the error.
-e exits on a non‑zero status unless the command is part of an if, while, until, &&, ||, or a conditional expression.-o pipefail makes the exit status of a pipeline the first non‑zero status, so a failure in the middle of cmd1 | cmd2 | cmd3 aborts the script.-o errtrace propagates trap ERR into functions and subshells, giving a single place to log the error.LOG=$(grep -i "error" /var/log/app.log). set -e does not abort if grep fails; capture the status immediately: LOG=$(grep …); rc=$?; if [[ $rc -ne 0 ]]; then …; fi.(…) inherits -e, but the parent will continue unless you check the subshell’s exit code.|| true only when you deliberately want to ignore a specific failure; otherwise, let -e surface it.#!/usr/bin/env bash
set -e -o pipefail -o errtrace
trap 'echo "ERROR: $BASH_COMMAND failed with exit $?'" >&2' ERR
LOG=$(grep -i "error" /var/log/app.log) || {
echo "Log grep failed – aborting" >&2
exit 1
}
RESULT=$( (some_long_command) ); rc=$?; if [[ $rc -ne 0 ]]; then echo "Subshell failed"; exit $rc; fi
-e fall short.If your log‑parsing logic relies heavily on nested command substitutions or complex pipelines that produce intermediate files, let us know. That information can change whether a blanket set -e is sufficient or whether you need more granular checks.
Use comments to ask for clarification. Post a solution as an answer.
5,170 reputation · 16 May 2025, 02:04 UTC
With set -e -o pipefail the exit status of a simple assignment like
LOG=$(grep -i error /var/log/app.log | wc -l) is the status of the
entire pipeline inside the substitution. If grep exits non‑zero, the
assignment fails and -e aborts the script immediately.
Problems arise when the substitution is used inside a context that
suppresses -e – for example, inside a test expression, a logical
conjunction, or a subshell. In those cases the outer shell receives only
the substitution’s *output*, not its status, and the script keeps running.
set -euo pipefail
# Capture the status explicitly so it is not masked by a surrounding
# construct.
LOG=$(grep -i error /var/log/app.log | wc -l) || {
echo "Log parsing failed (grep exited $?)"
exit 1
}
# If the substitution is inside a conditional, check $?.
if [[ $(grep -q "ERROR" /var/log/app.log) ]]; then
echo "Found an error"
else
echo "No errors"
fi
Adding set -o errtrace also propagates trap ERR into subshells,
so any cleanup or custom error handling defined in a function or subshell
still runs before the script terminates.
In short: the global set -e -o pipefail is almost always sufficient, but
whenever a command substitution appears inside a conditional or a
subshell, capture its exit status immediately or perform an explicit
|| exit check to guarantee the failure surfaces.