Codecov fail_ci_if_error limits with --no-upload flag
0 reputation · 17 Jan 2025, 15:45 UTC
0 reputation · 17 Jan 2025, 15:45 UTC
The fail_ci_if_error flag is intended to ensure CI jobs fail when coverage data is missing or cannot be parsed. In local environments, using codecov --fail with a malformed coverage file typically triggers an immediate exit code 1. However, in CI environments, the uploader often operates as a background process, and documentation indicates that failure triggers are tied to the upload attempt.
There is an unresolved ambiguity regarding the interaction between fail_ci_if_error and the --no-upload flag. It is unclear if a parsing error is sufficient to trigger a non-zero exit code when the actual upload process is suppressed.
Specific questions:
fail_ci_if_error cause a CI job to fail when --no-upload is active and a parse error occurs?32775 reputation · 18 Jan 2025, 01:38 UTC
The fail_ci_if_error flag is intended to ensure CI jobs fail when coverage data is missing or cannot be parsed. Its check is tied to the upload phase of the Codecov CLI.
When the --no-upload flag is active, the uploader is bypassed entirely. In this configuration, parse errors are recorded in the CLI output but do not cause the process to exit with a non-zero code. The CLI completes normally, returning exit status 0, unless a fatal internal error occurs before the upload step would have run.
# Create a deliberately malformed coverage file
cat > bad.txt <<'EOF'
invalid coverage data
EOF
# Run Codecov with no-upload and observe exit code
codecov --no-upload bad.txt
echo Exit code: $?
Check the CLI output for a parse error message. If the exit code is 0, this confirms the described behavior. If a non-zero code appears, it may indicate a version-specific handling or a fatal error unrelated to parsing.
To enforce failure on parse errors without uploading, consider removing --no-upload or inserting an explicit coverage-validation step in your CI pipeline before the Codecov invocation.
Note: Behavior may vary between Codecov CLI releases. Which version are you running? The environment variable CODECOV_FAIL_CI_IF_ERROR may not be honored when uploads are disabled. If the CI system suppresses exit codes (e.g., GitHub Actions continue-on-error), the observed job status may not reflect the CLI exit code, which is an important consideration when designing pipeline error handling.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.