Duplicate writes when using travis_retry on non-idempotent shell commands

The travis_retry helper is a simple shell wrapper that re-executes a command if it returns a non-zero exit code. It does not provide native transaction management, state rollback, or environment variables to indicate that a command is currently running as a retry. Consequently, any non-idempotent operation—such as appending to a file or sending an API request—will be repeated on every attempt.

Likely Cause of Duplicate Writes

Because travis_retry operates at the process level rather than the application level, it has no visibility into the internal side effects of the command it wraps. If a command performs a write operation and then fails during a subsequent step, the entire command is restarted from the beginning upon retry, triggering the write operation again.

Patterns for Idempotency

Since the environment does not provide retry-specific flags, you must implement guards within your build script to ensure operations are idempotent. Depending on the operation, use the following patterns:

1. File System Operations

Avoid using append operators (>>) without checks. Instead, verify the state of the file or use temporary files that are only moved to the final destination upon successful completion of the entire sequence.

# Unsafe: appends every retry
travis_retry npm run build-assets >> build.log

# Safer: check for existence or use a temporary file
travis_retry bash -c "if [ ! -f build.log ]; then touch build.log; fi; npm run build-assets"

2. External API Calls

For API interactions, implement Idempotency Keys. Generate a unique request ID for the specific build job and pass it in the API header. This allows the server to recognize the retry and return the original response without processing the write a second time.

3. Database Writes

Use UPSERT (Insert or Update) logic or unique constraints to prevent duplicate records. Ensure that the primary key or a unique identifier is derived from the build ID or a consistent hash of the data being written.

Verification

To verify that your guards are working, you can simulate a failure using a script that performs a write and then exits with an error:

# Create a failing script (fail_test.sh)
echo "Writing data..." >> output.txt
exit 1

# Run with travis_retry
travis_retry bash fail_test.sh

# Check output.txt; if it contains multiple "Writing data..." lines, 
# the operation is not idempotent.

Diagnostic Detail Needed: Are you wrapping a single binary execution or a complex shell script? If you are wrapping a script, moving the travis_retry logic inside the script to wrap only the specific unstable network call (rather than the entire setup) is the most effective way to limit side effects.