Duplicate writes in Jenkins Pipeline retry step during transient failures

Short answer

No. The retry step has no built-in option to detect that a side-effect already succeeded, and it cannot be configured or extended to skip re-execution based on prior writes. It is purely a control-flow wrapper: when the block throws, the entire block runs again, up to the configured count. Jenkins has no visibility into what your block did to a file, database, or API before it failed, so idempotency is entirely your responsibility — either inside the block or in the external system you write to.

Why duplicates happen

The classic failure mode is a write that succeeds but whose confirmation is lost: the database insert commits, then the network drops before the response arrives, the step throws, and retry runs the block again — inserting a second row. The same applies to appending to files, publishing messages, or calling non-idempotent HTTP endpoints. retry catches any exception and re-executes unconditionally; it keeps no record of partial progress.

What to do instead

Make the retried block idempotent using one of these patterns:

• Check before write. For files, guard with fileExists; for databases, query for the record first. This works when the check and write target the same agent and the state is observable.
• Let the external system enforce uniqueness. Use a database unique constraint or an idempotency key derived from something stable across retries — e.g. ${env.JOB_NAME}-${env.BUILD_NUMBER} — and treat a constraint violation as success rather than failure.
• Split the block. Put only the failure-prone operation inside retry, and keep one-time writes outside it, so a retry never re-runs the write at all.

def idemKey = "${env.JOB_NAME}-${env.BUILD_NUMBER}".replaceAll('[^A-Za-z0-9-]', '-')
retry(3) {
    if (!fileExists("done-${idemKey}.marker")) {
        sh "./upload.sh --idempotency-key ${idemKey}"
        writeFile file: "done-${idemKey}.marker", text: 'ok'
    }
}

Note that RETRY_COUNT only tells you which attempt is running — it does not tell you whether a previous attempt partially succeeded, so it cannot replace a state check.

Verify the behavior yourself

On a test controller, run a retry(3) block that appends a line to a file and then throws on the first two attempts (e.g. if (env.RETRY_COUNT.toInteger() < 3) { error 'boom' }). You will see the line appended three times. Add a fileExists guard and repeat: the line appears once. This confirms both the problem and the fix on your exact Jenkins version.

Assumptions

This reflects long-standing behavior of the Pipeline Basic Steps plugin; there is no announced core mechanism for automatic side-effect detection. If your writes go to a remote API, the strongest fix is an idempotency key accepted by that API — local checks cannot see state on the far side of a failed request.