Does CircleCI Docker executor preserve environment variables across re-queued jobs?

No, the CircleCI Docker executor does not preserve environment variables across re-queued jobs. Each time a job is re-queued or restarted, CircleCI instantiates a fresh container based on the specified Docker image. Any environment variables dynamically exported to the shell during the previous execution are lost because the container's memory and shell state are destroyed upon failure or completion.

Mechanism of State Loss

In the Docker executor, the environment is initialized from the .circleci/config.yml definition and any project-level environment variables configured in the CircleCI web UI. When a job fails and is re-run, the system does not "resume" the previous container; it triggers a new lifecycle. Consequently, there is no native inheritance of the runtime shell environment from a failed attempt.

Recommended Persistence Strategy

To maintain dynamic variables across re-queued jobs, you must externalize the state to the filesystem. The standard approach involves writing the variables to a file and utilizing the persist_to_workspace feature.

1. Export to File: Write the dynamic variable to a file (e.g., .env or a specific state file) during the initial run.
2. Persist: Use the persist_to_workspace step to save that file to the CircleCI workspace.
3. Restore and Source: In the re-queued job (or subsequent jobs), use attach_workspace and source the file to re-populate the shell environment.

# Example: Saving a dynamic variable
echo "DYNAMIC_VAR=$VALUE" > dynamic_env.txt
# Then use persist_to_workspace in the config

# Example: Recovering in a re-queued/subsequent job
export $(cat dynamic_env.txt | xargs)

Security Consideration

Avoid writing sensitive secrets to workspace files. Files persisted to the workspace may be accessible to other jobs within the same pipeline. For sensitive dynamic data, consider using an external secret manager or encrypted store.

Verification

To verify this behavior in your specific environment, you can run a job that exports a variable, intentionally triggers a failure (e.g., exit 1), and then re-runs the job to check if the variable persists using printenv.

Diagnostic Detail Needed: Are you attempting to persist these variables across a manual re-run of a single job, or across different jobs within a sequential pipeline workflow? The latter is the primary use case for workspaces, while the former may require a change in how you handle idempotency in your scripts.