Nomad Reschedule Stanza and Task Writes: Where Should Idempotency Live?

Direct answer

Assuming Nomad 1.x, the scheduler provides no documented duplicate-write mechanism and injects no idempotency key. The restart and reschedule stanzas only decide when a process starts; the task write path must supply whatever uniqueness the datastore enforces.

Confirmed: stable values by re-execution path

Across a restart, the task runs inside the same allocation, so NOMAD_ALLOC_ID, NOMAD_ALLOC_NAME, and NOMAD_SHORT_ALLOC_ID keep their values and can key writes deduplicated within one allocation. NOMAD_JOB_NAME, NOMAD_TASK_NAME, and NOMAD_NAMESPACE are also stable, but they are shared by every attempt.

Across a reschedule, Nomad creates a replacement allocation, so NOMAD_ALLOC_ID and NOMAD_ALLOC_NAME change. No documented per-attempt identifier survives a reschedule. NOMAD_ALLOC_INDEX is the only plausible cross-reschedule key, but whether it is reused or incremented is version-sensitive and should be treated as unverified until observed on the running server.

Likely explanation: idempotency belongs in the task

Because the scheduler only controls process timing, the write path must enforce uniqueness. Two patterns fit the confirmed behavior:

• Continuation of the same logical work: key on a deterministic business or natural key derived from job, task, and input. The datastore unique constraint rejects duplicates.
• Distinct execution per attempt: key an external dedup table on NOMAD_ALLOC_ID plus an attempt counter maintained by the task. The counter advances on restart; a reschedule receives a new NOMAD_ALLOC_ID.

The one diagnostic that flips the recommendation

Does a reschedule mean the same logical write continues, or that a new execution begins? Continuation needs a job-scoped natural key; distinct execution needs alloc-ID-plus-attempt keying. Confirm this intent before choosing the key.

Verification on your cluster

Because stanza defaults and environment stability vary across Nomad 1.x, verify against the deployed version before depending on any field:

nomad job inspect <job>

Run a count=1 service, record NOMAD_ALLOC_ID and NOMAD_ALLOC_INDEX, drain the node to force a reschedule, then diff the new allocation environment. Trigger an in-allocation restart and confirm NOMAD_ALLOC_ID is unchanged.