git index.lock error: retrying without duplicate writes
0 reputation · 16 Oct 2022, 15:35 UTC
0 reputation · 16 Oct 2022, 15:35 UTC
When multiple Git processes attempt to modify the index simultaneously, Git creates an .git/index.lock file. If a second process starts while the lock exists, Git aborts with the message fatal: Unable to create '.git/index.lock': File exists. The goal is to retry the operation after the lock is released, but without re‑writing the same objects or duplicating work that may have already been performed by the first process.
index.lock files may persist if a process crashes, requiring manual removal or a special flag.1. Does Git provide a documented flag or configuration that automatically waits for index.lock to be released and then retries the operation without re‑creating objects?
2. If no such flag exists, what is the recommended pattern for safely retrying an index‑modifying command while ensuring idempotency and avoiding duplicate writes?
3. How does NFS‑specific locking behavior affect the reliability of index.lock and what mitigations are documented in Git’s official guidance?
33775 reputation · 16 Oct 2022, 21:49 UTC
No. Modern Git (2.x) does not expose a documented flag or configuration that waits for .git/index.lock to clear and then automatically retries an index-modifying command. Settings such as core.filesRefLockTimeout govern reference locks, not the index lock, and --no-optional-locks only suppresses optional refreshes. You must serialize writers yourself or wrap the command in a bounded retry loop.
Likely explanation: two Git processes raced to create .git/index.lock. The first succeeded; the second saw the file and aborted before touching the index. That abort alone does not mean objects were duplicated. Git objects are content-addressed, so re-running git add for the same paths after the lock clears normally creates no new blobs, and the index simply converges to the last writer's state.
Confirmed risk: the real duplicate-work danger is commits. If another process already committed while your process was blocked, a blind retry can produce an empty commit or a redundant one. The index write is cheap to repeat; the commit is not.
Serialize every index writer for the repository with an external guard lock. Do not use .git/index.lock itself as the guard—deleting or reusing it can corrupt the index. A separate flock around the Git command is the portable baseline:
flock .git/guard.lock git add -A
flock .git/guard.lock git commit -m "apply changes"If you cannot hold a global lock, use only a bounded backoff loop around lock acquisition. Wait while the lock file exists, then re-check repository state before re-running the write:
while [ -e .git/index.lock ]; do sleep 1; done
git --no-optional-locks add -A
git rev-parse HEAD > /tmp/head.before
git diff --cached --quiet || echo "staged changes present"
git --no-optional-locks commit -m "apply changes" || trueAfter any retry, compare git rev-parse HEAD and git diff --cached against pre-retry values. If HEAD is unchanged and the index is clean, treat the work as already done. Do not add --allow-empty unless empty commits are intentional.
On NFS, index.lock relies on atomic exclusive-create semantics that client caching can delay or violate. The lock may be invisible to a second client or falsely reported as present. There is no index-specific NFS retry knob; the documented-safe mitigation is a single writer or a reliable lock mechanism outside Git.
Git does not auto-clear stale index.lock files. Remove one only after confirming no live Git process is using the repository—otherwise deleting a live lock can corrupt the index.
To confirm the behavior in your environment, run two concurrent git add processes in a scratch repository and verify the second fails with the expected message. Then check whether core.filesRefLockTimeout is set and confirm it does not affect the index lock in your Git version.
One missing detail changes the recommendation: is your repository on a local disk or a network mount such as NFS? On local disk, the flock wrapper is usually sufficient. On NFS, you should treat the repository as single-writer or move the lock to a reliable external service.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.