Question
Which lock‑handling strategy should Composer adopt to mitigate latency when multiple installs run concurrently?
Lax Cedar
0 reputation · 17 May 2023, 22:51 UTC
132.2K views
Loading…
0 reputation · 17 May 2023, 22:51 UTC
Composer creates a temporary lock file (.composer.lock.tmp) during dependency resolution. The file is locked exclusively until the install completes, which blocks any other Composer process that attempts to acquire the lock in the same repository.
When two or more PHP Composer processes start simultaneously, the first process obtains the exclusive lock, while the others block until the lock is released. Composer does not provide a timeout, retry mechanism, or shared‑read capability for the lock file. This behaviour is not documented in the official manual or issue tracker, and it can cause unpredictable latency spikes in CI/CD pipelines that invoke Composer in parallel.
The core uncertainty is whether Composer should implement a shared or advisory lock that allows multiple processes to read the lock file concurrently while preventing concurrent writes, or whether it should fail immediately, wait, or retry with back‑off when a lock is already held.
.composer.lock while protecting against concurrent writes?32275 reputation · 18 May 2023, 00:48 UTC
Composer should adopt an advisory lock that allows multiple processes to share a read lock on the temporary lock file while keeping an exclusive write lock for install operations. Additionally, it should implement a retry loop with exponential back‑off (e.g., 100 ms, 200 ms, 400 ms, … up to 30 s) so that a waiting process does not stall indefinitely.
composerflock) are non‑blocking for readers and still prevent concurrent writers, which matches the goal of concurrent installs that only read the lock file..composer.lock.tmp and is currently locked exclusively until install completes.flock; network filesystems may not.Modify Composer’s lock acquisition code to use flock($fd, LOCK_SH) for read‑only operations (e.g., composer show) and flock($fd, LOCK_EX) for installs.
Wrap the lock call in a loop that attempts acquisition, sleeps with an exponential back‑off, and aborts after a configurable timeout (default 30 s). Expose the timeout via an env var like COMPOSER_LOCK_RETRY_TIMEOUT.
Run two terminals in the same repo: composer install in the first, then composer install --no-scripts in the second. Verify the second logs a short wait and then proceeds.
In a CI pipeline, trigger two parallel jobs on the same repository and compare job durations before and after implementing the advisory lock. A noticeable reduction in total pipeline time indicates success.
If your CI agents mount the repo via NFS or other network FSes, confirm that flock is supported; otherwise consider a file‑based lock fallback.
Do you run Composer on a network‑mounted repository (e.g., NFS, SMB) in your CI environment? If so, the advisory lock may not be reliable, and an alternative locking strategy might be required.
Advisory locks keep the lock semantics lightweight and OS‑agnostic while allowing concurrent reads. The back‑off loop prevents a single long‑running install from blocking the entire CI pipeline, which is essential when multiple jobs share the same repository. Exposing the timeout via configuration gives teams the flexibility to tune for their specific workloads.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.