Throttling Resource Contention with Counting Semaphores

The Cost of Unbounded Concurrency

When a batch job fans out across a third-party API or a database, the instinct is to launch as many concurrent requests as the runtime allows. That instinct usually ends in 429 Too Many Requests responses, connection-pool exhaustion, or a system that spends more time context-switching than doing useful work.

The missing piece is not more hardware; it is admission control. You need a way to say, "only three threads may run this operation at once, and everyone else waits in line." A counting semaphore is the primitive built for exactly that sentence.

Counting Semaphores vs. Mutexes

A mutex is a binary lock: one thread holds it or nobody does. A counting semaphore instead maintains a pool of permits initialized to a number you choose. Picture a bike rental stand with ten bikes rather than a single bathroom key.

• acquire() takes a permit and decrements the count. If the count is zero, the calling thread blocks until a permit is returned.
• release() returns a permit, increments the count, and can wake one waiting thread.

Because permits are not owned by a thread, semaphores fit two jobs a mutex cannot: throttling concurrent access to a bounded resource, and signaling between threads where a producer can release a permit before any consumer has tried to acquire it.

A Worked Example: Capping API Calls at Three

Suppose an external image service allows three concurrent requests per account and you must process a batch of 100 images. A semaphore enforces that cap no matter how many worker threads the application spawns.

// Java-style pseudocode, not a runnable program
Semaphore apiSemaphore = new Semaphore(3);

void processImage(Image img) {
    boolean acquired = false;
    try {
        apiSemaphore.acquire();      // blocks while all 3 permits are in use
        acquired = true;
        callExternalService(img);     // at most 3 calls in flight
    } catch (InterruptedException e) {
        Thread.currentThread().interrupt();
    } finally {
        if (acquired) {
            apiSemaphore.release();  // return only a permit we actually took
        }
    }
}

The acquired flag is the detail most implementations get wrong. If acquire() throws because the thread was interrupted while waiting, the finally block must not call release(). Releasing a permit nobody took inflates the pool, and the throttle silently drifts above the limit you promised the provider.

Checking That the Throttle Actually Holds

Instrument the guarded section with an atomic counter: increment on entry, decrement on exit. Launch 10 threads against the 3-permit semaphore and log the counter's peak value. If it ever exceeds 3, the semaphore is misconfigured or some code path bypasses it. A second check is a bounded wait: tryAcquire(500, TimeUnit.MILLISECONDS) returning false under load confirms the pool is saturated and callers are receiving backpressure instead of overrunning the downstream service.

Failure Modes Worth Designing Around

• Permit leaks: a path that acquires but never releases, typically an early return that skips cleanup, permanently shrinks the pool until the count reaches zero and every new caller hangs.
• Deadlock by ordering: if two threads acquire multiple semaphores in different orders, each can end up holding a permit the other needs. Fix it by acquiring semaphores in one documented order everywhere.
• Priority inversion: a high-priority thread can stall behind a low-priority permit holder, producing latency spikes that average-throughput dashboards will not show.

These risks also mark where a semaphore is the wrong tool:

Requirement	Mutex	Counting semaphore
Protect one shared variable or invariant	Yes	No
Cap concurrent API or database calls	No	Yes
Manage a fixed pool of connections or workers	No	Yes
Enforce ownership: only the locker may unlock	Yes	No

Three Rules to Implement By

Set the permit count from the downstream system's limits, not your CPU core count. Release only permits you actually acquired, inside a finally block guarded by an acquired flag. And on paths where an indefinite hang is worse than deferred work, prefer tryAcquire with a timeout so saturation degrades into backpressure rather than a stuck thread.