Go Weighted Semaphores: Throttle Concurrency Safely

The Problem: Uncontrolled Parallelism

In microservice architectures, a single HTTP handler can easily spawn dozens or hundreds of goroutines for background work, caching, or external API calls. When the load spikes, the number of concurrent goroutines can grow without bound, exhausting OS thread pools, memory, or remote service capacity. A common symptom is a sudden drop in response latency or a surge in connection errors from downstream services.

Weighted Semaphores Explained

The golang.org/x/sync/semaphore package exposes a lightweight weighted semaphore. Two core methods drive its behaviour:

• Acquire(ctx context.Context, n int64) error – Blocks until n units are available or ctx is cancelled.
• Release(n int64) – Frees n units, potentially unblocking waiting goroutines.

Unlike a binary sync.Mutex, a weighted semaphore lets you model resources that cost different amounts of capacity. For example, a goroutine that streams a large file might acquire 5 units, while a quick lookup might only need 1.

Using sema with HTTP Clients

Consider a service that makes outbound HTTP requests to an API with a strict rate limit of 10 concurrent connections. A weighted semaphore with a maximum of 10 units can enforce this ceiling without adding extra latency to the request path.

Because Acquire accepts a context.Context, you can propagate cancellation from an incoming request or a timeout, ensuring that goroutines do not linger if the client disconnects.

Example: Throttling 1000 Requests

package main

import (
    "context"
    "fmt"
    "net/http"
    "time"

    "golang.org/x/sync/semaphore"
)

func main() {
    // A weighted semaphore that allows at most 10 concurrent HTTP calls.
    const maxConcurrent int64 = 10
    sem := semaphore.NewWeighted(maxConcurrent)

    // Simulate 1000 incoming requests.
    for i := 0; i < 1000; i++ {
        go func(id int) { // id is only for logging
            // Each request may need 1 unit of capacity.
            ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
            defer cancel()

            if err := sem.Acquire(ctx, 1); err != nil {
                fmt.Printf("%d: acquire failed: %v\n", id, err)
                return
            }
            defer sem.Release(1)

            // Perform the network call.
            // In production, use a real client; here we just sleep.
            time.Sleep(100 * time.Millisecond)
            fmt.Printf("%d: completed\n", id)
        }(i)
    }

    // Wait long enough for all goroutines to finish.
    time.Sleep(12 * time.Second)
}

Key points:

• The semaphore’s capacity is set to 10, matching the external API limit.
• Each goroutine acquires 1 unit; if an operation required more, you could pass a larger n.
• A 5‑second timeout ensures that a goroutine that cannot acquire before the deadline aborts, preventing a backlog.

Trade‑offs & Common Pitfalls

• Deadlocks: If a goroutine requests more units than the semaphore’s capacity (e.g., Acquire(ctx, 15) with a max of 10), it will block forever unless the context expires. Always validate that n <= maxConcurrent before calling Acquire.
• Under‑utilisation: Setting the capacity too low throttles throughput. Measure the typical burst of requests and set maxConcurrent accordingly.
• Memory footprint: The semaphore itself is very small (a few dozen bytes). The real cost is the number of goroutines you spawn. Use the semaphore to limit goroutine creation if needed.
• Context leakage: If you reuse a context.Context that has already been cancelled, Acquire will return immediately with an error. Always create a fresh context per operation.

Actionable Steps

1. Measure the maximum concurrent load your downstream service can handle.
2. Instantiate a semaphore.NewWeighted(max) with that limit.
3. Wrap each concurrent operation with Acquire(ctx, 1) (or the appropriate weight) and defer Release(1).
4. Use context timeouts to avoid indefinite blocking.
5. Instrument by logging or metrics the number of blocked goroutines or semaphore wait times to fine‑tune the capacity.

By following these steps, you can keep your Go service responsive, protect downstream APIs, and maintain predictable resource usage.