Choosing Between a Shared http:Client and Per‑Request Clients in Ballerina Swan Lake

Decision: Shared http:Client vs Per‑Request Client

When designing outbound HTTP calls in a Ballerina Swan Lake service, you must decide whether to reuse a single http:Client instance with connection pooling or to instantiate a new client for each request. This decision affects latency, resource consumption, and the reliability of the service under load.

Constraints and goals

• Minimize handshake latency (DNS, TLS, TCP SYN).
• Avoid exhausting the client host’s ephemeral ports.
• Keep the implementation simple and thread‑safe.
• Allow graceful shutdown without leaking sockets.

Comparison of options

Option	Connection reuse	DNS/TLS overhead	Ephemeral‑port risk	Code complexity	Retry handling	Shutdown safety
Shared module‑level client	Yes – pooled connections are reused across strands	Amortized; only on pool miss or idle timeout	Low – limited by pool size	Low – client declared once and injected	Manual – configure retryConfig if needed	Safe – call close() during service stop
Per‑request client	No – each request gets a fresh socket	Full DNS resolution, TCP handshake, TLS negotiation per call	High – can approach the OS ephemeral‑port limit under concurrency	Very low – client created and closed in the resource	Manual – each client would need its own retry config	Risky – must ensure close() is called before the response is sent; otherwise sockets leak

Trade‑off explanation

A shared client leverages Ballerina’s internal connection pool, which is safe for concurrent use from multiple strands. The pool reduces the frequency of expensive operations such as DNS lookups and TLS handshakes, leading to lower average latency and predictable socket usage. The per‑request approach avoids any pooling complexity but incurs the full cost of establishing a new connection for every outbound call. Under sustained load, the per‑request model can quickly consume available ephemeral ports, causing bind() failures or increased latency due to port‑allocation contention.

Concrete implementation

Shared client (module‑scope)

// shared_client.bal
import ballerina/http;

// Declare the client once at module scope.
// Adjust pool limits and timeouts as needed for your workload.
http:Client sharedClient = new (
    {
        poolConfig: {
            maxIdleConnections: 100,
            maxPendingConnections: 1000
        },
        timeout: 30s
    }
);

service /api on new http:Listener(9090) {
    resource function get greeting() returns string {
        // The client is safe to use from multiple strands concurrently.
        http:Response resp = check sharedClient->get("https://httpbin.org/get");
        return check resp.getTextPayload();
    }
}

// Optional: graceful shutdown hook to release pooled connections.
service /api on new http:Listener(9090) {
    // Attach a stop listener if your service framework supports it.
    // For illustration, assume a function `onStop` is invoked during shutdown.
    // function onStop() {
    //     check sharedClient.close();
    // }
}

Per‑request client

// per_request.bal
import ballerina/http;

service /api on new http:Listener(9090) {
    resource function get greeting() returns string {
        // A new client is created for each request.
        http:Client client = new ();
        http:Response resp = check client->get("https://httpbin.org/get");
        // It is essential to close the client after the response is obtained.
        check client.close();
        return check resp.getTextPayload();
    }
}

Validation steps

1. Build and run the shared‑client service: bal run shared_client.bal
2. In another terminal, generate concurrent traffic (e.g., hey -z 1m -c 50 http://localhost:9090/greeting).
3. While the load is active, inspect the TCP state to the target host: ss -tp | grep :9090 or netstat -anp | grep ESTAB. You should see a relatively small, steady number of ESTABLISHED connections that matches the pool size.
4. Stop the load test and repeat the experiment with the per‑request version (bal run per_request.bal). Observe a higher connection count that rises with concurrency and may approach the limit shown by cat /proc/sys/net/ipv4/ip_local_port_range.
5. Compare latency metrics reported by the load‑testing tool; the shared client typically shows lower average latency because the TLS handshake is reused.
6. After stopping the service, verify that no sockets remain in ESTAB or TIME_WAIT for the shared client when a graceful shutdown hook calls sharedClient.close(). For the per‑request variant, ensure each request’s client.close() is executed; otherwise sockets will linger.

Limits and checks

• Default pool settings (e.g., maxIdleConnections) differ between Ballerina 1.x and Swan Lake releases. Inspect the http:ClientConfiguration record in your local distribution to confirm the exact field names and defaults.
• A shared client does not automatically retry failed idempotent requests. If retries are required, configure retryConfig on the client or implement a retry wrapper.
• Using insecure TLS settings (insecureSkipVerification: true) may work in a development environment but will fail against production services that enforce certificate validation.
• Never invoke close() on a shared client while there are in‑flight requests borrowing connections from its pool; doing so can terminate those requests prematurely.
• To verify that connection reuse is occurring, enable Ballerina HTTP tracing (bal run --trace) and look for log messages indicating “reused connection” versus “new connection”.