net/http and httptest: Simulating API dependencies without production credentials
0 reputation · 01 Jun 2022, 19:30 UTC
0 reputation · 01 Jun 2022, 19:30 UTC
Integration testing for Go applications often requires verifying interactions with external REST APIs. To avoid using production credentials or incurring costs during CI/CD cycles, the net/http/httptest package allows for the creation of a local server that mimics the target API's behavior.
The goal is to ensure the application's HTTP client correctly handles various response payloads and status codes by injecting the local server's URL into the client configuration. However, relying on http.DefaultClient within these tests introduces uncertainty regarding resource management and execution time.
Given that http.DefaultClient does not define a default timeout, how should the client be configured to prevent goroutine leaks if the httptest.Server handler hangs? What is the recommended approach for ensuring the local server is fully decommissioned across multiple test cases to avoid port exhaustion?
34775 reputation · 02 Jun 2022, 06:37 UTC
To prevent goroutine leaks when a httptest.Server handler hangs, you must replace http.DefaultClient with a custom http.Client that defines a explicit Timeout. Because the default client has no timeout, a stalled mock server will cause the test goroutine to block indefinitely, potentially hanging the entire CI/CD pipeline.
Initialize a scoped client within your test function or as part of your application's dependency injection. This ensures that the request is terminated regardless of the server's state.
// Recommended client configuration for tests
client := &http.Client{
Timeout: 5 * time.Second,
}
To avoid port exhaustion and ensure a clean state across multiple test cases, use the defer server.Close() pattern immediately after the server is instantiated. This ensures the listener is closed and the associated resources are released as soon as the test function returns.
For suites with many sub-tests, the recommended approach is to instantiate the server within the specific t.Run block rather than globally. This isolates the mock behavior and ensures a fresh port is allocated and released for every single case.
The following structure demonstrates the safe integration of a timed-out client and a decommissioned server:
func TestAPIInteraction(t *testing.T) {
// 1. Setup mock server
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
w.Write([]byte(`{"status": "success"}`))
}))
// 2. Ensure decommissioning to prevent port exhaustion
defer server.Close()
// 3. Configure client to prevent goroutine leaks
client := &http.Client{
Timeout: 2 * time.Second,
}
// Inject server.URL into your application logic
resp, err := client.Get(server.URL)
if err != nil {
t.Fatalf("Request failed: %v", err)
}
defer resp.Body.Close()
}
This approach assumes the use of Go 1.12 or later. To verify that the timeout is functioning correctly, you can temporarily introduce a time.Sleep longer than the client timeout inside the httptest.Server handler; the test should fail with a context deadline exceeded error rather than hanging.
Diagnostic Detail: Are you using a shared http.Transport across multiple clients? If so, the transport's IdleConnTimeout may also need configuration to ensure connections are fully purged.
Use comments to ask for clarification. Post a solution as an answer.
34,775 reputation · 01 Jun 2022, 22:25 UTC
A custom http.Transport with MaxIdleConns set to 1 or IdleConnTimeout set to 0 prevents stale connection reuse across test isolation boundaries. This ensures each httptest.Server instance operates with a clean connection scope, avoiding unexpected port reuse or leftover TLS sessions when running many integration tests in a suite. Practical verification: after test execution, observe that connection resources are released and no lingering ports persist, confirming proper cleanup.