Bazel Remote HTTP Caching: Architecture for Distributed Build Performance

The Problem: Redundant Compilation Across Machines

When multiple developers or CI agents build the same Bazel targets, each compiles dependencies independently. A large Java or C++ codebase can spend hours on identical work across machines. Remote HTTP caching solves this by storing build artifacts on a shared server, letting workers skip compilation when inputs haven't changed.

Requirements for a Working Solution

• Content-addressed storage: Cache keys must be SHA-256 hashes of action inputs to guarantee reproducibility.
• Authentication and TLS: Prevent unauthorized access and protect artifacts in transit.
• High availability: The cache server must respond quickly; unavailability blocks remote workers.
• Compression support: Large binaries need efficient encoding to balance storage and transfer costs.
• Offline operation: Workers should continue functioning when the network is down and sync later.

Smallest Suitable Design

Use Bazel 7.x+ built-in HTTP backend with a reverse proxy (nginx, envoy) in front of an object store (S3, GCS, or local filesystem). This avoids running a custom cache server while supporting all required features.

Core Configuration

Add to .bazelrc:

build --remote_cache=https://cache.example.com/cache
build --remote_auth=bearer
build --remote_header=Authorization:Bearer $(cat ~/.bazel_token)
build --remote_timeout=60
build --remote_max_retries=3
build --remote_upload_local_results=true
build --remote_verify=False

Trust and Data Boundaries

The HTTP server authenticates requests using bearer tokens or HTTP basic auth. TLS terminates at the reverse proxy, which then communicates with the object store over private network. Cache keys are untrusted user input; Bazel validates them as SHA-256 hex strings before any lookup.

Operational Checks

1. Verify cache location:

   $ bazel info repository_cache
   repository_cache: /path/to/.bazel/cache

2. Test artifact caching:

   $ bazel clean --expunge
   $ bazel build //path/to:target
   $ bazel build //path/to:target  # Second run should hit remote cache

3. Monitor traffic:

   $ sudo tcpdump -i any port 443 host cache.example.com
   $ # Observe GET/PUT requests during builds

4. Check cache status:

   $ bazel aquery //path/to:target --output=text

Failure Modes and Mitigation

Mode	Symptom	Mitigation
Server unavailable	Build hangs or fails with timeout	Set --remote_timeout and --remote_max_retries; use local fallback cache
Network latency	Remote cache slower than local
Cache poisoning	Corrupted artifacts break builds	--remote_verify; use immutable cache keys
Credential leak	Unauthorized cache access

When This Design Changes

If you need fine-grained access control per target, or must support non-SHA-256 keys, consider Google's Buildbarn or Buildkite's remote exec setup. For environments with strict air-gap requirements, use a local filesystem cache with periodic rsync instead of HTTP.

Verification Checklist

• Run bazel clean --expunge then two consecutive builds to confirm second build hits remote cache (look for remote cache hit in output).
• Monitor server logs for 200 responses on cache GET requests.
• Measure build time difference between clean local build and cached remote build.