Distributed Memcached Caching: Client-Side Hashing vs Centralized Lookup

Decision: Where to Resolve Cache Keys

When read traffic repeatedly asks the database for the same rows—user profiles, session data, or social graph edges—the database becomes the bottleneck. Moving that hot data into an in-memory cache reduces query volume, but the application still needs to know which cache server holds a given key. The decision is between a centralized lookup service and client-side distributed hashing.

Useful takeaway: client-side hashing removes an extra network hop and avoids a central coordinator, but it shifts routing logic into the application and requires consistent hashing to avoid mass cache invalidation when nodes change.

Supported Options Compared

Factor	Centralized Cache Router	Client-Side Distributed Hashing
Network path per read	Application → router → cache (two hops)	Application → cache (one hop)
Failure exposure	Router becomes a bottleneck or failure point unless made redundant	Losing one node loses only the keys mapped to that node
Operational complexity	Higher; requires running and scaling a routing tier	Medium; routing logic lives in the client library
Key remapping on node change	Managed by the router	Depends on hashing scheme; naive modulo remaps most keys

Note: The failure behavior of a centralized router depends on implementation. A redundant router tier can eliminate the single point of failure, but it adds cost and operational surface area. The table describes common trade-offs, not guaranteed behavior in every deployment.

Trade-Offs of Memcached

Memcached is an in-memory key-value store chosen for simplicity and throughput. It does not write data to disk by default, and standard Memcached does not replicate keys across nodes. These properties create specific engineering risks:

• Cold start: After a cluster restart, the cache is empty. A sudden traffic spike can send thousands of refill requests to the database simultaneously, overwhelming it.
• No native replication: If a node fails, keys assigned to that node are unavailable until the application refetches them from the database and stores them elsewhere.
• Application-managed serialization: Memcached stores opaque values. Versioning, encoding, and cache invalidation are the application's responsibility.

Public descriptions of Facebook's Memcached deployment exist in engineering literature, but those specifics are treated here as unverified model knowledge. This guide covers the general distributed-caching decision, not a confirmed reproduction of any Facebook system.

Implementation: Consistent Hashing on the Client

A common mistake is to use simple modulo hashing (hash(key) % server_count). That is cheap to compute, but adding or removing a server changes the divisor and remaps nearly every key to a different node. Consistent hashing places servers and keys on the same logical ring; when a node is added or removed, only a small fraction of keys move.

The following pseudocode shows the structure of a client-side resolver. It assumes the application knows the current list of Memcached nodes and uses a hash ring implementation provided by the client library.

# Run this logic inside the application data-access layer.
# Required: network access from application hosts to Memcached nodes (default port 11211).
# Required: a consistent-hashing library or internal ring implementation.

def resolve_server(key, hash_ring):
    return hash_ring.get_node(key)

def get_cached_value(key):
    server = resolve_server(key, hash_ring)
    value = memcached_client.get(server, key)
    if value is None:
        value = database_query(key)
        memcached_client.set(server, key, value, ttl=300)
    return value

Where to run: Application servers or service instances that read from Memcached.

Permissions: The application process needs outbound TCP access to Memcached nodes on the configured port, and read/write access to the backend database for cache misses.

Placeholders: Replace hash_ring with your client library's consistent-hashing object; replace ttl=300 with a cache lifetime suited to your data.

Risks: If all nodes are unreachable, every request becomes a cache miss and hits the database. Use circuit breakers or graceful degradation to avoid turning a cache outage into a database outage.

Validation and Diagnostics

These checks should be performed in a staging environment before production rollout. Do not treat the figures below as guaranteed measurements; measure them in your own environment.

1. Key distribution: Run stats items on each Memcached node after loading a representative workload. Item counts should be roughly balanced across nodes. Significant skew indicates a hashing or key-design problem.
2. Latency baseline: Measure GET latency from Memcached and equivalent database query latency under load. Compare the two to confirm the cache provides a meaningful reduction for your workload.
3. Node failure simulation: Stop one Memcached process. Confirm the application continues serving requests by fetching missing keys from the database and that only the affected key range experiences misses.
4. Node addition or removal: Add or remove a node and measure the fraction of keys that move to a different node. With consistent hashing, this should be small; with modulo hashing, it will be large.

Rollback

Enabling the cache changes the application's data path. To roll back, update the application configuration to bypass Memcached and route all reads directly to the database. Before disabling the cache in production, confirm the database can absorb the full read load without exceeding connection limits or latency targets.

Limitations

• This guide does not verify any specific Facebook architecture or configuration.
• Memcached behavior varies by client library and server version; consult your library's documentation for consistent-hashing support.
• All latency and distribution figures are illustrative and must be validated in your environment.