Diagnosing Memcached LRU Evictions: From Rising Miss Rates to Root Cause

Your application's cache hit ratio is sliding, latency is creeping up, and the stats output shows the evicted counter climbing steadily. That combination almost always means memcached's least-recently-used (LRU) eviction is discarding items faster than your workload can tolerate. This guide walks through recognizing the condition, narrowing the cause, and applying the right fix — because "add more memory" is only correct about half the time.

Recognizing the condition

Memcached evictions are normal under memory pressure, but they become a problem when they evict items that would have been read again. The recognizable pattern is three signals moving together:

• evicted counter increasing between two stats samples taken minutes apart.
• Cache hit ratio (get_hits / (get_hits + get_misses)) dropping below your baseline.
• bytes sitting at or near limit_maxbytes, meaning the cache is permanently full.

A single eviction spike after a deploy or a cache warm-up is not this condition. You want a sustained upward trend in evicted correlated with rising misses.

Cause and diagnostic table

Cause	What to check	Tell-tale sign
maxbytes too small for working set	bytes vs limit_maxbytes	bytes pinned at limit; evictions track traffic
Items too large, rejected or churned	Item sizes vs -I max item size (default 1 MB)	STORE errors client-side; uneven slab usage
TTLs too long or absent	Sample keys with stats cachedump	Old items occupying space; low natural expiry
Hot-key imbalance across slabs	stats slabs per-class eviction counts	One slab class evicting while others idle
Client-side serialization masking pressure	Connection pool and thread settings	Evictions look fine but misses still high

Ordered checks

Run these from a host with network access to the memcached port (default 11211). Read-only stats commands need no special privileges; use nc or your client's stats call.

1. Confirm eviction activity

echo stats | nc localhost 11211 | grep -E 'evicted|bytes|limit_maxbytes|get_hits|get_misses'

Sample twice, five minutes apart, under normal load. A rising evicted delta with bytes at the limit confirms active LRU eviction. If bytes is well below the limit and evictions are still rising, suspect slab-class imbalance (check 4) rather than total capacity.

2. Verify the memory limit is what you think it is

Compare limit_maxbytes against the -m flag in the service unit or startup script. A common finding: a config management change reset -m to the 64 MB default while the host has gigabytes free. This is the cheapest fix in the list.

3. Audit item sizes and TTLs

On a non-production instance or during low traffic, inspect a slab class:

echo "stats cachedump 1 20" | nc localhost 11211

This lists keys with size and expiry for slab class 1. Look for items with no expiry (or multi-day TTLs) that are read rarely — they hold memory until evicted. Also check whether any values approach the max item size; oversized SET attempts fail client-side, which shows up as misses that no amount of memory will fix.

4. Check slab-class distribution

echo stats slabs | nc localhost 11211

Memcached partitions memory into slab classes by item size. If one class shows heavy evicted counts while others have free chunks, your working set is concentrated in one size band. Modern memcached (1.5+) has a slab rebalancer and automover enabled by default; on older builds, a restart is the only way to rebalance, which means a full cache flush.

5. Rule out client behavior

If server-side numbers look healthy but the application still misses, check the client: a connection pool that is too small serializes requests and inflates perceived latency, and inconsistent key hashing across app instances can split the working set. Verify all app nodes use the same server list and hashing configuration.

Fixes tied to findings

• maxbytes undersized: raise -m and restart the service. Restarting drops all cached data, so schedule it and expect a warm-up miss spike. Leave headroom for the OS — memcached should not compete with the page cache on a shared host.
• TTLs too long: set expirations that reflect actual reuse windows. Shorter TTLs let natural expiry do the work eviction was doing, and expired items are reclaimed without LRU churn.
• Oversized items: either compress values client-side, split large objects, or raise -I deliberately. Raising -I changes slab geometry, so restart and re-verify with stats slabs.
• Slab imbalance: on 1.5+, confirm the rebalancer is active; on older versions, plan a rolling restart across the pool rather than a single flush.
• Client misconfiguration: align pool sizes with concurrency and confirm consistent hashing. No server change needed.

Avoid flush_all as a diagnostic step. It invalidates everything, produces a thundering-herd miss wave against your backing store, and tells you nothing about why evictions happened.

Verifying the fix

After any change, repeat check 1: sample evicted, bytes, and hit/miss counters twice under comparable load. Success looks like the eviction rate falling toward zero while the hit ratio recovers to baseline. If you raised memory, also confirm bytes now plateaus below the new limit rather than immediately filling it — filling instantly suggests the working set is larger than estimated.

When to escalate

Escalate beyond tuning when any of these hold: the working set genuinely exceeds what a single host can hold (consider sharding across more memcached instances or a different cache tier); evictions persist after doubling memory with unchanged hit ratio (re-examine whether the data is cacheable at all — very low reuse means caching adds overhead without benefit); or you see eviction behavior that differs between clients using the text and binary protocols, which warrants a version compatibility review before further changes.

Behavior described here reflects memcached 1.5 and later defaults, including the segmented LRU. If you run an older release, verify eviction and slab behavior against your version's release notes before applying slab-related fixes.