Stopping Memcached Evictions with Slab Allocation Tuning

You are monitoring your Memcached instance and notice a high rate of evictions, yet your system monitor shows plenty of available RAM. This paradox usually isn't a memory leak; it is a symptom of internal fragmentation within the slab allocation system. Because Memcached avoids the overhead of frequent system-level memory requests, it pre-allocates memory into fixed-size buckets. If your data doesn't fit those buckets efficiently, you waste space and trigger premature data removal.

To solve this, you need to align Memcached's slab classes with your application's specific payload sizes. This ensures that memory is utilized effectively and that items aren't evicted simply because one specific size bucket is full.

How Slab Allocation Manages Memory

Memcached uses a slab allocation strategy to maintain high performance under heavy write loads. Instead of requesting a new block of memory from the OS for every key-value pair, it divides its total allocated memory (defined by the -m flag) into slab classes.

Each slab class represents a specific range of data sizes. Within these classes, memory is further divided into equal-sized slots. When you store an item, Memcached identifies the smallest slab class that can accommodate that item's size and assigns it to an available slot. This prevents the memory fragmentation that typically occurs when an application frequently allocates and deallocates varying sizes of memory.

The Risk of Internal Fragmentation

The trade-off for this speed is internal fragmentation. If your application stores objects that are 65 bytes, but the available slab classes are 64 bytes and 80 bytes, every single object will be placed in an 80-byte slot. This results in 15 bytes of wasted space per item.

More critically, slab allocation is not automatically dynamic. If the 80-byte slab class is completely full, Memcached cannot "borrow" space from the 64-byte class, even if that class is entirely empty. In this scenario, Memcached will evict existing items from the 80-byte class to make room for new ones, leading to a lower cache hit rate despite having total free memory available in other classes.

Diagnostic Example: Identifying Slab Misalignment

To determine if your slab configuration is causing evictions, you must inspect the real-time usage of each class. Run the following command on the host where Memcached is running:

# Inspect slab statistics (requires access to the memcached port, usually 11211)
memcached-stat slabs

Analyze the output for a mismatch between cur_items and evicted. Consider this hypothetical scenario:

Class	Item Size	Cur Items	Evicted	Status
1	64B	100	0	Underutilized
2	80B	50,000	12,000	Saturated
3	112B	50	0	Underutilized

In this case, the 80B class is heavily saturated and triggering evictions, while the surrounding classes are nearly empty. This indicates your data payload is consistently falling just above 64 bytes, creating a bottleneck in a single slab class.

Tuning the Configuration

To mitigate this, you can adjust the number of slab classes during startup using the -s flag. Increasing the number of classes provides a finer-grained distribution of sizes, reducing the gap between the item size and the slot size.

# Restart memcached with 2GB of RAM and 128 slab classes (default is typically 64)
# Run as root or via sudo to ensure proper memory allocation
sudo memcached -m 2048 -s 128

Risk: Increasing the number of classes too far can lead to more fragmented memory across the entire instance, as memory is spread across more buckets. Always monitor the evictions metric after a change to ensure the hit rate improves.

Verifying the Result

After restarting with new parameters, perform a load test with your production payload sizes and run memcached-stat slabs again. You are looking for a more even distribution of cur_items across multiple classes and a significant decrease in the evicted count for the previously saturated class.

Because changing slab settings requires a restart, this operation will clear all existing cached data. Plan for a cache-warming period to avoid a database spike upon restart.