Transparent Huge Pages on Linux: Enabling, Tuning, and Monitoring

Useful Answer

To enable Transparent Huge Pages (THP) on a running system, run the following as root:

# echo "always" > /sys/kernel/mm/transparent_hugepage/enabled
# echo "always" > /sys/kernel/mm/transparent_hugepage/defrag

After that, verify the state:

# cat /sys/kernel/mm/transparent_hugepage/enabled
# cat /sys/kernel/mm/transparent_hugepage/defrag

If both files show [always], THP is fully enabled. You can disable it with never.

How Transparent Huge Pages Work

THP allows the kernel to map contiguous 2 MiB (or 1 GiB on some architectures) pages for anonymous memory and file mappings. This reduces the number of page‑table entries and TLB misses, lowering page‑fault overhead for large allocations. THP does not replace the conventional 4 KiB pages; it simply offers a larger granularity when the kernel deems it beneficial.

The kernel exposes two control knobs:

• /sys/kernel/mm/transparent_hugepage/enabled – always, never, or madvise (per‑process control).
• /sys/kernel/mm/transparent_hugepage/defrag – always, defer, or never; controls how aggressively the kernel defragments memory to form huge pages.

Enabling THP System‑Wide

Root privileges are required because the files live in /sys/kernel/mm/transparent_hugepage. A convenient, persistent method is to create a systemd unit:

[Unit]
Description=Enable Transparent Huge Pages
After=local-fs.target

[Service]
Type=oneshot
ExecStart=/bin/sh -c "echo always > /sys/kernel/mm/transparent_hugepage/enabled && echo always > /sys/kernel/mm/transparent_hugepage/defrag"
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Enable and start the unit:

# systemctl enable --now enable-thp.service

This runs at boot and keeps THP enabled between reboots.

Per‑Process Control with madvise

When enabled is set to madvise, the kernel respects MADV_HUGEPAGE hints from user space. A minimal C example demonstrates this:

#include <unistd.h>
#include <sys/mman.h>
#include <stdio.h>

int main(void) {
    size_t size = 3 * 1024 * 1024; // 3 MiB
    void *p = mmap(NULL, size, PROT_READ | PROT_WRITE,
                    MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
    if (p == MAP_FAILED) { perror("mmap"); return 1; }
    if (madvise(p, size, MADV_HUGEPAGE) != 0) {
        perror("madvise"); return 1;
    }
    /* Use the memory */
    for (size_t i = 0; i < size; i += 4096) *((char *)p + i) = 0;
    printf("Allocated %zu bytes with MADV_HUGEPAGE\n", size);
    return 0;
}

Compile and run as a normal user (no root needed). Verify that the process now uses huge pages:

# cat /proc/$(pgrep -f "./huge_test")/smaps | grep AnonHugePages
AnonHugePages:  3072 kB

Monitoring THP Usage

Several tools expose THP statistics:

• vmstat -m – shows HugePages counters.
• grep Hugepages* /proc/meminfo – lists total, free, reserved, and surplus huge pages.
• Per‑process smaps – AnonHugePages shows the amount of THP used by a single process.

Example:

# vmstat -m
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0    0  123456  65432  98765    0    0    12    34  567  890  3  1 95  1  0
HugePages_Total:    0
HugePages_Free:    0
HugePages_Rsvd:    0
HugePages_Surp:    0

Limitations and Common Mistakes

• Fragmentation – Enabling THP globally can increase memory fragmentation, especially on workloads that frequently allocate and free small pages. This may hurt overall memory pressure.
• File System Support – Some filesystems (e.g., XFS) do not support THP for file mappings, leading to inconsistent behavior.
• Workload Suitability – THP only benefits allocations >= 2 MiB. Applications that work with many small pages (e.g., key‑value stores with tiny records) see little or no improvement.
• Defrag Setting – Setting defrag to always can cause the kernel to spend extra time moving pages around, which may degrade performance under heavy memory pressure.
• Persistence – Modifying the files at runtime works for the current session, but the change is lost on reboot unless you add the echo commands to a startup script or systemd unit.
• Kernel Support – THP requires the kernel to be compiled with CONFIG_TRANSPARENT_HUGEPAGE. Most mainstream distributions enable it by default, but some minimal or hardened kernels may disable it.
• Per‑Process Control Misuse – Using MADV_HUGEPAGE on small allocations (<2 MiB) has no effect and may waste space if the kernel later splits the huge page.

When to Consider Disabling THP

If you observe increased memory fragmentation, higher swap activity, or degraded performance on a particular workload, try disabling THP temporarily:

# echo "never" > /sys/kernel/mm/transparent_hugepage/enabled
# echo "never" > /sys/kernel/mm/transparent_hugepage/defrag

After disabling, monitor the same metrics to confirm improvement.

Summary

Transparent Huge Pages can provide measurable latency reductions for large, contiguous allocations. The key steps are:

• Set enabled to always or madvise and defrag to always for system‑wide use.
• Use madvise or prctl for per‑process control when madvise is active.
• Monitor vmstat, /proc/meminfo, and per‑process smaps to verify usage.
• Beware of fragmentation, filesystem support, and workload suitability.

Apply the configuration, test with your specific workloads, and adjust as needed to balance performance and memory efficiency.