Transparent Huge Pages on Linux: Choosing always, madvise, or never

The decision: choose a THP mode before memory pressure appears

Transparent Huge Pages (THP) is a kernel feature that automatically groups anonymous memory into 2 MB pages instead of the usual 4 KB pages. The goal is fewer TLB misses and faster memory access for large workloads. The problem is that THP is not free: allocating and compacting huge pages can stall applications, increase memory fragmentation, and inflate resident set size. Database and latency-sensitive workloads often behave worse with THP enabled, while large sequential analytics jobs can benefit.

Useful takeaway: do not enable THP unconditionally. Start from your distribution default, then use madvise for mixed workloads or never for latency-sensitive and database workloads after staging tests. Reserve always for workloads already known to benefit from huge anonymous pages.

Constraints and assumptions

• Root access is required to change the THP mode at runtime or persist it through the bootloader.
• The kernel must be built with CONFIG_TRANSPARENT_HUGEPAGE; most mainstream distributions enable it.
• Behavior below assumes a modern mainstream kernel (5.x or 6.x). Older kernels may expose different sysfs paths or lack some options.
• Distribution tuned profiles and init scripts can rewrite the THP setting at boot or when a profile activates, so manual changes must be re-checked after profile changes.
• NUMA layout, application memory layout, and whether the runtime can issue madvise(MADV_HUGEPAGE) all affect whether a mode produces any benefit.

Supported options compared

Mode	What it does	Best suited for	Main risk
always	Marks all eligible anonymous mappings for huge pages automatically.	Large sequential, memory-heavy batch jobs with known huge-page behavior.	Allocation stalls during compaction, memory fragmentation, higher RSS, poor database behavior.
madvise	Enables THP only for regions explicitly advised with madvise(MADV_HUGEPAGE).	Mixed workloads and instrumented applications or allocators that opt in.	No effect unless the application or allocator requests huge pages; benefits are opt-in.
never	Disables THP for new anonymous allocations.	Latency-sensitive services, databases, and workloads requiring predictable tail latency.	More TLB misses and higher CPU work for very large heaps or SGAs.

THP is distinct from static hugepages (vm.nr_hugepages or Hugetlb in /proc/meminfo). Do not confuse the two mechanisms.

Trade-offs to weigh

always

always simplifies adoption because applications need no changes, but it can trigger page-table migration and compaction at allocation time. Those stalls show up as latency spikes. It can also increase RSS because partially used huge pages still consume a full 2 MB frame.

madvise

madvise is safer for general-purpose hosts because THP is not applied globally. The downside is that uninstrumented applications see no benefit; you must confirm that the runtime, language allocator, or database actually calls madvise(MADV_HUGEPAGE).

never

never gives the most predictable latency, which is why it is commonly chosen for database workloads. The cost is more TLB pressure when the working set is very large. Test with production-like memory size and concurrency before accepting that trade-off.

Check the current setting

Run this on the target host. Reading the file does not require root; writing it does.

cat /sys/kernel/mm/transparent_hugepage/enabled

Expected check: the active option is bracketed, for example always [madvise] never. If the bracket is on always, THP is being applied to eligible anonymous memory by default.

Change the mode temporarily

As root, write the desired mode to the same sysfs file. Replace madvise with never or always as appropriate.

echo madvise > /sys/kernel/mm/transparent_hugepage/enabled

Relevant risks: this affects future allocations only. Existing anonymous memory is not retroactively converted or restored. The change may trigger compaction for subsequent huge-page allocations.

Rollback: restore the previous mode by writing it back, for example echo always > /sys/kernel/mm/transparent_hugepage/enabled, then confirm the bracket moved in the enabled file.

Persist across reboots

Add a kernel boot parameter to the bootloader configuration. For example, in /etc/default/grub:

GRUB_CMDLINE_LINUX_DEFAULT="... transparent_hugepage=never"

Then regenerate the bootloader configuration, such as update-grub on Debian-based systems or grub2-mkconfig -o /boot/grub2/grub.cfg on RHEL-based systems, depending on your distribution.

If your host uses tuned, check the active profile with tuned-adm active. Some profiles force a THP mode and can override the boot parameter when activated. Re-check /sys/kernel/mm/transparent_hugepage/enabled after reboot and after any tuned profile activation.

Validate with a representative workload

1. Record a baseline before the load:

grep -E 'AnonHugePages|HugePages_' /proc/meminfo

1. Run your staging load under the candidate THP mode.
2. Check /proc/meminfo again. With never, new anonymous allocations should not grow AnonHugePages. With madvise, growth depends on whether the workload advised huge pages.
3. Monitor application latency percentiles and vmstat 1 during the load. Compare madvise and never in staging before changing production.

Do not treat a single /proc/meminfo snapshot as proof; compare before and after under the same load shape.

Limitations

• Runtime changes affect future allocations only; existing anonymous memory is not converted.
• madvise has no effect unless the application or allocator calls madvise(MADV_HUGEPAGE).
• Disabling THP may increase TLB pressure for very large heaps; validate with production-like memory and concurrency.
• Distro tuned profiles or init scripts can override manual settings on boot or profile activation.

Practical verification checklist

• cat /sys/kernel/mm/transparent_hugepage/enabled shows the intended mode bracketed after reboot.
• The same check still shows the intended mode after a tuned profile activation.
• grep -E 'AnonHugePages|HugePages_' /proc/meminfo behaves as expected before and after the workload.
• Bootloader command line contains transparent_hugepage= if persistence is required.
• Staging latency percentiles and stall behavior are acceptable for the chosen mode.