libhugetlbfs

The hugeadm command is used to display and configure the system's huge page pools. The pool size is defined by a minimum threshold (guaranteed to be allocated by the kernel) and an optional maximum threshold (allowing dynamic allocation if requested by applications).

hugeadm [options]

The pagesize utility is used to determine the memory page sizes supported by the system. This information is useful for configuring huge page pools (e.g., with hugeadm), launching applications with hugectl, or writing portable shell scripts.

By default, running pagesize without arguments returns the system's base page size (the value returned by getpagesize(3)).

The hugeedit command allows you to modify the ELF header of a binary to set a default policy for backing its text or data segments with hugepages.

By setting these bits in the ELF header, you eliminate the need to use hugectl or set environment variables every time the binary is executed.

If you run hugeedit without any arguments, it will display the current default settings for each segment in the specified binary.

# View current defaults for a binary
hugeedit my_binary

# Set the binary to use hugepages for text segments by default
hugeedit --text my_binary

# Set the binary to use hugepages for data segments by default
hugeedit --data my_binary

# Disable hugepage backing for all segments (use small pages by default)
hugeedit --disable my_binary

The ld.hugetlbfs command replaces the standard ld linker to enable hugepage support during the linking process.

To use this with gcc, you must pass the -B flag to specify the non-standard location where the linker resides. This is typically /usr/share/libhugetlbfs. You can automate this by setting the CFLAGS environment variable.

Note: Using the -Wl,--hugetlbfs-align option provides greater runtime flexibility via the HUGETLB_ELFMAP environment variable, allowing you to control which segments are placed in hugepages at execution time.

# Using gcc with ld.hugetlbfs
gcc -B /usr/share/libhugetlbfs my_program.c -o my_program

# Or via CFLAGS
export CFLAGS="-B /usr/share/libhugetlbfs"
gcc $CFLAGS my_program.c -o my_program

The hugectl utility allows you to run a command with a specific policy for backing memory regions (text, data, BSS, heap, or shared memory) with hugepages. This is used to improve performance for applications with large address spaces by reducing TLB misses.

Note on Text/Data/BSS: To effectively back these segments, the target process must be relinked to align the ELF segments on a hugepage boundary.

Note on Heap: The --heap option uses the glibc morecore hook. It does not affect brk() segments, and applications using custom allocators may not benefit from this option.

hugectl [options] command {arguments}

You can use libhugetlbfs to back an application's text, data, malloc(), or shared memory with hugepages without modifying the application's source code. This is achieved by preloading the library using the LD_PRELOAD environment variable.

Note: For effective backing of text and data segments, the application should ideally be relinked using ld.hugetlbfs helpers to ensure ELF segments are correctly aligned. Without relinking, only malloc() and shared memory are guaranteed to be backed, and text/data segments may only be partially backed if they are large enough.

export LD_PRELOAD=libhugetlbfs.so
[environment options] target_application

To diagnose problems or errors encountered while using the library, increase the diagnostic output by setting the following environment variables:

• HUGETLB_VERBOSE=99: Increases the verbosity of the library's output.
• HUGETLB_DEBUG=yes: Enables extra internal checking to help identify the root cause of issues.

If problems persist, collect the debug output along with strace logs, /proc/pid/maps, and gdb backtraces before seeking community support.

export HUGETLB_VERBOSE=99
export HUGETLB_DEBUG=yes
./your_application

If using libhugetlbfs results in slower application performance, consider the following steps:

1. Test different configurations: Experiment with different combinations of hugetlb malloc and the two supported link types to identify the optimal setup for your specific workload.
2. Evaluate suitability: Note that huge pages are a niche performance optimization. They are not universally beneficial and can specifically degrade performance in certain application types.
3. Verify page allocation: Ensure you have sufficient huge pages allocated in the system. Even if you believe you have enough, try increasing the allocation to a significantly higher number to rule out exhaustion.

Use get_hugepage_region() to allocate a memory region of size len bytes that is backed by hugepages. This is useful for applications with large address spaces to reduce TLB misses.

Unlike get_huge_pages(), the len parameter does not need to be hugepage-aligned, though alignment requirements may result in some wasted memory.

On success, the function returns a pointer to the allocated memory. On error, it returns NULL and sets errno based on the underlying mmap() failure.

#include <hugetlbfs.h>

void *get_hugepage_region(size_t len, ghr_t flags);