Monthly Archives: September 2007

  • Linux hugepages

    Hugepages is a mechanism that allows the Linux kernel to utilize the multiple page size capabilities of modern hardware architectures. Linux uses pages as the basic unit of memory, where physical memory is partitioned and accessed using the basic page unit. The default page size is 4096 Bytes in the x86 architecture. Hugepages allows large amounts of memory to be utilized with a reduced overhead. Linux uses “Translation Lookaside Buffers” (TLB) in the CPU architecture. These buffers contain mappings of virtual memory to actual physical memory addresses. So utilizing a huge amount of physical memory with the default page size consumes the TLB and adds processing overhead. 

     The Linux kernel is able to set aside a portion of physical memory to be able be addressed using a larger page size. Since the page size is higher, there will be less overhead managing the pages with the TLB. In the Linux 2.6 series of kernels, hugepages is enabled using the CONFIG_HUGETLB_PAGE feature when the kernel is built. Systems with large amount of memory can be configured to utilize the memory more efficiently by setting aside a portion dedicated for hugepages. The actual size of the page is dependent on the system architecture.

    A typical x86 system will have a Huge Page Size of 2048 kBytes. The huge page size may be found by looking at the /proc/meminfo :

    # cat /proc/meminfo |grep Hugepagesize Hugepagesize: 2048 kB

    Number of Hugepages can be allocated using the /proc/sys/vm/nr_hugepages entry, or by using the sysctl command.

    To view the current setting using the /proc entry:
    # cat /proc/sys/vm/nr_hugepages 0

    To view the current setting using the sysctl command:
    # sysctl vm.nr_hugepages vm.nr_hugepages = 0

    To set the number of huge pages using /proc entry:
    # echo 5 > /proc/sys/vm/nr_hugepages

    To set the number of hugepages using sysctl:
    # sysctl -w vm.nr_hugepages=5 vm.nr_hugepages = 5

    It may be necessary to reboot to be able to allocate the number of hugepages that is needed. This is because hugepages requires large areas of contiguous physical memory. Over time, physical memory may be mapped and allocated to pages, thus the physical memory can become fragmented. If the hugepages are allocated early in the boot process, fragmentation is unlikely to have occurred.

    It is recommended that the /etc/sysctl.conf file should be used to allocate hugepages at boot time. For example, to allocate 5 hugepages at boot time add the line below to the sysctl.conf file :

    vm.nr_hugepages = 5

    Some more useful Linux kernel parameters: