Configuring Huge Pages in Red Hat Enterprise Linux 4 or 5

source

Before configuring Big Pages, ensure to have read Section 14.3, “Sizing Big Pages and Huge Pages”.

In Red Hat Enterprise Linux 4 or 5 the size of the Huge Pages pool is specified by the desired number of Huge Pages. To calculate the number of Huge Pages you first need to know the Huge Page size. To obtain the size of Huge Pages, execute the following command:

$ grep Hugepagesize /proc/meminfo
Hugepagesize:     2048 kB
$

The output shows that the size of a Huge Page on this system is 2MB. This means if a 1GB Huge Pages pool should be allocated, then 512 Huge Pages need to be allocated. The number of Huge Pages can be configured and activated by setting nr_hugepages in the proc file system. For example, to allocate 512 Huge Pages, execute:

# echo 512 > /proc/sys/vm/nr_hugepages

Alternatively, you can use sysctl(8) to change it:

# sysctl -w vm.nr_hugepages=512

To make the change permanent, add the following line to the file /etc/sysctl.conf. This file is used during the boot process. The Huge Pages pool is usually guaranteed if requested at boot time:

# echo "vm.nr_hugepages=512" >> /etc/sysctl.conf

If you allocate a large number of Huge Pages, the execution of the above commands can take a while. To verify whether the kernel was able to allocate the requested number of Huge Pages, run:

$ grep HugePages_Total /proc/meminfo
HugePages_Total:   512
$

The output shows that 512 Huge Pages have been allocated. Since the size of Huge Pages is 2048 KB, a Huge Page pool of 1GB has been allocated and pinned in physical memory.

If HugePages_Total is lower than what was requested with nr_hugepages, then the system does either not have enough memory or there are not enough physically contiguous free pages. In the latter case the system needs to be rebooted which should give you a better chance of getting the memory.

To get the number of free Huge Pages on the system, execute:

$ grep HugePages_Free /proc/meminfo

Free system memory will automatically be decreased by the size of the Huge Pages pool allocation regardless whether the pool is being used by an application like Oracle DB or not:

$ grep MemFree /proc/meminfo

Note

In order that an Oracle database can use Huge Pages in Red Hat Enterprise Linux 4 or 5, you also need to increase the ulimit parameter "memlock" for the oracle user in /etc/security/limits.conf if "max locked memory" is not unlimited or too small, see ulimit -a or ulimit -l. An example can be seen below.

oracle           soft    memlock         1048576
oracle           hard    memlock         1048576

The memlock parameter specifies how much memory the oracle user can lock into its address space. Note that Huge Pages are locked in physical memory. The memlock setting is specified in KB and must match the memory size of the number of Huge Pages that Oracle should be able to allocate. So if the Oracle database should be able to use 512 Huge Pages, then memlock must be set to at least 512 * Hugepagesize, which on this system would be 1048576 KB (512*1024*2). If memlock is too small, then no single Huge Page will be allocated when the Oracle database starts. For more information on setting shell limits, see Chapter 22, Setting Shell Limits for Your Oracle User.

Log in as the oracle user again and verify the new memlock setting by executing ulimit -l before starting the database.

After an Oracle DB startup you can verify the usage of Huge Pages by checking whether the number of free Huge Pages has decreased:

$ grep HugePages_Free /proc/meminfo

To free the Huge Pages pool, you can execute:

# echo 0 > /proc/sys/vm/nr_hugepages

This command usually takes a while to finish.