transparent huge page

RAC - Disable Transparent HugePages on SLES11, RHEL6 and OEL6 [ID 1557478.1]

Applies to:

Oracle Database - Enterprise Edition
Linux x86
Linux x86-64

Description

Starting RedHat 6, OEL 6 and SLES 11 Linux, Transparent HugePages are implemented and enabled (default) in an attempt to improve the memory management.  Transparent HugePages are similar to the HugePages that have been available in previous Linux releases.  The main difference is that the Transparent HugePages are set up dynamically at run time by the khugepaged thread in kernel while the regular HugePages had to be preallocated at the boot up time.

Because Transparent HugePages are known to cause unexpected node reboots and performance problems with RAC, Oracle strongly advises to disable the use of Transparent HugePages.

Note:  This ONLY applies to the new feature Transparent HugePages, Oracle highly recommends the use of standard HugePages that were recommended for previous releases of Linux.  See Document 361323.1 for additional information on HugePages.

Occurrence

 Starting SLES11, Redhat 6, and OEL 6, the transparent huge page is introduced and enabled by default, and this can cause node reboots and performance problem.

Symptoms

Node reboots.

The ocssd.log may show some of the threads are blocked (but this does not show all the time):

2013-05-01 14:30:45.255: [    CSSD][224204544]clssscMonitorThreads clssnmvKillBlockThread not scheduled for 7500 msecs
2013-05-01 14:30:46.945: [    CSSD][224204544]clssscMonitorThreads clssnmvWorkerThread not scheduled for 8030 msecs

Workaround

To check if the Transparent HugePages are enabled in your server execute the following:

# cat /sys/kernel/mm/transparent_hugepage/enabled

If "enabled" is NOT set to "never", the Transparent HugePages are being used.

You can also issue:

# grep AnonHugePages /proc/meminfo

If the output contains a line like "AnonHugepages: xxxx kB", the kernel is using Transparent HugePages.

Because the kernel currently uses Transparent HugePages only for the anonymous memory blocks like stack and heap, the value of AnonHugepages in /proc/meminfo is the current amount of Transparent HugePages that the kernel is using.


To disable Transparent HugePages boot time, add the following lines in /etc/rc.local and reboot the server:

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

Patches

none