Hugepages are Not used by Database Buffer Cache if you have use_indirect_data_buffers=true

Hugepages are Not used by Database Buffer Cache [ID 829850.1]

Applies to:

Oracle Server - Enterprise Edition - Version 11.1.0.7 and later
Linux x86
***Checked for relevance on 27-Apr-2012***

Symptoms

Following the steps in Note 748637.1 you have configured your Linux for Very Large Memory with HugePages.
As described on Note 749851.1 you have MEMORY_TARGET & MEMORY_MAX_TARGET initialization parameters disabled as the use of 11g Automatic Memory Management (AMM) is incompatible with HugePages.
Your SGA size is less then the size of the HugePages_Total (HugePages_Total*Hugepagesize)
But despite this the HugePages are not used by the database buffer cache.


Here I would like to show you an example of a possible scenario:

Suppose to have a RDBMS instance ASM enabled:

ASM
*.shared_pool_size 67108864 --> 64MB
*.db_cache_size 25165824 --> 24MB
*.log_buffer 4194304 --> 4MB

Checking on /proc/meminfo you can verify how the HugePages are configured on your system before and after.

- Before ASM is running:

$ grep -i huge /proc/meminfo
HugePages_Total: 5120
HugePages_Free: 5120
HugePages_Rsvd: 0
Hugepagesize: 2048 kB

where:
HugePages_Total is the size of the pool of hugepages.
HugePages_Free is the number of hugepages in the pool that are not yet allocated.
HugePages_Rsvd is short for "reserved," and is the number of hugepages
for which a commitment to allocate from the pool has been made, but no

- After ASM is running:



$ grep -i huge /proc/meminfo

HugePages_Total:  5120

HugePages_Free:   5078

HugePages_Rsvd:     11

Hugepagesize:     2048 kB



5120 - 5078 = 42 * 2MB = 84MB

             11 * 2MB = 22MB

             ---------------

                        106MB

RDBMS instance:

DB
*.DB_BLOCK_BUFFERS 393216
*.db_block_size 8192 --> 3GB
*.use_indirect_data_buffers=true
*.shared_pool_size 318767104 --> 304MB
*.log_buffer 1048576 -->  10MB



$ grep -i huge /proc/meminfo
HugePages_Total: 5120
HugePages_Free: 4970
HugePages_Rsvd: 80
Hugepagesize: 2048 kB

--> 5078 - 4970 = 108 * 2MB = 216MB
                   80 * 2MB = 160MB
                   ----------------
                              376MB

Cause

This is an expected behavior and it is not a product defect.

Using VLM (used by the Database Buffer cache, which is the only way), you cannot use HugePages (for the Database Buffer cache). HugePages can be used for other parts of SGA like shared_pool, large_pool etc. only

That is because
- memory allocation for VLM is done by creation pseudo-files under ramfs/tmpfs
- HugePages does not get allocated using the memory (pseudo) file systems

Solution

Using USE_INDIRECT_DATA_BUFFERS=TRUE you will use HugePages just for Shared_pool, Large_pool etc. but not for the Database Buffer cache.

If you do not use that (and even use DB_BLOCK_BUFFERS vs DB_CACHE_SIZE) you can have HugePages also for the Database Buffer cache. But of course in this case you are not using VLM.