Native Memory is an area which is usually used by the JVM for it’s internal operations and to execute the JNI codes. The JVM Uses Native Memory for Code Optimization and for loading the classes and libraries along with the intermediate code generation.
The Size of the Native Memory depends on the Architecture of the Operating System and the amount of memory which is already commited to the Java Heap. Native memory is an Process Area where the
- JNI codes gets loaded or
- JVM Libraries gets loaded or
- Native Performance packs or
- Proxy Modules gets loaded.
Some times you can see your java process is consuming more memory than you specified in your application (Startup scripts, Xmx) and your Unix or Linux box is running out of memory. Then you restart the process, and the same thing happens again after couple of weeks. If this type of things happened repeatedly, then your system is suffering from a potential memory leak.
You can use below commands to get the native memory consumption by a process.
- pmap command on Solaris & Linux
- svmon command on AIX
The native heap also known as C-Heap is storing objects such as MMAP file, other JVM and third party native code objects.
So if you start debugging why is the process consuming more memory than you allocated? Is it a bug or something else happening? First of all, part of it can definitely be the indication of native code leaking memory. And in most of the cases it is completely normal behaviour of the JVM.
PMAP
The pmap command reports the memory map of a process or processes. You can use below options for pmap.
-x extended Show the extended format.
-d device Show the device format.
$ pmap 8096
8096: /u01/app/oracle/product/fmw/jrockit-jdk1.6.0_37-R28.2.5-4.1.0/bin/java -jrockit -Xms4096m -Xmx4096m -Dweblogic.log.RedirectStdoutToServerLogEnabled=true -server -XX:+UseStringCache -Djrockit.oomdiagnostics=true -Dweblogic.Name=apptest_soa1 -Djava.security.policy=/u01/app/oracle/product/fmw/wlserver_10.3/server/lib/weblogic.polic
0000000000400000 76K r-x-- /u01/app/oracle/product/fmw/jrockit-jdk1.6.0_37-R28.2.5-4.1.0/bin/java
0000000000612000 4K rw--- /u01/app/oracle/product/fmw/jrockit-jdk1.6.0_37-R28.2.5-4.1.0/bin/java
0000000002434000 341820K rw--- [ anon ]
0000000040066000 8K rwx-- [ anon ]
0000000044a06000 12K rw--- [ anon ]
00000000fe180000 4225536K rw--- [ anon ]
0000003992800000 112K r-x-- /lib64/ld-2.5.so
0000003992a1c000 4K r---- /lib64/ld-2.5.so
0000003993400000 520K r-x-- /lib64/libm-2.5.so
0000003993482000 2044K ----- /lib64/libm-2.5.so
0000003993681000 4K r---- /lib64/libm-2.5.so
0000003993682000 4K rw--- /lib64/libm-2.5.so
0000003993c00000 8K r-x-- /lib64/libdl-2.5.so
0000003994000000 88K r-x-- /lib64/libpthread-2.5.so
0000003994a08000 4K rw--- /lib64/librt-2.5.so
0000003996216000 8K rw--- [ anon ]
0000003997800000 68K r-x-- /lib64/libresolv-2.5.so
0000003997811000 2048K ----- /lib64/libresolv-2.5.so
0000003997a11000 4K r---- /lib64/libresolv-2.5.so
0000003997a12000 4K rw--- /lib64/libresolv-2.5.so
0000003997a13000 8K rw--- [ anon ]
000000399b000000 52K r-x-- /lib64/libgcc_s-4.1.2-20080825.so.1
000000399b00d000 2048K ----- /lib64/libgcc_s-4.1.2-20080825.so.1
000000399b20d000 4K rw--- /lib64/libgcc_s-4.1.2-20080825.so.1
00007f50309b4000 64K rw--- [ anon ]
00007f5030a44000 35472K rw--- [ anon ]
00007f5032cf8000 16K r-x-- /lib64/libnss_dns-2.5.so
00007f5032cfc000 2044K ----- /lib64/libnss_dns-2.5.so
.............
.............trimmed
.............
00007fffa0c0a000 80K rwx-- [ stack ]
00007fffa0c1e000 28K rw--- [ anon ]
00007fffa0c72000 4K r-x-- [ anon ]
ffffffffff600000 4K r-x-- [ anon ]
total 6891548K
$pmap -x 8096
8096: /u01/app/oracle/product/fmw/jrockit-jdk1.6.0_37-R28.2.5-4.1.0/bin/java -jrockit -Xms4096m -Xmx4096m -Dweblogic.log.RedirectStdoutToServerLogEnabled=true -server -XX:+UseStringCache -Djrockit.oomdiagnostics=true -Dweblogic.Name=apptest_soa1 -Djava.security.policy=/u01/app/oracle/product/fmw/wlserver_10.3/server/lib/weblogic.polic
Address Kbytes RSS Dirty Mode Mapping
0000000000400000 76 36 0 r-x-- java
0000000000612000 4 4 4 rw--- java
0000000002434000 341820 336172 336172 rw--- [ anon ]
000000399b000000 52 16 0 r-x-- libgcc_s-4.1.2-20080825.so.1
000000399b00d000 2048 0 0 ----- libgcc_s-4.1.2-20080825.so.1
000000399b20d000 4 4 4 rw--- libgcc_s-4.1.2-20080825.so.1
00007f50309b4000 64 64 64 rw--- [ anon ]
00007f5030a44000 35472 35472 35472 rw--- [ anon ]
00007f5032cf8000 16 16 0 r-x-- libnss_dns-2.5.so
00007f5032cfc000 2044 0 0 ----- libnss_dns-2.5.so
00007f5032efb000 4 4 4 r---- libnss_dns-2.5.so
000007f5035059000 1020 0 0 ----- librmi.so
00007f5035158000 4 4 4 rw--- librmi.so
00007f5035159000 3328 3328 3328 rw--- [ anon ]
00007f5035499000 1028 36 0 rw-s- WLS_DIAGNOSTICS000000.DAT
00007f503559a000 34560 34560 34560 rw--- [ anon ]
00007f5037790000 5632 5632 5632 rw--- [ anon ]
..............
..............
..............
00007f5037d10000 396 172 0 r-x-- libMapUpgradeMgr.so
00007f50b8f62000 188 184 184 rw--- libjvm.so
00007f50b8f91000 696 684 684 rw--- [ anon ]
00007f50b904e000 8 8 8 rw--- [ anon ]
00007fffa0c0a000 80 32 32 rwx-- [ stack ]
00007fffa0c1e000 28 28 28 rw--- [ anon ]
00007fffa0c72000 4 4 0 r-x-- [ anon ]
ffffffffff600000 4 0 0 r-x-- [ anon ]
---------------- ------ ------ ------
total kB 6891548 5531992 5515296
$ pmap -d 8096
8096: /u01/app/oracle/product/fmw/jrockit-jdk1.6.0_37-R28.2.5-4.1.0/bin/java -jrockit -Xms4096m -Xmx4096m -Dweblogic.log.RedirectStdoutToServerLogEnabled=true -server -XX:+UseStringCache -Djrockit.oomdiagnostics=true -Dweblogic.Name=apptest_soa1 -Djava.security.policy=/u01/app/oracle/product/fmw/wlserver_10.3/server/lib/weblogic.policy
Address Kbytes Mode Offset Device Mapping
0000000000400000 76 r-x-- 0000000000000000 000:00015 java
0000000000612000 4 rw--- 0000000000012000 000:00015 java
0000000002434000 341820 rw--- 0000000000000000 000:00000 [ anon ]
00007f5035058000 4 r-x-- 0000000000000000 000:00015 librmi.so
00007f5035059000 1020 ----- 0000000000001000 000:00015 librmi.so
00007f5035158000 4 rw--- 0000000000000000 000:00015 librmi.so
00007f5035159000 3328 rw--- 0000000000000000 000:00000 [ anon ]
00007f5035499000 1028 rw-s- 0000000000000000 000:00017 WLS_DIAGNOSTICS000000.DAT
00007f503559a000 34560 rw--- 0000000000000000 000:00000 [ anon ]
00007f5037790000 5632 rw--- 0000000000000000 000:00000 [ anon ]
00007f5037d10000 396 r-x-- 0000000000000000 000:00015 libMapUpgradeMgr.so
00007f5037d73000 1024 ----- 0000000000063000 000:00015 libMapUpgradeMgr.so
00007f5037e73000 56 rw--- 0000000000063000 000:00015 libMapUpgradeMgr.so
00007f5037e81000 1136 r-x-- 0000000000000000 000:00015 libXOM.so
...................
...................
...................
00007fffa0c1e000 28 rw--- 0000000000000000 000:00000 [ anon ]
00007fffa0c72000 4 r-x-- 0000000000000000 000:00000 [ anon ]
ffffffffff600000 4 r-x-- 0000000000000000 000:00000 [ anon ]
mapped: 6891548K writeable/private: 5624952K shared: 1300K
$ pmap -d 8096 | grep mapped
mapped: 6891548K writeable/private: 5624952K shared: 1300K
$ pmap -d 8096 | grep mapped | awk '{print $4}'
5624904K
The above commands gives the exact memory (private memory) usage 5624904K (~5.36GB) a particular Java process even though the max heap size of the java process is 4GB.
If you want to know which process is consuming more native memory, write a script to get list of all processes in the box and get the memory usage of each process by above pmap command.
Gather the memory usage details on hourly basis or daily basis by putting this script in cronjob.
If the memory usage for a particular process is keep on increasing, it will indicate the native memory leak
