I am currently having some issues running Java. It won't start because of heap issues. But I have more than 9 GB Ram free (or even 16 GB if you assumed the cache would be empty). This is the error I get (and the free command)
root@server: ~ # java
Error occurred during initialization of VM
Could not reserve enough space for object heap
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
root@server: ~ # free
total used free shared buffers cached
Mem: 25165824 15941148 9224676 0 0 7082176
-/+ buffers/cache: 8858972 16306852
Swap: 0 0 0
I am running a 64 bit Debian on a virtualized server. The virtualization software is OpenVZ. This is my Java version (I can execute this command after I stop two of my VM's (4 currently running)):
root@server: ~ # java -version
java version "1.7.0_45"
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
What could I do?
As requested:
root@server: ~ # cat /proc/meminfo
MemTotal: 25165824 kB
MemFree: 11723412 kB
Cached: 4597552 kB
Active: 9692308 kB
Inactive: 3322544 kB
Active(anon): 7411960 kB
Inactive(anon): 1005340 kB
Active(file): 2280348 kB
Inactive(file): 2317204 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 960 kB
Writeback: 0 kB
AnonPages: 8417300 kB
Shmem: 21504 kB
Slab: 427452 kB
SReclaimable: 383424 kB
SUnreclaim: 44028 kB
Reguest2:
root@server: ~ # cat /proc/user_beancounters
Version: 2.5
uid resource held maxheld barrier limit failcnt
10023468: kmemsize 399250512 506245120 5053325720 5558658292 0
lockedpages 0 8 246744 246744 0
privvmpages 6005602 6291447 6291456 6291456 221
shmpages 8576 8608 579124 579124289562 0
dummy 0 0 9223372036854775807 9223372036854775807 0
numproc 598 1236 30000 30000 0
physpages 4634494 6291456 6291456 6291456 0
vmguarpages 0 0 6291456 9223372036854775807 0
oomguarpages 1529371 2144671 6291456 9223372036854775807 0
numtcpsock 62 164 30000 30000 0
numflock 25 39 1000 1100 0
numpty 13 24 512 512 0
numsiginfo 10 75 1024 1024 0
tcpsndbuf 3330352 4153232 1179110194 1684441906 0
tcprcvbuf 1216896 34410032 1179110194 1684441906 0
othersockbuf 270504 537552 589555096 1094886808 0
dgramrcvbuf 0 67048 589555096 589555096 0
numothersock 287 333 30000 30000 0
dcachesize 355559855 446054103 1103879952 1136996352 0
numfile 4766 7745 250000 250000 0
dummy 0 0 9223372036854775807 9223372036854775807 0
dummy 0 0 9223372036854775807 9223372036854775807 0
dummy 0 0 9223372036854775807 9223372036854775807 0
numiptent 14 14 1000 1000 0
root@server: ~ # java
Error occurred during initialization of VM
Could not reserve enough space for object heap
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
root@server: ~ # cat /proc/user_beancounters
Version: 2.5
uid resource held maxheld barrier limit failcnt
10023468: kmemsize 399246622 506245120 5053325720 5558658292 0
lockedpages 0 8 246744 246744 0
privvmpages 6005601 6291447 6291456 6291456 233
shmpages 8576 8608 579124 579124289562 0
dummy 0 0 9223372036854775807 9223372036854775807 0
numproc 598 1236 30000 30000 0
physpages 4635460 6291456 6291456 6291456 0
vmguarpages 0 0 6291456 9223372036854775807 0
oomguarpages 1529376 2144671 6291456 9223372036854775807 0
numtcpsock 64 164 30000 30000 0
numflock 25 39 1000 1100 0
numpty 13 24 512 512 0
numsiginfo 10 75 1024 1024 0
tcpsndbuf 3365232 4153232 1179110194 1684441906 0
tcprcvbuf 1249664 34410032 1179110194 1684441906 0
othersockbuf 270504 537552 589555096 1094886808 0
dgramrcvbuf 0 67048 589555096 589555096 0
numothersock 287 333 30000 30000 0
dcachesize 355559855 446054103 1103879952 1136996352 0
numfile 4768 7745 250000 250000 0
dummy 0 0 9223372036854775807 9223372036854775807 0
dummy 0 0 9223372036854775807 9223372036854775807 0
dummy 0 0 9223372036854775807 9223372036854775807 0
numiptent 14 14 1000 1000 0
Best Answer
OpenVZ & Memory
The
failcntis going up onprivvmpages, so your container is unable to allocate any more virtual memory space from the host:Note that virtual memory != physical memory. Processes can allocate up to somewhere around the addressable amount of virtual memory (32bit ~ 2G - 4G, 64bit 8 TB - 256 TB) but that doesn't mean physical memory pages are being used ( a page being a 4KB chunk of memory).
physpagesis the number of physical memory pages your container can use.oomguarpagesis the guaranteed memory pages the container will receive when the host is memory constrained.privvmpagesis the number of virtual memory pages your container can usevmguarpagesis the guaranteed amount of virtual memory in the same wayJava
Oracle Java will always allocate one contiguous chunk of virtual memory. Running
javawith no arguments on a box results in 5M of real memory used (RSS), but 660M of VM space allocated (VSZ):Looking at the memory segments for the
javaprocess in it'ssmapsfile shows a chunk of about 500MB allocated, the rest is memory mapped files and normal java stuff.On a system that's been up for a while the available VM space becomes fragmented as processes use/free parts of it. A
grep Vmalloc /proc/meminfowill give youVmallocChunkwhich is the largest free chunk currently available. If this is low, the system will try and allocate more whenjavarequests it, after all it's virtually unlimited on a 64bit box.Fix
Tell your host to configure
privvmpagesandvmguarpagesmuch higher. There's no need for them to be the same as physical memory as that impacts the way linux memory worksYou might be able to work around the problem temporarily by dropping your file cache
echo 1 > /proc/sys/vm/drop_cachesbut that's only temporary.You can limit the chunk of memory
javatries to allocate at run time with a minimumXmsor while running with maximumXmx. Runningjavawith these options on my machine:reduces the total virtual size to 140MB or so with only a 10MB contiguous chunk for the java heap allocated.