Command Line Tool for monitoring Java Heap

Try watch. From the manpage:

Name

watch - execute a program periodically, showing output fullscreen

Synopsis

watch [-dhvt] [-n <seconds>] [--differences[=cumulative]] [--help] [--interval=<seconds>] [--no-title] [--version] <command>

Description

watch runs command repeatedly, displaying its output (the first screenfull). This allows you to watch the program output change over time. By default, the program is run every 2 seconds; use -n or --interval to specify a different interval.

The -d or --differences flag will highlight the differences between successive updates. The --cumulative option makes highlighting "sticky", presenting a running display of all positions that have ever changed. [...]

watch will run until interrupted.

Note that "realtime" would have to be approximated by "once a second" (for example) here...

OpenVZ & Memory

The failcnt is going up on privvmpages, so your container is unable to allocate any more virtual memory space from the host:

root@server: ~ # cat /proc/user_beancounters
Version: 2.5
       uid  resource                     held              maxheld              barrier                limit              failcnt
            privvmpages               6005601              6291447              6291456              6291456                 >233<
            physpages                 4635460              6291456              6291456              6291456                    0
            vmguarpages                     0                    0              6291456  9223372036854775807                    0
            oomguarpages              1529376              2144671              6291456  9223372036854775807                    0

Note that virtual memory != physical memory. Processes can allocate up to somewhere around the addressable amount of virtual memory (32bit ~ 2G - 4G, 64bit 8 TB - 256 TB) but that doesn't mean physical memory pages are being used ( a page being a 4KB chunk of memory).

physpages is the number of physical memory pages your container can use.
oomguarpages is the guaranteed memory pages the container will receive when the host is memory constrained.
privvmpages is the number of virtual memory pages your container can use
vmguarpages is the guaranteed amount of virtual memory in the same way

Java

Oracle Java will always allocate one contiguous chunk of virtual memory. Running java with no arguments on a box results in 5M of real memory used (RSS), but 660M of VM space allocated (VSZ):

  PID COMMAND                        VSZ   RSS
20816 java                        667496  4912 

Looking at the memory segments for the java process in it's smaps file shows a chunk of about 500MB allocated, the rest is memory mapped files and normal java stuff.

On a system that's been up for a while the available VM space becomes fragmented as processes use/free parts of it. A grep Vmalloc /proc/meminfo will give you VmallocChunk which is the largest free chunk currently available. If this is low, the system will try and allocate more when java requests it, after all it's virtually unlimited on a 64bit box.

Fix

Tell your host to configure privvmpages and vmguarpages much higher. There's no need for them to be the same as physical memory as that impacts the way linux memory works

You might be able to work around the problem temporarily by dropping your file cache echo 1 > /proc/sys/vm/drop_caches but that's only temporary.

You can limit the chunk of memory java tries to allocate at run time with a minimum Xms or while running with maximum Xmx. Running java with these options on my machine:

java -Xms10M -Xmx10M

reduces the total virtual size to 140MB or so with only a 10MB contiguous chunk for the java heap allocated.