Top showing virtual memory usage of hundreds of gigabytes

htopmemorytop

On a new virtual machine, top shows that a process of mine consumes 100 GB of virtual memory. I've read that it's possible because of overcommit, however, it's a bit too much. I've never seen anything like this, especially in a VM having just 6 GB real memory, 2 GB swap and a single 40 GB HD partition.

unreal_memory

The funny process with 101 GB VIRT is Eclipse IDE (started a few minutes ago) and there's another one, WebKitWebProcess with 98 GB VIRT. All other processes have sane VIRT values of about 1-2 GB. All three tools seem to agree on that: top, htop and ps.

  • Is there any good reason for the processes to request that much virtual memory?
  • Isn't such behavior detrimental to the system?
  • What are the current limits on the total virtual memory?

Update

Details:
Linux 4.15.0-38-generic x86_64 GNU/Linux, Linux Mint 19 Tara
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
Version: 2018-09 (4.9.0) Build id: 20180917-1800

The content of /proc//maps starts with

00400000-00401000 r-xp 00000000 08:01 132092                             /usr/lib/jvm/java-8-oracle/jre/bin/java
00600000-00601000 r--p 00000000 08:01 132092                             /usr/lib/jvm/java-8-oracle/jre/bin/java
00601000-00602000 rw-p 00001000 08:01 132092                             /usr/lib/jvm/java-8-oracle/jre/bin/java
014fc000-02ebe000 rw-p 00000000 00:00 0                                  [heap]
c0000000-d0000000 rw-p 00000000 00:00 0 
d0000000-100000000 ---p 00000000 00:00 0 
100000000-1012c6000 rw-p 00000000 00:00 0 
1012c6000-140000000 ---p 00000000 00:00 0 
7ef800000000-7ef800004000 rw-p 00000000 00:00 0 
7ef800004000-7ef8000ec000 rw-p 00000000 00:00 0 
7ef8000ec000-7ef800100000 rw-p 00000000 00:00 0 
7ef800100000-7f0800000000 rw-p 00000000 00:00 0 
7f0800000000-7f1000000000 ---p 00000000 00:00 0

The last two shown lines line seem to be the culprit as 0x7f0800000000 - 0x7ef800100000 ≈ 68.7e9. From what I've found, it's an unmapped private region with no permissions. That's all I can say….

Best Answer

Most likely it is mmap'ing every file it can index. I've seen similar with Kafka mapping thousands of files but usually the virtual footprint is much higher (~10x) times). You can run:

cat /proc/{PID}/maps

(where {PID} is your Eclipse process ID) to see if this is the case. If it is indeed the case, you might have to raise vm.max_map_count in sysctl if your virtual footprint is too large.

Related Question