VirtualBox is using far too much RAM for your system, and you are swapping constantly.
I retouched the image to make it visible:
VirtualBox have a virtual memory requirement that are almost the same size of your physical RAM.
How much memory have you dedicated to the virtual machine? In my experience, everything that goes more than 1/3 of your total physical RAM is asking for troubles and slowdowns.
The java process under it could be also part of the problem (it's at 2.8G!) --- chances are that VirtualBox and that process are competing for RAM and livelock together swapping like crazy.
More detailed: that the Virtual Size is the memory the process has "mapped", that is, the memory it thinks it has --- only part of that is really in RAM at any moment, and that part is the one marked with RES (for resident). If the program tries to access memory which is in its virtual pool but not in RAM, and the memory is full, some other memory must be swapped out to make place for the new one.
So if you have two process with a big difference between Virtual and Resident memory, chances are that the two are swapping out the other one at every scheduling switch --- making the system slow to a crawl --- swap memory can be millions of time slower than RAM.
The fact that it happens after a long period of inactivity is probably due the fact that the system has settled on a partition of the RAM among the process that more or less satisfy all of them for that workload. When you change the kind of workload again, different part of the programs are loaded, and the swap frenzy starts. Chances are that if you wait the necessary time (could be hours!) the system will more or less find another decent equilibrium; it's the transitory which is bad.
About your case:
With 4G of RAM, having a virtual machine of 1.5G is ok, if it's the only memory hungry process in the system. Probably that one and the IDE are too much.
Dirty (and dangerous) Trick
To have back a responsive system, the faster way I know is to force swap off; but this is quite dangerous sometime. The basic idea is to do a
sudo swapoff -a
wait for finishing...
sudo swapon -a
This will try to put all the memory back in RAM. If it can't, processes will be unmercifully killed along the way. To avoid the death of some innocent and important system process(1) the best way is to try to close the memory-hog before doing the swapoff.
(1) the OOM (out-of-memory) killer of the kernel will try to kill the correct processes, but it's not failproof.
Best Answer
If you don't need compositing/desktop effects, then Xubuntu would be the better option.
It has a nice tool in its Settings menu to turn them off, and quite frankly you won't lose anything by installing Xubuntu if you're only going to use it for web/java development. You'll be using the same apps as you would do in Ubuntu.