From what I understand, the swap space can be used to swap out inactive processes to free up memory. But can a single process take advantage of swap space? For example, if my system has little memory, but a large harddrive with large swap space, can a process requiring large amounts of memory still run (albeit slowly) by taking advantage of the large disk swap space? Is any action required on my part or will Linux automatically take care of this?
Memory – Can Swap Space Accommodate a Single Process?
memoryswap
Related Solutions
Getting swap back to 0 is not a useful goal.
There is nothing ipso facto wrong about having things in swap. It is quite possible for a program to load resources it doesn't actually use, and for the kernel to notice this and swap them out, freeing the memory for use by programs that can actually make use of it right now. This situation comes up a lot in today's modern bloatware world, where programs depend so much on huge libraries others provide, even though they need only a tiny fraction of the library's full capability.
The only hard numbers you have provided — 200 GB in 30 minutes — also looks quite good to me. That's 114 MByte/sec, which is an impressive copy rate, considering that you're copying a file within a single physical volume. It wasn't that long ago that 100 MByte/sec on purely sequential reads was pretty impressive. You're managing better than that with interleaved reads and writes!
Bottom line, I think you're barking up the wrong tree.
Having such a huge swap nowadays is often a bad idea. By the time the OS swapped just a few GB of memory to swap, your system had already crawled to death (like what you saw)
It's better to use zram
with a small backup swap partition. Many OSes like ChromeOS, Android and various Linux distros (Lubuntu, Fedora) have enabled zram by default for years, especially for systems with less RAM. It's much faster than swap on HDD and you can clearly feel the system responsiveness in this case. Less so on an SSD, but according to the benchmark results here it still seems faster even with the default lzo algorithm. You can change to lz4 for even better performance with a little bit less compression ratio. It's decoding speed is nearly 5 times faster than lzo based on official benchmark
In fact Windows 10 and macOS also use similar pagefile compression techniques by default
There's also zswap
although I've never used it. Probably worth a try and compare which one is better for your usecases
After that another suggestion is to reduce the priority of those IO-bound processes and possibly leave a terminal running on higher priority so that you can run commands on it right away even when the system is on a high load
Further reading
- Arch Linux - Improving performance - Zram or zswap
- Enable ZSwap to increase performance
- Enable zRAM for improved memory handling and less swapping
- Running out of RAM in Ubuntu? Enable ZRAM
- Difference between ZRAM and ZSWAP
- zram vs zswap vs zcache Ultimate guide: when to use which one
- Linux, SSD and swap
- https://wiki.debian.org/ZRam
- https://www.kernel.org/doc/Documentation/blockdev/zram.txt
- https://wiki.gentoo.org/wiki/Zram
Best Answer
You don't have to do anything special, apart from providing swap space and mounting it.
For processors with a MMU (memory management unit, i.e. most processors since the early 90's) the full address space can be used, although there doesn't have to be real memory (RAM) at all address range locations. If some memory location is addressed, RAM will be "attached" using a lookup mechanism. If no free RAM is available, some block is freed up by writing its contents to the swap space.
Of course the processor keeps track of what memory address range was written out, so that when some address in that range is once more needed the right block is "swapped in" (most often after freeing up some other RAM by writing its contents out).
The actual detail for each operating system differ, and the article in wikipedia on paging is a good start if you want more information.
Be forewarned with what you try, competing swapping applications easily makes your system slower by a few orders of magnitude, making it look like your machine is dead.