Rsync Memory Usage – Optimize Performance

14.04memory usagersyncxubuntu

I am running on Xubuntu 14.04 and experiencing high memory usage of rsync.

With a fresh start, the system is OK.

I have rsync running in a cronjob, after/during rsync memory usage increases and stays high (seems like rsync is not freeing the memory).

Any hints/suggestions/etc???

rsync command:

sudo rsync -auAXv --delete --ignore-errors /* /media/edisz/backup_hdd/ --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found}

Here's the htop output (after rsync):

screenshot of terminal showing htop

And free -m shows:

             total       used       free     shared    buffers     cached
Mem:          3816       3386        429         19        230       1000
-/+ buffers/cache:       2155       1660
Swap:         3954          0       3954

OK, firefox is eating ~400MB, but then there are still 1700MB left…

And before rsync:

screenshot of terminal showing htop output before rsync

             total       used       free     shared    buffers     cached
Mem:          3816       1497       2318         39         71        731
-/+ buffers/cache:        694       3121
Swap:         3954          0       3954

Best Answer

Dropping caches after rsync seem to fix the problem:

sudo sh -c "sync; echo 3 > /proc/sys/vm/drop_caches"

Related Solutions

Ubuntu – Reduce memory usage

You can easily install and try out different desktop environments (instead of the default, Unity). Once installed, you can switch between different desktop environments at the log-in screen (there is an icon near your name, which, when clicked, allows you to select). enter image description here

Probably in your case, with 1GB ram, I would suggest the lubuntu desktop (based on LXDE). To install, open a terminal and type

sudo apt-get install lubuntu-desktop

If you don't want the whole environment, you can install LXDE by itself

sudo apt-get install lxde

Most everything will be the same but you'll have a new look that is less memory intensive (see many screenshots HERE). Firefox will be there, and the terminal will look a little different.

If you don't like the LXDE look, you can read about other desktop environments HERE. Options include Xfce (Xubuntu) and KDE (Kubuntu).

Ubuntu – How to read memory usage

Type free -h in console. You will get something like this: total used free shared buffers cached Mem: 15G 9,1G 6,1G 270M 524K 7,6G -/+ buffers/cache: 1,5G 13G Swap: 0B 0B 0B

The first line says that out of 15G memory installed (marketologist's 16) I have 9 used for apps and 7,6 for filesystem cache. The third line shows that I have swap disabled.

Best Answer

Related Solutions

Ubuntu – Reduce memory usage

Ubuntu – How to read memory usage

Related Question