I have a Debian Sid box, freshly installed about a month ago, with no desktop apps running at the moment.
Any command I try to run (via ssh) makes the hard disk work very hard. The system worked flawlessly a few weeks ago while running Ubuntu 8.04. Hardware should be ok, smart tools report no problem, nor do memory checks, etc.
I suspect the problem is related to swapping:
$ free -h
total used free shared buffers cached
Mem: 998M 986M 12M 0B 464K 904M
-/+ buffers/cache: 81M 917M
Swap: 511M 511M 0B
Testing stuff, I see that I'm unable to disable swap without rebooting, even though most physical ram is used for caches:
# swapoff -a
swapoff: /swapfile: swapoff failed: Cannot allocate memory
When running the script mentioned in this stackoverflow answer, the highest swap-using process was using barely 1.5megs, and the total count was:
"Overall swap used: 18376 kB"
…which is far from the half gig that htop or free report.
I tried changing the swappiness to 1 half a month ago, but there's no difference (same memory usage patterns).
Any idea what could be the culprit?
Edit: here's the /proc/meminfo contents (similar situation, but 702megs of cache instead of 904, and chromium still open):
MemTotal: 1022464 kB
MemFree: 14344 kB
Buffers: 192 kB
Cached: 719816 kB
SwapCached: 2212 kB
Active: 304304 kB
Inactive: 657344 kB
Active(anon): 294128 kB
Inactive(anon): 651264 kB
Active(file): 10176 kB
Inactive(file): 6080 kB
Unevictable: 0 kB
Mlocked: 0 kB
HighTotal: 127336 kB
HighFree: 316 kB
LowTotal: 895128 kB
LowFree: 14028 kB
SwapTotal: 524284 kB
SwapFree: 0 kB
Dirty: 4 kB
Writeback: 0 kB
AnonPages: 239528 kB
Mapped: 11968 kB
Shmem: 703760 kB
Slab: 28924 kB
SReclaimable: 12116 kB
SUnreclaim: 16808 kB
KernelStack: 2200 kB
PageTables: 5812 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 1035516 kB
Committed_AS: 2193368 kB
VmallocTotal: 122880 kB
VmallocUsed: 14092 kB
VmallocChunk: 105548 kB
HardwareCorrupted: 0 kB
AnonHugePages: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 911352 kB
DirectMap2M: 0 kB
And here's the top -b -n 1 output:
top - 22:00:18 up 1 day, 2:34, 4 users, load average: 1,97, 1,25, 0,88
Tasks: 137 total, 3 running, 134 sleeping, 0 stopped, 0 zombie
%Cpu(s): 4,9 us, 3,0 sy, 2,7 ni, 83,2 id, 6,1 wa, 0,0 hi, 0,0 si, 0,0 st
KiB Mem: 1022464 total, 1008788 used, 13676 free, 188 buffers
KiB Swap: 524284 total, 524284 used, 0 free, 719792 cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
7861 stenyak 20 0 6336 1252 976 R 18,0 0,1 0:00.03 top
5194 isabel 30 10 44372 5344 1404 R 12,0 0,5 0:36.11 glmatrix
17940 root 20 0 103m 6844 2828 S 6,0 0,7 11:02.21 Xorg
1 root 20 0 2280 28 0 S 0,0 0,0 0:02.49 init
2 root 20 0 0 0 0 S 0,0 0,0 0:00.00 kthreadd
3 root 20 0 0 0 0 S 0,0 0,0 0:15.25 ksoftirqd/0
5 root 20 0 0 0 0 S 0,0 0,0 0:00.00 kworker/u:0
6 root rt 0 0 0 0 S 0,0 0,0 0:00.93 migration/0
7 root rt 0 0 0 0 S 0,0 0,0 0:00.46 watchdog/0
8 root rt 0 0 0 0 S 0,0 0,0 0:00.73 migration/1
10 root 20 0 0 0 0 S 0,0 0,0 0:17.30 ksoftirqd/1
12 root rt 0 0 0 0 S 0,0 0,0 0:00.43 watchdog/1
13 root 0 -20 0 0 0 S 0,0 0,0 0:00.00 cpuset
14 root 0 -20 0 0 0 S 0,0 0,0 0:00.00 khelper
15 root 20 0 0 0 0 S 0,0 0,0 0:00.00 kdevtmpfs
16 root 0 -20 0 0 0 S 0,0 0,0 0:00.00 netns
17 root 20 0 0 0 0 S 0,0 0,0 0:00.44 sync_supers
18 root 20 0 0 0 0 S 0,0 0,0 0:00.00 bdi-default
19 root 0 -20 0 0 0 S 0,0 0,0 0:00.00 kintegrityd
20 root 0 -20 0 0 0 S 0,0 0,0 0:00.00 kblockd
22 root 20 0 0 0 0 S 0,0 0,0 0:00.07 khungtaskd
23 root 20 0 0 0 0 D 0,0 0,0 2:50.47 kswapd0
24 root 25 5 0 0 0 S 0,0 0,0 0:00.00 ksmd
25 root 20 0 0 0 0 S 0,0 0,0 0:00.00 fsnotify_mark
26 root 0 -20 0 0 0 S 0,0 0,0 0:00.00 crypto
99 root 20 0 0 0 0 S 0,0 0,0 0:00.00 khubd
115 root 0 -20 0 0 0 S 0,0 0,0 0:00.00 ata_sff
116 root 20 0 0 0 0 S 0,0 0,0 0:00.00 scsi_eh_0
117 root 20 0 0 0 0 S 0,0 0,0 0:00.00 scsi_eh_1
118 root 20 0 0 0 0 S 0,0 0,0 0:01.66 kworker/u:1
152 root 20 0 0 0 0 S 0,0 0,0 0:02.12 jbd2/sda1-8
153 root 0 -20 0 0 0 S 0,0 0,0 0:00.00 ext4-dio-unwrit
280 root 20 0 2944 352 128 S 0,0 0,0 0:00.30 udevd
423 root 0 -20 0 0 0 S 0,0 0,0 0:00.00 hd-audio0
1286 root 20 0 0 0 0 S 0,0 0,0 0:07.06 jbd2/sda2-8
1287 root 0 -20 0 0 0 S 0,0 0,0 0:00.00 ext4-dio-unwrit
1561 root 20 0 0 0 0 S 0,0 0,0 0:03.09 flush-8:0
1603 root 20 0 2376 48 0 S 0,0 0,0 0:00.33 rpcbind
1634 statd 20 0 2648 68 64 S 0,0 0,0 0:00.00 rpc.statd
1639 root 0 -20 0 0 0 S 0,0 0,0 0:00.00 rpciod
1641 root 0 -20 0 0 0 S 0,0 0,0 0:00.00 nfsiod
1648 root 20 0 2576 0 0 S 0,0 0,0 0:00.00 rpc.idmapd
2019 root 20 0 29088 976 0 S 0,0 0,1 0:22.26 rsyslogd
2123 root 20 0 1876 84 24 S 0,0 0,0 0:00.23 acpid
2161 message+ 20 0 3756 728 0 S 0,0 0,1 0:02.06 dbus-daemon
2213 daemon 20 0 2152 20 0 S 0,0 0,0 0:00.00 atd
2231 root 20 0 4488 220 128 S 0,0 0,0 0:00.00 bluetoothd
2242 root 10 -10 0 0 0 S 0,0 0,0 0:00.00 krfcommd
2259 avahi 20 0 3272 292 168 S 0,0 0,0 0:00.66 avahi-daemon
2262 avahi 20 0 3156 24 0 S 0,0 0,0 0:00.00 avahi-daemon
2265 root 20 0 5968 180 152 S 0,0 0,0 0:00.00 bitlbee
2271 root 20 0 0 0 0 S 0,0 0,0 0:11.69 kworker/0:2
2304 root 20 0 19056 700 220 S 0,0 0,1 0:00.75 NetworkManager
2334 root 20 0 5536 176 108 S 0,0 0,0 0:02.86 cron
2350 root 20 0 6700 492 152 S 0,0 0,0 0:00.01 sudo
2351 root 20 0 15304 5880 1428 R 0,0 0,6 61:42.79 iotop
2359 root 20 0 24256 600 56 S 0,0 0,1 0:01.12 polkitd
2390 root 20 0 6892 368 180 S 0,0 0,0 0:00.15 modem-manager
2392 root 20 0 7536 80 0 S 0,0 0,0 0:00.13 cupsd
2396 root 20 0 20964 516 296 S 0,0 0,1 0:00.15 gdm3
2712 root 20 0 10068 196 84 S 0,0 0,0 0:11.72 kerneloops
2714 root 20 0 6896 2108 372 S 0,0 0,2 0:05.69 ddclient - slee
2802 Debian-+ 20 0 7424 100 40 S 0,0 0,0 0:00.05 exim4
2877 daemon 20 0 1868 236 172 S 0,0 0,0 0:01.22 uptimed
2881 root 20 0 6464 208 104 S 0,0 0,0 0:00.27 sshd
2913 root 20 0 1848 72 8 S 0,0 0,0 0:00.00 minissdpd
2936 root 20 0 4328 40 36 S 0,0 0,0 0:00.00 getty
2937 root 20 0 4328 40 36 S 0,0 0,0 0:00.00 getty
2938 root 20 0 4328 40 36 S 0,0 0,0 0:00.00 getty
2939 root 20 0 4328 40 36 S 0,0 0,0 0:00.00 getty
2940 root 20 0 4328 40 36 S 0,0 0,0 0:00.00 getty
2941 root 20 0 4328 40 36 S 0,0 0,0 0:00.00 getty
2961 root 20 0 23384 540 156 S 0,0 0,1 0:00.88 accounts-daemon
2965 root 20 0 29216 292 64 S 0,0 0,0 0:01.35 console-kit-dae
3050 root 20 0 27408 320 0 S 0,0 0,0 0:00.48 upowerd
3115 stenyak 20 0 6260 320 84 S 0,0 0,0 0:00.51 screen
3119 stenyak 20 0 6260 248 40 S 0,0 0,0 0:00.61 screen
3121 stenyak 20 0 6260 132 16 S 0,0 0,0 0:00.48 screen
3124 stenyak 20 0 6388 1004 572 S 0,0 0,1 1:33.81 p2p.sh
3125 stenyak 20 0 34176 5604 1764 S 0,0 0,5 1:57.10 quasselcore
3129 stenyak 20 0 55268 2972 652 S 0,0 0,3 3:00.32 bot.py
3280 rtkit 21 1 18980 180 64 S 0,0 0,0 0:01.23 rtkit-daemon
3367 root 20 0 0 0 0 S 0,0 0,0 0:00.00 kauditd
3684 root 20 0 2940 340 112 S 0,0 0,0 0:00.00 udevd
3685 root 20 0 2940 208 0 S 0,0 0,0 0:00.00 udevd
5845 root 20 0 6444 904 592 S 0,0 0,1 0:00.00 cron
5876 stenyak 20 0 1932 428 368 S 0,0 0,0 0:00.00 sh
5878 stenyak 20 0 6048 1232 1080 S 0,0 0,1 0:00.00 memmon.sh
7857 stenyak 20 0 5060 508 448 S 0,0 0,0 0:00.00 sleep
9071 root 20 0 9816 860 200 S 0,0 0,1 0:00.10 sshd
9080 stenyak 20 0 10180 1112 136 S 0,0 0,1 0:06.93 sshd
9081 stenyak 20 0 9280 3232 148 S 0,0 0,3 0:00.62 bash
9436 root 20 0 7396 1188 684 S 0,0 0,1 0:07.05 bitlbee
10285 root 20 0 0 0 0 S 0,0 0,0 0:00.00 kworker/1:0
12728 stenyak 20 0 7960 2960 648 S 0,0 0,3 23:35.01 htop
12950 root 20 0 9832 80 76 S 0,0 0,0 0:00.08 sshd
12955 stenyak 20 0 9832 0 0 S 0,0 0,0 0:00.04 sshd
12956 stenyak 20 0 9304 36 32 S 0,0 0,0 0:00.64 bash
17939 root 20 0 23284 1080 320 S 0,0 0,1 0:00.11 gdm-simple-slav
18062 root 20 0 29808 964 364 S 0,0 0,1 0:00.38 gdm-session-wor
20066 root 20 0 0 0 0 S 0,0 0,0 0:09.72 kworker/0:0
28369 isabel 20 0 1932 112 108 S 0,0 0,0 0:00.03 sh
28404 isabel 20 0 3868 260 68 S 0,0 0,0 0:00.20 ssh-agent
28407 isabel 20 0 3516 0 0 S 0,0 0,0 0:00.00 dbus-launch
28408 isabel 20 0 3420 304 116 S 0,0 0,0 0:00.24 dbus-daemon
28416 isabel 20 0 6344 744 548 S 0,0 0,1 0:00.29 xfconfd
28422 isabel 20 0 8336 496 248 S 0,0 0,0 0:01.46 xscreensaver
28424 isabel 20 0 21932 580 340 S 0,0 0,1 0:00.17 xfce4-session
28430 isabel 20 0 24392 2032 1160 S 0,0 0,2 0:16.32 xfwm4
28432 isabel 20 0 117m 2072 908 S 0,0 0,2 0:01.26 xfce4-panel
28434 isabel 20 0 22520 256 256 S 0,0 0,0 0:00.06 Thunar
28436 isabel 20 0 21016 688 460 S 0,0 0,1 0:00.06 xfsettingsd
28437 isabel 20 0 119m 3344 1312 S 0,0 0,3 0:03.50 xfdesktop
28441 isabel 20 0 21072 272 268 S 0,0 0,0 0:00.07 kerneloops-appl
28443 isabel 20 0 29816 540 228 S 0,0 0,1 0:00.10 polkit-gnome-au
28450 isabel 20 0 263m 544 276 S 0,0 0,1 0:00.30 xfce4-volumed
28452 isabel 20 0 63936 488 0 S 0,0 0,0 0:00.31 tracker-store
28457 isabel 20 0 23240 768 472 S 0,0 0,1 0:00.18 xfce4-settings-
28460 isabel 9 -11 99532 500 248 S 0,0 0,0 0:00.38 pulseaudio
28470 isabel 20 0 21936 612 312 S 0,0 0,1 0:00.20 panel-6-systray
28472 isabel 20 0 339m 1348 552 S 0,0 0,1 0:00.42 xfce4-mixer-plu
28473 isabel 20 0 22028 2292 1740 S 0,0 0,2 0:59.33 xfce4-oragecloc
28514 isabel 20 0 371m 21m 2428 S 0,0 2,1 3:46.22 chromium
28517 isabel 20 0 111m 3020 288 S 0,0 0,3 0:00.58 chromium
28518 isabel 20 0 1948 64 0 S 0,0 0,0 0:00.00 chromium-sandbo
28519 isabel 20 0 120m 3324 288 S 0,0 0,3 0:00.20 chromium
28522 isabel 20 0 128m 3252 204 S 0,0 0,3 0:00.01 chromium
28558 isabel 20 0 208m 52m 476 S 0,0 5,3 1:25.92 chromium
28573 isabel 20 0 271m 103m 1244 S 0,0 10,3 3:01.64 chromium
28693 isabel 20 0 120m 3836 628 S 0,0 0,4 0:43.86 chromium
28696 isabel 20 0 83088 6412 108 S 0,0 0,6 0:16.55 GoogleTalkPlugi
28706 isabel 20 0 159m 6896 972 S 0,0 0,7 9:00.72 chromium
32332 root 20 0 0 0 0 S 0,0 0,0 0:06.24 kworker/1:2
32351 stenyak 9 -11 99108 1032 192 S 0,0 0,1 0:00.51 pulseaudio
32455 root 20 0 9816 108 76 S 0,0 0,0 0:00.11 sshd
32460 stenyak 20 0 9992 264 144 S 0,0 0,0 0:07.43 sshd
32462 stenyak 20 0 9300 1976 108 S 0,0 0,2 0:01.31 bash
Best Answer
You can't swapoff because the amount of swapped memory can't be overtaken by your RAM. You are getting legitimate error message.
Small snippet.
In my opinion, your workload increases your RAM demand. You are running a workload that requires more memory. Usage of the entire swap indicates that. Also, changing swappiness to 1 might not be a wise decision. Setting swappiness to 1 does not indicate that swapping will not be done. It just indicates how aggressive kernel will be in respect of swapping, it does not eliminate swapping. Swapping will happen if needs to be done.
Also, I don't know why you are trying to disable swap. Unless you have tons and tons of RAM, you should not disable swap.
Of course, you can reboot and swap usage will be zero then. And you can safely swapoff then. But, that doesn't solve the problem in long term.
Would you mind, posting the
/proc/meminfooutput.