Ubuntu – High System Load, Low CPU/RAM Utilization on Ubuntu 15.04

15.04cpucpu loadkernelserver

not really a system administrator here but really trying to just set up a server (a rented VDS, really) for some friends.

I recently transferred basically game servers/MySQL/web sites over from one VPS to another – while there hasn't been any issues on the new one I keep seeing my system load spike and take up both processors; previous server system load averaged at about .3-.5. Previous server was on Ubuntu 14, I exported a list of packages I installed from there and apt-get installed them on the new server; I also rsync'd most of the files from the old server over as well (I'm thinking I copied over something bad that's messing with my kernel…)

Anyway, here is the results of my uname -a:

 Linux ophq 3.19.0-18-generic #18-Ubuntu SMP Tue May 19 18:31:35 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

And the results of landscape-sysinfo/logging in screen:

  Welcome to Ubuntu 15.04 (GNU/Linux 3.19.0-18-generic x86_64)
  System load:  2.13                Processes:           11
  Usage of /:   22.6% of 196.64GB   Users logged in:     1
  Memory usage: 32%                 IP address for eth0: 123.123.123.123
  Swap usage:   0%

(currently one game server is in use hence the memory usage – I have to reduce how much RAM is allocated to Minecraft from the default values)

Result of top:
http://ericbarber.me/serverproblem/top.png

To add to this – if I hit F and then hit S on 'Process Status' and resort the top lists I have 2 commands listed under 'D'… kworker/u30:0 and kworker/u30:1 which leads me to my kernel assumption…

I'm totally stumped on why load average is so high – I had my users test on both MC and the CS:GO servers and they aren't experiencing any lag – I also tested the web servers and they're delivering pages extremely fast (in comparison to the old server.)

I thought it may be an interrupt issue, so here's the results of cat /proc/interrupts:

http://ericbarber.me/serverproblem/interrupts2.png

Along with this, another question suggested running grep . -r /sys/firmware/acpi/interrupts/ and disabling any values above 0…although all my values are 0 unfortunately.

same url as above serverproblem/interrupts.png

I installed perf and did a quick 30 second report – but I don't understand this output too much:

same url as above serverproblem/perf.png

I'll omit CPU info, but it is an Intel Xeon CPU E5-2690, 2 cores, 2gb RAM, and I believe about 500gb harddrive. My apologies if this is a dumb question or has been asked before – I've been working on this for a few hours now and I'm running into dead-ends with Google past just starting over from scratch…which preferably I would like to avoid.

Apologies on the links..new user limitations.

Edit:
To add, the results of mpstat:

Linux 3.19.0-18-generic (ophq)  06/05/2015  _x86_64_    (2 CPU)

02:10:35 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal       %guest  %gnice   %idle
02:10:35 PM  all    7.28    0.00    1.72   47.13    0.00    0.09    0.53        0.00    0.00   43.24

Best Answer

This ended up being what I believe to be a kernel bug. Upon updating to 4.0.0-040000-generic #201504121935 my CPU wait has been normal and system load under .10 in most cases unless something is happening on the hosted servers.

Anyway, I used the following link to help : http://ubuntuhandbook.org/index.php/2015/04/upgrade-to-linux-kernel-4-0-in-ubuntu/

and just to keep in compliance with the rules, I did the following as root and then rebooted the machine:

wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.0-vivid/linux-headers-4.0.0-040000_4.0.0-040000.201504121935_all.deb
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.0-vivid/linux-image-4.0.0-040000-generic_4.0.0-040000.201504121935_amd64.deb
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.0-vivid/linux-headers-4.0.0-040000-generic_4.0.0-040000.201504121935_amd64.deb
dpkg -i linux-headers-4.0.0*.deb linux-image-4.0.0*.deb
update-grub

As far as how I came to this - after reading through countless forums and newsgroups/mailing lists and getting nowhere (tried messing with BIOs, boot options, commit=60, disabling services, changing physical server location, etc.) I decided to either downgrade or update the kernel...being that 15.04 is new I updated. Still unsure the root cause as I haven't seen any other reports of this issue, my assumption is when I used rsync from my old 14.10 system a faulty driver was copied over or a faulty kernel file - why 4.0.0 fixes this is beyond me...but at least no more kworker writing every 5 seconds to kern.log and my harddrives.

Related Question