Conclusion and a workaround
After exhausting all options on Windows, I decided to switch to Linux, mostly because I was frustrated with inability to profile and debug in detail.
I have moved the whole setup to Ubuntu 14.04. I first tried XAMPP but gave up because of conflicts between XAMPP and MySQL and MySQL Workbench. Then I moved to vanilla MySQL (5.5, I think) and vanilla Apache 2.
However, I was still left with the same problem – no visible bottleneck and resources still underutilized. I suspected throttling in TCP sockets (used between Perl code and MySQL), but further profiling proved this not to be the case.
Then, I turned my attantion to Perl DBI module DBD::SQL, thinking that it may be doing some throttlinig. I did some tests where I replaced DBI calls in Perl with system calls (system("mysql -e'INSERT INTO blah blah …'). I have determined that the performance did not change, therefore absolving DBI as a culprit.
I need to add one important detail now: I was in fact always running a number of my Perl scripts in parallel. Given that the CPU has 8 cores, this is necessary to utilize all of them, of course.
Further debugging showed that almost all my perl processes which were supposed to work furiously
were sleeping most of the time. Ubunty System Monitor shawed them as waiting on Waiting Channels wait_answer_interruptible or unix_stream_recvmsg. CPU History graph in System Monitor showed all perl processes jumping to 100% CPU utilization and then dropping to ~0% in unison. I suspected that MySQL server is not configured for multi threading, but htop showed 17 mysqld threads activated, confirming that all should be ok.
I suspected that all MySQL threads were waiting on the same semaphore and were locked out for most of the time. I dreaded delving into the dark bowels of MySQL trying to figure out what goes on inside. Instead, I decided to replace MySQL with MariaDB, even though MariaDB seems to have had the same issue originally when I was running it on Windows.
Lo and behold – this finally worked. My perl scripts were screaming.
One last problem remained: I had a very rudimentary method of parallelising the perl scripts: I would just run 10 or 20 with their respective loads and hope that they would utilize all the resources.
This has obvious drawbacks: if too many processes are spawned, the OS may spend too much time swapping them (although not a serious issue with only 20 processes, it becomes an issue with e.g. 1000). If not enough processes are spawned (e.g. less than 8, for each core) the CPU will not be utilized fully for sure. If too many processes exhaust RAM, Linux will turn to disk and will start swapping. As soon as this starts happening, everything grinds to a halt.
I searched but could not find a perl library/script/code which would spawn new processes only when CPU, memory and disk are under utilized. Hence I created my own: raspawn.pl (resource aware spawn) which I placed on github. Raspawn.pl spawns a number of processes while trying to keep resources utilization just below the maximum. It constantly checks the CPU, memory and disk utilization and only if all are less than ~90% utilized, starts a new process.
Finally, this worked. I can now process my whole load in around 7 days, instead of many months...
In some cases (perhaps most) the servers are already at capacity physically. An increase in the number of CPU's would require a motherboard swap. To add RAM to an existing server could be expensive, depending on how old the server is. Memory modules more than 5 years old and sourced from a dealer can be prohibitively expensive.
What all this amounts to is that upgrading from an 8 processor box with 32GB RAM to a 64 processor box with 128GB would involve purchasing an entire new server to replace the old one. This is still vertical scaling from the database/application point of view. A server that supports this many CPU's is likely far more expensive than 7 servers with 8CPU's each (keeping the original one as the 8th).
As per mustaccio's comment: High-end hardware is not just about more processors and RAM; designing hardware to support throughput and power/cooling requirements for dozens of multicore processors is far from trivial. Thus, a 256-core 2TB RAM server might cost $150000, which is about an order of magnitude more expensive than 64 4-core 32GB commodity servers
Best Answer
"You can't throw hardware at a performance problem." Or, rather, you can't do that twice.
One trick is to virtualize a big hunk of iron by having multiple VMs, each running an instance of MySQL. This does "horizontal" scaling on a single "vertically" scaled server. This at least helps with the brick wall I mentioned. But then you have a big, costly, single-point-of-failure.
My usual answer is "let's see if we can program smarter".
A recurring example is Data Warehousing. DBAs come to this (and other) forums bemoaning that their 'report' (a big
SELECTwith a bigGROUP BY) is taking too long. I point out the beauty of a Summary table -- 10x speedup, 10x smaller disk footprint. Poof, problem gone. No hardware upgrade could have gotten 10x.Replication provides nearly infinite scaling, nearly linear, of reads.
Sharding (horizontal scaling) provides nearly infinite scaling, nearly linearly, but it does not work for all applications.
The best I see in Vertical is adding enough RAM to turn an I/O-bound query into a CPU-bound query running in the buffer_pool. I regularly see a 10x improvement for that. But it is a one-time improvement. That is, adding even more RAM won't help any more.
So, to your title question, I say a resounding No.
OK, Vertical may help, but be aware of its limits.