I have found one acceptable (but not entirely satisfying) solution to the problem:
I went to Control Panel -> Power Options -> Change plan settings -> Change advanced power settings -> Processor power management -> Minimum processor state and I changed it from 5% to 100%.
All the sluggishness disappeared, at the cost of 10 more watts of power consumption when idle. PassMark PerformanceTest finds my machine to be at 94% compared to the rest of the world, and the machine certainly feels like that: CPU utilization when idle is down to 1%, and there is no annoying lag anymore, at all. I am so happy.
The thing is, I cannot understand how this can be, so I am afraid that there may be something more sinister at play. A "Minimum processor state" of only 5% is quite common in power profiles of modern computers, and of course it is not causing the kind of sluggishness that I have been experiencing.
So, it appears that on my system, the module responsible for controlling the throttling of the CPU (whatever this may be, hardware, BIOS, windows, I don't know,) is doing a terribly bad job: it is supposed to allow the CPU to run at maximum speed when needed, but it never does, not even when running the PassMark benchmark.
So, I am wondering: what is the root cause of this behavior?
Another strange thing is that the "Windows Experience Index" is reporting roughly the same numbers now as before: about 7.8 for the processor, 7.9 for the memory, and 6.6 for the graphics. (The mediocre graphics performance is mainly because I am using the CPU's integrated graphics for the time being.) This could be because Microsoft's benchmark is clever enough to enable full CPU while benchmarking and restore it once it is done, but I do not know that it does that for a fact, so it could also be an indication that something else, more strange is going on.
Feel free to add an answer if you have any more insights on this.
Best Answer
Some time ago I wrote Windows Perfmon: The Top Ten Counters at my old blog. The article is a quick introduction to using Perfmon to determine where hardware bottlenecks are: processor, memory, disk, or network (and secondarily, to determine current operational loads on these resources). I can't emphasize this step enough: first make sure you know which hardware resource is overused. Then go hunting the resource hog.
Once you've determined which resource is overused, you have two choices: add more of that resource (get a faster CPU, add memory, get a faster disk or NIC) ... or track down the top users of that resource, and either kill them or take steps to make them less piggish.
The easiest way to track down top resource hogs is with Process Explorer. Be sure to run it from an account with Admin privs. Here's a quick-and-dirty list of things to look for:
Good luck!