Difference Between Hyper-Threading and Multiple Cores – Explained

Theoretically the efficiency runs somewhere between having the full extra cores and hyper-threading but Intel CPUs have a higher IPC so the increase is rarely enough for an 8core AMD CPU to beat out an Intel quad core with hyper-threading but obviously it will vary from one workload to another.

The simple answer is that not all threads are executing simultaneously. For a fuller explanation, read on.

The operating system's task scheduler is generally thought of to schedule applications, and doing so allows you to perform one task while the computer is working on another. In the old days, the litmus test of multitasking was formatting a floppy disk while doing something else. If you wanted to really put the OS to the test, you'd format a floppy disk while downloading a file over a modem connected to the serial port. As the hardware became powerful enough to actually do it in a meaningful way, video playback sometimes featured in such tests as well. If the OS's task scheduler could handle running those tasks smoothly, then it could handle anything.

However, the task scheduler doesn't actually schedule applications (processes), it schedules threads. Each application has at least one thread, but can potentially use a large number of threads to split the work it does into related or independent parts. For example, it is common for an application to have one thread that handles the user interface, and to create another thread when the user initiates a potentially long-running operation (which might be things like printing, recalculating a spreadsheet, a development environment doing a symbol lookup, etc. etc.). Some programming environments introduce some amount of threads invisibly-to-the-programmer; for example, Java and .NET might do garbage collection in a separate thread, which is out of the immediate control of the programmer. Some programs create a number of threads early on and pool them, because creating new threads is a comparatively expensive operation (so you don't necessarily want to have to create a thread every time you need one). Anything that does a preview is usually done in a separate thread, so the remainder of the UI remains responsive while the preview is being generated. And so on. Taken together, all this means that the number of threads on the system at any time can easily be many times the number of processes.

Each thread can be in one of a few possible states, but the most important distinction is between running, runnable and waiting states; the terminology can differ slightly, but that's the general idea. At any one time, only one thread per virtual (because of hyperthreading and similar technologies) CPU core can be running (that is, executing machine code instructions), but any number of threads can be runnable (meaning that it is a candidate to get the CPU the next time the scheduler needs to make a decision about which thread should be allowed to run). Waiting (also known as blocked) threads are just that, waiting for something - the most common cases probably are that it is waiting for user, disk or network I/O (user input in particular is exceptionally slow).

The thread count you see in the task manager is the total number of threads in any of these states. For example, the Windows 7 system I am typing this on currently has around 70 processes started but almost 900 threads. With all the background processes to handle various tasks and how they are probably subdivided into a multitude of threads each, this is not an outrageous number.

Going a bit more into the depths of technical implementation, at the very core of a pre-emptively multitasking operating system's task scheduler is usually some kind of hardware interrupt hook. This means that the kernel can halt the CPU when it has no useful work to perform (this is almost certainly one of the reasons, if not the reason, why Linux checks the HLT instruction on boot on IA-32-compatible CPUs, and probably does similar checks on other architectures), safe in the knowledge that at some reasonably determinate future time, an interrupt will fire and the task scheduler will be invoked. Since the interrupt fires regardless of what other work the CPU is performing (that's the idea behind interrupts), the scheduler gets executed regularly and gets a chance to determine which thread should be executed during the following time slice. Since context switches are relatively expensive it's usually possible (at least through the source code) to tune how aggressively the scheduler switches between threads; switching threads more often causes the system to become more responsive, but the switching overhead means that the overall time to finish a given set of tasks is longer. The fastest system will be one that only switches between threads when the running thread is no longer possible to run (meaning it gets blocked waiting on something, or it has finished its job) because that minimizes the overhead, whereas the most responsive system will switch between threads every time the scheduler is invoked because that minimizes the average time to wait before a particular thread gets CPU time. The ideal setting usually is somewhere in between these two, and the tradeoff between those choices is likely one big reason why Linux offers multiple schedulers to choose from as well as some tuning parameters through the kernel configuration.

Cooperatively multitasking OSes and environments, on the other hand (Windows 3.x being one example), rely on each application to regularly cede control to the scheduler. There's usually an API function specifically meant to do that, and oftentimes many API functions will do it as part of their internal execution flow, because it helps make the user experience smoother. That design approach works well as long as all applications are well-behaved and cede control with short intervals during any long-running operations (long-running meaning more than a small fraction of a second), but an application that does not can clog up the entire system. This is one big reason why Windows 3.x did so poorly on the multitasking test I mentioned above, while OS/2 strolled along merrily while performing the same tasks on the same hardware: an application could tell the floppy disk drive to write a certain sector, and the time it took to do so before the call returned could actually be measurable (tens to hundreds of milliseconds or more); a preemptively multitasking system would have its scheduler break in on its next scheduled invocation, notice that the thread that is currently "running" is actually blocked by the write call and simply switch to another thread that is runnable. (In practice it's a little more involved, but that's the general idea.)

In both preemptively multitasking and cooperative environments, there is also the possibility of different threads having different priorities. For example, it is probably more important to execute in a timely fashion the thread that is receiving data over a communications link than the one that updates the system time display, so the receiving thread has high priority and the time display updater thread has low priority. Thread priorities play a role in the scheduler's decision of which thread to allow to execute (for example, very simplified, high priority threads should always execute before low priority threads, so even if the low priority thread has work left to do, if the high priority thread becomes runnable it takes precedence), but such specific scheduling decisions do not affect the underlying mechanism design.