Lately, I've been experiencing video loading behavior issues again, even after applying what I suggested in my original answer.
Dash playback
From what I understand, the underlying issue is YouTube's implementation of Dynamic Adaptive Streaming over HTTP. From what I can tell, there are both server-side and client-side implementations at work here. The Wikipedia article linked above says:
Google's YouTube experimented with supporting MPEG-DASH on the server side. Google Chrome supports it on the client side. However, the implementation of the feature has resulted in video playback being severely degraded by various bugs, such as the video quality options being randomly greyed out and unselectable without multiple refreshes of the page.
Which sounds very familiar.
Solution
The currently very popular approach to solve this issue is to use the YouTubeCenter extension, which is available for a wide variety of browsers.
It allows you to disable Dash playback, but also notes:
Please note that by disabling Dash Playback will result in 480p and 1080p not being available. This is because of a recent YouTube change.
Disabling Dash playback
To disable Dash playback, just open the settings menu:
And untick the corresponding checkbox on the player page:
The result will be that videos will load as fast as possible and fully.
Please note, YouTubeCenter removes advertisements by default. If you want to support your favorite YouTube channels, be sure to consider disabling this feature.
Original answer:
Disabling the internal Flash plugin of Chrome
Seems like this is again a case of rubber duck debugging...
After posting this, I thought that I should probably check if I can simply disable the built-in Flash player in Chrome and see if that has an effect. The process is actually outlined in the Adobe Flash Player Help.
Grab the installer for the latest Flash Player from Adobe and install it.
Go to chrome://plugins/.
Click the Details button to expand the details about single plugins:
There should be 2 entries for the Adobe Flash Player. One is the built-in version in Chrome, the other is the normal installation. Disable the built-in one.
You'll recognize the internal one easily, as it resides in the PepperFlash folder in the Chrome installation directory.
Now the caching behavior should be identical to the behavior observed in Firefox.
Please note that disabling the built-in version of Flash is usually discouraged.
Using the SmartVideo extension
Alex pointed out an extension called SmartVideo. This extension gives you more control over the buffering behavior of Chrome.
I tested it for a while and am quite pleased with the results. Especially the option Ensure that videos are buffered even if they are paused affects the undesired behavior as outlined in my question.
It has several advantages over disabling PepperFlash. First of all, it gives quite a few options which I didn't even have in mind before. Like preventing YouTube to auto-play videos.
Using PepperFlash is also an advantage when using multiple monitors. If you have a fullscreen video on one screen, and another Chrome window on the second, giving focus to the second Chrome window will still keep the fullscreen video in the foreground. In contrast, if you use the external Flash player, focusing the second Chrome window would pop the taskbar in front of the fullscreen video.
Best Answer
I was able to fix this. I have an external audio interface that is connected via USB. It was certainly working, I heard system audio and audio from some music software that I use just fine, but something went wonky for some reason and changing the USB port that it was connected to fixed the issue with videos. I have no idea why this was, I hadn't moved the USB port since the last boot-up so I guess Windows just decided it didn't like it overnight or something.