MacOS – Notification sounds not playing after upgrading to Yosemite

Every time a system notification sound should be played, the screen simply flashes. The Flash screen option in Accessibility is turned off but the system seems to be smart enough to flash screen instead of playing a notification sound when it is unable to do so.

Every other sounds works, even Skype's notifications.

I've checked the permissions in /System/Library/Sounds and I do have read permissions there. I can even play the sounds manually.

Any ideas what could be the cause of the problem and how to fix it?

EDIT: I have made sure that the "Sound effects" output device is set to the correct one.