MacOS – Mac Pro Dual Displays

I've not had your issue exactly, BUT I've found DVI connection to be the most reliable with external monitors and OS X. AS you said, you intended to try DVI-D next. I imagine that would get both monitors firing up.

Side note: OS X 10.8 seems to despise 10.5. I've had a bunch of unrelated madness crop up due to failed mountain lion install and then recovery via at some point using 10.5 install disc. Disastrous results.

I would probably install the old OS onto an external USB drive to be sure it's not a failure unrelated to the OS.

It's rare that the OS breaks an adapter that was working - even if the change happened at the same time as an OS update. Also, you can usually rule out software by updating to 10.9.3 (in your case) or to the next point update in case there was an error that was caught in the drivers or settings.