You haven't specified if this is a new build, or if this is your existing system, but here are a few non obvious things to check:
Before I offer some suggestions, here is why putting your motherboard in an oven is a REALLY BAD IDEA
Capacitors are full of electrolytes which react to heat just like water. Putting a capacitor into an oven could figuratively boil off all the liquid inside. You do not want to heat up capacitors.
Ever heard of xbox's getting the red ring of death, and that you can fix it with the 'towel trick'? Bad idea.
https://electronics.stackexchange.com/questions/18866/can-capacitors-be-exposed-to-temperatures-above-the-max-operating-temp
4 Pin Connector
Since removing the 4 pin connector causes it to work (as far as you have stated), then this is likely a power problem. Two very important things to consider.
Accidentally using the power supply 4 pin
Many 24 pin power connectors are also 20 pin connectors. Those 4 extra pins will fit into the cpu 4 pin header, but they have the wrong pinout. While this is a very slim chance it is the answer, I have seen this before from new system builders.
Accidentally using a GPU 4 pin connector
I've also seen 6 pin and 8 pin gpu headers plugged into the cpu 4 pin headers.
Bent CPU pin
A bent pin on the cpu can also cause the problems you are describing.
Bad Capacitors
If this isn't a new build, check to see if you have any blown capacitors.
http://en.wikipedia.org/wiki/Capacitor_plague
Modular Power Supplies
If you replaced your power supply and it happens to be modular, make sure you use the cables that come with the PSU. There is no standardization between manufactures for modular pinouts. You can kill your components if you don't.
See this super user question How do I interpret the Asus motherboard LED and beep patterns?
Is the CPU bad?
I've repaired computers for 15 years. I've only ever seen 2 cpu's 'go bad'. It is almost always the power supply then the motherboard. I suggest that you buy a new motherboard.
Best Answer
When things go wrong, you should find out what is causing the problem before you start replacing components. Otherwise, you don't know what you did to fix it and you might end up changing all the parts out instead of building/buying a new PC.
Can you boot and get into the BIOS/CMOS? If not, check these possibilities:
If that doesn't work, power down, unplug the HDD and try to boot with just your video card, memory, CPU (and fan), and PSU. You don't need a HDD to get into the CMOS/BIOS.