PC getting stuck at the Intel boot screen

there is this strange problem I am facing for quite a while now at the time of starting up my pc.

WHERE it occurs: The problem occurs at the very first screen that appears after pressing power button where there are two ‘Intel’ logos with few two digit alphabets which keep on changing at the right bottom of the screen like: BA BC EB usually this screen last for 2, 3 seconds.

WHEN it occurs: BUT at the times when the problem occurs my PC remain stuck at this screen at the BA point (see image attached), with no CPU activity happening it just remain stagnant there.

Other Important Observations/Information:

• This problem ALWAYS occurs when I Restart my PC or even when I
  Shutdown and start it again without switching off main power (UPS +
  Main switch)

• I have also noticed when I have a bootable USB attached to PC at the time of booting up it gets stuck at the BC point instead of BA

• To correct it I have power it off, switch off electric supply behind
  including UPS, Main power, wait for 3,4 seconds and power it on
  again, than it all works fine.

• I have changed my power supply for the PC as thinking that this issues is somehow related to power problem but didn’t helped, I have also reinstalled windows after this problem but this problem is still there.

• PC configuration: 64 bit system | Intel Core 2 Quad Q8200 | 4 GB Ram |

• Screen shot of screen where PC gets stuck

Kindly help me out with how to get rid of this problem as it gets really problematic in many ways including while installing windows or when you have to restart your pc etc.

Thankyou!