If you need to do this manually on a file (or files) you can run the following command to remove the Quarantine flag:
xattr -d com.apple.quarantine /PATH/TO/FILE(S)
So in your case since the quarantine flag is not being removed you will want to run the command with sudo as the primary cause for the flag not being removed after hitting "Open" is because you don't have permissions to do so.
Not nearly as complicated, but perhaps of value, is the app Choosy. It allows you to choose your browser every time you open a link outside of a browser. While this might sound onerous, it's actually pretty transparent, and I find it very useful in the leadup to going live with a new web project and testing in multiple browsers over and over and over…
Might suit the bill if one added keystroke isn't a problem. Of course, once you're inside Safari, links clicked will open in Safari (although there's even a way to modify this behavior.)
Best Answer
For Opera (and possibly for all Chromium-based browsers) it's enough to run the binary with
--no-default-browser-check
flag.