I don't recommend Stealth Mode for this reason, constant pop-ups.
Mac Help says:
If you don’t either allow or deny access, the message remains
displayed, and any attempts to connect to the app are denied.
But in my experience, they dismiss instantly.
Unless you want to manage the firewall with one of the third-party apps--which are certainly available—Apple in their wisdom hath now given us an application firewall, and that's what we've got.
I fully understand your needs, but do not know specific answer.
Since you are a programer,you will be able to figure it out using the
Configuring IPFW firewalls on OS X
if you need special features like:
Firewall filters that include qualifiers on host or network addresses
Firewall filters that operate on other than TCP or UDP protocols
Firewall filters that include the whole range of ipfw qualifiers,
such as IP options, ICMP types or TCP flags
Per-filter logging configuration, including the ability to log
allowed connections and the option to not log certain types of denied
connections
NAT port forwarding or other custom NAT configuration
Different filter configurations on different network interfaces.
A persistant, searchable firewall log entry database with graphical
log viewer
Scriptable control of your firewall, such as via cron or other shell
automation
- Access and ability to easily edit the raw firewall configuration
text, including an integrated ipfw filter syntaxchecker.
...then you should consider using Flying Buttress.
The Author has stopped supporting it, but can be reached here:
Web site http://personalpages.tds.net/~brian_hill
Support web site http://personalpages.tds.net/~brian_hill
Support e-mail brianhill@mac.com
One more try: with something called Little Snitch
Silent Mode – Decide Later
There are times where you don’t want to get interrupted by any network related notifications. With Silent Mode you can quickly choose to silence all connection warnings for a while. You can then later review the Silent Mode Log to define permanent rules for connection attempts that occurred during that time.
Best Answer
Check /Applications/Utilities/Console.app after the firewall is turned off.
There might be a message about the change.