How to test if a web-services client application supports TLS

Although the example you cite in your question is trivial to achieve with a proxy because the URLs are not encrypted, and therefore easy to add to a blacklist, it IS possible to inspect HTTPS traffic going through a proxy.

Enterprise deployments usually achieve this by deploying an internally trusted certificate to their entire installed end user machines. Connections to the proxy server are done via this certificate (whether the users realise it or not), where the proxy software can decrypt the payload, inspect it and decide on its validity. The onward connection to the end site is done with "real" certs.

This is a bit of a sad state of affairs really, as it breaks the trusted model of SSL and TLS - but I know for a fact it's done - as it happens where I work.

I figured it out. :D This solution satisfies all of my requirements and meets all of my goals, perfectly. Performance isn't too bad, either, considering the level of indirection that is necessary to achieve this.

The general approach is thus:

1. Set up a local Certificate Authority (CA), and generate an RSA "server key" and "client key" (I used 256-bit encryption). For this, I used Easy-RSA version 3.0.0-rc2.

2. Run any bog standard HTTP proxy on the "Debian Box" (the server on the public Internet), making sure to have it listen on localhost only (it should NOT be exposed to the public Internet). For my purposes I used Privoxy, but Squid would've worked just as well. Since it's only listening on localhost, authentication is not necessary (unless there are processes running on your box that you don't trust; in which case, yikes...)

3. Download stunnel and install it on both the client and server. The process for doing this is going to be OS-specific; in my case, I chose to compile stunnel from source (paranoia...) for Windows, which was a rather involved process I won't detail here. On the server side, it was available in the package manager :)

4. Stunnel's configuration was quite daunting at first, but it's simpler than it seems! Basically, on the server, you need something like the below "server's stunnel.conf". On the client, you need something like the below "client's stunnel.conf".

5. Start Privoxy; start stunnel on the server, pointing it to the config file; start stunnel on the client, pointing it to the config file. There's really nothing all that special about Privoxy's config; the default was fine for me.

6. In Firefox, your browser of choice on the client side, set the HTTP and HTTPS proxy to be the same as the port your client's stunnel is listening on -- probably something like localhost:8080.

I should probably note that if your local network's proxy demands some kind of authentication, you will either have to get stunnel to authenticate for you, or else use another local intercepting proxy and chain them together -- something like Firefox -> stunnel -> local authenticating proxy -> LAN proxy/gateway -> internet -> your server's stunnel -> privoxy.

That's a lot of copying, but it works!

;This is the *client's* stunnel.conf.
[https]
accept = localhost:9020
connect = your.lan.proxy:80
client = yes
protocol = connect
;protocolHost should be the same as the "accept" for the server
protocolHost = 1.2.3.4:443
;Same CAfile, different cert and key pair
CAfile = ca.crt
cert = client.crt
key = client.key
;VERY IMPORTANT!!! Make sure it's really your server and not a MITM attempt by your local network by making sure that the certificate authority "ca.crt" really signed the server's cert
verify = 2
;More performance tweaks...
sessionCachetimeout = 600
sessionCacheSize = 200
TIMEOUTidle = 600

.

;This is the *server's* stunnel.conf.
[https]
;1.2.3.4 is a publicly-routable, static IP address that can be connected to by your box that's under the firewall
accept = 1.2.3.4:443
;localhost:8118 is an example of where your local forwarding HTTP(S) proxy might reside.
connect = localhost:8118
CAfile = ca.crt
cert = server.crt
key = server.key
;VERY IMPORTANT!!! Without this, anyone in the world can use your public stunnel port as an open proxy!
verify = 2
;Set some timeouts higher for performance reasons
sessionCacheTimeout = 600
sessionCacheSize = 200
TIMEOUTidle = 600

Once everything is configured, the end result ends up looking something like this:

1. Your web browser connects to localhost:9020 (stunnel) and treats it like a proxy that can accept HTTP and/or HTTPS connections.
2. Once stunnel gets a connection from your browser, it reaches out, through your firewall's proxy/gateway, to establish a TLS session with your remote server. At this point, your client verifies your server's PKI certificate, and vice versa.
3. Once the TLS session is established with your remote server, stunnel passes along the data coming from your browser, e.g. an HTTP request or an SSL tunnel request, through the local proxy and directly to your server. This channel is encrypted, so your local network can't tell what the data contains, they can only guess by doing traffic analysis.
4. Once the stunnel instance running on your server starts receiving data, it opens a connection to e.g. localhost:8118, which would be where your HTTP(S) proxy server, in my case Privoxy, is listening.
5. Privoxy then acts like a normal forwarding HTTP proxy server, and forwards your requests on to the public Internet through the server's ISP.

The amount of sockets and buffers involved makes this method very high overhead, especially if you're nesting an SSL connection through the proxy, but it has the advantage that your local network has no way of knowing which sites you're visiting over SSL. I mean, it knows you're visiting your server, but aside from that, it doesn't know if you're visiting Gmail or SuperUser or whatever. And your local gateway has no way of filtering or blocking you.