How to tell gpg that I do not want password caching for a specific program

There is the gnome-keyring-daemon and seahorse which makes key & password management very easy.

Basically if you're running gnome-keyring-daemon as a gpg agent, it has the ability to unlock your GPG keys automatically. It does this by maintaining a password keyring, which contains the passwords to things like web sites, GPG keys, SSH keys, etc. This password keyring is then secured with it's own password. So you unlock it, and the gnome keyring unlocks everything else.
As an added bonus, gnome-keyring-daemon has a "login" keyring, which if it's password matches your user password, the keyring is automatically unlocked when you log in.

Configuration

How to get this working? Just install gnome-keyring-daemon and seahorse. The package should do all the system configuration for you. Just make sure you're not starting another keyring daemon or GPG agent. Whichever starts last "wins", and the gnome keyring starts in the PAM stack, so extremely early.

If your GPG keys are stored in ~/.gnupg, it will automatically pick them up and act as the GPG agent for them. Same goes for SSH keys stored in ~/.ssh

The first time you try to use the private key, you'll get a dialog that looks like this: (I triggered it by a simple command line gpg -d myfile.gpg)
Just select "Automatically unlock this keyring whenever I'm logged in"

Now we haven't really talked about seahorse. That's because it's not strictly necessary. All this has been done with just the regular gnome-keyring-daemon. However with seahorse you can view and manage all your keys & keyrings. And if you use centralized authentication (LDAP), you'll need to use it when you change your login password to also change the password on the "login" keyring to match it.

Other passwords

As alluded to earlier, gnome-keyring-daemon can also store web site passwords. Last time I checked chrome supports this, but firefox does not. However there is one trick to getting it working.
By default you'll have 2 keyrings, a "login" keyring, and a "default" keyring. The "default" keyring is the default (hence the name). But it's a separate keyring, so it doesn't automatically get unlocked. In seahorse, if you right-click the "login" keyring, there's an option to "set as default". Select this and it'll start getting used for passwords. I personally just delete the "default" one and use "login" for everything.