Caching bitmap means that images and other bitmap resources are locally stored on the client computer for reusing them later. This way, the remote server or PC doesn't send images twice reducing the amout of data sent and saving your bandwidth usage.
The option makes particularly sense for slow (low bandwidth) connections, less if you connect to a machine in the same local area network.
If you enable the option the Remote Desktop client caches bitmaps into a BMC file located on the client hard disk in (example for Windows XP)
C:\Documents and Settings\<username>\Local Settings\
Application Data\Microsoft\Terminal Server Client\Cache
folder.
Note (interesting if your computer is part of a domain): since the cache folder is stored in a "Local Settings" folder, it won't be replicated as part of a roaming user profile.
Answering my own question. :-P
Shortly after posting, I googled across an application called Karabiner that does precisely what I needed it to do (in addition to a lot of other stuff).
Specifically, you can map the left and right Command keys to be Control keys in only Remote Desktop Connection and VMs.
Best Answer
Remote Desktop Connection Client vs. Remote Desktop
They appear to have the same functionality.
However they are for different versions of OSX. See below.
Microsoft Remote Desktop Connection Client for Mac Version 2.1.1
...
Source Microsoft Remote Desktop Connection Client for Mac Version 2.1.1
Microsoft Remote Desktop
...
Source Microsoft Remote Desktop