Removing voices
All voices are in /System/Library/Speech/Voices. Just like most other Library items in OS X, they are inherited, which means you could install them in /Library/Speech/Voices and ~/Library/Speech/Voices.
Removing items from /System/Library is indeed usually not a good thing to do, but here, it does not have any practical consequences beyond making the deleted voices unavailable. Indeed, the list in the System Preferences is dynamically generated from the items found in the said folder, just like the Sound alerts, or the wallpapers. You can confirm this by adding / removing / renaming elements in the /System/Library/Sounds folder, for example.
There's no other way to manage the speech items, but since this one is without consequences as long as you leave at least one voice, it is actually pretty nice to simply manipulate the filesystem. I have personally long removed Alex, along with other /System/Library and /Library items.
Installing voices / what to back up
As you underlined, Lion provides a “Customize” item in the list of voices that allows you to download voices in many languages. Therefore, if you ever delete the original “Alex” voice (the heavy one, 400+MB), you can download it again through this menu.
The “funny” voices, however, are not downloadable. You'll need to back them up if you want to remove them but keep a way to get them back (or you could use a package manager like Pacifist to extract only them from the Lion installer package, but that's a lot of hassle for not much). Nevertheless, considering they weigh, all together, less than 35MB, I'd let them in place.
Please note that you must leave at least one voice for the “Customize” item to be accessible. So, even if you want to remove all voices, leave at least one there, or you won't be able to install any again. “Fred” is the lightest “serious” voice, so I'd advise leaving it in any case.
Size note / a bit of history
As for the relative sizes, Alex is very heavy because it is of much higher quality, and is actually from a totally different generation than the others: it shipped with Leopard, while others have been around since Mac OS 7 (had a lot of fun with them, actually ;) ). Of course, Alex would have filled a full hard drive at the time. There's no "logic" that's shared between voices in those packages, hence the heavy weight of installing new “modern” voices in other languages.
You are not free to use them for any purpose, only for personal, non-commercial use while running Apple software. Read Mac OS X Lion Software License Agreement for full details (section G, it is rather short and clear):
Voices Subject to the terms and conditions of this License, you may use the system voices included in the Apple Software (“System Voices”) (i) while running the Apple Software and (ii) to create your own original content and projects for your personal, non-commercial use. No other use of the System Voices is permitted by this License, including but not limited to the use, reproduction, display, performance, recording, publishing or redistribution of any of the System Voices in a profit, non-profit, public sharing or commercial context.
Best Answer
Speech does support dictation, menu item and button selection, and some programs have "speakable items" for their own features.
It doesn't natively support the rest, but you can write scripts or Automator workflows to do what you want, and make them "speakable". When you've finished writing the scripts, use Speech to "Make this speakable" while each file is selected in the Finder, one at a time. This will put an alias in the Speakable Items folder, so make sure you put them somewhere where you won't then rename, move, or delete them.