Removing voices
All voices are in /System/Library/Speech/Voices. Just like most other Library items in OS X, they are inherited, which means you could install them in /Library/Speech/Voices and ~/Library/Speech/Voices.
Removing items from /System/Library is indeed usually not a good thing to do, but here, it does not have any practical consequences beyond making the deleted voices unavailable. Indeed, the list in the System Preferences is dynamically generated from the items found in the said folder, just like the Sound alerts, or the wallpapers. You can confirm this by adding / removing / renaming elements in the /System/Library/Sounds folder, for example.
There's no other way to manage the speech items, but since this one is without consequences as long as you leave at least one voice, it is actually pretty nice to simply manipulate the filesystem. I have personally long removed Alex, along with other /System/Library and /Library items.
Installing voices / what to back up
As you underlined, Lion provides a “Customize” item in the list of voices that allows you to download voices in many languages. Therefore, if you ever delete the original “Alex” voice (the heavy one, 400+MB), you can download it again through this menu.
The “funny” voices, however, are not downloadable. You'll need to back them up if you want to remove them but keep a way to get them back (or you could use a package manager like Pacifist to extract only them from the Lion installer package, but that's a lot of hassle for not much). Nevertheless, considering they weigh, all together, less than 35MB, I'd let them in place.
Please note that you must leave at least one voice for the “Customize” item to be accessible. So, even if you want to remove all voices, leave at least one there, or you won't be able to install any again. “Fred” is the lightest “serious” voice, so I'd advise leaving it in any case.
Size note / a bit of history
As for the relative sizes, Alex is very heavy because it is of much higher quality, and is actually from a totally different generation than the others: it shipped with Leopard, while others have been around since Mac OS 7 (had a lot of fun with them, actually ;) ). Of course, Alex would have filled a full hard drive at the time. There's no "logic" that's shared between voices in those packages, hence the heavy weight of installing new “modern” voices in other languages.
Another shot in the dark...but this page suggests downgrading to Perian 1.2.2 on Lion (see "Solution 1").
The latest version 1.2.3 of Perian doesn't work on Mac OS X Lion. So if you have a Mac running Mac OS X 10.7, say, Lion, you have to downgrade the previous version, say 1.2.2. The version of 1.2.3 of Perian does not support QuickTime 10 in Lion and fails to fix the loss of audio after one second in xvid files with ac3. So if you have a Mac with Lion, converting video to QuickTime is highly recommended.
However, that particular page is all about how to play AVI files with Quicktime, so it may not work, but might be worth a try?
Best Answer
In those cases i use a little app called AppCleaner.
The just drag the app in question in to it and it will find all associated files to that program and ask you if you want to delete them.
If you do not have the original app any more, you could use Terminal and find the files and delete them.
type
mdfind TechSmithand see what comes up