To get the behavior you want you need to do a few things in addition to stopping the pop-up delay:
Install CCSM if not installed already.
apt-get install compizconfig-settings-manager from the terminal or search for "Advanced Desktop Effects Settings (CCSM)" in the Ubuntu Software Center.
Select the System Settings option in the power button menu (top right corner) and launch CCSM from there.
For Ubuntu 13.04 (Ringtail)
This may also work with > 12.10; unfortunately Unity changes significantly between versions.
Select the 'Ubuntu Unity Plugin' in 'Desktop'.
Under the 'Switcher' tab un-tick 'Show live previews of windows in the switcher'.
I'm not sure the popup delay exists in 13.04; I don't seem to notice any on my machine at least.
This section also allows you to swap the bindings of the switcher for all viewports and the viewport switcher.
For Ubuntu < 12.04
Select the Static Application Switcher in 'Window Management'.
Under the 'appearance' tab check 'icon' and 'show icon only'
And as @fossfreedom said, you can get rid of the .2 second delay "feature" under the 'behavior' tab by reducing 'popup window delay' to 0.
If you don't want them in rows you can switch to the 'application switcher' and change the settings similarly. Just make sure you do this by de-selecting the 'static application switcher' first, or else you'll get binding conflicts.
I found it to be compiz doing the things - I did some customization and at first I wanted ctrl+alt to bring up my launcher - but setting this in compiz makes ctrl+alt+t bring up trash instead of terminal, I bound my commands otherwise and worked around the issue, but I'll leave this post open for future people with the same issue. Or maybe to get the problem solved, or at least an explanation to why it behaves at it does!
Compiz seems buggy anyhow - lot of CPU usage etc. so I guess there's a good chance this is some compiz bug too.
It ends up that there is a delay of 2ms set in the alt-tab switcher (called the Static Application Switcher) by default.
To change it, use you can use CompizConfig Settings Manager . After you install it, run it via the dash by searching for compiz or just run
And then set
Popup Window Delayto 0 in the behavior tab.
And crank up the speed: