According to John Gruber, this is not possible, unfortunately:
AirPlay is the killer feature for Apple TV. But perhaps we’d better keep that in the future tense for now: it’s going to be the killer feature for Apple TV. After installing today’s various software updates for iOS devices, the only apps I’ve seen where AirPlay works for video are the built-in iPod app (a.k.a. “Video” on the iPad and iPod Touch) and YouTube. In other apps, AirPlay is audio-only.
Okay, after dealing with this for quite some time, I finally found a solution that worked for me. I figured I'd come and share for others as well.
In order to resolve the choppy playback, I ultimately went in to Settings -> General -> Reset -> Reset All Settings. After resetting, the device downloaded the very latest OS for the Apple TV, and after everything was done, the Airplay quality between the MacBook and AppleTV is actually pretty decent.
In testing, I was able to mirror my display without any noticeable choppiness. I was also able to extend my desktop on to the TV, and this worked surprisingly well. Even playing YouTube videos was of decent quality, with almost zero choppiness during playback.
What I have not tried yet... and will update this answer later on, is reconnecting my photo streams. As it's common to have 1000+ photos in my shared streams, I'm concerned that this may have slowed the device down (just a guess, honestly). I'll do some more testing today, mirroring as is, then adding the photo streams and comparing, to see if that causes any issue. If it does, I can always reset again.
Best Answer
It's a Treasure Hunt!
On the iPad:
Click the wrench icon to open the Settings dialog. One of the settings is AirPlay - Bluetooth. Set that to On, and the usual audio output selector will appear in the toolbar.
On the iPhone:
Click the gear icon to open an untitled two-sheet settings dialog. Select the Song sheet. The AirPlay - Bluetooth selector is there.