Yosemite Startup Manager After Downgrading to Mavericks

1. Restart your Mac to Recovery Mode by pressing cmd R at startup
2. Launch Terminal from the menubar Utilities->Terminal

3. Enter the following command at the Terminal prompt:

   defaults write com.apple.DiskUtility DUDebugMenuEnabled 1
   

4. Press enter
5. Quit Terminal
6. Start Disk Utility and enable Show every Partition in the Debug menu
7. After a few seconds all partitions should be visible
8. Activate the partition Macintosh HD if it's present and greyed out.
9. Erasing/formating of Macintosh HD should be possible now.

If this doesn't work quit Disk Utility and according to this site

10. Start Terminal and enter following command at the Terminal prompt:

    diskutil cs list
    

    This will give you a list of the CoreStorage volumes on your system. Your old FileVault 2-encrypted drive should be the only one listed.

11. Copy the Logical Volume Group (LVG) alphanumeric UUID of your CoreStorage volume. The LVG should be the first UUID listed and it’s the one you want to delete.

12. Next, run the following command:

    diskutil cs delete lvgUUID #replace lvgUUID here by the UUID you found above
    

    This will delete your CoreStorage volume and reformat it as an unencrypted HFS+ volume.

If it's still impossible to erase Macintosh HD or if it's not present consider the Internet Recovery Mode or a USB Recovery drive and format the whole internal drive.

13. Restart to Internet Recovery Mode by pressing alt cmd R at startup.
    The prerequisites are the latest firmware update installed, either ethernet or WLAN (WPA/WPA2) and a router with DHCP activated.
    On a 50 Mbps-line it takes about 4 min (presenting a small animated globe) to boot to the recovery netboot image which is loaded from an Apple/Akamai server.
14. In the opening window choose Disk Utility and format/partition the internal drive. Choose Mac OS Extended (Journaled) and GUID partition table (available with the Options... button). Quit Disk Utility.
15. Plug in your Time Machine backup drive.
16. Start Time Machine and recover your old system

It appears to be quite complicated to boot an OS other than Mac OS from USB. See this thread on Ask Ubuntu for discussion on someone trying to do similar.

I know you don't want to install anything on the host system, but installing rEFInd or the older rEFIt may get you there. In that aforementioned thread a few people mentioned that once they had it installed a non-Mac OS USB drive would be listed as bootable from the bootloader menu.