Try New OS Releases – Methods Without Committing

I'm not really sure what you want, but if you want it to behave this way you should install Ubuntu on the usb disk rather than installing the Live-CD on it.

The answer is "it depends."

If your GRUB is an EFI version of GRUB, then in theory it can replace rEFIt. Doing so on a Mac is a bit tricky, though, and I don't have step-by-step instructions for doing the job.

Most Ubuntu installations on Macs install Ubuntu in BIOS mode, though, and therefore use a BIOS version of GRUB. In this configuration, both boot loaders are (more-or-less) necessary; rEFIt is unable to boot a Linux kernel directly without jumping through awkward hoops, and the BIOS version of GRUB can't boot OS X.

An alternative you might want to consider is replacing rEFIt, which is no longer being developed, with its fork rEFInd, which is. Unlike rEFIt, rEFInd can boot a Linux (3.3.0 or later) kernel directly, so by installing rEFInd, you can bypass GRUB. The procedure to do this would be:

1. Install rEFIt from OS X by using its install.sh script.
2. Manually install the driver for whatever filesystem you use on Linux's /boot partition, or on the root (/) partition if you don't have a separate /boot partition. See the rEFInd documentation for details on its drivers. (You'll just need to copy one file.)
3. Reboot. rEFInd should appear and show you an option for OS X and at least one for Linux (with a penguin), and one for GRUB (with either a generic icon or an Ubuntu logo). Select the one that boots via GRUB.
4. In Ubuntu, locate the mkrlconf.sh script that comes with rEFInd and run it with sudo. This creates a configuration file that helps boot the kernel directly.
5. Reboot. You should now be able to launch the direct-boot option(s) in rEFInd. (The ones with the Linux penguin icons.) If you have problems at this point, you'll have to resolve them, or maybe just use rEFInd as you'd been using rEFIt before.
6. If everything's working, you can optionally remove the old GRUB entry from rEFInd's menu as follows:
   • If you'd been using a BIOS-mode GRUB, locate the refind.conf file (on the ESP or the OS X / partition, depending on how you installed rEFInd), uncomment the scanfor line, and remove the hdbios option from that line. Note that if you also boot Windows, though, this will remove Windows from the boot options, so you probably don't want to do this if you're triple-booting with Windows.
   • If you'd been using an EFI-mode GRUB, you can either remove its files (probably in /boot/efi/EFI/ubuntu) or use the dont_scan_files or dont_scan_dirs option in refind.conf to restrict scanning for GRUB.