Ubuntu – Ubuntu as guest OS (with Vista host) stuck at 800×600 resolution

You need to install the VBox guest utilities to add support for the virtualised graphics hardware.

sudo apt-get install virtualbox-guest-utils virtualbox-guest-x11 virtualbox-guest-dkms

Previously you might have needed the "ose" versions:

sudo apt-get install virtualbox-ose-guest-utils virtualbox-ose-guest-x11 virtualbox-ose-guest-dkms

You need to install Virtualbox Guest Additions in the guest machine. You can do that by selecting "Install Guest Additions" in VirtualBox menu, and VirtualBox will automatically mount Virtualbox guest addition CD to the guest machine. If it is not mounted, you can do it manually by mapping VBoxGuestAdditions.iso to guest CD drive in VirtualBox device menu; the ISO should be found under /opt/VirtualBox . Once the ISO is mounted by the guest machine, you will see the CD icon in your guest desktop. Right click the icon, choose "Open with autorun prompt" and it will automatically compile and install kernel for your guest. That is it.

Having guest kernel addition installed, you will be able to change screen resolution from inside the guest machine, or you can use "Auto adjudst guest display" in VirtualBox "Machine" menu. I prefer to the latter.

I noticed however that some Ubuntu updates will disable/break guest kernel addition. When that happens, you just need to repeat the process above.