In principle evince can be controlled through its dbus api. An example of its application is the SyncTeX plugin for gedit, which allows easier editing and browsing of latex source files in combination with evince. On the other hand, the dbus api itself has been found to change sensibly from gnome 2 to 3 and it's somewhat hidden so you'll have to dig into the evince source to find the exposed methods.
Concerning annotations: as of version 2.31 evince supports adding (but not deleting) annotations. This feature is currently broken in evince 3.2 (the version shipped with Ubuntu 12.04) due to a regression but a fix has been released upstream.
In your case, the program Evince was installed on your system as a component of the ubuntu-desktop metapackage. There are several excellent answers here about what a metapackage is and the implications of uninstalling a component of a metapackage and the metapackage itself. Here are two:
What is the difference between a meta-package and a package?
How to determine if a package is a meta-package from the command line?
Coming back to your case, if you run apt-cache show ubuntu-desktop and examine the output, you'll see a section called Depends:. This lists all the software, including Evince, specified by the ubuntu-desktop metapackage. If you attempt to uninstall any of the packages in that list, the ubuntu-desktop metapackage will also be uninstalled (along with other dependencies unique to the package you want to uninstall in the first place).
Going further down, you'll see this:
Description-en: The Ubuntu desktop system
This package depends on all of the packages in the Ubuntu desktop system
.
It is also used to help ensure proper upgrades, so it is recommended that
it not be removed.
In other words, sometime in the future, when you want to upgrade your system to the next permissible version of Ubuntu, the upgrade process will want to refer to the ubuntu-desktop metapackage. At that time, it will be advisable that you re-install ubuntu-desktop before undertaking the upgrade step. Until that time, not having ubuntu-desktop on your system doesn't mean you'll be without a "desktop"; its absence has no adverse day-to-day consequences.
As a precaution, the next time you want to uninstall anything, run a simulation first by leaving out sudo and by including -s. For a really scary simulation, look at the output of apt-get purge -s zenity. Then, if you have doubts, you could research the matter as you did for Evince!
Best Answer
Under Evince 3.4 you can activate a "back" button by editing the toolbar (Edit → Toolbar; drag and drop functions to toolbar):
I don't think there's any hotkey by default but you might be able to add a custom one.
If you're looking for customization you might be better off with a PDF viewer like Okular or qpdfview anyway. Both should offer the function you're searching for.