This is an attempt to provide a complete answer by summarizing bits from individual already existing answers, with some added content. The list tries to include all possibilities for a general case, even if some can be ruled out for this particular case.
CPU instruction set difference
As described by Joel Coehoorn, for a native x64 code a larger register set is available, as well as some additional instructions (e.g. all x64 CPUs are guaranteed to implement SSE and SSE2 instructions). This means native x64 OS and drivers can be more efficient.
Better RAM utilization
64b is able to use full 4 GB (and even more when available). Moreover, some OS internal limits (paged/non-paged pool, maximum cache size) are increased with 64b OS. As a result, both memory paging and file caching may be more efficient.
Different hard disc position
When you have two OSes installed on two partitions of the same disc, one of those two partitions is closes to the outer edge, which is generally faster.
Different OS state
When you have two OSes, each of them may be in a bit different state. One of them may be more fragmented than the other one, one of them may have a bit cleaner registry or less installed services or background applications than the other one.
Better drivers
Another possibility is x64 drivers might be of a better quality than x86 not because of some fundamental architecture differences, but simply because their vendor took more effort to optimize and fine tune them.
64b addressing
I add this only for completeness sake, as for vast majority of everyday applications this difference plays no role, and it definitely is not relevant to what Jasper describes in his question.
If you have an application which processes huge amounts of data (e.g. working in Photoshop with huge pictures), running 64b version can make a huge difference provided you have 4 GB or more of RAM installed.
What does "faster" really mean?
Here is the list from a comment on other answer, what exactly are the "faster things" Jasper can see:
- windows animating smoother
Hard to explain. Windows animation is done by the GPU in Windows 7 and the GPU is the same. The load on CPU is negligible. (I suppose you are using Aero desktop).
- certain photo editing programs drawing faster (both x86 and x64)
- HD videos running smoother (on MPC-HC x64), and an overall more stable experience.
Could be better tuned drivers or drivers taking advantage of x64, .
- files being copied a lot faster
Could be caused by a faster partition (not related to OS at all) or by better cache handling in x64 OS.
- The games didn't show any change.
Understandable. The games performance is determined by CPU and GPU. In case of CPU limitation the limiting code is the game itself, which is the same x86 code with little interfacing to x64 OS. The CPU performance is therefore the same. The GPU is the same. The driver and OS overhead in a well programmed game is low, therefore any driver or OS differences are not likely to show here.
Best Answer
x86 refers to 32-bit software. x86_64 and x64 both refer to 64-bit software. Running 32-bit software on a 64-bit version of Linux is not the same as on Windows where 32-bit apps are thrown into a Program Files (x86) folder, some modifications are needed such as installing ia32 libraries.