Since there are no answers to this question yet, and all the discussion took place in comments, let's write a summary. And please, up vote the relevant author in their comments on the question itself. Please please please.
Per @ghoppe;
When using the iPod app in list view (list view = the view with selections for your chosen category), the "More …" category on the very bottom right hand side has a few additional options, name there is "Compilations" view.
This view is identical to Album view, except it only shows albums that have two or more unique artists in it.
If this is a view you use often, you could put it on the category bar so it's quick/easy to get to. Perhaps a customization like: "Artist - Compilations - Songs - Videos - More …"
This of course does nothing to prevent seeing compilations when in an Artist entry.
(More is required and cannot be replaced nor moved.)
My suggestion was to default your browsing habits to using Album view instead. Instead of picking a song by an artist, you will always see a track-ordered list of songs after picking an album.
There's also the search box you can use to type in an artist/album, and pick the relevant view you want from it (scroll to the top of any view, or if you're already at the top swipe down once more to pull the top down, and the search box will be revealed).
I think this might be your problem: Windows 7 64
If you can, try to sync your phone from a 32 bit machine. Since the dawn of the iPhone Apple hasn't played nicely with 64 bit flavors of Windows. This fellow says there's a HotFix from MSFT to fix the issue.
Best Answer
This isn't through the iPod app, but it (should) delete all the music from your device.
Settings > General > Usage > Music > Edit (button, upper right hand corner) > "-" button > Delete.