MP4Tools works for this. It does pass-thru of h.264 and audio streams, or you can convert as you see fit. It's free, but $5 to remove the nag screen that pops up any time you open a file, and to get batch operations. I haven't tested it with chapters and subs, but I think it can deal with them fine.
One word of caution: if you have files with AC3 audio that you want to leave in as a 2nd channel, it has an option to do pass-thru of that, and add a stereo AAC track, but it doesn't re-order them, which can cause issues with QuickTime X/iPad/AppleTV.
MP4Tools doesn't work for anything below 10.6, so MKVtools is a good alternative for 10.5. Same developer and basic functionality, it should work for your purposes fine.
The other option is doing it manually using command line tools, mkvtoolnix to extract the streams from the mkv, and MP4Box to repackage in an MP4. If you need to convert audio to AAC, ffmpeg is the a good starting point, but there are other options. Unless you need a degree of control that MP4Tools doesn't provide, I'd say stay away from the command line stuff. It's useful, but more hassle than it's worth if MP4Tools does what you need.
Another good tool is Subler, which is a nice tagging program. Unfortunately lacking a batch mode, but it's otherwise very good - much faster at writing the tags than other programs I've tried. And as long as the files have season & episode numbers, the tag auto-lookup works great.
Indeed, you should use QuickTime rather than iMovie: iMovie will import (and therefore usually encode) the movie, but will also force you to export, as you have noted.
The main interest of using QuickTime is to use its editing features.
Assuming you're using QuickTime X (Snow Leopard / Lion), open your movie file, and hit ⌘T (or Edit > Trim). The timeline will morph into a yellow double-ended scrollbar so that you can trim the sequence.
Adjust it to fit the sequence you want, and validate the trimming with a click on the Trim yellow button. You can then simply save the file (⌘S), without the need for an export that will necessarily reduce quality.
Update seeing that you need to remove parts of the video.
You'll need the more advanced editing features of QuickTime 7. To install it on Snow Leopard or Lion, refer to Installing QuickTime Player 7 on Mac OS X v10.6 or Later on Apple's KB.
You'll then be able to open your video with QT7, select parts of your video with I (set input point) and O (set output point), and delete the part with a stroke of ⌫. Then, simply save the file, as said earlier.
You might need a license key for QT7, though (I don't remember if the latest versions of the OS need them or activate the Pro features for free).
Best Answer
You could do that with ffmpeg:
This will just change containers, but obviously will only work with a codec that ffmpeg supports.
Edit: if you absolutely need a GUI, you could use Automator to create a Drag & Drop target, like this: