MacOS – Virtual video capture device

I've found that "New Screen Recording" functionality in QuickTime Player combined with iMovie editing capabilities is enough for my needs. Maybe you can give it a try? Best of all, it's already installed on your Mac. Unfortunately, I can't compare it with Camtasia, as I haven't used it.

Yes, I would recommend to convert it to a format that iMove can import. MKV is just a container format so you probably already have an iMovie-compatible inside that package. Under Snow Leopard just install Perian and open it in QuickTime X and then go to File->Save as ... and choose a suitable format from the dropdown menu.

If you need more control over your conversion, you could also convert it to an iMovie-compatible format using any of the video conversion tools available for the Mac. For a long time VisualHub was the king of video format conversion, unfortunately techspansion the company behind it, closed its virtual doors. There are two main open-source options derived from VisualHub, ReduxZero and TranscoderRedux they both should do the trick. Another option is Subler which will also convert to and from MKV video containers to an iMovie-compatible format.