Convert MTS video to OGV (OGG) format via Terminal

Because the video is buried inside a full SWF file you won't be able to convert it -- it's much like if you had a Mac app that happens to play a built-in video: you're effectively trying to convert the app to a video, two very different kinds of things.

Instead you'll need to extract the video. You can use a task-specific extrator, and since you asked for a free way to do it, I suggest the Flaex app. There are a couple of other paid extractors for the Mac including Flash Decompiler and SWF Decompiler. I don't have first-hand experience with any of the extractors so I can't specifically recommend one.

The other possible not-free way to extract the video is by opening the swf file with the full Adobe Flash Pro, or have someone do it for you. Note that Adobe provides a fully-functional 30-day trial, so at least in the short-term it's a free option as well.

I know relatively easy and free way that I will share

1. Subtitles have to be converted to SRT format. You may do that with Jubler.

2. Then I'm using Handbrake to convert to h.264 with soft subtitles. Here is the guide.

3. Alternatively you may use MKVTools to convert video without reencoding (in case it's already in h.264) to save time and after that add subtitles with Subler. You may try to import SSA subtitles (I don't have such on hand now to check if SSA will work with Subler so please report back if it will)