Ubuntu – Fix MKV to MP4 Transcoding Script Issues

shell-scripttranscodeUbuntuvideo

Ok, this one has got me scratching my head – specially as I'm a scripting nOOb, and my *nix expeirence is basic to say the least.

Summary

Need to identify why the shell script for converting filers from one media container to another is generating errors and issues.

Script

I have a shell script provided by a friend. The script is used on a Ubuntu 10.10 machine for converting MKV video files to MP4 (MPEG-4). The script in question is as follows (forgive the lack of commenting):

#!/usr/bin/env bash

if [ -f $1 ] ; then
    filename=$(basename $1)
    extension=${filename##*.}
    name=${filename%.*}
    fname=$1
    video=`mediainfo --Inform=Video\;%ID% "${fname}"`
    audio=`mediainfo --Inform=Audio\;%ID% "${fname}"`
    fps=`mediainfo --Inform=Video\;%FrameRate% "${fname}"`
    `mkvextract tracks ${fname} 1:${name}.h264 2:${name}.ac3`
    `a52dec ${name}.ac3 -o wavdolby > ${name}.wav`
    `faac ${name}.wav -o ${name}.m4a`
    `MP4Box -add ${name}.m4a -add ${name}.h264 -fps $fps ${name}.mp4`
    `rm ${name}.m4a ${name}.ac3 ${name}.h264 ${name}.wav`
fi

Problem

skip errors when running a52dec

The first error happens on only selected MKV files, where when it goes to execute a52dec as part of the audio conversion process.

What happens is a skip error keeps appearing over and over in the console/terminal window. The a52dec process has created the expected file name (filename.wav), but keeps generating these skip messages and proceeds no further (despite having left for half an hour at one point just to see if it would proceed past the problem)

From looking at the Mediainfo output on a couple of different files, it looks like things are coming unstuck where the file includes 5.1 channel audio. I of course have zero idea how I would overcome this issue.

IsoMedia: command not found

This happens all the time when the script has finished running, which to this n00b suggests that it can't find the application in question.

However, I cannot find this package, or if part of a larger package which one it is to install.

More confusing, this file isn't called at any point during the script, and only is called after the rm command to clean up is run.

Extra info

Example of Mediainfo output for a MKV file that transcodes without issues

General
Unique ID                        : 233323168834975742075458986504469215458 (0xAF886862D1B0BB1B9427E04C90A1F8E2)
Complete name                    : \\192.168.2.5\video\sorted\CSI NY\CSI.New.York.S07E10.720p.HDTV.X264-DIMENSION.mkv
Format                           : Matroska
File size                        : 1.09 GiB
Duration                         : 41mn 30s
Overall bit rate                 : 3 768 Kbps
Encoded date                     : UTC 2010-12-03 20:40:51
Writing application              : mkvmerge v3.1.0 ('Happy up here') built on Jan 19 2010 12:09:24
Writing library                  : libebml v0.7.9 + libmatroska v0.8.1

Video
ID                               : 1
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : High@L4.1
Format settings, CABAC           : Yes
Format settings, ReFrames        : 8 frames
Format settings, GOP             : M=6, N=12
Codec ID                         : V_MPEG4/ISO/AVC
Duration                         : 41mn 30s
Bit rate                         : 3 381 Kbps
Width                            : 1 280 pixels
Height                           : 720 pixels
Display aspect ratio             : 16:9
Frame rate                       : 23.976 fps
Color space                      : YUV
Chroma subsampling               : 4:2:0
Bit depth                        : 8 bits
Scan type                        : Progressive
Bits/(Pixel*Frame)               : 0.153
Stream size                      : 982 MiB (88%)
Writing library                  : x264 core 110 r1804 e89c4cf
Encoding settings                : cabac=1 / ref=8 / deblock=1:0:0 / analyse=0x3:0x113 / me=umh / subme=9 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-2 / threads=12 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / constrained_intra=0 / bframes=5 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=2pass / mbtree=1 / bitrate=3381 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=1:1.00
Language                         : English

Audio
ID                               : 2
Format                           : AC-3
Format/Info                      : Audio Coding 3
Mode extension                   : CM (complete main)
Codec ID                         : A_AC3
Duration                         : 41mn 30s
Bit rate mode                    : Constant
Bit rate                         : 384 Kbps
Channel(s)                       : 2 channels
Channel positions                : Front: L R
Sampling rate                    : 48.0 KHz
Bit depth                        : 16 bits
Compression mode                 : Lossy
Stream size                      : 114 MiB (10%)

Example of Mediainfo output for a MKV file that causes a52dec issues

General
Unique ID                        : 173353892635048029459501626055714892286 (0x826ABECAAEC6D2638DD0EC376D6369FE)
Complete name                    : \\192.168.2.5\video\sorted\Conan\conan.2010.11.25.jim.parsons.720p.hdtv.x264-bff.mkv
Format                           : Matroska
File size                        : 1.09 GiB
Duration                         : 42mn 1s
Overall bit rate                 : 3 720 Kbps
Encoded date                     : UTC 2010-11-26 05:45:43
Writing application              : mkvmerge v3.1.0 ('Happy up here') built on Jan 19 2010 12:09:24
Writing library                  : libebml v0.7.9 + libmatroska v0.8.1

Video
ID                               : 2
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : High@L3.1
Format settings, CABAC           : Yes
Format settings, ReFrames        : 3 frames
Codec ID                         : V_MPEG4/ISO/AVC
Duration                         : 42mn 1s
Bit rate                         : 3 272 Kbps
Width                            : 1 280 pixels
Height                           : 720 pixels
Display aspect ratio             : 16:9
Frame rate                       : 29.970 fps
Color space                      : YUV
Chroma subsampling               : 4:2:0
Bit depth                        : 8 bits
Scan type                        : Progressive
Bits/(Pixel*Frame)               : 0.118
Stream size                      : 961 MiB (86%)
Writing library                  : x264 core 85 r1442tw
Encoding settings                : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=6 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=30 / sliced_threads=0 / nr=0 / decimate=1 / mbaff=0 / constrained_intra=0 / bframes=6 / b_pyramid=1 / b_adapt=1 / b_bias=0 / direct=1 / wpredb=1 / wpredp=0 / keyint=240 / keyint_min=24 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=2pass / mbtree=1 / bitrate=3272 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=1:1.00
Language                         : English

Audio
ID                               : 1
Format                           : AC-3
Format/Info                      : Audio Coding 3
Mode extension                   : CM (complete main)
Codec ID                         : A_AC3
Duration                         : 42mn 1s
Bit rate mode                    : Constant
Bit rate                         : 448 Kbps
Channel(s)                       : 6 channels
Channel positions                : Front: L C R, Side: L R, LFE
Sampling rate                    : 48.0 KHz
Bit depth                        : 16 bits
Compression mode                 : Lossy
Stream size                      : 135 MiB (12%)

Best Answer

MKV to MP4 Remuxing Script

This is based off thewinchester's script posted in his answer here. I started off with his and it eventually morphed into a rewrite of most of it, but the concept is the same.

The basic function is to take an MKV file containing h.264 video and some audio and convert it to an MP4/M4V file.

My goal is to make the resulting files compatible with the iPad and AppleTV, but it should work with anything that can handle MP4/M4V files. Video is not transcoded, so quality remains the same, audio is converted if necessary, but higher quality tracks are kept if possible.

Changes

A partial list of the changes I've made:

  • Handling multiple files or directories passed as arguments
  • Uses (and requires) the higher quality NeroAAC to encode AAC audio from AC3 or DTS
  • Handling multiple audio tracks in accordance with the AppleTV standard to allow for proper playback and surround sound on AppleTV:
    • 160kB/s stereo AAC
    • (Optional) AC-3 (typically surround sound) as the second track, disabled
  • Converting DTS to AC-3
  • Handling multiple tracks regardless of order (i.e. AAC, Video, AC3)
  • Various coding changes to make it easier to modify later

Requirements

The script requires the aforementioned NeroAAC, as well as Aften (AC3 encode) and libdca/dcadec (DTS decode). As with the original script, ffmpeg is also required.

Known Issues

The script is fairly bulletproof - if it runs into something it can't deal with, it's typically ignored or quits. The original files are left intact, so there's little to no chance of data loss.

I've used it on a lot of my media library and many of the changes have been to fix bugs I've run into.

That said, there are a few things to be aware of, and room for improvement:

  • Handling multiple audio tracks of the same type - 1 AC3 and 1 AAC is fine, but 2 AAC tracks won't work. This shouldn't come up often, but keep it in mind.
  • No support for non-audio/video tracks. Subtitles, chapters and artwork are all ignored.
  • No detection of surround sound AAC. The script copies any AAC track, it won't down mix to stereo. Technically for iPad and AppleTV support, the main audio track should be stereo AAC, but I think in most cases an iPad or AppleTV can handle it gracefully.

Code and use

I'm posting it here hoping that someone will find it useful - please feel free to use and modify as you see fit for non-commercial uses. If you come up with an interesting modification it might be nice if you shared it here or elsewhere.

#!/bin/bash

#Close stdin - avoid accidental keypresses causing problems
exec 0>&-

# Find MKV files
for file in "$@";
do
  find "$file" -type f -not -name ".*" | grep .mkv$ | while read file
  do
    fileProper=$(readlink -f "$file") # full path of file
    pathNoExt=${fileProper%.*} # full path minus extension

    #Check if M4V already exists
    if [ -f "$pathNoExt".m4v ]; then
      echo "M4V already exists, stopping"
    else
      # Get number of tracks
      numberOfTracks=`mkvmerge -i "$fileProper" | grep "Track ID" | wc -l`
      echo "Found $numberOfTracks Tracks"

      # Set base extraction command
      extractCmd+=(mkvextract tracks "$fileProper")

      # Determine type of tracks
      for (( i=1; i<=$numberOfTracks; i++ ))
      do
         trackType=`mkvmerge -i "$fileProper" | grep "Track ID $i" | sed -e 's/^.*: //'`
         if [[ "$trackType" == *video* ]]; then
            echo "Track $i is Video"
            extractCmd+=( $i:"$pathNoExt".264)
            fps=`mkvinfo "$fileProper" | grep duration | sed -e 's/.*(//' -e 's/f.*//' | sed -n ${i}p`
         elif [[ "$trackType" == "audio (A_AAC)" ]]; then
            echo "Track $i is AAC"
            extractCmd+=( $i:"$pathNoExt".aac)
         elif [[ "$trackType" == "audio (A_AC3)" ]]; then
            echo "Track $i is AC3"
            extractCmd+=( $i:"$pathNoExt".ac3)
         elif [[ "$trackType" == "audio (A_DTS)" ]]; then
            echo "Track $i is DTS"
            extractCmd+=( $i:"$pathNoExt".dts)
         fi
         # Insert cases for handling other audio and non-AV tracks here
       done

        "${extractCmd[@]}" # Extract Tracks

        # Check files and encode audio if neccessary
        if [ -f "$pathNoExt".264 ]; then
            # Video file exists
            mp4BoxCmd+=(MP4Box -new "$pathNoExt".m4v -add "$pathNoExt".264 -fps $fps)
            if [ -f "$pathNoExt".aac ]; then
                # AAC exists
                mp4BoxCmd+=( -add "$pathNoExt".aac)
                if [ -f "$pathNoExt".ac3 ]; then
                    mp4BoxCmd+=( -add "$pathNoExt".ac3:disable)
                elif [ -f "$pathNoExt".dts ]; then
                    # Encode DTS to AC3
                    dcadec -o wavall "$pathNoExt".dts | aften -v 0 - "$pathNoExt".ac3
                    mp4BoxCmd+=( -add "$pathNoExt".ac3:disable)
                fi
            else # Encode AAC from AC3 or DTS
                if [ -f "$pathNoExt".ac3 ]; then
                    ffmpeg -i "$pathNoExt".ac3 -acodec pcm_s16le -ac 2 -f wav - | neroAacEnc -lc -br 160000 -ignorelength -if - -of "$pathNoExt".aac
                    mp4BoxCmd+=( -add "$pathNoExt".aac -add "$pathNoExt".ac3:disable)
                elif [ -f "$pathNoExt".dts ]; then
                  ffmpeg -i "$pathNoExt".dts -acodec pcm_s16le -ac 2 -f wav - | neroAacEnc -lc -br 160000 -ignorelength -if - -of "$pathNoExt".aac
                    # Encode DTS to AC3
                    dcadec -o wavall "$pathNoExt".dts | aften -v 0 - "$pathNoExt".ac3
                    mp4BoxCmd+=( -add "$pathNoExt".aac -add "$pathNoExt".ac3:disable)
                else
                    echo "Warning: no audio file found"
                fi
            fi
            # Create m4v
            "${mp4BoxCmd[@]}"
        else
            echo "Error: no video file found"
        fi
  #remove temporary track files
  rm -f "$pathNoExt".aac "$pathNoExt".dts "$pathNoExt".ac3 "$pathNoExt".264 "$pathNoExt".wav
  fi
 done
done
Related Question