My new Canon Vixia HF G30 video camera has an option to encode directly to 35 Mbit/sec Mp4 at 1080p60 and it does a nice job. I primarily use it to record my daughter's high school basketball games.
I need to make highlights from the camera files AND I WISH TO RETAIN AS MUCH QUALITY AND CLARITY AS POSSIBLE. Here is the workflow that I have developed with my limited understanding of FFmpeg:
STEP 1.) Join the camera produced mp4 clips into one mp4 file representing one game. Here is a representative command that I use for this step from a Windows command prompt (DOS box):
ffmpeg -f concat -i "\\mylancomputer\f\Canon\2013.09.21_san_francisco_game2\gameclipsfromcamera.txt" -c copy "\\mylancomputer\f\Canon\2013.09.21_san_francisco_game2\joined_fullgame.mp4"
This joins the game clips into a game video with no transcoding (I believe), and so this is good (it is quick and no additional compression artifacts are introduced).
STEP 2.) Cut highlight clips from the game video. Here is a representative command that I use for this step:
ffmpeg -ss 00:00:06.00 -i "\\mylancomputer\f\Canon\2013.09.21_san_francisco_game2\joined_fullgame.mp4" -acodec copy -vcodec copy -t 00:00:20.00 "\\mylancomputer\f\Canon\2013.09.21_san_francisco_game2\1.mp4"
This produces the following 20 second clip starting at 6 seconds into the game video:
Highlight Sample (85 MB)
STEP 3.) I require a 2-second pause at the beginning of each highlight clip where a text overlay identifies my daughter. To get this I make a bitmap image from the first frame of the clip :
ffmpeg -i "\\mylancomputer\f\Canon\2013.09.21_san_francisco_game2\1.mp4" -ss 00:00:00.00 -f image2 -vframes 1 -deinterlace "\\mylancomputer\f\Canon\2013.09.21_san_francisco_game2\1freezeframe.bmp"
Here is the result.
Then loop the video for 2 seconds with CRF 0 to introduce no additional artifacts:
ffmpeg -loop 1 -r 59.97 -i "\\mylancomputer\f\Canon\2013.09.21_san_francisco_game2\1freezeframe.bmp" -c:v libx264 -crf 0 -pix_fmt yuv420p -t 2 "\\Excelhero\f\Canon\2013.09.21_san_francisco_game2\1freezeframe.mp4"
Then add the text overlay label identifying Fiona:
ffmpeg -i "\\mylancomputer\f\Canon\2013.09.21_san_francisco_game2\1freezeframe.mp4" -vf drawtext="fontfile=/Windows/Fonts/Corbelb.ttf:text='Fiona':fontsize=40:fontcolor=yellow:x=1321:y=417" -b:v 35M -minrate 35M -maxrate 35M -bufsize 35M -profile:v high -level:v 4.2 -refs 2 -pix_fmt yuv420p -bf 0 -r 59.97 "\\mylancomputer\f\Canon\2013.09.21_san_francisco_game2\1freezeframeannotated.mp4"
Which produces this file: [ ehasamples.excelhero.com/video/1freezeframeannotated.mp4 ]
(SuperUser only allowed me to include 2 live links)
Now we add a silent sound stream to the clip:
ffmpeg -f lavfi -i aevalsrc=0:0:sample_rate=48000 -i "\\mylancomputer\f\Canon\2013.09.21_san_francisco_game2\1freezeframeannotated.mp4" -shortest -c:v copy -c:a aac -b:a 255k -strict -2 "\\mylancomputer\f\Canon\2013.09.21_san_francisco_game2\1freezeframeannotatedwithsilentsound.mp4"
Resulting in this: [ ehasamples.excelhero.com/video/1freezeframeannotatedwithsilentsound.mp4 ]
(SuperUser only allowed me to include 2 live links)
The above is a 2-second video with my daughter labeled in suspended animation with a blank sound stream.
STEP 4.) The final operation is to join the 2-second video to the full clip from the camera. I can easily re-render by filter producing one output file from these two input files in this fashion:
ffmpeg -i "\\mylancomputer\f\Canon\2013.09.21_san_francisco_game2\1freezeframeannotatedwithsilentsound.mp4" -i "\\Excelhero\f\Canon\2013.09.21_san_francisco_game2\1.mp4" -filter_complex "[0:0] [0:1] [1:0] [1:1] concat=n=2:v=1:a=1 [v] [a]" -map "[v]" -map "[a]" -c:v libx264 -r 59.97 "\\mylancomputer\f\Canon\2013.09.21_san_francisco_game2\1done.mp4"
But this is not a great solution. It is slow because the original footage from the video camera is re-rendered with the introduction of new compression artifacts. I can choose to add " -crf 0 " to get a lossless re-rendering, but this bloats the size of the clip by well over 1000%, in this case bringing our sample clip to over a gigabyte. So this is not a practical solution.
So I want to use the concat demuxer instead of the filter:
ffmpeg -f concat -i "\\mylancomputer\f\Canon\2013.09.21_san_francisco_game2\clipfiles.txt" -c copy "\\mylancomputer\f\Canon\2013.09.21_san_francisco_game2\1doneBAD.mp4"
With this approach the operation is quick as there is no re-encoding. BUT, the resulting file does not play properly. Here it is:
[ ehasamples.excelhero.com/video/1doneBAD.mp4 ]
(SuperUser only allowed me to include 2 live links)
The 2-second image displays the entire 22 seconds. Although the video does not play properly, the sound seems to work just fine. So my guess is that the two files (the clip from the camera and the 2-second label clip) are simply too far apart on their relative encoding. I have attempted to get them as close to identical as I can in terms of encoding by using MediaInfo and by using FFProbe, and the above series of commands are my best, but apparently are not sufficient.
So the question I have is simply this: is it possible with FFmpeg to make my labeled clip enough like the encoded footage from the camera in order for the Concat demux to succeed? If so, how?
This entire workflow is driven by batch file and is really simple on my part and so I like it and hope to use this methodology to allow me to quickly edit highlights from games going forward.
Conversely, is there a better (quicker, no transcoding) workflow using FFmpeg?
Thank you kindly for your assistance.
Best Answer
You might want to use a faster lossless codec that will take more temporary space, like ffvhuff or something, if you have fast disks. Otherwise, use
-preset ultrafast
with-crf 0
. (same as-qp 0
, enables lossless mode.)x264
slow
only compresses a tiny better thansuperfast
in lossless mode, BTW. Maybe if you had animation with some bit-identical blocks more than 1 frame back, so multiple ref frames could help, then higher settings would do more. My findings, for a 1h:22m NTSC DV deinterlace (720x480p60), were 27.9GB (superfast
) vs 27.0GB (slow
), or 55.0GB for ffvhuff, or 37.6GB for ffv1 (default settings, somewhat smaller with even slower ffv1 compress/decompress settings) all in yuv420). CABAC encode/decode at that bitrate takes a ton of CPU; I should have used ultrafast instead of superfast, or just-x264-params cabac=0
.TL;DR: use libx264
-preset ultrafast
or ffvhuff for intermediate files.ffv1 or h.264 with CABAC aren't worth the encode/decode CPU time when file size doesn't really matter. And huffyuv doesn't do yuv 4:2:0, you need ffvhuff for that. Lagarith is GPL, but decoder-only in ffmpeg, and the encoder isn't ported to anything but windows. (speed vs. compression tradeoff probably not too impressive vs. x264 either, except maybe for noiseless sources like animation, where the prediction / run-length stuff before the entropy coder will do well.)
Also, pretty sure you could use
-vf drawtext
during the bmp -> 2 sec video step. And why are you using hard-CBR (bitrate 35M, max-bitrate 35M) for that? Why not just lossless for that step, too?Not usually useful to specific -profile for x264. It sets the profile flags in the output to the lowest it can, given what it puts into the bitstream. (i.e. it will set profile to high if
8x8dct=1
, and work out what level based on rez, bitrate, and ref frames.) edit:-profile
also forces x264 to lower ref frame count or other settings needed to keep to the limits for the given profile. Still, rare to need to use it, unless targeting some HW decoder.What IS useful on the lossy final encode is using
-preset slower
or something to have x264 use more CPU time, but get more quality for the same bitrate. This pretty much replaces having to tweak all the knobs, like ref frames, trellis, adaptive b frames, motion search, etc.To try to answer the actual question, the things that should matter for being able to concat different h.264 streams (from your camera's encoder and from BMP -> x264) should just be resolution, colorspace (yuv420 vs yuv444 or something), and probably some h.264 stream parameters, like interlaced or not.
If you're planning to keep the entire 35Mbit/s thing around, and don't want to get it to a reasonable file size that you could send over the internet, then you'd want to match whatever your camera's hardware encoder is doing. Or, you could run the whole thing through x264, which will take time, but with
-preset veryslow
you can probably do 10Mbit/s, or maybe even 5, with little noticeable quality loss. Try-crf 18
, that's generally pretty much transparent. (Yeah you can still easily notice the difference if you pause and flip back and forth between x264 and source frames).If should be possible to do the whole process in a single invocation of ffmpeg. The filtergraph that ultimately ends in the concat filter can have chains that generate 2 sec of repeated still image + silence interspersed with chains that are just inputfile->output.
(Or if you really want to not xcode the camera output, and can figure out what's different between your camera's output and x264: one invocation of ffmpeg per still image, and then one final concat.)
Hardware encoders in phones and cameras are usually pretty bad. The only way they look good is by throwing a lot of bitrate at the problem. x264 can usually do a lot better, esp. with -preset slow, slower, or veryslow. Obviously there will be another generation of loss, but you can typically cut the video bitrate down to 2Mbit/s for 1080p@24fps hollywood movies with very sharp detail. Noisy handheld sources with high motion the whole time will take more bits, but
-crf 18
is constant quality, so I'd recommend that as something that will be good enough for viewing even up close on a good monitory. I'd probably still save the original sources, though, since storage only gets cheaper, and you can't ever recover the original quality from the x264 output. Still, I'd just keep them to have them. If you're giving this file to anyone, or copying it over the internet, x264-preset slow
does good stuff. Even the default-preset medium
is good. If you don't set a target bitrate or quality, the default is I think-crf 23
.I'm not doing much to answer how to get x264 to make output you can concat with your camera's non-xcoded bitstream, since that's not something I've had to do, and it doesn't really interest me to find out, sorry. Mostly answering so anyone who wanted to follow your starting point won't be led to -crf 0 or something silly like that. :P