Linux – How to create a video from images using FFmpeg

There are several ways to get an "uncompressed" AVI out of ffmpeg, but I suspect you actually mean "lossless." Both terms have a fair bit of wiggle room in their definitions, as you will see.

I'm going to anchor this discussion with the 720p HD version of Big Buck Bunny, since it's a freely-available video we can all test with and get results we can compare. The raw data rate of 1280×720p video at 24 fps is very nearly equal to that of your stated 1024×768 at 29.97 fps goal, so my results should be a pretty good guide to the data rates you can expect on your footage.

Automatic Listing of Available Options

The following POSIX command¹ gives you a list that mostly² matches what we discuss below:

$ ffmpeg -codecs 2> /dev/null | grep '^..EV..S ' | grep -vE 'bitmap|image'

You might want to run that command on your own machine to see what your build of FFmpeg will support. FFmpeg is rarely built with every possible encoder enabled.

Now let's discuss those options.

Fully Uncompressed

If your definition of "uncompressed" is the form the video is in right before it's turned to photons by a digital display, the closest I see in the ffmpeg -codecs list are -c:v r210, r10k, v410, v308, ayuv and v408. These are all substantially the same thing, differing only in color depth, color space, and alpha channel support.

• R210 and R10K are 4:4:4 RGB at 10 bits per component (bpc), so they both require about 708 Mbit/s for 720p in my testing. (That's about ⅓ TB per hour, friends!)

  These codecs both pack the 3×10 bit color components per pixel into a 32-bit value for ease of manipulation by computers, which like power-of-2 sizes. The only difference between these codecs is which end of the 32-bit word the two unused bits are on. This trivial difference is doubtless because they come from competing companies, Blackmagic Design and AJA Video Systems, respectively.

  Although these are trivial codecs, you will probably have to download the Blackmagic and/or AJA codecs to play files using them on your computer. Both companies will let you download their codecs without having bought their hardware first, since they know you may be dealing with files produced by customers who do have some of their hardware.

• V410 is essentially just the YUV version of R210/R10K; their data rates are identical. This codec may nevertheless encode faster, because ffmpeg is more likely to have an accelerated color space conversion path between your input frames' color space and this color space.

  I cannot recommend this codec, however, since I was unable to get the resulting file to play in any software I tried, even with the AJA and Blackmagic codecs installed.

• V308 is the 8 bpc variant of V410, so it comes to 518 Mbit/s in my testing. As with V410, I was unable to get these files to play back in normal video player software.

• AYUV and V408 are essentially the same thing as V308, except that they include an alpha channel, whether it is needed or not! If your video doesn't use transparency, this means you pay the size penalty of the 10 bpc R210/R10K codecs above without getting the benefit of the deeper color space.

  AYUV does have one virtue: it is a "native" codec in Windows Media, so it doesn't require special software to play.

  V408 is supposed to be native to QuickTime in the same way, but the V408 file wouldn't play in either QuickTime 7 or 10 on my Mac.

So, putting all this together, if your PNGs are named frame0001.png and so forth:

$ ffmpeg -i frame%04d.png -c:v r10k output.mov
  ...or...                -c:v r210 output.mov
  ...or...                -c:v v410 output.mov
  ...or...                -c:v v408 output.mov
  ...or...                -c:v v308 output.mov
  ...or...                -c:v ayuv output.avi

Notice that I have specified AVI in the case of AYUV, since it's pretty much a Windows-only codec. The others may work in either QuickTime or AVI, depending on which codecs are on your machine. If one container format doesn't work, try the other.

The above commands — and those below, too — assume your input frames are already the same size as you want for your output video. If not, add something like -s 1280x720 to the command, before the output file name.

Compressed RGB, But Also Lossless

If, as I suspect, you actually mean "lossless" instead of "uncompressed," a much better choice than any of the above is Apple QuickTime Animation, via -c:v qtrle

I know you said you wanted an AVI, but the fact is that you're probably going to have to install a codec on a Windows machine to read any of the AVI-based file formats mentioned here, whereas with QuickTime there's a chance the video app of your choice already knows how to open a QuickTime Animation file. (The AYUV codec above is the lone exception I'm aware of, but its data rate is awfully high, just to get the benefit of AVI.)

ffmpeg will stuff qtrle into an AVI container for you, but the result may not be very widely compatible. In my testing, QuickTime Player will gripe a bit about such a file, but it does then play it. Oddly, though, VLC won't play it, even though it's based in part on ffmpeg. I'd stick to QT containers for this codec.

The QuickTime Animation codec uses a trivial RLE scheme, so for simple animations, it should do about as well as Huffyuv below. The more colors in each frame, the more it will approach the bit rate of the fully uncompressed options above. In my testing with Big Buck Bunny, I was able to get ffmpeg to give me a 165 Mbit/s file in RGB 4:4:4 mode, via -pix_fmt rgb24.

Although this format is compressed, it will give identical output pixel values to your PNG input files, for the same reason that PNG's lossless compression doesn't affect pixel values.

The ffmpeg QuickTime Animation implementation also supports -pix_fmt argb, which gets you 4:4:4:4 RGB, meaning it has an alpha channel. In a very rough sort of way, it is the QuickTime equivalent to -c:v ayuv, mentioned above. Because of the lossless compression, though, it comes to only 214 Mbit/s, less than ⅓ the data rate of AYUV with zero loss in quality or features.

There are variants of QuickTime Animation with fewer than 24 bits per pixel, but they're best used for progressively simpler animation styles. ffmpeg appears to support only one of the other formats defined by the spec, -pix_fmt rgb555be, meaning 15 bpp big-endian RGB. It's tolerable for some video, and is fine for most screencast captures and simple animations. If you can accept the color space decimation, you may find its 122 Mbit/s data rate appealing.

Putting all this together:

$ ffmpeg -i frame%04d.png -c:v qtrle -pix_fmt rgb24    output.mov
  ...or...                           -pix_fmt argb     output.mov
  ...or...                           -pix_fmt rgb555be output.mov

Effectively Lossless: The YUV Trick

Now, the thing about RGB and 4:4:4 YUV is that these encodings are very easy for computers to process, but they ignore a fact about human vision, which is that our eyes are more sensitive to black and white differences than color differences.

Video storage and delivery systems therefore almost always use fewer bits per pixel for color information than for luminance information. This is called chroma subsampling. The most common schemes are 4:2:0 and 4:2:2.

The data rate of 4:2:0 YUV is just 50% higher than for black and white (Y only) uncompressed video and ½ the data rate of 4:4:4 RGB or YUV.

4:2:2 is a kind of halfway point between 4:2:0 and 4:4:4. It is twice the data rate of Y-only video and ⅔ the data rate of 4:4:4.

You also sometimes see 4:1:1, as in the old DV camera standard. 4:1:1 has the same uncompressed data rate as 4:2:0, but the color information is arranged differently.

The point of all this is that if you're starting with a 4:2:0 H.264 file, re-encoding it to 4:4:4 uncompressed RGB buys you absolutely nothing over 4:2:0 losslessly compressed YUV. This is true even if you know your workflow is otherwise 4:4:4 RGB, since it's a trivial conversion; video hardware and software do such conversions on the fly routinely.

You really only need 4:4:4 when you're pixel peeping or you're doing pixel-level color changes on the video, and you need to preserve exact pixel values. Visual effects (VFX) work is easier to do with a 4:4:4 pixel format, for example, so high-end VFX houses are often willing to tolerate the higher data rates it requires.

Effectively Lossless: Codec Choices

Once you open yourself up to YUV codecs with color decimation, your options open up, too. ffmpeg has many effectively lossless codecs.

Huffyuv

The most widely compatible option is Huffyuv. You get this via -c:v huffyuv.

The original Windows Huffyuv codec only supports two pixel formats: RGB24 and YUV 4:2:2. (Actually, it supports two flavors of YUV 4:2:2, differing only in the order of the bytes on disk.)

Older versions of the FFmpeg Huffyuv codec did not include RGB24 support, so if you try it and FFmpeg tells you it is going to use the yuv422p pixel format, you need to upgrade.

FFmpeg also has a Huffyuv variant codec called FFVHuff, which supports YUV 4:2:0. This variant isn't compatible with the Windows DirectShow Huffyuv codec, but it should open in any software based on libavcodec, such as VLC.

• RGB24 — RGB 4:4:4 is essentially the same thing as QuickTime Animation's RGB24 color space option. The two codecs will differ somewhat in compression for a given file, but they will usually be close.

  It is also essentially the same thing as the YUV 4:4:4 mode used by the V308 option above. The color space difference makes no practical difference, since the color space conversion is easy to do in real time.

  Due to Huffyuv's lossless compression, I was able to get a test video to compress to about 251 Mbit/s in RGB24 mode, with identical visual quality to what you'd get from V308 or AYUV. If AVI is an absolute must for you, installing the Huffyuv codec is probably less painful than paying the 3× data rate cost of AYUV.

• YUV 4:2:2 — This mode is far more practical for video than RGB24, which is doubtless why the ffmpeg developers chose to implement it first. As you'd expect from the theoretical ⅔ reduction discussed above, my test file encoded to 173 Mbit/s. That's pretty much exactly ⅔, if you take into account the fact that the audio track was unchanged between these two tests.

• YUV 4:2:0 — This option decimates the color information more than 4:2:2 does, dropping the data rate to 133 Mbit/s in my testing.

Putting all this together:

$ ffmpeg -i frame%04d.png -c:v huffyuv -pix_fmt rgb24   output.avi
  ...or...                             -pix_fmt yuv422p output.avi
  ...or...                -c:v ffvhuff -pix_fmt yuv420p output.avi

Although the ffvhuff codec defaults to 4:2:0 as I write this, and indeed only supports that pixel format in the release version I'm using, this is changing, so you should include the flag in case this default changes.

Ut Video

A more recent option in the same spirit as Huffyuv and FFVHuff is Ut Video. Like Huffyuv, there is a Windows video codec which means any Windows program that can play a movie can play videos using this codec with the codec installed. Unlike Huffyuv, there is also a Mac video codec as well, so you aren't restricted to software based on FFmpeg or libavcodec to read these files on Macs.

This codec is very flexible in terms of color spaces, so I will just give a few examples of common color spaces:

• 4:4:4 RGB via -f avi -c:v utvideo -pix_fmt rgb24 gives 178 Mbit/sec output

• 4:4:4 YUV via -f avi -c:v utvideo -pix_fmt yuv444p gives 153 Mbit/sec output

• 4:2:2 YUV via -f avi -c:v utvideo -pix_fmt yuv422p gives 123 Mbit/sec output

• 4:2:0 YUV via -f avi -c:v utvideo -pix_fmt yuv420p gives 100 Mbit/sec output

I suspect 4:4:4 YUV does better than 4:4:4 RGB in this test despite these two being technically equivalent because the source video is 4:2:0 YUV, so arranging the data in YUV format allows for better lossless compression by grouping the partially-redundant U and V channels together in the file.

FFV1

Another interesting option in this space is FFmpeg's own FFV1 codec. This is mostly used as an archival codec rather than a playback or editing codec, but since so much software is either based on the libavcodec library underpinning FFmpeg or can be lashed to libavcodec via tools like ffdshow, it may be useful to you anyway.

By default, ffmpeg will preserve your input files' color space when using a flexible codec like FFV1, so that if you feed it one of the official Big Buck Bunny MP4 files, which use 4:2:0 YUV, that's what you'll get out unless you give a -pix_fmt flag to ffmpeg. This results in a 63 Mbit/s output file.

If you force FFV1 to use a 4:4:4 YUV color space with -pix_fmt yuv444p, the file size only goes up to 86 Mbit/sec, but it's buying us nothing in this case since we're encoding from a 4:2:0 original. However, if you feed in a set of PNGs instead, as in the original question, the output file is likely to use the bgra or bgr0 color space, which are just rearrangements of the argb and rgb24 color spaces brought up above.

Lossless H.264

Another interesting alternative is Lossless H.264. This is pretty much an x264-only thing as of this writing, but those using FFmpeg on the encoding side are likely to be using other software that includes libx264 on the decoding side, too, such as VLC.

The simplest way to get such a file is:

$ ffmpeg -i frame%04d.png -c:v libx264 -qp 0 -f mp4 output.mp4

The -qp 0 flag is the key: higher values give lossy compression. (You can also give -crf 0 to get the same effect.)

As with FFV1, ffmpeg will try to guess the best output color space given the input color space, so for comparison to the results above, I ran multiple encode passes on the Big Buck Bunny source file with different color spaces:

• yuv444p: This is what ffmpeg chooses when you give it an RGB PNG stream, as in the original question, and results in a 44 Mbit/sec file with our test file

• yuv422p: This is similar to the default color space for Huffyuv, but we get a 34 Mbit/sec file in this case, quite a savings!

• yuv420p: This is the default for the Big Buck Bunny official MP4s I'm testing with, and results in a 29 Mbit/sec file.

Beware that you're trading a lot of compatibility to get such small file sizes. That's why I didn't even bother trying to stuff this into an AVI or MOV container. It's so closely tied to x264 that you might as well use its standard container type (MP4) instead. You could also use something like Matroska for this.

You can trade off some of that bit rate for a faster encoding time by adding -preset ultrafast. That increased my test file's bit rate to 44 Mbit/s in YUV 4:2:2 mode, but encoded much faster, as promised. The docs claim that -preset veryslow is also worthwhile, but it resulted in a much longer encode time while only saving a tiny bit of space; I can't recommend it.

Others

ffmpeg also supports decode-only mode for Lagarith and encode-only mode for Lossless JPEG. These two codecs are actually somewhat similar, and should give files a bit smaller than Huffyuv with the same quality. If the ffmpeg developers ever add Lagarith encoding, it would be a strong alternative to Huffyuv. I cannot recommend Lossless JPEG, though, since it doesn't enjoy wide decoding support.

Perceptually Lossless: Or, You Can Probably Get Away With Some Loss

Then there are the codecs that are perceptually lossless. Unless you're pixel peeping, you almost certainly cannot tell that these give different visual results than the previous two groups. By giving up on the idea of absolutely zero change between the video capture sensor and the display device, you buy considerable savings:

• Apple ProRes: -c:v prores or -c:v prores_ks — ProRes is a profile-based codec, meaning that there are several variants, each with a different quality vs. space tradeoff:

  • ProRes 4444 encodes our test video using only 114 Mbit/s, yet is VFX quality. There are currently three different prores* codecs in FFmpeg, but only prores_ks supports ProRes 4444, as I write this, via the -profile:v 4444 option.

    If you're wondering why you'd bother going with ProRes 4444 over Lossless H.264, it comes down to compatibility, decoding speed, predictability, and the alpha channel.

  • ProRes 422 saves even more space, needing only 84 Mbit/s to give a result you can tell from ProRes 4444 only by pixel-peeping. Unless you need the alpha channel offered by ProRes 4444, there's probably no reason to insist on ProRes 4444.

    ProRes 422 is a closer competitor to the Lossless H.264 option above, since neither supports an alpha channel. You'll want to tolerate the higher bit rate of ProRes if you need compatibility with Apple pro video apps, a lower CPU overhead for encoding and decoding, or predictable bit rates. The latter is important with hardware encoders, for example. On the other hand, if you can cope with the compatibility problems of Lossless H.264, you get the option of using the 4:2:0 color space, which isn't an option from any ProRes profile.

    All three of the ProRes encoders in FFmpeg support the ProRes 422 profile, so the simplest option is to use -c:v prores, rather than -c:v prores_ks -profile hq, or depend on the auto-profile feature of prores_ks to do the right thing.

  There are even more parsimonious ProRes profiles, but they're meant for either SD video or as proxies for full-res files.

  The main problem with ProRes is that it doesn't yet have wide support outside the Apple and pro video worlds.

• Avid's DNxHD is a similar codec to ProRes, but isn't tied to the Apple pro video world. Avid offers freely-downloadable codecs for both Windows and Macintosh, and FFmpeg now supports it via -c:v dnxhd.

  Because DNxHD is a profile-based codec like ProRes, you choose the profile from the predefined set, and that tells the codec which frame size, frame rate, and bit rate to use. For the Big Buck Bunny test file, the -b:v 60M profile is most appropriate. Unsurprisingly, the resulting file is about 59 Mbit/s.

• Low-loss MJPEG: -vcodec mjpeg -qscale:v 1 — This is far more common than lossless JPEG. In fact, this was once a quite common video editing codec, and it's still frequently used by things like networked streaming video cameras. All that history means it is easy to find software that supports it.

  Expect pretty wide variability in data rates from this codec. A test I just made here gave me 25 Mbit/s for 720p video. That's high enough compression to make me nervous about lossiness, but it looked pretty good to me. Based on data rate alone, I'd say it's probably on par quality-wise to 12 Mbit/s MPEG-2 or 6 Mbit/s H.264.

Putting all this together:

$ ffmpeg -i frame%04d.png -c:v prores_ks -profile:v 4444 output.mov
  ...or...                -c:v prores_ks -profile:v hq   output.mov
  ...or...                -c:v prores                    output.mov
  ...or...                -c:v dnxhd -b:v 60M            output.mov
  ...or...                -c:v mjpeg -qscale:v 1         output.avi

Bottom line on these methods is that unless you're doing something very demanding, "good enough" really is good enough.

---

Footnotes and Digressions

1. The command should work as-given on Linux, macOS, the BSDs, and Unix. If you're on Windows, you can get a POSIX command line via Cygwin or WSL.

2. There are several reasons why the list produced by that command doesn't perfectly match the set of codecs I've chosen to discuss above:

   • The second grep is meant to filter out inappropriate encoders like bmp which are not "video" codecs, despite being tagged V in this list. While technically you could probably stuff many of these into a container like AVI, MP4, or MKV to get a single-file video, that file won't likely be readable by anything but a program based on ffmpeg or libavcodec.

     There are a few exceptions to this, such as that -f avi -c:v ljpeg gives something you could call "Lossless MJPEG," but as a rule, we're not interested in stuffing many still-image files into an A/V container here to make a movie. We want widely-recognized video codecs here, not semantic trickery.

   • The command currently fails to filter out some inappropriate encoders such as GIF because they are not currently described in the ffmpeg -codecs output as bitmap or image file formats.

     GIF is an interesting case: it supports multiple image frames in a single GIF file with timing information for motion playback, but for several reasons, it is entirely inappropriate to our discussion here.

   • A few of the options that are shown are obsolete or never really got much traction, such as flashsv, dirac, and snow, so it's not worth discussing them above.

   • Some of the options in that list are meant only for use in pipelines between ffmpeg instances or between ffmpeg and another program, such as rawvideo and wrapped_avframe, and so are inappropriate for our purposes here.

   • Near the end of the discussion above, I judiciously expand the scope of the question a bit to include a few carefully chosen lossy options, so they don't pass the first grep filter in the above command.

Chroma subsampling

From your console output:

No pixel format specified, yuvj444p for H.264 encoding chosen.  
Use -pix_fmt yuv420p for compatibility with outdated media players.

Meaning you need to add -pix_fmt yuv420p as an output option. Otherwise ffmpeg may (depending on your inputs, ffmpeg version, and the encoder being used) try to avoid chroma subsampling. Technically this is good, but most (all?) non-FFmpeg based players such as QuickTime will not decode it properly.

Changing frame sizes

From your console output:

Input stream #0:0 frame changed from size:126x126 fmt:yuvj444p to
size:1024x768 fmt:yuvj444p

This is telling you that your input frames are not all of the same size and may cause unexpected behavior.

[swscaler @ 0257f640] deprecated pixel format used, make sure you did set range correctly

This is a warning (more for developers than users) that you can ignore.

Audio

You are re-encoding the audio. Instead you can stream copy the audio since your output container (mp4) should support MP3 audio. This will avoid re-encoding and therefore preserve the quality and will also work much faster.

Example

ffmpeg -i img%01d.jpg -i "03 eminem - when I'm Gone.mp3" -c:v libx264 \
-pix_fmt yuv420p -c:a copy -shortest output.mp4