FFmpeg can capture images from videos that can be used as thumbnails to represent the video. Most common ways of doing that are captured in the FFmpeg Wiki.
But, I don't want to pick random frames at some intervals. I found some options using filters on FFmpeg to capture scene changes:
The filter thumbnail tries to find the most representative frames in the video:
ffmpeg -i input.mp4 -vf "thumbnail,scale=640:360" -frames:v 1 thumb.png
and the following command selects only frames that have more than 40% of changes compared to previous (and so probably are scene changes) and generates a sequence of 5 PNGs.
ffmpeg -i input.mp4 -vf "select=gt(scene\,0.4),scale=640:360" -frames:v 5 thumb%03d.png
Info credit for the above commands to Fabio Sonnati.
The second one seemed better as I could get n images and pick the best. I tried it and it generated the same image 5 times.
Some more investigation led me to:
ffmpeg -i input.mp4 -vf "select=gt(scene\,0.5)" -frames:v 5 -vsync vfr out%02d.png
-vsync vfr ensures that you get different images. This still always picks the first frame of the video, in most cases the first frame is credits/logo and not meaningful, so I added a -ss 3 to discard first 3 seconds of the video.
My final command looks like this:
ffmpeg -ss 3 -i input.mp4 -vf "select=gt(scene\,0.5)" -frames:v 5 -vsync vfr out%02d.jpg
This was the best I could do. I have noticed that since I pick only 5 videos , all of them are mostly from beginning of the video and may miss out on important scenes that occur later in the video
I would like to pick your brains for any other better options.
Best Answer
How about looking for, ideally, the first >40%-change frame within each of 5 time spans, where the time spans are the 1st, 2nd, 3rd, 4th, and 5th 20% of the video.
You could also split it into 6 time spans and disregard the 1st one to avoid credits.
In practice, this would mean setting the fps to a low number while applying your scene change check and your argument to throw out the first bit of the video.
...something like: