How to specify unordered image list?

Question

I'm trying to figure out how to specify a specific list of images to be converted into a video. I do know that we can do something like:

ffmpeg -i image_04%d.png

That would pick all the images from the folder that match the sequence. However in my case the image files are no necessarily in the order as its name implies. The reason is that the order is kept on a database and the file names are essentially the database row id.

How could I specify the correct input sequence? I'm essentially calling ffmpeg from code and not from the command line. So any changes ideas to the code are also welcomed.

Thanks!

Answer 1

Well, the solution was to patch the image2 class to support a list with the numbers to pick from. Than I provided to ffmpeg the input file pattern along with an array of numbers with the specified order to read the input. This worked perfectly.

Answer 2

Your application could create symbolic links starting at 0001 that respect the order of the original frames, then pass that sequence into ffmpeg. Once the video is complete delete the symlinks and you're done.

Answer 3

Here is a script along Kevin's idea which works for me. You might want to replace the file name pattern (shot*.png) and the output filename movie.mp4. All frame_ ... files are removed by the script when finished.

# script to create movie from multiple png files
# taken in choronoligcal order

# start at 0
count=0
# take all files named shot - something ordered by date
for f in `ls -rt shot*.png`
do 
  # get the index in 0000 format
  printf -v counts "%04d" $count
    # link file to a frame_0000  named symbolic link
    ln -s $f frame_$counts.png
    # increment counter
  count=`expr $count + 1`
done
# create movie
# slowed down by factor 5
# ffmpeg -f image2 -i frame_%04d.png -vcodec mpeg4 -vf "setpts=5*PTS" movie.mp4
ffmpeg -i frame_%04d.png movie.mp4
# remove the links
rm frame_*.png

Answer 4

Let's say we have a list of file names in the correct order like this:

frames=(
    "5K1OCNKToUu.png"  "kJuKFQS0Fgnp.png" "00v4U4JTyUn.png"  "3sg9sDwPZoX.png"
    "1KsuEk9mboa9.png" "qNrI8zlRBmW.png"  "MOvSca3wlPsP.png" "5rXcxfGQXunY.png"
    "hjruIcoN0aTn.jpg" "OhRttnWtKy.png"   "e2Qj8jCixc.png"   "Uze2H7vzrt4.png"
    "n14qhmjiBW3.png"  "ZDMvY4g1hzgS.png" "ibnb7MxELyGp.png" "9c8QGWmBEDNg.png"
    "STQT0t7oqPEK.png" "jI7UvpbLDWPc.png" "6clazeaUAJHv.png" "ylJ40r9uMK9d.png"
    "RICq5KV00P6.png"  "zjCLrappFMPq.png" "TJQTDv313KBo.png" "Gu3pLpWylsuo.png"
    "Ksym4SB6VYNv.png" "rIyj0LJIjBVX.png" "pSUm2J8xYU.png"   "Rnsr0H0m7p9A.png"
    "x4vVomlOolxt.png" "2W1QURLQUyE8.png" "m3JgtDzQ0VgE.png" "CrjN9TVJKMAU.png"
    "IO6pnF83ivqo.png" "hY15nsYDvr4h.png" "1GagDdBM9L7.png"
)

and we want to create an animated GIF out of these with a certain frame rate e.g. FPS=30. This can be done like in the other answers by creating symbolic links e.g. named 001.png to 035.png and then use:

ffmpeg -i 03%d.png

Another way is to make use of ffmpeg's concat-feature. Unfortunately this is a tad easier said than done, because the the feature expects video streams, meaning the images need to be looped as long as needed.

The command to concatenate three images with 500ms inbetween is this:

ffmpeg -f concat -safe 0 -i <(cat <<EOF
file '$(pwd)/1KsuEk9mboa9.png'
duration 0.5
file '$(pwd)/hjruIcoN0aTn.png'
duration 0.5
file '$(pwd)/n14qhmjiBW3.png'
duration 0.5
EOF
) -framerate 2 out.gif

Tested with ffmpeg version 3.0.1-3.

Explanation:

• The concat demuxer expects a file with a list of relative file names prepended by the keyword file. In order to not clutter the current working directory (or maybe we don't have writing permissions), we use process substition <( ... ). But this creates a file in /dev/fd/ and if relative file names are used, then this results in error messages like this:

  [concat @ 0xc181c0] Impossible to open '/dev/fd/5K1OCNKToUu.png'
  

  That's why the absolute path is given. But absolute paths are not allowed by default, resulting in:

  [concat @ 0x17da1c0] Unsafe file name '/home/5K1OCNKToUu.png'
  

  In order to solve that -safe 0 is used.

• When trying to specify an input frame rate with -r

  ffmpeg -f concat -safe 0 -r 2 -i <(cat <<EOF
  file '$(pwd)/1KsuEk9mboa9.png'
  file '$(pwd)/hjruIcoN0aTn.png'
  file '$(pwd)/n14qhmjiBW3.png'
  EOF
  ) -framerate 2 out.gif
  

  errors like this occur:

  [concat @ 0x1458220] DTS -230575710986777 < 0 out of order
  DTS -230575710986777, next:40000 st:0 invalid dropping
  PTS -230575710986777, next:40000 invalid dropping st:0
  

  The resulting GIF works like expected anyway. Using the duration option of concat explicitly solves these warnings/errors.

  Note the comment for -r

  As an input option, ignore any timestamps stored in the file and instead generate timestamps assuming constant frame rate fps. This is not the same as the -framerate option used for some input formats like image2 or v4l2 (it used to be the same in older versions of FFmpeg). If in doubt use -framerate instead of the input option -r.

  My take on this is that the above warnings from concat, which notify you that it couldn't find a playing length for images, don't result in bad behavior because the framerate is forced after the concat warnings with -r, which works, but may not be the intended way. I would only use this when manually writing ffmpeg commands, but not in scripts.

Putting together a small script to work with the list of file names specified in the beginning:

function makeGif() {
    local targetName=$1; shift
    local FPS=$1; shift
    # concat doesn't recognize .33 as returned by bc by default
    local delay=$(bc <<< "scale=5; x=1/$FPS; if (x<1) print 0; x")
    local list
    for file in $@; do
        list+=$(printf "\nfile '$(pwd)/$file'\nduration $delay")
    done
    ffmpeg -f concat -safe 0 -i <(cat <<< "$list") -r $FPS "$targetName".gif
    #ffmpeg -f concat -safe 0 -i <(cat <<< "$list") -r $FPS -c:v libx264 -crf 5 -pix_fmt yuv420p "$targetName".mkv
}
makeGif out 30 ${frames[@]}

The uncommented line in the code above will give a H264-encoded mkv.

Heterogenous image types

If your image list contains images of different types, e.g. by replacing hjruIcoN0aTn.png with hjruIcoN0aTn.jpg in frames, then this usage of concat won't work. Images in other formats than the first specified will be dropped and throw errors like:

[png @ 0x1ec8260] Invalid PNG signature 0xFFD8FFE000104A46.
Error while decoding stream #0:0: Invalid data found when processing input

In this case you will have to use the concat filter instead of the simple demuxer. The command with the three files from above will then have to be changed to this:

ffmpeg \
-f image2 -loop 1 -thread_queue_size 4096 -framerate 30 -t 0.5 -i "$(pwd)/1KsuEk9mboa9.png"  \
-f image2 -loop 1 -thread_queue_size 4096 -framerate 30 -t 0.5 -i "$(pwd)/hjruIcoN0aTn.jpg"  \
-f image2 -loop 1 -thread_queue_size 4096 -framerate 30 -t 0.5 -i "$(pwd)/n14qhmjiBW3.png"   \
-filter_complex 'concat=n=3:v=1 [vmerged]' \
-map '[vmerged]' -r 30 out.gif

Explanation:

• Each options used is explained really well here. Note that the -framerate option is specific to the image importer and is at least technically different from -r, practically in this case they produce the same results.

• The -thread_queue_size seems to be necessary, because we extend one image for a quite long 500ms, resulting in these error messages:

  [image2 @ 0x18856e0] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
  [image2 @ 0x188a100] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
  [image2 @ 0x188b9e0] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
  

  The description of the option gave me reason to think that the overly large delay is at fault:

  -thread_queue_size size (input) This option sets the maximum number of queued packets when reading from the file or device. With low latency / high rate live streams, packets may be discarded if they are not read in a timely manner; raising this value can avoid it.

Putting all this in a script again:

function makeGif() {
    local targetName=$1; shift
    local FPS=$1; shift
    local delay=$(bc <<< "scale=5; x=1/$FPS; if (x<1) print 0; x")
    local list=()
    local nFiles=0
    for file in $@; do
        list+=( -f image2 -loop 1 -thread_queue_size 4096 -framerate $FPS -t $delay -i "$(pwd)/$file" )
        nFiles=$((nFiles+1))
    done
    ffmpeg ${list[@]} -filter_complex "concat=n=$nFiles:v=1 [vmerged]" -map '[vmerged]' -r $FPS "$targetName".gif
    #ffmpeg ${list[@]} -filter_complex "concat=n=$nFiles:v=1 [vmerged]" -map '[vmerged]' -r $FPS -c:v libx264 -crf 5 -pix_fmt yuv420p "$targetName".mkv
}
makeGif out 30 ${frames[@]}

For some very weird reason the above command doesn't work as is !!!. In my case it dropped 33 of the 35 frames specified without a warning or error message:

frame=    2 fps=0.0 q=-0.0 Lsize=       2kB time=00:00:00.07 bitrate= 179.7kbits/s dup=0 drop=33 speed=0.155x

The reason that this is weird is, that it works perfectly like intended for double the delay, i.e. each image being shown for 2 frames, i.e. changing 1/$FPS to 2/$FPS:

frame=   70 fps=0.0 q=-0.0 Lsize=     701kB time=00:00:02.33 bitrate=2465.4kbits/s speed= 2.9x 

This fact suggests an easy workaround: encode with double the framerate and display each image for 2 frames. But that is wasteful and not favorable anyway.