Related to Understanding/controlling MLT melt slideshow?, I am trying to understand how does melt's luma mixer work, especially in context of short (small frame number) durations.
For instance, if I use something like this specification tmppics/pic_01.jpg length=6 -mix 2 -mixer luma
I would have expected 2 frames of fade of pic_01, then 2 frames full pic_01, then 2 frames of fade again - however, I've both experienced results that are like this, and those that are not.
To clarify this, I've developed a bash script, which uses ImageMagick convert
to generate test images, then melt
to make a "slideshow" with fades out of these images, ffmpeg
to convert that to animated .gif, then ImageMagick convert
and montage
to obtain a film strip (sprite sheet) of the video .gif (Ubuntu 18.04, melt 6.6.0, ffmpeg version 3.4.4-0ubuntu0.18.04.1, ImageMagick 6.9.7-4 Q16 x86_64 20170114).
Here is the bash script, melt-test-strip.sh
:
#!/usr/bin/env bash
FRAMERATE=25
echo "
description=DV PAL
frame_rate_num=$FRAMERATE
frame_rate_den=1
width=720
height=576
progressive=0
sample_aspect_num=59
sample_aspect_den=54
display_aspect_num=4
display_aspect_den=3
colorspace=601
" > my-melt.profile
mkdir tmppics
convert -background lightblue -fill blue -size 3840x2160 -pointsize 200 -gravity center label:"Test A" tmppics/pic_01.jpg
convert -background lightblue -fill blue -size 2160x3840 -pointsize 200 -gravity center label:"Test B" tmppics/pic_02.jpg
if [ -z "$IMGDURATIONF" ]; then
IMGDURATIONF=6 # picture duration, frames
fi
if [ -z "$FADEDURATIONF" ]; then
FADEDURATIONF=2 # single-end fade duration, frames
fi
melt -verbose -profile ./my-melt.profile \
tmppics/pic_01.jpg length=$IMGDURATIONF \
tmppics/pic_02.jpg length=$IMGDURATIONF -mix $FADEDURATIONF -mixer luma \
colour:black length=$IMGDURATIONF -mix $FADEDURATIONF -mixer luma \
-consumer avformat:meltout.mp4 vcodec=libx264 an=1
# auxiliary: just for creating sprite sheet/film strip:
melt -verbose -profile ./my-melt.profile tmppics/pic_01.jpg length=$IMGDURATIONF -consumer avformat:meltout-01.mp4 vcodec=libx264 an=1
melt -verbose -profile ./my-melt.profile tmppics/pic_02.jpg length=$IMGDURATIONF -consumer avformat:meltout-02.mp4 vcodec=libx264 an=1
melt -verbose -profile ./my-melt.profile colour:black length=$IMGDURATIONF -consumer avformat:meltout-b.mp4 vcodec=libx264 an=1
# convert to gif to obtain sprite sheet/film strip from:
ffmpeg \
-i meltout.mp4 \
-r 25 \
-vf scale=256:-1 \
-y meltout.gif
# auxiliary: just for creating sprite sheet/film strip:
ffmpeg -i meltout-01.mp4 -r $FRAMERATE -vf scale=256:-1 -y meltout-01.gif
ffmpeg -i meltout-02.mp4 -r $FRAMERATE -vf scale=256:-1 -y meltout-02.gif
ffmpeg -i meltout-b.mp4 -r $FRAMERATE -vf scale=256:-1 -y meltout-b.gif
convert meltout.gif -coalesce meltoutc.gif
convert meltout-01.gif -coalesce meltoutc-01.gif
convert meltout-02.gif -coalesce meltoutc-02.gif
convert meltout-b.gif -coalesce meltoutc-b.gif
FRAMETHICK=5
#~ montage temp.gif -tile x1 -geometry '1x1+0+0<' -border 5 -bordercolor "rgb(200, 200, 200)" -label 'Image' -quality 100 meltout.png
# "%p index of image in current image list" is "t=> index of current image (s) in list" in fx:
montage -label 'Frame %[fx:t+1]/%n' meltoutc.gif -tile x1 -geometry '1x1+0+0<' -frame $FRAMETHICK -bordercolor "rgb(200, 200, 200)" -quality 100 meltout.png
# here from the .gif - otherwise for replicating images: montage in.jpg +clone +clone +clone -tile x4 -geometry +0+0 out.jpg
montage -label 'Frame %[fx:t+1]/%n' meltoutc-01.gif -tile x1 -geometry '1x1+0+0<' -frame $FRAMETHICK -bordercolor "rgb(200, 200, 200)" -quality 100 meltout-01.png
montage -label 'Frame %[fx:t+1]/%n' meltoutc-02.gif -tile x1 -geometry '1x1+0+0<' -frame $FRAMETHICK -bordercolor "rgb(200, 200, 200)" -quality 100 meltout-02.png
montage -label 'Frame %[fx:t+1]/%n' meltoutc-b.gif -tile x1 -geometry '1x1+0+0<' -frame $FRAMETHICK -bordercolor "rgb(200, 200, 200)" -quality 100 meltout-b.png
# for offsetting:
# (gif) frame width/height
fw=$(convert meltoutc-01.gif[0] -format "%w" info:)
fh=$(convert meltoutc-01.gif[0] -format "%w" info:)
# strip width/height
sw=$(convert meltout.png -format "%w" info:)
sh=$(convert meltout.png -format "%h" info:)
echo fw $fw fh $fh sw $sw sh $sh
convert -size $(( (IMGDURATIONF-FADEDURATIONF)*(fw+2*FRAMETHICK) ))x$sh xc:white meltout-02.png +append meltout-02B.png
# IMGDURATIONF-FADEDURATIONF to get to the start of second clip; +IMGDURATIONF from there to get to end of second clip, and -FADEDURATIONF from there to get to start of third clip
convert -size $(( (IMGDURATIONF-FADEDURATIONF+IMGDURATIONF-FADEDURATIONF)*(fw+2*FRAMETHICK) ))x$sh xc:white meltout-b.png +append meltout-bB.png
montage -geometry '+0+0' meltout.png meltout-01.png meltout-02B.png meltout-bB.png -tile 1x meltout-all.png
eog meltout-all.png
So, if you call bash melt-test-strip.sh
, you get the defaults, IMGDURATIONF=6 and FADEDURATIONF=2, for which the output is this (click for full size):
The only way the luma-mixed result (on top) makes sense to me, is if the starting frame of the new clip participates in the mix with 0% - which is why, for the second image clip, we observe 1 frame fade + 3 frames full + 1 frame fade (instead of 2 frames fade + 2 frames full + 2 frames fade, which I'd expect).
Is this correct?
Reading https://www.mltframework.org/docs/melt/#mixes I cannot really tell if this interpretation is correct.
If I run the script with other parameters, like IMGDURATIONF=8 FADEDURATIONF=3 bash melt-test-strip.sh
, then the output is:
... in which case the interpretation holds (if every new clip's first frame participates in the mix with 0%, that explains why we're seeing 2 frames fade + 3 frames full + 2 frames fade for second clip = 7 frames in all, instead of the requested 8) - but now I'm not sure if this is just an artefact of my script (as opposed to the true behavior of melt
s mix).
Can anyone confirm if this is how melt
s mixer works - and if not, explain how can it be understood?