Bash script: automate ffmpeg encoding for mpeg-das

Question

I'm writing a bash file to create video encoding and concatenation for dash live streaming use, Basically it read an input video folder, encodes all videos into three resolution formats, after that it concatening them to create three adaption sets.

DIAGRAM:

This script checks for fps conformance,

force/scaling resolution if the input is not 1920 x 1080p,

Insert the channel logo png,

Cut the end of all videos input in order to make them finish with closed gop, this to ensure that there are not videos with audio and video track with different lenght.

ISSUE:

Actually I'm not sure that the concatenation process respects the closed GOP alignment as it after the encoding.. I try to cut also the end of the concatenation result in order to make it finish without decimals on a closed gop too, but im unable to erase all decimals from the total duration:

total duration in seconds: 826.795000
total duration corrected in seconds: 826

But the real duration measured by ffprobe is

824.044000

I try to check keyframes alignment with mp4box has they teach without any result:

MP4Box -info TRACK_ID source1.mp4 2>&1 | grep GOP

This is the first time that i work with "video scripts" and probably i don't know what input to give for TRACK_ID

BASH SCRIPT:

#!/bin/bash
#CANCAT 0.2

cd input
times=()
fps=()
for f in *.mp4; do

    _t=$(ffprobe -i "$f" -show_entries format=duration -v quiet -of csv="p=0")
    times+=("$_t")

    _f=$(ffmpeg -i "$f" 2>&1 | sed -n "s/.*, \\(.*\\) fp.*/\\1/p")
    fps+=("$_f")
done
#SUM ALL DURATIONS
TOTALDURATION=$( echo "${times[@]}" | sed 's/ /+/g' | bc )
#DELETE DECIMAL
DURROUND=$(echo "$TOTALDURATION" | cut -d'.' -f1) 
#GET REST OF DIVISION BY 2 AS GOP
TOTDELTA="$((DURROUND%2))"
#SUBTRACT DELTA FROM TOTAL DURATION
TOTDUR="$(($DURROUND-$TOTDELTA))"

#GET NUMBER OF ELEMENTS IN FPS ARRAY  
tLen=${#fps[@]}
#CHECK FPS EQUALITY     
for tLen in "${fps[@]:1}"; do
    if [[ $tLen != ${fps[0]} ]]; then
        printf "WARNING: VIDEO’S FRAME-RATE ARE NOT EQUALS, THE PROCESS CAN’T START."
        printf "%s\\0" "${fps[@]}" |
            sort -zu |
            xargs -0 printf " %s"
        printf "\\n"
       exit 1
    fi
done
for f in *.mp4; do
#GET DURATION OF EACH VIDEO
DUR="$(ffprobe -i "$f" -show_entries format=duration -v quiet -of csv="p=0")"
DUR=$(echo "$DUR" | cut -d'.' -f1) # DELETE DECIMAL
#GET FPS OF EACH VIDEO
FPS="$(ffmpeg -i "$f" 2>&1 | sed -n "s/.*, \(.*\) fp.*/\1/p")"
#ROUND FPS OF EACH VIDEO
FPSC=$( echo "($FPS+0.5)/1" | bc )
#REMOVE EXTENSION FROM VIDEO FILE NAME
NAME=$(echo "$f" | cut -d'.' -f1)

#GET GOP
GOP="$((FPSC*2))"
DELTADUR="$((DUR%2))"
DUR="$(($DUR-$DELTADUR))"

#ENCODE 1080p
ffmpeg -y -i "$f" -i ../logo/logo.png -c:a aac -b:a 384k -ar 48000 -ac 2 -async 1 -c:v libx264 -x264opts keyint=$GOP:min-keyint=$GOP:no-scenecut -bf 0 -r $FPSC -b:v 4800k -maxrate 4800k -bufsize 3000k -profile:v main -crf 22 -t $DUR -filter_complex "[0:v][1:v]overlay=main_w-overlay_w-10:10,scale=1920:1080,setsar=1" ../buffer/${NAME}-1080.mp4

#ENCODE 720p
ffmpeg -y -i ../buffer/${NAME}-1080.mp4 -c:a aac -b:a 256k -ar 48000 -ac 2 -async 1 -c:v libx264 -x264opts keyint=$GOP:min-keyint=$GOP:no-scenecut -bf 0 -s 1280x720 -r $FPSC -b:v 2400k -maxrate 2400k -bufsize 1500k -profile:v main -crf 22 -t $DUR ../buffer/${NAME}-720.mp4

#ENCODE 360p
ffmpeg -y -i ../buffer/${NAME}-720.mp4 -c:a aac -b:a 128k -ar 48000 -ac 2 -async 1 -c:v libx264 -x264opts keyint=$GOP:min-keyint=$GOP:no-scenecut -bf 0 -s 640x360 -r $FPSC -b:v 800k -maxrate 800k -bufsize 500k -profile:v main -crf 22 -t $DUR ../buffer/${NAME}-360.mp4
done


#enter in buffer
cd ..
cd buffer

#CONCAT 1080 SET
# with a bash for loop
for f in ./*1080.mp4; do echo "file '$f'" >> 1080list.txt; done

ffmpeg -f concat -safe 0 -i 1080list.txt -t $TOTDUR -c copy ../output/1080set.mp4

#CONCAT 720 SET
# with a bash for loop
for f in ./*720.mp4; do echo "file '$f'" >> 720list.txt; done

ffmpeg -f concat -safe 0 -i 720list.txt -t $TOTDUR -c copy ../output/720set.mp4

#CONCAT 360 SET
# with a bash for loop
for f in ./*360.mp4; do echo "file '$f'" >> 360list.txt; done

ffmpeg -f concat -safe 0 -i 360list.txt -t $TOTDUR -c copy ../output/360set.mp4

#CLEAN BUFFER
rm *.mp4
rm *.txt

echo "CONCAT COMPLETED:"
echo "frame-rate: $fps"
echo "total duration in seconds: $TOTALDURATION"
echo "total duration corrected in seconds: $TOTDUR"

The full file with relative folders:

BASH SCRIPT WITH FOLDERS

RESULT VIDEO

There is someone who can help me to understand why I can not eliminate the decimals of the total duration during concat? And how to check overall keyframes allignment? Also any impovement that i ignore is welcome!

Thanks a lot!

Massimo

Answer 1

I understand that there was a big mistake in the algorithm: the total duration is calculated before the single videos cut process, so the total duration is always more long then the effective concatention duration, i fix it,

now

total duration in seconds: 824.044000
total duration corrected in seconds: 824

ffprobe measure: 824.012000, im unable to get it without decimals and check the overall keyframes alignment.

#!/bin/bash
#CANCAT 0.2

cd input
fps=()
# GET FPS OF ALL VIDEOS INTO ARRAY
for f in *.mp4; do

    _f=$(ffmpeg -i "$f" 2>&1 | sed -n "s/.*, \\(.*\\) fp.*/\\1/p")
    fps+=("$_f")
done

#GET NUMBER OF ELEMENTS IN FPS ARRAY  
tLen=${#fps[@]}
#CHECK FPS EQUALITY     
for tLen in "${fps[@]:1}"; do
    if [[ $tLen != ${fps[0]} ]]; then
        printf "WARNING: VIDEO’S FRAME-RATE ARE NOT EQUALS, THE PROCESS CAN’T START."
        printf "%s\\0" "${fps[@]}" |
            sort -zu |
            xargs -0 printf " %s"
        printf "\\n"
       exit 1
    fi
done
for f in *.mp4; do
#GET DURATION OF EACH VIDEO
DUR="$(ffprobe -i "$f" -show_entries format=duration -v quiet -of csv="p=0")"
DUR=$(echo "$DUR" | cut -d'.' -f1) # DELETE DECIMAL
#GET FPS OF EACH VIDEO
FPS="$(ffmpeg -i "$f" 2>&1 | sed -n "s/.*, \(.*\) fp.*/\1/p")"
#ROUND FPS OF EACH VIDEO
FPSC=$( echo "($FPS+0.5)/1" | bc )
#REMOVE EXTENSION FROM VIDEO FILE NAME
NAME=$(echo "$f" | cut -d'.' -f1)

#GET GOP
GOP="$((FPSC*2))"
DELTADUR="$((DUR%2))"
DUR="$(($DUR-$DELTADUR))"

#ENCODE 1080p
ffmpeg -y -i "$f" -i ../logo/logo.png -c:a aac -b:a 384k -ar 48000 -ac 2 -async 1 -c:v libx264 -x264opts keyint=$GOP:min-keyint=$GOP:no-scenecut -bf 0 -r $FPSC -b:v 4800k -maxrate 4800k -bufsize 3000k -profile:v main -crf 22 -t $DUR -filter_complex "[0:v][1:v]overlay=main_w-overlay_w-10:10,scale=1920:1080,setsar=1" ../buffer/${NAME}-1080.mp4

#ENCODE 720p
ffmpeg -y -i ../buffer/${NAME}-1080.mp4 -c:a aac -b:a 256k -ar 48000 -ac 2 -async 1 -c:v libx264 -x264opts keyint=$GOP:min-keyint=$GOP:no-scenecut -bf 0 -s 1280x720 -r $FPSC -b:v 2400k -maxrate 2400k -bufsize 1500k -profile:v main -crf 22 -t $DUR ../buffer/${NAME}-720.mp4

#ENCODE 360p
ffmpeg -y -i ../buffer/${NAME}-720.mp4 -c:a aac -b:a 128k -ar 48000 -ac 2 -async 1 -c:v libx264 -x264opts keyint=$GOP:min-keyint=$GOP:no-scenecut -bf 0 -s 640x360 -r $FPSC -b:v 800k -maxrate 800k -bufsize 500k -profile:v main -crf 22 -t $DUR ../buffer/${NAME}-360.mp4
done


#enter in buffer
cd ..
cd buffer
times=()
for f in *1080.mp4; do
    _t=$(ffprobe -i "$f" -show_entries format=duration -v quiet -of csv="p=0")
    times+=("$_t")
done
#SUM ALL DURATIONS
TOTALDURATION=$( echo "${times[@]}" | sed 's/ /+/g' | bc )
#DELETE DECIMAL
DURROUND=$(echo "$TOTALDURATION" | cut -d'.' -f1) 
#GET REST OF DIVISION BY 2 AS GOP
TOTDELTA="$((DURROUND%2))"
#SUBTRACT DELTA FROM TOTAL DURATION
TOTDUR="$(($DURROUND-$TOTDELTA))"


#CONCAT 1080 SET
# with a bash for loop
for f in ./*1080.mp4; do echo "file '$f'" >> 1080list.txt; done

ffmpeg -f concat -safe 0 -i 1080list.txt -t $TOTDUR -c copy ../output/1080set.mp4

#CONCAT 720 SET
# with a bash for loop
for f in ./*720.mp4; do echo "file '$f'" >> 720list.txt; done

ffmpeg -f concat -safe 0 -i 720list.txt -t $TOTDUR -c copy ../output/720set.mp4

#CONCAT 360 SET
# with a bash for loop
for f in ./*360.mp4; do echo "file '$f'" >> 360list.txt; done

ffmpeg -f concat -safe 0 -i 360list.txt -t $TOTDUR -c copy ../output/360set.mp4

#CLEAN BUFFER
rm *.mp4
rm *.txt

echo "CONCAT COMPLETED:"
echo "frame-rate: $fps"
echo "total duration in seconds: $TOTALDURATION"
echo "total duration corrected in seconds: $TOTDUR"