execute command in bash script until output exceed

Question

I use a command which parses video files for certain frames and returning their timecode, when found. At the moment, I have to execute the command, wait, until the values printed to stdout reach the desired position and then abort the execution using Ctrl+C.

As I have to watch the process and to abort the execution in the right moment to get the information I need, I thought, I could automate this to some degree by creating a bash script.

I am not certain, if it can be done in bash, as I don't exactly know, how to abort the execution in connection with the values it writes to stdout.

The output of the command looks like

0.040000
5.040000
10.040000
15.040000
18.060000
(...)

I tried

until [[ "$timecode" -gt 30 ]]; do
  timecode=$(mycommand)
  sleep 0.1
done

echo "Result: $timecode"

or

while [[ "$timecode" -le 30 ]]; do
  timecode=$(mycommand)
  sleep 0.1
done

echo "Result: $timecode"

which both seem to result in the command being executed until it finishes and afterwards the rest of the loop is being processed. But I want to evaluate the output while the command executes and break execution depending on the output.

Additional information

The command has no capability to be stopped at a certain point in the stream. It parses the whole file and gives the results unless signalled to stop. This was my first shot.

The execution time of the command is very long as the files I parse are ~2GB. As I don't need all frames of the file but only a few around a given timecode, I never let it execute until it finished.

The output of the command varies from file to file, so I can't look for an exact value. If I knew the exact value, I probably wouldn't have to look for it.

The destination time code - in the example it is specified by "-gt 30" - is different for every file I will have to parse, so I will have to put this into a command line parameter once the script works. I would also have to make sure to get back more than the last value of the execution but about the last 5 values. For these two I already have Ideas.

I'm totally stuck on that one and have not even an idea what to google for.

Thank you for your input!

Manuel

---

With the answers of PSkocik and Kyle Burton, I was able to integrate the suggested solution into my script. It doesn't work and I don't see, why.

Here the complete script including the external command providing the output:

 #!/usr/bin/env bash
 set -eu -o pipefail

 parser () {
   local max="$1"
   local max_int

   max_int="${max%.*}"

   while read tc;
     do
       local tc_int
       tc_int="${tc%.*}"
       echo $tc

       if (( "$tc_int" >= "$max_int" )); then
         echo "Over 30: $tc";
         exec 0>&-
         return 0
       fi

     done
 }

 ffprobe "$1" -hide_banner -select_streams v -show_entries frame=key_frame,best_effort_timestamp_time -of csv=nk=1:p=0:s="|" -v quiet | sed -ne "s/^1|//p" | parser 30

I don't get any output from the "echo $tc" but the ffprobe is running - I can see it in top. It runs until I stop the script using Ctrl+C.

---

Thank you Kyle for your big efforts in this. I'd never come to such a conclusion. I changed the commandline of ffprobe to your suggestion

 ffprobe "$1" -hide_banner -select_streams v -show_entries frame=key_frame,best_effort_timestamp_time -of csv=nk=1:p=0:s="|" -v quiet | cut -f2 -d\| | parser 30

and now, I'm getting results while ffprobe runs. But... the way you changed the command returns all frames, ffprobe finds and not only the Keyframes. The original output of the ffprobe command looks like

 1|0.000000
 0|0.040000
 0|0.080000
 0|0.120000
 0|0.160000
 0|0.200000
 (...)

The 0 at the beginning of the line means: this is no keyframe. The 1 at the beginning of the line means: this is a keyframe.

The script is intended to provide only the keyframes around a certain timecode of the video file. The way you changed the command, it now provides all frames of the video file what makes the resulting output useless. It has to be filtered for all lines starting with zero to be dropped.

As I don't exactly understand, why this doesn't work with sed, I can only try to find a solution by try and error, facilitating different tools to filter the output. But if the filtering itself causes the problem, we might have hit a wall here.

Answer 1

I think PSkocik's approach makes sense. I think all you need to do is run your mycommand and pipe it into your while loop. If you put PSkocik's code in a file wait-for-max.sh then you should be able to run it as:

mycommand | bash wait-for-max.sh

After working with M. Uster in comments above, we've come up with the following solution:

#!/usr/bin/env bash
set -eu -o pipefail

# echo "bash cutter.sh rn33.mp4"

# From: https://stackoverflow.com/questions/45304233/execute-command-in-bash-script-until-output-exceeds-certain-value
# test -f stack_overflow_q45304233.tar ||  curl -k -O https://84.19.186.119/stack_overflow_q45304233.tar
# test -f stack_overflow_q45304233.tar ||  curl -k -O https://84.19.186.119/stack_overflow_q45304233.tar
# test -f rn33.mp4 || curl -k -O https://84.19.186.119/rn33.mp4

function parser () {
  local max="$1"
  local max_int

  # NB: this removes everything after the decimal point
  max_int="${max%.*}"

  # I added a line number so I could match up the ouptut from this function
  # with the output captured by the 'tee' command
  local lnum="0"
  while read -r tc;
    do

      lnum="$(( 1 + lnum ))"

      # if a blank line is read, just ignore it and continue
     if [ -z "$tc" ]; then
       continue
     fi

     local tc_int
     # NB: this removes everything after the decimal point
     tc_int="${tc%.*}"
     echo "Read[$lnum]: $tc"

     if (( "$tc_int" >= "$max_int" )); then
       echo "Over 30: $tc";
       # This closes stdin on this process, which will cause an EOF on the
       # process writing to us across the pipe
       exec 0>&-
       return 0
     fi

    done
}

# echo "bash version:    $BASH_VERSION"
# echo "ffprobe version: $(ffprobe -version | head -n1)"
# echo "sed version:     $(sed --version | head -n1)"

# NB: by adding in the 'tee ffprobe.out' into the pipeline I was able to see
# that it was producing lines like:
#
# 0|28.520000
# 1|28.560000
#
#
# changing the sed to look for any single digit and a pipe fixed the script
# another option is to use cut, see below, which is probalby more robust.

# ffprobe "$1" \
#   -hide_banner \
#   -select_streams v \
#   -show_entries frame=key_frame,best_effort_timestamp_time \
#   -of csv=nk=1:p=0:s="|" \
#   -v quiet 2>&1 | \
#   tee ffprobe.out |
#   sed -ne "s/^[0-9]|//p" | \
#   parser 30


ffprobe "$1" \
    -hide_banner \
    -select_streams v \
    -show_entries frame=key_frame,best_effort_timestamp_time \
    -of csv=nk=1:p=0:s="|" \
    -v quiet 2>&1 | \
    cut -f2 -d\| | \
    parser 30

Answer 2

If you have process a that's outputting stuff to stdout and process b that reads the outputted stuff via a pipe:

a | b

all b has to usually do to kill a when a certain item is outputted is to close its standard input.

A sample b:

b()
{
    while read w;
        do case $w in some_pattern)exec 0>&-;; esac; 
        echo $w
    done
}

This closing of stdin (filedescriptor 0) will cause the producer process to be killed by SIGPIPE the moment it tries to make its next write.

Answer 3

The answer to my question has finally been found by the help of PSkocik and intense support of Kyle Burton. Thanks to both of you!

I didn't know, that it is possible to pipe the output of commands executed in a script to a function that belongs to the script. This was the first piece of information necessary.

And I didn't know, how to evaluate the piped information inside the function properly and how to signal from inside the function, that the execution of the command generating the values should be terminated.

Additionally, Kyle found, that the filtering I did by piping the original output to sed and the resulting data to the function inside the script prohibited the script to function as designed. I'm still uncertain, why - but it definitively does.

The original command generating the output is now being piped as it is to the internal function of the script. The filtering is being done inside the function to avoid the problem with sed. Now everything works as expected and I can continue completing the script.

This is the working code of the soultion:

 #!/usr/bin/env bash
 set -eu -o pipefail

 function parser () {
   local max="$1"
   local max_int

   max_int="${max%.*}"

   while read tc;
     do

      #If line is empty, continue
      if [ -z "$tc" ]; then
        continue
      fi

      #If first char is 0 (=non-Index Frame), continue
      local iskey="${tc:0:1}";

      if [ $iskey == "0" ]; then
        continue
      fi

      #Return timecode if intended maximum has been reached
      local val="${tc:2:10}"
      local tc_int
      tc_int="${val%.*}"

      if (( "$tc_int" >= "$max_int" )); then
        echo "First index frame at/after given Timecode: $tc";
        exec 0>&-
        return 0
      fi

     done
 }

 ffprobe "$1" -hide_banner -select_streams v -show_entries frame=key_frame,best_effort_timestamp_time -of csv=nk=1:p=0:s="|" -v quiet | parser "$2"

Usage:

 ./script.sh "Name of Movie.avi" 30

where 30 represents the timecode at which the next found index frame is being searched and returned.