How to add a progress bar to a shell script?

2018-12-31 14:23发布

问题:

When scripting in bash or any other shell in *NIX, while running a command that will take more than a few seconds, a progress bar is needed.

For example, copying a big file, opening a big tar file.

What ways do you recommend to add progress bars to shell scripts?

回答1:

You can implement this by overwriting a line. Use \\r to go back to the beginning of the line without writing \\n to the terminal.

Write \\n when you\'re done to advance the line.

Use echo -ne to:

  1. not print \\n and
  2. to recognize escape sequences like \\r.

Here\'s a demo:

echo -ne \'#####                     (33%)\\r\'
sleep 1
echo -ne \'#############             (66%)\\r\'
sleep 1
echo -ne \'#######################   (100%)\\r\'
echo -ne \'\\n\'

In a comment below, puk mentions this \"fails\" if you start with a long line and then want to write a short line: In this case, you\'ll need to overwrite the length of the long line (e.g., with spaces).



回答2:

You may also be interested in how to do a spinner:

Can I do a spinner in Bash?

Sure!

i=1
sp=\"/-\\|\"
echo -n \' \'
while true
do
    printf \"\\b${sp:i++%${#sp}:1}\"
done

Each time the loop iterates, it displays the next character in the sp string, wrapping around as it reaches the end. (i is the position of the current character to display and ${#sp} is the length of the sp string).

The \\b string is replaced by a \'backspace\' character. Alternatively, you could play with \\r to go back to the beginning of the line.

If you want it to slow down, put a sleep command inside the loop (after the printf).

A POSIX equivalent would be:

sp=\'/-\\|\'
printf \' \'
while true; do
    printf \'\\b%.1s\' \"$sp\"
    sp=${sp#?}${sp%???}
done

If you already have a loop which does a lot of work, you can call the following function at the beginning of each iteration to update the spinner:

sp=\"/-\\|\"
sc=0
spin() {
   printf \"\\b${sp:sc++:1}\"
   ((sc==${#sp})) && sc=0
}
endspin() {
   printf \"\\r%s\\n\" \"$@\"
}

until work_done; do
   spin
   some_work ...
done
endspin


回答3:

Some posts have showed how to display the command\'s progress. In order to calculate it, you\'ll need to see how much you\'ve progressed. On BSD systems some commands, such as dd(1), accept a SIGINFO signal, and will report their progress. On Linux systems some commands will respond similarly to SIGUSR1. If this facility is available, you can pipe your input through dd to monitor the number of bytes processed.

Alternatively, you can use lsof to obtain the offset of the file\'s read pointer, and thereby calculate the progress. I\'ve written a command, named pmonitor, that displays the progress of processing a specified process or file. With it you can do things, such as the following.

$ pmonitor -c gzip
/home/dds/data/mysql-2015-04-01.sql.gz 58.06%

An earlier version of Linux and FreeBSD shell scripts appears on my blog.



回答4:

use the linux command pv:

http://linux.die.net/man/1/pv

it doesn\'t know the size if it\'s in the middle of the stream, but it gives a speed and total and from there you can figure out how long it should take and get feedback so you know it hasn\'t hung.



回答5:

Got an easy progress bar function that i wrote the other day:

#!/bin/bash
# 1. Create ProgressBar function
# 1.1 Input is currentState($1) and totalState($2)
function ProgressBar {
# Process data
    let _progress=(${1}*100/${2}*100)/100
    let _done=(${_progress}*4)/10
    let _left=40-$_done
# Build progressbar string lengths
    _fill=$(printf \"%${_done}s\")
    _empty=$(printf \"%${_left}s\")

# 1.2 Build progressbar strings and print the ProgressBar line
# 1.2.1 Output example:                           
# 1.2.1.1 Progress : [########################################] 100%
printf \"\\rProgress : [${_fill// /\\#}${_empty// /-}] ${_progress}%%\"

}

# Variables
_start=1

# This accounts as the \"totalState\" variable for the ProgressBar function
_end=100

# Proof of concept
for number in $(seq ${_start} ${_end})
do
    sleep 0.1
    ProgressBar ${number} ${_end}
done
printf \'\\nFinished!\\n\'

Or snag it from,
https://github.com/fearside/ProgressBar/



回答6:

I was looking for something more sexy than the selected answer, so did my own script.

Preview

\"progress-bar.sh

Source

I put it on github progress-bar.sh

progress-bar() {
  local duration=${1}


    already_done() { for ((done=0; done<$elapsed; done++)); do printf \"▇\"; done }
    remaining() { for ((remain=$elapsed; remain<$duration; remain++)); do printf \" \"; done }
    percentage() { printf \"| %s%%\" $(( (($elapsed)*100)/($duration)*100/100 )); }
    clean_line() { printf \"\\r\"; }

  for (( elapsed=1; elapsed<=$duration; elapsed++ )); do
      already_done; remaining; percentage
      sleep 1
      clean_line
  done
  clean_line
}

Usage

 progress-bar 100


回答7:

GNU tar has a useful option which gives a functionality of a simple progress bar.

(...) Another available checkpoint action is ‘dot’ (or ‘.’). It instructs tar to print a single dot on the standard listing stream, e.g.:

$ tar -c --checkpoint=1000 --checkpoint-action=dot /var
...

The same effect may be obtained by:

$ tar -c --checkpoint=.1000 /var


回答8:

A simpler method that works on my system using the pipeview ( pv ) utility.

srcdir=$1
outfile=$2


tar -Ocf - $srcdir | pv -i 1 -w 50 -berps `du -bs $srcdir | awk \'{print $1}\'` | 7za a -si $outfile


回答9:

I would also like to contribute my own progress bar

It achieves sub-character precision by using Half unicode blocks

\"enter

Code is included



回答10:

This lets you visualize that a command is still executing:

while :;do echo -n .;sleep 1;done &
trap \"kill $!\" EXIT  #Die with parent if we die prematurely
tar zxf packages.tar.gz; # or any other command here
kill $! && trap \" \" EXIT #Kill the loop and unset the trap or else the pid might get reassigned and we might end up killing a completely different process

This will create an infinite while loop that executes in the background and echoes a \".\" every second. This will display . in the shell. Run the tar command or any a command you want. When that command finishes executing then kill the last job running in the background - which is the infinite while loop.



回答11:

Haven\'t seen anything similar so... my very simple solution:

#!/bin/bash
BAR=\'####################\'   # this is full bar, mine is 20 chars
for i in {1..20}; do
    echo -ne \"\\r${BAR:0:$i}\" # print $i chars of $BAR from 0 position
    sleep .1
done
  • echo -n - print without new line at the end
  • echo -e - interpret special characters while printing
  • \"\\r\" - carriage return, a special char to return to the beginning of the line

I used it long time ago in a simple \"hacking video\" to simulate typing code. ;)



回答12:

My solution displays the percentage of the tarball that is currently being uncompressed and written. I use this when writing out 2GB root filesystem images. You really need a progress bar for these things. What I do is use gzip --list to get the total uncompressed size of the tarball. From that I calculate the blocking-factor needed to divide the file into 100 parts. Finally, I print a checkpoint message for each block. For a 2GB file this gives about 10MB a block. If that is too big then you can divide the BLOCKING_FACTOR by 10 or 100, but then it\'s harder to print pretty output in terms of a percentage.

Assuming you are using Bash then you can use the following shell function

untar_progress () 
{ 
  TARBALL=$1
  BLOCKING_FACTOR=$(gzip --list ${TARBALL} |
    perl -MPOSIX -ane \'$.==2 && print ceil $F[1]/50688\')
  tar --blocking-factor=${BLOCKING_FACTOR} --checkpoint=1 \\
    --checkpoint-action=\'ttyout=Wrote %u%  \\r\' -zxf ${TARBALL}
}


回答13:

First of all bar is not the only one pipe progress meter. The other (maybe even more known) is pv (pipe viewer).

Secondly bar and pv can be used for example like this:

$ bar file1 | wc -l 
$ pv file1 | wc -l

or even:

$ tail -n 100 file1 | bar | wc -l
$ tail -n 100 file1 | pv | wc -l

one useful trick if you want to make use of bar and pv in commands that are working with files given in arguments, like e.g. copy file1 file2, is to use process substitution:

$ copy <(bar file1) file2
$ copy <(pv file1) file2

Process substitution is a bash magic thing that creates temporary fifo pipe files /dev/fd/ and connect stdout from runned process (inside parenthesis) through this pipe and copy sees it just like an ordinary file (with one exception, it can only read it forwards).

Update:

bar command itself allows also for copying. After man bar:

bar --in-file /dev/rmt/1cbn --out-file \\
     tape-restore.tar --size 2.4g --buffer-size 64k

But process substitution is in my opinion more generic way to do it. An it uses cp program itself.



回答14:

Most unix commands will not give you the sort of direct feedback from which you can do this. Some will give you output on stdout or stderr that you can use.

For something like tar you could use the -v switch and pipe the output to a program that updates a small animation for each line it reads. As tar writes out a list of files it\'s unravelled the program can update the animation. To do a percent complete you would have to know the number of files and count the lines.

cp doesn\'t give this sort of output as far as I know. To monitor the progress of cp you would have to monitor the source and destination files and watch the size of the destination. You could write a small c program using the stat (2) system call to get the file size. This would read the size of the source then poll the destination file and update a % complete bar based on the size of the file written to date.



回答15:

I prefer to use dialog with the --gauge param. Is used very often in .deb package installations and other basic configuration stuff of many distros. So you don\'t need to reinvent the wheel... again

Just put an int value from 1 to 100 @stdin. One basic and silly example:

for a in {1..100}; do sleep .1s; echo $a| dialog --gauge \"waiting\" 7 30; done

I have this /bin/Wait file (with chmod u+x perms) for cooking purposes :P

#!/bin/bash
INIT=`/bin/date +%s`
NOW=$INIT
FUTURE=`/bin/date -d \"$1\" +%s`
[ $FUTURE -a $FUTURE -eq $FUTURE ] || exit
DIFF=`echo \"$FUTURE - $INIT\"|bc -l`

while [ $INIT -le $FUTURE -a $NOW -lt $FUTURE ]; do
    NOW=`/bin/date +%s`
    STEP=`echo \"$NOW - $INIT\"|bc -l`
    SLEFT=`echo \"$FUTURE - $NOW\"|bc -l`
    MLEFT=`echo \"scale=2;$SLEFT/60\"|bc -l`
    TEXT=\"$SLEFT seconds left ($MLEFT minutes)\";
    TITLE=\"Waiting $1: $2\"
    sleep 1s
    PTG=`echo \"scale=0;$STEP * 100 / $DIFF\"|bc -l`
    echo $PTG| dialog --title \"$TITLE\" --gauge \"$TEXT\" 7 72
done

if [ \"$2\" == \"\" ]; then msg=\"Espera terminada: $1\";audio=\"Listo\";
else msg=$2;audio=$2;fi 

/usr/bin/notify-send --icon=stock_appointment-reminder-excl \"$msg\"
espeak -v spanish \"$audio\"

So I can put:

Wait \"34 min\" \"warm up the oven\"

or

Wait \"dec 31\" \"happy new year\"



回答16:

for me easiest to use and best looking so far is command pv or bar like some guy already wrote

for example: need to make a backup of entire drive with dd

normally you use dd if=\"$input_drive_path\" of=\"$output_file_path\"

with pv you can make it like this :

dd if=\"$input_drive_path\" | pv | dd of=\"$output_file_path\"

and the progress goes directly to STDOUT as this:

    7.46GB 0:33:40 [3.78MB/s] [  <=>                                            ]

after it is done summary comes up

    15654912+0 records in
    15654912+0 records out
    8015314944 bytes (8.0 GB) copied, 2020.49 s, 4.0 MB/s


回答17:

Many answers describe writing your own commands for printing out \'\\r\' + $some_sort_of_progress_msg. The problem sometimes is that printing out hundreds of these updates per second will slow down the process.

However, if any of your processes produce output (eg 7z a -r newZipFile myFolder will output each filename as it compresses it) then a simpler, fast, painless and customisable solution exists.

Install the python module tqdm.

$ sudo pip install tqdm
$ # now have fun
$ 7z a -r -bd newZipFile myFolder | tqdm >> /dev/null
$ # if we know the expected total, we can have a bar!
$ 7z a -r -bd newZipFile myFolder | grep -o Compressing | tqdm --total $(find myFolder -type f | wc -l) >> /dev/null

Help: tqdm -h. An example using more options:

$ find / -name \'*.py\' -exec cat \\{} \\; | tqdm --unit loc --unit_scale True | wc -l

As a bonus you can also use tqdm to wrap iterables in python code.

https://github.com/tqdm/tqdm/blob/master/README.rst#module



回答18:

This is only applicable using gnome zenity. Zenity provides a great native interface to bash scripts: https://help.gnome.org/users/zenity/stable/

From Zenity Progress Bar Example:

#!/bin/sh
(
echo \"10\" ; sleep 1
echo \"# Updating mail logs\" ; sleep 1
echo \"20\" ; sleep 1
echo \"# Resetting cron jobs\" ; sleep 1
echo \"50\" ; sleep 1
echo \"This line will just be ignored\" ; sleep 1
echo \"75\" ; sleep 1
echo \"# Rebooting system\" ; sleep 1
echo \"100\" ; sleep 1
) |
zenity --progress \\
  --title=\"Update System Logs\" \\
  --text=\"Scanning mail logs...\" \\
  --percentage=0

if [ \"$?\" = -1 ] ; then
        zenity --error \\
          --text=\"Update canceled.\"
fi


回答19:

I used an answer from Creating string of repeated characters in shell script for char repeating. I have two relatively small bash versions for scripts that need to display progress bar (for example, a loop that goes through many files, but not useful for big tar files or copy operations). The faster one consists of two functions, one to prepare the strings for bar display:

preparebar() {
# $1 - bar length
# $2 - bar char
    barlen=$1
    barspaces=$(printf \"%*s\" \"$1\")
    barchars=$(printf \"%*s\" \"$1\" | tr \' \' \"$2\")
}

and one to display a progress bar:

progressbar() {
# $1 - number (-1 for clearing the bar)
# $2 - max number
    if [ $1 -eq -1 ]; then
        printf \"\\r  $barspaces\\r\"
    else
        barch=$(($1*barlen/$2))
        barsp=$((barlen-barch))
        printf \"\\r[%.${barch}s%.${barsp}s]\\r\" \"$barchars\" \"$barspaces\"
    fi
}

It could be used as:

preparebar 50 \"#\"

which means prepare strings for bar with 50 \"#\" characters, and after that:

progressbar 35 80

will display the number of \"#\" characters that corresponds to 35/80 ratio:

[#####################                             ]

Be aware that function displays the bar on the same line over and over until you (or some other program) prints a newline. If you put -1 as first parameter, the bar would be erased:

progressbar -1 80

The slower version is all in one function:

progressbar() {
# $1 - number
# $2 - max number
# $3 - number of \'#\' characters
    if [ $1 -eq -1 ]; then
        printf \"\\r  %*s\\r\" \"$3\"
    else
        i=$(($1*$3/$2))
        j=$(($3-i))
        printf \"\\r[%*s\" \"$i\" | tr \' \' \'#\'
        printf \"%*s]\\r\" \"$j\"
    fi
}

and it can be used as (the same example as above):

progressbar 35 80 50

If you need progressbar on stderr, just add >&2 at the end of each printf command.



回答20:

To indicate progress of activity, try the following commands:

while true; do sleep 0.25 && echo -ne \"\\r\\\\\" && sleep 0.25 && echo -ne \"\\r|\" && sleep 0.25 && echo -ne \"\\r/\" && sleep 0.25 && echo -ne \"\\r-\"; done;

OR

while true; do sleep 0.25 && echo -ne \"\\rActivity: \\\\\" && sleep 0.25 && echo -ne \"\\rActivity: |\" && sleep 0.25 && echo -ne \"\\rActivity: /\" && sleep 0.25 && echo -ne \"\\rActivity: -\"; done;

OR

while true; do sleep 0.25 && echo -ne \"\\r\" && sleep 0.25 && echo -ne \"\\r>\" && sleep 0.25 && echo -ne \"\\r>>\" && sleep 0.25 && echo -ne \"\\r>>>\"; sleep 0.25 && echo -ne \"\\r>>>>\"; done;

OR

while true; do sleep .25 && echo -ne \"\\r:Active:\" && sleep .25 && echo -ne \"\\r:aCtive:\" && sleep .25 && echo -ne \"\\r:acTive:\" && sleep .25 && echo -ne \"\\r:actIve:\" && sleep .25 && echo -ne \"\\r:actiVe:\" && sleep .25 && echo -ne \"\\r:activE:\"; done;

One can use flags/variables inside the while loop to check and display the value/extent of progress.



回答21:

Based on the work of Edouard Lopez, I created a progress bar that fits the size of the screen, whatever it is. Check it out.

\"enter

It\'s also posted on Git Hub.

#!/bin/bash
#
# Progress bar by Adriano Pinaffo
# Available at https://github.com/adriano-pinaffo/progressbar.sh
# Inspired on work by Edouard Lopez (https://github.com/edouard-lopez/progress-bar.sh)
# Version 1.0
# Date April, 28th 2017

function error {
  echo \"Usage: $0 [SECONDS]\"
  case $1 in
    1) echo \"Pass one argument only\"
    exit 1
    ;;
    2) echo \"Parameter must be a number\"
    exit 2
    ;;
    *) echo \"Unknown error\"
    exit 999
  esac
}

[[ $# -ne 1 ]] && error 1
[[ $1 =~ ^[0-9]+$ ]] || error 2

duration=${1}
barsize=$((`tput cols` - 7))
unity=$(($barsize / $duration))
increment=$(($barsize%$duration))
skip=$(($duration/($duration-$increment)))
curr_bar=0
prev_bar=
for (( elapsed=1; elapsed<=$duration; elapsed++ ))
do
  # Elapsed
prev_bar=$curr_bar
  let curr_bar+=$unity
  [[ $increment -eq 0 ]] || {  
    [[ $skip -eq 1 ]] &&
      { [[ $(($elapsed%($duration/$increment))) -eq 0 ]] && let curr_bar++; } ||
    { [[ $(($elapsed%$skip)) -ne 0 ]] && let curr_bar++; }
  }
  [[ $elapsed -eq 1 && $increment -eq 1 && $skip -ne 1 ]] && let curr_bar++
  [[ $(($barsize-$curr_bar)) -eq 1 ]] && let curr_bar++
  [[ $curr_bar -lt $barsize ]] || curr_bar=$barsize
  for (( filled=0; filled<=$curr_bar; filled++ )); do
    printf \"▇\"
  done

  # Remaining
  for (( remain=$curr_bar; remain<$barsize; remain++ )); do
    printf \" \"
  done

  # Percentage
  printf \"| %s%%\" $(( ($elapsed*100)/$duration))

  # Return
  sleep 1
  printf \"\\r\"
done
printf \"\\n\"
exit 0

Enjoy



回答22:

Using suggestions listed above, I decided to implement my own progress bar.

#!/usr/bin/env bash

main() {
  for (( i = 0; i <= 100; i=$i + 1)); do
    progress_bar \"$i\"
    sleep 0.1;
  done
  progress_bar \"done\"
  exit 0
}

progress_bar() {
  if [ \"$1\" == \"done\" ]; then
    spinner=\"X\"
    percent_done=\"100\"
    progress_message=\"Done!\"
    new_line=\"\\n\"
  else
    spinner=\'/-\\|\'
    percent_done=\"${1:-0}\"
    progress_message=\"$percent_done %\"
  fi

  percent_none=\"$(( 100 - $percent_done ))\"
  [ \"$percent_done\" -gt 0 ] && local done_bar=\"$(printf \'#%.0s\' $(seq -s \' \' 1 $percent_done))\"
  [ \"$percent_none\" -gt 0 ] && local none_bar=\"$(printf \'~%.0s\' $(seq -s \' \' 1 $percent_none))\"

  # print the progress bar to the screen
  printf \"\\r Progress: [%s%s] %s %s${new_line}\" \\
    \"$done_bar\" \\
    \"$none_bar\" \\
    \"${spinner:x++%${#spinner}:1}\" \\
    \"$progress_message\"
}

main \"$@\"


回答23:

APT style progress bar (Does not break normal output)

\"enter

EDIT: For an updated version check my github page

I was not satisfied with the responses on this question. What I was personally looking for was a fancy progress bar as is seen by APT.

I had a look at the C source code for APT and decided to write my own equivalent for bash.

This progress bar will stay nicely at the bottom of the terminal and will not interfere with any output sent to the terminal.

Please do note that the bar is currently fixed at 100 characters wide. If you want scale it to the size of the terminal, this is fairly easy to accomplish as well (The updated version on my github page handles this well).

I will post my script here. Usage example:

source ./progress_bar.sh
echo \"This is some output\"
setup_scroll_area
sleep 1
echo \"This is some output 2\"
draw_progress_bar 10
sleep 1
echo \"This is some output 3\"
draw_progress_bar 50
sleep 1
echo \"This is some output 4\"
draw_progress_bar 90
sleep 1
echo \"This is some output 5\"
destroy_scroll_area

The script (I strongly recommend the version on my github instead):

#!/bin/bash

# This code was inspired by the open source C code of the APT progress bar
# http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/trusty/apt/trusty/view/head:/apt-pkg/install-progress.cc#L233

#
# Usage:
# Source this script
# setup_scroll_area
# draw_progress_bar 10
# draw_progress_bar 90
# destroy_scroll_area
#


CODE_SAVE_CURSOR=\"\\033[s\"
CODE_RESTORE_CURSOR=\"\\033[u\"
CODE_CURSOR_IN_SCROLL_AREA=\"\\033[1A\"
COLOR_FG=\"\\e[30m\"
COLOR_BG=\"\\e[42m\"
RESTORE_FG=\"\\e[39m\"
RESTORE_BG=\"\\e[49m\"

function setup_scroll_area() {
    lines=$(tput lines)
    let lines=$lines-1
    # Scroll down a bit to avoid visual glitch when the screen area shrinks by one row
    echo -en \"\\n\"

    # Save cursor
    echo -en \"$CODE_SAVE_CURSOR\"
    # Set scroll region (this will place the cursor in the top left)
    echo -en \"\\033[0;${lines}r\"

    # Restore cursor but ensure its inside the scrolling area
    echo -en \"$CODE_RESTORE_CURSOR\"
    echo -en \"$CODE_CURSOR_IN_SCROLL_AREA\"

    # Start empty progress bar
    draw_progress_bar 0
}

function destroy_scroll_area() {
    lines=$(tput lines)
    # Save cursor
    echo -en \"$CODE_SAVE_CURSOR\"
    # Set scroll region (this will place the cursor in the top left)
    echo -en \"\\033[0;${lines}r\"

    # Restore cursor but ensure its inside the scrolling area
    echo -en \"$CODE_RESTORE_CURSOR\"
    echo -en \"$CODE_CURSOR_IN_SCROLL_AREA\"

    # We are done so clear the scroll bar
    clear_progress_bar

    # Scroll down a bit to avoid visual glitch when the screen area grows by one row
    echo -en \"\\n\\n\"
}

function draw_progress_bar() {
    percentage=$1
    lines=$(tput lines)
    let lines=$lines
    # Save cursor
    echo -en \"$CODE_SAVE_CURSOR\"

    # Move cursor position to last row
    echo -en \"\\033[${lines};0f\"

    # Clear progress bar
    tput el

    # Draw progress bar
    print_bar_text $percentage

    # Restore cursor position
    echo -en \"$CODE_RESTORE_CURSOR\"
}

function clear_progress_bar() {
    lines=$(tput lines)
    let lines=$lines
    # Save cursor
    echo -en \"$CODE_SAVE_CURSOR\"

    # Move cursor position to last row
    echo -en \"\\033[${lines};0f\"

    # clear progress bar
    tput el

    # Restore cursor position
    echo -en \"$CODE_RESTORE_CURSOR\"
}

function print_bar_text() {
    local percentage=$1

    # Prepare progress bar
    let remainder=100-$percentage
    progress_bar=$(echo -ne \"[\"; echo -en \"${COLOR_FG}${COLOR_BG}\"; printf_new \"#\" $percentage; echo -en \"${RESTORE_FG}${RESTORE_BG}\"; printf_new \".\" $remainder; echo -ne \"]\");

    # Print progress bar
    if [ $1 -gt 99 ]
    then
        echo -ne \"${progress_bar}\"
    else
        echo -ne \"${progress_bar}\"
    fi
}

printf_new() {
    str=$1
    num=$2
    v=$(printf \"%-${num}s\" \"$str\")
    echo -ne \"${v// /$str}\"
}


回答24:

I did a pure shell version for an embedded system taking advantage of:

  • /usr/bin/dd\'s SIGUSR1 signal handling feature.

    Basically, if you send a \'kill SIGUSR1 $(pid_of_running_dd_process)\', it\'ll output a summary of throughput speed and amount transferred.

  • backgrounding dd and then querying it regularly for updates, and generating hash ticks like old-school ftp clients used to.

  • Using /dev/stdout as the destination for non-stdout friendly programs like scp

The end result allows you to take any file transfer operation and get progress update that looks like old-school FTP \'hash\' output where you\'d just get a hash mark for every X bytes.

This is hardly production quality code, but you get the idea. I think it\'s cute.

For what it\'s worth, the actual byte-count might not be reflected correctly in the number of hashes - you may have one more or less depending on rounding issues. Don\'t use this as part of a test script, it\'s just eye-candy. And, yes, I\'m aware this is terribly inefficient - it\'s a shell script and I make no apologies for it.

Examples with wget, scp and tftp provided at the end. It should work with anything that has emits data. Make sure to use /dev/stdout for programs that aren\'t stdout friendly.

#!/bin/sh
#
# Copyright (C) Nathan Ramella (nar+progress-script@remix.net) 2010 
# LGPLv2 license
# If you use this, send me an email to say thanks and let me know what your product
# is so I can tell all my friends I\'m a big man on the internet!

progress_filter() {

        local START=$(date +\"%s\")
        local SIZE=1
        local DURATION=1
        local BLKSZ=51200
        local TMPFILE=/tmp/tmpfile
        local PROGRESS=/tmp/tftp.progress
        local BYTES_LAST_CYCLE=0
        local BYTES_THIS_CYCLE=0

        rm -f ${PROGRESS}

        dd bs=$BLKSZ of=${TMPFILE} 2>&1 \\
                | grep --line-buffered -E \'[[:digit:]]* bytes\' \\
                | awk \'{ print $1 }\' >> ${PROGRESS} &

        # Loop while the \'dd\' exists. It would be \'more better\' if we
        # actually looked for the specific child ID of the running 
        # process by identifying which child process it was. If someone
        # else is running dd, it will mess things up.

        # My PID handling is dumb, it assumes you only have one running dd on
        # the system, this should be fixed to just get the PID of the child
        # process from the shell.

        while [ $(pidof dd) -gt 1 ]; do

                # PROTIP: You can sleep partial seconds (at least on linux)
                sleep .5    

                # Force dd to update us on it\'s progress (which gets
                # redirected to $PROGRESS file.
                # 
                # dumb pid handling again
                pkill -USR1 dd

                local BYTES_THIS_CYCLE=$(tail -1 $PROGRESS)
                local XFER_BLKS=$(((BYTES_THIS_CYCLE-BYTES_LAST_CYCLE)/BLKSZ))

                # Don\'t print anything unless we\'ve got 1 block or more.
                # This allows for stdin/stderr interactions to occur
                # without printing a hash erroneously.

                # Also makes it possible for you to background \'scp\',
                # but still use the /dev/stdout trick _even_ if scp
                # (inevitably) asks for a password. 
                #
                # Fancy!

                if [ $XFER_BLKS -gt 0 ]; then
                        printf \"#%0.s\" $(seq 0 $XFER_BLKS)
                        BYTES_LAST_CYCLE=$BYTES_THIS_CYCLE
                fi
        done

        local SIZE=$(stat -c\"%s\" $TMPFILE)
        local NOW=$(date +\"%s\")

        if [ $NOW -eq 0 ]; then
                NOW=1
        fi

        local DURATION=$(($NOW-$START))
        local BYTES_PER_SECOND=$(( SIZE / DURATION ))
        local KBPS=$((SIZE/DURATION/1024))
        local MD5=$(md5sum $TMPFILE | awk \'{ print $1 }\')

        # This function prints out ugly stuff suitable for eval() 
        # rather than a pretty string. This makes it a bit more 
        # flexible if you have a custom format (or dare I say, locale?)

        printf \"\\nDURATION=%d\\nBYTES=%d\\nKBPS=%f\\nMD5=%s\\n\" \\
            $DURATION \\
            $SIZE \\
            $KBPS \\
            $MD5
}

Examples:

echo \"wget\"
wget -q -O /dev/stdout http://www.blah.com/somefile.zip | progress_filter

echo \"tftp\"
tftp -l /dev/stdout -g -r something/firmware.bin 192.168.1.1 | progress_filter

echo \"scp\"
scp user@192.168.1.1:~/myfile.tar /dev/stdout | progress_filter


回答25:

In case you have to show a temporal progress bar (by knowing in advance the showing time), you can use Python as follows:

#!/bin/python
from time import sleep
import sys

if len(sys.argv) != 3:
    print \"Usage:\", sys.argv[0], \"<total_time>\", \"<progressbar_size>\"
    exit()

TOTTIME=float(sys.argv[1])
BARSIZE=float(sys.argv[2])

PERCRATE=100.0/TOTTIME
BARRATE=BARSIZE/TOTTIME

for i in range(int(TOTTIME)+1):
    sys.stdout.write(\'\\r\')
    s = \"[%-\"+str(int(BARSIZE))+\"s] %d%% \"
    sys.stdout.write(s % (\'=\'*int(BARRATE*i), int(PERCRATE*i)))
    sys.stdout.flush()
    SLEEPTIME = 1.0
    if i == int(TOTTIME): SLEEPTIME = 0.1
    sleep(SLEEPTIME)
print \"\"

Then, assuming you saved the Python script as progressbar.py, it\'s possible to show the progress bar from your bash script by running the following command:

python progressbar.py 10 50

It would show a progress bar sized 50 characters and \"running\" for 10 seconds.



回答26:

I have built on the answer provided by fearside

This connects to an Oracle database to retrieve the progress of an RMAN restore.

#!/bin/bash

 # 1. Create ProgressBar function
 # 1.1 Input is currentState($1) and totalState($2)
 function ProgressBar {
 # Process data
let _progress=(${1}*100/${2}*100)/100
let _done=(${_progress}*4)/10
let _left=40-$_done
# Build progressbar string lengths
_fill=$(printf \"%${_done}s\")
_empty=$(printf \"%${_left}s\")

# 1.2 Build progressbar strings and print the ProgressBar line
# 1.2.1 Output example:
# 1.2.1.1 Progress : [########################################] 100%
printf \"\\rProgress : [${_fill// /#}${_empty// /-}] ${_progress}%%\"

}

function rman_check {
sqlplus -s / as sysdba <<EOF
set heading off
set feedback off
select
round((sofar/totalwork) * 100,0) pct_done
from
v\\$session_longops
where
totalwork > sofar
AND
opname NOT LIKE \'%aggregate%\'
AND
opname like \'RMAN%\';
exit
EOF
}

# Variables
_start=1

# This accounts as the \"totalState\" variable for the ProgressBar function
_end=100

_rman_progress=$(rman_check)
#echo ${_rman_progress}

# Proof of concept
#for number in $(seq ${_start} ${_end})

while [ ${_rman_progress} -lt 100 ]
do

for number in _rman_progress
do
sleep 10
ProgressBar ${number} ${_end}
done

_rman_progress=$(rman_check)

done
printf \'\\nFinished!\\n\'


回答27:

#!/bin/bash

function progress_bar() {
    bar=\"\"
    total=10
    [[ -z $1 ]] && input=0 || input=${1}
    x=\"##\"
   for i in `seq 1 10`; do
        if [ $i -le $input ] ;then
            bar=$bar$x
        else
            bar=\"$bar  \"
       fi
    done
    #pct=$((200*$input/$total % 2 + 100*$input/$total))
    pct=$(($input*10))
    echo -ne \"Progress : [ ${bar} ] (${pct}%) \\r\"    
    sleep 1
    if [ $input -eq 10 ] ;then
        echo -ne \'\\n\'
    fi

}

could create a function that draws this on a scale say 1-10 for the number of bars :

progress_bar 1
echo \"doing something ...\"
progress_bar 2
echo \"doing something ...\"
progress_bar 3
echo \"doing something ...\"
progress_bar 8
echo \"doing something ...\"
progress_bar 10


回答28:

Here is how it might look

Uploading a file

[##################################################] 100% (137921 / 137921 bytes)

Waiting for a job to complete

[#########################                         ] 50% (15 / 30 seconds)

Simple function that implements it

You can just copy-paste it to your script. It does not require anything else to work.

PROGRESS_BAR_WIDTH=50  # progress bar length in characters

draw_progress_bar() {
  # Arguments: current value, max value, unit of measurement (optional)
  local __value=$1
  local __max=$2
  local __unit=${3:-\"\"}  # if unit is not supplied, do not display it

  # Calculate percentage
  if (( $__max < 1 )); then __max=1; fi  # anti zero division protection
  local __percentage=$(( 100 - ($__max*100 - $__value*100) / $__max ))

  # Rescale the bar according to the progress bar width
  local __num_bar=$(( $__percentage * $PROGRESS_BAR_WIDTH / 100 ))

  # Draw progress bar
  printf \"[\"
  for b in $(seq 1 $__num_bar); do printf \"#\"; done
  for s in $(seq 1 $(( $PROGRESS_BAR_WIDTH - $__num_bar ))); do printf \" \"; done
  printf \"] $__percentage%% ($__value / $__max $__unit)\\r\"
}

Usage example

Here, we upload a file and redraw the progress bar at each iteration. It does not matter what job is actually performed as long as we can get 2 values: max value and current value.

In the example below the max value is file_size and the current value is supplied by some function and is called uploaded_bytes.

# Uploading a file
file_size=137921

while true; do
  # Get current value of uploaded bytes
  uploaded_bytes=$(some_function_that_reports_progress)

  # Draw a progress bar
  draw_progress_bar $uploaded_bytes $file_size \"bytes\"

  # Check if we reached 100%
  if [ $uploaded_bytes == $file_size ]; then break; fi
  sleep 1  # Wait before redrawing
done
# Go to the newline at the end of upload
printf \"\\n\"


回答29:

#!/bin/bash
tot=$(wc -c /proc/$$/fd/255 | awk \'/ /{print $1}\')
now() {
echo $(( 100* ($(awk \'/^pos:/{print $2}\' < /proc/$$/fdinfo/255)-166) / (tot-166) )) \"%\"
}
now;
now;
now;
now;
now;
now;
now;
now;
now;

output:

0 %
12 %
25 %
37 %
50 %
62 %
75 %
87 %
100 %

note: if instead of 255 you put 1 you will monitor the standard in...with 2 the standard out (but you must modify the source to set \"tot\" to the projected output file size)



回答30:

To make a tar progress bar

tar xzvf pippo.tgz |xargs -L 19 |xargs -I@ echo -n \".\"

Where \"19\" is the number of files in the tar divided the length of the intended progress bar. Example: the .tgz contains 140 files and you\'ll want a progress bar of 76 \".\", you can put -L 2.

You\'ll need nothing else.



标签: bash shell zsh