How can I execute parallel “for” loops in Bash?

问题:

I have been trying to parallelize the following script, specifically the for loop. How can I do that?

#!/bin/bash
for i in `cat /root/vms`;
do
    /usr/bin/sshpass -p 'test' /usr/bin/ssh -o StrictHostKeyChecking=no \
        -l testuser $i -t 'echo test | sudo -S yum update -y'
done

回答1:

Replace

/usr/bin/sshpass ...

with

/usr/bin/sshpass ... &

回答2:

You can do it quite succinctly with GNU Parallel like this:

parallel -a /root/vms /usr/bin/sshpass -p \'test\' /usr/bin/ssh -o StrictHostKeyChecking=no -l testuser {} -t \'echo test \| sudo -S yum update -y\'

So, if your /root/vms contains:

vm-ubuntuLTS
vm-centos
vm-debian
vm-arch

and you add the --dry-run option to see what it would do, without actually doing anything:

parallel --dry-run -a /root/vms /usr/bin/sshpass -p \'test\' /usr/bin/ssh -o StrictHostKeyChecking=no -l testuser {} -t \'echo test \| sudo -S yum update -y\'

Sample Output

/usr/bin/sshpass -p 'test' /usr/bin/ssh -o StrictHostKeyChecking=no -l testuser vm-debian -t 'echo test | sudo -S yum update -y'
/usr/bin/sshpass -p 'test' /usr/bin/ssh -o StrictHostKeyChecking=no -l testuser vm-centos -t 'echo test | sudo -S yum update -y'
/usr/bin/sshpass -p 'test' /usr/bin/ssh -o StrictHostKeyChecking=no -l testuser vm-ubuntuLTS -t 'echo test | sudo -S yum update -y'
/usr/bin/sshpass -p 'test' /usr/bin/ssh -o StrictHostKeyChecking=no -l testuser vm-arch -t 'echo test | sudo -S yum update -y'

---

Rather than repeating all your ssh options, consider putting them into a file at $HOME/.ssh/config like this:

Host vm-centos
HostName vm-centos
User freddy
   StrictHostKeyChecking no

Host vm-arch
HostName vm-arch
   User frog
   Port 2222
   ServerAliveInterval 10

回答3:

GNU Parallel has a --nonall option and an environment variable to set the ssh-command to use:

PARALLEL_SSH="/usr/bin/sshpass -p test /usr/bin/ssh -o StrictHostKeyChecking=no -l testuser -t"
export PARALLEL_SSH
parallel --slf /root/vms --nonall 'echo test | sudo -S yum update -y'

If the things you want to run is a bit more complex, you can make a function and have GNU Parallel transfer that:

complex_task() {
  # Do complex task
  echo test | sudo -S yum update -y
  # and a lot more complex stuff
}
export -f complex_task
PARALLEL_SSH="/usr/bin/sshpass -p test /usr/bin/ssh -o StrictHostKeyChecking=no -l testuser -t"
export PARALLEL_SSH
parallel --slf /root/vms --env complex_task --nonall complex_task