perl: executing multiple systems processes and wai

Question

Currently in my Perl script I make a call like the following:

system(" ./long_program1 & ./long_program2 & ./long_program3 & wait ");

I would like to be able to log when each of the long running commands executes while still executing them asyncronously. I know that the system call causes perl to make a fork, so is something like this possible? Could this be replaced by multiple perl fork() and exec() calls?

Please help me find a better solution.

Answer 1

Two alternatives:

---

use IPC::Open3 qw( open3 );

sub launch {
   open(local *CHILD_STDIN, '<', '/dev/null') or die $!;
   return open3('<&CHILD_STDIN', '>&STDOUT', '>&STDERR', @_);
}

my %children;
for my $cmd (@cmds) {
   print "Command $cmd started at ".localtime."\n";
   my $pid = launch($cmd);
   $children{$pid} = $cmd;
}

while (%children) {
   my $pid = wait();
   die $! if $pid < 1;
   my $cmd = delete($children{$pid});
   print "Command $cmd ended at ".localtime." with \$? = $?."\n";
}

I use open3 since it it's shorter than a even trivial fork+exec and since it doesn't misattribute exec errors to the command you launch like a trivial fork+exec.

---

use threads;

my @threads;
for my $cmd (@cmds) {
   push @threads, async {
      print "Command $cmd started at ".localtime."\n";
      system($cmd);
      print "Command $cmd ended at ".localtime." with \$? = $?."\n";
   };
}

$_->join() for @threads;

Answer 2

Yes, definitely. You can fork off a child process for each of the programs to be executed.

You can either do system() or exec() after forking, depending on how much processing you want your Perl code to do after the system call finishes (since exec() is very similar in functionality to system(); exit $rc;)

foreach my $i (1, 2, 3) {
    my $pid = fork();
    if ($pid==0) { # child
        exec("./long_program$i");
        die "Exec $i failed: $!\n";
    } elsif (!defined $pid) {
        warn "Fork $i failed: $!\n";
    }
}

1 while wait() >= 0;

Please note that if you need to do a lot of forks, you are better off controlling them via Parallel::ForkManager instead of doing forking by hand.