Proper shell execution in PHP

Question

The problem

I was using a function that made use of proc_open() to invoke shell commands. It seems the way I was doing STDIO was wrong and sometimes caused PHP or the target command to lock up. This is the original code:

function execute($cmd, $stdin=null){
    $proc=proc_open($cmd,array(0=>array('pipe','r'),1=>array('pipe','w'),2=>array('pipe','w')),$pipes);
    fwrite($pipes[0],$stdin);                fclose($pipes[0]);
    $stdout=stream_get_contents($pipes[1]);  fclose($pipes[1]);
    $stderr=stream_get_contents($pipes[2]);  fclose($pipes[2]);
    return array( 'stdout'=>$stdout, 'stderr'=>$stderr, 'return'=>proc_close($proc) );
}

It works most of the time, but that is not enough, I want to make it work always.

The issue lies in stream_get_contents() locking up if the STDIO buffers exceed 4k of data.

Test Case

function out($data){
    file_put_contents('php://stdout',$data);
}
function err($data){
    file_put_contents('php://stderr',$data);
}
if(isset($argc)){
    // RUN CLI TESTCASE
    out(str_repeat('o',1030);
    err(str_repeat('e',1030);
    out(str_repeat('O',1030);
    err(str_repeat('E',1030);
    die(128); // to test return error code
}else{
    // RUN EXECUTION TEST CASE
    $res=execute('php -f '.escapeshellarg(__FILE__));
}

We output a string twice to STDERR and STDOUT with the combined length of 4120 bytes (exceeding 4k). This causes PHP to lock up on both sides.

Solution

Apparently, stream_select() is the way to go. I have the following code:

function execute($cmd,$stdin=null,$timeout=20000){
    $proc=proc_open($cmd,array(0=>array('pipe','r'),1=>array('pipe','w'),2=>array('pipe','w')),$pipes);
    $write  = array($pipes[0]);
    $read   = array($pipes[1], $pipes[2]);
    $except = null;
    $stdout = '';
    $stderr = '';
    while($r = stream_select($read, $write, $except, null, $timeout)){
        foreach($read as $stream){

            // handle STDOUT
            if($stream===$pipes[1])
/*...*/         $stdout.=stream_get_contents($stream);

            // handle STDERR
            if($stream===$pipes[2])
/*...*/         $stderr.=stream_get_contents($stream);
        }

        // Handle STDIN (???)
        if(isset($write[0])) ;

// the following code is temporary
$n=isset($n) ? $n+1 : 0; if($n>10)break; // break while loop after 10 iterations

    }
}

The only remaining piece of the puzzle is handling STDIN (see the line marked (???)). I figured out STDIN must be supplied by whatever is calling my function, execute(). But what if I don't want to use STDIN at all? In my testcase, above, I didn't ask for input, yet I'm supposed to do something to STDIN.

That said, the above approach still freezes at stream_get_contents(). I'm quite unsure what to do/try next.

Credits

The solution was suggested by Jakob Truelsen, as well as discovering the original issue. The 4k tip was also his idea. Prior to this I was puzzled as to why the function was working fine (didn't know it all depended on buffer size).

Answer 1

while($r = stream_select($read, $write, $except, null, $timeout)){

As far as I know this will set $r to the number of changed streams, which may be 0 and the loop would no longer continue. I would personally recode this as described in the PHP manual:

while(false !== ($r = stream_select($read, $write, $except, null, $timeout))){

As far as your STDIN is concerned if your process is not interactive then the STDIN may not be necessary. What is the process you are executing?

Answer 2

The whole problem with hanging in stream_get_contents is in the way how process is created. The correct way is to open STDOUT with read/write mode of pipe, eg:

$descriptor = array (0 => array ("pipe", "r"), 1 => array ("pipe", "rw"), 2 => array ("pipe", "rw"));
//Open the resource to execute $command
$t->pref = proc_open($command,$descriptor,$t->pipes);
//Set STDOUT and STDERR to non-blocking 
stream_set_blocking ($t->pipes[0], 0);
stream_set_blocking ($t->pipes[1], 0);

This is obvious that when stream_get_contents wants to read the STDOUT pipe it needs read mode. The same bug with hang/freeze/block is in this nice class https://gist.github.com/Arbow/982320

Then blocking disappears. But read does not read nothing.

Answer 3

You've missed this note in the PHP manual for stream_select():

When stream_select() returns, the arrays read, write and except are modified to indicate which stream resource(s) actually changed status.

You need to re-create the arrays before calling stream_select() each time.

Depending on the process you're opening, this may be why your example still blocks.

Answer 4

Well, seems a year passed and forgot this thing is still pending!

However, I wrapped up this mess in a nice PHP class which you can find on Github.

The main remaining problem is that reading STDERR causes the PHP script to block, so it has been disabled.

On the bright side, thanks to events and some nice coding (I hope!), one can actually interact with the process being executed (hence the class name, InterExec). So you can have bot-style behavior in PHP.