Sync local and remote folders using rsync from php

问题:

How can I sync local and remote folders using rsync from inside a php script without beeing prompted to type a password?

I have already set up a public key to automate the login on remote server for my user.

So this is running without any problem from cli:

rsync -r -a -v -e "ssh -l user" --delete ~/local/file 111.111.11.111:~/remote/;

But, when I try to run the same from a PHP script (on a webpage in my local server):

$c='rsync -r -a -v -e "ssh -l user" --delete ~/local/file 111.111.11.111:~/remote/';
//exec($c,$data);
passthru($c,$data);
print_r($data);

This is what I receive:

255

And no file is uploaded from local to remote server.

Searching on the net I found this clue:

"You could use a combination of BASh and Expect shell code here, but it wouldn't be very secure, because that would automate the root login. Generate keys for nobody or apache (whatever user as which Apache is being executed). Alternatively, install phpSuExec, Suhosin, or suPHP so that the scripts are run as the user for which they were called."

Well, I don't know how to "automate the root login" for PHP, which is running as "apache". Maybe to make the script run as the actual user is a better idea, I don't know... Thanks!

UPDATE: - As this works fine:

passthru('ssh user@111.111.11.111 | ls',$data);

Returning a list of the home foder, I can be sure, there is no problem with the automatic login. It mast be something with rsync running from the PHP script.

• I also have chmod -R 0777 on the local and remote folders just in case. But I didn't get it yet.

UPDATE:

All the problem is related to the fact that "when ran on commandline, ssh use the keyfiles on $HOME/.ssh/, but under PHP, it's ran using Apache's user, so it might not have a $HOME; much less a $HOME/.ssh/id_dsa. So, either you specifically tell it which keyfile to use, or manually create that directory and its contents."

While I cannot get rsync, this is how I got to transfer the file from local to remote:

if($con=ssh2_connect('111.111.11.111',22)) echo 'ok!';
if(ssh2_auth_password($con,'apache','xxxxxx')) echo ' ok!';
if(ssh2_scp_send($con,'localfile','/remotefolder',0755)) echo ' ok!';

Local file needs: 0644 Remote folder needs: 0775

I guess if the solution wouldn't be to run php with the same user bash does...

@Yzmir Ramirez gave this suggestion: "I don't think you want to "copy the key somewhere where apache can get to it" - that's a security violation. Better to change the script to run as a secure user and then setup the .ssh keys for passwordless login between servers.

This is something I have to invest some more time. If somebody know how to do this, please, it would be of great help.

回答1:

When I set this same thing up in an application of ours, I also ran into 255 errors, and found that they can mean a variety of things; it's not a particularly helpful error code. In fact, even now, after the solution's been running smoothly for well over a year, I still see an occasional 255 show up in the logs.

I will also say that getting it to work can be a bit of a pain, since there are several variables involved. One thing I found extremely helpful was to construct the rsync command in a variable and then log it. This way, I can grab the exact rsync command being used and try to run it manually. You can even su to the apache user and run it from the same directory as your script (or whatever your script is setting as the cwd), which will make it act the same as when it's run programmatically; this makes it far simpler to debug the rsync command since you're not having to deal with the web server. Also, when you run it manually, if it's failing for some unstated reason, add in verbosity flags to crank up the error output.

Below is the code we're using, slightly edited for security. Note that our code actually supports rsync'ing to both local and remote servers, since the destination is fully configurable to allow easy test installations.

try {
    if ('' != $config['publishSsh']['to']['remote']):
    //we're syncing with a remote server
        $rsyncToRemote = escapeshellarg($config['publishSsh']['to']['remote']) . ':';
        $rsyncTo = $rsyncToRemote . escapeshellarg($config['publishThemes']['to']['path']);
        $keyFile = $config['publishSsh']['to']['keyfile'];
        $rsyncSshCommand = "-e \"ssh -o 'BatchMode yes' -o 'StrictHostKeyChecking no' -q -i '$keyFile' -c arcfour\"";
    else:
    //we're syncing with the local machine
        $rsyncTo = escapeshellarg($config['publishThemes']['to']['path']);
        $rsyncSshCommand = '';
    endif;

    $log->Message("Apache running as user: " . trim(`whoami`), GLCLogger::level_debug);
    $deployCommand = "
        cd /my/themes/folder/; \
        rsync \
        --verbose \
        --delete \
        --recursive \
        --exclude '.DS_Store' \
        --exclude '.svn/' \
        --log-format='Action: %o %n' \
        --stats \
        $rsyncSshCommand \
        ./ \
        $rsyncTo \
         2>&1
    "; //deployCommand
    $log->Message("Deploying with command: \n" . $deployCommand, GLCLogger::level_debug);
    exec($deployCommand, $rsyncOutputLines, $returnValue);
    $log->Message("rsync status code: $returnValue", GLCLogger::level_debug);
    $log->Message("rsync output: \n" . implode("\n", $rsyncOutputLines), GLCLogger::level_debug);
    if (0 != $returnValue):
        $log->Message("Encountered error while publishing themes: <<<$returnValue>>>");
        throw new Exception('rsync error');
    endif;

    /* ... process output ... */
} catch (Exception $e) {
    /* ... handle errors ... */
}

A couple of things to notice about the code:

• I'm using exec() so that I can capture the output in a variable. I then parse it so that I can log and report the results in terms of how many files were added, modified, and removed.

• I'm combining rsync's standard output and standard error streams and returning both. I'm also capturing and checking the return result code.

• I'm logging everything when in debug mode, including the user Apache is running as, the rsync command itself, and the output of the rsync command. This way, it's trivially easy to run the same command as the same user with just a quick copy-and-paste, as I mention above.

• If you're having problems with the rsync command, you can adjust its verbosity without impact and see the output in the log.

In my case, I was able to simply point to an appropriate key file and not be overly concerned about security. That said, some thoughts on how to handle this:

• Giving Apache access to the file doesn't mean that it has to be in the web directory; you can put the file anywhere that's accessible by the Apache user, even on another network machine. Depending on your other layers of security, this may or may not be an acceptable compromise.

• Depending on the data you're copying, you may be able to heavily lock down the permissions of the ssh user on the other machine, so that if someone unscrupulous managed to get in there, they'd have minimal ability to do damage.

• You can use suEXEC to run the script as a user other than the Apache user, allowing you to lock access to your key to that other user. Thus, someone compromising the Apache user simply would not have access to the key.

I hope this helps.

回答2:

When you let the web server run the command, it runs it as its own user ("nobody" or "apache" or similar) and looks for a private key in that user's home directory/.ssh, while the private key you setup for your own user is in "/home/you/.ssh/keyfile", or however you named it.

Make ssh use that specific key with the -i parameter:

ssh -i /home/you/.ssh/keyfile

and it should work

回答3:

A Simple but In-direct solution to this problem is here, which i am using.

DO NOT run rsync directly from php, it does have issues as well as it can be security risk to do so. Rather, just prepare two scripts. In php script, you just have to change value of one variable on filesystem from 0 to 1.

Now, on ther other side, make an rsync script, that will run forever, and will have following logic. "" If the file has 0 in it, then do not run rsync, if it is 1, then run the rsync, then change its value back to 0 after successful run of rsync; ""

I am doing it as well.