Read remote file using python subprocess and ssh?

问题:

How can I read data from a big remote file using subprocess and ssh?

回答1:

import subprocess
ssh = subprocess.Popen(['ssh', 'user@host', 'cat', 'path/to/file'],
                       stdout=subprocess.PIPE)
for line in ssh.stdout:
    line  # do stuff

回答2:

The answer above will work, but you'll have to setup your ssh login to use no password between your boxes. There are other ways to transfer files between computers using Python. A simple way, without authentication is to setup an apache server and use an http request.

回答3:

For performance improvement, which is important when the file is big, there is rsync. For more information about the exact improvement see following post and the answer from Rafa: How does `scp` differ from `rsync`?

The algorithm would then be the following using rsync

import subprocess

subprocess.Popen(["rsync", host-ip+'/path/to/file'],stdout=subprocess.PIPE)
for line in ssh.stdout:
    line  # do stuff

回答4:

Use iter with readline to read each full line:

for i in iter(f.stdout.readline,"")