python pipe subprocess i/o over socket

2019-08-07 03:56发布

问题:

I know there are similar questions out there, but I'm having trouble with this concrete example, and haven't found a good answer. I'm trying to set up a remote backup server for dar, along these lines. I've asked a separate question about doing this by invoking netcat with subprocess.Popen, but I'd prefer to set up the sockets and do all the piping with python if possible. There will be several gigs transferred, so I can't just read all the input first and then pass it on.

The problem is that the server doesn't seem to be reading the data.

At the moment, I've got the following code:

from socket import socket, AF_INET, SOCK_STREAM
import sys
import SocketServer
import subprocess

class DarHandler(SocketServer.BaseRequestHandler):
    def handle(self):
        print('entering handler')
        data = self.request.recv(1024).strip()
        print('got: ' + data)
        if data == 'xform':
            s = socket(AF_INET, SOCK_STREAM)
            s.bind(('',0))
            myaddr, myport = s.getsockname()
            print('bound new socket to {0}:{1}'.format(myaddr, myport))
            self.request.send(str(myport))
            s.listen(1)
            conn, remoteaddr = s.accept()
            print('accepted connection from {0}:{1}'.format(*remoteaddr))
            xform_input = conn.makefile('rb',0)
            proc = subprocess.Popen(
                ['/usr/bin/dar_xform', '-s', '10k', '-', 'archives/sockbackup',],
                 stdin=xform_input
            )
            return_code = proc.wait()
            print('dar_xform returned {0}'.format(return_code))
            conn.close()
            self.request.send(str(return_code))
        else:
            self.request.send('bad request')
        print('send result, exiting handler')

server_address = ('localhost', 18010)
def server():
    server = SocketServer.TCPServer(server_address, DarHandler)
    print('listening')
    server.serve_forever()

def client():
    sock = socket(AF_INET, SOCK_STREAM)
    print('connecting to server')
    sock.connect(('localhost', 18010))
    print('connected, sending request')
    sock.send('xform')
    print('waiting for response')
    port = sock.recv(1024)
    print('got: ' + port)
    s = socket(AF_INET, SOCK_STREAM)
    s.connect(('localhost', int(port)))
    print('connected to dar_xform port')
    dar_output = s.makefile('wb',0)
    return_code = subprocess.call(
          ['/usr/bin/dar', '-B', 'config/test.dcf', '-c', '-',], 
          stdout=dar_output
    )
    print('dar returned {0}'.format(return_code))
    s.close()
    result = sock.recv(1024)
    print('received: ' + result)
    sock.close()
    print('socket closed, exiting')

if __name__ == "__main__":
    if sys.argv[1].startswith('serv'):
        server()
    else:
        client()

I get the following output from the server side when I run the client:

listening
entering handler
got: xform
bound new socket to 0.0.0.0:41658
accepted connection from 127.0.0.1:42440

Then it just sits there. Meanwhile, dar runs on the client and the client is stuck waiting for a response from the server:

connecting to server
connected, sending request
waiting for response
got: 41300
connected to dar_xform port


 --------------------------------------------
 53 inode(s) saved
 with 0 hard link(s) recorded
 0 inode(s) changed at the moment of the backup
 0 inode(s) not saved (no inode/file change)
 0 inode(s) failed to save (filesystem error)
 1 inode(s) ignored (excluded by filters)
 0 inode(s) recorded as deleted from reference backup
 --------------------------------------------
 Total number of inodes considered: 54
 --------------------------------------------
 EA saved for 0 inode(s)
 --------------------------------------------
dar returned 0

回答1:

It looks like the dar_xform command on the server is failing...

Try providing a full path to the dar_xform command to subprocess.Popen(...) and see if you continue to have a problem. What you've described looks suspiciously like a PATH issue, especially since it works at the shell prompt.