How to receive pickle via subprocess.Popen

Question

getPickle.py

import pickle
import subprocess

cmd = ['rsh', 'host1', 'sendPickle.py']
p  = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p.communicate()
results = pickle.load(stdout)
print results

sendPickle.py

import pickle
import sys

to_return = {'a':1, 'b': 2}
pickle.dump(to_return, sys.stdout)

OUTPUT:

File "getPickle" line 10, in <module>
    results = pickle.load(stdout)
AttributeError: 'str' object has no attribute 'readline'

What can I do to get pickle back from stdout?

Thank you

Answer 1

Use pickle.loads to load the pickle from a string. pickle.load is for loading from a stream.

Two unrelated remarks:

• if you are using Python 2, you probably want to import cPickle as pickle because the C version is many time faster and just as powerful.

• unless you specifically want to support older Python versions, it is a good idea to use protocol=-1 on the dump side in order specify the latest Pickle protocol, which is more efficient than the default lowest/oldest version.

Answer 2

@user4815162342 gives the correct answer. To make it crystal clear, here's example code showing how to retrieve the pickled object.

Note that it's a bit of an unusual case to pair dump and loads, but that's what needs to be done as p.communicate returns the string from stdout.

>>> import pickle
>>> import subprocess as sp
>>> cmd = ['python', 'sendPickle.py']
>>> p = sp.Popen(cmd, stdout=sp.PIPE, stderr=sp.PIPE)
>>> stdout, stderr = p.communicate()
>>> stdout
"(dp0\nS'a'\np1\nI1\nsS'b'\np2\nI2\ns."
>>> results = pickle.loads(stdout)
>>> results
{'a': 1, 'b': 2}