Interprocess communication with a modified python

问题:

TL;DR: How can I spawn a different python interpreter (from within python) and create a communication channel between the parent and child when stdin/stdout are unavailable?

---

I would like my python script to execute a modified python interpreter and through some kind of IPC such as multiprocessing.Pipe communicate with the script that interpreter runs.

Lets say I've got something similar to the following:

subprocess.Popen(args=["/my_modified_python_interpreter.exe",
                       "--my_additional_flag",
                       "my_python_script.py"])

Which works fine and well, executes my python script and all.

I would now like to set up some kind of interprocess communication with that modified python interpreter.

Ideally, I would like to share something similar to one of the returned values from multiprocessing.Pipe(), however I will need to share that object with the modified python process (and I suspect multiprocessing.Pipe won't handle that well even if I do that).

Although sending text and binary will be sufficient (I don't need to share python objects or anything), I do need this to be functional on all major OSes (windows, Linux, Mac).

Some more use-case/business explanation

More specifically, the modified interpreter is the IDAPython interpreter that is shipped with IDA to allow scripting within the IDA tool.

Unfortunately, since stdio is already heavily used for the existing user interface functionalities (provided by IDA), I cannot use stdin/stdout for the communication.

---

I'm searching for possibilities that are better than the one's I thought of:

1. Use two (rx and tx channels) hard-disk files and pass paths to both as the arguments.
2. Use a local socket and pass a path as an argument.
3. Use a memory mapped file and the tagname on windows and some other sync method on other OSes.

回答1:

After some tinkering with the multiprocessing.Pipe function and the multiprocesing.Connection objects it returns, I realized that serialization of Connection objects is far simpler that I originally thought.

A Connection object has three descripting properties:

1. fileno - A handle. An arbitrary file descriptor on Unix and a socket on windows.
2. readable - A boolean controlling whether Connection object can be read.
3. writable - A boolean controlling whether Connection object can be written.

All three properties are accessible as object attributes and are controllable through the Connection class constructor.

It appears that if:

1. The process calling Pipe spawns a child process and shares the connection.fileno() number.
2. The child process creates a Connection object using that file descriptor as the handle.
3. Both interpreters implement the Connection object roughly the same (And this is the risky part, I guess).

It is possible to Connection.send and Connection.recv between those two processes although they do not share the same interpreter build and the multiprocessing module was not actually used to instantiate the child process.

EDIT:

Please note the Connection class is available as multiprocessing.connection.Connection in python3 and as _multiprocessing.Connection in python2 (which might suggest it's usage is discouraged. YMMV)

回答2:

Going with the other answer of mine turned out to be a mistake. Because of how handles are inherited in python2 on Windows I couldn't get the same solution to work on Windows machines. I ended up using the far superior Listener and Client interfaces also found in the multiprocessing module.

This question of mine discusses that mistake.