I'm new to programming and have been wrestling this problem for a while now...
I'm writing a Python script to automate an interactive command-line based Perl script so that it may run on every file in a specified directory. I can get the program to boot with subprocess.call and subprocess.Popen, but from there I either open the program and it sits there waiting in the shell for a user input, or I open it an infinite amount of times. I need the Perl script to open once per file, accept user inputs (with something similar to "enter" being pressed after each input, of course, and a command including the name of the file, which should be str(files) in the example code below), begin the Perl script's calculations with a user input of "4", and then have it close.
I know this is possible, but I've tried 1000 different approaches to no avail. Could someone give me an "explain it like I'm 5" as to how I can get this Python and Perl script talking to each other as if a user was inputting commands?
Here's what I have so far:
for files in os.listdir("In_Files"):
print "Performing calculations on " + str(files) + " ..."
os.chdir("/home/MTS/Dropbox/dir/In_Filess")
subprocess.Popen("run_command.command -f a", shell=True) #this boots up the Perl script
time.sleep(3) #waits for the script to open...
Popen.communicate(input="1") #user inputs begin here
Popen.wait(0) #am I trying to set a return value here so that the inputs may proceed once the previous one has been accepted?
Popen.communicate(input=str(files)) #this is where the name of the file the Perl script is working on is entered
Popen.wait(0)
Popen.communicate(input="4")
Popen.wait(0)
I keep reading a lot about stdin pipes but don't quite understand them. Are they used for this problem or should another approach be used? Also, I've already looked at Pexpect but that doesn't seem to be working.
Your help is immensely appreciated.
To run a command for each file from a given directory:
#!/usr/bin/env python
import os
from subprocess import Popen, PIPE
working_dir = b"/home/MTS/Dropbox/dir/In_Files"
for filename in os.listdir(working_dir):
print("Performing calculations on {filename!r}...".format(**vars()))
p = Popen(["run_command.command", "-f", "a"], cwd=working_dir, stdin=PIPE)
p.communicate(input=b"\n".join([b"1", filename, b"4"]))
if p.returncode != 0: # non-zero exit status may indicate failure
raise Error
See @skmrx's answer for the explanation of why your current code fails.
Pipes are one way for multiple processes to communicate and are widely used for that. A pipe is a pair of file descriptors with one opened for writing and the other for reading. If data is written to the former, reading from the latter will return back the same data.
Popen can create pipes for the standard input (stdin), standard output (stdout) and standard error (stderr) file descriptors of the process it executes - but you have to explicitly ask for it. This way, you can write something to the stdin pipe which will be passed on to the stdin of the child. Similarly, if you want to read the output of the child process, you can read from the stdout pipe. This serves your purpose well and hence your approach is correct.
Now let's see your code:
subprocess.Popen("run_command.command -f a", shell=True)
This will execute the command, but won't create the pipes to allow communication with the child process. If you look at the documentation, it says that you need to explicitly ask it to create pipes through keyword arguments such as stdin=subprocess.PIPE.
time.sleep(3)
Generally, scripts that take input line-by-line don't care about when they receive input. So you should be able to safely remove this.
Popen.communicate(input="1")
Look at the documentation again:
Popen.communicate(input=None, timeout=None)
Interact with process: Send data to stdin. Read data from stdout and stderr, until end-of-file is reached. Wait for process to terminate.
You can call communicate() only once and you must pass in all the input as one string and it returns the data printed by the child process once it terminates. This also means that the calls to wait() are unncessary. Note that communicate() returns the a tuple with contents of what the child printed to stdout and stderr. You may need these.
