Launch a completely independent process

Question

I wanted to initiate a process from my python script (main.py), specifically I want to run the below command

`nohup python ./myfile.py &`

and this file myfile.py should even after my main python script exits.

Additionally I wish to get the pid of the new process.

I tried os.spawnl*, os.exec* & subprocess.Popen methods, all are terminating my myfile.py if my main.py script exits.

I may be missing something.

Update: Can I use os.startfile with xdg-open? Is it a right approach?

Example

a = subprocess.Popen([sys.executable, "nohup /usr/bin/python25 /long_process.py &"],\
     stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
print a.pid

If I check ps aux | grep long_process, I could not see any process running.

long_process.py which keeps on printing some text: no exit.

Am I doing anything wrong here?

Answer 1

You can use os.fork().

import os
pid=os.fork()
if pid==0: # new process
    os.system("nohup python ./myfile.py &")
    exit()
# parent process continues

Answer 2

I could not see any process running.

You don't see any process running because the child python process exits immediately. The Popen arguments are incorrect as @user4815162342 says in the comment.

To launch a completely independent process, you could use python-daemon package or use systemd/supervisord/etc:

#!/usr/bin/python25
import daemon
from long_process import main

with daemon.DaemonContext():
    main()

Though it might be enough in your case, to start the child with correct Popen arguments:

with open(os.devnull, 'r+b', 0) as DEVNULL:
    p = Popen(['/usr/bin/python25', '/path/to/long_process.py'],
              stdin=DEVNULL, stdout=DEVNULL, stderr=STDOUT, close_fds=True)
time.sleep(1) # give it a second to launch
if p.poll(): # the process already finished and it has nonzero exit code
    sys.exit(p.returncode)

If the child process doesn't require python2.5 then you could use sys.executable instead (to use the same Python version as the parent).

Note: the code closes DEVNULL in the parent without waiting for the child process to finish (it has no effect on the child).

Answer 3

You open your long-running process and keep a pipe to it. So you expect to talk to it. When yor launcher script exits, you can no longer talk to it. The long-running process receives a SIGPIPE and exits.

The following just worked for me (Linux, Python 2.7).

Create a long-running executable:

$ echo "sleep 100" > ~/tmp/sleeper.sh

Run Python REPL:

$ python
>>>

import subprocess
import os
p = subprocess.Popen(['/bin/sh', os.path.expanduser('~/tmp/sleeper.sh')])
# look ma, no pipes!
print p.pid
# prints 29893

Exit the REPL and see the process still running:

>>> ^D
$ ps ax | grep sleeper
29893 pts/0    S      0:00 /bin/sh .../tmp/sleeper.sh
29917 pts/0    S+     0:00 grep --color=auto sleeper

If you want to first communicate to the started process and then leave it alone to run further, you have a few options:

• Handle SIGPIPE in your long-running process, do not die on it. Live without stdin after the launcher process exits.
• Pass whatever you wanted using arguments, environment, or a temporary file.
• If you want bidirectional communication, consider using a named pipe (man mkfifo) or a socket, or writing a proper server.
• Make the long-running process fork after the initial bi-direcional communication phase is done.