Clean shutdown of a Python script

Question

I have a threaded server written in Python that I start using the following shell script:

#!/bin/bash

base_path="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

public_dns=$(curl -s http://169.254.169.254/latest/meta-data/public-hostname)
echo $public_dns > "$base_path/client/address"

cd "$base_path/server"
python "server.py" &
echo $! > "$base_path/server_pid"
echo "Server running"

I echo the PID to a file so that I can shutdown a server using another shell script:

#!/bin/bash

base_path="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

kill -9 `cat "$base_path/server_pid"`
rm "$base_path/server_pid"
rm "$base_path/client/address"

I know, however, that this is a bad approach considering the server has many threads that have I/O into network and hdd... So what I would like to do is have the second script somehow interact with the server and tell it to start a shutdown sequence which would cleanly close all the threads, close & archive logs etc.

Now I know about atexit and I tested it this way:

import atexit
def byebye(o):
    o.write('stop')
    o.flush()
    o.close()

o = open('log','w')
o.write('start')
o.flush()

atexit.register(byebye, o)

while True:
    pass

But when I kill -9 the process, byebye() is not fired. Should I use a command other than the almighty kill -9? How would I go about shutting down the process?

Answer 1

It seems to me that your are trying to implement a daemon. There is various references about daemon implementations in python :

Previous stackoverlow question : How do you create a daemon in Python?

PEP regarding daemon : http://www.python.org/dev/peps/pep-3143/

python module implementing PEP3143 : http://pypi.python.org/pypi/python-daemon/

Note that the PEP is a draft.

More about daemons : http://en.wikipedia.org/wiki/Daemon_%28computing%29

Typically, daemon are started, stopped and restarted as :

mydaemon start
mydaemon stop
mydaemon restart

So you dont need to know the PID or anything else to stop it.

Answer 2

I'm not particularly used to programming with threads, but rather than sending kill -9 (which corresponds to SIGKILL), you could send SIGINT or some other user defined signal. SIGINT (kill -2 on my system) is nice because that one is already understood by python. (When python catches that signal, it raises a KeyboardInterrupt), but any signal will work. You just need to register a signal handler that exits your program cleanly.

Answer 3

kill 9 is atomic powered - you don't get to clean up after yourself. A better approach would be to use a different, gentler, signal (HUP is commonly used to signal a server process that it is time to shut down, for example), and teach your python code how to handle it gracefully.

The signal module documentation should get you started.