Python Tornado updating shared data between reques

Question

I have a Python Tornado app. The app contains request handlers, for which I am passing data to like (the code below is not complete, and is just to illustrate what I want):

configs = {'some_data': 1, # etc.
          }

class Application(tornado.web.Application):
    def __init__(self):
        handlers = [('/pageone', PageOneHandler, configs),
                    ('/pagetwo', PageTwoHandler, configs)]
        settings = dict(template_path='/templates',
                    static_path='/static', debug=False)
        tornado.web.Application.__init__(self, handlers, **settings)

# Run the instance
# ... code goes here ...
application = Application()
http_server = tornado.httpserver.HTTPServer(application)
# ... other code (bind to port, etc.)

# Callback function to update configs
some_time_period = 1000 # Once an second
tornado.ioloop.PeriodicCallback(update_configs, some_time_period).start()
tornado.ioloop.IOLoop.instance().start()

I want the update_configs function to update the configs variable defined above and have this change propagate through the handlers. For example (I know this doesn't work):

def update_configs():
    configs['some_data'] += 1

# Now, assuming PageOneHandler just prints out configs['some_data'], I'd expect
# the output to be: "1" on the first load, "2" if I load the page a second
# later, "4" if I load the page two seconds after that, etc.

The problem is, the configs variable is passed along to the handlers during creation in the constructor for the Application class. How can I update configs['some_data'] in the periodic callback function?

My actual use case for this mechanism is to refresh the data stored in the configs dictionary from the database every so often.

Is there an easy way to do this without fiddling around with application.handlers (which I have tried for the past hour or so)?

Answer 1

Well, the simplest thing would be to pass the entire config dict to the handlers, rather than just the individual values inside the dict. Because dicts are mutable, any change you make to the values in the dict would then propagate to all the handlers:

import tornado.web
import tornado.httpserver

configs = {'some_data': 1, # etc.
          }

def update_configs():
    print("updating")
    configs['some_data'] += 1

class PageOneHandler(tornado.web.RequestHandler):
    def initialize(self, configs):
        self.configs = configs
    def get(self):
        self.write(str(self.configs) + "\n")


class PageTwoHandler(tornado.web.RequestHandler):
    def initialize(self, configs):
        self.configs = configs

    def get(self):
        self.write(str(self.configs) + "\n")


class Application(tornado.web.Application):
    def __init__(self):
        handlers = [('/pageone', PageOneHandler, {'configs' : configs}),
                ('/pagetwo', PageTwoHandler, {'configs': configs})]
        settings = dict(template_path='/templates',
                    static_path='/static', debug=False)
        tornado.web.Application.__init__(self, handlers, **settings)

# Run the instance
application = Application()
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(8888)

# Callback function to update configs
some_time_period = 1000 # Once an second
tornado.ioloop.PeriodicCallback(update_configs, some_time_period).start()
tornado.ioloop.IOLoop.instance().start()

Output:

dan@dantop:~> curl localhost:8888/pageone
{'some_data': 2}
dan@dantop:~> curl localhost:8888/pageone
{'some_data': 3}
dan@dantop:~> curl localhost:8888/pagetwo
{'some_data': 4}
dan@dantop:~> curl localhost:8888/pageone
{'some_data': 4}

To me this approach makes the most sense; the data contained in configs doesn't really belong to any one instance of a RequestHandler, it's global state shared by all RequsetHandlers, as well as your PeriodicCallback. So I don't think it makes sense to try to create X numbers of copies of that state, and then try to keep all those different copies in sync manually. Instead, just share the state across your whole process using either a custom object with class variables, or a dict, as shown above.

Answer 2

Another strategy, in addition to what dano mentions above is to attach the shared data to the Application object.

class MyApplication(tornado.web.Application):

    def __init__(self):

       self.shared_attribute = foo;

        handlers = [#your handlers here]        
        settings = dict(#your application settings here)
        super().__init__(handlers, **settings)


server = tornado.httpserver.HTTPServer(MyApplication())
server.listen(8888)
tornado.ioloop.IOLoop.instance().start()

Next you can access shared_attribute defined above in all your request handlers using self.application.shared_attribute.

You update it at one place and it immediately reflects in all your subsequent calls to the request handlers.