Python. Tornado. Non-blocking xmlrpc client

问题:

Basically we can call xmlrpc handlers following way:

import xmlrpclib
s = xmlrpclib.ServerProxy('http://remote_host/rpc/')
print s.system.listmethods()

In tornado we can integrate it like this:

import xmlrpclib
import tornado.web

s = xmlrpclib.ServerProxy('http://remote_host/rpc/')

class MyHandler(tornado.web.RequestHandler):
    def get(self):
        result = s.system.listmethods()

I have following, a little bit newbie, questions:

1. Will result = s.system.listmethods() block tornado?
2. Are there any non-blocking xmlrpc clients around?
3. How can we achieve result = yield gen.Task(s.system.listmethods)?

回答1:

1.Yes it will block tornado, since xmlrpclib uses blocking python sockets (as it is)

2.Not that I'm aware of, but I'll provide a solution where you can keep xmlrpclib but have it async

3.My solution doesn't use tornado gen.

Ok, so one useful library to have at mind whenever you're doing networking and need to write async code is gevent, it's a really good high quality library that I would recommend to everyone.

Why is it good and easy to use ?

• You can write asynchronous code in a synchronous manner (so that makes it easy)

• All you have to do, to do so is monkey patch with one simple line :

  from gevent import monkey; monkey.patch_all()

When using tornado you need to know two things (that you may already know) :

• Tornado only supports asynchronous views when acting as a HTTPServer (WSGI isn't supported for async views)
• Async views need to terminate the responses by themselves you do by using either self.finish() or self.render() (which calls self.finish())

Ok so here's an example illustrating what you would need with the necessary gevent integration with tornado :

# Python immports
import functools

# Tornado imports
import tornado.ioloop
import tornado.web
import tornado.httpserver

# XMLRpc imports
import xmlrpclib


# Asynchronous gevent decorator
def gasync(func):
    @tornado.web.asynchronous
    @functools.wraps(func)
    def f(self, *args, **kwargs):
        return gevent.spawn(func, self, *args, **kwargs)
    return f


# Our XML RPC service
xml_service = xmlrpclib.ServerProxy('http://remote_host/rpc/')


class MyHandler(tornado.web.RequestHandler):
    @gasync
    def get(self):
        # This doesn't block tornado thanks to gevent
        # Which patches all of xmlrpclib's socket calls
        # So they no longer are blocking
        result = xml_service.system.listmethods()

        # Do something here

        # Write response to client
        self.write('hello')
        self.finish()


# Our URL Mappings
handlers = [
   (r"/", MyHandler),
]


def main():
    # Setup app and HTTP server
    application = tornado.web.Application(handlers)
    http_server = tornado.httpserver.HTTPServer(application)
    http_server.listen(8000)

    # Start ioloop
    tornado.ioloop.IOLoop.instance().start()


if __name__ == "__main__":
    main()

So give the example a try (adapt it to your needs obviously) and you should be good to go.

No need to write any extra code, gevent does all the work of patching up python sockets so they can be used asynchronously while still writing code in a synchronous fashion (which is a real bonus).

Hope this helps :)

回答2:

I do not think so. Because Tornado has it's own ioloop, but gevent's ioloop is libevent. So gevent will block Tornado's ioloop.