I have a python code that upload files to another server when a button is click. After clicking the button, web server will start uploading file to another server that took a long process, during that time the the browser is loading, and I can't open another page of my website.
I try to use async, but async example all uses httpclient which I am not using, then I modify the code like below
The upload itself is a synchronous code, and I don't want to modify it, what I want is a wrapper or something that can allow the client to open another page and does not lock the session while processing. Is there such a way?
def Upload(self, filehost, uniqueno, username, password):
Filename = self.GetFile(uniqueno)
uploadfunc = None
if (upload == "mega"):
uploadfunc = MegaWrapper.Upload
result = Thread(target=self.MyUpload, args=(uploadfunc, MyConfig.Folder + Filename, username, password))
result.start()
result.join(5 * 60)
return self.UploadResult
def MyUpload(self, uploadfunc, filename, username, password):
str = uploadfunc(filename, username, password)
self.UploadResult = str
return str
def post(self):
filehost = self.get_argument("filehost")
crc = self.get_argument("crc")
username = self.get_argument("username")
password = self.get_argument("password")
fileList = dal.FileListByCRC(crc)
messagetype = "error"
mesasge = ""
if (len(fileList) > 0):
try:
url = self.Upload(filehost, fileList[0].UniqueNo , username, password)
message = url
if (StringOps.StartWith(url, "http")):
messagetype = "info"
message = "Success Uploading : " + url
except Exception, ex:
messagetype = "error"
message = "Fail Uploading : " + str(ex)
self.render("upload.html", title=MyConfig.DefaultSiteTitle + " - Administrative Page - Upload/Delete", messagetype=messagetype, message=message, crc=self.get_argument("crc"), filehost=self.get_argument("filehost"), username=self.get_argument("username"), password=self.get_argument("password"))
You basically have to write your own handler that also make use of the IOLoop correctly. Look into tornado.ioloop.IOLoop.add_callback
and friends. That being said, this wasn't intuitive and took me some time to figure out how to do async downloading of large binary data, and at the end of it I concluded that this probably isn't the best tool for serving large binary data that exceed a typical transmission window.
Of course, you probably want to know how to start, here is something to help you get started:
import time
import logging
from datetime import timedelta
from threading import Thread
from tornado.httpserver import HTTPServer
import tornado.ioloop
import tornado.web
logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)
def blocking_task(callback, seconds=10):
time.sleep(seconds)
callback('slept for %d seconds' % seconds)
class CallbackRequestHandler(tornado.web.RequestHandler):
wait = timedelta(seconds=1)
@tornado.web.asynchronous
def get(self):
tornado.ioloop.IOLoop.current().add_callback(self.start_loop)
def start_loop(self):
self.task = Thread(target=blocking_task,
kwargs={'callback': self.write})
self.task.start()
logger.info('starting %s', self.task)
self.loop()
def loop(self):
if self.task.is_alive():
logger.info('waiting on %s', self.task)
tornado.ioloop.IOLoop.current().add_timeout(self.wait, self.loop)
else:
logger.info('finished on %s', self.task)
self.finish()
tornado_app = tornado.web.Application([
(r"/", CallbackRequestHandler),
])
if __name__ == "__main__":
http_server = HTTPServer(tornado_app)
http_server.listen(9090)
tornado.ioloop.IOLoop.instance().start()
An example run with two clients poking at this instance at http://localhost:9090
.
$ python unblock.py
INFO:__main__:starting <Thread(Thread-1, started 139876219111168)>
INFO:__main__:waiting on <Thread(Thread-1, started 139876219111168)>
INFO:__main__:waiting on <Thread(Thread-1, started 139876219111168)>
INFO:__main__:starting <Thread(Thread-2, started 139876210718464)>
INFO:__main__:waiting on <Thread(Thread-2, started 139876210718464)>
INFO:__main__:waiting on <Thread(Thread-1, started 139876219111168)>
INFO:__main__:waiting on <Thread(Thread-2, started 139876210718464)>
INFO:__main__:waiting on <Thread(Thread-1, started 139876219111168)>
INFO:__main__:waiting on <Thread(Thread-2, started 139876210718464)>
INFO:__main__:waiting on <Thread(Thread-1, started 139876219111168)>
INFO:__main__:waiting on <Thread(Thread-2, started 139876210718464)>
INFO:__main__:waiting on <Thread(Thread-1, started 139876219111168)>
INFO:__main__:waiting on <Thread(Thread-2, started 139876210718464)>
INFO:__main__:waiting on <Thread(Thread-1, started 139876219111168)>
INFO:__main__:waiting on <Thread(Thread-2, started 139876210718464)>
INFO:__main__:waiting on <Thread(Thread-1, started 139876219111168)>
INFO:__main__:waiting on <Thread(Thread-2, started 139876210718464)>
INFO:__main__:waiting on <Thread(Thread-1, started 139876219111168)>
INFO:__main__:waiting on <Thread(Thread-2, started 139876210718464)>
INFO:__main__:waiting on <Thread(Thread-1, started 139876219111168)>
INFO:__main__:waiting on <Thread(Thread-2, started 139876210718464)>
INFO:__main__:finished on <Thread(Thread-1, stopped 139876219111168)>
INFO:tornado.access:200 GET / (::1) 10010.21ms
INFO:__main__:waiting on <Thread(Thread-2, started 139876210718464)>
INFO:__main__:finished on <Thread(Thread-2, stopped 139876210718464)>
INFO:tornado.access:200 GET / (::1) 10004.87ms
This is probably NOT the optimum solution as I don't make use of the newer/better methods such as concurrent.futures
and the related hooks that Tornado provides, but hopefully this example code will point you towards the right direction.