{var%20f='http://v.t.sina.com.cn/share/share.php?appkey=1515056452',u=z||d.location,p=['&url=',e(u),'&title=',e(t||d.title),'&source=',e(r),'&sourceUrl=',e(l),'&content=',c||'gb2312','&pic=',e(p||'')].join('');function%20a(){if(!window.open([f,p].join(''),'mb',['toolbar=0,status=0,resizable=1,width=440,height=430,left=',(s.width-440)/2,',top=',(s.height-430)/2].join('')))u.href=[f,p].join('');};if(/Firefox/.test(navigator.userAgent))setTimeout(a,0);else%20a();})(screen,document,encodeURIComponent,'','','https://www.manongdao.com/data/attach/logo/logo.png', '推荐 Rolldiameter 的问题《Fastest parallel requests in Python》','https://www.manongdao.com/q-1193887.html','页面编码gb2312|utf-8默认gb2312'));){kind=link}
I need to keep making many requests to about 150 APIs, on different servers. I work with the trading, time is crucial, I can not waste 1 millisecond.
The solution and problems I found were these:
- Async using Asyncio: I do not want to rely on a single thread, for some reason it may get stucked.
- Threads: Is it really reliable on Python to use threads? Do I have the risk of 1 thread make
other get stucked? - Multiprocesses: If a have on process controlling the others, would I loose to much time in interprocess communication?
Maybe a solution that uses all of that.
If there is no really good solution in Python, what should I use instead?
# Using Asyncio
import asyncio
import requests
async def main():
loop = asyncio.get_event_loop()
future1 = loop.run_in_executor(None, requests.get, 'http://www.google.com')
future2 = loop.run_in_executor(None, requests.get, 'http://www.google.co.uk')
response1 = await future1
response2 = await future2
print(response1.text)
print(response2.text)
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
# Using Threads
from threading import Thread
def do_api(url):
#...
#...
#...
#...
for i in range(50):
t = Thread(target=do_apis, args=(url_api[i],))
t.start()
One can easily spend 5x more time on doing the same amount of work, if bad approach was selected. Check the [ Epilogue ] section below as as to see one such exemplified code ( an MCVE-example ), where any of the Threads and/or Processes were way slower, than a pure
[SERIAL]-form of the process-execution. So indeed a due care will be necessary here and in every real-world use-case.The long story short :
HFT/Trading may benefit from an intentionally restricted-duration
asynciocode, as was in detail demonstrated below, so as to benefit from transport-latency masking ( interleaved progress of execution, due to having to still wait for a delivery of a remote-processing results - so can do some useful work in the meantime, letting the I/O-related waiting threads stay idle and handling some other work in the meantime ). Computing heavy tasks or tight, the less very tight request/response-behavioural patterns will not be able to use this, right due to computing intesive nature ( no reason there to go idle at all, so no beneficial CPU-releases will ever happen ) or due to having a need to avoid any ( potentially deteriorating ) in-determinism in code-execution tight response time-window.Threads are an a priori lost game in standard python interpreter. The central GIL-lock stepping enforces a pure-
[SERIAL]code execution, one-after-another( round-robin scheduling ordered ) as explained here and interactively demonstrated ( here + code included ) - click+to zoom, until you see 1-tick per pixel resolution, and you will see how often other cores go and try to ask for GIL-lock acquisition and fail to get it, and you will also never see more than a one-and-only-one green-field of a CPU-execution in any column, so a pure-[SERIAL]-code execution happens even in a crowd of python-threads ( the real-time goes to the right in the graphs ).Processes-based multiprocessing is quite expensive tool, yet gives one a way, how to escape from the trap of the GIL-lock internally
[SERIAL]-ised python flow of processing. Inter-process communication is expensive, if performed using the standardmultiprocessing.Queue, but HFT/trading platforms may enjoy much faster / lower latency tools for truly distributed, multi-host, performance-motivated designs. Details go beyond this format, yet after tens of years using microseconds-shaving for ultimate response robustness and latency minimisation for such a distributed-computing trading system.The Computer Science has taught me a lot lessons on doing this right.
From a pure Computer-Science point of view, the approach to the problem ( a solution not being a parallel in its nature ) proposed here by @Felipe Faria made me to post this answer.
I will forget now about all HFT-trading-tricks and just decompose the concept of latency masking ( asking 150+ API calls across a global internet for some data is by far not a true
[PARALLEL]process-flow organisation ).The
example.comurl-target, used in the simplified test code, looks for my test-site having about some~ 104-116 [ms]network transport-latency. So my side has about that amount of CPU-idle time once each request has been dispatched over the network ( and there will never be an answer arriving sooner than that~ 100 ms).Here, the time, the ( principally that very loooooooooooong ) latency, can become hidden right by letting the CPU handle more threads do another request, as the one that have already sent one, no matter what, have to wait. This is called a latency-masking and it may help reduce the end-to-end run-time, even inside GIL-stepped pythonic threads ( that otherwise must have been for years fully avoided in the true and hardcore HPC-grade parallel-code ). For details, one may read about GIL-release time, and one may also deduce, or observe in test, the upper-limit of such latency-masking, if there are going to be way more requests in the salvo, than there are GIL-lock thread switching ( forced transfers of execution ), than one's actual network transport-latency.
So the latency masking tricks demasked:
The simplified experiment has shown, that the fired salvo of 25 test calls took
~ 273 [ms]in batch,whereas each of the 25, latency-masked, calls has taken
~ 232.6-266.9 [ms]i.e. the responses were heavily latency-masked, being just loosely concurrently monitored from "outside" of their respective context-managers by the orchestrating tooling inside the event-loopasync/awaitmechanics, for their respective async completion.The powers of the latency-masking could be seen from the fact, that the first call
launch_id:< 0>to the API has finished as the last but one (!)This was possible as the url-retrieve process takes so long without having anything to do with the local CPU-workload ( which is IDLE until anything gets there-and-back to first start any processing on the fetched data ).
This is also the reason for which latency-masking does not help "so impressively well" for processes, where each
[ns]-shaving is in place, like the said HPC-processing or in HFT-trading engines.Epilogue: the same work may take 5x longer ...
All the run-time times are in [us].
Both the Process- and Thread-based forms of a just-
[CONCURRENT]-processing have accumulated immense instantiation overheads and results-collection and transfer overheads ( the threading with additional, indeterministic variability of run-time ), whereas the pure-[SERIAL]process-flow was by far the fastest and the most efficient way to get the job done. For largerf-s these overheads will grow beyond all limits and may soon introduce O/S swapping and other system-resources deteriorating side-effects, so be careful.Instead of using multithreading or
asyncio.executor, you should useaiohttpinstead, which is the equivalent ofrequestsbut with asynchronous support.Outputs:
As you can see 100 websites from across the world were successfully reached (with or without
https) in about 4 seconds withaiohttpon my internet connection (Miami, Florida). Keep in mind the following can slow down the program by a fewms:printstatements (yes, including the ones placed in the code above).The example above has both instances of the above, and therefore it is arguably the least-optimized way of doing what you have asked. However, I do believe it is a great start for what you are looking for.