Suppose I have two iterators, and I want to compute
fancyoperation1(iter1), fancyoperation2(iter2)
Normally, I would simply use fancyoperation1(iter1), fancyoperation2(iter2)
. However, if these iterators are linked to a single source, perhaps tee
d from a single iterator, I can't do this without keeping a lot of temporary data in memory. In that case, I know of several options:
- I could rewrite
fancyoperation1
andfancyoperation2
into a single function that does both at the same time, but that may be a lot of code duplication, and I may not understand or have the source code for either function. Also, this would need to be done anew for every pair of operations. - I could use threading. The synchronization can probably be written once in a helper function, and the overhead probably wouldn't be too bad as long as I don't need to switch threads too often.
- I could keep a lot of temporary data in memory.
I don't really like the drawbacks of those options, though. Is there a way to do what I want in one thread, without rewriting things or using large amounts of memory? I tried to do it with coroutines, but Python's yield
doesn't seem to be powerful enough.
(I do not currently have this problem, but I'm wondering what to do if it ever comes up.)