Google Cloud Dataflow Worker Threading

2019-08-14 13:27发布

问题:

Say we have one worker with 4 CPU cores. How does parallelism configured in Dataflow worker machines? Do we parallelize beyond # of cores?

Where would this type of information be available?

回答1:

One worker thread is used per core, and each worker thread independently processes a chunk of the input space.