I got this error on Dataflow log:
(11b55950300ec4ce): Workflow failed. Causes: (11b55950300ec459): The Dataflow appears to be stuck. Please reach out to the Dataflow team at http://stackoverflow.com/questions/tagged/google-cloud-dataflow.
The JobID is: 2017-11-13_20_58_16-5812097262489045871
I tried the exact same code and it is working on other project...
Any help is appreciated,
Yosua
It turns the dataflow cannot spawn workers because of mismatch between the default network's region (in asia-southeast1 in my case) and the zone parameters (default: us-central1-f).
So to solve this issue I need to explicitly set the network, subnet, and zone i the pipeline parameter.
Note: I found this unintuitive but it is okay to have region as us-central1 and zone outside us-central1 since the region parameter is where we deploy the jobs and zone is where we spawn the worker.
Reference: https://cloud.google.com/dataflow/pipelines/specifying-exec-params
