How to change the region/zone where dataflow job o

Question

Is there where I can set in google dataprep, the region where the generated dataflow job will be running? For example I'm transforming CSV data in a bucket in Europe to a BigQuery Dataset in Europe. But the dataflow job is running in US-central-1, not good.

Thx in advance for your help.

Answer 1

Changing the below directories to my own EU directories worked for me, as suggested by @markov00

• Upload directory
• Job Run directory
• Temp directory

Answer 2

As of this writing, Google Cloud DataPrep can currently only be run in Iowa (us-central-1). Result from: https://cloud.google.com/dataprep/pricing

Accessibility of products by region can be seen in the following link, but currently it doesn't have DataPrep Beta listed. https://cloud.google.com/about/locations/

Answer 3

You can change where DataPrep jobs run using the profile page on the dataprep app: https://clouddataprep.com/profile (the user icon on top right corner, than click on your account name) Here you can find the buckets used for:

• Upload directory
• Job Run directory
• Temp directory

Change them to your custom ones creating buckets on your preferred region.

Answer 4

Cloud Dataprep uses Cloud Dataflow service underneath. This service can run at more than one location. In fact, when you submit a Cloud Dataflow job, you can specify a regional endpoint for job management and execution by using the --region option. In the above-mentioned documents, "Iowa" represents a drop-down menu that offers other choices than Iowa itself, ex.: London, Belgium and Frankfurt in Europe.