google dataflow firewall rule/network tag

2019-05-28 14:32发布

问题:

I am trying to connect to cassandra (which have firewall rule tags) from google dataflow batch, , moreover the dataflow service will create workers with 'dataflow' tagged.

is it possible to change the default tag ? or add more tags? .

回答1:

I believe it is currently not possible to change the labels with which Dataflow service creates the VMs. However, it is possible to control which GCE network the Dataflow workers will belong to, by specifying the parameter --network: The Google Compute Engine network for launching Compute Engine instances to run your pipeline (see Specifying Pipeline Execution Parameters).

You may consider creating a network for your Dataflow workers and using that in the firewall rules.



回答2:

There does not appear to be a way to change the tag. However, firewall rules can select instances by service account in lieu of network tag, and the service account can be specified through the --serviceAccount account option. See "Specifying a user-managed controller service account" within the Dataflow documentation.