Dataflow pipeline and pubsub emulator

问题:

I'm trying to setup my development environment. Instead of using google cloud pubsub in production, I've been using the pubsub emulator for development and testing. To achieve this I set the following environment variable:

export PUBSUB_EMULATOR_HOST=localhost:8586

This worked for the python google pubsub library but when I switched to using java apache beam for google dataflow, the pipeline still points to production google pubsub. Is there a setting, environment variable or method on the pipeline that I need to set so that the pipeline reads for the local pubsub emulator?

回答1:

I found the solution in the PubsubOptions interface, and extending it for my own PipelineOptions implementation. Then using setPubsubRootUrl() to set it to localhost:port of the emulator.