How do I add Java dependencies to a Google Dataflo

Question

My Java project has lots of jars from third party libraries as well as my own code. How do I deploy these so that Google Cloud Dataflow can use them?

There is documentation on how do this in Python, but not Java.

Answer 1

You don't need to do anything special: simply add those jars to your classpath and Dataflow will automatically make them available to your workers. This is the same mechanism Dataflow uses to make your pipeline's code per se available to the workers.

For more details, please see the filesToStage option in Specifying Execution Parameters. And please let us know if this option is not doing what you expect or if you're looking for something else.