grpc with dataflow - java.lang.NoSuchMethodError

Question

I have a grpc service code compiled with maven protoc plugin 0.4.2 and grpc version 0.8.0. I am trying to use this generated code with datafflow sdk 1.4.0 but I get this error when i try to create a grpc client using the blocking stub.

com.google.cloud.dataflow.sdk.Pipeline$PipelineExecutionException: java.lang.NoSuchMethodError: io.grpc.protobuf.ProtoUtils.marshaller(Lcom/google/protobuf/MessageLite;)Lio/grpc/MethodDescriptor$Marshaller;
    at com.google.cloud.dataflow.sdk.Pipeline.run(Pipeline.java:186)

I am not sure if this is because of some discrepancy with between the versions of protobuf used by the dataflow sdk and the grpc generated code. I also get this error when i try to output a protobuf message from one of the DoFn's in my pipeline

com.google.cloud.dataflow.sdk.Pipeline$PipelineExecutionException: java.lang.NoSuchMethodError: com.google.protobuf.GeneratedMessage.computeStringSize(ILjava/lang/Object;)I
    at com.google.cloud.dataflow.sdk.Pipeline.run(Pipeline.java:186)

I have excluded protobuf from the dataflow sdk dependency in maven and the only protobuf and groc packages that show up in my dependency tree are 3.0.0-alpha-3.1 and 0.8.0 respectively.

Answer 1

One quick note: The PipelineExecutionException simply indicates that an error occurred within user code when executing the pipeline, so it is not of much relevance. You'll want to focus on the cause of that failure and its stack trace.

Your two errors are slightly different from each other, but each one suggests that the version of the protocol buffers library used at runtime does not match the version that generated the protocol buffer message classes.

Answer 2

ProtoUtils.marshaller(MessageLite) was added in grpc-java 0.9.0; grpc-java 0.8.0 used a Parser object instead. Try updating your grpc-java version.

It's a similar issue with protobuf. The method was added in v3.0.0-alpha-4.