java.lang.NoSuchMethodError：scala.reflect.api.Java

我已经使用Spark 1.6和Cassandra connector 1.4.3从火花数据写入卡桑德拉。

现在，我们升级到Spark 2.1.0和我试图升级cassandra connector to 2.0.0-M3 ，但返回错误如下： -

java.lang.NoSuchMethodError：scala.reflect.api.JavaUniverse.runtimeMirror（Ljava /郎/类加载器;）Lscala /反映/ API / JavaMirrors $ JavaMirror; 在com.datastax.spark.connector.util.JavaApiHelper $ .mirror（JavaApiHelper.scala：25）在com.datastax.spark.connector.util.JavaApiHelper $ .getTypeTag（JavaApiHelper.scala：29）在com.datastax.spark .connector.util.JavaApiHelper.getTypeTag（JavaApiHelper.scala）在com.datastax.spark.connector.japi.CassandraJavaUtil.typeTag（CassandraJavaUtil.java:159）在com.datastax.spark.connector.japi.CassandraJavaUtil.mapToRow（CassandraJavaUtil的.java：在com.datastax.spark.connector.japi.CassandraJavaUtil.mapToRow（CassandraJavaUtil.java:1205）在com.datastax.spark.connector.japi.CassandraJavaUtil.mapToRow（CassandraJavaUtil.java:1222）在COM 1172）。 cisco.cmcdp.EBdoUtils.upsertToJobAuditTable（EBdoUtils.java:370）在com.cisco.cmcdp.EBdoContactPublisher.main（EBdoContactPublisher.java:245）在sun.reflect.NativeMethodAccessorImpl.invoke0（本机方法）在sun.reflect.NativeMethodAccessorImpl。调用（NativeMethodAccessorImpl.java:57）在sun.reflect.DelegatingMethodAccessorImpl.invoke（DelegatingMethodAccessorImpl.java:4 3）在java.lang.reflect.Method.invoke（Method.java:606）在org.apache.spark.deploy.yarn.ApplicationMaster $$不久$ 2.run（ApplicationMaster.scala：637）

pom.xml中：

    <!-- Spark and Spark SQL, Hive Dependencies -->
     <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_2.11</artifactId>
        <version>2.1.0-mapr-1703</version>
                    <exclusions>
            <exclusion>
                <groupId>com.google.guava</groupId>
                <artifactId>guava</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql_2.11</artifactId>
        <version>2.1.0-mapr-1703</version>
                    <exclusions>
            <exclusion>
                <groupId>com.google.guava</groupId>
                <artifactId>guava</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-hive_2.11</artifactId>
        <version>2.1.0-mapr-1703</version>
                    <exclusions>
            <exclusion>
                <groupId>com.google.guava</groupId>
                <artifactId>guava</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming_2.11</artifactId>
        <version>2.1.0-mapr-1703</version>
                    <exclusions>
            <exclusion>
                <groupId>com.google.guava</groupId>
                <artifactId>guava</artifactId>
            </exclusion>
        </exclusions>
    </dependency>


    <!--Spark Cassandra Connector -->
    <dependency>
        <groupId>com.datastax.cassandra</groupId>
        <artifactId>cassandra-driver-core</artifactId>
        <version>3.0.0</version>
        <exclusions>
            <exclusion>
                <groupId>org.xerial.snappy</groupId>
                <artifactId>snappy-java</artifactId>
            </exclusion>
            <exclusion>
                <groupId>com.google.guava</groupId>
                <artifactId>guava</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>com.datastax.cassandra</groupId>
        <artifactId>cassandra-driver-mapping</artifactId>
        <version>3.0.0</version>
        <exclusions>
            <exclusion>
                <groupId>com.google.guava</groupId>
                <artifactId>guava</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>org.apache.cassandra</groupId>
        <artifactId>cassandra-clientutil</artifactId>
        <version>2.0.0</version>

        <exclusions>
            <exclusion>
                <groupId>org.xerial.snappy</groupId>
                <artifactId>snappy-java</artifactId>
            </exclusion>
            <exclusion>
                <groupId>com.google.guava</groupId>
                <artifactId>guava</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>com.datastax.spark</groupId>
        <artifactId>spark-cassandra-connector_2.10</artifactId>
        <version>2.0.0-M3</version>
            <exclusions>
            <exclusion>
                <groupId>com.google.guava</groupId>
                <artifactId>guava</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>com.datastax.spark</groupId>
        <artifactId>spark-cassandra-connector_2.10</artifactId>
        <version>2.0.0-M3</version>
        <exclusions>
            <exclusion>
                <groupId>com.google.guava</groupId>
                <artifactId>guava</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

请建议卡桑德拉连接器的适当的方式和版本。

问候，RISHABH

这个问题主要是由于不适当的Scala版本的使用，你可能有斯卡拉2.10和Scala 2.11库在类路径的组合。请确保您使用的是星火2.x中，星火卡桑德拉连接器2.x和斯卡拉2.11（虽然斯卡拉2.10是支持的，但它能够更好地使用Scala的2.11）。看看下面的工作Maven配置 -

    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_2.11</artifactId>
        <version>2.1.0</version>
        <exclusions>
            <exclusion>
                <groupId>commons-codec</groupId>
                <artifactId>commons-codec</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql_2.11</artifactId>
        <version>2.1.1</version>
        <exclusions>
            <exclusion>
                <groupId>commons-codec</groupId>
                <artifactId>commons-codec</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming_2.11</artifactId>
        <version>2.1.1</version>
    </dependency>
    <dependency>
        <groupId>com.datastax.spark</groupId>
        <artifactId>spark-cassandra-connector_2.11</artifactId>
        <version>2.0.1</version>
    </dependency>   
    <dependency>
        <groupId>commons-codec</groupId>
        <artifactId>commons-codec</artifactId>
        <version>1.10</version>
    </dependency>

我希望它能帮助。