Can't find spark-hbase mvn dependency

2019-05-06 02:39发布

问题:

I want to read table in HBase using Spark. I added the following dependency:

libraryDependencies += "org.apache.hbase" %% "hbase-spark" % "2.0.0-SNAPSHOT"

as mentioned in hbase website, but the dependency can't be resolved !!

I also tried different versions (1.2.0, 1.1.2) and it didn't work.

回答1:

It doesn't look like hbase-spark is in maven central, which is the default repository that dependencies will be retrieved from.

You'll need to configure your build management tool (unclear if you're using Maven or SBT) to use the correct repository.

From the project page you can use

  • https://repository.apache.org/content/repositories/snapshots for snapshots
  • https://repository.apache.org/content/repositories/releases for releases

Currently, however, it looks like only snapshots are available.

Additionally, as mentioned in the comment from the author below, the line

libraryDependencies += "org.apache.hbase" %% "hbase-spark" % "2.0.0-SNAPSHOT"

should actually be

libraryDependencies += "org.apache.hbase" % "hbase-spark" % "2.0.0-SNAPSHOT"

The double % is not needed in this case as you do no want to append the scala version to the artifact name.



回答2:

  1. add this repository

    cloudera https://repository.cloudera.com/artifactory/cloudera-repos/

  2. then u can download it by maven:

    org.apache.hbase hbase-spark 1.2.0-cdh5.7.0



回答3:

I was facing the same issue while using hbase-spark dependency in pom.xml

<!-- https://mvnrepository.com/artifact/org.apache.hbase/hbase-spark -->
<dependency>
   <groupId>org.apache.hbase</groupId>
   <artifactId>hbase-spark</artifactId>
   <version>2.0.0-alpha4</version>
</dependency>

I resolve the above issue by using following step:

  • Download the hbase-spark dependency jar from maven repository by click on the blue marking area as below image.

  • Create a lib folder inside /src/main/resources as per the attached image.

  • Copy the downloaded jar into lib folder and add the system path of jar file in pom.xml

.

    <!-- https://mvnrepository.com/artifact/org.apache.hbase/hbase-spark -->
<dependency>
   <groupId>org.apache.hbase</groupId>
   <artifactId>hbase-spark</artifactId>
   <version>2.0.0-alpha4</version>
   <scope>system</scope>
   <systemPath>${project.basedir}/src/main/resources/lib/hbase-spark-2.0.0-alpha4.jar</systemPath>
</dependency>

Hope It will help you.