sbt build failed for spark scala with xgboost

问题:

./build/sbt clean package gives the below error:

Resolving org.fusesource.jansi#jansi;1.4 ...
   [warn]   ::::::::::::::::::::::::::::::::::::::::::::::
   [warn]   ::          UNRESOLVED DEPENDENCIES         ::
   [warn]   ::::::::::::::::::::::::::::::::::::::::::::::
   [warn]   :: ml.dmlc.xgboost#xgboost4j_2.10;0.7: not found
   [warn]   :: ml.dmlc.xgboost#xgboost4j-spark_2.10;0.7: not found
   [warn]   ::::::::::::::::::::::::::::::::::::::::::::::

build.sbt looks like below:

name := "xgboostproj"
version := "1.0"
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.6.1"
libraryDependencies += "org.apache.spark" %% "spark-mllib" % "1.6.1"
resolvers += Resolver.mavenLocal
libraryDependencies += "ml.dmlc.xgboost" %% "xgboost4j" % "0.7"
libraryDependencies += "ml.dmlc.xgboost" %% "xgboost4j-spark" % "0.7"

Thanks in Advance!

回答1:

xgboost jars must be built locally and published to your local maven repository for your set up to work. The instructions for this is published here

Also additionally the dependencies should be like below (the groupId is ml.dmlc)

libraryDependencies += "ml.dmlc" %% "xgboost4j" % "0.7"
libraryDependencies += "ml.dmlc" %% "xgboost4j-spark" % "0.7"

回答2:

I think the dependencies should be changed to:

libraryDependencies += "ml.dmlc" % "xgboost4j" % "0.7"
libraryDependencies += "ml.dmlc" % "xgboost4j-spark" % "0.7"

Also make sure your sbt is looking at the local maven repo. These were the two things that I needed to fix to make it work. Also make sure the jar files exist in the m2 repo. This is assuming you used mvn install to build the jvm packages.