H2O上H2OContext.getOrCreate失败(H2O fails on H2OConte

2019-10-28 12:01发布

我试着写在斯卡拉/星火/ H2O的示例程序。 该程序编译,而是抛出一个异常H2OContext.getOrCreate

object App1 extends App{

         val conf = new SparkConf()
         conf.setAppName("AppTest")
         conf.setMaster("local[1]")
         conf.set("spark.executor.memory","1g");
         val sc = new SparkContext(conf)

         val spark = SparkSession.builder
            .master("local")
            .appName("ApplicationController")
            .getOrCreate()

         import spark.implicits._  

         val h2oContext = H2OContext.getOrCreate(sess) // <--- error here
         import h2oContext.implicits._

         val rawData = sc.textFile("c:\\spark\\data.csv")        
         val data = rawData.map(line => line.split(',').map(_.toDouble))
         val response: RDD[Int] = data.map(row => row(0).toInt)

         val str = "count: " + response.count()
         val h2oResponse: H2OFrame = response.toDF

         sc.stop
         spark.stop
}

这是异常日志:

例外在线程“主”了java.lang.RuntimeException:当经由--packages选项使用苏打水作为火花包,“no.priv.garshol.duke:公:1.2”的依赖性,必须明确指定由错误引起的在星火依赖分辨率。 在org.apache.spark.h2o.H2OContext.init(H2OContext.scala:117)

文章来源: H2O fails on H2OContext.getOrCreate