Error in Caching a Table in SparkSQL

2019-07-13 01:37发布

I am trying to cache a Table available in Hive(using spark-shell). Given below is my code

scala> val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)

scala> hiveContext.cacheTable("sparkdb.firsttable")

and I am getting the below Exception

org.apache.spark.sql.catalyst.analysis.NoSuchTableException
    at org.apache.spark.sql.hive.client.ClientInterface$$anonfun$getTable$1.apply(ClientInterface.scala:112)

The table firsttable is available in database sparkdb(in Hive). Looks like the issue seems to be in providing database name. How do I achieve this?

PS : HiveQL query like the one shown below does work without any issues

scala> hiveContext.sql("select * from sparkdb.firsttable")

Find below results from few other method calls

scala> hiveContext.tables("sparkdb")
res14: org.apache.spark.sql.DataFrame = [tableName: string, isTemporary: boolean]

scala> hiveContext.tables("sparkdb.firsttable")
res15: org.apache.spark.sql.DataFrame = [tableName: string, isTemporary: boolean]

1条回答
Melony?
2楼-- · 2019-07-13 02:19

Aha! I was right, this seems to be SPARK-8105. So, for now, your best bet is to do the select * and cache that.

查看更多
登录 后发表回答