Hive Tables are created from spark but are not vis

2019-06-03 17:08发布

From spark using:

DataFrame.write().mode(SaveMode.Ignore).format("orc").saveAsTable("myTableName")

Table is getting saved I can see using below command's hadoop fs -ls /apps/hive/warehouse\test.db' where test is my database name

drwxr-xr-x - psudhir hdfs 0 2016-01-04 05:02 /apps/hive/warehouse/test.db/myTableName

but when I trying to check tables in Hive I cannot view them either with command SHOW TABLES from hiveContext.

3条回答
你好瞎i
2楼-- · 2019-06-03 17:40

I think you need to run INVALIDATE METADATA; in the hive console to refresh the databases and view your new table.

查看更多
Deceive 欺骗
3楼-- · 2019-06-03 17:42

You have to copy the hive-site.xml file (mine is located at /etc/hive/conf.dist/hive-site.xml) to Spark conf folder (mine is located at /etc/spark/conf/)

sudo cp /etc/hive/conf.dist/hive-site.xml /etc/spark/conf/

Restart Spark and it should work.

查看更多
SAY GOODBYE
4楼-- · 2019-06-03 17:50
sudo cp /etc/hive/conf.dist/hive-site.xml /etc/spark/conf/

This worked for me in a Cloudera quick start Virtual Box.

查看更多
登录 后发表回答