spark - scala - How can I check if a table exists

问题:

I have to check whether a table exists in hive using spark(1.6.2) scala

If it doesn't I have to create an empty dataframe and save that as a hive table.

If it exists, then overwrite the existing table.

I need a function that returns boolean based on which I could take the above mentioned decisions(of whether to create new table or overwrite existing one)

回答1:

1.x:

def tableExists(table: String, sqlContext: SQLContext) =
  sqlContext.tableNames.contains(table)

2.x:

def tableExists(table: String, spark: SparkSession) =
  spark.catalog.tableExists(table)

2.1.x or later.

You can use spark.catalog.tableExists. Credits go to Huseyin Oktay for pointing that out.

回答2:

We also can define it with a database name as below.

1.6.x

sqlContext.tableNames("db_name").contains("tbl_name")

2.x:

spark.catalog.tableExists("db_name", "tbl_name")