How to write Spark Structured Streaming Data into

2019-04-14 23:30发布

How to write Spark Structured Streaming Data into Hive?

There is df.write().saveAsTable(tablename) however I am not sure if this writes streaming data

I normally do df.writeStream().trigger(new ProcessingTime(1000)).foreach(new KafkaSink()).start() to write streaming data into Kafka but I don't see anything similar to write streaming data into Hive data warehouse. any ideas?

1条回答
太酷不给撩
2楼-- · 2019-04-14 23:38
df.createOrReplaceTempView("mytable") 
spark.sql("create table mytable as select * from mytable");

or

df.write().mode(SaveMode.Overwrite).saveAsTable("dbName.tableName");
查看更多
登录 后发表回答