-->

Structured Streaming中window不存在问题。

2020-04-11 12:21发布

问题:

今天运行Spark官网的Demo http://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#handling-late-data-and-watermarking

代码如下:

import spark.implicits._

val words = ... // streaming DataFrame of schema { timestamp: Timestamp, word: String }

// Group the data by window and word and compute the count of each group
val windowedCounts = words
    .withWatermark("timestamp", "10 minutes")
    .groupBy(
        window($"timestamp", "10 minutes", "5 minutes"),
        $"word")
    .count()

然后提示window不存在,请问这个需要引入什么包么?看官方也没说,谁能帮帮我,在线等~

回答1:

解决了,引入 import org.apache.spark.sql.functions._ 就好了。