Convert between Streaming Dataset and DStream

2019-02-19 11:13发布

问题:

Is it possible to convert a streaming o.a.s.sql.Dataset to DStream? If so, how?

I know how to convert it to RDD, but it is in a streaming context.

回答1:

It is not possible. Structured Streaming and legacy Spark Streaming (DStreams) use completely different semantics and are not compatible with each other so:

  • DStream cannot be converted to Streaming Dataset.
  • Streaming Dataset cannot be converted to DStream.