Can a DStream
have type parameter
s?
If yes, how?
When I try lazy val qwe = mStream.mapWithState(stateSpec)
on myDStream: DStream[(A, B)]
(class parameter), I get:
value mapWithState is not a member of org.apache.spark.streaming.dstream.DStream[(A, B)]
lazy val qwe = mStream.mapWithState(stateSpec)
Substantial subset of the Spark API requires implicit
ClassTags
(see Scala: What is a TypeTag and how do I use it?) andPairDStreamFunctions.mapWithState
is no different. Check class definition:and:
If want to create a function which operates on a generic pair streams and uses
mapWithState
you should at least provideClassTags
forKeyType
andValueType
types:If
StateType
andMappedType
are parametrized as well you'll needClassTags
for these too: