在属于RDD对象改变的属性(Changing an attribute in an object t

2019-10-23 03:13发布

我有以下代码:

      def generateStoriesnew(outputPath: String, groupedRDD:RDD[(String,Iterable[String])], isInChurnMode: Boolean, isInChurnPeriod: Boolean) {
val windowedRDD = groupedRDD.map(SOME CODE)
 var  windowedRDD2 = windowedRDD.filter(r => r != null).map(a=>a.churnPeriod(isInChurnPeriod,isInChurnMode))  
val prettyStringRDD = windowedRDD2.map(r => {
  r.toString
})
prettyStringRDD.saveAsTextFile(outputPath)

}

这里是ChurnPriod函数的代码:

def churnPeriod( churnPeriod:Boolean, churnMode: Boolean): Unit = {  
if (churnMode && rootEventType.equalsIgnoreCase("c")){
  var churnCustStory: CustStoryN = null
  var nonChurnCustStory: CustStoryN = null
  var churnPeriodEventStory: mutable.MutableList[StoryEventN] = null
  var NonChurnEventstory: mutable.MutableList[StoryEventN] = null
  churnPeriodEventStory = new mutable.MutableList[StoryEventN]
  NonChurnEventstory = new mutable.MutableList[StoryEventN]     
  var lastEventChurnPeriod = true
  var currentEventStory = eventStory
  var max = currentEventStory.length
  println(max);
  if (currentEventStory.size > 0) {
    for (i <- 0 until currentEventStory.length) {
      var currentEvent = currentEventStory(i)
      if (currentEvent.timeSenseRootEvent < 90) {          
        churnPeriodEventStory.+=(currentEvent)
        //lastEventChurnPeriod = true
      }
      else {
        NonChurnEventstory.+=(currentEvent)
        lastEventChurnPeriod = false
      }
    }
  }
if (churnPeriod)
  eventStory = churnPeriodEventStory
else
  eventStory=null

}}

但流失周期函数不改变eventstory其是custstory类的成员。 我缺少什么吗?

    class CustStoryN (val custId:String,
             var rootEventType:String,
             var rootEventTime:Long,
             var eventStory:mutable.MutableList[StoryEventN])

我的假设是两种:1.map不是,我2.churnPeriod功能永远不会得到所谓的3.I不能改变eventstory这是卡斯特故事类的成员没有人有任何想法,我怎么能解决的功能转变权这个问题?

Answer 1:

这将是微不足道的通过调试来确定。 只要把几个断点,你可以看到,如果该程序的函数内部停止,确实发生了什么转变。

我的猜测是,这个问题是这一行: if (currentEventStory.size > 0)因此,开始于大小为0的列表保持在大小为0,直到永远。 另一种选择是churnPeriod是不正确的,因此,你计算了很多,但从来没有分配给eventStory变量。

您的代码确实需要一个良好的清理;-)



文章来源: Changing an attribute in an object that belongs to RDD
标签: scala class rdd