Driver容错安全性是什么

本篇内容主要讲解“Driver容错安全性是什么”，感兴趣的朋友不妨来看看。本文介绍的方法操作简单快捷，实用性强。下面就让小编来带大家学习“Driver容错安全性是什么”吧!

成都创新互联公司专注于企业营销型网站、网站重做改版、巴里坤哈萨克网站定制设计、自适应品牌网站建设、HTML5建站、商城网站建设、集团公司官网建设、成都外贸网站制作、高端网站制作、响应式网页设计等建站业务，价格优惠性价比高，为巴里坤哈萨克等各大城市提供网站开发制作服务。

从数据层面，ReceivedBlockTracker为整个Spark Streaming应用程序记录元数据信息。

从调度层面，DStreamGraph和JobGenerator是Spark Streaming调度的核心，记录当前调度到哪一进度，和业务有关。

ReceivedBlockTracker在接收到元数据信息后调用addBlock方法，先写入磁盘中，然后在写入内存中。

Driver容错安全性是什么

根据batchTime分配属于当前BatchDuration要处理的数据到timToAllocatedBlocks数据结构中。

Driver容错安全性是什么

Time类的是一个case class，记录时间，重载了操作符，隐式转换，值得借鉴。

case classTime(private valmillis: Long) {
  defmilliseconds: Long = millis
  def< (that: Time): Boolean = (this.millis < that.millis)
  def<= (that: Time): Boolean = (this.millis <= that.millis)
  def> (that: Time): Boolean = (this.millis > that.millis)
  def>= (that: Time): Boolean = (this.millis >= that.millis)
  def+ (that: Duration): Time = newTime(millis + that.milliseconds)
  def- (that: Time): Duration = newDuration(millis - that.millis)
  def- (that: Duration): Time = newTime(millis - that.milliseconds)
  // Java-friendlier versions of the above.
  defless(that: Time): Boolean = this< that
  deflessEq(that: Time): Boolean = this<= that
  defgreater(that: Time): Boolean = this> that
  defgreaterEq(that: Time): Boolean = this>= that
  defplus(that: Duration): Time = this+ that
  defminus(that: Time): Duration = this- that
  defminus(that: Duration): Time = this- that
  deffloor(that: Duration): Time = {
    valt = that.milliseconds
    newTime((this.millis / t) * t)
  }
  deffloor(that: Duration, zeroTime: Time): Time = {
    valt = that.milliseconds
    newTime(((this.millis - zeroTime.milliseconds) / t) * t + zeroTime.milliseconds)
  }
  defisMultipleOf(that: Duration): Boolean =
    (this.millis % that.milliseconds == 0)
  defmin(that: Time): Time = if(this< that) this elsethat
  defmax(that: Time): Time = if(this> that) this elsethat
  defuntil(that: Time, interval: Duration): Seq[Time] = {
    (this.milliseconds) until (that.milliseconds) by (interval.milliseconds) map (newTime(_))
  }
  defto(that: Time, interval: Duration): Seq[Time] = {
    (this.milliseconds) to (that.milliseconds) by (interval.milliseconds) map (newTime(_))
  }
  override deftoString: String = (millis.toString + " ms")
}
objectTime {
  implicit valordering = Ordering.by((time: Time) => time.millis)
}