spark-0.8.0源码剖析--standalone模式集群并行和单机并行

1在集群用的remoteactor进行 master worker  通信 进行集群的管理。  executor和调度器通信进行执行器管理， blockmanageractor和主节点的masterblockmanager进行通信进行存储管理.

private[spark] class StandaloneExecutorBackend(

    driverUrl: String,

    executorId: String,

    hostPort: String,

    cores: Int)

  extends Actor

  with ExecutorBackend

  with Logging {

case LaunchTask(taskDesc) =>

      logInfo("Got assigned task " + taskDesc.taskId)

      if (executor == null) {

        logError("Received launchTask but executor was null")

        System.exit(1)

      } else {

        executor.launchTask(this, taskDesc.taskId, taskDesc.serializedTask)

      }

在每个executor中执行task

  

  def launchTask(context: ExecutorBackend, taskId: Long, serializedTask: ByteBuffer) {

    threadPool.execute(new TaskRunner(context, taskId, serializedTask))

  }