hadoop&spark mapreduce对比 & 框架设计和理解

Hadoop MapReduce:



MapReduce在每次执行的时候都要从磁盘读数据，计算完毕后都要把数据放到磁盘



spark map reduce:









RDD is everything for dev:



Basic Concepts:





Graph RDD:



Spark Runtime:



schedule:



Depency Type:



Scheduler Optimizations：



Event Flow:



Submit Job:



New Job Instance:



Job In Detail:



executor.launchTask:



Standalone：



Work Flow:



Standalone detail:



Driver application to Clustor:



Worker Exception:



Executor Exception:



Master Exception:



Master HA: