Initial job has not accepted any resources; check your cluster UI to ensure that workers are registe

spark在提交任务时，出现如下错误：

15/03/26 22:29:36 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory15/03/26 22:29:51 WARN TaskSchedulerImpl: Initial job has not accepted any resources;
check your cluster UI to ensure that workers are registered and have sufficient memory15/03/26 22:30:06 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory15/03/26
22:30:21 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory



从警告信息上看，初始化job时没有获取到任何资源；提示检查集群，确保workers可以被注册并有足够的内存资源。

如上问题产生的原因是多方面的，可能原因如下：

1.因为提交任务的节点不能和spark工作节点交互，因为提交完任务后提交任务节点上会起一个进程，展示任务进度，大多端口为4044，工作节点需要反馈进度给该该端口，所以如果主机名或者IP在hosts中配置不正确。所以检查下主机名和ip是否配置正确

2.有可能是内存不足

检查内存

conf.set("spark.executor.memory", "3000m")

Make sure to set

SPARK_LOCAL_IP

and

SPARK_MASTER_IP

.

查看8080端口，确保一些workers保持Alive状态，确保 some cores 是可利用的。