Spark to yarn启动的时候报错:org.apache.spark.SparkException: Yarn application has already ended!
场景:Spark to yarn启动的时候报错
原因:虚拟内存不够
解决方案:
可通过以下两种方式解决:
调大虚拟内存比:
yarn.nodemanager.vmem-pmem-ratio 默认2.1倍
或者关闭虚拟内存检查:
yarn.nodemanager.vmem-check-enabled 默认true
这是犯错的场景:
输入命令如下:
./spark-submit --master yarn --class org.apache.spark.examples.SparkPi ../examples/jars/spark-examples_2.11-2.2.1.jar 1
报错如下:
org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master.
打开hadoop的web端
解决方案如下:
切换到hadoop目录下
cd /home/hadoop-2.6.5/etc/hadoop
vim yarn-site.xml
<property>
<name>yarn.nodemanager.vmem-pmem-ratio</name>
<value>3</value>
</property>
重启yarn
stop-yarn.sh
start-yarn.sh
重新输入指令
./spark-submit --master yarn --class org.apache.spark.examples.SparkPi ../examples/jars/spark-examples_2.11-2.2.1.jar 1
- spark streaming restart error: org.apache.spark.SparkException: Yarn application has already ended!
- 提交spark的代码的时候出现Exception in thread "main" org.apache.SparkException:Yarn application has already end
- spark-shell启动报错:Yarn application has already ended! It might have been killed or unable to launch application master
- spark-shell启动报错:Yarn application has already ended! It might have been killed or unable to launch ap
- spark在yarn上运行报错:Yarn application has already ended
- Spark集群启动的时候出现failed to launch: nice -n 0 /soft/spark/bin/spark-class org.apache.spark.deploy.worker
- Yarn application has already ended! It might have been killed or unable to launch application master
- spark-shell on yarn 出错(arn application already ended,might be killed or not able to launch applic)解决
- Hive On Spark报错:Failed to execute spark task, org.apache.hadoop.hive.ql.metadata.HiveException
- hadoop报错:org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container.
- 新建servlet后再配置web.xml后tomcat不能启动,报错:Caused by: org.apache.catalina.LifecycleException: Failed to star
- SVN问题之——org.apache.subversion.javahl.ClientException: Attempted to lock an already-locked dir
- SVN更新代码时报错:org.apache.subversion.javahl.ClientException: Attempted to lock an already-locked dir
- maven WEB项目启动没问题访问页面就报错:org.apache.jasper.JasperException: Unable to compile class for JSP
- org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container
- hadoop错误org.apache.hadoop.yarn.exceptions.YarnException Unauthorized request to start container
- 项目启动后发生org.apache.jasper.JasperException: Failed to load or instantiate TagLibraryValidator class: o
- 启动tomcat卡在starting状态,然后访问页面就报错:org.apache.jasper.JasperException: Unable to compile class for JSP