启动YARN时，ResourceManager 和 NodeManager报错 Error found before invoking supervisord dictionary update s

在通过CM启动Yarn服务时候，不同的版本可能都会遇到同样的一个问题，如下图所示：



Error found before invoking supervisord: dictionary update sequence element #89 has length 1; 2 is required

之前在cm5.5、cm5.7以及现在的cm5.11都出现过这样的问题。

在cm5.5中，修改文件

/opt/cloudera-manager/cm-5.5.0/lib64/cmf/agent/src/cmf/util.py

修改内容如下，原文件:

pipe = subprocess.Popen(['/bin/bash', '-c', ". %s; %s; env" % (path, command)],
stdout=subprocess.PIPE, env=caller_env)

修改为：

pipe = subprocess.Popen(['/bin/bash', '-c', ". %s; %s; env | grep -v { | grep -v }" % (path, command)],
stdout=subprocess.PIPE, env=caller_env)

也即在env的后面加了

| grep -v { | grep -v }

这玩意，启动成功。

但cm5.7及以后的版本中，貌似不存在这个路径或文件里面不包含这部分内容，仔细查看日志文件

cloudera-scm-agent.log

，如下图(cm5.11)：



涉及到的文件为

/opt/cloudera-manager/cm-5.11.1/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.11.1-py2.6.egg/cmf/util/__init__.py

果然在这个文件里面包含上述修改的内容，尝试着按照上述修改。

然后重启agent，再启动Yarn，这次成功了。

如果其他版本遇到类似问题，可尝试上述2个文件。

以上只针对完全离线安装的CDH，如果是完全在线或者半离线(rpm包)安装，可能路径不一样。楼主从来只用完全离线方式安装，所以其他2种方式路径未知。

可尝试用find命令查找,如

find / -name 'util.py'

PS：用完全离线的方式安装肯定优于在线和半离线方式，以下摘自官方文档

1.Distribution of CDH as a single object

Instead of having a separate package for each part of CDH, parcels have just a single object to install. This makes it easier to distribute software to a cluster that is not connected to the Internet.

2. Internal consistency

All CDH components are matched, eliminating the possibility of installing parts from different versions of CDH.

3. Installation outside of /usr

In some environments, Hadoop administrators do not have privileges to install system packages. These administrators needed to use CDH tarballs, which do not provide the infrastructure that packages do. With parcels, administrators can install to /opt, or anywhere else, without completing the additional manual steps of regular tarballs.

Note: With parcels, the path to the CDH libraries is /opt/cloudera/parcels/CDH/lib instead of the usual /usr/lib. Do not link /usr/lib/ elements to parcel-deployed paths, because the links may cause scripts that distinguish between the two paths to not work.

4. Installation of CDH without sudo

Parcel installation is handled by the Cloudera Manager Agent running as root or another user, so you can install CDH without sudo.

5. Decoupled distribution from activation

With side-by-side install capabilities, you can stage a new version of CDH across the cluster before switching to it. This allows the most time-consuming part of an upgrade to be done ahead of time without affecting cluster operations, thereby reducing downtime.

6. Rolling upgrades

Packages require you to shut down the old process, upgrade the package, and then start the new process. Any errors in the process can be difficult to recover from, and upgrading requires extensive integration with the package management system to function seamlessly. With parcels, when a new version is staged side-by-side, you can switch to a new minor version by simply changing which version of CDH is used when restarting each process. You can then perform upgrades with rolling restarts, in which service roles are restarted in the correct order to switch to the new version with minimal service interruption. Your cluster can continue to run on the existing installed components while you stage a new version across your cluster, without impacting your current operations. Major version upgrades (for example, CDH 4 to CDH 5) require full service restarts because of substantial changes between the versions. Finally, you can upgrade individual parcels or multiple parcels at the same time.

7. Upgrade management

Cloudera Manager manages all the steps in a CDH version upgrade. With packages, Cloudera Manager only helps with initial installation.

8. Additional components

Parcels are not limited to CDH. Cloudera Impala, Cloudera Search, LZO, Apache Kafka, and add-on service parcels are also available.

9. Compatibility with other distribution tools

Cloudera Manager works with other tools you use for download and distribution. For example, you can use Puppet. Or, you can download the parcel to Cloudera Manager Server manually if your cluster has no Internet connectivity and then have Cloudera Manager distribute the parcel to the cluster.