Ubuntu平台下源码编译Ambari 2.4
2017-03-01 15:57
507 查看
Ubuntu/UbuntuKylin平台下源码编译Ambari 2.4
1、Ubuntu环境配置
1.1 网络配置
kylin@Ubuntu:~$ sudo vim /etc/network/interfaces \# interfaces(5) file used by ifup(8) and ifdown(8) auto ens32 iface ens32 inet static address 192.168.1.166 gateway 192.168.1.1 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 重启系统 kylin@Ubuntu:~$ sudo /etc/init.d/networking restart [ ok ] Restarting networking (via systemctl): networking.service.
1.2 Ubuntu安装SSH
sudoapt−getinstallssh sudo apt-get install rsynckylin@Ubuntu:~$ dpkg -l | grep ssh ii libssh-4:amd64 0.7.3-1 amd64 tiny C SSH library (OpenSSL flavor) ii libssh-gcrypt-4:amd64 0.7.3-1 amd64 tiny C SSH library (gcrypt flavor) ii openssh-client 1:7.3p1-1 amd64 secure shell (SSH) client, for secure access to remote machines ii openssh-server 1:7.3p1-1 amd64 secure shell (SSH) server, for secure access from remote machines ii openssh-sftp-server 1:7.3p1-1 amd64 secure shell (SSH) sftp server module, for SFTP access from remote machines ii ssh 1:7.3p1-1 all secure shell client and server (metapackage) ii ssh-import-id 5.6-0ubuntu1 all securely retrieve an SSH public key and install it locally
1.3 如何让ubuntukylin开机进入纯命令模式
找了搜索了很多方法都是针对ubuntu，ubuntukylin没有用！！kylin@Ubuntu:~sudoservicelightdmstartkylin@Ubuntu: sudo lightdm start
Failed to use bus name org.freedesktop.DisplayManager, do you have appropriate permissions?
1.4 JDK
kylin@Ubuntu:~$ dpkg -l | grep jdk kylin@Ubuntu:~$ dpkg -l | grep java ii gir1.2-javascriptcoregtk-4.0:amd64 2.14.0-1 amd64 JavaScript engine library from WebKitGTK+ - GObject introspection data ii libjavascriptcoregtk-1.0-0:amd64 2.4.11-3 amd64 JavaScript engine library from WebKitGTK+ ii libjavascriptcoregtk-4.0-18:amd64 2.14.0-1 amd64 JavaScript engine library from WebKitGTK+ kylin@Ubuntu:~$ kylin@Ubuntu:~$ java -version 程序 'java' 已包含在下列软件包中： * default-jre * gcj-5-jre-headless * openjdk-8-jre-headless * gcj-4.8-jre-headless * gcj-4.9-jre-headless * gcj-6-jre-headless * openjdk-9-jre-headless 请尝试：sudo apt install <选定的软件包> kylin@Ubuntu:~$ sudo vi .profile #set for java export JAVA_HOME=/home/kylin/jdk1.7.0_67 export CLASSPATH=.:$CLASSPATH:$JAVA_HOME/lib export _JAVA_OPTIONS="-Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true" export PATH=$PATH:$JAVA_HOME/bin kylin@Ubuntu:~$ source .profile kylin@Ubuntu:~$ java -version java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
1.5 Maven
kylin@Ubuntu:~$ vi .profile #set for maven export MAVEN_HOME=/home/kylin/apache-maven-3.0.5 export PATH=$PATH:$MAVEN_HOME/bin kylin@Ubuntu:~$ source .profile
maven有2个位置可以选择配置M2_HOME/conf/setting.xml和~/.m2/setting.xml
前者是全局性的，后者是针对用户的。
查询资料是说用户配置优先级高于全局配置
kylin@Ubuntu:~/apache-maven-3.0.5/conf$ cp settings.xml ~/.m2/
1.6 Node.js
kylin@Ubuntu:~$ sudo tar -zxvf node-v0.10.44-linux-x64.tar.gz -C /opt kylin@Ubuntu:~$ vi .profile #set for nodejs export NODE_HOME=/home/kylin/node-v0.10.44-linux-x64 export PATH=$PATH:$NODE_HOME/bin kylin@Ubuntu:~$ source .profile
配置淘宝源
kylin@Ubuntu:~$ npm config set registry https://registry.npm.taobao.org kylin@Ubuntu:~$ npm info underscor
安装brunch
kylin@Ubuntu:~$ npm install -g brunch@1.7.20 npm WARN deprecated minimatch@1.0.0: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue npm WARN optional dep failed, continuing fsevents@0.3.8 npm WARN deprecated minimatch@0.2.14: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue npm WARN deprecated graceful-fs@2.0.3: graceful-fs v3.0.0 and before will fail on node releases >= v7.0. Please update to graceful-fs@^4.0.0 as soon as possible. Use 'npm ls graceful-fs' to find it in the tree. /home/kylin/node-v0.10.44-linux-x64/bin/brunch -> /home/kylin/node-v0.10.44-linux-x64/lib/node_modules/brunch/bin/brunch > brunch@1.7.20 postinstall /home/kylin/node-v0.10.44-linux-x64/lib/node_modules/brunch > node setup.js postinstall brunch@1.7.20 /home/kylin/node-v0.10.44-linux-x64/lib/node_modules/brunch ├── debug@0.7.4 ├── async-waterfall@0.1.5 ├── commonjs-require-definition@0.1.2 ├── async-each@0.1.6 ├── commander@2.0.0 ├── ncp@0.4.2 ├── mkdirp@0.3.5 ├── init-skeleton@0.2.4 (rimraf@2.2.8) ├── read-components@0.6.1 (component-builder@0.10.1) ├── source-map@0.1.43 (amdefine@1.0.1) ├── coffee-script@1.8.0 ├── anymatch@1.0.0 (minimatch@1.0.0) ├── chokidar@0.12.6 (readdirp@1.3.0) ├── anysort@1.0.1 (anymatch@1.3.0) ├── pushserve@0.1.6 (connect-slashes@0.0.11, express@3.3.8) └── loggy@0.2.2 (growl@1.8.1, ansi-color@0.2.1, date-utils@1.2.21)
1.7 安装rpmbuild
ambari编译时依赖rpmbuild打包成rpm安装包，因而需要rpm套件，很幸运，ubuntu可以安装这一套件：kylin@Ubuntu:~$ sudo apt-get install -y rpm yum
1.8 Python
Ubuntu默认是Python 2.7，直接设置setuptools-0.6c11-py2.7.eggkylin@Ubuntu:~$ sudo sh setuptools-0.6c11-py2.7.egg Processing setuptools-0.6c11-py2.7.egg Copying setuptools-0.6c11-py2.7.egg to /usr/local/lib/python2.7/dist-packages Adding setuptools 0.6c11 to easy-install.pth file Installing easy_install script to /usr/local/bin Installing easy_install-2.7 script to /usr/local/bin Installed /usr/local/lib/python2.7/dist-packages/setuptools-0.6c11-py2.7.egg Processing dependencies for setuptools==0.6c11 Finished processing dependencies for setuptools==0.6c11
1.9 git
kylin@Ubuntu:~$ sudo apt-get install -y git
2 源码编译Ambari
2.1 下载源码
apache-ambari-2.4.2-src.tar.gz解压缩
kylin@Ubuntu:~$ tar -zxvf apache-ambari-2.4.2-src.tar.gz
2.2 安装node-gyp
进入源码根目录安装node-gypkylin@Ubuntu:~$ cd apache-ambari-2.4.2-src/ kylin@Ubuntu:~/apache-ambari-2.4.2-src$ ls ambari-admin ambari-client ambari-funtest ambari-metrics ambari-server ambari-views contrib docs LICENSE.txt NOTICE.txt pom.xml ambari-agent ambari-common ambari-logsearch ambari-project ambari-shell ambari-web dev-support KEYS logo.png pass.txt version kylin@Ubuntu:~/apache-ambari-2.4.2-src$ npm install node-gyp
2.3 准备编译
kylin@Ubuntu:~/apache-ambari-2.4.2-src$ mvn versions:set -DnewVersion=2.4.2.0.0 kylin@Ubuntu:~/apache-ambari-2.4.2-src$ pushd ambari-metrics ~/apache-ambari-2.4.2-src/ambari-metrics ~/apache-ambari-2.4.2-src kylin@Ubuntu:~/apache-ambari-2.4.2-src/ambari-metrics$ cd .. kylin@Ubuntu:~/apache-ambari-2.4.2-src$ popd ~/apache-ambari-2.4.2-src
2.4 开始编译
kylin@Ubuntu:~/apache-ambari-2.4.2-src$ mvn -B clean install package jdeb:jdeb -DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl
3、遇到问题：
问题1、Too many files with unapproved license
[ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.11:check (default) on project ambari: Too many files with unapproved license: 780 See RAT report in: /home/kylin/apache-ambari-2.4.2-src/target/rat.txt -> [Help 1]解决办法：加上-Drat.skip=true
kylin@Ubuntu:~/apache-ambari-2.4.2-src$ mvn -B install package jdeb:jdeb -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -Preplaceurl
问题2、#include
问题3、UnknownHostException Unknown host
问题3.1 Unknown host grafanarel.s3.amazonaws.com
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (default) on project ambari-metrics-grafana: An Ant BuildException has occured: java.net.UnknownHostException: grafanarel.s3.amazonaws.com[ERROR] around Ant part …… @ 5:217 in /home/kylin/apache-ambari-2.4.2-src/ambari-metrics/ambari-metrics-grafana/target/antrun/build-Download Ambari Grafana.xml: Unknown host grafanarel.s3.amazonaws.com
问题3.2 Unknown host private-repo-1.hortonworks.com
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (default) on project ambari-metrics-timelineservice: An Ant BuildException has occured: java.net.UnknownHostException: private-repo-1.hortonworks.com[ERROR] around Ant part …… @ 5:265 in /home/kylin/apache-ambari-2.4.2-src/ambari-metrics/ambari-metrics-timelineservice/target/antrun/build-Download HBase.xml: Unknown host private-repo-1.hortonworks.com
问题3.3 Unknown host www.w3.org
[ERROR] Failed to execute goal org.codehaus.mojo:xml-maven-plugin:1.0:transform (default) on project ambari-server: Failed to transform input file /home/kylin/apache-ambari-2.4.2-src/ambari-server/target/findbugs/findbugsXml.html: java.net.UnknownHostException: www.w3.org: Unknown host www.w3.org -> [Help 1]解决办法：重新执行
kylin@Ubuntu:~/apache-ambari-2.4.2-src$ mvn -B install package jdeb:jdeb -DnewVersion=2.4.2.0.0 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true
问题4、findbugsXml.html
[ERROR] Failed to execute goal org.codehaus.mojo:xml-maven-plugin:1.0:transform (default) on project ambari-server: Failed to transform input file /home/kylin/apache-ambari-2.4.2-src/ambari-server/target/findbugs/findbugsXml.html: The entity name must immediately follow the ‘&’ in the entity reference. -> [Help 1]解决办法1：
重新执行
kylin@Ubuntu:~/apache-ambari-2.4.2-src$ mvn -B install package jdeb:jdeb -DnewVersion=2.4.2.0.0 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true
解决办法2:
如果网络情况不好，这一步就很难过去。那只好修改pom.xml文件，将findbugs内容注释掉：
kylin@Ubuntu:~/apache-ambari-2.4.2-src/ambari-server$ vim pom.xml
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>findbugs-maven-plugin</artifactId>
<version>3.0.3</version>
<configuration>
<failOnError>false</failOnError>
<threshold>Low</threshold>
<findbugsXmlOutputDirectory>${project.basedir}/target/findbugs</findbugsXmlOutputDirectory>
</configuration>
<executions>
<!--
<execution>
<phase>verify</phase>
<goals>
<goal>check</goal>
</goals>
</execution>
-->
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>xml-maven-plugin</artifactId>
<version>1.0</version>
<executions>
<!--
<execution>
<phase>verify</phase>
<goals>
<goal>transform</goal>
</goals>
</execution>
-->
</executions>问题5、 is not a valid ‘control’ directory
参考：https://issues.apache.org/jira/browse/AMBARI-12977问题5.1 ambari-metrics-grafana
Caused by: org.vafer.jdeb.PackagingException: “/home/kylin/apache-ambari-2.4.2-src/ambari-metrics/ambari-metrics-grafana/src/main/package/deb/control” is not a valid ‘control’ directory)kylin@Ubuntu:~/apache-ambari-2.4.2-src/ambari-metrics/ambari-metrics-grafana$ vim pom.xml
在最后添加一个添加，如下代码
<plugin>
<groupId>org.vafer</groupId>
<artifactId>jdeb</artifactId>
<version>1.0.1</version>
<executions>
<execution>
<!--Stub execution on direct plugin call - workaround for ambari deb build process-->
<id>stub-execution</id>
<phase>none</phase>
<goals>
<goal>jdeb</goal>
</goals>
</execution>
</executions>
<configuration>
<skip>true</skip>
<attach>false</attach>
<submodules>false</submodules>
<controlDir>${project.basedir}/../src/main/package/deb/control</controlDir>
</configuration>
</plugin>
</plugins>
</build>问题5.2 ambari-funtest
[ERROR] Failed to create debian package /home/kylin/apache-ambari-2.4.2-src/ambari-funtest/target/ambari-funtest_2.4.2.0.0_all.deborg.vafer.jdeb.PackagingException: “/home/kylin/apache-ambari-2.4.2-src/ambari-funtest/src/main/package/deb/control” is not a valid ‘control’ directory)
解决办法：在上面添加一个元素
kylin@Ubuntu:~/apache-ambari-2.4.2-src/ambari-funtest$ vi pom.xml
<plugin>
<groupId>org.vafer</groupId>
<artifactId>jdeb</artifactId>
<version>1.0.1</version>
<executions>
<execution>
<!--Stub execution on direct plugin call - workaround for ambari deb build process-->
<id>stub-execution</id>
<phase>none</phase>
<goals>
<goal>jdeb</goal>
</goals>
</execution>
</executions>
<configuration>
<skip>true</skip>
<attach>false</attach>
<submodules>false</submodules>
<controlDir>${project.basedir}/../src/main/package/deb/control</controlDir>
</configuration>
</plugin>
</plugins>
<resources>
<resource>问题5.3 ambari-logsearch
[ERROR] Failed to execute goal org.vafer:jdeb:1.0.1:jdeb (default-cli) on project ambari-logsearch: Failed to create debian package /home/kylin/apache-ambari-2.4.2-src/ambari-logsearch/target/ambari-logsearch_2.4.2.0.0_all.deb: “/home/kylin/apache-ambari-2.4.2-src/ambari-logsearch/src/main/package/deb/control” is not a valid ‘control’ directory) -> [Help 1]解决办法：在上面添加一个元素
kylin@Ubuntu:~/apache-ambari-2.4.2-src/ambari-logsearch$ vim pom.xml
<plugin>
<groupId>org.vafer</groupId>
<artifactId>jdeb</artifactId>
<version>1.0.1</version>
<executions>
<execution>
<!--Stub execution on direct plugin call - workaround for ambari deb build process-->
<id>stub-execution</id>
<phase>none</phase>
<goals>
<goal>jdeb</goal>
</goals>
</execution>
</executions>
<configuration>
<skip>true</skip>
<attach>false</attach>
<submodules>false</submodules>
<controlDir>${project.basedir}/../src/main/package/deb/control</controlDir>
</configuration>
</plugin>
</plugins>
</build>4、编译完成
[INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Ambari Main ....................................... SUCCESS [1.620s] [INFO] Apache Ambari Project POM ......................... SUCCESS [0.078s] [INFO] Ambari Web ........................................ SUCCESS [59.983s] [INFO] Ambari Views ...................................... SUCCESS [0.906s] [INFO] Ambari Admin View ................................. SUCCESS [19.604s] [INFO] ambari-metrics .................................... SUCCESS [0.434s] [INFO] Ambari Metrics Common ............................. SUCCESS [0.421s] [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [2.242s] [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [1.049s] [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.697s] [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [2.175s] [INFO] Ambari Metrics Storm Sink (Legacy) ................ SUCCESS [1.617s] [INFO] Ambari Metrics Collector .......................... SUCCESS [13.404s] [INFO] Ambari Metrics Monitor ............................ SUCCESS [1.155s] [INFO] Ambari Metrics Grafana ............................ SUCCESS [15.134s] [INFO] Ambari Metrics Assembly ........................... SUCCESS [1:38.729s] [INFO] Ambari Server ..................................... SUCCESS [2:03.242s] [INFO] Ambari Functional Tests ........................... SUCCESS [0.445s] [INFO] Ambari Agent ...................................... SUCCESS [19.449s] [INFO] Ambari Client ..................................... SUCCESS [0.021s] [INFO] Ambari Python Client .............................. SUCCESS [0.741s] [INFO] Ambari Groovy Client .............................. SUCCESS [3.822s] [INFO] Ambari Shell ...................................... SUCCESS [0.056s] [INFO] Ambari Python Shell ............................... SUCCESS [0.744s] [INFO] Ambari Groovy Shell ............................... SUCCESS [1.118s] [INFO] ambari-logsearch .................................. SUCCESS [0.014s] [INFO] Ambari Logsearch Appender ......................... SUCCESS [0.287s] [INFO] Ambari Logsearch Solr Client ...................... SUCCESS [1.501s] [INFO] Ambari Logsearch Portal ........................... SUCCESS [3:58.085s] [INFO] Ambari Logsearch Log Feeder ....................... SUCCESS [7:39.905s] [INFO] Ambari Logsearch Assembly ......................... SUCCESS [0.155s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 17:49.903s [INFO] Finished at: Wed Mar 01 15:43:16 CST 2017 [INFO] Final Memory: 231M/1188M [INFO] ------------------------------------------------------------------------
备注：
Ambari Server will create following packagesRPM will be created under
AMBARI_DIR/ambari-server/target/rpm/ambari-server/RPMS/noarch.
DEB will be created under AMBARI_DIR/ambari-server/target/
Ambari Agent will create following packages
RPM will be created under
AMBARI_DIR/ambari-agent/target/rpm/ambari-agent/RPMS/x86_64.
DEB will be created under AMBARI_DIR/ambari-agent/target
Ubuntu 14
1)Ambari Repositories
Base URL http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.4.2.0
Repo File http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.4.2.0/ambari.list
Tarball md5 http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.4.2.0/ambari-2.4.2.0-ubuntu14.tar.gz
2)HDP 2.5 Repositories
Repo File http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.5.3.0/hdp.list
Tarball md5 http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.5.3.0/HDP-2.5.3.0-ubuntu14-deb.tar.gz
Tarball md5 http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ubuntu14/HDP-UTILS-1.1.0.21-ubuntu14.tar.gz
内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理 
相关文章推荐
- Ubuntu/UbuntuKylin平台下源码编译Ambari 2.4（含较多问题解决方案）
- Ubuntu环境下 Android平台源码的下载及编译
- ubuntu环境：下载、编译模拟器平台android内核源码
- ubuntu 下基于源码的android平台搭建与内核编译
- Ubuntu环境下 Android平台源码的下载及编译
- ubuntu下mtk6573平台android源码编译命令
- Android平台源码下载以及编译--ubuntu 11.04
- Ubuntu环境Android平台源码下载及编译 .
- centos6下源码编译安装LAMP平台(apache2.4+php5.4+mysql5.5.30)
- Ubuntu环境下 Android平台源码的下载及编译
- Ubuntu环境Android平台源码下载及编译
- Ambari2.4 源码编译
- ubuntu10.04下audacious2.4源码编译过程（解决2.3cue的bug）
- AMD64位ubuntu9.04下编译Android源码
- Ubuntu 下编译Android源码全过程
- Ubuntu 9.10下的Android源码下载及编译
- 在ubuntu 上下载 编译android 源码
- ubuntu下基于源码的android平台搭建
- Ubuntu 安装texlive 并源码编译安装texworks lyx