Hadoop安装记录(伪分布式)
2016-03-13 20:19
274 查看
下面给大家分享一下我在安装Hadoop时候的一些步骤记录,希望能给大家带来帮助,我安装的是单节点的伪分布式。
1.基本环境:VMware8.0 + CentOS 6.0
2.软件版本:hadoop-0.20.2-cdh3u1.tar.gz + jdk-6u29-linux-i586.bin
第一步,将所需要的软件从本地共享到虚拟机,我们可以使用VMware Tools的Share共享文件夹功能
我们在/usr下新建/hadoopsoft文件夹,把需要的软件拷贝进去:
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/889e3ef351d753c271300a167d1407a7.png)
第二步,安装jdk
首先改变hadoopsoft目录为可执行目录,并安装jdk
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/e86ccd1259ac6a3f7f49dd1382010e88.png)
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/71f5268ffa044ef59aa4123dcd6f0af4.png)
第三步,安装hadoop
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/b9f9322a8a632d4cb659d1852f97650c.png)
第四步,创建hadoop的用户组,并创建相应用户hadoopuser
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/063f737463de4215f3f2a789bddf3e3a.png)
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/5c4aa0038959f786ba87fb51204b413e.png)
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/d47a3eafe8cf6f57e959b0575f819227.png)
第五步,配置环境变量
打开/etc/profile:
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/6fb9dca388a48999dd5784f2f5e6da9a.png)
配置完环境变量以后需要重启一下CentOS,然后验证是否成功:
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/2e9f7b2f846a87ba979d662afee0c230.png)
第六步,配置SSH免登陆认证
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/735e8fe0594aac29b13d779545d2fd69.png)
如果配置不成功需要赋予authorized_keys相应的权限!
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/186f054ed1e16c558e24036518ac20bb.png)
第七步,配置hadoop配置文件
1.hadoop-env.sh 为hadoop添加java环境变量
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/dbd39acd2f97f726880fb5ab8f982329.png)
(为了一下配置的方便,我们先为ip配置一个host)
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/beaec718315d12b89213594a36237eb7.png)
同时也要修改/hadoop/conf下的masters和slaves文件,修改成相应的host名。
2.core-site.xml
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/4455c98e1c36b0253b1313614c5e4127.png)
3.mapred-site.xml
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/3d807ea3efa907e891f0937059901c1f.png)
第八步,为hadoopuser用户赋予hadoop文件夹的使用权限
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/a2da4de2ae748cb44de5cae5cd529926.png)
第九步,格式化namenode
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/1882f60b21469d93312d7827106998c9.png)
如果格式化不成功,需要删除hadoop临时文件夹下的文件,然后重新执行namenode格式化:
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/3134c6d2709c2730aeb121fa915af456.png)
第十步,启动hadoop
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/7279444e95479285990f60f86cb13bb7.png)
如果这些节点都已经启动,则代表hadoop已经正常启动,通过在浏览器里查看:
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/5246a02ae4181dae5f38fbc1a9925ceb.png)
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/0a07de83e83647a177d3a38d61a57469.png)
以上就是我在安装单节点hadoop的配置过程,如有什么疑问欢迎下方跟评论!
1.基本环境:VMware8.0 + CentOS 6.0
2.软件版本:hadoop-0.20.2-cdh3u1.tar.gz + jdk-6u29-linux-i586.bin
第一步,将所需要的软件从本地共享到虚拟机,我们可以使用VMware Tools的Share共享文件夹功能
我们在/usr下新建/hadoopsoft文件夹,把需要的软件拷贝进去:
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/889e3ef351d753c271300a167d1407a7.png)
第二步,安装jdk
首先改变hadoopsoft目录为可执行目录,并安装jdk
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/e86ccd1259ac6a3f7f49dd1382010e88.png)
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/71f5268ffa044ef59aa4123dcd6f0af4.png)
第三步,安装hadoop
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/b9f9322a8a632d4cb659d1852f97650c.png)
第四步,创建hadoop的用户组,并创建相应用户hadoopuser
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/063f737463de4215f3f2a789bddf3e3a.png)
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/5c4aa0038959f786ba87fb51204b413e.png)
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/d47a3eafe8cf6f57e959b0575f819227.png)
第五步,配置环境变量
打开/etc/profile:
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/6fb9dca388a48999dd5784f2f5e6da9a.png)
配置完环境变量以后需要重启一下CentOS,然后验证是否成功:
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/2e9f7b2f846a87ba979d662afee0c230.png)
第六步,配置SSH免登陆认证
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/735e8fe0594aac29b13d779545d2fd69.png)
如果配置不成功需要赋予authorized_keys相应的权限!
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/186f054ed1e16c558e24036518ac20bb.png)
第七步,配置hadoop配置文件
1.hadoop-env.sh 为hadoop添加java环境变量
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/dbd39acd2f97f726880fb5ab8f982329.png)
(为了一下配置的方便,我们先为ip配置一个host)
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/beaec718315d12b89213594a36237eb7.png)
同时也要修改/hadoop/conf下的masters和slaves文件,修改成相应的host名。
2.core-site.xml
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/4455c98e1c36b0253b1313614c5e4127.png)
3.mapred-site.xml
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/3d807ea3efa907e891f0937059901c1f.png)
第八步,为hadoopuser用户赋予hadoop文件夹的使用权限
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/a2da4de2ae748cb44de5cae5cd529926.png)
第九步,格式化namenode
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/1882f60b21469d93312d7827106998c9.png)
如果格式化不成功,需要删除hadoop临时文件夹下的文件,然后重新执行namenode格式化:
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/3134c6d2709c2730aeb121fa915af456.png)
第十步,启动hadoop
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/7279444e95479285990f60f86cb13bb7.png)
如果这些节点都已经启动,则代表hadoop已经正常启动,通过在浏览器里查看:
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/5246a02ae4181dae5f38fbc1a9925ceb.png)
![](https://oscdn.geek-share.com/Uploads/Images/Content/202001/28/0a07de83e83647a177d3a38d61a57469.png)
以上就是我在安装单节点hadoop的配置过程,如有什么疑问欢迎下方跟评论!
相关文章推荐
- linux下如何用命令配置网卡、dns、网关、如何查看网卡状态、如何查看网卡信息。
- 在Hadoop上安装Hive
- 在Hadoop上安装HBase
- ganglia监控kafka
- centos6用户和权限管理
- 安装GDK
- 线段树(维护最大值):HDU Billboard
- Nginx日志文件切割
- Linux内核启动分析过程
- Linux学习计划
- Photoshop的评价
- 【BZOJ1095】【ZJOI2007】捉迷藏 括号序列+线段树维护
- Linux使用RPM方式安装MySQL
- Nginx配置文件nginx.conf中文详解(总结)
- grep,egrep的用法
- linux作为服务器,利用top命令查看服务进程的耗用情况
- nginx(七)gzip压缩功能
- Properties的用法和案例分析
- CentOS7在VM中连接Xshell
- java从菜鸟到架构师的必看书籍 .