Ubuntu16.04安装配置hadoop-2.7.4
2017-09-15 16:35
519 查看
写下来自己记录下,搭hadoop的伪分布式虽然很简单,但是有比较多的坑!
1.首先去下载包,然后解压
这里就不用多做说明了
2.接下来是profile的配置,这个东西比较的烦人:
export JAVA_HOME=/opt/jdk
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export HADOOP_HOME=/opt/hadoop-2.7.4
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop这个是直接可以用的,我试过!
3.最恶心的就是下面一点了:
hadoop配置文件:
1.core-site.xml
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>file:/opt/hadoop-2.7.4/tmp</value>
<description>Abase for other temporary directories.</description>
</property>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>2.hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/opt/hadoop-2.7.4/tmp/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/opt/hadoop-2.7.4/tmp/dfs/data</value>
</property>
</configuration>这里你首先要把那些个文件建好,tmp,dfs/data,dfs/name
运行过程是这样的:
1.先格式化namenode:
./hadoop namenode -format
2.格式化成功就可以运行了:
./sbin/start-dfs.sh注意:但是实际上你是不能运行的报的错是这个
Starting namenodes on [localhost]
localhost: Error: JAVA_HOME is not set and could not be found.
localhost: Error: JAVA_HOME is not set and could not be found.
Starting secondary namenodes [account.jetbrains.com]
account.jetbrains.com: Error: JAVA_HOME is not set and could not be found.但是实际上你的JAVA_HOME是配置了,解决方法如下:
vi /home/hadoop-2.7.4/etc/hadoop/hadoop-env.sh 设置JAVA_HOME
vi /home/hadoop-2.7.4/etc/hadoop/mapred-env.sh 设置JAVA_HOME
去配置下这个,我被这个搞死了!
1.首先去下载包,然后解压
这里就不用多做说明了
2.接下来是profile的配置,这个东西比较的烦人:
export JAVA_HOME=/opt/jdk
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export HADOOP_HOME=/opt/hadoop-2.7.4
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop这个是直接可以用的,我试过!
3.最恶心的就是下面一点了:
hadoop配置文件:
1.core-site.xml
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>file:/opt/hadoop-2.7.4/tmp</value>
<description>Abase for other temporary directories.</description>
</property>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>2.hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/opt/hadoop-2.7.4/tmp/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/opt/hadoop-2.7.4/tmp/dfs/data</value>
</property>
</configuration>这里你首先要把那些个文件建好,tmp,dfs/data,dfs/name
运行过程是这样的:
1.先格式化namenode:
./hadoop namenode -format
2.格式化成功就可以运行了:
./sbin/start-dfs.sh注意:但是实际上你是不能运行的报的错是这个
Starting namenodes on [localhost]
localhost: Error: JAVA_HOME is not set and could not be found.
localhost: Error: JAVA_HOME is not set and could not be found.
Starting secondary namenodes [account.jetbrains.com]
account.jetbrains.com: Error: JAVA_HOME is not set and could not be found.但是实际上你的JAVA_HOME是配置了,解决方法如下:
vi /home/hadoop-2.7.4/etc/hadoop/hadoop-env.sh 设置JAVA_HOME
vi /home/hadoop-2.7.4/etc/hadoop/mapred-env.sh 设置JAVA_HOME
去配置下这个,我被这个搞死了!
相关文章推荐
- Ubuntu16.04之Hadoop2.7.3集群安装配置教程
- Hadoop安装教程_单机/伪分布式配置_Hadoop2.7.3/Ubuntu16.04
- Hadoop安装教程_单机/伪分布式配置_Hadoop2.6.4/Ubuntu16.04
- 大数据-ubuntu16.04上Hadoop安装教程_单机配置
- hadoop学习笔记之前期准备 ubuntu16.04虚拟机安装、WMwareTools安装及网络配置
- 从零开始安装配置Hadoop 2.7.2+Spark 2.0.0到Ubuntu 16.04
- 大数据基础(五)从零开始安装配置Hadoop 2.7.2+Spark 2.0.0到Ubuntu 16.04
- Hadoop安装教程_单机配置_Hadoop1.2.1/Ubuntu16.04
- Ubuntu16.04 下 hadoop的安装与配置(伪分布式环境)
- Ubuntu16.04下伪分布式环境搭建之hadoop2.6.0、jdk1.7、Hbase0.98的安装与配置
- Hadoop安装(版本一)———Ubuntu16.04 下 hadoop的安装与配置(伪分布式环境)
- ubuntu16.04 64位安装HTS-2.3(包括前端配置)【待更】
- Hadoop安装教程_单机/伪分布式配置_Hadoop2.6.0/Ubuntu14.04
- ubuntu16.04环境下安装配置openface人脸识别程序
- Ubuntu 13.10下Hadoop 2.2 安装、配置、编译(伪分布式)
- Hadoop安装教程_单机/伪分布式配置_Hadoop2.6.0/Ubuntu14.04
- ubuntu16.04安装jdk并配置
- [置顶] Ubuntu16.04下安装tensorflow并配置GPU
- 新手推荐:Hadoop安装教程_单机/伪分布式配置_Hadoop-2.7.1/Ubuntu14.04