core-site.xml、hdfs-site.xml、yarn-site.xml 和一些jdk等环境变量
2020-07-12 16:58
274 查看
hadoop +zookeeper的文件配置
hadoop-env.sh:
'#' The java implementation to use. export JAVA_HOME=/opt/jdk1.8.0_221
core-site.xml
<configuration> <property> <name>fs.defaultFS</name> <value>hdfs://mycluster</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/opt/soft/hadoop260/tmp</value> </property> <property> <name>ha.zookeeper.quorum</name> <value>hadoop01:2181,hadoop02:2181,hadoop03:2181</value> </property> <property> <name>hadoop.proxyuser.root.hosts</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.root.groups</name> <value>*</value> </property> </configuration>
hdfs-site.xml
<configuration> <property> <name>dfs.nameservices</name> <value>mycluster</value> </property> <property> <name>dfs.ha.namenodes.mycluster</name> <value>nn1,nn2</value> </property> <property> <name>dfs.namenode.rpc-address.mycluster.nn1</name> <value>hadoop01:9000</value> </property> <property> <name>dfs.namenode.rpc-address.mycluster.nn2</name> <value>hadoop02:9000</value> </property> <property> <name>dfs.namenode.http-address.mycluster.nn1</name> <value>hadoop01:50070</value> </property> <property> <name>dfs.namenode.http-address.mycluster.nn2</name> <value>hadoop02:50070</value> </property> <property> <name>dfs.namenode.shared.edits.dir</name> <value>qjournal://hadoop01:8485;hadoop02:8485;hadoop03:8485/mycluster</value> </property> <property> <name>dfs.ha.fencing.methods</name> <value>sshfence</value> </property> <!--在hadoop里配置免密登录--> <property> <name>dfs.ha.fencing.ssh.private-key-files</name> <value>/root/.ssh/id_rsa</value> </property> <!--接收journalnode信息--> <property> <name>dfs.journalnode.edits.dir</name> <value>/opt/soft/hadoop260/data/jn</value> </property> <property> <name>dfs.permissions.enable</name> <value>false</value> </property> <!----> <property> <name>dfs.client.failover.proxy.provider.mycluster</name> <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value> </property> <!--DataNode副本个数--> <property> <name>dfs.replication</name> <value>3</value> </property> <!--助手配置--> <property> <name>dfs.namenode.secondary.http-address</name> <value>hadoop03:50090</value> </property> <property> <name>dfs.ha.automatic-failover.enabled</name> <value>true</value> </property> </configuration>
mapred-site.xml
<configuration> <property> <name>mapreduce.framework.name </name><value>yarn</value> </property> </configuration>
yarn-site.xml
<configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.resourcemanager.ha.enabled</name> <value>true</value> </property> <property> <name>yarn.resourcemanager.cluster-id</name> <value>cluster-yarn1</value> </property> <property> <name>yarn.resourcemanager.ha.rm-ids</name> <value>rm1,rm2</value> </property> <property> <name>yarn.resourcemanager.hostname.rm1</name> <value>hadoop01</value> </property> <property> <name>yarn.resourcemanager.hostname.rm2</name> <value>hadoop02</value> </property> <property> <name>yarn.resourcemanager.zk-address</name> <value>hadoop01:2181,hadoop02:2181,hadoop03:2181</value> </property> <property> <name>yarn.resourcemanager.recovery.enabled</name> <value>true</value> </property> <property> <name>yarn.resourcemanager.store.class</name> <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value> </property> </configuration>
Hadoop环境变量配置
vi /etc/profile
export HADOOP_HOME=/opt/hadoop export HADOOP_MAPRED_HOME=$HADOOP_HOME export HADOOP_COMMON_HOME=$HADOOP_HOME export HADOOP_HDFS_HOME=$HADOOP_HOME export YARN_HOME=$HADOOP_HOME export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib" export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
jdk环境:
export JAVA_HOME=/opt/jdk1.8.0_221 exportCLASSPATH=.:$JAVA_HOME/rt.jar:$JAVA_HOME/tools.jar:$JAVA_HOME/dt.jar export JRE_HOME=$JAVA_HOME/jre export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
相关文章推荐
- hadoop搭建四个配置文件(core-site.xml hdfs-site.xml mapred-site.xml yarn-site.xml )的简单介绍
- hadoop搭建四个配置文件(core-site.xml hdfs-site.xml mapred-site.xml yarn-site.xml )的简单介绍
- HDFD 四个配置文件(core-site.xml hdfs-site.xml mapred-site.xml yarn-site.xml )的简单介绍
- Hadoop小兵笔记【一】Hadoop安装过程中core-site.xml,hdfs-site.xml,mapred-site.xml配置文件详解
- Hadoop配置: core-site.xml hdfs-site.xml mapred-site.xml
- hadoop三个配置文件的参数含义说明core-site.xml,hdfs-site.xml,mapred-site.xml
- Hadoop笔记之core-site.xml && hdfs-site.xml&&mapred-site.xml分析&&hadoop等集群环境搭建梳理
- Hadoop学习5_Hadoop:hdfs-site.xml,mapred-site.xml,core-site.xml 配置项清单
- hadoop三个配置文件的参数含义说明core-site.xml、hdfs-site.xml、mapred-site.xml
- hadoop1.21配置+hdfs-site.xml+mapred-site.xml+Core-site.xml
- HDFS For core-site.xml
- hadoop三个配置文件的参数含义说明core-site.xml、hdfs-site.xml、mapred-site.xml
- HDFS 2.7.4中hdfs-site.xml参数未配置引发的一些异常
- hadoop配置文件core-site.xml、hdfs-site.xml,mapred-site.xml详解
- yarn-site.xml
- Hadoop 參數設定 – hdfs-site.xml
- yarn-site.xml相关配置参数
- Hadoop的core-site.xml配置文件
- hadoop2.2.0 core-site.xml--file system properties
- yarn-site.xml