根据硬件配置后mapred-site.xml和yarn-site.xml
2016-01-30 13:45
288 查看
机器总共16G内存,sqoop导入数据时大约需要2G左右
mapred-site.xml
yarn-site.xml
mapred-site.xml
<configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> <property> <name>mapreduce.jobhistory.address</name> <value>master:10020</value> <description>MapReduce JobHistory Server IPC host:port</description> </property> <property> <name>mapreduce.jobhistory.webapp.address</name> <value>master:19888</value> <description>MapReduce JobHistory Server Web UI host:port</description> </property> <property> <name>mapred.child.java.opts</name> <value>-Xmx3000m -Xmx3000m -XX:-UseGCOverheadLimit</value> </property> <property> <name>mapreduce.reduce.memory.mb</name> <value>4096</value> </property> <property> <name>mapreduce.map.memory.mb</name> <value>4096</value> </property> <property> <name>mapreduce.task.timeout</name> <value>600000</value> </property> </configuration>
yarn-site.xml
<configuration> <property> <name>yarn.resourcemanager.hostname</name> <value>master</value> </property> <property> <description>The address of the applications manager interface in the RM.</description> <name>yarn.resourcemanager.address</name> <value>${yarn.resourcemanager.hostname}:8032</value> </property> <property> <description>The address of the scheduler interface.</description> <name>yarn.resourcemanager.scheduler.address</name> <value>${yarn.resourcemanager.hostname}:8030</value> </property> <property> <description>The http address of the RM web application.</description> <name>yarn.resourcemanager.webapp.address</name> <value>${yarn.resourcemanager.hostname}:8088</value> </property> <property> <description>The https adddress of the RM web application.</description> <name>yarn.resourcemanager.webapp.https.address</name> <value>${yarn.resourcemanager.hostname}:8090</value> </property> <property> <name>yarn.resourcemanager.resource-tracker.address</name> <value>${yarn.resourcemanager.hostname}:8031</value> </property> <property> <description>The address of the RM admin interface.</description> <name>yarn.resourcemanager.admin.address</name> <value>${yarn.resourcemanager.hostname}:8033</value> </property> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.scheduler.maximum-allocation-mb</name> <value>9216</value> <discription>每个节点可用内存,单位MB,默认8182MB</discription> </property> <property> <name>yarn.nodemanager.vmem-pmem-ratio</name> <value>2.1</value> </property> <property> <name>yarn.nodemanager.resource.memory-mb</name> <value>10240</value> </property> </configuration>
相关文章推荐
- js弹窗层支持html代码
- react-native之js事件发送并得到返回结果
- runtime error 扩栈
- Leetcode题解(十八)
- Hue 3.9 on HDP2.3.4 安装备忘
- Android判断用户是平板还是手机的方法
- Android分享笔记(1) 获取屏幕尺寸,包括状态栏
- Java基础(八)——集合
- LeetCode 228 Summary Ranges(值域)(*)
- 左旋转字符串
- zabbix搭建
- keepalived+mysql双主高可用配置
- C语言之链表————(转载)
- mysql 优化技巧
- 联想正遭遇第四道坎 柳传志对症下药
- Android 向右滑动销毁(finish)Activity, 随着手势的滑动而滑动并跳转的效果(二)
- CC150 chapter 8 OOD
- 【选择图片Jar】完成选择图片后:预览图片的实现(八)
- final 修饰符能提升 Java 程序的性能吗
- 二叉搜索树