hadoop-运行hadoop jar hadoop-examples-1.2.1.jar wordcount /wc/input/ /wc/output/
2016-01-20 15:07
387 查看
查看结果:
在地址栏:输入 主机名:50070
点击_logs 下面的进入统计单词的结果。
[root@redhat5 hadoop-1.2.1]# hadoop jar hadoop-examples-1.2.1.jar wordcount /wc/input/ /wc/output/
16/01/20 22:27:56 INFO input.FileInputFormat: Total input paths to process : 7
16/01/20 22:27:56 INFO util.NativeCodeLoader: Loaded the native-hadoop library
16/01/20 22:27:56 WARN snappy.LoadSnappy: Snappy native library not loaded
16/01/20 22:27:59 INFO mapred.JobClient: Running job: job_201601202034_0003
16/01/20 22:28:00 INFO mapred.JobClient: map 0% reduce 0%
16/01/20 22:29:27 INFO mapred.JobClient: map 28% reduce 0%
16/01/20 22:30:25 INFO mapred.JobClient: map 42% reduce 0%
16/01/20 22:30:27 INFO mapred.JobClient: map 42% reduce 9%
16/01/20 22:30:29 INFO mapred.JobClient: map 57% reduce 9%
16/01/20 22:30:34 INFO mapred.JobClient: map 57% reduce 14%
16/01/20 22:30:41 INFO mapred.JobClient: map 57% reduce 19%
16/01/20 22:31:01 INFO mapred.JobClient: map 71% reduce 19%
16/01/20 22:31:04 INFO mapred.JobClient: map 85% reduce 19%
16/01/20 22:31:10 INFO mapred.JobClient: map 85% reduce 28%
16/01/20 22:31:25 INFO mapred.JobClient: map 100% reduce 28%
16/01/20 22:31:36 INFO mapred.JobClient: map 100% reduce 92%
16/01/20 22:31:37 INFO mapred.JobClient: map 100% reduce 100%
16/01/20 22:31:52 INFO mapred.JobClient: Job complete: job_201601202034_0003
16/01/20 22:31:52 INFO mapred.JobClient: Counters: 29
16/01/20 22:31:52 INFO mapred.JobClient: Job Counters
16/01/20 22:31:53 INFO mapred.JobClient: Launched reduce tasks=1
16/01/20 22:31:53 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=372203
16/01/20 22:31:53 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
16/01/20 22:31:53 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
16/01/20 22:31:53 INFO mapred.JobClient: Launched map tasks=7
16/01/20 22:31:53 INFO mapred.JobClient: Data-local map tasks=7
16/01/20 22:31:53 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=127071
16/01/20 22:31:53 INFO mapred.JobClient: File Output Format Counters
16/01/20 22:31:53 INFO mapred.JobClient: Bytes Written=6549
16/01/20 22:31:53 INFO mapred.JobClient: FileSystemCounters
16/01/20 22:31:53 INFO mapred.JobClient: FILE_BYTES_READ=10600
16/01/20 22:31:53 INFO mapred.JobClient: HDFS_BYTES_READ=16263
16/01/20 22:31:53 INFO mapred.JobClient: FILE_BYTES_WRITTEN=462821
16/01/20 22:31:53 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=6549
16/01/20 22:31:53 INFO mapred.JobClient: File Input Format Counters
16/01/20 22:31:53 INFO mapred.JobClient: Bytes Read=15486
16/01/20 22:31:53 INFO mapred.JobClient: Map-Reduce Framework
16/01/20 22:31:53 INFO mapred.JobClient: Map output materialized bytes=10636
16/01/20 22:31:53 INFO mapred.JobClient: Map input records=389
16/01/20 22:31:53 INFO mapred.JobClient: Reduce shuffle bytes=10636
16/01/20 22:31:53 INFO mapred.JobClient: Spilled Records=1196
16/01/20 22:31:53 INFO mapred.JobClient: Map output bytes=21294
16/01/20 22:31:53 INFO mapred.JobClient: Total committed heap usage (bytes)=1323302912
16/01/20 22:31:53 INFO mapred.JobClient: CPU time spent (ms)=38790
16/01/20 22:31:53 INFO mapred.JobClient: Combine input records=1761
16/01/20 22:31:53 INFO mapred.JobClient: SPLIT_RAW_BYTES=777
16/01/20 22:31:53 INFO mapred.JobClient: Reduce input records=598
16/01/20 22:31:53 INFO mapred.JobClient: Reduce input groups=427
16/01/20 22:31:53 INFO mapred.JobClient: Combine output records=598
16/01/20 22:31:53 INFO mapred.JobClient: Physical memory (bytes) snapshot=1390465024
16/01/20 22:31:53 INFO mapred.JobClient: Reduce output records=427
16/01/20 22:31:53 INFO mapred.JobClient: Virtual memory (bytes) snapshot=4970704896
16/01/20 22:31:53 INFO mapred.JobClient: Map output records=1761
在命令行:查看
在命令下查看结果:
[root@redhat5 conf]# hadoop fs -lsr /wc/input/
-rw-r--r-- 1 root supergroup 7457 2016-01-20 06:20 /wc/input/capacity-scheduler.xml
-rw-r--r-- 1 root supergroup 378 2016-01-20 06:21 /wc/input/core-site.xml
-rw-r--r-- 1 root supergroup 327 2016-01-20 06:21 /wc/input/fair-scheduler.xml
-rw-r--r-- 1 root supergroup 4644 2016-01-20 06:21 /wc/input/hadoop-policy.xml
-rw-r--r-- 1 root supergroup 358 2016-01-20 06:21 /wc/input/hdfs-site.xml
-rw-r--r-- 1 root supergroup 2033 2016-01-20 06:21 /wc/input/mapred-queue-acls.xml
-rw-r--r-- 1 root supergroup 289 2016-01-20 06:21 /wc/input/mapred-site.xml
[root@redhat5 conf]# hadoop fs -lsr /wc/output/
-rw-r--r-- 1 root supergroup 0 2016-01-20 22:31 /wc/output/_SUCCESS
drwxr-xr-x - root supergroup 0 2016-01-20 22:27 /wc/output/_logs
drwxr-xr-x - root supergroup 0 2016-01-20 22:27 /wc/output/_logs/history
-rw-r--r-- 1 root supergroup 30337 2016-01-20 22:27 /wc/output/_logs/history/job_201601202034_0003_1453300078993_root_word+count
-rw-r--r-- 1 root supergroup 48088 2016-01-20 22:27 /wc/output/_logs/history/job_201601202034_0003_conf.xml
-rw-r--r-- 1 root supergroup 6549 2016-01-20 22:31 /wc/output/part-r-00000
[root@redhat5 conf]# hadoop fs -text /wc/output/part-r-00000
在地址栏:输入 主机名:50070
点击_logs 下面的进入统计单词的结果。
[root@redhat5 hadoop-1.2.1]# hadoop jar hadoop-examples-1.2.1.jar wordcount /wc/input/ /wc/output/
16/01/20 22:27:56 INFO input.FileInputFormat: Total input paths to process : 7
16/01/20 22:27:56 INFO util.NativeCodeLoader: Loaded the native-hadoop library
16/01/20 22:27:56 WARN snappy.LoadSnappy: Snappy native library not loaded
16/01/20 22:27:59 INFO mapred.JobClient: Running job: job_201601202034_0003
16/01/20 22:28:00 INFO mapred.JobClient: map 0% reduce 0%
16/01/20 22:29:27 INFO mapred.JobClient: map 28% reduce 0%
16/01/20 22:30:25 INFO mapred.JobClient: map 42% reduce 0%
16/01/20 22:30:27 INFO mapred.JobClient: map 42% reduce 9%
16/01/20 22:30:29 INFO mapred.JobClient: map 57% reduce 9%
16/01/20 22:30:34 INFO mapred.JobClient: map 57% reduce 14%
16/01/20 22:30:41 INFO mapred.JobClient: map 57% reduce 19%
16/01/20 22:31:01 INFO mapred.JobClient: map 71% reduce 19%
16/01/20 22:31:04 INFO mapred.JobClient: map 85% reduce 19%
16/01/20 22:31:10 INFO mapred.JobClient: map 85% reduce 28%
16/01/20 22:31:25 INFO mapred.JobClient: map 100% reduce 28%
16/01/20 22:31:36 INFO mapred.JobClient: map 100% reduce 92%
16/01/20 22:31:37 INFO mapred.JobClient: map 100% reduce 100%
16/01/20 22:31:52 INFO mapred.JobClient: Job complete: job_201601202034_0003
16/01/20 22:31:52 INFO mapred.JobClient: Counters: 29
16/01/20 22:31:52 INFO mapred.JobClient: Job Counters
16/01/20 22:31:53 INFO mapred.JobClient: Launched reduce tasks=1
16/01/20 22:31:53 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=372203
16/01/20 22:31:53 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
16/01/20 22:31:53 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
16/01/20 22:31:53 INFO mapred.JobClient: Launched map tasks=7
16/01/20 22:31:53 INFO mapred.JobClient: Data-local map tasks=7
16/01/20 22:31:53 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=127071
16/01/20 22:31:53 INFO mapred.JobClient: File Output Format Counters
16/01/20 22:31:53 INFO mapred.JobClient: Bytes Written=6549
16/01/20 22:31:53 INFO mapred.JobClient: FileSystemCounters
16/01/20 22:31:53 INFO mapred.JobClient: FILE_BYTES_READ=10600
16/01/20 22:31:53 INFO mapred.JobClient: HDFS_BYTES_READ=16263
16/01/20 22:31:53 INFO mapred.JobClient: FILE_BYTES_WRITTEN=462821
16/01/20 22:31:53 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=6549
16/01/20 22:31:53 INFO mapred.JobClient: File Input Format Counters
16/01/20 22:31:53 INFO mapred.JobClient: Bytes Read=15486
16/01/20 22:31:53 INFO mapred.JobClient: Map-Reduce Framework
16/01/20 22:31:53 INFO mapred.JobClient: Map output materialized bytes=10636
16/01/20 22:31:53 INFO mapred.JobClient: Map input records=389
16/01/20 22:31:53 INFO mapred.JobClient: Reduce shuffle bytes=10636
16/01/20 22:31:53 INFO mapred.JobClient: Spilled Records=1196
16/01/20 22:31:53 INFO mapred.JobClient: Map output bytes=21294
16/01/20 22:31:53 INFO mapred.JobClient: Total committed heap usage (bytes)=1323302912
16/01/20 22:31:53 INFO mapred.JobClient: CPU time spent (ms)=38790
16/01/20 22:31:53 INFO mapred.JobClient: Combine input records=1761
16/01/20 22:31:53 INFO mapred.JobClient: SPLIT_RAW_BYTES=777
16/01/20 22:31:53 INFO mapred.JobClient: Reduce input records=598
16/01/20 22:31:53 INFO mapred.JobClient: Reduce input groups=427
16/01/20 22:31:53 INFO mapred.JobClient: Combine output records=598
16/01/20 22:31:53 INFO mapred.JobClient: Physical memory (bytes) snapshot=1390465024
16/01/20 22:31:53 INFO mapred.JobClient: Reduce output records=427
16/01/20 22:31:53 INFO mapred.JobClient: Virtual memory (bytes) snapshot=4970704896
16/01/20 22:31:53 INFO mapred.JobClient: Map output records=1761
在命令行:查看
在命令下查看结果:
[root@redhat5 conf]# hadoop fs -lsr /wc/input/
-rw-r--r-- 1 root supergroup 7457 2016-01-20 06:20 /wc/input/capacity-scheduler.xml
-rw-r--r-- 1 root supergroup 378 2016-01-20 06:21 /wc/input/core-site.xml
-rw-r--r-- 1 root supergroup 327 2016-01-20 06:21 /wc/input/fair-scheduler.xml
-rw-r--r-- 1 root supergroup 4644 2016-01-20 06:21 /wc/input/hadoop-policy.xml
-rw-r--r-- 1 root supergroup 358 2016-01-20 06:21 /wc/input/hdfs-site.xml
-rw-r--r-- 1 root supergroup 2033 2016-01-20 06:21 /wc/input/mapred-queue-acls.xml
-rw-r--r-- 1 root supergroup 289 2016-01-20 06:21 /wc/input/mapred-site.xml
[root@redhat5 conf]# hadoop fs -lsr /wc/output/
-rw-r--r-- 1 root supergroup 0 2016-01-20 22:31 /wc/output/_SUCCESS
drwxr-xr-x - root supergroup 0 2016-01-20 22:27 /wc/output/_logs
drwxr-xr-x - root supergroup 0 2016-01-20 22:27 /wc/output/_logs/history
-rw-r--r-- 1 root supergroup 30337 2016-01-20 22:27 /wc/output/_logs/history/job_201601202034_0003_1453300078993_root_word+count
-rw-r--r-- 1 root supergroup 48088 2016-01-20 22:27 /wc/output/_logs/history/job_201601202034_0003_conf.xml
-rw-r--r-- 1 root supergroup 6549 2016-01-20 22:31 /wc/output/part-r-00000
[root@redhat5 conf]# hadoop fs -text /wc/output/part-r-00000
相关文章推荐
- OpenSSH客户端漏洞:CVE-2016-0777和CVE-2016-0778
- apache域名重定向301跳转 .htaccess的写法
- Linux系统中的信号量(semphore)与互斥体(mutex)
- centos 6.5 升级php到5.6.17版本
- Linux查找和替换目录下所有字符串【转】
- CentOS开启mcrypt模块
- ANALYSIS AND EXPLOITATION OF A LINUX KERNEL VULNERABILITY (CVE-2016-0728)
- nginx log打印自定义头
- Tomcat报错“connector attribute SSLCertificateFile must be defined...”
- CentOS配置网易163 yum源
- Citrix桌面虚拟化平台交付推荐版本及相关hotfix
- Tomcat 7下SSL配置出现异常: '/root/.keystore' did not find a matching property
- wps for linux wps-office_10.1.0.5460~a20p1_i386.deb字体库缺失
- Zabbix的snmp监控一些snmp常用的一些OID (KEY)
- 突然出现 -bash: pod: command not found 的解决方法
- RedHat Enterprise Linux 6.4使用Centos 6 的yum源
- nginx 内置变量大全
- lnmp一键安装后不同域名登入不同站点
- Zabbix2.4.X_监控SNMP
- linux基础命令之:vi模式下查找和替换