您的位置:首页 > 运维架构

hadoop-运行hadoop jar hadoop-examples-1.2.1.jar wordcount /wc/input/ /wc/output/

2016-01-20 15:07 387 查看
查看结果:



在地址栏:输入   主机名:50070

点击_logs 下面的进入统计单词的结果。

[root@redhat5 hadoop-1.2.1]# hadoop jar hadoop-examples-1.2.1.jar wordcount /wc/input/ /wc/output/

16/01/20 22:27:56 INFO input.FileInputFormat: Total input paths to process : 7

16/01/20 22:27:56 INFO util.NativeCodeLoader: Loaded the native-hadoop library

16/01/20 22:27:56 WARN snappy.LoadSnappy: Snappy native library not loaded

16/01/20 22:27:59 INFO mapred.JobClient: Running job: job_201601202034_0003

16/01/20 22:28:00 INFO mapred.JobClient:  map 0% reduce 0%

16/01/20 22:29:27 INFO mapred.JobClient:  map 28% reduce 0%

16/01/20 22:30:25 INFO mapred.JobClient:  map 42% reduce 0%

16/01/20 22:30:27 INFO mapred.JobClient:  map 42% reduce 9%

16/01/20 22:30:29 INFO mapred.JobClient:  map 57% reduce 9%

16/01/20 22:30:34 INFO mapred.JobClient:  map 57% reduce 14%

16/01/20 22:30:41 INFO mapred.JobClient:  map 57% reduce 19%

16/01/20 22:31:01 INFO mapred.JobClient:  map 71% reduce 19%

16/01/20 22:31:04 INFO mapred.JobClient:  map 85% reduce 19%

16/01/20 22:31:10 INFO mapred.JobClient:  map 85% reduce 28%

16/01/20 22:31:25 INFO mapred.JobClient:  map 100% reduce 28%

16/01/20 22:31:36 INFO mapred.JobClient:  map 100% reduce 92%

16/01/20 22:31:37 INFO mapred.JobClient:  map 100% reduce 100%

16/01/20 22:31:52 INFO mapred.JobClient: Job complete: job_201601202034_0003

16/01/20 22:31:52 INFO mapred.JobClient: Counters: 29

16/01/20 22:31:52 INFO mapred.JobClient:   Job Counters 

16/01/20 22:31:53 INFO mapred.JobClient:     Launched reduce tasks=1

16/01/20 22:31:53 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=372203

16/01/20 22:31:53 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0

16/01/20 22:31:53 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0

16/01/20 22:31:53 INFO mapred.JobClient:     Launched map tasks=7

16/01/20 22:31:53 INFO mapred.JobClient:     Data-local map tasks=7

16/01/20 22:31:53 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=127071

16/01/20 22:31:53 INFO mapred.JobClient:   File Output Format Counters 

16/01/20 22:31:53 INFO mapred.JobClient:     Bytes Written=6549

16/01/20 22:31:53 INFO mapred.JobClient:   FileSystemCounters

16/01/20 22:31:53 INFO mapred.JobClient:     FILE_BYTES_READ=10600

16/01/20 22:31:53 INFO mapred.JobClient:     HDFS_BYTES_READ=16263

16/01/20 22:31:53 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=462821

16/01/20 22:31:53 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=6549

16/01/20 22:31:53 INFO mapred.JobClient:   File Input Format Counters 

16/01/20 22:31:53 INFO mapred.JobClient:     Bytes Read=15486

16/01/20 22:31:53 INFO mapred.JobClient:   Map-Reduce Framework

16/01/20 22:31:53 INFO mapred.JobClient:     Map output materialized bytes=10636

16/01/20 22:31:53 INFO mapred.JobClient:     Map input records=389

16/01/20 22:31:53 INFO mapred.JobClient:     Reduce shuffle bytes=10636

16/01/20 22:31:53 INFO mapred.JobClient:     Spilled Records=1196

16/01/20 22:31:53 INFO mapred.JobClient:     Map output bytes=21294

16/01/20 22:31:53 INFO mapred.JobClient:     Total committed heap usage (bytes)=1323302912

16/01/20 22:31:53 INFO mapred.JobClient:     CPU time spent (ms)=38790

16/01/20 22:31:53 INFO mapred.JobClient:     Combine input records=1761

16/01/20 22:31:53 INFO mapred.JobClient:     SPLIT_RAW_BYTES=777

16/01/20 22:31:53 INFO mapred.JobClient:     Reduce input records=598

16/01/20 22:31:53 INFO mapred.JobClient:     Reduce input groups=427

16/01/20 22:31:53 INFO mapred.JobClient:     Combine output records=598

16/01/20 22:31:53 INFO mapred.JobClient:     Physical memory (bytes) snapshot=1390465024

16/01/20 22:31:53 INFO mapred.JobClient:     Reduce output records=427

16/01/20 22:31:53 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=4970704896

16/01/20 22:31:53 INFO mapred.JobClient:     Map output records=1761

在命令行:查看



在命令下查看结果:

[root@redhat5 conf]# hadoop fs -lsr /wc/input/

-rw-r--r--   1 root supergroup       7457 2016-01-20 06:20 /wc/input/capacity-scheduler.xml

-rw-r--r--   1 root supergroup        378 2016-01-20 06:21 /wc/input/core-site.xml

-rw-r--r--   1 root supergroup        327 2016-01-20 06:21 /wc/input/fair-scheduler.xml

-rw-r--r--   1 root supergroup       4644 2016-01-20 06:21 /wc/input/hadoop-policy.xml

-rw-r--r--   1 root supergroup        358 2016-01-20 06:21 /wc/input/hdfs-site.xml

-rw-r--r--   1 root supergroup       2033 2016-01-20 06:21 /wc/input/mapred-queue-acls.xml

-rw-r--r--   1 root supergroup        289 2016-01-20 06:21 /wc/input/mapred-site.xml

[root@redhat5 conf]# hadoop fs -lsr /wc/output/

-rw-r--r--   1 root supergroup          0 2016-01-20 22:31 /wc/output/_SUCCESS

drwxr-xr-x   - root supergroup          0 2016-01-20 22:27 /wc/output/_logs

drwxr-xr-x   - root supergroup          0 2016-01-20 22:27 /wc/output/_logs/history

-rw-r--r--   1 root supergroup      30337 2016-01-20 22:27 /wc/output/_logs/history/job_201601202034_0003_1453300078993_root_word+count

-rw-r--r--   1 root supergroup      48088 2016-01-20 22:27 /wc/output/_logs/history/job_201601202034_0003_conf.xml
-rw-r--r--   1 root supergroup       6549 2016-01-20 22:31 /wc/output/part-r-00000

[root@redhat5 conf]# hadoop fs -text /wc/output/part-r-00000

 
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: