hadoop2.6.0执行自带wordcount出现异常

[root@namenode hadoop-2.6.0]# bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar wordcount hdfsInput hdfsOutput

执行上述命令，结果如下

15/08/05 00:55:05 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id

15/08/05 00:55:05 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=

15/08/05 00:55:05 INFO input.FileInputFormat: Total input paths to process : 1

15/08/05 00:55:05 INFO mapreduce.JobSubmitter: number of splits:1

15/08/05 00:55:06 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1009436488_0001

15/08/05 00:55:06 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
15/08/05 00:55:06 INFO mapred.LocalJobRunner: OutputCommitter set in config null

15/08/05 00:55:06 INFO mapreduce.Job: Running job: job_local1009436488_0001

15/08/05 00:55:06 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter

15/08/05 00:55:06 INFO mapred.LocalJobRunner: Waiting for map tasks

15/08/05 00:55:06 INFO mapred.LocalJobRunner: Starting task: attempt_local1009436488_0001_m_000000_0

15/08/05 00:55:06 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]

15/08/05 00:55:06 INFO mapred.MapTask: Processing split: hdfs://namenode:9000/user/root/hdfsInput/test:0+45

15/08/05 00:55:06 INFO mapred.MapTask: (EQUATOR) 0 kvi 2621436(10485744)

15/08/05 00:55:06 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 10

15/08/05 00:55:06 INFO mapred.MapTask: soft limit at 8388608

15/08/05 00:55:06 INFO mapred.MapTask: bufstart = 0; bufvoid = 10485760

15/08/05 00:55:06 INFO mapred.MapTask: kvstart = 2621436; length = 655360

15/08/05 00:55:06 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer

15/08/05 00:55:07 INFO mapred.LocalJobRunner:

15/08/05 00:55:07 INFO mapred.MapTask: Starting flush of map output

15/08/05 00:55:07 INFO mapred.MapTask: Spilling map output

15/08/05 00:55:07 INFO mapred.MapTask: bufstart = 0; bufend = 97; bufvoid = 10485760

15/08/05 00:55:07 INFO mapred.MapTask: kvstart = 2621436(10485744); kvend = 2621388(10485552); length = 49/655360

15/08/05 00:55:07 INFO mapred.MapTask: Finished spill 0

15/08/05 00:55:07 INFO mapred.Task: Task:attempt_local1009436488_0001_m_000000_0 is done. And is in the process of committing

15/08/05 00:55:07 INFO mapred.LocalJobRunner: map

15/08/05 00:55:07 INFO mapred.Task: Task 'attempt_local1009436488_0001_m_000000_0' done.

15/08/05 00:55:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1009436488_0001_m_000000_0

15/08/05 00:55:07 INFO mapred.LocalJobRunner: map task executor complete.

15/08/05 00:55:07 INFO mapred.LocalJobRunner: Waiting for reduce tasks

15/08/05 00:55:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1009436488_0001_r_000000_0

15/08/05 00:55:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]

15/08/05 00:55:07 INFO mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@4d22c8ff

15/08/05 00:55:07 INFO reduce.MergeManagerImpl: MergerManager: memoryLimit=5189795, maxSingleShuffleLimit=1297448, mergeThreshold=51897, ioSortFactor=20, memToMemMergeOutputsThreshold=20

15/08/05 00:55:07 INFO mapred.LocalJobRunner: reduce task executor complete.

15/08/05 00:55:07 WARN mapred.LocalJobRunner: job_local1009436488_0001

java.lang.Exception: java.lang.RuntimeException: Invalid configuration: maxSingleShuffleLimit should be less than mergeThresholdmaxSingleShuffleLimit: 1297448mergeThreshold: 51897

at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)

at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529)

Caused by: java.lang.RuntimeException: Invalid configuration: maxSingleShuffleLimit should be less than mergeThresholdmaxSingleShuffleLimit: 1297448mergeThreshold: 51897

at org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl.<init>(MergeManagerImpl.java:203)

at org.apache.hadoop.mapreduce.task.reduce.Shuffle.createMergeManager(Shuffle.java:85)

at org.apache.hadoop.mapreduce.task.reduce.Shuffle.init(Shuffle.java:80)

at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:374)

at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

at java.util.concurrent.FutureTask.run(FutureTask.java:262)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

15/08/05 00:55:07 INFO mapreduce.Job: Job job_local1009436488_0001 running in uber mode : false

15/08/05 00:55:07 INFO mapreduce.Job: map 100% reduce 0%

15/08/05 00:55:07 INFO mapreduce.Job: Job job_local1009436488_0001 failed with state FAILED due to: NA

15/08/05 00:55:07 INFO mapreduce.Job: Counters: 26

File System Counters

FILE: Number of bytes read=269453

FILE: Number of bytes written=521887

FILE: Number of read operations=0

FILE: Number of large read operations=0

FILE: Number of write operations=0

HDFS: Number of bytes read=45

HDFS: Number of bytes written=0

HDFS: Number of read operations=6

HDFS: Number of large read operations=0

HDFS: Number of write operations=1

Map-Reduce Framework

Map input records=1

Map output records=13

Map output bytes=97

Map output materialized bytes=82

Input split bytes=110

Combine input records=13

Combine output records=8

Spilled Records=8

Failed Shuffles=0

Merged Map outputs=0

GC time elapsed (ms)=2

CPU time spent (ms)=0

Physical memory (bytes) snapshot=0

Virtual memory (bytes) snapshot=0

Total committed heap usage (bytes)=29556736

File Input Format Counters

Bytes Read=45

后经过脱离安全模式，发现运行无误[root@namenode hadoop-2.6.0]# bin/hadoop dfsadmin -safemode leave

[root@namenode hadoop-2.6.0]# bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar wordcount hdfsInput hdfsOutput

15/08/05 01:21:38 INFO client.RMProxy: Connecting to ResourceManager at namenode/192.68.4.185:8032

15/08/05 01:21:39 INFO input.FileInputFormat: Total input paths to process : 1

15/08/05 01:21:39 INFO mapreduce.JobSubmitter: number of splits:1

15/08/05 01:21:40 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1438762610458_0001

15/08/05 01:21:40 INFO impl.YarnClientImpl: Submitted application application_1438762610458_0001

15/08/05 01:21:41 INFO mapreduce.Job: The url to track the job: http://namenode:8088/proxy/application_1438762610458_0001/
15/08/05 01:21:41 INFO mapreduce.Job: Running job: job_1438762610458_0001

15/08/05 01:21:53 INFO mapreduce.Job: Job job_1438762610458_0001 running in uber mode : false

15/08/05 01:21:53 INFO mapreduce.Job: map 0% reduce 0%

15/08/05 01:22:08 INFO mapreduce.Job: map 100% reduce 0%

15/08/05 01:22:23 INFO mapreduce.Job: map 100% reduce 100%

15/08/05 01:22:23 INFO mapreduce.Job: Job job_1438762610458_0001 completed successfully

15/08/05 01:22:24 INFO mapreduce.Job: Counters: 49

File System Counters

FILE: Number of bytes read=82

FILE: Number of bytes written=210925

FILE: Number of read operations=0

FILE: Number of large read operations=0

FILE: Number of write operations=0

HDFS: Number of bytes read=155

HDFS: Number of bytes written=44

HDFS: Number of read operations=6

HDFS: Number of large read operations=0

HDFS: Number of write operations=2

Job Counters

Launched map tasks=1

Launched reduce tasks=1

Data-local map tasks=1

Total time spent by all maps in occupied slots (ms)=12350

Total time spent by all reduces in occupied slots (ms)=13065

Total time spent by all map tasks (ms)=12350

Total time spent by all reduce tasks (ms)=13065

Total vcore-seconds taken by all map tasks=12350

Total vcore-seconds taken by all reduce tasks=13065

Total megabyte-seconds taken by all map tasks=12646400

Total megabyte-seconds taken by all reduce tasks=13378560

Map-Reduce Framework

Map input records=1

Map output records=13

Map output bytes=97

Map output materialized bytes=82

Input split bytes=110

Combine input records=13

Combine output records=8

Reduce input groups=8

Reduce shuffle bytes=82

Reduce input records=8

Reduce output records=8

Spilled Records=16

Shuffled Maps =1

Failed Shuffles=0

Merged Map outputs=1

GC time elapsed (ms)=212

CPU time spent (ms)=3560

Physical memory (bytes) snapshot=309587968

Virtual memory (bytes) snapshot=1687298048

Total committed heap usage (bytes)=164040704

Shuffle Errors

BAD_ID=0

CONNECTION=0

IO_ERROR=0

WRONG_LENGTH=0

WRONG_MAP=0

WRONG_REDUCE=0

File Input Format Counters

Bytes Read=45

File Output Format Counters

Bytes Written=44