Win7下提交MapReduceJob到集群环境

1、下载hadoop-2.5.2.tar.gz，解压到D:\hadoop目录下

2、下载hadoop-common-2.2.0-bin-master.zip，将其中的hadoop.dll和winutils.exe复制到D:\hadoop\hadoop-2.5.2\bin目录下

3、配置环境变量HADOOP_HOME=D:\hadoop\hadoop-2.5.2，在PATH中添加%HADOOP_HOME%\bin

4、复制org.apache.hadoop.mapred.YARNRunner.java，修改如下三个地方
1) 替换Line390：391
    //vargs.add(MRApps.crossPlatformifyMREnv(jobConf, Environment.JAVA_HOME) + "/bin/java");
    vargs.add("$JAVA_HOME/bin/java");
2) 在Line456插入
    replaceEnvironment(environment);
3) 在Line638插入
  private void replaceEnvironment(Map<String, String> environment) {
    String tmpClassPath = (String)environment.get("CLASSPATH");
    tmpClassPath = tmpClassPath.replaceAll(";", ":");
    tmpClassPath = tmpClassPath.replaceAll("%PWD%", "\\$PWD");
    tmpClassPath = tmpClassPath.replaceAll("%HADOOP", "\\$HADOOP");
    tmpClassPath = tmpClassPath.replaceAll("\\\\", "/").replaceAll("%", "");
    environment.put("CLASSPATH", tmpClassPath);
  }
5、用新的YARNRunner.class替换hadoop-mapreduce-client-jobclient-2.5.2.jar中的同名文件

6、在Eclipse中创建一个JavaProject，新增一个用户库hadoop_2.5.2，将hadoop-2.5.2中的jar添加至这个用户库，在Project中引用这个用户库

7、编写wordcount程序，并提交到Hadoop集群，即可得到正确结果，不会报如下错误

Job job_1418033228434_0012 failed with state FAILED due to: Application application_1418033228434_0012 failed 2
times due to AM Container for appattempt_1418033228434_0012_000002 exited with  exitCode: 1 due to: Exception from container-launch: ExitCodeException exitCode=1:
/bin/bash: line 0: fg: no job control
[align=left]
[/align]
ExitCodeException exitCode=1:
/bin/bash: line 0: fg: no job control
[align=left]
[/align]
[align=left]       at org.apache.hadoop.util.Shell.runCommand( Shell.java:538 )[/align]
[align=left]       at org.apache.hadoop.util.Shell.run( Shell.java:455 )[/align]
[align=left]       at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute( Shell.java:702)[/align]
[align=left]       at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer( DefaultContainerExecutor.java:195)[/align]
[align=left]       at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call( ContainerLaunch.java:300)[/align]
[align=left]       at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call( ContainerLaunch.java:81)[/align]
       at java.util.concurrent.FutureTask.run( FutureTask.java:262 )
[align=left]       at java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1145)[/align]
[align=left]       at java.util.concurrent.ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:615)[/align]
[align=left]       at java.lang.Thread.run( Thread.java:744 )[/align]
[align=left]
[/align]
[align=left]
[/align]
[align=left]Container exited with a non-zero exit code 1[/align]
[align=left].Failing this attempt.. Failing the application.[/align]
2014-12-09 13:42:05,383 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1380) )
- Counters: 0

8、如果报下面的错误，请注意可能是你修改了hadoop-mapreduce-client-core-2.5.2.jar中的mapred-default.xml，比如在其中添加了mapreduce.application.classpath，请将修改还原。

Job job_1418033228434_0013 failed with state FAILED due to: Application application_1418033228434_0013
failed 2 times due to AM Container for appattempt_1418033228434_0013_000002 exited with  exitCode: 1 due to: Exception from container-launch: ExitCodeException exitCode=1:
[align=left]ExitCodeException exitCode=1:[/align]
[align=left]       at org.apache.hadoop.util.Shell.runCommand( Shell.java:538 )[/align]
[align=left]       at org.apache.hadoop.util.Shell.run( Shell.java:455 )[/align]
[align=left]       at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute( Shell.java:702)[/align]
[align=left]       at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer( DefaultContainerExecutor.java:195)[/align]
[align=left]       at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call( ContainerLaunch.java:300)[/align]
[align=left]       at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call( ContainerLaunch.java:81)[/align]
[align=left]       at java.util.concurrent.FutureTask.run( FutureTask.java:262 )[/align]
[align=left]       at java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1145)[/align]
[align=left]       at java.util.concurrent.ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:615)[/align]
[align=left]       at java.lang.Thread.run( Thread.java:744 )[/align]
[align=left]
[/align]
[align=left]
[/align]
[align=left]Container exited with a non-zero exit code 1[/align]
[align=left].Failing this attempt.. Failing the application.[/align]
2014-12-09 13:45:02,949 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1380) )
- Counters: 0