hadoop jar singlnode下运行的命令行输出
2014-05-23 03:02
357 查看
[hadoop@aaa ~]$ hadoop jar /opt/workspace/PGMdriver.jar GibbsSamplingDriver /output 14/05/22 20:39:13 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 14/05/22 20:39:13 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 14/05/22 20:39:14 INFO mapreduce.JobSubmitter: number of splits:2 14/05/22 20:39:14 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name 14/05/22 20:39:14 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 14/05/22 20:39:14 INFO Configuration.deprecation: mapred.cache.files.filesizes is deprecated. Instead, use mapreduce.job.cache.files.filesizes 14/05/22 20:39:14 INFO Configuration.deprecation: mapred.cache.files is deprecated. Instead, use mapreduce.job.cache.files 14/05/22 20:39:14 INFO Configuration.deprecation: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class 14/05/22 20:39:14 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class 14/05/22 20:39:14 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name 14/05/22 20:39:14 INFO Configuration.deprecation: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class 14/05/22 20:39:14 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 14/05/22 20:39:14 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 14/05/22 20:39:14 INFO Configuration.deprecation: mapred.cache.files.timestamps is deprecated. Instead, use mapreduce.job.cache.files.timestamps 14/05/22 20:39:14 INFO Configuration.deprecation: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class 14/05/22 20:39:14 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir 14/05/22 20:39:14 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1400777095877_0003 14/05/22 20:39:15 INFO impl.YarnClientImpl: Submitted application application_1400777095877_0003 to ResourceManager at /0.0.0.0:8032 14/05/22 20:39:15 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1400777095877_0003/ 14/05/22 20:39:15 INFO mapreduce.Job: Running job: job_1400777095877_0003 14/05/22 20:39:22 INFO mapreduce.Job: Job job_1400777095877_0003 running in uber mode : false 14/05/22 20:39:22 INFO mapreduce.Job: map 0% reduce 0% 14/05/22 20:39:30 INFO mapreduce.Job: map 100% reduce 0% 14/05/22 20:39:37 INFO mapreduce.Job: map 100% reduce 100% 14/05/22 20:39:38 INFO mapreduce.Job: Job job_1400777095877_0003 completed successfully 14/05/22 20:39:38 INFO mapreduce.Job: Counters: 43 File System Counters FILE: Number of bytes read=34651 FILE: Number of bytes written=308564 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=40 HDFS: Number of bytes written=33845 HDFS: Number of read operations=7 HDFS: Number of large read operations=0 HDFS: Number of write operations=2 Job Counters Launched map tasks=2 Launched reduce tasks=1 Other local map tasks=2 Total time spent by all maps in occupied slots (ms)=10975 Total time spent by all reduces in occupied slots (ms)=4275 Map-Reduce Framework Map input records=0 Map output records=200 Map output bytes=34045 Map output materialized bytes=34657 Input split bytes=40 Combine input records=0 Combine output records=0 Reduce input groups=176 Reduce shuffle bytes=34657 Reduce input records=200 Reduce output records=200 Spilled Records=400 Shuffled Maps =2 Failed Shuffles=0 Merged Map outputs=2 GC time elapsed (ms)=201 CPU time spent (ms)=2410 Physical memory (bytes) snapshot=662515712 Virtual memory (bytes) snapshot=2658840576 Total committed heap usage (bytes)=512229376 Shuffle Errors BAD_ID=0 CONNECTION=0 IO_ERROR=0 WRONG_LENGTH=0 WRONG_MAP=0 WRONG_REDUCE=0 File Input Format Counters Bytes Read=0 File Output Format Counters Bytes Written=33845
以上位运行成功
下面换了个jar，运行没错，但是结果不对，貌似没读到hdfs里的文件，命令行显示结果如下：
[hadoop@aaa~]$ hadoop jar /opt/workspace/PGM2205.jar /output 14/05/22 21:23:10 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 14/05/22 21:23:11 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 14/05/22 21:23:11 INFO mapreduce.JobSubmitter: number of splits:3 14/05/22 21:23:11 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name 14/05/22 21:23:11 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 14/05/22 21:23:11 INFO Configuration.deprecation: mapred.cache.files.filesizes is deprecated. Instead, use mapreduce.job.cache.files.filesizes 14/05/22 21:23:11 INFO Configuration.deprecation: mapred.cache.files is deprecated. Instead, use mapreduce.job.cache.files 14/05/22 21:23:11 INFO Configuration.deprecation: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class 14/05/22 21:23:11 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class 14/05/22 21:23:11 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name 14/05/22 21:23:11 INFO Configuration.deprecation: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class 14/05/22 21:23:11 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 14/05/22 21:23:11 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 14/05/22 21:23:11 INFO Configuration.deprecation: mapred.cache.files.timestamps is deprecated. Instead, use mapreduce.job.cache.files.timestamps 14/05/22 21:23:11 INFO Configuration.deprecation: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class 14/05/22 21:23:11 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir 14/05/22 21:23:12 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1400777095877_0009 14/05/22 21:23:12 INFO impl.YarnClientImpl: Submitted application application_1400777095877_0009 to ResourceManager at /0.0.0.0:8032 14/05/22 21:23:12 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1400777095877_0009/ 14/05/22 21:23:12 INFO mapreduce.Job: Running job: job_1400777095877_0009 14/05/22 21:23:18 INFO mapreduce.Job: Job job_1400777095877_0009 running in uber mode : false 14/05/22 21:23:18 INFO mapreduce.Job: map 0% reduce 0% 14/05/22 21:23:27 INFO mapreduce.Job: map 33% reduce 0% 14/05/22 21:23:29 INFO mapreduce.Job: map 100% reduce 0% 14/05/22 21:23:34 INFO mapreduce.Job: map 100% reduce 100% 14/05/22 21:23:34 INFO mapreduce.Job: Job job_1400777095877_0009 completed successfully 14/05/22 21:23:34 INFO mapreduce.Job: Counters: 43 File System Counters FILE: Number of bytes read=6 FILE: Number of bytes written=318973 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=60 HDFS: Number of bytes written=0 HDFS: Number of read operations=9 HDFS: Number of large read operations=0 HDFS: Number of write operations=2 Job Counters Launched map tasks=3 Launched reduce tasks=1 Other local map tasks=3 Total time spent by all maps in occupied slots (ms)=21498 Total time spent by all reduces in occupied slots (ms)=3932 Map-Reduce Framework Map input records=0 Map output records=0 Map output bytes=0 Map output materialized bytes=18 Input split bytes=60 Combine input records=0 Combine output records=0 Reduce input groups=0 Reduce shuffle bytes=18 Reduce input records=0 Reduce output records=0 Spilled Records=0 Shuffled Maps =3 Failed Shuffles=0 Merged Map outputs=3 GC time elapsed (ms)=143 CPU time spent (ms)=2560 Physical memory (bytes) snapshot=946880512 Virtual memory (bytes) snapshot=3543068672 Total committed heap usage (bytes)=717225984 Shuffle Errors BAD_ID=0 CONNECTION=0 IO_ERROR=0 WRONG_LENGTH=0 WRONG_MAP=0 WRONG_REDUCE=0 File Input Format Counters Bytes Read=0 File Output Format Counters Bytes Written=0为什么呢？
内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理 
相关文章推荐
- 命令行实现hadoop上传JAR文件，运行《hadoop权威指南第二版》测试最大气温的应用
- 将hadoop程序打成jar包,在linux下以命令行方式运行(例如单词计算程序)
- Hadoop学习1_在使用命令行运行WordCount时，遇到的jar命令说明
- hadoop 编译jar包和运行
- java带jar包的命令行运行
- 运行 bin/hadoop jar hadoop-*-examples.jar wordcount报错
- hadoop学习笔记（5）-运行任务(Job)小结：第三方jar包、hadoop jar命令
- 通过命令行运行jar文件
- hadoop 命令行运行任务
- hadoop 命令行运行
- java命令行运行jar里的main类
- 伪分布式环境下命令行正确运行hadoop示例wordcount
- 关于java运行命令行程序输出的问题
- java 命令行运行时加载外部jar包的方式
- 如何在命令行中运行 *.jar
- 用命令行运行hadoop程序WordCount，编译hadoop程序报错
- 在非web环境下，直接用java命令行运行程序时，对于通过getResources找不到jar包中的资源和目录的解决方法
- 命令行运行jar
- 加载依赖的jar包在命令行编译和运行java文件
- 伪分布式环境下命令行正确运行hadoop示例wordcount