hadoop mapreduce 计算平均气温的代码,绝对原创
2015-05-23 16:25
134 查看
[code]1901 46 1902 21 1903 48 1904 33 1905 43 1906 47 1907 31 1908 28 1909 26 1910 35 1911 30 1912 16 1913 29 1914 29 1915 5 1916 21 1917 22 1918 31 1919 27 1920 43 1921 34 1922 27 1923 26 以上为结果 package com.teset; import java.io.IOException; import java.util.StringTokenizer; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.conf.Configured; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.FloatWritable; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce.Reducer; import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; import org.apache.hadoop.util.Tool; import org.apache.hadoop.util.ToolRunner; public class AvgTemprature extends Configured implements Tool { public static class AvgMapper extends Mapper<LongWritable, Text, Text, IntWritable> { @Override protected void map(LongWritable key, Text value, Mapper<LongWritable, Text, Text, IntWritable>.Context context) throws IOException, InterruptedException { // map处理数据 String str = value.toString(); String year = null; int temprature = 0; StringTokenizer tokenstr = new StringTokenizer(str); int i = 0; while (tokenstr.hasMoreTokens()) { String tempstr = tokenstr.nextToken(); i++; if (i == 1) { year = tempstr; continue; } else if (i == 5 && Integer.parseInt(tempstr) != -9999) { temprature = Integer.parseInt(tempstr); context.write(new Text(year), new IntWritable(temprature)); break; } } } } public static class AvgReducer extends Reducer<Text, IntWritable, Text, IntWritable> { @Override protected void reduce(Text key, Iterable<IntWritable> values, Reducer<Text, IntWritable, Text, IntWritable>.Context context) throws IOException, InterruptedException { int sum=0; int count=0; for (IntWritable value : values) { sum=sum+value.get(); count++; } if (count!=0){ context.write(key, new IntWritable(sum/count)); } } } public static void main(String[] args) throws Exception { int res = ToolRunner.run(new Configuration(), new AvgTemprature(), args); System.exit(res); } @Override public int run(String[] arg0) throws Exception { Configuration conf = getConf(); Job job = new Job(conf, "AvgTem");// 任务名 job.setJarByClass(AvgTemprature.class);// 指定class // 输入和输出流 job.setMapperClass(AvgMapper.class);// map job.setReducerClass(AvgReducer.class); FileInputFormat.addInputPath(job, new Path(arg0[0])); FileOutputFormat.setOutputPath(job, new Path(arg0[1])); job.setCombinerClass(AvgReducer.class); job.setOutputValueClass(IntWritable.class); job.setOutputKeyClass(Text.class); job.waitForCompletion(true); return job.isSuccessful() ? 0 : 1; } }
相关文章推荐
- hadoop mapreduce 计算平均气温的代码,绝对原创
- 简单的java Hadoop MapReduce程序(计算平均成绩)从打包到提交及运行
- 简单的java Hadoop MapReduce程序(计算平均成绩)从打包到提交及运行
- 关于hadoop集群的简单性能测试——mapreduce性能,hive性能,并行计算分析(原创)
- [原创]Python小工具 —— 计算 文件 或 文件夹及其所有子文件夹中所有文件 的 代码行数
- (第4篇)hadoop之魂--mapreduce计算框架,让收集的数据产生价值
- hadoop的MapReduce在java驱动代码
- Hadoop系列之MapReduce(分布式计算测试)
- 用MFC实现代码获取当前EXE 执行的绝对路径,用于扩展相对路径扩展,以及计算文件大小功能
- Hadoop(三)——利用MapReduce求平均成绩
- hadoop_7 : MapReduce代码
- Hadoop MapReduce示例代码
- Hadoop MapReduce示例代码
- hadoop2.7.3用mapreduce计算pi值
- 大数据时代之hadoop(五):hadoop 分布式计算框架(MapReduce)
- hadoop 计算平均成绩
- 大数据-Hadoop-MapReduce (二) WrodCount单词计算
- Hadoop2.2.0 第一步完成MapReduce wordcount计算文本数量
- hadoop之魂--mapreduce计算框架,让收集的数据产生价值
- hadoop MapReduce实例解析,hadoop新旧两种代码风格1.0.x和1.2.x