hadoop排序组合键的使用情况
2015-08-10 12:53
295 查看
于hadoop当处理复杂的业务,需要使用组合键,与单纯的复杂的继承Writable接口,但继承WritableComparable<T>接口。事实上。WritableComparable<T>接口继承Writable和Comparable<T>接口,假设仅仅须要使用某一个类作为传值对象而不是作为key,继承Writable接口就可以。
上源代码:
上源代码:
public interface WritableComparable<T> extends Writable, Comparable<T> { }
public interface Writable { void write(DataOutput out) throws IOException; void readFields(DataInput in) throws IOException; }
public interface Comparable<T> { public int compareTo(T o); }下面是实现复合key的实例,亲測。可用
public class SortKey implements WritableComparable<SortKey>{ private Text name; private IntWritable right; public SortKey() { set(new Text(), new IntWritable()); } public SortKey(Text name, IntWritable right) { set(name, right); } private void set(Text name,IntWritable right){ this.name = name; this.right = right; } /** * @return the name */ public Text getName() { return name; } /** * @param name the name to set */ public void setName(Text name) { this.name = name; } /** * @return the right */ public IntWritable getRight() { return right; } /** * @param right the right to set */ public void setRight(IntWritable right) { this.right = right; } @Override public void write(DataOutput out) throws IOException { name.write(out); right.write(out); } @Override public void readFields(DataInput in) throws IOException { name.readFields(in); right.readFields(in); } @Override public int compareTo(SortKey o) { int cmp = name.compareTo(o.name); if(cmp != 0){ return cmp; }else{ return right.compareTo(o.right); } }
<span style="white-space:pre"> </span>//到眼下为止,你仅仅能将其作为key来使用,可是假设你须要依照key的某一个值来排序,下面是重点
static{ WritableComparator.define(SortKey.class, new Comparator()); } public static class Comparator extends WritableComparator{ private static final Text.Comparator TEXT_COMPARATOR = new Text.Comparator(); protected Comparator() { super(SortKey.class); } /* (non-Javadoc) * @see org.apache.hadoop.io.WritableComparator#compare(byte[], int, int, byte[], int, int) */ @Override public int compare(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2) { try{ int firstL1 = WritableUtils.decodeVIntSize(b1[s1]) + readVInt(b1, s1); int firstL2 = WritableUtils.decodeVIntSize(b2[s2]) + readVInt(b2, s2); return TEXT_COMPARATOR.compare(b1, s1, firstL1, b2, s2, firstL2); }catch(Exception e){ throw new IllegalArgumentException(e); } } } }
相关文章推荐
- 在CentOS上搭建PHP服务器环境
- opengl 顶点数组 学习笔记
- windows开启Apache的mod_rewrite模块
- Docker:搭建开发环境(运行Eclipse等图形化界面程序)
- Hadoop安装教程_分布式集群
- Linux 下 C 语言实现 Copy 复制命令
- Linux远程连接和密钥
- VS2008环境下配置OpenCV4.8.0
- apache配置详解
- DockerFile
- linux定时备份mysql数据库文件
- 嵌入式linux usb wifi移植[s3c6410平台 转载]
- linux下安装google protobuf(详细)
- 前往架构师的路上不得不知道的知识(1)——编程语言的概念
- cmake的两个命令: option 和 configure_file
- 如何前端优化提高页面响应性能-读《高性能网站建设指南》
- hdoj1010Starship Troopers (树dp,依赖背包)
- centos 6.5-hadoop-2.2.0配置hive1.2.1
- OpenCV参考手册之Mat类详解(二)
- jmeter与apache测试网站并发