您的位置:首页 > 运维架构

hadoop排序组合键的使用情况

2015-08-10 12:53 295 查看
于hadoop当处理复杂的业务,需要使用组合键,与单纯的复杂的继承Writable接口,但继承WritableComparable<T>接口。事实上。WritableComparable<T>接口继承Writable和Comparable<T>接口,假设仅仅须要使用某一个类作为传值对象而不是作为key,继承Writable接口就可以。

上源代码:
public interface WritableComparable<T> extends Writable, Comparable<T> {
}
public interface Writable {

void write(DataOutput out) throws IOException;

void readFields(DataInput in) throws IOException;
}
public interface Comparable<T> {

public int compareTo(T o);
}
下面是实现复合key的实例,亲測。可用

public class SortKey implements WritableComparable<SortKey>{

private Text name;
private IntWritable right;

public SortKey() {
set(new Text(), new IntWritable());
}

public SortKey(Text name, IntWritable right) {
set(name, right);
}

private void set(Text name,IntWritable right){
this.name = name;
this.right = right;
}

/**
* @return the name
*/
public Text getName() {
return name;
}

/**
* @param name the name to set
*/
public void setName(Text name) {
this.name = name;
}

/**
* @return the right
*/
public IntWritable getRight() {
return right;
}

/**
* @param right the right to set
*/
public void setRight(IntWritable right) {
this.right = right;
}

@Override
public void write(DataOutput out) throws IOException {
name.write(out);
right.write(out);
}

@Override
public void readFields(DataInput in) throws IOException {
name.readFields(in);
right.readFields(in);
}

@Override
public int compareTo(SortKey o) {
int cmp = name.compareTo(o.name);
if(cmp != 0){
return cmp;
}else{
return right.compareTo(o.right);
}
}
<span style="white-space:pre">	</span>//到眼下为止,你仅仅能将其作为key来使用,可是假设你须要依照key的某一个值来排序,下面是重点
static{
WritableComparator.define(SortKey.class, new Comparator());
}

public static class Comparator extends WritableComparator{

private static final Text.Comparator TEXT_COMPARATOR = new Text.Comparator();

protected Comparator() {
super(SortKey.class);
}

/* (non-Javadoc)
* @see org.apache.hadoop.io.WritableComparator#compare(byte[], int, int, byte[], int, int)
*/
@Override
public int compare(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2) {
try{
int firstL1 = WritableUtils.decodeVIntSize(b1[s1]) + readVInt(b1, s1);
int firstL2 = WritableUtils.decodeVIntSize(b2[s2]) + readVInt(b2, s2);
return TEXT_COMPARATOR.compare(b1, s1, firstL1, b2, s2, firstL2);
}catch(Exception e){
throw new IllegalArgumentException(e);
}
}
}

}
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: