您的位置:首页 > 运维架构

hadoop map-reduce倒排索引过程

2016-07-29 23:14 281 查看
---------------------------------mapper

//context.wirte("hello->a.txt", "1")
//context.wirte("hello->a.txt", "1")
//context.wirte("hello->a.txt", "1")

<"hello->a.txt", {1,1,1}>
---------------------------------reducer
/context.write("hello", "a.txt->3")
//context.write("hello", "b.txt->2")
//context.write("hello", "c.txt->2")

-----------------------------------------------mapper

//context.write("hello", "a.txt->3")
//context.write("hello", "b.txt->2")
//context.write("hello", "c.txt->2")

<"hello", {"a.txt->3", "b.txt->2", "c.txt->2"}>

-------------------------------- reducer

context.write("hello", "a.txt->3 b.txt->2 c.txt->2")

hello a.txt->3 b.txt->2 c.txt->2
jerry a.txt->1 b.txt->3 c.txt->1
tom a.txt->2 b.txt->1 c.txt->1

hello   hdfs://itcast:9000/ii/b.txt->2  hdfs://itcast:9000/ii/a.txt->3
jerry   hdfs://itcast:9000/ii/a.txt->1  hdfs://itcast:9000/ii/b.txt->3
tom     hdfs://itcast:9000/ii/b.txt->1  hdfs://itcast:9000/ii/a.txt->2
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签:  hadoop 索引