Scala简单单机actorwordcount
2017-10-20 15:07
253 查看
import scala.actors.{Actor, Future} import scala.collection.mutable.{HashSet, ListBuffer} import scala.io.Source class Task extends Actor { override def act() : Unit = { loop { react { case SubmitTask(filename) => { val result = Source.fromFile(filename).getLines().flatMap(_.split(" ")).map((_, 1)).toList.groupBy(_._1).mapValues(_.size) sender ! ResultTask(result) } case StopTask => { exit() } } } } } case class SubmitTask(filename : String) case class ResultTask(result : Map[String, Int]) case object StopTask object ActorWordCount { def main(args: Array[String]): Unit = { val replySet = new HashSet[Future[Any]]() val resultList = new ListBuffer[ResultTask]() val files = Array[String]("f://words.txt", "f://words.log") for (f <- files) { val actor = new Task val reply = actor.start() !! SubmitTask(f) replySet += reply } while (replySet.size > 0) { val toCompute = replySet.filter(_.isSet) for (f <- toCompute) { val result = f.apply().asInstanceOf[ResultTask] resultList += result replySet -= f } Thread.sleep(100) } //汇总功能 val fr = resultList.flatMap(_.result).groupBy(_._1).mapValues(_.foldLeft(0)(_+_._2)) println(fr) } }
相关文章推荐
- Scala用actor编写简单WordCount
- scala-单机wordcount
- 使用scala的Actor模拟计算多文件WordCount
- Scala 学习(六)--- 单机实现wordcount详细解读
- 快学Scala-Actor并发编程实现WordCount
- scala-eclipse 编写spark简单程序 WordCount
- 一个简单的WordCount程序
- Spark 使用Python在pyspark中运行简单wordcount
- SPARK-Shell 用Scala执行WordCount
- Spark+scala+Idea wordcount 示例
- Spark wordcount - Python, Scala, Java
- hadoop hdfs搭建 mapreduce环境搭建 wordcount程序简单注释
- Scala:简单使用Actor的消息发送与接收求和
- MapReduce 的简单例子 WordCount的实现
- Spark教程-构建Spark集群-配置Hadoop单机模式并运行Wordcount(1)
- scala版wordcount
- 用hadoop运行一个简单程序WordCount
- idea+maven+scala创建wordcount,打包jar并在spark on yarn上运行
- Scala之WordCount
- Scala 实例wordcount