spark--actions算子--takeSample
2017-07-13 09:41
218 查看
import org.apache.spark.{SparkConf, SparkContext} /** * Created by yz02 on 2017/6/16. */ object A_takeSample { System.setProperty("hadoop.home.dir","F:\\hadoop-2.6.5") def main(args: Array[String]): Unit = { val conf = new SparkConf().setAppName("takeSample_test").setMaster("local") val sc = new SparkContext(conf) //准备一下数据 val nameList : List[Int] = List(1,2,3,4,5) val numbers = sc.parallelize(nameList) //随机两个数据 val num = numbers.takeSample(true,2,1) for (x <- num) { println(x) } //随机4个数据,注意随机的数据可能是重复的 val num1 = numbers.takeSample(true,4,1) for (x <- num1) { println(x) } val num2 = numbers.takeSample(false,4,1) for (x <- num2) { println(x) } } }
运行结果:
1
3
4
4
5
2
5
3
1
2
相关文章推荐
- spark--Actions算子--countByKey
- spark--actions算子--reduce
- spark--actions算子--collect
- Spark算子[15]:sample、takeSample 源码实例详解
- [大数据之Spark]——Actions算子操作入门实例
- spark--actions算子--saveAsObjectFile
- Spark算子详解之reduceByKey_sample_take_takeSample_distinct_sortByKey_saveAsTextFile_intersection
- spark--actions算子--count
- spark--actions算子--saveAsTextFile
- spark--actions算子--takeOrdered
- Spark算子:RDD键值转换操作(4)–cogroup、join
- Spark算子:RDD基本转换操作(2)–coalesce、repartition
- (转)Spark 算子系列文章
- Spark算子[07]:reduce,reduceByKey,count,countByKey
- Spark算子[20]:saveAsHadoopDataset、saveAsNewAPIHadoopDataset 实例详解
- Spark算子(一)
- spark--transform算子--coalesce
- spark--transform算子--map
- spark--transform算子--sample
- Spark算子:RDD行动Action操作学习–countByKey、foreach、sortBy