spark idea 的配置问题
2016-03-31 22:21
302 查看
不知道下面的错误是为什么?
Error:scalac: missing or invalid dependency detected while loading class file 'RDD.class'.
Could not access term hadoop in package org.apache,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'RDD.class' was compiled against an incompatible version of org.apache.
我i下面这个version应该写啥?写1.6.1 不对啊,没有找到
想跑一下下面的代码,死活有问题,哎,我就很讨厌java这套东西,环境配置半天不说,还慢的要死
下面分享一个ibm 大数据基础的培训资料:
http://download.csdn.net/detail/wangyaninglm/9478412
Error:scalac: missing or invalid dependency detected while loading class file 'RDD.class'.
Could not access term hadoop in package org.apache,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'RDD.class' was compiled against an incompatible version of org.apache.
我i下面这个version应该写啥?写1.6.1 不对啊,没有找到
想跑一下下面的代码,死活有问题,哎,我就很讨厌java这套东西,环境配置半天不说,还慢的要死
/** * Created by Administrator on 2016/3/31. */ import org.apache.spark.{SparkConf, SparkContext} /** * Created by Administrator on 2016/3/31. */ import scala.math.random import org.apache.spark._ object simpleApp { def main(args: Array[String]) { val conf = new SparkConf().setAppName("Spark Pi").setMaster("local") val spark = new SparkContext(conf) val slices = if (args.length > 0) args(0).toInt else 2 val n = math.min(100000L * slices, Int.MaxValue).toInt // avoid overflow val count = spark.parallelize(1 until n, slices).map { i => val x = random * 2 - 1 val y = random * 2 - 1 if (x*x + y*y < 1) 1 else 0 }.reduce(_ + _) println("Pi is roughly " + 4.0 * count / n) spark.stop() } }
下面分享一个ibm 大数据基础的培训资料:
http://download.csdn.net/detail/wangyaninglm/9478412
相关文章推荐
- 重学STM32---(五)ADC
- 【bzoj1045】[HAOI2008] 糖果传递
- ASP.NET-ActionFilter过滤器用法实例
- linux下的hugepage
- Servlet形成二维码的输出
- Java并发:线程安全的容器:同步和并发
- 关于 Java 数组的 12 个最佳方法
- 地图定位
- 【bzoj3195】【jxoi2012】【奇怪的道路】【状压dp】
- HDU 1423 Greatest Common Increasing Subsequence(dp)
- 空间三维散点数据的线性拟合
- http协议
- LeetCode -- Set Matrix Zeroes
- Gradle DSL method not found: 'android()
- ios-完成任务状态栏弹出提示view的小框架设计
- 15教育技术冯志超之排序了
- Redis内存使用优化与存储
- 码农小汪-剑指Offer之27 -连续子数组的最大和
- Caffe相关知识记录
- C变量存储