编译Spark出现MQTT失败的解决

编译spark的时候一直在MQTT哪里出错，看一下具体所报的错。

[INFO]------------------------------------------------------------------------

[INFO] Building Spark Project External MQTT1.5.2

[INFO]------------------------------------------------------------------------

[WARNING] The POM fororg.eclipse.paho:org.eclipse.paho.client.mqttv3:jar:1.0.1 is missing, nodependency information available

[INFO]------------------------------------------------------------------------

[INFO] Reactor Summary:

[INFO]

[INFO] Spark Project Parent POM........................... SUCCESS [ 4.876 s]

[INFO] Spark Project Launcher............................. SUCCESS [ 10.635 s]

[INFO] Spark Project Networking........................... SUCCESS [ 8.669 s]

[INFO] Spark Project Shuffle StreamingService ............ SUCCESS [  3.816 s]

[INFO] Spark Project Unsafe............................... SUCCESS [ 15.871 s]

[INFO] Spark Project Core................................. SUCCESS [04:14 min]

[INFO] Spark Project Bagel................................ SUCCESS [ 22.915 s]

[INFO] Spark Project GraphX............................... SUCCESS [01:09 min]

[INFO] Spark Project Streaming............................ SUCCESS [01:59 min]

[INFO] Spark Project Catalyst............................. SUCCESS [02:27 min]

[INFO] Spark Project SQL.................................. SUCCESS [02:59 min]

[INFO] Spark Project ML Library........................... SUCCESS [03:14 min]

[INFO] Spark Project Tools................................ SUCCESS [ 14.214 s]

[INFO] Spark Project Hive................................. SUCCESS [02:10 min]

[INFO] Spark Project REPL................................. SUCCESS [ 48.593 s]

[INFO] Spark Project YARN................................. SUCCESS [ 55.566 s]

[INFO] Spark Project Hive Thrift Server................... SUCCESS [ 43.281 s]

[INFO] Spark Project Assembly............................. SUCCESS [02:41 min]

[INFO] Spark Project External Twitter..................... SUCCESS [ 17.123 s]

[INFO] Spark Project External Flume Sink.................. SUCCESS [ 24.566 s]

[INFO] Spark Project External Flume....................... SUCCESS [ 32.463 s]

[INFO] Spark Project External FlumeAssembly .............. SUCCESS [  3.693s]

[INFO] Spark Project External MQTT........................ FAILURE [  0.095s]

[INFO] Spark Project External MQTT Assembly............... SKIPPED

[INFO] Spark Project External ZeroMQ...................... SKIPPED

[INFO] Spark Project External Kafka....................... SKIPPED

[INFO] Spark Project Examples............................. SKIPPED

[INFO] Spark Project External KafkaAssembly .............. SKIPPED

[INFO] Spark Project YARN Shuffle Service................. SKIPPED

[INFO]------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO]------------------------------------------------------------------------

[INFO] Total time: 26:05 min

[INFO] Finished at:2015-11-21T01:15:18+08:00

[INFO] Final Memory: 70M/885M

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal on projectspark-streaming-mqtt_2.10: Could not resolve dependencies for projectorg.apache.spark:spark-streaming-mqtt_2.10:jar:1.5.2: Failure to findorg.eclipse.paho:org.eclipse.paho.client.mqttv3:jar:1.0.1 inhttp://repo1.maven.org/maven2
was cached in the local repository, resolutionwill not be reattempted until the update interval of offical has elapsed orupdates are forced -> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of theerrors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch toenable full debug logging.

[ERROR]

[ERROR] For more information about theerrors and possible solutions, please read the following articles:

[ERROR] [Help 1]http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

[ERROR]

[ERROR] After correcting the problems, youcan resume the build with the command

[ERROR]  mvn <goals> -rf :spark-streaming-mqtt_2.10

 

主要是缺少这个org.eclipse.paho:org.eclipse.paho.client.mqttv3:jar:1.0.1包，但是为什么就是下不来呢？

百度了一下，根据这篇文章http://www.iteblog.com/archives/999，我们添加这个InstallCert.java类的编译进去。但是发现没有这个目录$JAVA_HOME/lib/security/只有jre里面才有security这个目录，然后就把jssecacerts放在$JAVA_HOME/jre/lib/security/和$JAVA_HOME/lib下面。问题还是解决不了。

看了一下.m2里面，jar还是没有下载下来，既然这样，那我就直接自己下了。



根据pom文件里的配置

<repository>

     <id>mqtt-repo</id>

      <name>MQTT Repository</name>

     <url>https://repo.eclipse.org/content/repositories/paho-releases</url>

      <releases>

       <enabled>true</enabled>

      </releases>

      <snapshots>

       <enabled>false</enabled>

      </snapshots>

    </repository>

我们就自己去这个网站下载https://repo.eclipse.org/content/repositories/paho-releases



根据所需要的包进行下载，主要是org.eclipse.paho.client.mqttv3-1.0.1和java-parent，然后添加到.m2里面就可以解决问题了。