Spark生态之Alluxio学习13--Alluxio-1.3.0启动疑问
2016-11-16 19:48
369 查看
更多代码请见：https://github.com/xubo245/SparkLearning
Spark生态之Alluxio学习 版本：alluxio-1.3.0（tachyon），spark-1.5.2,hadoop-2.6.0
启动：
有问题吗？
不确定会不会有影响
但是加sudo启动不了
task.log：
参考
Spark生态之Alluxio学习 版本：alluxio-1.3.0（tachyon），spark-1.5.2,hadoop-2.6.0
1.解释
1.1 alluxio.env.sh 配置
# The directory where a worker stores in-memory data. (Default: /mnt/ramdisk).
# E.g. On linux, /mnt/ramdisk for ramdisk, /dev/shm for tmpFS; on MacOS, /Volumes/ramdisk for ramdisk
# ALLUXIO_RAM_FOLDER
export ALLUXIO_RAM_FOLDER=/home/hadoop/cloud/alluxio-1.3.0/ramdisk
# Address of the under filesystem address. (Default: ${ALLUXIO_HOME}/underFSStorage)
# E.g. "/my/local/path" to use local fs, "hdfs://localhost:9000/alluxio" to use a local hdfs
# ALLUXIO_UNDERFS_ADDRESS
export ALLUXIO_UNDERFS_ADDRESS=hdfs://Master:9000
# How much memory to use per worker. (Default: 1GB)
# E.g. "1000MB", "2GB"
# ALLUXIO_WORKER_MEMORY_SIZE
export ALLUXIO_WORKER_MEMORY_SIZE=6GB启动：
hadoop@Master:~/cloud/alluxio-1.3.0/conf$ ../bin/alluxio-stop.sh all Killed 0 processes on Master Killed 0 processes on Master Waiting for WORKERS tasks to finish... All WORKERS tasks finished, please analyze the log at /home/hadoop/cloud/alluxio-1.3.0/logs/task.log. hadoop@Master:~/cloud/alluxio-1.3.0/conf$ ../bin/alluxio-start.sh all Mount /home/hadoop/cloud/alluxio-1.3.0/bin Killed 0 processes on Master Killed 0 processes on Master Waiting for WORKERS tasks to finish... All WORKERS tasks finished, please analyze the log at /home/hadoop/cloud/alluxio-1.3.0/logs/task.log. Starting master @ Master. Logging to /home/hadoop/cloud/alluxio-1.3.0/logs Waiting for MASTER tasks to finish... All MASTER tasks finished, please analyze the log at /home/hadoop/cloud/alluxio-1.3.0/logs/task.log.
1.2 启动记录
2016-11-11 16:35:48,582 INFO MASTER Connecting to Mcnode1 as hadoop... 2016-11-11 16:35:48,595 INFO MASTER Connecting to Mcnode2 as hadoop... 2016-11-11 16:35:48,604 INFO MASTER Connecting to Mcnode3 as hadoop... 2016-11-11 16:35:48,627 INFO MASTER Connecting to Mcnode4 as hadoop... 2016-11-11 16:35:48,632 INFO MASTER Connecting to Mcnode5 as hadoop... 2016-11-11 16:35:48,649 INFO MASTER Connecting to Mcnode6 as hadoop... 2016-11-11 16:35:48,669 INFO MASTER Connecting to Mcnode7 as hadoop... /home/hadoop/cloud/alluxio-1.3.0/bin /home/hadoop/cloud/alluxio-1.3.0/bin /home/hadoop/cloud/alluxio-1.3.0/bin Formatting RamFS: /home/hadoop/cloud/alluxio-1.3.0/ramdisk (6gb) mount: only root can do that Starting worker @ Mcnode1. Logging to /home/hadoop/cloud/alluxio-1.3.0/logs /home/hadoop/cloud/alluxio-1.3.0/bin /home/hadoop/cloud/alluxio-1.3.0/bin Formatting RamFS: /home/hadoop/cloud/alluxio-1.3.0/ramdisk (6gb) mount: only root can do that Starting worker @ Mcnode3. Logging to /home/hadoop/cloud/alluxio-1.3.0/logs /home/hadoop/cloud/alluxio-1.3.0/bin Formatting RamFS: /home/hadoop/cloud/alluxio-1.3.0/ramdisk (6gb) mount: only root can do that Starting worker @ Mcnode2. Logging to /home/hadoop/cloud/alluxio-1.3.0/logs /home/hadoop/cloud/alluxio-1.3.0/bin Formatting RamFS: /home/hadoop/cloud/alluxio-1.3.0/ramdisk (6gb) mount: only root can do that Formatting RamFS: /home/hadoop/cloud/alluxio-1.3.0/ramdisk (6gb) Starting worker @ Mcnode4. Logging to /home/hadoop/cloud/alluxio-1.3.0/logs mount: only root can do that Starting worker @ Mcnode5. Logging to /home/hadoop/cloud/alluxio-1.3.0/logs Formatting RamFS: /home/hadoop/cloud/alluxio-1.3.0/ramdisk (6gb) Formatting RamFS: /home/hadoop/cloud/alluxio-1.3.0/ramdisk (6gb) mount: only root can do that mount: only root can do that Starting worker @ Mcnode6. Logging to /home/hadoop/cloud/alluxio-1.3.0/logs Starting worker @ Mcnode7. Logging to /home/hadoop/cloud/alluxio-1.3.0/logs Connection to mcnode1 closed. Connection to mcnode3 closed. Connection to mcnode2 closed. Connection to mcnode4 closed. Connection to mcnode5 closed. Connection to mcnode6 closed. Connection to mcnode7 closed.
2.疑问：
显示mount: only root can do that有问题吗？
不确定会不会有影响
但是加sudo启动不了
hadoop@Master:~/cloud/alluxio-1.3.0/conf$ ../bin/alluxio-start.sh all SudoMount /home/hadoop/cloud/alluxio-1.3.0/bin Killed 0 processes on Master Killed 0 processes on Master Waiting for WORKERS tasks to finish... All WORKERS tasks finished, please analyze the log at /home/hadoop/cloud/alluxio-1.3.0/logs/task.log. Starting master @ Master. Logging to /home/hadoop/cloud/alluxio-1.3.0/logs Waiting for MASTER tasks to finish...
task.log：
2016-11-11 16:33:40,423 INFO MASTER Connecting to Mcnode1 as hadoop... 2016-11-11 16:33:40,435 INFO MASTER Connecting to Mcnode2 as hadoop... 2016-11-11 16:33:40,444 INFO MASTER Connecting to Mcnode3 as hadoop... 2016-11-11 16:33:40,463 INFO MASTER Connecting to Mcnode4 as hadoop... 2016-11-11 16:33:40,480 INFO MASTER Connecting to Mcnode5 as hadoop... 2016-11-11 16:33:40,486 INFO MASTER Connecting to Mcnode6 as hadoop... 2016-11-11 16:33:40,526 INFO MASTER Connecting to Mcnode7 as hadoop... /home/hadoop/cloud/alluxio-1.3.0/bin /home/hadoop/cloud/alluxio-1.3.0/bin /home/hadoop/cloud/alluxio-1.3.0/bin /home/hadoop/cloud/alluxio-1.3.0/bin /home/hadoop/cloud/alluxio-1.3.0/bin /home/hadoop/cloud/alluxio-1.3.0/bin [sudo] password for hadoop: [sudo] password for hadoop: /home/hadoop/cloud/alluxio-1.3.0/bin [sudo] password for hadoop: [sudo] password for hadoop: [sudo] password for hadoop: [sudo] password for hadoop: [sudo] password for hadoop: 2016-11-11 16:35:40,728
参考
【1】https://github.com/Alluxio/alluxio 【2】http://www.alluxio.org/ 【3】http://spark.apache.org/docs/1.5.2/programming-guide.html 【4】https://github.com/xubo245/SparkLearning
内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理 
相关文章推荐
- Spark生态之Alluxio学习8---集群版搭建问题之集群无法全部启动
- Spark生态之Alluxio学习10--集群问法全部启动问题解决
- Spark生态之Alluxio学习12--spark调用alluxio-1.3.0配置
- Spark生态之Alluxio学习20--疑问：为什么D7-D9到最后的count时间都是30s左右？
- Spark生态之Alluxio学习21--疑问：为什么master看不到D9,而work节点有D9部分数据？
- Spark生态之Alluxio学习14--alluxio内存文件加载方式和分布情况分析
- Spark生态之Alluxio学习19--alluxio性能提升分析与实验1
- Spark生态之Alluxio学习3---机器重启后数据存储位置的变化
- Spark生态之Alluxio学习7--下载源码通过maven安装成功
- Spark生态之Alluxio学习15--alluxio性能分析和加速方式
- Spark生态之Alluxio学习1---单机版搭建和运行
- Spark生态之Alluxio学习17--多次运行Space Usage增加
- Spark生态之Alluxio学习25--spark从HDFS和Alluxio读取时间比较
- Spark生态之Alluxio学习2---Spark从tachyon中读取文件
- Spark生态之Alluxio学习11--alluxio-1.3.0集群配置
- Spark生态之Alluxio学习18--alluxio work 3000出现问题
- Spark生态之Alluxio学习24--分别读取HDFS和Alluxio的数据进行line count比较分析
- Spark生态之Alluxio学习5--tachyon的几个问题（待解决）
- Spark生态之Alluxio学习4---下载源码通过maven install安装失败记录
- Spark生态之Alluxio学习22--saveAsTextFile alluxio后count （有bug）