hue安装与部署

运行环境

centOS 6.6

hadoop 2.4.0

hive 1.2.0

spark 1.4.1

HUE 3.9

介绍:

Hue是一个开源的Apache Hadoop UI系统，最早是由Cloudera Desktop演化而来，由Cloudera贡献给开源社区，它是基于Python Web框架Django实现的。通过使用Hue我们可以在浏览器端的Web控制台上与Hadoop集群进行交互来分析处理数据，例如操作HDFS上的数据，运行MapReduce Job等等。

HUE功能:

数据库查询编辑器,支持 Hive, Impala, MySql, PostGres, Sqlite and Oracle

动态查询仪表盘,支持 Solr

支持 Spark 编辑器和仪表盘

浏览器查看状态,支持 YARN, HDFS, Hive table Metastore, HBase, ZooKeeper

支持 Pig Editor, Sqoop2, Oozie workflows 编辑器和仪表盘

将数据导入hdfs

在centOS编译需要的依赖

ant

asciidoc

cyrus-sasl-devel

cyrus-sasl-gssapi

gcc

gcc-c++

krb5-devel

libtidy (for unit tests only)

libxml2-devel

libxslt-devel

make

mvn (from maven package or maven3 tarball)

mysql

mysql-devel

openldap-devel

python-devel

sqlite-devel

openssl-devel (for version 7+)

gmp-devel

添加maven源

sudo wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo

安装依赖

sudo yum install apache-maven ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi gcc gcc-c++ krb5-devel libxml2-devel libxslt-devel make mysql mysql-devel openldap-devel python-devel sqlite-devel gmp-devel

源码下载

下载，编译源码(hue 3.9),编译时间较长

git clone https://github.com/cloudera/hue.git branch-3.9
cd branch-3.9
make apps

编译完后也可以选择安装

make install

在

desktop/conf/pseudo-distributed.ini

中修改配置

假设所有服务都运行在testHost下

[desktop]
# hue webServer 地址和端口号
http_host=192.168.47.213
http_port=8001

# Webserver runs as this user
server_user=hue
server_group=hue

# This should be the Hue admin and proxy user
default_user=hue

# This should be the hadoop cluster admin
default_hdfs_superuser=admin

[hadoop]
fs_defaultfs=tdhdfs
webhdfs_url=http://192.168.47.216:50070/webhdfs/v1/
hadoop_conf_dir=/usr/install/hadoop/etc/hadoop/conf

[[yarn_clusters]]
# Enter the host on which you are running the ResourceManager
resourcemanager_host=192.168.47.216

# The port where the ResourceManager IPC listens on
resourcemanager_port=23140

# Whether to submit jobs to this cluster
submit_to=True

# URL of the ResourceManager API
resourcemanager_api_url=http://192.168.47.216:23188

# URL of the ProxyServer API
proxy_api_url=http://192.168.47.216:8888

# URL of the HistoryServer API
history_server_api_url=http://192.168.47.216:19888

[beeswax]
# If Kerberos security is enabled, use fully-qualified domain name (FQDN).
hive_server_host=192.168.47.213

# Port where HiveServer2 Thrift server runs on.
hive_server_port=10000

# Hive configuration directory, where hive-site.xml is located
hive_conf_dir=/usr/install/apache-hive-1.2.0-bin/conf

[zookeeper]
host_ports=zk1:2181,zk2:2181,zk3:2181

启动

build/env/bin/supervisor

运行界面

一些问题

使用hive 0.13.0时会报

can't fetch result之类的错误

是版本原因，升级hive版本为1.2.0问题消失

连接hive error

Could not install table: Error creating table sample_07: Could not start SASL: Error in sasl_client_start (-4) SASL(-4): no mechanism available: No worthy mechs found (code THRIFTTRANSPORT): TTransportException('Could not start SASL: Error in sasl_client_start (-4) SASL(-4): no mechanism available: No worthy mechs found',).

解决方案：

yum -y install cyrus-sasl-plain cyrus-sasl-devel cyrus-sasl-gssapi