您的位置:首页 > 产品设计 > UI/UE

impala+hue

2016-02-23 00:43 465 查看
下载impala 和 hue
http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_vd_cdh_package_tarball.html https://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/5.5.0/RPMS/x86_64/
impala

impala-catalog

impala-state-store

impala-server

hue

rpm -ivh impala-xxxx.rpm --force --nodeps (不安装所有依赖)

否则会有下面的报错:

<span style="font-family:Consolas;background-color: rgb(255, 255, 255);">error: Failed dependencies:
        bigtop-utils >= 0.7 is needed by impala-2.3.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el7.x86_64
        hadoop is needed by impala-2.3.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el7.x86_64
        hadoop-hdfs is needed by impala-2.3.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el7.x86_64
        hadoop-yarn is needed by impala-2.3.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el7.x86_64
        hadoop-mapreduce is needed by impala-2.3.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el7.x86_64
        hbase is needed by impala-2.3.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el7.x86_64
        hive >= 0.12.0+cdh5.1.0 is needed by impala-2.3.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el7.x86_64
        zookeeper is needed by impala-2.3.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el7.x86_64
        hadoop-libhdfs is needed by impala-2.3.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el7.x86_64
        avro-libs is needed by impala-2.3.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el7.x86_64
        parquet is needed by impala-2.3.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el7.x86_64
        sentry >= 1.3.0+cdh5.1.0 is needed by impala-2.3.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el7.x86_64
        sentry is needed by impala-2.3.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el7.x86_64
        /lib/lsb/init-functions is needed by impala-2.3.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el7.x86_64
        libhdfs.so.0.0.0()(64bit) is needed by impala-2.3.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el7.x86_64
</span>


wget -O
/etc/yum.repos.d/ http://www.apache.org/dist/bigtop/stable/repos/centos7/bigtop.repo

yum install bigtop-utils

yum install lsb

#添加impala_home

vi /etc/profile

export JAVA_HOME=/usr/java/latest

export CLASSPATH=.:%JAVA_HOME%/lib/dt.jar:%JAVA_HOME%/lib/tools.jar:%JAVA_HOME%/lib/jdbc.jar

export PATH=$PATH:$JAVA_HOME/bin

export HADOOP_HOME=/usr/hadoop/latest

export HADOOP_PREFIX=$HADOOP_HOME

export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop

export HIVE_HOME=/usr/hive/latest

export PATH=$HIVE_HOME/bin:$PATH

#impala

export IMPALA_HOME=/usr/lib/impala

export IMPALA_CONF_DIR=/etc/impala/conf

:wq

source /etc/profile

在各节点装好对应的服务后拷贝或者软链接core-site.xml,hdfs-site.xml,hbase-stie.xml到impala的conf目录

ln -s $HADOOP_CONF_DIR/core-site.xml $IMPALA_CONF_DIR/core-site.xml

ln -s $HADOOP_CONF_DIR/hdfs-site.xml $IMPALA_CONF_DIR/hdfs-site.xml

ln -s $HIVE_HOME/conf/hive-site.xml $IMPALA_CONF_DIR/hive-site.xml

下面是 $IMPALA_HOME/lib 中需要解决的死链接。

./hbase-annotations.jar
./hbase-common.jar
./hive-shims.jar
./hbase-client.jar
./hbase-protocol.jar
./hive-beeline.jar
./hadoop-yarn-server-applicationhistoryservice.jar
./hive-ant.jar
./avro.jar
./hadoop-annotations.jar
./hadoop-auth.jar
./hive-common.jar
./hadoop-aws.jar
./hadoop-common.jar
./hadoop-hdfs.jar
./hadoop-mapreduce-client-common.jar
./hive-exec.jar
./hadoop-mapreduce-client-core.jar
./hive-serde.jar
./hadoop-mapreduce-client-jobclient.jar
./hive-hbase-handler.jar
./hive-metastore.jar
./hadoop-mapreduce-client-shuffle.jar
./hive-service.jar
./hadoop-yarn-api.jar
./hive-shims-scheduler.jar
./hadoop-yarn-client.jar
./hadoop-yarn-common.jar
./hadoop-yarn-server-common.jar
./hadoop-yarn-server-nodemanager.jar
./hive-shims-common.jar
./hadoop-yarn-server-resourcemanager.jar
./hadoop-yarn-server-web-proxy.jar
./zookeeper.jar
./sentry-core-common.jar
./sentry-core-model-db.jar
./sentry-core-model-search.jar
./libhdfs.so
./sentry-policy-common.jar
./sentry-policy-db.jar
./sentry-provider-cache.jar
./parquet-hadoop-bundle.jar
./sentry-provider-common.jar
./sentry-provider-db-sh.jar
./sentry-provider-file.jar
./sentry-binding-hive.jar
./libhadoop.so
./libhadoop.so.1.0.0
./libhdfs.so.0.0.0


从官网下载相应的tar包,从中解压出需要的jar库文件放到 $IMPALA_HOME/lib/cdh5_lib/ 下面

http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_vd_cdh_package_tarball.html

然后全部加上链接

ln --force -s $HADOOP_HOME/lib/native/libhadoop.so $IMPALA_HOME/lib/libhadoop.so

ln --force -s $HADOOP_HOME/lib/native/libhdfs.so $IMPALA_HOME/lib/libhdfs.so

ln --force -s $IMPALA_HOME/lib/cdh5_lib/avro-1.7.6-cdh5.5.2.jar $IMPALA_HOME/lib/avro.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hive-exec-1.1.0-cdh5.5.2.jar $IMPALA_HOME/lib/hive-exec.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-annotations-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-annotations.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hive-hbase-handler-1.1.0-cdh5.5.2.jar $IMPALA_HOME/lib/hive-hbase-handler.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-auth-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-auth.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hive-metastore-1.1.0-cdh5.5.2.jar $IMPALA_HOME/lib/hive-metastore.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-aws-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-aws.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hive-serde-1.1.0-cdh5.5.2.jar $IMPALA_HOME/lib/hive-serde.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-common-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-common.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hive-service-1.1.0-cdh5.5.2.jar $IMPALA_HOME/lib/hive-service.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-hdfs-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-hdfs.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hive-shims-0.23-1.1.0-cdh5.5.2.jar $IMPALA_HOME/lib/hive-shims.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-mapreduce-client-common-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-mapreduce-client-common.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hive-shims-1.1.0-cdh5.5.2.jar $IMPALA_HOME/lib/hive-shims.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-mapreduce-client-core-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-mapreduce-client-core.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hive-shims-common-1.1.0-cdh5.5.2.jar $IMPALA_HOME/lib/hive-shims-common.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-mapreduce-client-jobclient.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hive-shims-scheduler-1.1.0-cdh5.5.2.jar $IMPALA_HOME/lib/hive-shims-scheduler.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-mapreduce-client-shuffle-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-mapreduce-client-shuffle.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/parquet-hadoop-bundle-1.5.0-cdh5.5.2.jar $IMPALA_HOME/lib/parquet-hadoop-bundle.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-yarn-api-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-yarn-api.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/sentry-binding-hive-1.5.1-cdh5.5.2.jar $IMPALA_HOME/lib/sentry-binding-hive.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-yarn-client-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-yarn-client.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/sentry-core-common-1.5.1-cdh5.5.2.jar $IMPALA_HOME/lib/sentry-core-common.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-yarn-common-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-yarn-common.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/sentry-core-model-db-1.5.1-cdh5.5.2.jar $IMPALA_HOME/lib/sentry-core-model-db.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-yarn-server-applicationhistoryservice-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-yarn-server-applicationhistoryservice.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/sentry-core-model-search-1.5.1-cdh5.5.2.jar $IMPALA_HOME/lib/sentry-core-model-search.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-yarn-server-common-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-yarn-server-common.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/sentry-policy-common-1.5.1-cdh5.5.2.jar $IMPALA_HOME/lib/sentry-policy-common.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-yarn-server-nodemanager-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-yarn-server-nodemanager.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/sentry-policy-db-1.5.1-cdh5.5.2.jar $IMPALA_HOME/lib/sentry-policy-db.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-yarn-server-resourcemanager-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-yarn-server-resourcemanager.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/sentry-provider-cache-1.5.1-cdh5.5.2.jar $IMPALA_HOME/lib/sentry-provider-cache.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hadoop-yarn-server-web-proxy-2.6.0-cdh5.5.2.jar $IMPALA_HOME/lib/hadoop-yarn-server-web-proxy.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/sentry-provider-common-1.5.1-cdh5.5.2.jar $IMPALA_HOME/lib/sentry-provider-common.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hive-ant-1.1.0-cdh5.5.2.jar $IMPALA_HOME/lib/hive-ant.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/sentry-provider-db-1.5.1-cdh5.5.2.jar $IMPALA_HOME/lib/sentry-provider-db-sh.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hive-beeline-1.1.0-cdh5.5.2.jar $IMPALA_HOME/lib/hive-beeline.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/sentry-provider-file-1.5.1-cdh5.5.2.jar $IMPALA_HOME/lib/sentry-provider-file.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hive-common-1.1.0-cdh5.5.2.jar $IMPALA_HOME/lib/hive-common.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/zookeeper-3.4.5-cdh5.5.2.jar $IMPALA_HOME/lib/zookeeper.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hbase-annotations-1.0.0-cdh5.5.2.jar $IMPALA_HOME/lib/hbase-annotations.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hbase-common-1.0.0-cdh5.5.2.jar $IMPALA_HOME/lib/hbase-common.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hbase-client-1.0.0-cdh5.5.2.jar $IMPALA_HOME/lib/hbase-client.jar

ln --force -s $IMPALA_HOME/lib/cdh5_lib/hbase-protocol-1.0.0-cdh5.5.2.jar $IMPALA_HOME/lib/hbase-protocol.jar

最后需要把jdbc driver 放到 /var/lib/impala 或者使用链接放到这里(如下),否则 impala-catalog服务启动不了。
ln -s $JAVA_HOME/lib/jdbc.jar /var/lib/impala/jdbc.jar

在hdfs-site.xml中增加以下内容:

<!--impala-->

<property>

<name>dfs.client.read.shortcircuit</name>

<value>true</value>

</property>

<property>

<name>dfs.domain.socket.path</name>

<value>/var/run/hadoop-hdfs/dn._PORT</value>

</property>

<property>

<name>dfs.datanode.hdfs-blocks-metadata.enabled</name>

<value>true</value>

</property>

<property>

<name>dfs.client.file-block-storage-locations.timeout.millis</name>

<value>10000</value>

</property>
在/etc/run/下创建hadoop-hdfs文件夹,所属用户hadoop。

mkdir /var/run/hadoop-hdfs
编辑/etc/default/impala配置文件

IMPALA_CATALOG_SERVICE_HOST=namenode

IMPALA_STATE_STORE_HOST=namenode

HADOOP_CONF_DIR=$IMPALA_CONF_DIR

HIVE_CONF_DIR=$IMPALA_CONF_DIR

service impala-state-store start

service impala-catalog start

service impala-server start
OK!



内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: