hadoop上hive安装配置及报错处理

参考网址：http://nunknown.com/study/282/#3

1. 在master上安装hive和mysql，mysql作为元数据存储位置：

hive下载： wget http://mirror.bit.edu.cn/apache/hive/hive-2.1.1/apache-hive-2.1.1-bin.tar.gz

解压： tar -zxf apache-hive-2.1.1-bin.tar.gz

配置环境变量：

HIVE_HOME=/home/hadoop/apache-hive-1.2.1-bin

PATH=$PATH:$HIVE_HOME/bin

2. Metastore：

Ubuntu上安装MySQL：

apt-getinstall mysql-server
apt-getisntall mysql-client

执行：apt-get isntall mysql-client

E: Invalid operation isntall

原因：

apt-getinstall mysql-server命令会安装以下包：

apparmor

mysql-client

mysql-common

mysql-server

mysql-server

mysql-server-core

验证：netstat -tap | grep mysql

进入mysql： mysql -u root -p 密码：111111

创建Hive用户：

CREATEUSER 'hive' IDENTIFIED BY '111111';
GRANTALL PRIVILEGES ON *.* TO 'hive'@'master' WITH GRANT OPTION;//分配权限
flushprivileges;//跟新配置

报错：

connection refuse （[mysqld]）

/etc/mysql/my.cnf中设置了

bind_address=127.0.0.1

说明：这种情况可以TCP/IP连接，不能使用hostname连接

注释掉就可以了

建立 Hive 专用的元数据库：

create database hive;

3. hive启动：

启动前：

schematool -initSchema -dbType mysql createDatabaseIfNotExist=true

这里使用mysql数据库而不是默认数据库

报错处理：

[Fatal Error] hive-site.xml:1:2: The markup in the documentpreceding the root element must be well-formed.

<1?xmlversion=”1.0” encoding=”UTF-8”standalone=”no”?>

多添加了一个“1”

Caused by: MetaException(message:Version information notfound in metastore. )

修改conf/hive-site.xml 中的“hive.metastore.schema.verification” 值为 false

Exception in thread "main"java.lang.RuntimeException: java.net.ConnectException: Call Frommaster/172.16.*.* to master:9000 failed on connection exception:java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused

解决：namenode未启动，重新启动即可

Exception in thread "main"java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative pathin absolute URI: ${system:java.io.tmpdir%7D/$%7Bhive.session.id%7D_resources
Caused by: java.net.URISyntaxException: Relative pathin absolute URI: ${system:java.io.tmpdir%7D/$%7Bhive.session.id%7D_resources

解决：路径设置问题

<property>
<name>hive.exec.local.scratchdir</name>
<value>/usr/local/apache-hive-2.1.1-bin/iotmp</value>
<!--value>${system:java.io.tmpdir}/${system:user.name}</value-->
<description>Localscratch space for Hive jobs</description>
</property>
<property>
<name>hive.downloaded.resources.dir</name>
<value>/tmp/hive/resources</value>
<!--value>${system:java.io.tmpdir}/${hive.session.id}_resources</value-->
<description>Temporarylocal directory for added resources in the remote filesystem.</description>
</property>

4000
移动到其它节点：

scp -r /usr/local/apache-hive-2.1.1-bin node1:/usr/local

报错：

Caused by: java.sql.SQLException: null,  message from server: "Host'116.62.*.*' is not allowed to connect to this MySQL server"

mysql没有授权该IP：可以在创建数据库是授权不限制IP登陆

grantall PRIVILEGES on *.* to hive@'%' identified by '111111';
flushprivileges;

ACID and Transactions in Hive参见网址：

https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions

https://cwiki.apache.org/confluence/display/Hive/Streaming+Data+Ingest#StreamingDataIngest-TransactionBatch