Hive remote install mode (mysql) step by step

Prerequisite:

Hadoop cluster is avalable;

Mysql installed on namenode;

Step1: download the latest hive tar packages and extract .

Download:http://apache.dataguru.cn/hive/hive-0.12.0/

tar xvf tar xvf hive-0.12.0.tar.gz

Step2: set the environment for hive.

# su to root add hive home

Vim /etc/profile

export HIVE_HOME=/home/hadoop/hive/hive-0.12.0

export PATH =$PATH:$HIVE_HOME/bin;

Step3: do this in mysql Create the connection user hive with password hive.

mysql –u root –p

CREATE USER 'hive'@'%' IDENTIFIED BY 'hive';

GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%' WITH GRANT OPTION;

flush privileges;

Step4: copy the mysql connector to hive/lib folder.

copy mysql-connector-java-5.1.6-bin.jar to hive/lib/

Step5:update the hive-site.xml

Vim hive-site.xml

The sample content is following:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>

<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://namenode:3306/hive?createDatabaseIfNotExist=true&useUnicode=true&characterEncoding=latin1</value>
</property>

<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>

<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>

<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive</value>
</property>

<property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
</property>
<property>

<name>hive.stats.dbconnectionstring</name>
<value>jdbc:mysql://namenode:3306/hive_stats?useUnicode=true&characterEncoding=latin1&user=hive&password=hive&createDatabaseIfNotExist=true</value>
<description>The default connection string for the database that stores temporary hive statistics.</description>

</property>

<property>
<name>hive.stats.dbconnectionstring</name>
<value>jdbc:mysql://namenode:3306/hive_stats?useUnicode=true&characterEncoding=utf8&user=hive&password=hive&createDatabaseIfNotExist=true</value>
</property>

<property>
<name>hive.stats.dbclass</name>
<value>jdbc:mysql</value>
</property>

<property>
<name>hive.stats.jdbcdriver</name>
<value>com.mysql.jdbc.Driver</value>
</property>

</configuration>

Step6: hive shell, test the install.

Hive;

Show tables;

Then you can go to mysql , check the metastore database hive,

Show tables to check the metastore tables in mysql.

Step7:quick start:

create table student(sid int,sname string) row format delimited fields terminated by '\t' stored as textfile;

load data local inpath '/home/hadoop/data/student.txt' overwrite into table student;

select * from student;

cd /data

vim student.txt
1        yaoxiaohua
2        yaoyuanyie
3        yaotianyie

我一开始在安装的时候,hive启动还总是出一些问题,是hive-site.xml的配置造成的.

可以开启调试模试,根据日志查找原因.

启动调试模式进行操作:

hive -hiveconf hive.root.logger=DEBUG,console

我的mysql安装的比较早,一开始居然忘记root密码了,如果你也遇到了这种情况,

可以参见:

http://blog.sina.com.cn/s/blog_4488002e0100z574.html

主要思路是使用安全登录命令,然后update user表的信息.

mysqld_safe --skip-grant-tables

可能会用到的一些命令:

rpm -qa | grep mysql　　// 这个命令就会查看该操作系统上是否已经安装了mysql数据库

Service mysqld status //检查mysqld的状态

Chkconfig mysqld on //设置开机自动启动mysql服务