hadoop-hive安装配置

hadoop-hive安装配置

下载hive-0.8.1.tar.gz ---------在ahdoop1.0.0这个版本上要使用这个版本以上

把这个文件解压在/hadoop/app目录下

1.添加bin/hive-config.sh，添加jdk支持

exportJ***A_HOME=/usr/java/jdk1.7.0_02

exportHIVE_HOME=/hadoop/app/hive-0.8.1

exportHADOOP_HOME=/hadoop/app/hadoop-1.0.0

Java代码

root@master:/data/soft#hadoop fs –mkdir /tmp

root@master:/data/soft#hadoop fs –mkdir /usr/hive/warehouse

root@master:/data/soft#hadoop fs –chmod g+w /tmp

root@master:/data/soft# hadoop fs –chmod g+w /usr/hive/warehouse ------------最好把这个目录创建

通过我的试验，以上创建目录的步骤是可以省略的，Hive会自动创建需要的目录

3.修改conf/hive-default.xml，这个是hive的关键配置，所以一般不要直接修改，新建hive-site.xml文件，将修改的内容在这个里面配置。

[root@hadoop-m conf]# cat /hadoop/app/hive-0.8.1/conf/hive-site.xml

<configuration>

<property>

<name>hive.metastore.local</name>

<value>true</value>

</property>

<property>

<name>hive.exec.scratchdir</name>

<value>/hadoop/app/hive-0.8.1/tmp</value>

<description>Scratch space for Hive jobs</description>

</property>

<property>

<name>hive.querylog.location</name>

<value>/hadoop/app/hive-0.8.1/querylog</value>

</property>

<property>

<name>hive.hwi.listen.host</name>

<value>0.0.0.0</value>

<description>This is the host address the Hive Web Interface will listen on</description>

</property>

<property>

<name>hive.hwi.listen.port</name>

<value>9999</value>

<description>This is the port the Hive Web Interface will listen on</description>

</property>

<property>

<name>javax.jdo.option.ConnectionURL</name>

<value>jdbc:mysql://192.168.10.33:3306/hive?characterEncoding=UTF-8</value>

<description>JDBC connect string for a JDBC metastore</description>

</property>

<property>

<name>hive.metastore.warehouse.dir</name>

<value>hdfs://master:9000/cucrz/hive/warehouse</value>

<value>/cucrz/hive/warehouse</value>

<description>location of default database for the warehouse</description>

</property>

<property>

<name>javax.jdo.option.ConnectionDriverName</name>

<value>com.mysql.jdbc.Driver</value>

</property>

<property>

<name>javax.jdo.option.ConnectionUserName</name>

<value>hive</value> #mysql登陆名

<description>username to use against metastore database</description>

</property>



<property>

<name>javax.jdo.option.ConnectionPassword</name>

<value>111</value> #mysql密码

<description>password to use against metastore database</description>

</property>

</configuration>

进入/home/hadoop/hive-0.8.1目录

用命令：bin/hive开启hive

Hive使用的语法与SQL差不多

检查：

c. 检查

$ hive

hive> use dw2;

OK

Time taken: 3.43 seconds

hive> create table hwz2(id int, name string ) row format delimited FIELDS TERMINATED BY ',';

OK

Time taken: 2.519 seconds

hive> show tables;

OK

hwz2

Time taken: 0.419 seconds

hive> load data local inpath 'demo.txt' overwrite into table hwz2;

Copying data from file:/home/hadoop/demo.txt

Copying file: file:/home/hadoop/demo.txt

Loading data to table dw2.hwz2

Deleted hdfs://master:9000/user/hive/warehouse/dw2.db/hwz2

OK

Time taken: 0.557 seconds

hive> select * from hwz2;

OK

12 jack

12 jack

12 jack

12 jack

12 jack

12 jack

12 jack

12 jack

$ hadoop dfs -lsr /user/hive

Warning: $HADOOP_HOME is deprecated.

drwxr-xr-x - hadoop supergroup 0 2012-03-22 15:36 /user/hive/warehouse

drwxr-xr-x - hadoop supergroup 0 2012-03-22 15:48 /user/hive/warehouse/dw2.db

drwxr-xr-x - hadoop supergroup 0 2012-03-22 15:48 /user/hive/warehouse/dw2.db/hwz2

-rw-r--r-- 2 hadoop supergroup 1201 2012-03-22 15:48 /user/hive/warehouse/dw2.db/hwz2/demo.txt

drwxr-xr-x - hadoop supergroup 0 2012-03-22 12:36 /user/hive/warehouse/hwz

drwxr-xr-x - hadoop supergroup 0 2012-03-22 15:36 /user/hive/warehouse/hwz2

-rw-r--r-- 2 hadoop supergroup 1201 2012-03-22 15:36 /user/hive/warehouse/hwz2/demo.txt

$ hadoop dfs -cat /user/hive/warehouse/dw2.db/hwz2/demo.txt |head

Warning: $HADOOP_HOME is deprecated.

12,jack

12,jack

12,jack

12,jack

12,jack

12,jack

12,jack

12,jack

12,jack

12,jack

d. 在mysql中验证建立的新表

mysql> use hive;

Database changed

mysql> show tables;

+-----------------+

| Tables_in_hive |

+-----------------+

| BUCKETING_COLS |

| CDS |

| COLUMNS_V2 |

| DATABASE_PARAMS |

| DBS |

| PARTITION_KEYS |

| SDS |

| SD_PARAMS |

| SEQUENCE_TABLE |

| SERDES |

| SERDE_PARAMS |

| SORT_COLS |

| TABLE_PARAMS |

| TBLS |

+-----------------+

14 rows in set (0.00 sec)

错误：

[hadoop@master data]$ hive

WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.

Logging initialized using configuration in jar:file:/app/hive/lib/hive-common-0.9.0.jar!/hive-log4j.properties

Hive history file=/tmp/hadoop/hive_job_log_hadoop_201209251016_560764333.txt

当登录hive的时候 老是有个警告：

WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.

根据警告的提示 只需要将 一个参数值从 org.apache.hadoop.metrics.jvm.EventCounter 修改为 org.apache.hadoop.log.metrics.EventCounter 即可。

解决的办法：

-- 将conf下面所有的配置文件都cp 一份

cd $HIVE_HOME/conf

cp hive-default.xml.template hive-default.xml

cp hive-default.xml.template hive-site.xml

cp hive-env.sh.template hive-env.sh

cp hive-log4j.properties.template hive-log4j.properties

--修改参数

vi hive-log4j.properties

找到 参数项 log4j.appender.EventCounter

将这一项的值修改为： org.apache.hadoop.log.metrics.EventCounter

修改之后就好了 再次登录hive 警告就消失了。

[hadoop@master conf]$ hive

Logging initialized using configuration in file:/app/hive/conf/hive-log4j.properties

Hive history file=/tmp/hadoop/hive_job_log_hadoop_201209251022_944441939.txt

hive> exit