TPC-DS性能测试及使用方法

一、安装TPC-DS工具

1、下载工具

 下载地址：http://www.tpc.org/tpc_documents_current_versions/download_programs/tools-download-request.asp?bm_type=TPC-DS&bm_vers=2.6.0&mode=CURRENT-ONLY

注意：必须输入邮箱，他会发下载地址到邮箱中，点击下载地址即可下载。

2、解压编译工具

   1）解压

    执行命令：unzip 944eb36c-5624-45ea-bece-646814a75b63-tpc-ds-tool.zip

   2）进入tools目录编译

    执行命令：make

  以上TPC-DS测试环境就在准备好了

二、创建TPC-DS测试需要用到的表

1、准备好创建表语句

TPC-DS已经提前准备好了创建表相关的SQL文件

文件位于:tools目录下，具体文件如下：

tpcds.sql 创建25张表的sql语句

tpcds_ri.sql 创建表与表之间关系的sql语句

tpcds_source.sql

2、利用数据库连接工具(如：navicat preminer)创建相应的数据库和表

    将准备好的建表语句复制到工具中创建表

三、生成数据并将数据导入到相应的表中

1、生成数据

到tools目录下执行如下命令：

-dir 生成数据存放目录

-scale 生成数据大小

1）命令1:./dsdgen -DIR /part2/tpcds/v2.6.0/datas/ -SCALE 1 （scale表示产生1G测试数据）

2）命令2：./dsdgen -DIR /part2/tpcds/v2.6.0/datas/ -SCALE 10 -parallel 4 -child 1 （并行产生1g数据）

2、对生成的数据进行处理（不处理无法导入到表中）—每一行多了一个“|”

在/part2/tpcds/v2.6.0/datas/目录下创建目录handled

命令：mkdir handled

在/part2/tpcds/v2.6.0/datas/目录下执行如下命令：

for i in `ls *.dat`
do
name="handled/$i"
echo $name
`touch $name`
`chmod 777 $name`
sed 's/|$//' $i >> $name;
done

3、将数据加载到表中

copy call_center from '/part2/tpcds/v2.6.0/datas/handled/call_center.dat' with delimiter as '|' NULL '';
copy catalog_page from '/part2/tpcds/v2.6.0/datas/handled/catalog_page.dat' with delimiter as '|' NULL '';
copy catalog_returns from '/part2/tpcds/v2.6.0/datas/handled/catalog_returns.dat' with delimiter as '|' NULL '';
copy catalog_sales from '/part2/tpcds/v2.6.0/datas/handled/catalog_sales.dat' with delimiter as '|' NULL '';
copy customer from '/part2/tpcds/v2.6.0/datas/handled/customer.dat' with delimiter as '|' NULL '';
copy customer_address from '/part2/tpcds/v2.6.0/datas/handled/customer_address.dat' with delimiter as '|' NULL '';
copy customer_demographics from '/part2/tpcds/v2.6.0/datas/handled/customer_demographics.dat' with delimiter as '|' NULL '';
copy date_dim from '/part2/tpcds/v2.6.0/datas/handled/date_dim.dat' with delimiter as '|' NULL '';
copy dbgen_version from '/part2/tpcds/v2.6.0/datas/handled/dbgen_version.dat' with delimiter as '|' NULL '';
copy household_demographics from '/part2/tpcds/v2.6.0/datas/handled/household_demographics.dat' with delimiter as '|' NULL '';
copy income_band from '/part2/tpcds/v2.6.0/datas/handled/income_band.dat' with delimiter as '|' NULL '';
copy inventory from '/part2/tpcds/v2.6.0/datas/handled/inventory.dat' with delimiter as '|' NULL '';
copy item from '/part2/tpcds/v2.6.0/datas/handled/item.dat' with delimiter as '|' NULL '';
copy promotion from '/part2/tpcds/v2.6.0/datas/handled/promotion.dat' with delimiter as '|' NULL '';
copy reason from '/part2/tpcds/v2.6.0/datas/handled/reason.dat' with delimiter as '|' NULL '';
copy ship_mode from '/part2/tpcds/v2.6.0/datas/handled/ship_mode.dat' with delimiter as '|' NULL '';
copy store from '/part2/tpcds/v2.6.0/datas/handled/store.dat' with delimiter as '|' NULL '';
copy store_returns from '/part2/tpcds/v2.6.0/datas/handled/store_returns.dat' with delimiter as '|' NULL '';
copy store_sales from '/part2/tpcds/v2.6.0/datas/handled/store_sales.dat' with delimiter as '|' NULL '';
copy time_dim from '/part2/tpcds/v2.6.0/datas/handled/time_dim.dat' with delimiter as '|' NULL '';
copy warehouse from '/part2/tpcds/v2.6.0/datas/handled/warehouse.dat' with delimiter as '|' NULL '';
copy web_page from '/part2/tpcds/v2.6.0/datas/handled/web_page.dat' with delimiter as '|' NULL '';
copy web_returns from '/part2/tpcds/v2.6.0/datas/handled/web_returns.dat' with delimiter as '|' NULL '';
copy web_sales from '/part2/tpcds/v2.6.0/datas/handled/web_sales.dat' with delimiter as '|' NULL '';
copy web_site from '/part2/tpcds/v2.6.0/datas/handled/web_site.dat' with delimiter as '|' NULL '';

四、生成99个查询语句

1、修改query_template下query1-99模板，在行尾加define _END = “”，否则执行生成命令会出错;

   1)、编写shell脚本update_query.sh，并添加如下内容：

#!/bin/bash

COUNTER=1
while [ $COUNTER -lt 100 ]
do
echo $COUNTER
echo "define _END = \"\";">>query$COUNTER.tpl
COUNTER=`expr $COUNTER + 1`
done

   2)、将update_query.sh复制到query_template目录下，并执行脚本

    ./update_query.sh

2、到/part2/tpc-ds/v2.6.0/tools目录下执行如下命令：

     ./dsqgen -output_dir /part2/tpcds/querydata/ -input ../query_templates/templates.lst -scale 1 -dialect oracle -directory ../query_templates

注意：生成的语句某些数据库语法可能不支持，如：PostgreSQL

使用说明持续更新中…….