YCSB的安装编译与运用--测试HBase

YCSB的安装编译与运用

YCSB是一款开源的能够为数据库做基准测试的工具，如果你在考虑哪款数据库，不妨先用YCSB做一些基准测试，根据们的性能做对比，从而帮助自己做出选择。这款产品是雅虎的的贡献，致谢！

安装：

1. 可以从https://github.com/brianfrankcooper/YCSB/wik直接下载zip包解压直接使用，但是我们的HBASE集群的版本是1.2.0，而最新的zip包里只有HBase10Binding,没有Hase12Biding,所以不能直接用，我试过，虽然集群连接没有问题，但是会读写失败。故而，我选择了下载源代码YCSBMaster，自己编译

2.编译时，用最新版本的maven即可，mven -version,

Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-04T03:39:06+08:00)

Maven home: /Users/tuyoo/Applications/apache-maven-3.5.0

Java version: 1.8.0_144, vendor: Oracle Corporation

Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre

Default locale: zh_CN, platform encoding: UTF-8

OS name: "mac os x", version: "10.12.3", arch: "x86_64", family: "mac",

按以上配置好环境即可

3.cd到YCSB-Master根目录，查年pom.xml查看HBase对应的版本，最新的是hbase1.2.5,网上说应该改成自己的版本，如1.2.0，但是我试了下，不行，会出现读写失败，所以不用改变，1.2.5兼容1.2.0，已试

4.在ycsb-master目录下，输入

maven clean package等待编译打包成功后，在distribution下的target下就能找到编译好的zip包ycsb-0.13.0-SNAPSHOT.tar.gz，上传到集群，解压，在hbase12-biding下建立conf文件夹，把集群下的core-site.xml,hdfs-site.xml,hbase-site.xml,放入其中，由于是自己编译的，所以有关的jar包都已下载好，不用像网上说的还要去把hbase,hadoop下的jar包拷过来放在hbase12-biding的lib下。





5.在hbase shell下创建usertable表，也可以不建，相应的下面的命令中的表参数要变，

成功后运行命令 

load数据：

bin/ycsb  load hbase12 -P workloads/workloada -cp /home/fanchangyou/ycsb-0.13.0-SNAPSHOT/hbase12-binding/ -p table=usertable  -p columnfamily=cf1 -s

运行数据：

bin/ycsb  run hbase12 -P workloads/workloadc -cp /home/fanchangyou/ycsb-0.13.0-SNAPSHOT/hbase12-binding/ -p table=usertable  -p columnfamily=cf1 -s

bin/ycsb  load hbase12 -P workloads/myworkload_load -cp /home/fanchangyou/ycsb-0.13.0-SNAPSHOT/hbase12-binding/ -p table=usertable  -p columnfamily=cf1 -s

bin/ycsb  run hbase12 -P workloads/myworkload_read -cp /home/fanchangyou/ycsb-0.13.0-SNAPSHOT/hbase12-binding/ -p table=usertable  -p columnfamily=cf1 -s

6.可以看控制台分析测试情况





7.workload的相关配置：请看官网或是YCSB下的workload/workload_template

Uniform（等概率随机选择记录）、Zipfian（随机选择记录，存在热纪录）和Latest（近期写入的记录是热记录）

workload: workload class to use (e.g. com.yahoo.ycsb.workloads.CoreWorkload)

db: database class to use. Alternatively this may be specified on the command line. (default: com.yahoo.ycsb.BasicDB)

exporter: measurements exporter class to use (default: com.yahoo.ycsb.measurements.exporter.TextMeasurementsExporter)

exportfile: path to a file where output should be written instead of to stdout (default: undefined/write to stdout)

threadcount: number of YCSB client threads. Alternatively this may be specified on the command line. (default: 1)

measurementtype: supported measurement types are histogram and timeseries (default: histogram)

fieldcount: the number of fields in a record (default: 10)

fieldlength: the size of each field (default: 100)

readallfields: should reads read all fields (true) or just one (false) (default: true)

readproportion: what proportion of operations should be reads (default: 0.95)

updateproportion: what proportion of operations should be updates (default: 0.05)

insertproportion: what proportion of operations should be inserts (default: 0)

scanproportion: what proportion of operations should be scans (default: 0)

readmodifywriteproportion: what proportion of operations should be read a record, modify it, write it back (default: 0)

requestdistribution: what distribution should be used to select the records to operate on – uniform, zipfian or latest (default: uniform)

maxscanlength: for scans, what is the maximum number of records to scan (default: 1000)

scanlengthdistribution: for scans, what distribution should be used to choose the number of records to scan, for each scan, between 1 and maxscanlength (default: uniform)

insertorder: should records be inserted in order by key (“ordered”), or in hashed order (“hashed”) (default: hashed)

operationcount: number of operations to perform.

maxexecutiontime: maximum execution time in seconds. The benchmark runs until either the 

operation count has exhausted or the maximum specified time has elapsed, whichever is earlier.

table: the name of the table (default: usertable)

recordcount: number of records to load into the database initially (default: 0)

core_workload_insertion_retry_limit: number of attempts for any failed insert operation (default: 0)

core_workload_insertion_retry_interval: interval between retries, in seconds (default: 3)

hdrhistogram.percentiles: comma seperated list of percentile values to be calculated for each measurement (default: 95,99)

hdrhistogram.fileoutput=true|false This option will enable periodical writes of the interval histogram into an output file. The path can be set through the ‘hdrhistogram.output.path’ property.

insertstart: The index of the record to start at.

insertcount: The number of records to insert.