ceph存储 ceph文件系统测试工具filebench介绍

filebench是一款位于Stony Brook的File system and
Storage Lab (FSL)开发的测试文件系统的负载。

Filebench 是一款文件系统性能的自动化测试工具，它通过快速模拟真实应用服务器的负载来测试文件系统的性能。它不仅可以仿真文件系统微操作（如 copyfiles, createfiles, randomread, randomwrite ），而且可以仿真复杂的应用程序（如 varmail, fileserver, oltp, dss, webserver, webproxy ）。 Filebench 比较适合用来测试文件服务器性能，但同时也是一款负载自动生成工具，也可用于文件系统的性能。

和传统的iozone不同，这款benchmark可以模拟这个类型的负载，如mailserver, fileserver等等。和各个知名的mailserver,fileserver负载相比，这款benchmark可以让你修改负载其中的参数（比如程序需要测试的file size大小等）。某天浏览FAST2012会议，发现很多论文都是采用该benchmark测试，估计公认度会越来越高。

下载地址：http://sourceforge.net/apps/mediawiki/filebench/index.php?title=Main_Page

Features

Filebench includes many features to facilitate file system benchmarking:

Multiple workload types support via loadable personalities
Ships with more than 40 pre-defined personalities, including the one that describe mail, web, file, and database servers behaviour
Easy to add new personalities using reach Workload Model Language (WML)
Multi-process and multi-thread workload support
Configurable directory hierarchies with depth, width, and file sizes set to given statistical distributions
Support of asynchronous I/O and process synchronization primitives

Integrated statistics for throughput, latency, and CPU cycle counts per system call
Tested on Linux, FreeBSD, and Solaris platforms (should work for any POSIX-compliant Operating System)

Installation

Getting and Installing Filebench

Filebench is distributed as a compressed tar-ball with sources. Download the newest version of Filebench
here.
Uncompress it and follow the regular procedure for building UNIX sources:

./configure

make

sudo make install

If all of these steps complete successfully, you're ready to run Filebench. In case you run into compilation error or warning, please, report it using
Bug Tracking System.

Filebench does not have any mandatory program/library dependencies except libc. If you want command line auto-completion to work in Filebench you need to install
libtecla prior
to Filebench.

Files in the Installation

Filebench's

make install

installs single

/usr/local/bin/filebench

binary and a number of workload personalities to

/usr/local/share/filebench/workloads/

directory.Default installation directories might vary from distribution to distribution and you can set them to other values during

./config

uration. Workload personalities are stored in files with

.f

extension. More than 40 pre-defined personalities are already included in Filebench package.

Running

Filebench generates I/O operations by executing a workload personality, which defines the workload to apply to the system and might provide various tunables to customize the workload. As it was mentioned earlier, Filebench
is shipped with a library of these personalities, ready to use. Below we describe a use case when one wants to use a pre-defined workload. If you want to define a new personality and run it, read
Writing Workload Models
page.

Interactively

If the directory where the

filebench

binary was installed is in the shell

PATH

, you can start Filebench by simply executing

filebench

command.Filebench prompt will appear after that:

user@host$ filebench
Filebench Version 1.4.9
IMPORTANT: Virtual address space randomization is enabled on this machine!
It is highly recommended to disable randomization to provide stable Filebench runs.
Echo 0 to /proc/sys/kernel/randomize_va_space file to disable the randomization.
WARNING: Could not open /proc/sys/kernel/shmmax file!
It means that you probably ran Filebench not as a root. Filebench will not increase shared
region limits in this case, which can lead to the failures on certain workloads.
11431: 0.000: Allocated 170MB of shared memory
filebench> quit

You can type Filebench commands now. Type

quit

to exit the prompt.

One can see two warnings above:

A lot of Linux distributions enable address space randomization. This prevents Filebench from mapping a shared memory region to the same address in different processes. Disable address space randomization (

echo
0 > /proc/sys/kernel/randomize_va_space

) for stable operation of multi-process workloads.
Second warning informs that Filebench was not able to increase shared memory region size. You can either:

Run Filebench as root
Increase shared memory region size to 256MB (

sudo echo 268435456 > /proc/sys/kernel/randomize_va_space

) and ignore this warning

If one disables address space randomization and runs Filebench as root, the output looks much cleaner:

user@host$ sudo su
[sudo] password for user:
root@host# echo 0 > /proc/sys/kernel/randomize_va_space
root@host# go_filebench
Filebench Version 1.4.9
12102: 0.000: Allocated 170MB of shared memory
filebench>

Now, one can load and run individual workload personalities with full control over their parameters. The following example demonstrates how to run

fileserver

workload personality:

root@user# go_filebench
Filebench Version 1.4.9
12324: 0.000: Allocated 170MB of shared memory
filebench> load fileserver
12462: 2.869: FileServer Version 2.2 personality successfully loaded
12462: 2.869: Usage: set $dir=<dir>
12462: 2.869:        set $meanfilesize=<size>     defaults to 131072
12462: 2.869:        set $nfiles=<value>      defaults to 10000
12462: 2.869:        set $nthreads=<value>    defaults to 50
12462: 2.869:        set $meanappendsize=<value>  defaults to 16384
12462: 2.869:        set $iosize=<size>  defaults to 1048576
12462: 2.869:        set $meandirwidth=<size> defaults to 20
12462: 2.869: (sets mean dir width and dir depth is calculated as log (width, nfiles)
12462: 2.869:        run runtime (e.g. run 60)
filebench> set $dir=/mnt
filebench> run 60
12462: 4.909: Creating/pre-allocating files and filesets
12462: 4.918: Fileset bigfileset: 10000 files, avg dir width = 20, avg dir depth = 3.1, 1240.757MB
12462: 5.280: Removed any existing fileset bigfileset in 1 seconds
12462: 5.280: making tree for filset /tmp/bigfileset
12462: 5.290: Creating fileset bigfileset...
12462: 6.080: Preallocated 7979 of 10000 of fileset bigfileset in 1 seconds
12462: 6.080: waiting for fileset pre-allocation to finish
12466: 6.080: Starting 1 filereader instances
12467: 6.081: Starting 50 filereaderthread threads
12462: 7.137: Running...
12462: 67.142: Run took 60 seconds...
12462: 67.145: Per-Operation Breakdown
statfile1            128311ops     2138ops/s   0.0mb/s      0.0ms/op     2320us/op-cpu [0ms - 0ms]
deletefile1          128316ops     2138ops/s   0.0mb/s      0.2ms/op     2535us/op-cpu [0ms - 458ms]
closefile3           128323ops     2139ops/s   0.0mb/s      0.0ms/op     2328us/op-cpu [0ms - 0ms]
readfile1            128327ops     2139ops/s 283.8mb/s      0.1ms/op     2460us/op-cpu [0ms - 267ms]
openfile2            128329ops     2139ops/s   0.0mb/s      0.0ms/op     2332us/op-cpu [0ms - 2ms]
closefile2           128332ops     2139ops/s   0.0mb/s      0.0ms/op     2332us/op-cpu [0ms - 0ms]
appendfilerand1      128337ops     2139ops/s  16.6mb/s      0.1ms/op     2377us/op-cpu [0ms - 559ms]
openfile1            128343ops     2139ops/s   0.0mb/s      0.0ms/op     2353us/op-cpu [0ms - 2ms]
closefile1           128349ops     2139ops/s   0.0mb/s      0.0ms/op     2317us/op-cpu [0ms - 1ms]
wrtfile1             128352ops     2139ops/s 265.2mb/s      0.1ms/op     2601us/op-cpu [0ms - 268ms]
createfile1          128358ops     2139ops/s   0.0mb/s      0.1ms/op     2396us/op-cpu [0ms - 267ms]
12462: 67.145: IO Summary: 1411677 ops, 23526 ops/s, (2139/4278 r/w), 565mb/s, 393us cpu/op, 0.2ms latency
12462: 67.145: Shutting down processes
root@user#

As you can see, we first loaded

fileserver

personality using

load

command. Filebench located corresponding

.f

in the directory with pre-defined workloads. After that tunables of the workload personality can be set. We change the benchmark directory to

/mnt

where the file system we want to benchmark is presumably mounted. To start workload for 60 seconds we execute

run 60

command. In response, Filebench first created a file system tree with the properties defined in the personality file and then spanned all required processes and threads. After 60 seconds of the run the statistics is printed and Filebench
exits.

Non-interactively

If you wish to run Filebench in non-interactive mode, you can use

-f

option. However, you need to add '

run <time>

' to the end of the workload personality file. I prefer to create a copy of original workload file for that:

root@host# cp /usr/local/share/filebench/workloads/fileserver.f /tmp/fileserver-noninteractive.f
root@host# vim /tmp/fileserver-noninteractive.f (add 'run 60' to the end of this file)
root@host# filebench -f /tmp/fileserver-noninteractive.f

After that you will see traditional Filebench output.

文档提到的2个注意点

1. filebench默认开启地址空间randomization (后面有解释，什么是address space randomization)。如果测试多进程的负载，为了稳定操作可以选择关闭该选项。方法是：

echo 0 > /proc/sys/kernel/randomize_va_space

2. 如果想增加filebench的共享内存，有2个方法

   a. 用root用户执行。（su）

   b.

echo 268435456 > /proc/sys/kernel/randomize_va_space （增加到256MB）

运行

你可以选择导入别人已经定制好的负载类型（如fileserver）。也可以自己使用workload model language定制特定的负载。下面介绍的是文档演示的导入已经定制好的fileserver负载（模拟）。

1. 进入filebench

2. load fileserver (filebench导入对应的~.f文件，代码自带，也可以从这个地址下载http://sourceforge.net/apps/mediawiki/filebench/index.php?title=Pre-defined_personalities)

3. set $dir=/mnt （测试mnt目录）

4. run 60 （运行60秒，运行完毕后打印统计信息）

非交互式运行

执行一条命令即可完成测试，无需要像前面一步一步的设置各个参数。注意在配置文件的末尾添加

'

run <time>

'
http://sourceforge.net/apps/mediawiki/filebench/index.php?title=Main_Page#Support