From: Erwan Velu <erwan@enovance.com>
To: fio@vger.kernel.org
Subject: Pull Request for Adding 2 new tools around fio
Date: Tue, 25 Jun 2013 19:13:12 +0200 [thread overview]
Message-ID: <51C9CFA8.6080703@enovance.com> (raw)
Hey Jens,
I'm been using fio for years and that's a wonderful tool. Since I'm
working at eNovance, I'm facing situations where I shall compare servers
and disks in a clusterized environnement.
In my last case, I had 6 servers with 33 disks each leading to a serious
problem. How can I test and visualize performances in various io pattern :
- read, write, randread, randwrite
- 4k, 128k, 1m block size
- one disk after the other (to estimate the performance of each disk
helping tracking the weaks)
- all the disks at the same time (to estimate the ability of the host to
manage all disks at the same time)
To help setting up this benchmark, I've been doing two tools :
- genfio to generate the fio script files
Details of its behavior & options can be found in the commit itself
(also available with the -help option)
A single command like 'genfio -d sdb,sdc,sdd,sde -a -b 4k,128k,1m -r
100' generates a single fio file that will runs 60 tests on 4 disks
during 100 seconds each. That does avoid human mistake and ease
reproductability
https://github.com/enovance/fio/commit/ca0ede58c1ca10469d9937110c3faa30ce88ca1f
- fio2gnuplot turns the output log from fio into gnuplot traces
It does help tracing many log files in a friendly way by using 3D graphs
(to avoid 2D overlapping) or generates several histograms to compare
min/max/avg/stddev of some selected outputfile. If outputfiles
are well named (like if you use genfio) you can use regexp to select the
files you want to graph.
You can so select logs by the io pattern, the name of disk, etc...
More details in the commit or in the --help option
https://github.com/enovance/fio/commit/1fa4c6436b31b71ab2bb373a5033a62bca47bce0
I've been uploading some sample output here :
http://www.flickr.com/photos/ennael/sets/72157634249027122/
In a fraction of seconds, you are able to visualize the global
performances with the 3D graph and compared values betweens runs by
using the histograms. On a real 6 servers, 197 disks (33 each), it
helped at detecting the weak devices very quickly.
I hope this work will have some interest for the fio project or maybe
some other users.
This two tools are available on my 'tools' branch of my github
repository. It's on top the current master so very easy to merge. Both
tools are splitted in two commits.
https://github.com/enovance/fio/tree/tools
Sincerly,
Erwan Velu
next reply other threads:[~2013-06-25 17:46 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-06-25 17:13 Erwan Velu [this message]
2013-06-26 2:06 ` Pull Request for Adding 2 new tools around fio Ni, Xun
2013-06-27 9:02 ` Jens Axboe
2013-06-27 9:31 ` Georg Schönberger
2013-07-05 13:25 ` Erwan Velu
2013-07-05 21:17 ` Jens Axboe
2013-07-11 10:20 ` Erwan Velu
2013-07-13 13:35 ` Jens Axboe
2013-07-14 19:59 ` Erwan Velu
-- strict thread matches above, loose matches on Subject: below --
2013-06-26 6:49 Erwan Velu
[not found] ` <91E2D863603AD4478F101CE81E76E45D017C5181@SHSMSX103.ccr.corp.intel.com>
2013-06-26 7:04 ` Erwan Velu
2013-06-26 7:24 ` Ni, Xun
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=51C9CFA8.6080703@enovance.com \
--to=erwan@enovance.com \
--cc=fio@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox