Pull Request for Adding 2 new tools around fio

Pull Request for Adding 2 new tools around fio

[Erwan Velu]

Hey Jens,

I'm been using fio for years and that's a wonderful tool. Since I'm 
working at eNovance, I'm facing situations where I shall compare servers 
and disks in a clusterized environnement.

In my last case, I had 6 servers with 33 disks each leading to a serious 
problem. How can I test and visualize performances in various io pattern :
- read, write, randread, randwrite
- 4k, 128k, 1m block size
- one disk after the other (to estimate the performance of each disk 
helping tracking the weaks)
- all the disks at the same time (to estimate the ability of the host to 
manage all disks at the same time)

To help setting up this benchmark, I've been doing two tools :
- genfio to generate the fio script files
Details of its behavior & options can be found in the commit itself 
(also available with the -help option)
A single command like 'genfio -d sdb,sdc,sdd,sde -a -b 4k,128k,1m -r 
100' generates a single fio file that will runs 60 tests on 4 disks 
during 100 seconds each. That does avoid human mistake and ease 
reproductability

https://github.com/enovance/fio/commit/ca0ede58c1ca10469d9937110c3faa30ce88ca1f

- fio2gnuplot turns the output log from fio into gnuplot traces
It does help tracing many log files in a friendly way by using 3D graphs 
(to avoid 2D overlapping) or generates several histograms to compare 
min/max/avg/stddev of some selected outputfile. If outputfiles
are well named (like if you use genfio) you can use regexp to select the 
files you want to graph.
You can so select logs by the io pattern, the name of disk, etc...
More details in the commit or in the --help option
https://github.com/enovance/fio/commit/1fa4c6436b31b71ab2bb373a5033a62bca47bce0

I've been uploading some sample output here : 
http://www.flickr.com/photos/ennael/sets/72157634249027122/

In a fraction of seconds, you are able to visualize the global 
performances with the 3D graph and compared values betweens runs by 
using the histograms. On a real 6 servers, 197 disks (33 each), it 
helped at detecting the weak devices very quickly.

I hope this work will have some interest for the fio project or maybe 
some other users.

This two tools are available on my 'tools' branch of my github 
repository. It's on top the current master so very easy to merge. Both 
tools are splitted in two commits.

https://github.com/enovance/fio/tree/tools

Sincerly,
Erwan Velu

RE: Pull Request for Adding 2 new tools around fio

[Ni, Xun]

Hello, Erwan:

   Good to hear this, your work is exactly what I plan to do before.

Thanks,
Xun

-----Original Message-----
From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On Behalf Of Erwan Velu
Sent: Wednesday, June 26, 2013 1:13 AM
To: fio@vger.kernel.org
Subject: Pull Request for Adding 2 new tools around fio

Hey Jens,

I'm been using fio for years and that's a wonderful tool. Since I'm working at eNovance, I'm facing situations where I shall compare servers and disks in a clusterized environnement.

In my last case, I had 6 servers with 33 disks each leading to a serious problem. How can I test and visualize performances in various io pattern :
- read, write, randread, randwrite
- 4k, 128k, 1m block size
- one disk after the other (to estimate the performance of each disk helping tracking the weaks)
- all the disks at the same time (to estimate the ability of the host to manage all disks at the same time)

To help setting up this benchmark, I've been doing two tools :
- genfio to generate the fio script files Details of its behavior & options can be found in the commit itself (also available with the -help option) A single command like 'genfio -d sdb,sdc,sdd,sde -a -b 4k,128k,1m -r 100' generates a single fio file that will runs 60 tests on 4 disks during 100 seconds each. That does avoid human mistake and ease reproductability

https://github.com/enovance/fio/commit/ca0ede58c1ca10469d9937110c3faa30ce88ca1f

- fio2gnuplot turns the output log from fio into gnuplot traces It does help tracing many log files in a friendly way by using 3D graphs (to avoid 2D overlapping) or generates several histograms to compare min/max/avg/stddev of some selected outputfile. If outputfiles are well named (like if you use genfio) you can use regexp to select the files you want to graph.
You can so select logs by the io pattern, the name of disk, etc...
More details in the commit or in the --help option
https://github.com/enovance/fio/commit/1fa4c6436b31b71ab2bb373a5033a62bca47bce0

I've been uploading some sample output here : 
http://www.flickr.com/photos/ennael/sets/72157634249027122/

In a fraction of seconds, you are able to visualize the global performances with the 3D graph and compared values betweens runs by using the histograms. On a real 6 servers, 197 disks (33 each), it helped at detecting the weak devices very quickly.

I hope this work will have some interest for the fio project or maybe some other users.

This two tools are available on my 'tools' branch of my github repository. It's on top the current master so very easy to merge. Both tools are splitted in two commits.

https://github.com/enovance/fio/tree/tools

Sincerly,
Erwan Velu
--
To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Pull Request for Adding 2 new tools around fio

[Jens Axboe]

On Tue, Jun 25 2013, Erwan Velu wrote:
> Hey Jens,
> 
> I'm been using fio for years and that's a wonderful tool. Since I'm working
> at eNovance, I'm facing situations where I shall compare servers and disks
> in a clusterized environnement.
> 
> In my last case, I had 6 servers with 33 disks each leading to a serious
> problem. How can I test and visualize performances in various io pattern :
> - read, write, randread, randwrite
> - 4k, 128k, 1m block size
> - one disk after the other (to estimate the performance of each disk helping
> tracking the weaks)
> - all the disks at the same time (to estimate the ability of the host to
> manage all disks at the same time)
> 
> To help setting up this benchmark, I've been doing two tools :
> - genfio to generate the fio script files
> Details of its behavior & options can be found in the commit itself (also
> available with the -help option)
> A single command like 'genfio -d sdb,sdc,sdd,sde -a -b 4k,128k,1m -r 100'
> generates a single fio file that will runs 60 tests on 4 disks during 100
> seconds each. That does avoid human mistake and ease reproductability
> 
> https://github.com/enovance/fio/commit/ca0ede58c1ca10469d9937110c3faa30ce88ca1f
> 
> - fio2gnuplot turns the output log from fio into gnuplot traces
> It does help tracing many log files in a friendly way by using 3D graphs (to
> avoid 2D overlapping) or generates several histograms to compare
> min/max/avg/stddev of some selected outputfile. If outputfiles
> are well named (like if you use genfio) you can use regexp to select the
> files you want to graph.
> You can so select logs by the io pattern, the name of disk, etc...
> More details in the commit or in the --help option
> https://github.com/enovance/fio/commit/1fa4c6436b31b71ab2bb373a5033a62bca47bce0
> 
> I've been uploading some sample output here :
> http://www.flickr.com/photos/ennael/sets/72157634249027122/
> 
> In a fraction of seconds, you are able to visualize the global performances
> with the 3D graph and compared values betweens runs by using the histograms.
> On a real 6 servers, 197 disks (33 each), it helped at detecting the weak
> devices very quickly.
> 
> I hope this work will have some interest for the fio project or maybe some
> other users.
> 
> This two tools are available on my 'tools' branch of my github repository.
> It's on top the current master so very easy to merge. Both tools are
> splitted in two commits.
> 
> https://github.com/enovance/fio/tree/tools

Nifty stuff! Thanks, I'll take a look at it and get it integrated.

-- 
Jens Axboe

Re: Pull Request for Adding 2 new tools around fio

[Georg Schönberger]

Hey there,

> > I'm been using fio for years and that's a wonderful tool. Since I'm
> > working
> > at eNovance, I'm facing situations where I shall compare servers
> > and disks
> > in a clusterized environnement.

I am also using fio for quite a time and I really like it a lot!

> > To help setting up this benchmark, I've been doing two tools :
> > - genfio to generate the fio script files
> > Details of its behavior & options can be found in the commit itself
> > (also
> > available with the -help option)
> > A single command like 'genfio -d sdb,sdc,sdd,sde -a -b 4k,128k,1m
> > -r 100'
> > generates a single fio file that will runs 60 tests on 4 disks
> > during 100
> > seconds each. That does avoid human mistake and ease
> > reproductability

These tools look really nice!

I will pick up the topic of fio tools and also introduce a tool based on fio that
is implementing a recent SNIA specification:
* Solid State Storage (SSS) Performance Test Specification (PTS)
- http://www.snia.org/tech_activities/standards/curr_standards/pts

I was looking for an open source tool implementing this specification, but I could not found one.
Therefore I have written a performance tool implementing the standard, in Python using fio:
* TKperf
- http://git.thomas-krenn.com/TKperf_v1.git/

* TKperf is starting a test for a device and generates plots and a pdf report:
- http://www.thomas-krenn.com/de/wikiDE/images/e/e5/TKperf-Report-IntelDCS3700.pdf
(a report for the recent Intel Data Center SSDs)

* More information (video, slides, wiki article)
- http://www.thomas-krenn.com/de/wiki/Event-News:_LinuxTag_2013_SSD_and_HDD_performance_testing_with_TKperf
- http://www.thomas-krenn.com/de/wiki/TKperf

> > I hope this work will have some interest for the fio project or
> > maybe some
> > other users.

My intention is also to increase the degree of popularity for fio, as it is awesome!
Regards, Georg

Re: Pull Request for Adding 2 new tools around fio

[Erwan Velu]

On 27/06/2013 11:02, Jens Axboe wrote:
> On Tue, Jun 25 2013, Erwan Velu wrote:
> Nifty stuff! Thanks, I'll take a look at it and get it integrated. 

Hey Jens,

Anything wrong with my work ?  Looks like the merge didn't occured.

Feel free to ask if something is bad,

Thanks,

Erwan

Re: Pull Request for Adding 2 new tools around fio

[Jens Axboe]

On Fri, Jul 05 2013, Erwan Velu wrote:
> On 27/06/2013 11:02, Jens Axboe wrote:
> >On Tue, Jun 25 2013, Erwan Velu wrote:
> >Nifty stuff! Thanks, I'll take a look at it and get it integrated.
> 
> Hey Jens,
> 
> Anything wrong with my work ?  Looks like the merge didn't occured.
> 
> Feel free to ask if something is bad,

Nothing wrong, just haven't gotten around to it yet. A bit busy these
days with work and a pending move, so I probably wont get to it for a
week or two more. Sorry.

-- 
Jens Axboe

Re: Pull Request for Adding 2 new tools around fio

[Erwan Velu]

On 05/07/2013 23:17, Jens Axboe wrote:
> Nothing wrong, just haven't gotten around to it yet. A bit busy these 
> days with work and a pending move, so I probably wont get to it for a 
> week or two more. Sorry. 
Sorry I forgot to answer your email.

Sure there is now problem with that. I perfectly understand it could 
take time.
By the way, I've been adding new features to it on the same branch.

Sincerly,
Erwan

Re: Pull Request for Adding 2 new tools around fio

[Jens Axboe]

On Thu, Jul 11 2013, Erwan Velu wrote:
> On 05/07/2013 23:17, Jens Axboe wrote:
> >Nothing wrong, just haven't gotten around to it yet. A bit busy these days
> >with work and a pending move, so I probably wont get to it for a week or
> >two more. Sorry.
> Sorry I forgot to answer your email.
> 
> Sure there is now problem with that. I perfectly understand it could take
> time.
> By the way, I've been adding new features to it on the same branch.

Pulled in, thanks! I'll get to play with it for real next week. In the
future, if you have updates, please send a proper pull request to this
list (and CC me) and include a full diff of the changes as well in the
email. Basically do a git request-pull, then append the full diff of all
changes as well.

-- 
Jens Axboe

Re: Pull Request for Adding 2 new tools around fio

[Erwan Velu]

Le 13/07/2013 15:35, Jens Axboe a écrit :
> Pulled in, thanks! I'll get to play with it for real next week. In the future, if you have updates, please send a proper pull request to this list (and CC me) and include a full diff of the changes as well in the email. Basically do a git request-pull, then append the full diff of all changes as well. 
Thanks Jens !

I'll proceed this way with pleasure. I'm already working on patches for genfio.

The thing I like on fio : Everytime I need something that I'm not using.... fio does have it.
Just decovered exec_{pre|post}run solving perfectly my need.

Cheers,
Erwan

RE: Pull Request for Adding 2 new tools around fio

[Erwan Velu]

Before what sorry ? Do you have the same needs as me ? Does this tool solves them ?

Erwan

"Ni, Xun" <xun.ni@intel.com> a écrit :

>Hello, Erwan:
>
>   Good to hear this, your work is exactly what I plan to do before.
>
>Thanks,
>Xun
>
>-----Original Message-----
>From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On Behalf Of Erwan Velu
>Sent: Wednesday, June 26, 2013 1:13 AM
>To: fio@vger.kernel.org
>Subject: Pull Request for Adding 2 new tools around fio
>
>Hey Jens,
>
>I'm been using fio for years and that's a wonderful tool. Since I'm working at eNovance, I'm facing situations where I shall compare servers and disks in a clusterized environnement.
>
>In my last case, I had 6 servers with 33 disks each leading to a serious problem. How can I test and visualize performances in various io pattern :
>- read, write, randread, randwrite
>- 4k, 128k, 1m block size
>- one disk after the other (to estimate the performance of each disk helping tracking the weaks)
>- all the disks at the same time (to estimate the ability of the host to manage all disks at the same time)
>
>To help setting up this benchmark, I've been doing two tools :
>- genfio to generate the fio script files Details of its behavior & options can be found in the commit itself (also available with the -help option) A single command like 'genfio -d sdb,sdc,sdd,sde -a -b 4k,128k,1m -r 100' generates a single fio file that will runs 60 tests on 4 disks during 100 seconds each. That does avoid human mistake and ease reproductability
>
>https://github.com/enovance/fio/commit/ca0ede58c1ca10469d9937110c3faa30ce88ca1f
>
>- fio2gnuplot turns the output log from fio into gnuplot traces It does help tracing many log files in a friendly way by using 3D graphs (to avoid 2D overlapping) or generates several histograms to compare min/max/avg/stddev of some selected outputfile. If outputfiles are well named (like if you use genfio) you can use regexp to select the files you want to graph.
>You can so select logs by the io pattern, the name of disk, etc...
>More details in the commit or in the --help option
>https://github.com/enovance/fio/commit/1fa4c6436b31b71ab2bb373a5033a62bca47bce0
>
>I've been uploading some sample output here : 
>http://www.flickr.com/photos/ennael/sets/72157634249027122/
>
>In a fraction of seconds, you are able to visualize the global performances with the 3D graph and compared values betweens runs by using the histograms. On a real 6 servers, 197 disks (33 each), it helped at detecting the weak devices very quickly.
>
>I hope this work will have some interest for the fio project or maybe some other users.
>
>This two tools are available on my 'tools' branch of my github repository. It's on top the current master so very easy to merge. Both tools are splitted in two commits.
>
>https://github.com/enovance/fio/tree/tools
>
>Sincerly,
>Erwan Velu
>--
>To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html
>--
>To unsubscribe from this list: send the line "unsubscribe fio" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Pull Request for Adding 2 new tools around fio

[Erwan Velu]

On 26/06/2013 08:55, Ni, Xun wrote:
> I have my own tool, it's different with each test cases, it solve my problem, I looked your test results and found that yours is much more powerfully than mine.
>
> Maybe my English is not very well, so the 'before' means I have this idea 2-3 month ago and plan to write a tool like yours
>
> Thanks,
> Xun
>
Glad to see my work is useful for others ;o)
If you have comments, ideas or patches, I'll be glad to receive them.

Cheers,
Erwan

RE: Pull Request for Adding 2 new tools around fio

[Ni, Xun]

Sure, glad to have the same idea with you.

Thanks,
Xun

-----Original Message-----
From: Erwan Velu [mailto:erwan@enovance.com] 
Sent: Wednesday, June 26, 2013 3:05 PM
To: Ni, Xun
Cc: fio@vger.kernel.org
Subject: Re: Pull Request for Adding 2 new tools around fio

On 26/06/2013 08:55, Ni, Xun wrote:
> I have my own tool, it's different with each test cases, it solve my problem, I looked your test results and found that yours is much more powerfully than mine.
>
> Maybe my English is not very well, so the 'before' means I have this 
> idea 2-3 month ago and plan to write a tool like yours
>
> Thanks,
> Xun
>
Glad to see my work is useful for others ;o) If you have comments, ideas or patches, I'll be glad to receive them.

Cheers,
Erwan