Re: blktrace with fio replay for benchmarking vendor offerings

[Jiri Horky <jiri.horky@cesnet.cz>]

[-- Attachment #1: Type: text/plain, Size: 2730 bytes --]

Hi Jarle,

On 05/16/2012 09:21 PM, Jarle Bjørgeengen wrote:
> Hello,
>
> I'm involved in the purchasing process of block storage systems, and 
> research viable benchmarking strategies for specifying and verifying 
> performance requirements.
>
> Ideally I would like to capture our traces of our current daily 
> production workload  with blktrace, attach the traces to the tender 
> and require the configuration to be able to run 2x that kind of 
> workload. During acceptance I would like to hook enough hardware to 
> saturate the system with the same workload, and measure that the 
> requirement has been met.
>
> I'm interested in comments about the practical viability of such 
> approach if anyone have similar experiences.
>
actually, I am in the very same situation at Institute of Physics, where 
we annually buy some 1PB of raw disk space (the money equivalent). We 
ended up with very similar approach that you described. We use IOreplay 
from IOapps (http://code.google.com/p/ioapps/) application to run the 
load previously recorded by strace. So it is done on a file level 
instead of the block level. This clearly has its advantages (you may use 
different file system, benchmarks are not fatal for running systems 
etc.), but of course some drawbacks as well (strace overhead, etc., see 
webpage). If you are interested and have any more question/suggestion 
about the IOreplay, just email me, I actually wrote it as a part of my 
master thesis.
> Some open questions:
>
> How safe is it to run blktrace on critical production environment? 
> What precautions should be made?
>
> Given that the current system consists of 3 HP EVA800 with X number of 
> LUNS about 50 hosts, and the new system likely is a single system with 
> 5 new servers running at at full speed, how much value will the 
> benchmark provide compared to "real world" ? Should I create equally 
> many luns and distribute load across the 5 machines?
>
> Is it best to scale the workload to 2X replaying all traces 
> simultanously with one fio-trace-replay/lun, and then dublicate alle 
> replays so that each lun serves two fio-replays rather than one, and 
> then 3 and so on.
>
> Any thoughts or comments are very much appreciated.
I would say that all above heavily depends on your environment, whether 
the access pattern is similar to all LUNS and whether all LUNS are more 
or less expected to expand similarly in the future. Personally, I would 
go with settings that are simpler to setup (from the options you have 
proposed) unless you really know your future requirements and you are 
sure that the results obtained in both ways differ significantly.

Regards
Jiri Horky


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5655 bytes --]