blktrace with fio replay for benchmarking vendor offerings

["Jarle Bjørgeengen" <jarle.bjorgeengen@usit.uio.no>]

Hello,

I'm involved in the purchasing process of block storage systems, and 
research viable benchmarking strategies for specifying and verifying 
performance requirements.

Ideally I would like to capture our traces of our current daily 
production workload  with blktrace, attach the traces to the tender and 
require the configuration to be able to run 2x that kind of workload. 
During acceptance I would like to hook enough hardware to saturate the 
system with the same workload, and measure that the requirement has been 
met.

I'm interested in comments about the practical viability of such 
approach if anyone have similar experiences.

Some open questions:

How safe is it to run blktrace on critical production environment? What 
precautions should be made?

Given that the current system consists of 3 HP EVA800 with X number of 
LUNS about 50 hosts, and the new system likely is a single system with 5 
new servers running at at full speed, how much value will the benchmark 
provide compared to "real world" ? Should I create equally many luns and 
distribute load across the 5 machines?

Is it best to scale the workload to 2X replaying all traces 
simultanously with one fio-trace-replay/lun, and then dublicate alle 
replays so that each lun serves two fio-replays rather than one, and 
then 3 and so on.

Any thoughts or comments are very much appreciated.

Best regards
Jarle Bj�rgeengen
University of Oslo / USIT