Re: Amount of data read with mixed workload sequential/random with percentage_random set

[Jens Axboe <axboe@kernel.dk>]

On Thu, Oct 10 2013, Juergen Salk wrote:
> * Jens Axboe <axboe@kernel.dk> [130925 15:01]:
> 
> > >>> --- snip ---
> > >>> [global]
> > >>> ioengine=sync
> > >>> direct=0
> > >>> bssplit=19k/25:177k/15:350k/60
> > >>> size=100m
> > >>> numjobs=4
> > >>> directory=/tmp
> > >>>
> > >>> [work]
> > >>> rw=randread
> > >>> --- snip ---
> > >>>
> > >>> $ fio jobfile.fio >fio.out
> > >>> $ grep io= fio.out
> > >>>   read : io=199968KB, bw=4892.6KB/s, iops=27, runt= 40872msec
> > >>>   read : io=200062KB, bw=5083.5KB/s, iops=28, runt= 39359msec
> > >>>   read : io=200156KB, bw=4989.1KB/s, iops=27, runt= 40112msec
> > >>>   read : io=199940KB, bw=4492.4KB/s, iops=24, runt= 44507msec
> > >>>    READ: io=800126KB, aggrb=17977KB/s, minb=4492KB/s, maxb=5083KB/s, mint=39359msec, maxt=44507msec
> > >>>
> > >>> [...]
> > >>>
> > >>> I am probably missing something obvious, but why does the job file 
> > >>> above result in 200 MB read by every process?
> > >>
> > >> It should not, that's definitely a bug. I'm guessing it's triggered by
> > >> the strange block sizes being used. Can you see if adding:
> > >>
> > >> random_generator=lfsr
> > >>
> > >> helps?
> > > 
> > > Thanks for your response, Jens. Yes it does. It's a bit confusing
> > > though, as the man page says "LFSR only works with single block
> > > sizes, not with workloads that use multiple block sizes. If used
> > > with such a workload, fio may read or write some blocks multiple
> > > times." Shouldn't this be read as "Don't use LFSR with mixed
> > > block sizes."?
> > 
> > That is correct, my suspicion is just that the current logic around when
> > to decide to run another loop is wrong. Right now we just look to see if
> > the remainder is smaller than the max block size, but that doesn't mean
> > that we necessarily have that big of a free chunk available. I suspect
> > you are hitting this because of the odd block sizes.
> > 
> > > I am sorry to keep on harping on the matter, but I am planning to
> > > use fio for simulating file sizes where total runtime will
> > > become a serious issue. And these simulations will definitely
> > > involve strange mixed block sizes ...
> > 
> > No worries, it's definitely a bug. Checking for a fix right now...
> 
> Just out of curiosity: Is there any news on that issue? 

I got it mostly fixed, but it gets quite complicated. Traveling this
week, will hopefully have some time to wrap it up.

-- 
Jens Axboe