Re: Patch to re-use already filled up pattern in io buffers

[Jens Axboe <jaxboe@fusionio.com>]

On 07/14/2010 08:33 AM, Jens Axboe wrote:
> On 07/14/2010 02:13 AM, Radha Ramachandran wrote:
>> Hi Jens,
>> I made changes to fio so we wld re-use the already populated io_u
>> buffer (when there is a non-random pattern) during writes. That way
>> only the header will be re-calculated for every I/O. This way the
>> buffer wld get populated in the beginning and as long as the
>> subsequent ios using the same io_u structure are writes and have same
>> or less block size, it wld get re-used. If any of the subsequent i/o
>> is a read or has a block size greater than the pre-filled one, then
>> the buffer is invalidated and will be re-filled at the next write.
>>
>> Reason for this risky change: (Performance)
>> I tested this change on a tmpfs(with no swap backing), with the
>> following config file:
>> [sscan_write]
>> filename=/mytmpfs/datafile.tmp
>> rw=write
>> bs=64k
>> size=3G
>> ioengine=libaio
>> iodepth=1024
>> iodepth_low=512
>> runtime=10800
>> bwavgtime=5000
>> thread=1
>> do_verify=0
>> verify=meta
>> verify_pattern=0x55aaa55a
>> verify_interval=4k
>> continue_on_error=1
>>
>> fio-1-41-6 gave 306MB/s and the new change had a performance of 1546MB/s
>>
>> Side effects/Risks:
>> There is a risk with this fix, that if the buffer gets corrupted then
>> the subsequent writes will also be corrupt. I think for both
>> sequential writes and random writes (with verify, where the I/O log is
>> replayed) we shld be able to find the first I/O that started with the
>> corruption and if the buffer is getting corrupted, there are other
>> issues here.
>>
>> Testing:
>> I have tested this fix with sequential write(verify)/random read write
>> mix combination(with verify).
>>
>> I think I have taken care of most of the case, but please let me know
>> if there is anything I have missed. I have attached the patch along
>> with this email. I think the performance improvement outweighs the
>> risk associated with the fix. But I will let you decide if you wld
>> like to pick it up.
> 
> I will pick this up, the fill time is the reason for some of the
> other hoops we jump through to try and avoid that when possible.
> I don't think the risk of memory corruption is something we need
> to consider. That could just as easily happen after the data
> has been read anyway, both cases would result in a verify error.
> 

I made a small change to turn ->buf_filled into an io_u
flag. It would be nice if you could retest the current
-git state and ensure that it works properly. I'm on the
road the next ~10 days, so wont have a lot of testing time
on my hands.

-- 
Jens Axboe