Re: [PATCH 0/1] Batched discard support

[Lukas Czerner <lczerner@redhat.com>]

On Thu, 30 Sep 2010, Dave Chinner wrote:

> On Wed, Sep 29, 2010 at 10:54:25AM +0200, Lukas Czerner wrote:
> > Hi,
> > 
> > I am working on something I have called "batched discard support" for Ext3
> > and Ext4 filesystems. Traditional discard support for filesystems like Ext4
> > has been implemented the way that whenever the file is unlinked the
> > disk-space that the file was using is trimmed (discarded) by
> > sb_issue_discard() to let the device know that this portion of disk is no
> > longer in use by the filesystem and can be safely used for wear-leveling.
> > 
> > However, this approach comes with very noticeable performance loss on most
> > of SSD devices and LUN's I have the opportunity to test it on. The fact is,
> > that bigger discard ranges are more efficient than smaller ones, so it make
> > sense try to batch the ranges together wherever it is possible.
> > 
> > I have introduced new filesystem independent ioctl (FITRIM) which can be used
> > to send the "trim this portion of filesystem" command down to the filesystem
> > which (if implemented) discards all free extents in that range.
> > 
> > The implementation for Ext3 and Ext4 is complete and you can see it here:
> > 
> > http://www.spinics.net/lists/linux-ext4/msg21050.html
> > 
> > Why I am sending it here to linux-fsdevel is because I am introducing new fs
> > independent ioctl and new member of super_operations (trim_fs) and we would
> > like let you know about this approach (which any filesystem can take
> > advantage from) and we would like your comment on this patch before we
> > send it upstream.
> 
> My first question is: how do you test a filesystem implements
> ->trim_fs correctly?
> 
> That is, if we are going to include a data-destroying ioctl, I
> really want some filesystem independent tests written first so that
> as filesystems implement ->trim_fs they can be tested for correct
> implementation.
> 
> Perhaps adding FITRIM support to xfs_io, and a generic test to
> xfstests would be the way to go. e.g. write a set of patterned files
> to the filesystem, unlink a number of the files, then run some trim
> commands on the filesystem exercising corner cases and check that
> none of the data in still-active files is damaged (e.g. via md5sum
> comparison)....
> 
> Cheers,
> 
> Dave.

Hi Dave,

When chasing bugs in those patches I have extensively used stress.sh
script which is part of my gensuit package:

https://sourceforge.net/projects/gensuit/

..and I found it very helpful for testing this. Basically, it does
exactly what you have proposed. It creates checksums of all files in
defined directory (e.g. /usr/share/doc) and run several processes which
will clear its working directory, then copy everything from this
previously checksummed directory (e.g. /usr/share/doc) into its working
directory, create list of files in working directory and its
checksums and compare it with the original list of checksums. Every
process works in the loop so it repeat remove->copy->check.

While the stress.sh tool is running I have run the FITRIM ioctl in the
loop, so when the FITRIM is really buggy (thus data-destroying) the
stress.sh will notice, because checksums will most change change.

As I said I found it sufficient to test FITRIM ioctl on ext3 and ext4
filesystem. So if you think it is good enough I will create new
xfstests test which will do the same thing as stress.sh. Actually I will
post this new test shortly if everyone is ok with this approach.

Thanks!
-Lukas