Test ext4/001

Test ext4/001

[Jan Kara]

Hi,

I've checked why test ext4/001 fails for me with DAX and after some
investigation I've realized that the test assumes that
extent_max_zeroout_kb is 32 KB and thus unwritten extent will get converted
to written as a whole and not split. With DAX that doesn't happen (because
of difference between EXT4_GET_BLOCKS_ flags passed in writeback path and
DAX write path) and so the result differs.

So I was wondering how to best fix this. Either we could switch
extent_max_zeroout_kb to 0 to make the result same (but that has a slight
disadvantage that we would lose testing of the zeroout logic) or we could
increase file size so that zeroout doesn't trigger or something else?
Anyone has some idea?

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: Test ext4/001

[Lukáš Czerner]

On Thu, 22 Oct 2015, Jan Kara wrote:

> Date: Thu, 22 Oct 2015 11:10:17 +0200
> From: Jan Kara <jack@suse.cz>
> To: linux-ext4@vger.kernel.org
> Cc: fstests@vger.kernel.org
> Subject: Test ext4/001
> 
> Hi,
> 
> I've checked why test ext4/001 fails for me with DAX and after some
> investigation I've realized that the test assumes that
> extent_max_zeroout_kb is 32 KB and thus unwritten extent will get converted
> to written as a whole and not split. With DAX that doesn't happen (because
> of difference between EXT4_GET_BLOCKS_ flags passed in writeback path and
> DAX write path) and so the result differs.
> 
> So I was wondering how to best fix this. Either we could switch
> extent_max_zeroout_kb to 0 to make the result same (but that has a slight
> disadvantage that we would lose testing of the zeroout logic) or we could
> increase file size so that zeroout doesn't trigger or something else?
> Anyone has some idea?
> 
> 								Honza

I'd rather still keep the test to exercise the zeroout logic,
because we've already had problems there and they may occur in the
future as well.

The easiest solution would be to create new test with zeroout turned
off and skip the old test for DAX and keep the current test as it is.

Thanks!
-Lukas

Re: Test ext4/001

[Theodore Ts'o]

On Thu, Oct 22, 2015 at 11:10:17AM +0200, Jan Kara wrote:
> 
> I've checked why test ext4/001 fails for me with DAX and after some
> investigation I've realized that the test assumes that
> extent_max_zeroout_kb is 32 KB and thus unwritten extent will get converted
> to written as a whole and not split. With DAX that doesn't happen (because
> of difference between EXT4_GET_BLOCKS_ flags passed in writeback path and
> DAX write path) and so the result differs.

Out of curiosity, how much memory are you using to test ext4 with DAX?
I assume you're doing something like what Matthew Wilcox documented
at: http://patchwork.ozlabs.org/patch/491107/

I should really figure out a way to automate doing DAX regression
testing using my scripts.

> So I was wondering how to best fix this. Either we could switch
> extent_max_zeroout_kb to 0 to make the result same (but that has a slight
> disadvantage that we would lose testing of the zeroout logic) or we could
> increase file size so that zeroout doesn't trigger or something else?
> Anyone has some idea?

The approach I would suggest is to fork 001.out to 001.out.zeroout and
001.out.nozerrout, and then test to see if our output file matches
either file.  That means we'll redirect the output to our own 001.tmp2
file and do the check against the two possible 001.out files in the
ext4/001 script, but the advantage of doing things that way is that is
that will also solve a false positive we're seeing when ext4
encryption is enabled, and for a similar reason (extent zero-out is
disabled when encryption is enabled).

Cheers,

						- Ted

Re: Test ext4/001

[Jan Kara]

On Thu 22-10-15 15:22:13, Ted Tso wrote:
> On Thu, Oct 22, 2015 at 11:10:17AM +0200, Jan Kara wrote:
> > 
> > I've checked why test ext4/001 fails for me with DAX and after some
> > investigation I've realized that the test assumes that
> > extent_max_zeroout_kb is 32 KB and thus unwritten extent will get converted
> > to written as a whole and not split. With DAX that doesn't happen (because
> > of difference between EXT4_GET_BLOCKS_ flags passed in writeback path and
> > DAX write path) and so the result differs.
> 
> Out of curiosity, how much memory are you using to test ext4 with DAX?
> I assume you're doing something like what Matthew Wilcox documented
> at: http://patchwork.ozlabs.org/patch/491107/
> 
> I should really figure out a way to automate doing DAX regression
> testing using my scripts.

Well, I took a machine with 128GB of RAM and use 2x16GB for test disks. I
have used the trick Matthew described but setting up plain ramdisk works as
well AFAICT.

> > So I was wondering how to best fix this. Either we could switch
> > extent_max_zeroout_kb to 0 to make the result same (but that has a slight
> > disadvantage that we would lose testing of the zeroout logic) or we could
> > increase file size so that zeroout doesn't trigger or something else?
> > Anyone has some idea?
> 
> The approach I would suggest is to fork 001.out to 001.out.zeroout and
> 001.out.nozerrout, and then test to see if our output file matches
> either file.  That means we'll redirect the output to our own 001.tmp2
> file and do the check against the two possible 001.out files in the
> ext4/001 script, but the advantage of doing things that way is that is
> that will also solve a false positive we're seeing when ext4
> encryption is enabled, and for a similar reason (extent zero-out is
> disabled when encryption is enabled).

Yeah, that's an interesting idea. I'll see how to make that work in the
cleanest possible way.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: Test ext4/001

[Dave Chinner]

On Thu, Oct 22, 2015 at 11:10:17AM +0200, Jan Kara wrote:
> Hi,
> 
> I've checked why test ext4/001 fails for me with DAX and after some
> investigation I've realized that the test assumes that
> extent_max_zeroout_kb is 32 KB and thus unwritten extent will get converted
> to written as a whole and not split. With DAX that doesn't happen (because
> of difference between EXT4_GET_BLOCKS_ flags passed in writeback path and
> DAX write path) and so the result differs.
> 
> So I was wondering how to best fix this. Either we could switch
> extent_max_zeroout_kb to 0 to make the result same (but that has a slight
> disadvantage that we would lose testing of the zeroout logic) or we could
> increase file size so that zeroout doesn't trigger or something else?
> Anyone has some idea?

Generic tests that have this problem with ext4 turn set zeroout to
0. You could always just filter the mount options to remove DAX from
this test - filtering mount options is not uncommon in XFS specific
tests that are designed to test specific functionality that may be
incompatible with the provided mount options.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com