[GIT PULL] fstests: updates on 2016-08-20

[GIT PULL] fstests: updates on 2016-08-20

[Eryu Guan]

Hi Dave,

Can you please pull the fstests update from the location below? This is a
normal update, which contains new generic and XFS tests and other fixes.

Thanks,
Eryu

The following changes since commit c760a54061d26890be3929e4c6659bf3dc9e0c6a:

  src/t_immutable: allow EPERM on immutable inode (2016-08-12 11:17:34 +0800)

are available in the git repository at:

  https://github.com/guaneryu/xfstests.git for-dave

for you to fetch changes up to 3c75489a57518745598e239ffeec2af64400f185:

  common/rc: improve _require_metadata_journaling() for ext4 (2016-08-20 00:54:28 +0800)

----------------------------------------------------------------

fstests: update on 2016-08-20

This update contains:
o New tests for generic and XFS
o Miscellaneous small fixes

----------------------------------------------------------------
Brian Foster (1):
      generic: shutdown fs after log recovery

Eric Sandeen (2):
      common/quota: add prjquota to _qmount_option() filter
      xfs: test xfs_quota "state" command

Eric Whitney (2):
      generic/135: redirect stderr from _scratch_mkfs
      common/rc: improve _require_metadata_journaling() for ext4

Omar Sandoval (1):
      src/test-nextquota: fix build failure with glibc 2.24

Xiao Yang (1):
      xfs/191: skip tests on older xfsprogs

 common/quota          |   1 +
 common/rc             |  18 +++--
 src/test-nextquota.c  |   9 ++-
 tests/generic/135     |   2 +-
 tests/generic/375     |  89 ++++++++++++++++++++++
 tests/generic/375.out |   2 +
 tests/generic/group   |   1 +
 tests/xfs/263         |  97 ++++++++++++++++++++++++
 tests/xfs/263.out     | 199 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/group       |   1 +
 10 files changed, 408 insertions(+), 11 deletions(-)
--
Eryu Guan
eguan@redhat.com

Re: [GIT PULL] fstests: updates on 2016-08-20

[Dave Chinner]

On Sat, Aug 20, 2016 at 10:44:01PM +0800, Eryu Guan wrote:
> Hi Dave,
> 
> Can you please pull the fstests update from the location below? This is a
> normal update, which contains new generic and XFS tests and other fixes.
> 
> Thanks,
> Eryu
> 
> The following changes since commit c760a54061d26890be3929e4c6659bf3dc9e0c6a:
> 
>   src/t_immutable: allow EPERM on immutable inode (2016-08-12 11:17:34 +0800)
> 
> are available in the git repository at:
> 
>   https://github.com/guaneryu/xfstests.git for-dave
> 
> for you to fetch changes up to 3c75489a57518745598e239ffeec2af64400f185:
> 
>   common/rc: improve _require_metadata_journaling() for ext4 (2016-08-20 00:54:28 +0800)
> 
> ----------------------------------------------------------------
> 
> fstests: update on 2016-08-20
> 
> This update contains:
> o New tests for generic and XFS
> o Miscellaneous small fixes
> 
> ----------------------------------------------------------------
> Brian Foster (1):
>       generic: shutdown fs after log recovery

Hi Eryu,

I just pulled this all in and got an unexpected surprise - this new
test killed all of my test machines. From your description ("normal
update") I didn't expect to see something like this occur - I pulled
it, confirmed commits match, then pushed it to my test machines
and started a test cycle. I expected to see it complete without any
significant problems.

The issue here is that this new test exercises a crash case and does
not have fixes that are upstream yet - we have review backlog that
has piled up while 4.8-rc1 regressions are being dealt with and
getting the xfsprogs rmap support reviewed and merged. Upstream can
only move as fast as review bandwidth will allow, and so sometimes
things don't get merged as quickly as we'd all like.

As such, can you try to hold off merging new tests that crash or
hang systems until the bug fixes have been committed in the upstream
repositories? This won't affect reviewers or testers (they grab
the test in themselves to exercise the problem), but for everyone
else merging it will just be a nuisance because there's nothing they
can do to make the test pass (excluding it is the only solution).

In future, maybe it would be a good idea to ask the patch submitter
to tell you when the fix for a dangerous test like this has been
merged? That way you can and use that to determine when you push it
out for everyone? If it's just a pass/fail test it really doesn't
matter, but dangerous tests need to be handled a bit more
carefully.

For now, I'm going to hold off pushing this update out so other
people don't have to work around this issue whilst we clear out the
upstream patch backlog. Hopefully that won't take too long.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [GIT PULL] fstests: updates on 2016-08-20

[Eryu Guan]

On Tue, Aug 23, 2016 at 04:54:47PM +1000, Dave Chinner wrote:
> On Sat, Aug 20, 2016 at 10:44:01PM +0800, Eryu Guan wrote:
> > Hi Dave,
> > 
> > Can you please pull the fstests update from the location below? This is a
> > normal update, which contains new generic and XFS tests and other fixes.
> > 
> > Thanks,
> > Eryu
> > 
> > The following changes since commit c760a54061d26890be3929e4c6659bf3dc9e0c6a:
> > 
> >   src/t_immutable: allow EPERM on immutable inode (2016-08-12 11:17:34 +0800)
> > 
> > are available in the git repository at:
> > 
> >   https://github.com/guaneryu/xfstests.git for-dave
> > 
> > for you to fetch changes up to 3c75489a57518745598e239ffeec2af64400f185:
> > 
> >   common/rc: improve _require_metadata_journaling() for ext4 (2016-08-20 00:54:28 +0800)
> > 
> > ----------------------------------------------------------------
> > 
> > fstests: update on 2016-08-20
> > 
> > This update contains:
> > o New tests for generic and XFS
> > o Miscellaneous small fixes
> > 
> > ----------------------------------------------------------------
> > Brian Foster (1):
> >       generic: shutdown fs after log recovery
> 
> Hi Eryu,
> 
> I just pulled this all in and got an unexpected surprise - this new
> test killed all of my test machines. From your description ("normal
> update") I didn't expect to see something like this occur - I pulled
> it, confirmed commits match, then pushed it to my test machines
> and started a test cycle. I expected to see it complete without any
> significant problems.

I saw only mount failures from this case in my testings (4.8-rc2
kernel), I didn't expect any crash either.

> 
> The issue here is that this new test exercises a crash case and does
> not have fixes that are upstream yet - we have review backlog that
> has piled up while 4.8-rc1 regressions are being dealt with and
> getting the xfsprogs rmap support reviewed and merged. Upstream can
> only move as fast as review bandwidth will allow, and so sometimes
> things don't get merged as quickly as we'd all like.

Understand.

> 
> As such, can you try to hold off merging new tests that crash or
> hang systems until the bug fixes have been committed in the upstream
> repositories? This won't affect reviewers or testers (they grab
> the test in themselves to exercise the problem), but for everyone
> else merging it will just be a nuisance because there's nothing they
> can do to make the test pass (excluding it is the only solution).

Sure, no problem. (ext4/022 has the same problem, it hangs kernel but
the fixes are not in upstream yet, it has 'dangerous' group though.)

> 
> In future, maybe it would be a good idea to ask the patch submitter
> to tell you when the fix for a dangerous test like this has been
> merged? That way you can and use that to determine when you push it
> out for everyone? If it's just a pass/fail test it really doesn't
> matter, but dangerous tests need to be handled a bit more
> carefully.

That sounds good, I'll pay some attentions to the patch status too.

> 
> For now, I'm going to hold off pushing this update out so other
> people don't have to work around this issue whilst we clear out the
> upstream patch backlog. Hopefully that won't take too long.

OKay. Thanks for all the information!

Thanks,
Eryu

Re: [GIT PULL] fstests: updates on 2016-08-20

[Darrick J. Wong]

On Tue, Aug 23, 2016 at 03:16:08PM +0800, Eryu Guan wrote:
> On Tue, Aug 23, 2016 at 04:54:47PM +1000, Dave Chinner wrote:
> > On Sat, Aug 20, 2016 at 10:44:01PM +0800, Eryu Guan wrote:
> > > Hi Dave,
> > > 
> > > Can you please pull the fstests update from the location below? This is a
> > > normal update, which contains new generic and XFS tests and other fixes.
> > > 
> > > Thanks,
> > > Eryu
> > > 
> > > The following changes since commit c760a54061d26890be3929e4c6659bf3dc9e0c6a:
> > > 
> > >   src/t_immutable: allow EPERM on immutable inode (2016-08-12 11:17:34 +0800)
> > > 
> > > are available in the git repository at:
> > > 
> > >   https://github.com/guaneryu/xfstests.git for-dave
> > > 
> > > for you to fetch changes up to 3c75489a57518745598e239ffeec2af64400f185:
> > > 
> > >   common/rc: improve _require_metadata_journaling() for ext4 (2016-08-20 00:54:28 +0800)
> > > 
> > > ----------------------------------------------------------------
> > > 
> > > fstests: update on 2016-08-20
> > > 
> > > This update contains:
> > > o New tests for generic and XFS
> > > o Miscellaneous small fixes
> > > 
> > > ----------------------------------------------------------------
> > > Brian Foster (1):
> > >       generic: shutdown fs after log recovery
> > 
> > Hi Eryu,
> > 
> > I just pulled this all in and got an unexpected surprise - this new
> > test killed all of my test machines. From your description ("normal
> > update") I didn't expect to see something like this occur - I pulled
> > it, confirmed commits match, then pushed it to my test machines
> > and started a test cycle. I expected to see it complete without any
> > significant problems.
> 
> I saw only mount failures from this case in my testings (4.8-rc2
> kernel), I didn't expect any crash either.

Is this in reference generic/323?  For whatever reason I get a flood
of overlapping kernel stack smeared all over ttyS0.

--D

> 
> > 
> > The issue here is that this new test exercises a crash case and does
> > not have fixes that are upstream yet - we have review backlog that
> > has piled up while 4.8-rc1 regressions are being dealt with and
> > getting the xfsprogs rmap support reviewed and merged. Upstream can
> > only move as fast as review bandwidth will allow, and so sometimes
> > things don't get merged as quickly as we'd all like.
> 
> Understand.
> 
> > 
> > As such, can you try to hold off merging new tests that crash or
> > hang systems until the bug fixes have been committed in the upstream
> > repositories? This won't affect reviewers or testers (they grab
> > the test in themselves to exercise the problem), but for everyone
> > else merging it will just be a nuisance because there's nothing they
> > can do to make the test pass (excluding it is the only solution).
> 
> Sure, no problem. (ext4/022 has the same problem, it hangs kernel but
> the fixes are not in upstream yet, it has 'dangerous' group though.)
> 
> > 
> > In future, maybe it would be a good idea to ask the patch submitter
> > to tell you when the fix for a dangerous test like this has been
> > merged? That way you can and use that to determine when you push it
> > out for everyone? If it's just a pass/fail test it really doesn't
> > matter, but dangerous tests need to be handled a bit more
> > carefully.
> 
> That sounds good, I'll pay some attentions to the patch status too.
> 
> > 
> > For now, I'm going to hold off pushing this update out so other
> > people don't have to work around this issue whilst we clear out the
> > upstream patch backlog. Hopefully that won't take too long.
> 
> OKay. Thanks for all the information!
> 
> Thanks,
> Eryu
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] fstests: updates on 2016-08-20

[Eryu Guan]

On Tue, Aug 23, 2016 at 08:10:43AM -0700, Darrick J. Wong wrote:
> > > Hi Eryu,
> > > 
> > > I just pulled this all in and got an unexpected surprise - this new
> > > test killed all of my test machines. From your description ("normal
> > > update") I didn't expect to see something like this occur - I pulled
> > > it, confirmed commits match, then pushed it to my test machines
> > > and started a test cycle. I expected to see it complete without any
> > > significant problems.
> > 
> > I saw only mount failures from this case in my testings (4.8-rc2
> > kernel), I didn't expect any crash either.
> 
> Is this in reference generic/323?  For whatever reason I get a flood
> of overlapping kernel stack smeared all over ttyS0.

If you're using my github tree, it should be generic/375. Dave hasn't
pushed this update to upstream tree yet, so you shouldn't see it if
you're using upstream tree.

Thanks,
Eryu

Re: [GIT PULL] fstests: updates on 2016-08-20

[Dave Chinner]

On Tue, Aug 23, 2016 at 08:10:43AM -0700, Darrick J. Wong wrote:
> On Tue, Aug 23, 2016 at 03:16:08PM +0800, Eryu Guan wrote:
> > On Tue, Aug 23, 2016 at 04:54:47PM +1000, Dave Chinner wrote:
> > > On Sat, Aug 20, 2016 at 10:44:01PM +0800, Eryu Guan wrote:
> > > > Hi Dave,
> > > > 
> > > > Can you please pull the fstests update from the location below? This is a
> > > > normal update, which contains new generic and XFS tests and other fixes.
> > > > 
> > > > Thanks,
> > > > Eryu
> > > > 
> > > > The following changes since commit c760a54061d26890be3929e4c6659bf3dc9e0c6a:
> > > > 
> > > >   src/t_immutable: allow EPERM on immutable inode (2016-08-12 11:17:34 +0800)
> > > > 
> > > > are available in the git repository at:
> > > > 
> > > >   https://github.com/guaneryu/xfstests.git for-dave
> > > > 
> > > > for you to fetch changes up to 3c75489a57518745598e239ffeec2af64400f185:
> > > > 
> > > >   common/rc: improve _require_metadata_journaling() for ext4 (2016-08-20 00:54:28 +0800)
> > > > 
> > > > ----------------------------------------------------------------
> > > > 
> > > > fstests: update on 2016-08-20
> > > > 
> > > > This update contains:
> > > > o New tests for generic and XFS
> > > > o Miscellaneous small fixes
> > > > 
> > > > ----------------------------------------------------------------
> > > > Brian Foster (1):
> > > >       generic: shutdown fs after log recovery
> > > 
> > > Hi Eryu,
> > > 
> > > I just pulled this all in and got an unexpected surprise - this new
> > > test killed all of my test machines. From your description ("normal
> > > update") I didn't expect to see something like this occur - I pulled
> > > it, confirmed commits match, then pushed it to my test machines
> > > and started a test cycle. I expected to see it complete without any
> > > significant problems.
> > 
> > I saw only mount failures from this case in my testings (4.8-rc2
> > kernel), I didn't expect any crash either.
> 
> Is this in reference generic/323?  For whatever reason I get a flood
> of overlapping kernel stack smeared all over ttyS0.

generic/375 is the new test that triggers unending recursive faults
across my machines. generic/323 has been modified since late 2014,
so if it's causing you problems, then it's probably a kernel
regression...

Cheers,

Dave.

-- 
Dave Chinner
david@fromorbit.com

Re: [GIT PULL] fstests: updates on 2016-08-20

[Dave Chinner]

On Sat, Aug 20, 2016 at 10:44:01PM +0800, Eryu Guan wrote:
> Eric Sandeen (2):
>       xfs: test xfs_quota "state" command

This test hard codes inode numbers in the golden output. Fails the
moment you change the mkfs parameters away from the default. e.g.
enable rmapbt=1:

xfs/263  - output mismatch (see /home/dave/src/xfstests-dev/results//xfs_1k/xfs/263.out.bad)
    --- tests/xfs/263.out       2016-08-23 16:55:38.026707187 +1000
    +++ /home/dave/src/xfstests-dev/results//xfs_1k/xfs/263.out.bad     2016-08-26 11:13:06.533209472 +1000
    @@ -5,15 +5,15 @@
     User quota state on SCRATCH_MNT (SCRATCH_DEV)
       Accounting: ON
       Enforcement: ON
    -  Inode: #131 (1 blocks, 1 extents)
    +  Inode: #195 (1 blocks, 1 extents)
     Group quota state on SCRATCH_MNT (SCRATCH_DEV)
       Accounting: OFF
    ...
    (Run 'diff -u tests/xfs/263.out /home/dave/src/xfstests-dev/results//xfs_1k/xfs/263.out.bad'  to see the entire diff)

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [GIT PULL] fstests: updates on 2016-08-20

[Eryu Guan]

On Fri, Aug 26, 2016 at 02:51:15PM +1000, Dave Chinner wrote:
> On Sat, Aug 20, 2016 at 10:44:01PM +0800, Eryu Guan wrote:
> > Eric Sandeen (2):
> >       xfs: test xfs_quota "state" command
> 
> This test hard codes inode numbers in the golden output. Fails the
> moment you change the mkfs parameters away from the default. e.g.
> enable rmapbt=1:
> 
> xfs/263  - output mismatch (see /home/dave/src/xfstests-dev/results//xfs_1k/xfs/263.out.bad)
>     --- tests/xfs/263.out       2016-08-23 16:55:38.026707187 +1000
>     +++ /home/dave/src/xfstests-dev/results//xfs_1k/xfs/263.out.bad     2016-08-26 11:13:06.533209472 +1000
>     @@ -5,15 +5,15 @@
>      User quota state on SCRATCH_MNT (SCRATCH_DEV)
>        Accounting: ON
>        Enforcement: ON
>     -  Inode: #131 (1 blocks, 1 extents)
>     +  Inode: #195 (1 blocks, 1 extents)
>      Group quota state on SCRATCH_MNT (SCRATCH_DEV)
>        Accounting: OFF
>     ...
>     (Run 'diff -u tests/xfs/263.out /home/dave/src/xfstests-dev/results//xfs_1k/xfs/263.out.bad'  to see the entire diff)

I noticed the hard-coded inode numbers too at review time, but I saw
that the mkfs options were hard-coded as well, so I think the inode
numbers wouldn't change too.

As stated in the commit log, xfs/263 requires patches to the upstream
kernel ("quota: fill in Q_XGETQSTAT inode information for inactive
quotas") and to xfsprogs ("xfs_quota: wire up XFS_GETQSTATV") to pass.

I just tested xfs/263 with rmapbt enabled, with latest xfsprogs you just
updated (which contains commit bb92709 "xfs_quota: wire up
XFS_GETQSTATV") and 4.8-rc3 kernel + kernel patch "quota: fill in
Q_XGETQSTAT inode information for inactive quotas" from Eric, and test
passed without problems. Perhaps you need that kernel patch to make it
pass?

But I agree that it'd better to filter the inode numbers to avoid future
changes to golden image. Should I drop it for now or we can fix it later
with a follow-up patch?

Thanks,
Eryu