public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH 5.15 00/15] xfs stable candidate patches for 5.15.y
@ 2022-06-03 18:46 Leah Rumancik
  2022-06-04  8:38 ` Amir Goldstein
  0 siblings, 1 reply; 9+ messages in thread
From: Leah Rumancik @ 2022-06-03 18:46 UTC (permalink / raw)
  To: leah.rumancik, linux-xfs, djwong; +Cc: Leah Rumancik

From: Leah Rumancik <lrumancik@google.com>

This first round of patches aims to take care of the easy cases - patches
with the Fixes tag that apply cleanly. I have ~30 more patches identified
which will be tested next, thanks everyone for the various suggestions
for tracking down more bug fixes. No regressions were seen during
testing when running fstests 3 times per config with the following configs:

xfs defaults
quota
quota 1k
v4
pmem and fsdax
realtime
8k directory blocks
external log
realtime and external log devices
realtime with 28k extents, external log devices
overlayfs atop xfs
overlayfs atop ext4
ext4 defaults

Thanks,
Leah

Brian Foster (1):
  xfs: punch out data fork delalloc blocks on COW writeback failure

Christoph Hellwig (1):
  xfs: remove xfs_inew_wait

Darrick J. Wong (7):
  xfs: remove all COW fork extents when remounting readonly
  xfs: only run COW extent recovery when there are no live extents
  xfs: prevent UAF in xfs_log_item_in_current_chkpt
  xfs: only bother with sync_filesystem during readonly remount
  xfs: don't generate selinux audit messages for capability testing
  xfs: use setattr_copy to set vfs inode attributes
  xfs: don't include bnobt blocks when reserving free block pool

Dave Chinner (4):
  xfs: check sb_meta_uuid for dabuf buffer recovery
  xfs: async CIL flushes need pending pushes to be made stable
  xfs: run callbacks before waking waiters in
    xlog_state_shutdown_callbacks
  xfs: drop async cache flushes from CIL commits.

Rustam Kovhaev (1):
  xfs: use kmem_cache_free() for kmem_cache objects

Yang Xu (1):
  xfs: Fix the free logic of state in xfs_attr_node_hasname

 fs/xfs/libxfs/xfs_attr.c      | 17 ++++-----
 fs/xfs/xfs_aops.c             | 15 ++++++--
 fs/xfs/xfs_bio_io.c           | 35 ------------------
 fs/xfs/xfs_buf_item_recover.c |  2 +-
 fs/xfs/xfs_extfree_item.c     |  6 +--
 fs/xfs/xfs_fsmap.c            |  4 +-
 fs/xfs/xfs_fsops.c            |  2 +-
 fs/xfs/xfs_icache.c           | 21 -----------
 fs/xfs/xfs_inode.h            |  4 +-
 fs/xfs/xfs_ioctl.c            |  2 +-
 fs/xfs/xfs_iops.c             | 58 ++---------------------------
 fs/xfs/xfs_linux.h            |  2 -
 fs/xfs/xfs_log.c              | 58 ++++++++++++-----------------
 fs/xfs/xfs_log_cil.c          | 70 +++++++++++++++++------------------
 fs/xfs/xfs_log_priv.h         |  3 +-
 fs/xfs/xfs_log_recover.c      | 24 +++++++++++-
 fs/xfs/xfs_mount.c            | 12 +-----
 fs/xfs/xfs_mount.h            | 15 ++++++++
 fs/xfs/xfs_pnfs.c             |  3 +-
 fs/xfs/xfs_reflink.c          |  5 ++-
 fs/xfs/xfs_super.c            | 30 ++++++++-------
 kernel/capability.c           |  1 +
 22 files changed, 154 insertions(+), 235 deletions(-)

-- 
2.36.1.255.ge46751e96f-goog


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 5.15 00/15] xfs stable candidate patches for 5.15.y
  2022-06-03 18:46 [PATCH 5.15 00/15] xfs stable candidate patches for 5.15.y Leah Rumancik
@ 2022-06-04  8:38 ` Amir Goldstein
  2022-06-06 15:55   ` Luis Chamberlain
  2022-06-06 17:42   ` Leah Rumancik
  0 siblings, 2 replies; 9+ messages in thread
From: Amir Goldstein @ 2022-06-04  8:38 UTC (permalink / raw)
  To: Leah Rumancik
  Cc: linux-xfs, Darrick J. Wong, Leah Rumancik, Luis R. Rodriguez,
	Theodore Tso

On Sat, Jun 4, 2022 at 6:53 AM Leah Rumancik <leah.rumancik@gmail.com> wrote:
>
> From: Leah Rumancik <lrumancik@google.com>
>
> This first round of patches aims to take care of the easy cases - patches
> with the Fixes tag that apply cleanly. I have ~30 more patches identified
> which will be tested next, thanks everyone for the various suggestions
> for tracking down more bug fixes. No regressions were seen during
> testing when running fstests 3 times per config with the following configs:

Hi Leah!

I'll let the xfs developers comment on the individual patches.
General comments about stable process and collaboration.

Some of the patches in this series are relevant to 5.10 and even apply
cleanly to 5.10 (see below).
They are in my queue but I did not get to test them thoroughly yet,
because I am working chronologically.

To avoid misunderstanding with stable maintainers, when you post to
stable, please make sure to state clearly in the cover letter that those
patches have only been tested on 5.15.y and should only be applied
to 5.15.y.
I know you have 5.15 in subject, but I would rather be safe than sorry.

Luis has advised me to post up to 10 patches in each round.
The rationale is that after we test and patches are applied to stable
regressions may be detected and reported by downstream users.

Regressions will be easier to bisect if there are less fixes in every
LTS release. For this reason, I am holding on to my part 2 patches
until 5.10.120 is released. LTS releases are usually on weekly basis
so the delay is not much.

I don't think that this series is terribly big, so I am fine with you
posting it at one go, but please consider splitting it pre 5.16
and post 5.16 or any other way that you see fit when you post
to stable, but let's wait for xfs developers review - if they tell you to
drop a few patches my comment will become moot ;-)

[...]

For the record:

> Brian Foster (1):
>   xfs: punch out data fork delalloc blocks on COW writeback failure
In my queue

>
> Darrick J. Wong (7):
>   xfs: remove all COW fork extents when remounting readonly
>   xfs: only run COW extent recovery when there are no live extents
Only first one is in my queue - needed backporting

>   xfs: prevent UAF in xfs_log_item_in_current_chkpt
In my queue

>   xfs: only bother with sync_filesystem during readonly remount
In my queue

>   xfs: use setattr_copy to set vfs inode attributes
In my queue - needed backporting

>   xfs: don't include bnobt blocks when reserving free block pool
>
> Dave Chinner (4):
>   xfs: check sb_meta_uuid for dabuf buffer recovery
In my queue

>
> Rustam Kovhaev (1):
>   xfs: use kmem_cache_free() for kmem_cache objects
In my queue

>
> Yang Xu (1):
>   xfs: Fix the free logic of state in xfs_attr_node_hasname
In my queue - needed backporting

Thanks,
Amir.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 5.15 00/15] xfs stable candidate patches for 5.15.y
  2022-06-04  8:38 ` Amir Goldstein
@ 2022-06-06 15:55   ` Luis Chamberlain
  2022-06-06 18:57     ` Leah Rumancik
  2022-06-06 17:42   ` Leah Rumancik
  1 sibling, 1 reply; 9+ messages in thread
From: Luis Chamberlain @ 2022-06-06 15:55 UTC (permalink / raw)
  To: Amir Goldstein
  Cc: Leah Rumancik, linux-xfs, Darrick J. Wong, Leah Rumancik,
	Theodore Tso

On Sat, Jun 04, 2022 at 11:38:35AM +0300, Amir Goldstein wrote:
> On Sat, Jun 4, 2022 at 6:53 AM Leah Rumancik <leah.rumancik@gmail.com> wrote:
> >
> > From: Leah Rumancik <lrumancik@google.com>
> >
> > This first round of patches aims to take care of the easy cases - patches
> > with the Fixes tag that apply cleanly. I have ~30 more patches identified
> > which will be tested next, thanks everyone for the various suggestions
> > for tracking down more bug fixes. No regressions were seen during
> > testing when running fstests 3 times per config with the following configs:

Leah,

It is great to see this work move forward.

How many times was fstest run *without* the patches to establish the
baseline? Do you have a baseline for known failures published somewhere?

For v5.10.y effort we aimed for 100 times so to ensure we have a high
confidence in the baseline. That baseline is here:

https://github.com/linux-kdevops/kdevops/tree/master/workflows/fstests/expunges/5.10.105/xfs/unassigned

For XFS the latest baseline we are tracking on kdevops is v5.17 and you can
see the current results here:

https://github.com/linux-kdevops/kdevops/tree/master/workflows/fstests/expunges/5.17.0-rc7/xfs/unassigned

This passed 100 loops of fstests already. The target "test steady state"
of 100 is set in kdevops using CONFIG_KERNEL_CI_STEADY_STATE_GOAL=100.

As discussed at LSFMM is there a chance we can collaborate on a baseline
together? One way I had suggested we could do this for different test
runners is to have git subtree with the expunges which we can all share
for different test runner.

The configuration used is dynamically generated for the target
test dev and pool, but the rest is pretty standard:

https://github.com/linux-kdevops/kdevops/blob/master/playbooks/roles/fstests/templates/xfs/xfs.config

Hearing that only 3 loops of running fstests is run gives me a bit of
concern for introducing a regression with a low failure rate. I realize
that we may be limited in resources to test running fstests in a loop
but just 3 tests should take a bit over a day. I think we can do better.
At the very last you can give me your baseline and I can try to confirm
if matches what I see. Then, 30 patches seems like a lot, so I think it
would be best to add patches to stable 10 at a time max.

  Luis

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 5.15 00/15] xfs stable candidate patches for 5.15.y
  2022-06-04  8:38 ` Amir Goldstein
  2022-06-06 15:55   ` Luis Chamberlain
@ 2022-06-06 17:42   ` Leah Rumancik
  2022-06-08  7:56     ` Amir Goldstein
  1 sibling, 1 reply; 9+ messages in thread
From: Leah Rumancik @ 2022-06-06 17:42 UTC (permalink / raw)
  To: Amir Goldstein
  Cc: linux-xfs, Darrick J. Wong, Luis R. Rodriguez, Theodore Tso

On Sat, Jun 04, 2022 at 11:38:35AM +0300, Amir Goldstein wrote:
> On Sat, Jun 4, 2022 at 6:53 AM Leah Rumancik <leah.rumancik@gmail.com> wrote:
> >
> > From: Leah Rumancik <lrumancik@google.com>
> >
> > This first round of patches aims to take care of the easy cases - patches
> > with the Fixes tag that apply cleanly. I have ~30 more patches identified
> > which will be tested next, thanks everyone for the various suggestions
> > for tracking down more bug fixes. No regressions were seen during
> > testing when running fstests 3 times per config with the following configs:
> 
> Hi Leah!
> 
> I'll let the xfs developers comment on the individual patches.
> General comments about stable process and collaboration.
> 
> Some of the patches in this series are relevant to 5.10 and even apply
> cleanly to 5.10 (see below).
> They are in my queue but I did not get to test them thoroughly yet,
> because I am working chronologically.
> 
> To avoid misunderstanding with stable maintainers, when you post to
> stable, please make sure to state clearly in the cover letter that those
> patches have only been tested on 5.15.y and should only be applied
> to 5.15.y.
> I know you have 5.15 in subject, but I would rather be safe than sorry.

Fair concern, will do.

> 
> Luis has advised me to post up to 10 patches in each round.
> The rationale is that after we test and patches are applied to stable
> regressions may be detected and reported by downstream users.
> 
> Regressions will be easier to bisect if there are less fixes in every
> LTS release. For this reason, I am holding on to my part 2 patches
> until 5.10.120 is released. LTS releases are usually on weekly basis
> so the delay is not much.
> 
> I don't think that this series is terribly big, so I am fine with you
> posting it at one go, but please consider splitting it pre 5.16
> and post 5.16 or any other way that you see fit when you post
> to stable, but let's wait for xfs developers review - if they tell you to
> drop a few patches my comment will become moot ;-)
> 

Sure, that is no problem for me, I'll go ahead and split into sets of 10
or fewer.

Thanks for the comments!
Leah

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 5.15 00/15] xfs stable candidate patches for 5.15.y
  2022-06-06 15:55   ` Luis Chamberlain
@ 2022-06-06 18:57     ` Leah Rumancik
  2022-06-06 19:30       ` Luis Chamberlain
  0 siblings, 1 reply; 9+ messages in thread
From: Leah Rumancik @ 2022-06-06 18:57 UTC (permalink / raw)
  To: Luis Chamberlain; +Cc: Amir Goldstein, linux-xfs, Darrick J. Wong, Theodore Tso

On Mon, Jun 06, 2022 at 08:55:24AM -0700, Luis Chamberlain wrote:
> On Sat, Jun 04, 2022 at 11:38:35AM +0300, Amir Goldstein wrote:
> > On Sat, Jun 4, 2022 at 6:53 AM Leah Rumancik <leah.rumancik@gmail.com> wrote:
> > >
> > > From: Leah Rumancik <lrumancik@google.com>
> > >
> > > This first round of patches aims to take care of the easy cases - patches
> > > with the Fixes tag that apply cleanly. I have ~30 more patches identified
> > > which will be tested next, thanks everyone for the various suggestions
> > > for tracking down more bug fixes. No regressions were seen during
> > > testing when running fstests 3 times per config with the following configs:
> 
> Leah,
> 
> It is great to see this work move forward.
> 
> How many times was fstest run *without* the patches to establish the
> baseline? Do you have a baseline for known failures published somewhere?

Currently, the tests are being run 10x per config without the patches.
If a failure is seen with the patches, the tests are rerun on the
baseline several hundred times to see if the failure was a regression or
to determine the baseline failure rate.

> 
> For v5.10.y effort we aimed for 100 times so to ensure we have a high
> confidence in the baseline. That baseline is here:
> 
> https://github.com/linux-kdevops/kdevops/tree/master/workflows/fstests/expunges/5.10.105/xfs/unassigned
> 
> For XFS the latest baseline we are tracking on kdevops is v5.17 and you can
> see the current results here:
> 
> https://github.com/linux-kdevops/kdevops/tree/master/workflows/fstests/expunges/5.17.0-rc7/xfs/unassigned
> 
> This passed 100 loops of fstests already. The target "test steady state"
> of 100 is set in kdevops using CONFIG_KERNEL_CI_STEADY_STATE_GOAL=100.
> 
> As discussed at LSFMM is there a chance we can collaborate on a baseline
> together? One way I had suggested we could do this for different test
> runners is to have git subtree with the expunges which we can all share
> for different test runner.
> 

Could you elaborate on this a bit? Are you hoping to gain insight from
comparing 5.10.y baseline with 5.15.y baseline or are you hoping to
allow people working on the same stable branch to have a joint record of
test run output?

> The configuration used is dynamically generated for the target
> test dev and pool, but the rest is pretty standard:
> 
> https://github.com/linux-kdevops/kdevops/blob/master/playbooks/roles/fstests/templates/xfs/xfs.config
> 
> Hearing that only 3 loops of running fstests is run gives me a bit of
> concern for introducing a regression with a low failure rate. I realize
> that we may be limited in resources to test running fstests in a loop
> but just 3 tests should take a bit over a day. I think we can do better.
> At the very last you can give me your baseline and I can try to confirm
> if matches what I see. 

I can go ahead and bump up the amount of test runs. It would be nice to
agree on the number of test runs and the specific configs to test. For a
fixed amount of resources there is a tradeoff between broader coverage
through more configs vs more solid results with fewer configs. I am not
sure where everyone's priorities lie.

After the new runs, I'll go ahead and post the baseline and send out a
link so we can compare.

> Then, 30 patches seems like a lot, so I think it
> would be best to add patches to stable 10 at a time max.

I am planning on batching into smaller groups, 10 at a time works for
me.

Best,
Leah

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 5.15 00/15] xfs stable candidate patches for 5.15.y
  2022-06-06 18:57     ` Leah Rumancik
@ 2022-06-06 19:30       ` Luis Chamberlain
  2022-06-08 22:16         ` Leah Rumancik
  0 siblings, 1 reply; 9+ messages in thread
From: Luis Chamberlain @ 2022-06-06 19:30 UTC (permalink / raw)
  To: Leah Rumancik; +Cc: Amir Goldstein, linux-xfs, Darrick J. Wong, Theodore Tso

On Mon, Jun 06, 2022 at 11:57:08AM -0700, Leah Rumancik wrote:
> On Mon, Jun 06, 2022 at 08:55:24AM -0700, Luis Chamberlain wrote:
> > On Sat, Jun 04, 2022 at 11:38:35AM +0300, Amir Goldstein wrote:
> > > On Sat, Jun 4, 2022 at 6:53 AM Leah Rumancik <leah.rumancik@gmail.com> wrote:
> > > >
> > > > From: Leah Rumancik <lrumancik@google.com>
> > > >
> > > > This first round of patches aims to take care of the easy cases - patches
> > > > with the Fixes tag that apply cleanly. I have ~30 more patches identified
> > > > which will be tested next, thanks everyone for the various suggestions
> > > > for tracking down more bug fixes. No regressions were seen during
> > > > testing when running fstests 3 times per config with the following configs:
> > 
> > Leah,
> > 
> > It is great to see this work move forward.
> > 
> > How many times was fstest run *without* the patches to establish the
> > baseline? Do you have a baseline for known failures published somewhere?
> 
> Currently, the tests are being run 10x per config without the patches.
> If a failure is seen with the patches, the tests are rerun on the
> baseline several hundred times to see if the failure was a regression or
> to determine the baseline failure rate.

This is certainly one way to go about it. This just means that you have
to do this work then as a second step. Whereas if you first have a high
confidence in a baseline you then are pretty certain you have a
regression once a test fails after you start testing deltas on
a stable release.

Average failure rates for non-deterministic tests tend to be about
1/2 - 1/30. Although things such as 1/60, anything beyond 1/100
exist is *very* rare. So running fstests just 10 times seems to me
rather low to have any sort of high confidence in a baseline.

> > For v5.10.y effort we aimed for 100 times so to ensure we have a high
> > confidence in the baseline. That baseline is here:
> > 
> > https://github.com/linux-kdevops/kdevops/tree/master/workflows/fstests/expunges/5.10.105/xfs/unassigned
> > 
> > For XFS the latest baseline we are tracking on kdevops is v5.17 and you can
> > see the current results here:
> > 
> > https://github.com/linux-kdevops/kdevops/tree/master/workflows/fstests/expunges/5.17.0-rc7/xfs/unassigned
> > 
> > This passed 100 loops of fstests already. The target "test steady state"
> > of 100 is set in kdevops using CONFIG_KERNEL_CI_STEADY_STATE_GOAL=100.
> > 
> > As discussed at LSFMM is there a chance we can collaborate on a baseline
> > together? One way I had suggested we could do this for different test
> > runners is to have git subtree with the expunges which we can all share
> > for different test runner.
> > 
> 
> Could you elaborate on this a bit? Are you hoping to gain insight from
> comparing 5.10.y baseline with 5.15.y baseline or are you hoping to
> allow people working on the same stable branch to have a joint record of
> test run output?

Not output, but to share failures known to exist per kernel release and
per filesystem, and even Linux distribution. We can shared this as
expressed in an expunge file which can be used as input to running
fstests so that tests are skipped for the release.

Annotations can be made with comments, you can see an existin list here:

https://github.com/linux-kdevops/kdevops/tree/master/workflows/fstests/expunges/

I currently track *.bad and *.dmesg outputs into gists and refer to them
with a URL. Likewise when possible I annotate the failure rate.

*If* it makes sense to collaborate on that front I can extract *just*
the expunges directory and make its own git subtree which then kdevops
uses. Other test runner can then use the same git tree as a git subtree.

> > The configuration used is dynamically generated for the target
> > test dev and pool, but the rest is pretty standard:
> > 
> > https://github.com/linux-kdevops/kdevops/blob/master/playbooks/roles/fstests/templates/xfs/xfs.config
> > 
> > Hearing that only 3 loops of running fstests is run gives me a bit of
> > concern for introducing a regression with a low failure rate. I realize
> > that we may be limited in resources to test running fstests in a loop
> > but just 3 tests should take a bit over a day. I think we can do better.
> > At the very last you can give me your baseline and I can try to confirm
> > if matches what I see. 
> 
> I can go ahead and bump up the amount of test runs. It would be nice to
> agree on the number of test runs and the specific configs to test. For a
> fixed amount of resources there is a tradeoff between broader coverage
> through more configs vs more solid results with fewer configs. I am not
> sure where everyone's priorities lie.

Sure, it is all a tradeoff. But given we want to strive to collaborate,
I'd hope we can strive for a reasoanably well tested baseline. Given
average failure rates for non deterministic tests linger aroun 1/2 -
1/30, and given it can take about 1 week to run fstests in a loop
100 times *for any filesystem*, I think it is reasonable to use 100 as
good baseline target for a "test steady state".

> After the new runs, I'll go ahead and post the baseline and send out a
> link so we can compare.

Groovy!

> > Then, 30 patches seems like a lot, so I think it
> > would be best to add patches to stable 10 at a time max.
> 
> I am planning on batching into smaller groups, 10 at a time works for
> me.

Great!

  Luis

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 5.15 00/15] xfs stable candidate patches for 5.15.y
  2022-06-06 17:42   ` Leah Rumancik
@ 2022-06-08  7:56     ` Amir Goldstein
  2022-06-08 22:24       ` Leah Rumancik
  0 siblings, 1 reply; 9+ messages in thread
From: Amir Goldstein @ 2022-06-08  7:56 UTC (permalink / raw)
  To: Leah Rumancik; +Cc: linux-xfs, Darrick J. Wong, Luis R. Rodriguez, Theodore Tso

On Mon, Jun 6, 2022 at 8:42 PM Leah Rumancik <leah.rumancik@gmail.com> wrote:
>
> On Sat, Jun 04, 2022 at 11:38:35AM +0300, Amir Goldstein wrote:
> > On Sat, Jun 4, 2022 at 6:53 AM Leah Rumancik <leah.rumancik@gmail.com> wrote:
> > >
> > > From: Leah Rumancik <lrumancik@google.com>
> > >
> > > This first round of patches aims to take care of the easy cases - patches
> > > with the Fixes tag that apply cleanly. I have ~30 more patches identified
> > > which will be tested next, thanks everyone for the various suggestions
> > > for tracking down more bug fixes. No regressions were seen during
> > > testing when running fstests 3 times per config with the following configs:
> >
> > Hi Leah!
> >
> > I'll let the xfs developers comment on the individual patches.
> > General comments about stable process and collaboration.
> >
> > Some of the patches in this series are relevant to 5.10 and even apply
> > cleanly to 5.10 (see below).
> > They are in my queue but I did not get to test them thoroughly yet,
> > because I am working chronologically.
> >
> > To avoid misunderstanding with stable maintainers, when you post to
> > stable, please make sure to state clearly in the cover letter that those
> > patches have only been tested on 5.15.y and should only be applied
> > to 5.15.y.
> > I know you have 5.15 in subject, but I would rather be safe than sorry.
>
> Fair concern, will do.
>
> >
> > Luis has advised me to post up to 10 patches in each round.
> > The rationale is that after we test and patches are applied to stable
> > regressions may be detected and reported by downstream users.
> >
> > Regressions will be easier to bisect if there are less fixes in every
> > LTS release. For this reason, I am holding on to my part 2 patches
> > until 5.10.120 is released. LTS releases are usually on weekly basis
> > so the delay is not much.
> >
> > I don't think that this series is terribly big, so I am fine with you
> > posting it at one go, but please consider splitting it pre 5.16
> > and post 5.16 or any other way that you see fit when you post
> > to stable, but let's wait for xfs developers review - if they tell you to
> > drop a few patches my comment will become moot ;-)
> >
>
> Sure, that is no problem for me, I'll go ahead and split into sets of 10
> or fewer.
>

FWIW, the following subset of your 5.15 patches (or backported version thereof)
have been sitting in my xfs-5.10.y-8 tag since Saturday and have been
spinning in kdevops since (~20 auto runs) with no regressions observed
from v5.10.y baseline:

xfs: punch out data fork delalloc blocks on COW writeback failure
xfs: remove all COW fork extents when remounting readonly
xfs: prevent UAF in xfs_log_item_in_current_chkpt
xfs: only bother with sync_filesystem during readonly remount
xfs: use setattr_copy to set vfs inode attributes
xfs: check sb_meta_uuid for dabuf buffer recovery
xfs: use kmem_cache_free() for kmem_cache objects
xfs: Fix the free logic of state in xfs_attr_node_hasname

So perhaps you could use that as the smaller subset for first posting.
To reduce review burden on xfs maintainers, I could break out of the
chronological patches order and use the same subset for my next set
of candidates for 5.10 after testing them in isolation on top of xfs-5.10.y-3
(at least the ones that apply out of order).

Thanks,
Amir.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 5.15 00/15] xfs stable candidate patches for 5.15.y
  2022-06-06 19:30       ` Luis Chamberlain
@ 2022-06-08 22:16         ` Leah Rumancik
  0 siblings, 0 replies; 9+ messages in thread
From: Leah Rumancik @ 2022-06-08 22:16 UTC (permalink / raw)
  To: Luis Chamberlain; +Cc: Amir Goldstein, linux-xfs, Darrick J. Wong, Theodore Tso

On Mon, Jun 06, 2022 at 12:30:59PM -0700, Luis Chamberlain wrote:
> On Mon, Jun 06, 2022 at 11:57:08AM -0700, Leah Rumancik wrote:
> > On Mon, Jun 06, 2022 at 08:55:24AM -0700, Luis Chamberlain wrote:
> > > On Sat, Jun 04, 2022 at 11:38:35AM +0300, Amir Goldstein wrote:
> > > > On Sat, Jun 4, 2022 at 6:53 AM Leah Rumancik <leah.rumancik@gmail.com> wrote:
> > > > >
> > > > > From: Leah Rumancik <lrumancik@google.com>
> > > > >
> > > > > This first round of patches aims to take care of the easy cases - patches
> > > > > with the Fixes tag that apply cleanly. I have ~30 more patches identified
> > > > > which will be tested next, thanks everyone for the various suggestions
> > > > > for tracking down more bug fixes. No regressions were seen during
> > > > > testing when running fstests 3 times per config with the following configs:
> > > 
> > > Leah,
> > > 
> > > It is great to see this work move forward.
> > > 
> > > How many times was fstest run *without* the patches to establish the
> > > baseline? Do you have a baseline for known failures published somewhere?
> > 
> > Currently, the tests are being run 10x per config without the patches.
> > If a failure is seen with the patches, the tests are rerun on the
> > baseline several hundred times to see if the failure was a regression or
> > to determine the baseline failure rate.
> 
> This is certainly one way to go about it. This just means that you have
> to do this work then as a second step. Whereas if you first have a high
> confidence in a baseline you then are pretty certain you have a
> regression once a test fails after you start testing deltas on
> a stable release.
> 
> Average failure rates for non-deterministic tests tend to be about
> 1/2 - 1/30. Although things such as 1/60, anything beyond 1/100
> exist is *very* rare. So running fstests just 10 times seems to me
> rather low to have any sort of high confidence in a baseline.
> 

Unfortunately, I am seeing some failures pop up with a fail rate of
~0.5-2% :( I typically end up rerunning failing tests up to 1000 times to
be confident about the failure rate on both the baseline and the backports
branch. Running each test 1000 times from the start is a bit much, but I
upped the test runs on both the baseline and backports branches to 100
runs per test to hopefully filter out some of the tests that fail more
consistently.


> > > 
> > > As discussed at LSFMM is there a chance we can collaborate on a baseline
> > > together? One way I had suggested we could do this for different test
> > > runners is to have git subtree with the expunges which we can all share
> > > for different test runner.
> > > 
> > 
> > Could you elaborate on this a bit? Are you hoping to gain insight from
> > comparing 5.10.y baseline with 5.15.y baseline or are you hoping to
> > allow people working on the same stable branch to have a joint record of
> > test run output?
> 
> Not output, but to share failures known to exist per kernel release and
> per filesystem, and even Linux distribution. We can shared this as
> expressed in an expunge file which can be used as input to running
> fstests so that tests are skipped for the release.
> 
> Annotations can be made with comments, you can see an existin list here:
> 
> https://github.com/linux-kdevops/kdevops/tree/master/workflows/fstests/expunges/
> 
> I currently track *.bad and *.dmesg outputs into gists and refer to them
> with a URL. Likewise when possible I annotate the failure rate.
> 
> *If* it makes sense to collaborate on that front I can extract *just*
> the expunges directory and make its own git subtree which then kdevops
> uses. Other test runner can then use the same git tree as a git subtree.

Personally, I don't think I would have much use for a git subtree. I have
been using expunges very sparingly - only for tests which cause crashes -
as I like to run even the failing tests to keep tabs on the failure rates.

> 
>   Luis

Best,
Leah

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 5.15 00/15] xfs stable candidate patches for 5.15.y
  2022-06-08  7:56     ` Amir Goldstein
@ 2022-06-08 22:24       ` Leah Rumancik
  0 siblings, 0 replies; 9+ messages in thread
From: Leah Rumancik @ 2022-06-08 22:24 UTC (permalink / raw)
  To: Amir Goldstein
  Cc: linux-xfs, Darrick J. Wong, Luis R. Rodriguez, Theodore Tso

On Wed, Jun 08, 2022 at 10:56:17AM +0300, Amir Goldstein wrote:
> On Mon, Jun 6, 2022 at 8:42 PM Leah Rumancik <leah.rumancik@gmail.com> wrote:
> >
> > On Sat, Jun 04, 2022 at 11:38:35AM +0300, Amir Goldstein wrote:
> > > On Sat, Jun 4, 2022 at 6:53 AM Leah Rumancik <leah.rumancik@gmail.com> wrote:
> > > >
> > > > From: Leah Rumancik <lrumancik@google.com>
> > > >
> 
> FWIW, the following subset of your 5.15 patches (or backported version thereof)
> have been sitting in my xfs-5.10.y-8 tag since Saturday and have been
> spinning in kdevops since (~20 auto runs) with no regressions observed
> from v5.10.y baseline:
> 
> xfs: punch out data fork delalloc blocks on COW writeback failure
> xfs: remove all COW fork extents when remounting readonly
> xfs: prevent UAF in xfs_log_item_in_current_chkpt
> xfs: only bother with sync_filesystem during readonly remount
> xfs: use setattr_copy to set vfs inode attributes
> xfs: check sb_meta_uuid for dabuf buffer recovery
> xfs: use kmem_cache_free() for kmem_cache objects
> xfs: Fix the free logic of state in xfs_attr_node_hasname
> 
> So perhaps you could use that as the smaller subset for first posting.
> To reduce review burden on xfs maintainers, I could break out of the
> chronological patches order and use the same subset for my next set
> of candidates for 5.10 after testing them in isolation on top of xfs-5.10.y-3
> (at least the ones that apply out of order).
> 
> Thanks,
> Amir.

Sure, good idea! I was going to split it into a smaller batch anyways,
so this selection works for me.

Best,
Leah

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2022-06-08 22:24 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-06-03 18:46 [PATCH 5.15 00/15] xfs stable candidate patches for 5.15.y Leah Rumancik
2022-06-04  8:38 ` Amir Goldstein
2022-06-06 15:55   ` Luis Chamberlain
2022-06-06 18:57     ` Leah Rumancik
2022-06-06 19:30       ` Luis Chamberlain
2022-06-08 22:16         ` Leah Rumancik
2022-06-06 17:42   ` Leah Rumancik
2022-06-08  7:56     ` Amir Goldstein
2022-06-08 22:24       ` Leah Rumancik

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox