[ANN] Discord server for testing Linux with kdevops

[ANN] Discord server for testing Linux with kdevops

[Luis Chamberlain]

I've setup a discord server for general discussions around Linux
kernel testing with kdevops. This should help with coordination
around kdevops in an accessible way for:

  * The shared kdevops repository and dependent trees on the linux-kdevops
    organization: https://github.com/linux-kdevops/
  * Sharing of expunges for fstests / blktests for different
    filesystems / configuration / kernel releases
  * Shared hardware resources such as the public Super Micro bigtwin server
    currently used to help test fstests and blktests
  * Future potential shared cloud credits
  * Streamlining reports for new issues found on stable kernels or
    Linus's tree or linux-next
  * Storing / sharing test failure artifacts

The discord server:

https://discord.gg/pWgZZhRp

  Luis

Collaborating updates of kdevops xfstests repo

[Amir Goldstein]

Hi Luis,

I have been on vacation and partly still on vacation.
Quick question.

Soon I will want to update the shared xfstetsts repo - it has not been
updated since v2022.07.10
I am still waiting on the SGID tests [1] to land, which may take a few more
weeks, because I have a bunch of 5.10 backports related to SGID fixes
that I would like to validate with those tests.

My question is, at the moment, who are the active kdevops users that you
know of that will be affected by an update like that?

Updating the kdevops clone of xfstests repo is going to have an immediate
side effect of "destabilizing" all the established baselines.

I will announce it on fstests lists, but wanted to give a heads up
and collaborate the update with the active kdevops users first.

Thanks,
Amir.

[1] https://lore.kernel.org/fstests/bc3f8e56-d5fa-bee2-741f-d2950ca6e304@fujitsu.com/

Re: Collaborating updates of kdevops xfstests repo

[Luis Chamberlain]

On Thu, Jul 21, 2022 at 11:15:04AM +0200, Amir Goldstein wrote:
> Hi Luis,
> 
> I have been on vacation and partly still on vacation.
> Quick question.
> 
> Soon I will want to update the shared xfstetsts repo - it has not been
> updated since v2022.07.10

That's slightly over a month.

> I am still waiting on the SGID tests [1] to land, which may take a few more
> weeks, because I have a bunch of 5.10 backports related to SGID fixes
> that I would like to validate with those tests.

Neat, good to know.

> My question is, at the moment, who are the active kdevops users that you
> know of that will be affected by an update like that?

SUSE has its own git subtree to manage their own delta and set of
changes, so typically updates don't affect that tree. But just in
case you can Cc Anthony.

I know Pankaj also uses it for his own ZNS development. And then there
are those that I am not sure of, but it's all good, I think its fair we
just set users to expect us to update maybe once or twice a month. Typically
I've found there may be small snafu bugs in one release and have to
update soon to fix those issues, or I just fix it myself and carry this
on our own tree until those fixes land upstream. This is why we have our
tree for it, so to give us the flexibility to ensure we can reach
stability.

If you look fstests has no release tags. Long term I think it would be
wonderful if we strive for that, perhaps in stride / linking with the
latest kernel release rc tag at that time, but what would a release
tag mean? Obviously some sort of stability, but how can we vet for this?

[0] https://github.com/linux-kdevops/fstests

> Updating the kdevops clone of xfstests repo is going to have an immediate
> side effect of "destabilizing" all the established baselines.

Sure.

> I will announce it on fstests lists, but wanted to give a heads up
> and collaborate the update with the active kdevops users first.

You can also just use the discord server.

  Luis

> Thanks,
> Amir.
> 
> [1] https://lore.kernel.org/fstests/bc3f8e56-d5fa-bee2-741f-d2950ca6e304@fujitsu.com/

Re: Collaborating updates of kdevops xfstests repo

[Theodore Ts'o]

On Thu, Jul 21, 2022 at 07:46:08AM -0700, Luis Chamberlain wrote:
> If you look fstests has no release tags. Long term I think it would be
> wonderful if we strive for that...

Since May, Zorro has started providing release tags --- it's tagged
when he pushes a batch to for-next, and so far there hasn't been any
changes between when a batch of updates get promoted from for-next to
the master branch.

% git tag -l | grep ^v2022
v2022.05.01
v2022.05.08
v2022.05.15
v2022.05.22
v2022.05.29
v2022.06.05
v2022.06.12
v2022.06.26
v2022.07.03

So if you want stability, you can wait until after the tag has been
promoted to the master branch.  I've using the for-next branch for
xfstests-bld, since I'm also doing development work for xfstests, and
before I release an prebuilt image for kvm-xfstests or gce-xfstests,
I've done a full sanity check run on at least ext4, xfs, btrfs, and
f2fs.  And aside from a relatively rare hiccups, I've found for-next
to be plenty stable enough for me.

Of course, it's a bit easier for me, since I've explicitly *not* been
promising that a reliable set of baseline configs that can be used for
drive-by testers.  OTOH, last I checked, kdevops hasn't published new
baselines since 5.17, which IMHO is way too old for drive-by testers
who need to test patches meant for upstream submission anyway...

Cheers,

					 -Ted

Re: Collaborating updates of kdevops xfstests repo

[Amir Goldstein]

On Thu, Jul 21, 2022 at 4:46 PM Luis Chamberlain <mcgrof@kernel.org> wrote:
>
> On Thu, Jul 21, 2022 at 11:15:04AM +0200, Amir Goldstein wrote:
> > Hi Luis,
> >
> > I have been on vacation and partly still on vacation.
> > Quick question.
> >
> > Soon I will want to update the shared xfstetsts repo - it has not been
> > updated since v2022.07.10
>
> That's slightly over a month.

Correction. It has not been updated since LSFMM on April 2022.
So far it didn't matter much for my tests of 5.10.y, because I was
backporting fixes from before v5.17 that already has fstests in April.

I will push an update before testing the next round of xfs 5.10.y backports.

Thanks,
Amir.

Re: Collaborating updates of kdevops xfstests repo

[Luis Chamberlain]

On Thu, Jul 21, 2022 at 10:59:48AM -0400, Theodore Ts'o wrote:
> On Thu, Jul 21, 2022 at 07:46:08AM -0700, Luis Chamberlain wrote:
> > If you look fstests has no release tags. Long term I think it would be
> > wonderful if we strive for that...
> 
> Since May, Zorro has started providing release tags --- it's tagged
> when he pushes a batch to for-next, and so far there hasn't been any
> changes between when a batch of updates get promoted from for-next to
> the master branch.
> 
> % git tag -l | grep ^v2022
> v2022.05.01
> v2022.05.08
> v2022.05.15
> v2022.05.22
> v2022.05.29
> v2022.06.05
> v2022.06.12
> v2022.06.26
> v2022.07.03

Fantastico.

> So if you want stability, you can wait until after the tag has been
> promoted to the master branch.  I've using the for-next branch for
> xfstests-bld, since I'm also doing development work for xfstests, and
> before I release an prebuilt image for kvm-xfstests or gce-xfstests,
> I've done a full sanity check run on at least ext4, xfs, btrfs, and
> f2fs.  And aside from a relatively rare hiccups, I've found for-next
> to be plenty stable enough for me.

This still begs the question of what can we count on for a tag, or if
want more.

> Of course, it's a bit easier for me, since I've explicitly *not* been
> promising that a reliable set of baseline configs that can be used for
> drive-by testers.

Git subtrees uses of kdevops can keep their own deltas / own fstests git
trees, etc. It was designed with that in mind.

> OTOH, last I checked, kdevops hasn't published new
> baselines since 5.17, which IMHO is way too old for drive-by testers
> who need to test patches meant for upstream submission anyway...

This is because of building confidence in a baseline on kdevops is done
first by striving for 100 consecutive runs of fstest and/or blktests
without failure and we lacked one for more recent kernels for a while.
Building confidence takes a while, but I consider it technical debt.

With a baseline with high confidence, it easy to bump one kernel
release, as the effort typically is just verifying prior known failures
(the status quo goes on) and picking up new ones. There would be high
confidence that these new failures are also regressions.

  Luis