[PATCH net-next] docs: netdev: add a note on selftest posting

[PATCH net-next] docs: netdev: add a note on selftest posting

[Jakub Kicinski]

We haven't had much discussion on the list about this, but
a handful of people have been confused about rules on
posting selftests for fixes, lately. I tend to post fixes
with their respective selftests in the same series.
There are tradeoffs around size of the net tree and conflicts
but so far it hasn't been a major issue.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
 Documentation/process/maintainer-netdev.rst | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/Documentation/process/maintainer-netdev.rst b/Documentation/process/maintainer-netdev.rst
index e497729525d5..1ac62dc3a66f 100644
--- a/Documentation/process/maintainer-netdev.rst
+++ b/Documentation/process/maintainer-netdev.rst
@@ -311,6 +311,14 @@ user space patches should form separate series (threads) when posted
 Posting as one thread is discouraged because it confuses patchwork
 (as of patchwork 2.2.2).
 
+Co-posting selftests
+--------------------
+
+Selftests should be part of the same series as the code changes.
+Specifically for fixes both code change and related test should go into
+the same tree (the tests may lack a Fixes tag, which is expected).
+Mixing code changes and test changes in a single commit is discouraged.
+
 Preparing changes
 -----------------
 
-- 
2.48.1

Re: [PATCH net-next] docs: netdev: add a note on selftest posting

[Matthieu Baerts]

Hi Jakub,

On 06/03/2025 19:05, Jakub Kicinski wrote:
> We haven't had much discussion on the list about this, but
> a handful of people have been confused about rules on
> posting selftests for fixes, lately. I tend to post fixes
> with their respective selftests in the same series.
> There are tradeoffs around size of the net tree and conflicts
> but so far it hasn't been a major issue.

Good idea to clarify this point!

> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> ---
>  Documentation/process/maintainer-netdev.rst | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/Documentation/process/maintainer-netdev.rst b/Documentation/process/maintainer-netdev.rst
> index e497729525d5..1ac62dc3a66f 100644
> --- a/Documentation/process/maintainer-netdev.rst
> +++ b/Documentation/process/maintainer-netdev.rst
> @@ -311,6 +311,14 @@ user space patches should form separate series (threads) when posted
>  Posting as one thread is discouraged because it confuses patchwork
>  (as of patchwork 2.2.2).
>  
> +Co-posting selftests
> +--------------------
> +
> +Selftests should be part of the same series as the code changes.
> +Specifically for fixes both code change and related test should go into
> +the same tree (the tests may lack a Fixes tag, which is expected).

Regarding the Fixes tag in the tests, could we eventually suggest using
the same one as for the code change?

Sometimes, I do that to get the corresponding test backported as well,
if there are no conflicts. That's good to have an easy way to check if
something has been correctly fixed on stable versions as well.

The only thing is with the selftests written in Python or Bash: it is
easy to get a situation where there are no conflicts, but the
modification doesn't work, e.g. some functions or variables are not
available, etc. The stable team will then not notice that during their
build tests. Not sure if my suggestion is safe to recommend then.

Anyway, good to recommend attaching tests in the net tree!

> +Mixing code changes and test changes in a single commit is discouraged.
> +
>  Preparing changes
>  -----------------
>  
Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.

Re: [PATCH net-next] docs: netdev: add a note on selftest posting

[Jakub Kicinski]

On Thu, 6 Mar 2025 19:22:49 +0100 Matthieu Baerts wrote:
> > +Co-posting selftests
> > +--------------------
> > +
> > +Selftests should be part of the same series as the code changes.
> > +Specifically for fixes both code change and related test should go into
> > +the same tree (the tests may lack a Fixes tag, which is expected).  
> 
> Regarding the Fixes tag in the tests, could we eventually suggest using
> the same one as for the code change?
> 
> Sometimes, I do that to get the corresponding test backported as well,
> if there are no conflicts. That's good to have an easy way to check if
> something has been correctly fixed on stable versions as well.

Hm, that's probably up to the stable team to decide. My intuition
is to reserve Fixes tags for fixes, and add another tag if necessary.
The mention of the Fixes tag was primarily because of NIPA checks...
A bit of a wink and a nod since we try not to speak about NIPA checks.

> The only thing is with the selftests written in Python or Bash: it is
> easy to get a situation where there are no conflicts, but the
> modification doesn't work, e.g. some functions or variables are not
> available, etc. The stable team will then not notice that during their
> build tests. Not sure if my suggestion is safe to recommend then.

Good point..

Re: [PATCH net-next] docs: netdev: add a note on selftest posting

[Aaron Conole]

Jakub Kicinski <kuba@kernel.org> writes:

> On Thu, 6 Mar 2025 19:22:49 +0100 Matthieu Baerts wrote:
>> > +Co-posting selftests
>> > +--------------------
>> > +
>> > +Selftests should be part of the same series as the code changes.
>> > +Specifically for fixes both code change and related test should go into
>> > +the same tree (the tests may lack a Fixes tag, which is expected).  
>> 
>> Regarding the Fixes tag in the tests, could we eventually suggest using
>> the same one as for the code change?
>> 
>> Sometimes, I do that to get the corresponding test backported as well,
>> if there are no conflicts. That's good to have an easy way to check if
>> something has been correctly fixed on stable versions as well.
>
> Hm, that's probably up to the stable team to decide. My intuition
> is to reserve Fixes tags for fixes, and add another tag if necessary.

+1

You could consider something like "Tests: xxx", but the problem is that
not every fix will perfectly map to a test, and as noted below, it is
possible that the fixes don't merge cleanly.  I don't know if there
would be a good tag that makes sense, really.

> The mention of the Fixes tag was primarily because of NIPA checks...
> A bit of a wink and a nod since we try not to speak about NIPA checks.
>
>> The only thing is with the selftests written in Python or Bash: it is
>> easy to get a situation where there are no conflicts, but the
>> modification doesn't work, e.g. some functions or variables are not
>> available, etc. The stable team will then not notice that during their
>> build tests. Not sure if my suggestion is safe to recommend then.

I also would note that I like the approach where the test cases detect
if the condition is even possible and [SKIP] if it isn't.  The reason is
that we can then run the latest and greatest copies of the selftest
suite against even older kernels.  Reality, it is not always possible to
do that either - so it probably needs to be case-by-case basis anyway.

> Good point..

Re: [PATCH net-next] docs: netdev: add a note on selftest posting

[Cong Wang]

On Thu, Mar 06, 2025 at 10:05:33AM -0800, Jakub Kicinski wrote:
> We haven't had much discussion on the list about this, but
> a handful of people have been confused about rules on
> posting selftests for fixes, lately. I tend to post fixes
> with their respective selftests in the same series.
> There are tradeoffs around size of the net tree and conflicts
> but so far it hasn't been a major issue.
> 
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>

Acked-by: Cong Wang <xiyou.wangcong@gmail.com>

Thanks for the effort!

Re: [PATCH net-next] docs: netdev: add a note on selftest posting

[Kuniyuki Iwashima]

From: Jakub Kicinski <kuba@kernel.org>
Date: Thu,  6 Mar 2025 10:05:33 -0800
> We haven't had much discussion on the list about this, but
> a handful of people have been confused about rules on
> posting selftests for fixes, lately. I tend to post fixes
> with their respective selftests in the same series.
> There are tradeoffs around size of the net tree and conflicts
> but so far it hasn't been a major issue.
> 
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> ---
>  Documentation/process/maintainer-netdev.rst | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/Documentation/process/maintainer-netdev.rst b/Documentation/process/maintainer-netdev.rst
> index e497729525d5..1ac62dc3a66f 100644
> --- a/Documentation/process/maintainer-netdev.rst
> +++ b/Documentation/process/maintainer-netdev.rst
> @@ -311,6 +311,14 @@ user space patches should form separate series (threads) when posted
>  Posting as one thread is discouraged because it confuses patchwork
>  (as of patchwork 2.2.2).
>  
> +Co-posting selftests
> +--------------------
> +
> +Selftests should be part of the same series as the code changes.
> +Specifically for fixes both code change and related test should go into
> +the same tree (the tests may lack a Fixes tag, which is expected).
> +Mixing code changes and test changes in a single commit is discouraged.

I guess an exception for the mixing is when a code change breaks a
selftest, or is it fine for NIPA ?  (still other CI may complain though)

Re: [PATCH net-next] docs: netdev: add a note on selftest posting

[Jakub Kicinski]

On Thu, 6 Mar 2025 16:41:41 -0800 Kuniyuki Iwashima wrote:
> > +Selftests should be part of the same series as the code changes.
> > +Specifically for fixes both code change and related test should go into
> > +the same tree (the tests may lack a Fixes tag, which is expected).
> > +Mixing code changes and test changes in a single commit is discouraged.  
> 
> I guess an exception for the mixing is when a code change breaks a
> selftest, or is it fine for NIPA ?  (still other CI may complain though)

If it breaks compilation yes, but that should almost never happen?
Functionality-wise it's fine, we don't expect patch-by-patch
compatibility of selftests. At least I don't recall it coming up
in discussions before.

Re: [PATCH net-next] docs: netdev: add a note on selftest posting

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to netdev/net-next.git (main)
by Paolo Abeni <pabeni@redhat.com>:

On Thu,  6 Mar 2025 10:05:33 -0800 you wrote:
> We haven't had much discussion on the list about this, but
> a handful of people have been confused about rules on
> posting selftests for fixes, lately. I tend to post fixes
> with their respective selftests in the same series.
> There are tradeoffs around size of the net tree and conflicts
> but so far it hasn't been a major issue.
> 
> [...]

Here is the summary with links:
  - [net-next] docs: netdev: add a note on selftest posting
    https://git.kernel.org/netdev/net-next/c/0ea09cbf8350

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html