Re: linus-next: improving functional testing for to-be-merged pull requests

Paul E. McKenney
2024-10-21T23:39:01Z

On Mon, Oct 21, 2024 at 12:07:13PM -0400, Sasha Levin wrote:
> Hi folks,
>
> The linux-next tree we all know and love is widely used by the kernel
> community for integration work. It offers several advantages:
>
> 1. Early detection of conflicts between matinainer trees
>
> 2. Catching most new build errors/warnings
>
> However, it faces significant testing challenges:
>
> 1. Contains a mix of "ready-to-go" code and experimental additions
>
> 2. A single "bad" piece of code can affect testing of everything else
>
> 3. Low barrier of entry, encouraging inclusion over exclusion
>
> 4. While linux-next offers early conflict resolution and
> identifies build issues, it is very difficult to actually test
> due to the abundance of runtime issues it tends to have
>
> These factors combine to make linux-next a valuable tool for integration
> but problematic for comprehensive testing.
>
> During the Maintainer's Summit, Linus Torvalds expressed concerns about
> the quality of testing that code receives before he pulls it. The
> subsequent discussion side-tracked to the testability of linux-next, but
> we didn't directly address Linus's original concern about pre-pull
> testing quality.

I have to ask...

Wouldn't more people testing -next result in more pressure to fix
linux-next problems quickly? Or perhaps more pressure for people to
avoid linux-next? But this later would also apply to a new linus-next.
Unless Linus were to start rejecting pull requests that had not been
in linu[sx]-next for "long enough", whatever that might be. ;-)

Thanx, Paul

> In an attempt to address the concerns, we're trying out a new "linus-next"
> tree is being created and maintained with the following characteristics:
>
> 1. Composed of pull requests sent directly to Linus
>
> 2. Contains branches destined for imminent inclusion by Linus
>
> 3. Higher code quality expectation (these are pull requests that
> maintainers expect Linus to pull)
>
> 4. Continuous tree (not daily tags like in linux-next),
> facilitating easier bisection
>
> The linus-next tree aims to provide a more stable and testable
> integration point compared to linux-next, addressing the runtime issues
> that make testing linux-next challenging and focusing on code that's
> about to be pulled by Linus.
>
> linus-next is (expected to be) particularly effective before the merge
> window opens, as maintainers tend to send their pull requests early,
> allowing for more thorough testing of to-be-merged changes.
>
> We also want to avoid altering the existing workflow. In particular:
>
> 1. No increase in latency. If anything, the expectation is that
> the cadence of merges would be improved given that Linus will
> need to do less builds and tests.
>
> 2. Require "sign up" for the tree like linux-next does. Instead,
> pull requests are monitored and grabbed directly from the
> mailing list.
>
> Tree location: `git://git.kernel.org/pub/scm/linux/kernel/git/sashal/linus-next.git linus-next`
>
> Current testing:
> - LKFT: https://qa-reports.linaro.org/lkft/sashal-linus-next/
> - KernelCI: https://t.ly/KEW7F
>
> Feedback and suggestions for improving usability are welcome!
>
> --
> Thanks,
> Sasha
>