Re: linus-next: improving functional testing for to-be-merged pull requests

Thorsten Leemhuis
2024-10-29T08:10:28Z

On 28.10.24 23:46, Sasha Levin wrote:
> On Mon, Oct 21, 2024 at 11:48:34PM -0700, Christoph Hellwig wrote:
>> On Mon, Oct 21, 2024 at 09:54:53PM -0700, Kees Cook wrote:
>>> For example, for a given PR, the bot can report:
>>>
>>> - Were the patches CCed to a mailing list?
>>> - A histogram of how long the patches were in next (to show bake times)
>>> - Are any patches associated with test failures? (0day and many other
>>> CIs are already running tests against -next; parse those reports)
>>>
>>> We could have a real pre-submit checker! :)
>>
>> That would be very useful.  Items 1 and 2 should be trivial, 3 would
>> require a bit of work but would still be very useful.
>
> If you've been following so far, there is a bot that is capable of doing
> most of the above
> (https://git.kernel.org/pub/scm/linux/kernel/git/sashal/next-
> analysis.git/).
>
> Here's a histogram that describes v6.12-rc4..v6.12-rc5 as far as how
> long commits spent in -next:

I took a quick look at that tree and histo.sh that lead to a few
questions here the code had no obvious answers to (or I missed them due
to the "quick" aspect):

* How does histo.sh handle changes where the commit-id changed between
the first time in -next and their merge into Linus' tree (while the
patch itself did not change)? For example due to a rebase or workflows
where the commit-id changes regularly, such as those used by the
bluetooth tree (for -fixes, as it queues them in their -next branch
first) or the -mm tree (for most of it iirc -- this made things hard in
a script of mine that looks up the arrival in -next)?

* Do those lore scripts detect if a committer adjusted the subject of a
patch that has been on lore?

* How do the scripts handle patches that changed a lot while they were
in -next? I know of one subsystem that regularly drops whole patch-sets
from their trees included in -next to replace them with newer versions
of said patch-sets -- and then the timer maybe should restart.

> This is where I think the value of linus-next comes during the -rc
> cycles: the (89 + 21) commits that haven't gone through the -next
> workflow before being pulled.
>
> I'm not looking to delay the process and
> add latency, I'm looking to plug a hole where code would flow directly
> to Linus's tree bypassing -next.

Overall after all the discussions in this thread I still fail to see why
we need a new tree for that. Why not make pending-fixes a bit more
prominent while motivating maintainers to have proper -fixes branches
included there?

Ciao, Thorsten