Re: [Ksummit-discuss] bug-introducing patches

[Sasha Levin <Alexander.Levin@microsoft.com>]

On Wed, May 02, 2018 at 08:06:20PM -0400, Theodore Y. Ts'o wrote:
>On Wed, May 02, 2018 at 10:41:56PM +0200, Geert Uytterhoeven wrote:
>>
>> Between v4.17-rc1 and v4.17-rc3, there are 660 non-merge commits, of which
>>   - 245 carry a Fixes tag,
>>   - 196 carry a CC stable,
>>   - 395 contain the string "fix".
>> (non-mutually exclusive)
>>
>> That leaves us with 200 commits not falling in the bugfix category.
>
>Some non-bug fixes are allowed in -rc2.  So perhaps what might be
>interesting is to look at v4.16 (which is completed), and look at the
>distribution of commits:
>
>	* regressions fixes (for bugs introduced during the current
>		release cycle)
>	* "normal" bug fixes
>	* commits which don't touch code (e.g., spelling or
>		documentation-only fixes)
>	* other commits (features or cleanup fixes)
>
>at each rcX level.  The historic "standard" has been feature commits
>in -rc1 and -rc2 (tolerated, but ideally should before the merge
>window), bug fixes / regressions in -rc3 and -rc4, and after -rc4,
>regression fixes only.  It would be interesting to see how well we
>have been holding to the historical ideal.
>
>It would then be intersting to use Sasha's analysis to see whether
>there are more bug fixes caused by regression fixes versus normal bug
>fixes, and whether or not they are common when fixes come "out of
>cycle" --- for example, a non-regression bug fix in -rc5 or -rc6.
>
>Because if that last is the case, then the prescription is very simple
>and not controversial --- bug fixes found post -rc4 should be held to
>the next merge window.
>
>If the concern is regression fixes which require one or two tries
>before they are fixed before 4.16-FINAL is released, then that's a
>"life is hard for AUTOSEL" issue, and I suspect Sasha will find that
>there is rather less sympathy for holding regression fixes for an
>extra week or two.
>
>If the concern is bug fixes that show up in -rc3 and -rc4, but which
>aren't hitting linux-next first, then holding bug fixes in linux-next
>for a week makes sense, and if that means that a bug fix found post
>-rc3 needs to marinate in linux-next for a week, and then it then
>misses the -rc4 "bug fix" deadline, we can have a discussion about
>whether bug fixes should be allowed in -rc5 after a week's marination.
>
>My personal opinion is "to hell with it, just wait until the next
>merge window" --- but this can cause more work for the stable
>maintainers since a lot of bug fixes would then land in -rc1.

I'll work on breaking up the 4.16 commits into categories, but one
interesting statistic I've noticed while starting the work is:

Fixes in -rc cycles:
rc1 68
rc2 147
rc3 88
rc4 121
rc5 40
rc6 193
rc7 98

Average days in -next, for a fix, per -rc cycle:
rc1 27.25
rc2 21.4286
rc3 22.5114
rc4 18.281
rc5 14.65
rc6 12.6166
rc7 8.70408

Fixes for bugs not introduced in current merge window:
rc1 40
rc2 113
rc3 61
rc4 79
rc5 25
rc6 139
rc7 72

So for some reason, there is a rush to push fixes for older bugs (that
were not introduced in the current merge window) to the point that rc7
commits that only existed for a few days are merged in to address older
bugs.