Re: Stop false review statements

[Krzysztof Kozlowski <krzk@kernel.org>]

On 16/05/2026 15:24, Laurent Pinchart wrote:
> On Sat, May 16, 2026 at 02:29:15PM +0200, Krzysztof Kozlowski wrote:
>> On 16/05/2026 14:23, Guenter Roeck wrote:
>>> On 5/16/26 05:16, Krzysztof Kozlowski wrote:
>>>> On 16/05/2026 14:11, Guenter Roeck wrote:
>>>>> On Sat, May 16, 2026 at 10:05:02AM +0200, Krzysztof Kozlowski wrote:
>>>>>> What the hell is that:
>>>>>>
>>>>>> https://lore.kernel.org/all/20260515190707.033BDC2BCB0@smtp.kernel.org/
>>>>>>
>>>>>> As a bot you CANNOT MAKE a Reviewer's statement of oversight. You are
>>>>>> not a damn human do be able to make such statement. You are a bot, a tool.
>>>>>>
>>>>>
>>>>> Where exactly do the rules say that ? I seem to miss that.
>>>>>
>>>>> There is a policy document about _contributions_ made by AI, but I don't
>>>>> see the one that says that AI agents must not provide Reviewed-by: tags.
>>>>
>>>> Quotes from the existing policy:
>>>>
>>>> 1. "By offering my Reviewed-by: tag, I state that:"
>>>>
>>>> Tool cannot use first person "I". Tool cannot "state that".
>>>>
>>>> 2. "A Reviewed-by tag is *a statement of opinion* that the patch is an
>>>>   appropriate modification of the kernel without any remaining serious"
>>>>
>>>> Tool cannot make a statement of opinion.
>>>>
>>>> 3. "Any interested reviewer (who has done the work) can offer a
>>>> Reviewed-by".
>>>>
>>>> Tool is not a reviewer as a person, thus above does not grant the tool
>>>> permission to offer a tag.
>>>
>>> I'd like to see that explicitly spelled out. Until then it is your opinion.
>>
>> It is not an opinion. It is written. I gave you quotes.
>>
>> Do you want to spell the rules of English language? That tool is not a
>> person?
>>
>> Shall I send the patch like:
>>
>>   Any interested reviewer (who has done the work) can offer a
>>   Reviewed-by.
>>  +In English "reviewer" is a person [1].
>>  + [1] https://en.wiktionary.org/wiki/reviewer
>>
>> Seriously, you expect to document the English language?
>>
>>>>>> Stop faking tags.
>>>>>>
>>>>>> And really, considering how many false positives Sashiko produces, how
>>>>>> poor review comments it gives, how many misleading comments, it's
>>>>>> unacceptable to me to consider that a review.
>>>>>>
>>>>>> Amount of useless noise Sashiko produces already changed my mind how
>>>>>> useful that tool is.
> 
> Note this isn't en entirely new situation. As a maintainer, you know how
> much you trust each reviewer. You will consider some R-b tags as a sign
> you don't even have to look at a patch, and will completely ignore some
> others. There's a whole continuum in the middle. In some ways, reviews
> by an LLM are similar. You will trust them or not trust them.
> 
> Except they're also very different.
> 
> The kernel needs more skilled reviewers (I don't think this is a
> controversial statement). We can't expect all newcomers to start with
> extensive experience from day one, so there's a learning curve. I
> believe it's fine for more junior reviewers to send R-b tags even if
> they miss some issue, as long as they genuinely try and improve (and, in
> some unfortunate cases, decide to leave if patch review turns out not to
> be for them). Those R-b tags may feel like a bit of noise in the
> beginning, but that's compensated by their value increasing over time.

Yes, I agree. Reviews from inexperienced people are sometimes fruitless
or pointless per actual value they bring, but they allow a person
(again: person) to grow in the community with a credits being the reward.

> 
> Bot reviews are not the same. Not only are they generated at a much
> larger scale than human reviews, they also won't learn from feedback you
> give them. Sure, the tools may be improved when cases of false positives
> are identified, and new LLMs may be trained with more (and better ?)
> data to improve the output, but they won't learn from the interactions.
> 
> How much value a maintainer sees in those reviews is up to individual
> maintainers. I will personally not consider a R-b tag from an LLM to
> mean that a patch is ready to be merged (and I believe you won't
> either). As such, I think that a R-b from an LLM is misleading and
> doesn't provide good value. At best it's free advertising for company
> making closed-source tools, which I don't think we should encourage.

That's different aspect than I raised. I agree with above approach but
it is more subjective.

What I brought is object: our docs clearly state that reviewer can offer
reviewed-by tag. They do not allow non-reviewers to offer a tag and
English is clear on that - only a person is a reviewer.

Dog is not a reviewer.

Hammer is not a reviewer.

Tool is not a reviewer.

Guenter did not bring any counter arguments that our docs ALLOW
non-person to provide a reviewed-by tag. I brought that arguments as
excerpt from our documented policy.

Best regards,
Krzysztof