* Re: Linking Patchwork with Sashiko?
[not found] ` <20260715005909.GF1656185@killaraus.ideasonboard.com>
@ 2026-07-15 16:28 ` Ihor Solodrai
2026-07-15 16:39 ` Laurent Pinchart
0 siblings, 1 reply; 12+ messages in thread
From: Ihor Solodrai @ 2026-07-15 16:28 UTC (permalink / raw)
To: Laurent Pinchart, Roman Gushchin
Cc: Mauro Carvalho Chehab, Derek Barbosa, Matthieu Baerts,
Konstantin Ryabitsev, Jason Gunthorpe, Steven Rostedt, users,
Linux Media Mailing List, Stephen Finucane, bpf, Chris Mason,
Christian Brauner, Alexei Starovoitov
On 7/14/26 5:59 PM, Laurent Pinchart wrote:
> [...]
>
> I expect maintainers who want to act on sashiko reviews to triage and
> verify them first before bothering authors, yes.
I can't help but chime in, because I find this opinion triggeringly
unreasonable.
As of today, the problem is *not* "thoughtful experts are bombarded
with low-quality feedback from llms", even though the experts might
feel like that frequently.
The problem is that the mailing list is flooded with slop to varying
degree depending on the subsystem. And review by humans, even with
powerful AI tools at hand, doesn't scale [1][2].
And the argument "I didn't sign up for AI reviews" is weak. If you've
sent a patch to the mailing list, and there are maintainer-sanctioned
bot messages in response, you've already opted-in to receive them.
Yes, it is annoying. It's annoying in the same way a dozen nits on
your patch from a human reviewer is annoying. But the contributors
generally accept this, otherwise their patches don't land.
Your job as a contributor is to make sure the patch is "good enough"
to land. Now, with ubiquitous AI tools, "good enough" means *at least*
that the AI bots don't find real bugs.
Yes, assessing whether AI report is "real" is work, but it's not
reasonable to push this work on already overwhelmed maintainers. And
if AI is wrong, again it's the job of the author to convince the
maintainers it's wrong.
I set up automated AI code reviews on BPF list using Chris Mason's
prompts in the fall last year [3], before sashiko was developed. And
originally we were very worried about the quality of the reviews.
Time has showed that we had to be more worried about the average
quality of incoming patch instead.
[1] https://lore.kernel.org/bpf/CAADnVQKsM8iOGO3ZL3LgbWigOBBMrGbTYZ_k_Ktz=+cVkvRLXg@mail.gmail.com/
[2] https://lore.kernel.org/all/20260703-vfs-summer-jam-22b2edbbbc44@brauner/
[3] https://lore.kernel.org/all/0d6aa077-1222-49a0-9554-dd922122a904@meta.com/
> I believe we should
> follow the first two recommendations of the Software Freedom Conservancy
> on using LLM-backed generative AI systems for FOSS contributions ([1]).
>
> [1] https://sfconservancy.org/llm-gen-ai/llm-backed-generative-ai-recommendations.html
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Linking Patchwork with Sashiko?
2026-07-15 16:28 ` Linking Patchwork with Sashiko? Ihor Solodrai
@ 2026-07-15 16:39 ` Laurent Pinchart
2026-07-15 17:12 ` Ihor Solodrai
2026-07-15 17:41 ` Jason Gunthorpe
0 siblings, 2 replies; 12+ messages in thread
From: Laurent Pinchart @ 2026-07-15 16:39 UTC (permalink / raw)
To: Ihor Solodrai
Cc: Roman Gushchin, Mauro Carvalho Chehab, Derek Barbosa,
Matthieu Baerts, Konstantin Ryabitsev, Jason Gunthorpe,
Steven Rostedt, users, Linux Media Mailing List, Stephen Finucane,
bpf, Chris Mason, Christian Brauner, Alexei Starovoitov
On Wed, Jul 15, 2026 at 09:28:03AM -0700, Ihor Solodrai wrote:
> On 7/14/26 5:59 PM, Laurent Pinchart wrote:
> > [...]
> >
> > I expect maintainers who want to act on sashiko reviews to triage and
> > verify them first before bothering authors, yes.
>
> I can't help but chime in, because I find this opinion triggeringly
> unreasonable.
>
> As of today, the problem is *not* "thoughtful experts are bombarded
> with low-quality feedback from llms", even though the experts might
> feel like that frequently.
>
> The problem is that the mailing list is flooded with slop to varying
> degree depending on the subsystem. And review by humans, even with
> powerful AI tools at hand, doesn't scale [1][2].
I certainly agree with you that this is a large problem. I don't think
we have a single problem though, but I can understand people who
consider the problem you're discribing as more urgent to address.
> And the argument "I didn't sign up for AI reviews" is weak. If you've
> sent a patch to the mailing list, and there are maintainer-sanctioned
> bot messages in response, you've already opted-in to receive them.
We're enabling bots on mailing lists that people are already subscribed
to. But I don't think that's a very important argument here.
> Yes, it is annoying. It's annoying in the same way a dozen nits on
> your patch from a human reviewer is annoying.
That I strongly disagree with. A humain reviewer sending a review with
dozens of nits usually doesn't annoy me, and doesn't make me feel
diminished as a human.
> But the contributors
> generally accept this, otherwise their patches don't land.
>
> Your job as a contributor is to make sure the patch is "good enough"
> to land. Now, with ubiquitous AI tools, "good enough" means *at least*
> that the AI bots don't find real bugs.
>
> Yes, assessing whether AI report is "real" is work, but it's not
> reasonable to push this work on already overwhelmed maintainers. And
> if AI is wrong, again it's the job of the author to convince the
> maintainers it's wrong.
We disagree on this as well. This would force contributors to constantly
justify their value against a machine that is known to produce a
non-negligible quantity of nonsense. Furthermore, if those generative AI
tools are as good as their supporters claim, the time they free for
maintainers should outweight the need workload to triage the comments.
> I set up automated AI code reviews on BPF list using Chris Mason's
> prompts in the fall last year [3], before sashiko was developed. And
> originally we were very worried about the quality of the reviews.
>
> Time has showed that we had to be more worried about the average
> quality of incoming patch instead.
>
> [1] https://lore.kernel.org/bpf/CAADnVQKsM8iOGO3ZL3LgbWigOBBMrGbTYZ_k_Ktz=+cVkvRLXg@mail.gmail.com/
> [2] https://lore.kernel.org/all/20260703-vfs-summer-jam-22b2edbbbc44@brauner/
> [3] https://lore.kernel.org/all/0d6aa077-1222-49a0-9554-dd922122a904@meta.com/
>
> > I believe we should
> > follow the first two recommendations of the Software Freedom Conservancy
> > on using LLM-backed generative AI systems for FOSS contributions ([1]).
> >
> > [1] https://sfconservancy.org/llm-gen-ai/llm-backed-generative-ai-recommendations.html
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Linking Patchwork with Sashiko?
2026-07-15 16:39 ` Laurent Pinchart
@ 2026-07-15 17:12 ` Ihor Solodrai
2026-07-15 17:35 ` Mark Brown
2026-07-15 19:06 ` Laurent Pinchart
2026-07-15 17:41 ` Jason Gunthorpe
1 sibling, 2 replies; 12+ messages in thread
From: Ihor Solodrai @ 2026-07-15 17:12 UTC (permalink / raw)
To: Laurent Pinchart
Cc: Roman Gushchin, Mauro Carvalho Chehab, Derek Barbosa,
Matthieu Baerts, Konstantin Ryabitsev, Jason Gunthorpe,
Steven Rostedt, users, Linux Media Mailing List, Stephen Finucane,
bpf, Chris Mason, Christian Brauner, Alexei Starovoitov
On 7/15/26 9:39 AM, Laurent Pinchart wrote:
> On Wed, Jul 15, 2026 at 09:28:03AM -0700, Ihor Solodrai wrote:
>> On 7/14/26 5:59 PM, Laurent Pinchart wrote:
>>> [...]
>>
>> Yes, assessing whether AI report is "real" is work, but it's not
>> reasonable to push this work on already overwhelmed maintainers. And
>> if AI is wrong, again it's the job of the author to convince the
>> maintainers it's wrong.
>
> We disagree on this as well. This would force contributors to constantly
> justify their value against a machine that is known to produce a
> non-negligible quantity of nonsense.
I know what you mean and I can relate. Going through walls of
AI-generated paragraphs is not fun, to put it lightly.
However it seems that you are suggesting to simply delegate this work
to maintainers, and I very much disagree with that approach (even
though I'm not a maintainer myself).
I hope you're not saying "let's just not use AI", because that ship
has sailed a while ago.
And if automated AI reviews are there, someone has to do the
triage. AI can do the triage itself only to a limited degree. For
example, there are instructions in review-prompts where AI checks
whether an issue has already been discussed. But it will never be
perfect.
I think it makes much more sense for the author of the patch to do the
triage, as they have the best context of the work, and they are
usually the most invested in their patch being merged in.
> Furthermore, if those generative AI
> tools are as good as their supporters claim, the time they free for
> maintainers should outweight the need workload to triage the comments.
Even if the AI tools were ~perfect with respect to code review, I
think people would still be signing off patches and taking
responsibility for them.
An imaginary perfect AI reviewer that maintainer can trust to land
code without looking at it would essentially become an AI maintainer.
Independent of whether you think it's possible, it hasn't happened
yet, so we have to deal with imperfect AI in the meanwhile.
> >> I set up automated AI code reviews on BPF list using Chris Mason's
>> prompts in the fall last year [3], before sashiko was developed. And
>> originally we were very worried about the quality of the reviews.
>>
>> Time has showed that we had to be more worried about the average
>> quality of incoming patch instead.
>>
>> [1] https://lore.kernel.org/bpf/CAADnVQKsM8iOGO3ZL3LgbWigOBBMrGbTYZ_k_Ktz=+cVkvRLXg@mail.gmail.com/
>> [2] https://lore.kernel.org/all/20260703-vfs-summer-jam-22b2edbbbc44@brauner/
>> [3] https://lore.kernel.org/all/0d6aa077-1222-49a0-9554-dd922122a904@meta.com/
>>
>>> I believe we should
>>> follow the first two recommendations of the Software Freedom Conservancy
>>> on using LLM-backed generative AI systems for FOSS contributions ([1]).
>>>
>>> [1] https://sfconservancy.org/llm-gen-ai/llm-backed-generative-ai-recommendations.html
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Linking Patchwork with Sashiko?
2026-07-15 17:12 ` Ihor Solodrai
@ 2026-07-15 17:35 ` Mark Brown
2026-07-15 18:39 ` Ihor Solodrai
2026-07-15 19:06 ` Laurent Pinchart
1 sibling, 1 reply; 12+ messages in thread
From: Mark Brown @ 2026-07-15 17:35 UTC (permalink / raw)
To: Ihor Solodrai
Cc: Laurent Pinchart, Roman Gushchin, Mauro Carvalho Chehab,
Derek Barbosa, Matthieu Baerts, Konstantin Ryabitsev,
Jason Gunthorpe, Steven Rostedt, users, Linux Media Mailing List,
Stephen Finucane, bpf, Chris Mason, Christian Brauner,
Alexei Starovoitov
[-- Attachment #1: Type: text/plain, Size: 1650 bytes --]
On Wed, Jul 15, 2026 at 10:12:03AM -0700, Ihor Solodrai wrote:
> On 7/15/26 9:39 AM, Laurent Pinchart wrote:
> > On Wed, Jul 15, 2026 at 09:28:03AM -0700, Ihor Solodrai wrote:
> >> Yes, assessing whether AI report is "real" is work, but it's not
> >> reasonable to push this work on already overwhelmed maintainers. And
> >> if AI is wrong, again it's the job of the author to convince the
> >> maintainers it's wrong.
> > We disagree on this as well. This would force contributors to constantly
> > justify their value against a machine that is known to produce a
> > non-negligible quantity of nonsense.
> I know what you mean and I can relate. Going through walls of
> AI-generated paragraphs is not fun, to put it lightly.
> However it seems that you are suggesting to simply delegate this work
> to maintainers, and I very much disagree with that approach (even
> though I'm not a maintainer myself).
This is something that already happens with human written reviews,
indeed I have a canned reply for one particular reviewer telling
submitters it's OK to ignore them. It goes in both directions - it can
be ignoring some issues or postponing them for later, or it can be
making sure that there's input from particular people.
> I think it makes much more sense for the author of the patch to do the
> triage, as they have the best context of the work, and they are
> usually the most invested in their patch being merged in.
It's not clear cut - untrustworthy tooling often causes problems, I'm
fairly sure I'm seeing issues that are the result of submitters getting
mislead by LLM reviews in a similar manner to some compiler warnings.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Linking Patchwork with Sashiko?
2026-07-15 16:39 ` Laurent Pinchart
2026-07-15 17:12 ` Ihor Solodrai
@ 2026-07-15 17:41 ` Jason Gunthorpe
2026-07-15 19:10 ` Laurent Pinchart
2026-07-15 19:38 ` Dmitry Torokhov
1 sibling, 2 replies; 12+ messages in thread
From: Jason Gunthorpe @ 2026-07-15 17:41 UTC (permalink / raw)
To: Laurent Pinchart
Cc: Ihor Solodrai, Roman Gushchin, Mauro Carvalho Chehab,
Derek Barbosa, Matthieu Baerts, Konstantin Ryabitsev,
Steven Rostedt, users, Linux Media Mailing List, Stephen Finucane,
bpf, Chris Mason, Christian Brauner, Alexei Starovoitov
On Wed, Jul 15, 2026 at 07:39:21PM +0300, Laurent Pinchart wrote:
> > Yes, assessing whether AI report is "real" is work, but it's not
> > reasonable to push this work on already overwhelmed maintainers. And
> > if AI is wrong, again it's the job of the author to convince the
> > maintainers it's wrong.
>
> We disagree on this as well. This would force contributors to constantly
> justify their value against a machine that is known to produce a
> non-negligible quantity of nonsense. Furthermore, if those generative AI
> tools are as good as their supporters claim, the time they free for
> maintainers should outweight the need workload to triage the comments.
I feel Sashiko and this whole AI world has created alot more work. It
is not saving time at all. Now I have to go and read every Sashiko
report to see if there is something important that normal review
missed. I have fix all the drive by 'this is already broken' stuff it
finds. We get way more patches than we did before. Now *everything*
has to be looked at more skeptically if came from some AI slop. I get
crazy real security reports that need special hand holding now that we
never got before.
Even on the submitting side now I have to run patches through all
kinds of AI tools to have a solid chance to navigate the new guantlet
upstream.
We aren't saving time, or creating less work. My hope is we are
creating better quality, higher security and less bugs.
Jason
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Linking Patchwork with Sashiko?
2026-07-15 17:35 ` Mark Brown
@ 2026-07-15 18:39 ` Ihor Solodrai
2026-07-15 20:32 ` Mark Brown
0 siblings, 1 reply; 12+ messages in thread
From: Ihor Solodrai @ 2026-07-15 18:39 UTC (permalink / raw)
To: Mark Brown
Cc: Laurent Pinchart, Roman Gushchin, Mauro Carvalho Chehab,
Derek Barbosa, Matthieu Baerts, Konstantin Ryabitsev,
Jason Gunthorpe, Steven Rostedt, users, Linux Media Mailing List,
Stephen Finucane, bpf, Chris Mason, Christian Brauner,
Alexei Starovoitov
On 7/15/26 10:35 AM, Mark Brown wrote:
> On Wed, Jul 15, 2026 at 10:12:03AM -0700, Ihor Solodrai wrote:
>> On 7/15/26 9:39 AM, Laurent Pinchart wrote:
>>> On Wed, Jul 15, 2026 at 09:28:03AM -0700, Ihor Solodrai wrote:
>
>>>> Yes, assessing whether AI report is "real" is work, but it's not
>>>> reasonable to push this work on already overwhelmed maintainers. And
>>>> if AI is wrong, again it's the job of the author to convince the
>>>> maintainers it's wrong.
>
>>> We disagree on this as well. This would force contributors to constantly
>>> justify their value against a machine that is known to produce a
>>> non-negligible quantity of nonsense.
>
>> I know what you mean and I can relate. Going through walls of
>> AI-generated paragraphs is not fun, to put it lightly.
>
>> However it seems that you are suggesting to simply delegate this work
>> to maintainers, and I very much disagree with that approach (even
>> though I'm not a maintainer myself).
>
> This is something that already happens with human written reviews,
> indeed I have a canned reply for one particular reviewer telling
> submitters it's OK to ignore them. It goes in both directions - it can
> be ignoring some issues or postponing them for later, or it can be
> making sure that there's input from particular people.
My objection is to a suggested expectation for every review to be
triaged by maintainers/reviewers *before* it's sent or the author
engages with it.
I think the author should be the first one to engage, it's a feedback
on their patch after all. And of course reviewers/maintainers are free
to jump in whenever they can and feel it's necessary, and they do. Opt-in.
>
>> I think it makes much more sense for the author of the patch to do the
>> triage, as they have the best context of the work, and they are
>> usually the most invested in their patch being merged in.
>
> It's not clear cut - untrustworthy tooling often causes problems, I'm
> fairly sure I'm seeing issues that are the result of submitters getting
> mislead by LLM reviews in a similar manner to some compiler warnings.
Yes, this is a valid point. I'm not arguing automated AI feedback is
always good. This is why the triage is needed in the first place.
I think one small thing that may help with this is teaching
contributors to not iterate on the revisions too quickly based solely
on AI feedback. And also try to iterate with AI reviews off-list first.
But that's marginal. Fundamentally, a human has to read AI feedback on
the patch in the end. I don't see a way around it, barring the singularity™.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Linking Patchwork with Sashiko?
2026-07-15 17:12 ` Ihor Solodrai
2026-07-15 17:35 ` Mark Brown
@ 2026-07-15 19:06 ` Laurent Pinchart
2026-07-15 21:13 ` Theodore Tso
1 sibling, 1 reply; 12+ messages in thread
From: Laurent Pinchart @ 2026-07-15 19:06 UTC (permalink / raw)
To: Ihor Solodrai
Cc: Roman Gushchin, Mauro Carvalho Chehab, Derek Barbosa,
Matthieu Baerts, Konstantin Ryabitsev, Jason Gunthorpe,
Steven Rostedt, users, Linux Media Mailing List, Stephen Finucane,
bpf, Chris Mason, Christian Brauner, Alexei Starovoitov
On Wed, Jul 15, 2026 at 10:12:03AM -0700, Ihor Solodrai wrote:
> On 7/15/26 9:39 AM, Laurent Pinchart wrote:
> > On Wed, Jul 15, 2026 at 09:28:03AM -0700, Ihor Solodrai wrote:
> >> On 7/14/26 5:59 PM, Laurent Pinchart wrote:
> >>> [...]
> >>
> >> Yes, assessing whether AI report is "real" is work, but it's not
> >> reasonable to push this work on already overwhelmed maintainers. And
> >> if AI is wrong, again it's the job of the author to convince the
> >> maintainers it's wrong.
> >
> > We disagree on this as well. This would force contributors to constantly
> > justify their value against a machine that is known to produce a
> > non-negligible quantity of nonsense.
>
> I know what you mean and I can relate. Going through walls of
> AI-generated paragraphs is not fun, to put it lightly.
>
> However it seems that you are suggesting to simply delegate this work
> to maintainers,
For authors who object to generative AI, yes. And most likely for junior
developers too, see below.
> and I very much disagree with that approach (even
> though I'm not a maintainer myself).
I had noticed that :-)
> I hope you're not saying "let's just not use AI", because that ship
> has sailed a while ago.
For the time being I'm not seeing this as a viable option in any way, so
I'm not calling for it.
> And if automated AI reviews are there, someone has to do the
> triage. AI can do the triage itself only to a limited degree. For
> example, there are instructions in review-prompts where AI checks
> whether an issue has already been discussed. But it will never be
> perfect.
>
> I think it makes much more sense for the author of the patch to do the
> triage, as they have the best context of the work, and they are
> usually the most invested in their patch being merged in.
The discussion regarding enabling sashiko for the linux-media mailing
list started with the question of whether or not authors should be
CC'ed. This came from a consideration on the impact of false positives
and other hallucinations on junior developers. They may have the best
context of the work, but they are also most at risk of not questioning
the validity of the reviews. A one-size-fits-them-all approach may not
be good, which is why I think some sort of opt-in option could address
many concerns. I don't expect immediate and universal enthousiasm
though.
> > Furthermore, if those generative AI
> > tools are as good as their supporters claim, the time they free for
> > maintainers should outweight the need workload to triage the comments.
>
> Even if the AI tools were ~perfect with respect to code review, I
> think people would still be signing off patches and taking
> responsibility for them.
My comment wasn't about signing off patches. I was questioning whether
the time that sashiko saves for a maintainer is smaller, similar, or
larger than the time it takes to triage the comments. If the tool saves
maintainers from having to review some of the code, or from considering
some classes of problems in their reviews, would triaging the comments
come even close to the time saved ?
> An imaginary perfect AI reviewer that maintainer can trust to land
> code without looking at it would essentially become an AI maintainer.
> Independent of whether you think it's possible, it hasn't happened
> yet, so we have to deal with imperfect AI in the meanwhile.
>
> > >> I set up automated AI code reviews on BPF list using Chris Mason's
> >> prompts in the fall last year [3], before sashiko was developed. And
> >> originally we were very worried about the quality of the reviews.
> >>
> >> Time has showed that we had to be more worried about the average
> >> quality of incoming patch instead.
> >>
> >> [1] https://lore.kernel.org/bpf/CAADnVQKsM8iOGO3ZL3LgbWigOBBMrGbTYZ_k_Ktz=+cVkvRLXg@mail.gmail.com/
> >> [2] https://lore.kernel.org/all/20260703-vfs-summer-jam-22b2edbbbc44@brauner/
> >> [3] https://lore.kernel.org/all/0d6aa077-1222-49a0-9554-dd922122a904@meta.com/
> >>
> >>> I believe we should
> >>> follow the first two recommendations of the Software Freedom Conservancy
> >>> on using LLM-backed generative AI systems for FOSS contributions ([1]).
> >>>
> >>> [1] https://sfconservancy.org/llm-gen-ai/llm-backed-generative-ai-recommendations.html
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Linking Patchwork with Sashiko?
2026-07-15 17:41 ` Jason Gunthorpe
@ 2026-07-15 19:10 ` Laurent Pinchart
2026-07-15 19:55 ` Mark Brown
2026-07-15 19:38 ` Dmitry Torokhov
1 sibling, 1 reply; 12+ messages in thread
From: Laurent Pinchart @ 2026-07-15 19:10 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Ihor Solodrai, Roman Gushchin, Mauro Carvalho Chehab,
Derek Barbosa, Matthieu Baerts, Konstantin Ryabitsev,
Steven Rostedt, users, Linux Media Mailing List, Stephen Finucane,
bpf, Chris Mason, Christian Brauner, Alexei Starovoitov
On Wed, Jul 15, 2026 at 02:41:53PM -0300, Jason Gunthorpe wrote:
> On Wed, Jul 15, 2026 at 07:39:21PM +0300, Laurent Pinchart wrote:
> > > Yes, assessing whether AI report is "real" is work, but it's not
> > > reasonable to push this work on already overwhelmed maintainers. And
> > > if AI is wrong, again it's the job of the author to convince the
> > > maintainers it's wrong.
> >
> > We disagree on this as well. This would force contributors to constantly
> > justify their value against a machine that is known to produce a
> > non-negligible quantity of nonsense. Furthermore, if those generative AI
> > tools are as good as their supporters claim, the time they free for
> > maintainers should outweight the need workload to triage the comments.
>
> I feel Sashiko and this whole AI world has created alot more work. It
> is not saving time at all. Now I have to go and read every Sashiko
> report to see if there is something important that normal review
> missed. I have fix all the drive by 'this is already broken' stuff it
> finds. We get way more patches than we did before. Now *everything*
> has to be looked at more skeptically if came from some AI slop. I get
> crazy real security reports that need special hand holding now that we
> never got before.
I'd even go as far as questioning if the additional work is resulting in
less time being spent on changes that would address classes of problems.
There are many areas in the kernel where APIs are hard to use correctly.
Improvements there (and I know there's ongoing work such as Kees' object
allocator functions for instance) may pay off better than spending time
discussing about one-time memory leaks in error paths of probe
functions.
> Even on the submitting side now I have to run patches through all
> kinds of AI tools to have a solid chance to navigate the new guantlet
> upstream.
>
> We aren't saving time, or creating less work. My hope is we are
> creating better quality, higher security and less bugs.
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Linking Patchwork with Sashiko?
2026-07-15 17:41 ` Jason Gunthorpe
2026-07-15 19:10 ` Laurent Pinchart
@ 2026-07-15 19:38 ` Dmitry Torokhov
1 sibling, 0 replies; 12+ messages in thread
From: Dmitry Torokhov @ 2026-07-15 19:38 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Laurent Pinchart, Ihor Solodrai, Roman Gushchin,
Mauro Carvalho Chehab, Derek Barbosa, Matthieu Baerts,
Konstantin Ryabitsev, Steven Rostedt, users,
Linux Media Mailing List, Stephen Finucane, bpf, Chris Mason,
Christian Brauner, Alexei Starovoitov
On Wed, Jul 15, 2026 at 02:41:53PM -0300, Jason Gunthorpe wrote:
> On Wed, Jul 15, 2026 at 07:39:21PM +0300, Laurent Pinchart wrote:
> > > Yes, assessing whether AI report is "real" is work, but it's not
> > > reasonable to push this work on already overwhelmed maintainers. And
> > > if AI is wrong, again it's the job of the author to convince the
> > > maintainers it's wrong.
> >
> > We disagree on this as well. This would force contributors to constantly
> > justify their value against a machine that is known to produce a
> > non-negligible quantity of nonsense. Furthermore, if those generative AI
> > tools are as good as their supporters claim, the time they free for
> > maintainers should outweight the need workload to triage the comments.
>
> I feel Sashiko and this whole AI world has created alot more work. It
> is not saving time at all. Now I have to go and read every Sashiko
> report to see if there is something important that normal review
> missed. I have fix all the drive by 'this is already broken' stuff it
> finds. We get way more patches than we did before. Now *everything*
> has to be looked at more skeptically if came from some AI slop. I get
> crazy real security reports that need special hand holding now that we
> never got before.
No, we are just becoming aware how awful our code actually is and how
may issues have been overlooked by experienced developers and
maintainers, not even talking about junior contributors.
So save all these "pre-existing issue" reports in todo/ and go over them
when you have a chance.
>
> Even on the submitting side now I have to run patches through all
> kinds of AI tools to have a solid chance to navigate the new guantlet
> upstream.
Yes, I experience the same. But that just shows that we are not perfect
and prone to making mistakes.
>
> We aren't saving time, or creating less work. My hope is we are
> creating better quality, higher security and less bugs.
I believe we are. Hopefully as we are fixing all these pre-existing
issues we will be seeing less of them...
Thanks.
--
Dmitry
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Linking Patchwork with Sashiko?
2026-07-15 19:10 ` Laurent Pinchart
@ 2026-07-15 19:55 ` Mark Brown
0 siblings, 0 replies; 12+ messages in thread
From: Mark Brown @ 2026-07-15 19:55 UTC (permalink / raw)
To: Laurent Pinchart
Cc: Jason Gunthorpe, Ihor Solodrai, Roman Gushchin,
Mauro Carvalho Chehab, Derek Barbosa, Matthieu Baerts,
Konstantin Ryabitsev, Steven Rostedt, users,
Linux Media Mailing List, Stephen Finucane, bpf, Chris Mason,
Christian Brauner, Alexei Starovoitov
[-- Attachment #1: Type: text/plain, Size: 1383 bytes --]
On Wed, Jul 15, 2026 at 10:10:46PM +0300, Laurent Pinchart wrote:
> On Wed, Jul 15, 2026 at 02:41:53PM -0300, Jason Gunthorpe wrote:
> > I feel Sashiko and this whole AI world has created alot more work. It
> > is not saving time at all. Now I have to go and read every Sashiko
> > report to see if there is something important that normal review
> > missed. I have fix all the drive by 'this is already broken' stuff it
> > finds. We get way more patches than we did before. Now *everything*
> > has to be looked at more skeptically if came from some AI slop. I get
> > crazy real security reports that need special hand holding now that we
> > never got before.
> I'd even go as far as questioning if the additional work is resulting in
> less time being spent on changes that would address classes of problems.
> There are many areas in the kernel where APIs are hard to use correctly.
> Improvements there (and I know there's ongoing work such as Kees' object
> allocator functions for instance) may pay off better than spending time
> discussing about one-time memory leaks in error paths of probe
> functions.
Yeah, that's been one of the big issues I've noticed with people
handling the LLM reviews poorly - people tend to end up with very
tactical approachs for issues that are not at all tactical. Which is
one of the symptoms of the stuff the LLMs generate themselves.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Linking Patchwork with Sashiko?
2026-07-15 18:39 ` Ihor Solodrai
@ 2026-07-15 20:32 ` Mark Brown
0 siblings, 0 replies; 12+ messages in thread
From: Mark Brown @ 2026-07-15 20:32 UTC (permalink / raw)
To: Ihor Solodrai
Cc: Laurent Pinchart, Roman Gushchin, Mauro Carvalho Chehab,
Derek Barbosa, Matthieu Baerts, Konstantin Ryabitsev,
Jason Gunthorpe, Steven Rostedt, users, Linux Media Mailing List,
Stephen Finucane, bpf, Chris Mason, Christian Brauner,
Alexei Starovoitov
[-- Attachment #1: Type: text/plain, Size: 1576 bytes --]
On Wed, Jul 15, 2026 at 11:39:41AM -0700, Ihor Solodrai wrote:
> On 7/15/26 10:35 AM, Mark Brown wrote:
> > On Wed, Jul 15, 2026 at 10:12:03AM -0700, Ihor Solodrai wrote:
> >> However it seems that you are suggesting to simply delegate this work
> >> to maintainers, and I very much disagree with that approach (even
> >> though I'm not a maintainer myself).
> > This is something that already happens with human written reviews,
> > indeed I have a canned reply for one particular reviewer telling
> > submitters it's OK to ignore them. It goes in both directions - it can
> > be ignoring some issues or postponing them for later, or it can be
> > making sure that there's input from particular people.
> My objection is to a suggested expectation for every review to be
> triaged by maintainers/reviewers *before* it's sent or the author
> engages with it.
> I think the author should be the first one to engage, it's a feedback
> on their patch after all. And of course reviewers/maintainers are free
> to jump in whenever they can and feel it's necessary, and they do. Opt-in.
OTOH I don't really want to have to not only look at whatever random
tool is sending reports but also have to check replies from submitters
for routine stuff, I already have a perfectly adequate supply of email.
What I'm seeing is more at the level where I'm likely to want to provide
some additional feedback to people, or where there's a lot of false
reports to ignore. If people look themselves and take appropriate
action then great, but it's not at the level of the build test bots.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Linking Patchwork with Sashiko?
2026-07-15 19:06 ` Laurent Pinchart
@ 2026-07-15 21:13 ` Theodore Tso
0 siblings, 0 replies; 12+ messages in thread
From: Theodore Tso @ 2026-07-15 21:13 UTC (permalink / raw)
To: Laurent Pinchart
Cc: Ihor Solodrai, Roman Gushchin, Mauro Carvalho Chehab,
Derek Barbosa, Matthieu Baerts, Konstantin Ryabitsev,
Jason Gunthorpe, Steven Rostedt, users, Linux Media Mailing List,
Stephen Finucane, bpf, Chris Mason, Christian Brauner,
Alexei Starovoitov
On Wed, Jul 15, 2026 at 10:06:54PM -0500, Laurent Pinchart wrote:
> This came from a consideration on the impact of false positives
> and other hallucinations on junior developers.
You keep claiming that there is an awful lot of false positives and
other hallucinations. That's an assertion that needs to be tested to
make sure that we're not relying on a few pieces of anecdata. How
does this compare with false positives and bogus recommendations from
certain human reviewers?
After all, I can think of at *least* one human reviewer whose reviews
are far worse than Sashiko's in false positives. That might or might
not be the same person as the one Mark has said that he has a stock
e-mail reply telling a junior developer that they should be free to
ignore reviews from that particular human being. :-)
Each maintainer can decide how much they trust Shashiko reviews. At
least for my subsystem, I haven't seen the hallucionations that you
are continually citing, certainly not as a common case. I will note
that even experienced human reviewers have occasionally flagged an
issue, only to say, "my bad, you're right, it's fine" after a
discussion with the patch author. And that's OK. When a human being
makes a false positive criticism, no one responds with "I am feeling
demeaned as a human being!" Perhaps we should be giving
tool-generated reviews (whether it is via checkpatch or Sashiko) with
a similar level of grace?
At least with tool-generated reviews, we can send bug reports asking
that the tool can be improved. At least with one particular human
reviewer that I have in mind, no attempts asking that person to
improve their reviews hasn't been helpful.
Bottom line --- if you are making the claim the Sashiko reviews are
rife with false positives, all I can say is <<Citation Needed>>.
- Ted
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2026-07-15 21:14 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20260604085201.177ad537@foz.lan>
[not found] ` <20260607195656.02788791@foz.lan>
[not found] ` <akQlPSNfGEllNVhe@debarbos-thinkpadt14gen5.rmtusma.csb>
[not found] ` <20260710074528.5a6e4457@foz.lan>
[not found] ` <20260710083845.23c753ca@foz.lan>
[not found] ` <87wlv2jq4t.fsf@linux.dev>
[not found] ` <20260713095538.3d5e86f1@foz.lan>
[not found] ` <20260713094120.GD1127719@killaraus.ideasonboard.com>
[not found] ` <20260713220427.582b28bf@foz.lan>
[not found] ` <7ia4mrvtrxjl.fsf@castle.c.googlers.com>
[not found] ` <20260715005909.GF1656185@killaraus.ideasonboard.com>
2026-07-15 16:28 ` Linking Patchwork with Sashiko? Ihor Solodrai
2026-07-15 16:39 ` Laurent Pinchart
2026-07-15 17:12 ` Ihor Solodrai
2026-07-15 17:35 ` Mark Brown
2026-07-15 18:39 ` Ihor Solodrai
2026-07-15 20:32 ` Mark Brown
2026-07-15 19:06 ` Laurent Pinchart
2026-07-15 21:13 ` Theodore Tso
2026-07-15 17:41 ` Jason Gunthorpe
2026-07-15 19:10 ` Laurent Pinchart
2026-07-15 19:55 ` Mark Brown
2026-07-15 19:38 ` Dmitry Torokhov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox