Re: sysbot AI patches and wireless

[Slawomir Stepien <sst@poczta.fm>]

On sie 06, 2026 12:15, Johannes Berg wrote:
> Hi!
> 
> > 1. Every AI-generated patch sent by syzbot has been pre-reviewed and
> > approved by a human engineer. The person who approved the patch is
> > listed in the From: and Signed-off-by: fields.
> 
> Yes, I realize that. But the experience still seems to be one of me
> effectively consuming pure LLM output, just via an intermediary. If I
> had the time to do that, then I'd be perfectly able to talk to an LLM
> directly, cutting out the intermediaries, and get the issues fixed that
> way - much, much faster than writing everything in emails.
> 
> If, on the other hand, the intermediaries actually do the necessary
> legwork etc. then I don't think this whole process is necessary; I don't
> think using "b4" would be a significant hurdle in the process, and
> that's really the only thing this helps with? Maybe LLM access (not
> everyone has effectively free tokens), but that's still enabled by
> syzbot providing the service over on the internal/upstream-moderation
> list I guess.
> 
> Now, of course now that I say this (and you disabled it) I guess I'll
> just see the patches pasted into an email manually instead, and I've
> lost the signal that I could use to just ignore them entirely, but at
> least I've been honest about it - and I guess I'll still silently drop
> patches where the "human engineer" has no idea what they're doing. Even
> having Reported-by: syzbot has been a signal of that to some extent.
> 
> I do think syzbot is a bit of a special case - at least if there's a
> reproducer it's trivial to tell the system "hack the code until the
> issue no longer reproduces" - but that's almost certainly guaranteed to
> not be a useful patch yet.
> 
> The system, in a case like this, is almost certainly going to provide a
> very narrow, targeted fix (with an annoying wall of text explaining
> exactly that), but I think that at least the human in the loop should
> actually take a step back from that and ask what the semantics of the
> code should be ... I've played this game with Slawomir's first patch
> myself, but that clearly cannot scale if the original intermediary
> doesn't want to do that.
> 
> Maybe it's something you can even tell the LLM to do, somehow, so the
> first draft is better. In this case, for example, why the hell did it
> decide that it made any sense to have multiple branches of the same
> switch statement - and there are even only two! - implement the same
> validation? At the very least I'd expect the "human engineer" to take
> that step back.
> 
> This is why I'm refusing these patches, because clearly nobody actually
> even bothers to look at the semantics of the code before or during the
> patching. Does pulling out the check outside of the switch change the
> order of errors? Yes. Does that matter? No, the new order of errors for
> NL80211_TDLS_ENABLE_LINK would actually - if you think about it (!) -
> make a lot more sense! Am I surprised the LLM doesn't do that when you
> tell it to make a targeted fix? Absolutely not.
> 
> But I really cannot make that judgement call myself for every single
> issue like that, if I could, see above, I could be doing all of this
> myself. Need the contributors to do that. Slawomir did that after I
> prompted (pun intended!) him to do that, and it didn't work out so well

It didn't? Or it did? :) Well I do not have new version of the patch yet, but I'm still
investigating correct approach and testing a lot. I see now, that I should do that in the 1st place.
I'm sorry for that.

> and we had a good discussion about it, but again, I can't provide that
> support all time.
> 
> > 2. For the patches already sent to LKML, the bot *does not* send
> > automatic replies and *does not* automatically submit newer versions.
> > Once the patch is on LKML, follow-ups, comments, and iterations are
> > handled entirely by the human developer who signed off on it, so you
> > have not been interacting with an LLM.
> 
> Thanks for clarifying. I've definitely seen pure LLM replies (sometimes
> even with the output saying things along the lines of "the reviewer said
> this, I'll explain...") but that might have been in other contexts,
> clearly it's not just syzbot which enables people doing things like
> that.
> 
> > I apologize for the frustration this may have caused. We'll stop
> > sending AI-assisted patches to the wireless subsystem.
> 
> Apology accepted, and thanks for disabling it.
> 
> Slawomir, Krystian, I assume you mean well and I apologise that you're
> getting caught in the cross-fire here. Even the targeted fixes are still
> fixes, so I understand from your perspective this still made sense, but
> from mine it just absolutely cannot scale, both in terms of long-term
> maintenance (sprinkling checks all over the code disregarding the
> architecture) and also in terms of review bandwidth etc. I'm sorry
> you're effectively the two first victims of this new process.

Thanks and now worries! It's a good lesson, that the overall subsystem might need a better
approach. I know, that I sometimes will overlook this.

In the end, I should be saying sorry for wasting your time. And also, I should say `thanks!` for all
the good hints and "prompts"!

-- 
Slawomir Stepien