Re: Introduce Sashiko (agentic review of Linux kernel changes)

[Roman Gushchin <roman.gushchin@linux.dev>]

"Lorenzo Stoakes (Oracle)" <ljs@kernel.org> writes:

> On Tue, Mar 17, 2026 at 03:31:11PM +0000, Roman Gushchin wrote:
>> Hello,
>>
>> I'm happy to share something my colleagues and I have been working on
>> for the last several months:
>> Sashiko - an agentic system for Linux kernel changes.
>>
>> First, Sashiko is available as a service at:
>>   * https://sashiko.dev
>>
>
> ...
>
> (For one I'm going to go fix some bugs on my series I saw reported there).
>
> I think over time as the approach/model is refined this will get a LOT
> better, it seems these things can acelerate quickly.

Hi Lorenzo,

Thank you for kind words!

RE false positives: I think Chris's prompts were initially heavily
biased towards avoiding false positives, but it comes at the cost of
missing real issues (in general, I don't have hard data on % of findings).
Now he also is looking to relax it a bit, to my knowledge.
But then there are different models in use, different protocols, etc.

I also have a notion of issue severity and I was thinking about
e.g. sending out only reviews revealing critical & high severity bugs
(e.g. memory corruptions & panics). Or maybe send the feedback to the
author in any case (e.g. for fixing typos), but cc maintainers only if
there are serious concerns.

And obviously no pressure, I won't enable any public email sending
unless there is a consensus across maintainers of the corresponding
subsystem.

>>
>> * What's next?
>>
>> This is our first version and it's obviously not perfect. There is a
>> long list of fixes and improvements to make. Please, don't expect it to
>> be 100% reliable, even though we'll try hard to keep it up and running.
>> Please use github issues or email me any bug reports and feature
>> requests, or send PR's.
>
> Of course, it's all much appreicated!
>
>>
>> As of now, Sashiko only provides a web interface;
>> however, Konstantin Ryabitsev is already adding sashiko.dev support to b4,
>> and SeongJae Park is adding support to hkml.
>> That was really fast, thank you!
>
> Thanks to Konstantantin and SJ too but the web interface is pretty nice I
> must say so thanks for that! :)
>
>>
>> We're working on adding an email interface to Sashiko, and soon Sashiko
>> will be able to send out reviews over email - similar to what the bpf
>> subsystem already has. It will be opt-in by subsystem and will have options
>
> Like I said, I think it's a bit premature for mm at least _at this point_
> but I'm sure it'll get there.

I'd really appreciate (and actually need) yours and other maintainers and
developers feedback here. Even though I can't fix every single false
positive as a code issue, I can hopefully tackle some common themes.

Chris did a fantastic work on the bpf subsystem (and several others) by
manually analyzing replies to the AI feedback and adjusting prompts. Now
we need to repeat this for all other subsystems.

>
> For now I think we need to get the false positive rate down a fair bit
> otherwise it might be a little distracitng.
>
> But people are _already_ integrating the web interface into workflows, I
> check it now, and Andrew is already very keen :) see:
>
> https://lore.kernel.org/all/20260317121736.f73a828de2a989d1a07efea1@linux-foundation.org/
> https://lore.kernel.org/all/20260317113730.45d5cef4ba84be4df631677f@linux-foundation.org/
>
>> to CC only the author of the patch, maintainers, volunteers, or send a
>> fully public reply. If you're a maintainer and have a strong preference
>> to get reviews over email, please let me know.
>
> Well as maintainer I think 'not quite yet' but probably soon is the answer
> on that one!
>
>>
>> We also desperately need better benchmarks, especially when it comes to
>> false positives. Having a decent vetted set of officially perfect
>> commits can help with this.
>
> Not sure perfect commits exist in the kernel certainly not mine :P

Same here :) This is why it's so hard.

>
>>
>> Finally, some subsystems have a good prompts coverage and some don't. It
>> doesn't have to be lengthy documentation (and it might actually be
>> counter-productive), but having a small list of things to look at - some
>> high-level concepts which are hard to grasp from the code, etc. - can
>> help a lot with both bug discovery and false positives.
>
> I guess best contributed to Chris's review-prompts repo right?

Both works for me now, we'll figure out with Chris how to sync our
prompts. The small problem is that we're using various models, tools and
review protocols and barely can test each other's setup. And it's all
very fragile, so it's not exactly trivial.
But we'll figure out something soon.

In general we need to carefully separate instructions (like which tools
to use, which prompts to load etc) from factual data. Then we can easily
use the factual data with various tooling around.

Thanks!