[MAINTAINERS SUMMIT] The place of AI code review in the Linux Kernel process

[MAINTAINERS SUMMIT] The place of AI code review in the Linux Kernel process

[Roman Gushchin]

Sashiko and AI code reviews in general have gained a lot of traction in
the kernel community over the last few months. I think it is fair to say
that at this point there are no more questions about the usefulness of
AI code review in general. However, there are many questions about how
to improve current workflows and better integrate them into the kernel
development process.

Some specific topics I propose we discuss:
* Review of LTS patches and kernel releases.
Currently, Sashiko reviews only publicly proposed changes, and it is
completely up to individual maintainers whether to take the findings
into account. By starting to review kernel releases and LTS backports,
we can likely significantly improve security and minimize the number of
regressions. However, this adds to the workload of maintainers, and we
need to agree on a specific process. For example, we could agree to
bring up only critical and high-severity issues and expect the authors
of corresponding changes to provide a fix-up or explain why it is not an
issue.

* How to maintain the long-term stability of Sashiko?
Several kernel engineers and maintainers have rightfully expressed
concerns about relying on infrastructure provided by a single company
without clear formal guarantees. It would be great to discuss what a
more sustainable model could realistically look like and how we might
get there.

* Handling of pre-existing bugs.
Currently, Sashiko reports pre-existing bugs alongside new issues
(while trying hard to highlight that these issues were not introduced by
the proposed change). This approach comes with significant pros (a
steady stream of bug fixes) and cons (additional noise and workload for
maintainers). I am considering a database of pre-existing issues to
ensure they are reported only once (or once per year), with an option
for the respective maintainers to flag them as false positives. This
will also provide maintainers an access to a deduplicated and ranked
list of potential issues in their subsystem’s codebase.

* Prompt development and testing.
Currently, prompts are maintained in two GitHub repositories and are
changed manually or with the help of AI coding agents. However, there is
no established practice for testing them, especially across various LLM
models. At the last LSFMMBPF conference, there was a discussion about
moving them into the kernel tree. I see some pros and cons to this
approach, but the ownership and testing models are not entirely clear.

* Interactive mode.
Many engineers have asked for some sort of interactive mode where they
can ask additional questions or follow up on the initial feedback from
Sashiko. I plan to add this to the local review mode, but for the
central public instance, it is problematic from both security and token
cost perspectives. Sashiko could analyze false-positive cases reported by
engineers, attempt to verify them, and automatically suggest specific
prompt adjustments. However, there is a non-trivial number of cases
where people are wrong to dismiss AI findings. How should we treat these
cases?

Thanks!

Re: [MAINTAINERS SUMMIT] The place of AI code review in the Linux Kernel process

[Miguel Ojeda]

On Wed, Jul 15, 2026 at 6:55 PM Roman Gushchin <roman.gushchin@linux.dev> wrote:
>
> * Review of LTS patches and kernel releases.

For the backporting case, I think this could help, and there are
certain things we could ask for, i.e. custom prompts.

It may also have less noise, in the sense that the AI has the original
commit in mainline as a baseline, i.e. it is not a completely new
patch of unknown quality.

I imagine Sasha et al. have experience with this.

> * How to maintain the long-term stability of Sashiko?

How hard do you think it would be to get at least one more AI company
to donate some tokens (for their own model)? Could that be a simple
start to begin reducing the single-provider issue?

> * Handling of pre-existing bugs.

Do you mean the AI would be able to query that database? Or directly
being part of the prompt for that subsystem/file if it is not too big?

Ideally some of those cases would just be a comment in the source code
(like we already do) -- local and in-band, and it may help other
readers too.

> * Prompt development and testing.

Yeah, it is pretty much a black box at the moment, especially since a
maintainer doesn't control the runs, i.e. even worse than a local LLM
that one can do several quick tries etc.

Regarding putting the prompts into the tree, that sounds similar to
the in-tree CI files. That could help, but I wonder if it should be as
general, shared guidelines for a subsystem (i.e. useful for humans
too) or truly LLM-only notes (i.e. to overcome certain weaknesses of
the AI etc.).

> * Interactive mode.

Could this access be restricted to, say, people with Gitolite access
or similar, and rate limited somehow?

I am pretty sure you already thought of this, but I wonder how feasible it is.

Thanks!

Cheers,
Miguel

Re: [MAINTAINERS SUMMIT] The place of AI code review in the Linux Kernel process

[Roman Gushchin]

Miguel Ojeda <miguel.ojeda.sandonis@gmail.com> writes:

> On Wed, Jul 15, 2026 at 6:55 PM Roman Gushchin <roman.gushchin@linux.dev> wrote:
>>
>> * Review of LTS patches and kernel releases.
>
> For the backporting case, I think this could help, and there are
> certain things we could ask for, i.e. custom prompts.

+1

> It may also have less noise, in the sense that the AI has the original
> commit in mainline as a baseline, i.e. it is not a completely new
> patch of unknown quality.
>
> I imagine Sasha et al. have experience with this.
>
>> * How to maintain the long-term stability of Sashiko?
>
> How hard do you think it would be to get at least one more AI company
> to donate some tokens (for their own model)? Could that be a simple
> start to begin reducing the single-provider issue?

It's hard to say. Technically it's fairly trivial to make Sashiko using
multiple models for discovery stages 1-7 in parallel, which will almost
certainly improve the review quality. In such case models will
effectively cooperate to produce better reviews, which is our goal.

>> * Handling of pre-existing bugs.
>
> Do you mean the AI would be able to query that database? Or directly
> being part of the prompt for that subsystem/file if it is not too big?

Yes, Sashiko can query the database before reporting pre-existing
issues, e.g. if there is an obvious but not urgent pre-existing bug
there is no point to report it again and again.

Thanks!

Re: [MAINTAINERS SUMMIT] The place of AI code review in the Linux Kernel process

[Mauro Carvalho Chehab]

On Wed, 15 Jul 2026 16:55:03 +0000
Roman Gushchin <roman.gushchin@linux.dev> wrote:

> Sashiko and AI code reviews in general have gained a lot of traction in
> the kernel community over the last few months. I think it is fair to say
> that at this point there are no more questions about the usefulness of
> AI code review in general. However, there are many questions about how
> to improve current workflows and better integrate them into the kernel
> development process.

Heh, I was about to propose the same theme. IMO, we do need to discuss
more about AI, specially after the discussions related to linux-media
and Sashiko that it is happening those days.

---

Besides code review, there's one area I also want to propose some
discussions: 

Use AI to fix documentation issues
----------------------------------

If we run:

	$ scripts/kernel-doc -Wall .

it will generate a ~77k warnings due to bad documentation. This
includes files that are not currently on Documentation, but even if
we pick just the ones included there, the number is really big.

Several of them are there since the beginning, and we ended removing
-Wall from kernel-doc generation to avoid a too noisy documentation
build.

I'm proposing to discuss using AI to at least partially address
the documentation warning noise and then finally enable -Wall for
kernel-doc warnings.

---


> Some specific topics I propose we discuss:
> * Review of LTS patches and kernel releases.
> Currently, Sashiko reviews only publicly proposed changes, and it is
> completely up to individual maintainers whether to take the findings
> into account. By starting to review kernel releases and LTS backports,
> we can likely significantly improve security and minimize the number of
> regressions. However, this adds to the workload of maintainers, and we
> need to agree on a specific process. For example, we could agree to
> bring up only critical and high-severity issues and expect the authors
> of corresponding changes to provide a fix-up or explain why it is not an
> issue.

Makes sense to me, but I would start with very specific prompts to
check issues where LLM is particularly good on checking, and where
compiler warnings don't report.

One such area is to check error paths: Kernel works reasonably
well when registration/memory allocation/resource allocation/...
succeeds, but there is a large number of fix patches that sometimes
take several years to be applied related to error condition
failures.

Also, if we're willing to do that, it is probably be worth to
have something similar to what we had in Coverity: an interface
where maintainers could navigate at the found issues, with their
fixes, ordered by priority, and eventually tag as false positive,
preventing future reports of the same issue.

> * How to maintain the long-term stability of Sashiko?
> Several kernel engineers and maintainers have rightfully expressed
> concerns about relying on infrastructure provided by a single company
> without clear formal guarantees. It would be great to discuss what a
> more sustainable model could realistically look like and how we might
> get there.

IMO, the solution is to use open source LLM models, preferably in
a way they could run outside the cloud.

> * Handling of pre-existing bugs.
> Currently, Sashiko reports pre-existing bugs alongside new issues
> (while trying hard to highlight that these issues were not introduced by
> the proposed change). This approach comes with significant pros (a
> steady stream of bug fixes) and cons (additional noise and workload for
> maintainers). 

> I am considering a database of pre-existing issues to
> ensure they are reported only once (or once per year), with an option
> for the respective maintainers to flag them as false positives. This
> will also provide maintainers an access to a deduplicated and ranked
> list of potential issues in their subsystem’s codebase.

Makes total sense to me.

> 
> * Prompt development and testing.
> Currently, prompts are maintained in two GitHub repositories and are
> changed manually or with the help of AI coding agents. However, there is
> no established practice for testing them, especially across various LLM
> models. At the last LSFMMBPF conference, there was a discussion about
> moving them into the kernel tree. I see some pros and cons to this
> approach, but the ownership and testing models are not entirely clear.

I don't think it is a good idea to move to the Kernel tree, but
it makes sense to have them on a shared repository (or database)
where each maintainer can be able to adjust the prompts as needed.

For this to work properly, IMO we need an interface to be able to
trigger a test on Sashiko using the new prompts and see the output
differences before/after the changes, before committing it to
production. Perhaps we can use github/gitlab workflows for such
purpose, I dunno.

> * Interactive mode.
> Many engineers have asked for some sort of interactive mode where they
> can ask additional questions or follow up on the initial feedback from
> Sashiko. I plan to add this to the local review mode, but for the
> central public instance, it is problematic from both security and token
> cost perspectives. Sashiko could analyze false-positive cases reported by
> engineers, attempt to verify them, and automatically suggest specific
> prompt adjustments. However, there is a non-trivial number of cases
> where people are wrong to dismiss AI findings. How should we treat these
> cases?

An interactive mode is important, together with the capability of
running it locally on affordable GPUs. For this to work, we may need
to have a Linux-Kernel sashiko trained model that could be executed
locally with a limited number of active parameters.

Thanks,
Mauro

Re: [MAINTAINERS SUMMIT] The place of AI code review in the Linux Kernel process

[Jason Gunthorpe]

On Wed, Jul 15, 2026 at 04:55:03PM +0000, Roman Gushchin wrote:
> * Handling of pre-existing bugs.
> Currently, Sashiko reports pre-existing bugs alongside new issues
> (while trying hard to highlight that these issues were not introduced by
> the proposed change). This approach comes with significant pros (a
> steady stream of bug fixes) and cons (additional noise and workload for
> maintainers). I am considering a database of pre-existing issues to
> ensure they are reported only once (or once per year), with an option
> for the respective maintainers to flag them as false positives. This
> will also provide maintainers an access to a deduplicated and ranked
> list of potential issues in their subsystem’s codebase.

I would really like a syzkaller like dashboard of all these
pre-existing issues and a nag/summary email so they actually get
fixed.

I've gone and fixed a bunch on my own, mostly out of fear that they
will just disappear and be lost, but it is an annoying urgency. I'd
rather have as a giant todo list (that maybe other people could help
with too)

Often alot of tokens are spent to find these things, it feels wrong
that they are effectively lost in the endless stream of reviews..

> * Prompt development and testing.
> Currently, prompts are maintained in two GitHub repositories and are
> changed manually or with the help of AI coding agents. However, there is
> no established practice for testing them, especially across various LLM
> models. At the last LSFMMBPF conference, there was a discussion about
> moving them into the kernel tree. I see some pros and cons to this
> approach, but the ownership and testing models are not entirely clear.

Having them in the kernel always felt better to me, I wonder what the
cons are.

> Many engineers have asked for some sort of interactive mode where they
> can ask additional questions or follow up on the initial feedback from
> Sashiko. I plan to add this to the local review mode, but for the
> central public instance, it is problematic from both security and token
> cost perspectives. 

Could there some way to download the entire context from all the passes
to a local environment and have a local llm chew on it to answer
questions?

Jason

Re: [MAINTAINERS SUMMIT] The place of AI code review in the Linux Kernel process

[Roman Gushchin]

Jason Gunthorpe <jgg@nvidia.com> writes:

> On Wed, Jul 15, 2026 at 04:55:03PM +0000, Roman Gushchin wrote:
>> * Handling of pre-existing bugs.
>> Currently, Sashiko reports pre-existing bugs alongside new issues
>> (while trying hard to highlight that these issues were not introduced by
>> the proposed change). This approach comes with significant pros (a
>> steady stream of bug fixes) and cons (additional noise and workload for
>> maintainers). I am considering a database of pre-existing issues to
>> ensure they are reported only once (or once per year), with an option
>> for the respective maintainers to flag them as false positives. This
>> will also provide maintainers an access to a deduplicated and ranked
>> list of potential issues in their subsystem’s codebase.
>
> I would really like a syzkaller like dashboard of all these
> pre-existing issues and a nag/summary email so they actually get
> fixed.
>
> I've gone and fixed a bunch on my own, mostly out of fear that they
> will just disappear and be lost, but it is an annoying urgency. I'd
> rather have as a giant todo list (that maybe other people could help
> with too)
>
> Often alot of tokens are spent to find these things, it feels wrong
> that they are effectively lost in the endless stream of reviews..
>
>> * Prompt development and testing.
>> Currently, prompts are maintained in two GitHub repositories and are
>> changed manually or with the help of AI coding agents. However, there is
>> no established practice for testing them, especially across various LLM
>> models. At the last LSFMMBPF conference, there was a discussion about
>> moving them into the kernel tree. I see some pros and cons to this
>> approach, but the ownership and testing models are not entirely clear.
>
> Having them in the kernel always felt better to me, I wonder what the
> cons are.

A minor problem is that for prompts there is no point to stick to the
kernel release schedule (merge window, etc), which will only slow down
things. So it requires Linus being onboard with the idea to accept these
pull requests all the time.

But also I hope that eventually (and I work on it) we can at least
enhance prompts in a semi-automatic way, based on the human feedback in
email replies to sashiko reviews. It's not trivial, requires
verification and likely a human in the loop at least initially,
but I hope that mid-term it might turn the whole process into
more-or-less continuously self-improving system.

Having prompts in the kernel tree will also have an interesting
side-effect: newer models will learn on SOME version of these prompts
and then get a newer version as a prompt. Idk how it will play out.

That said, I'm not against putting prompts into the kernel tree at all.
I'm just saying the testing and ownership questions are real and need
a real discussion.

>> Many engineers have asked for some sort of interactive mode where they
>> can ask additional questions or follow up on the initial feedback from
>> Sashiko. I plan to add this to the local review mode, but for the
>> central public instance, it is problematic from both security and token
>> cost perspectives. 
>
> Could there some way to download the entire context from all the passes
> to a local environment and have a local llm chew on it to answer
> questions?

Yeah, it's an option I though about. Like a download button which just
dumps all relevant context as an .md file, for example.

Thanks!

Re: [MAINTAINERS SUMMIT] The place of AI code review in the Linux Kernel process

[Dmitry Torokhov]

On Wed, Jul 15, 2026 at 04:55:03PM +0000, Roman Gushchin wrote:
> 
> * Handling of pre-existing bugs.
> Currently, Sashiko reports pre-existing bugs alongside new issues
> (while trying hard to highlight that these issues were not introduced by
> the proposed change). This approach comes with significant pros (a
> steady stream of bug fixes) and cons (additional noise and workload for
> maintainers). I am considering a database of pre-existing issues to
> ensure they are reported only once (or once per year), with an option
> for the respective maintainers to flag them as false positives. This
> will also provide maintainers an access to a deduplicated and ranked
> list of potential issues in their subsystem’s codebase.

Having database and either flagging issues or tweaking the prompts would
be nice.

Also I feel that if we simply had separate sections in the review, one
giving feedback (including "all clear") strictly on the patch itself,
and another one stating something like:

*** Pre-existng issues identified during patch analysis (address at will) ***

and then listing all other issues would help recipients (and maintainers
too).

Thanks.

-- 
Dmitry

Re: [MAINTAINERS SUMMIT] The place of AI code review in the Linux Kernel process

[Steven Rostedt]

On Wed, 15 Jul 2026 16:55:03 +0000
Roman Gushchin <roman.gushchin@linux.dev> wrote:

> Sashiko and AI code reviews in general have gained a lot of traction in
> 
> Some specific topics I propose we discuss:
> * Review of LTS patches and kernel releases.

Honestly, I think those that use LTS should get together and support
developers that will look at the reviews of LTS and push to get the
fixes for them. I keep hearing that LTS is used more than mainline and
upstream doesn't put as much priority in what goes into LTS (and stable
in general) as it does for mainline. Perhaps those that depend on LTS
and stable should be the ones doing the work.

> 
> * How to maintain the long-term stability of Sashiko?
> Several kernel engineers and maintainers have rightfully expressed
> concerns about relying on infrastructure provided by a single company
> without clear formal guarantees. It would be great to discuss what a
> more sustainable model could realistically look like and how we might
> get there.

As I believe Jason already mentioned, it would be great if we could
possibly get kernel specific model information that can be shared.
Possibly even with an open source model so those that want to buy a
NVidia Spark or something similar could run their own reviews ;-)

> 
> * Handling of pre-existing bugs.

I like the pre-existing bugs. But I do agree it shouldn't be sent to
the author of a patch. It should either be a separate review or email
that says "Oh I found these pre-existing bugs around the code that was
modified". I want to know what pre-existing bugs are in the code I
maintain, and I would love to make sure that it's public so that
someone else might want to do the fix ;-)

> 
> * Prompt development and testing.

No comment.

> 
> * Interactive mode.

Yes, this is something I would love to have. There's some unique ways
the code works that Sashiko could use to get better.

-- Steve

Re: [MAINTAINERS SUMMIT] The place of AI code review in the Linux Kernel process

[SJ Park]

On Wed, 15 Jul 2026 16:55:03 +0000 Roman Gushchin <roman.gushchin@linux.dev> wrote:

> Sashiko and AI code reviews in general have gained a lot of traction in
> the kernel community over the last few months. I think it is fair to say
> that at this point there are no more questions about the usefulness of
> AI code review in general. However, there are many questions about how
> to improve current workflows and better integrate them into the kernel
> development process.

It is very helpful for me.  Appreciate your efforts again.

> 
> Some specific topics I propose we discuss:
> * Review of LTS patches and kernel releases.
> Currently, Sashiko reviews only publicly proposed changes, and it is
> completely up to individual maintainers whether to take the findings
> into account. By starting to review kernel releases and LTS backports,
> we can likely significantly improve security and minimize the number of
> regressions. However, this adds to the workload of maintainers, and we
> need to agree on a specific process. For example, we could agree to
> bring up only critical and high-severity issues and expect the authors
> of corresponding changes to provide a fix-up or explain why it is not an
> issue.

I once made a mistake in a manual stable backport of a patch that was failed to
be automatically backported by the stable kernel maintainers.  I was able to
fix it because Sashiko found and reported the mistake, because I was Cc-ing a
mailing list that onboarded to Sashiko.  So I can show the clear advantage of
LTS patches.

But I'm also concerned if I will have to read and (optionally) reply to same
Sashiko findings that I already decided to ignore for each of the stable
backports.

Doing that for each release could also be useful at finding issues made during
the merges into the trees towards the mainline.  But I'm concerned if this can
be seen like ignoring each subsystem's decision about Sashiko.  My previous
concern about a case that I have to read Sashiko's comment that I already
decided to ignore again.  Probably just one more time, so less concerned than
the LTS case, but still concerned and better to be avoided.

> 
> * How to maintain the long-term stability of Sashiko?
> Several kernel engineers and maintainers have rightfully expressed
> concerns about relying on infrastructure provided by a single company
> without clear formal guarantees. It would be great to discuss what a
> more sustainable model could realistically look like and how we might
> get there.

I personally not really worrry about this at this stage.  To my understanding,
it is already Linux Foundation's open source project, and can learn with any
LLM including open models on any setup.  But, if this is a real concern, I'm
wondering if moving the entire infrastructure to Linux Foundation (like
kernel.org) and using an open model for the public instance can relieve the
concern.

I'm bit concerned if using an open model will increase the false positives of
Sashiko.  But I feel like that would still be useful for small subsystems
having no many reviewers.  And we might be able to let more people help
improving Sashiko prompts by running a setup same to the public one on their
local setup?  Like normal open source projects do.

> 
> * Handling of pre-existing bugs.
> Currently, Sashiko reports pre-existing bugs alongside new issues
> (while trying hard to highlight that these issues were not introduced by
> the proposed change). This approach comes with significant pros (a
> steady stream of bug fixes) and cons (additional noise and workload for
> maintainers). I am considering a database of pre-existing issues to
> ensure they are reported only once (or once per year), with an option
> for the respective maintainers to flag them as false positives. This
> will also provide maintainers an access to a deduplicated and ranked
> list of potential issues in their subsystem’s codebase.

I know Sashiko has evolved to handle this issue much better than the initial
days and appreciate the developers' efforts.  But, still it is true it still
has rooms to improve.

I think having the database will be super helpful.  I personally show its value
more on reducing duplicated reports.  And in this case, I would be less
bothered and even prefer repeated reporting if it mentions it is a duplicated
reports of an issue that I can check later on the database.

The false positive flagging would also be nice.  For a case that even the
maintainer can mistake or change minds, having a way to revoke it would also be
nice.  For cases that maintainer wants to fix in long term, but with low
priority, having the score would also be helpful.

> 
> * Prompt development and testing.
> Currently, prompts are maintained in two GitHub repositories and are
> changed manually or with the help of AI coding agents. However, there is
> no established practice for testing them, especially across various LLM
> models. At the last LSFMMBPF conference, there was a discussion about
> moving them into the kernel tree. I see some pros and cons to this
> approach, but the ownership and testing models are not entirely clear.

I find no reason to say no to the idea of having the prompts inside the kernel
tree.  I think those are at lest need to be hosted on some trusted safe place,
like kernel.org.

If we have the trusted found issues database, and if we have ways to give
feedback to issues.  Maybe we could use the feedback to the issues as a way to
measure if it is getting better or worse?

> 
> * Interactive mode.
> Many engineers have asked for some sort of interactive mode where they
> can ask additional questions or follow up on the initial feedback from
> Sashiko. I plan to add this to the local review mode, but for the
> central public instance, it is problematic from both security and token
> cost perspectives. Sashiko could analyze false-positive cases reported by
> engineers, attempt to verify them, and automatically suggest specific
> prompt adjustments. However, there is a non-trivial number of cases
> where people are wrong to dismiss AI findings. How should we treat these
> cases?

I really wish this feature.  Top two use cases I want to be in real are:

1. Applying patch on specific tree.
2. Review of a fixup patch.

I agree the cost and security are concerns.  I wonder if we could have a kind
of limitation for the request frequency and scope to mitigate those issues.
For example, we could restrict the specific tree for use case 1 to trees hosted
in kernel.org or signed by known and trusted developers.

> 
> Thanks!


Thanks,
SJ