Full subsystem codebase scan for identifying pre-existing issues?

Full subsystem codebase scan for identifying pre-existing issues?

[Joanne Koong]

Hi,

Is there a way for Sashiko to run a full scan of a subsystem/tree to
identify all the pre-existing issues so that they can be fixed
proactively? Currently, they're flagged on patch reviews, which can be
a bit frustrating for contributors who didn't introduce the problem,
and adds a bit of noise for maintainers/reviewers looking at the
Sashiko report.

Thanks,
Joanne

Re: Full subsystem codebase scan for identifying pre-existing issues?

[Derek Barbosa]

Hi,

On Wed, Jul 29, 2026 at 01:34:10PM -0700, Joanne Koong wrote:
> Hi,
> 
> Is there a way for Sashiko to run a full scan of a subsystem/tree to
> identify all the pre-existing issues so that they can be fixed
> proactively?

At the moment, no. At least nothing that I am aware of :)

There have been some small conversations about such functionality, but nothing
has really taken shape yet. One of the concerns was trying to validate the sheer
number of potential-false-positives, and how to present such information in an
actionable manner. IIRC syzbot/syzkaller's way of storing such data was used as
a comparison point here.

FWIW, Chris Mason has a repository for a tool that runs such scans on codebases
-- paired with tools like semcode. His prompts are the same ones that we
frequently sync back to Sashiko. It won't solve the other point you made, but it
may be worth looking into. [0][1]

> Currently, they're flagged on patch reviews, which can be
> a bit frustrating for contributors who didn't introduce the problem,
> and adds a bit of noise for maintainers/reviewers looking at the
> Sashiko report.

Unfortunately, you aren't the first to report a problem with these.

Currently, in the 11th stage instruction in the code, pre-existing issues are
propagated in the report like so:

<snip>

CRITICAL RULE: If a finding is flagged as pre-existing (`\"preexisting\":
true`), you MUST explicitly state in your inline comment that this issue is
pre-existing and was not introduced by the patch under review. Use phrasing like
\"This isn't a bug introduced by this patch, but...\" or \"This is a
pre-existing issue, but...\" to start the comment.

</snip>

Instead of them replying inline with "This isn't a bug introduced by this patch,
but by..." or "This is a pre-existing issue, but does..." how would you like to
see this information presented/conveyed?

I created an issue here [2]

> 
> Thanks,
> Joanne
> 

[0] https://github.com/masoncl/kres
[1] https://github.com/masoncl/review-prompts
[2] https://github.com/sashiko-dev/sashiko/issues/381

Thanks,

-- 
Derek <debarbos@redhat.com>

Re: Full subsystem codebase scan for identifying pre-existing issues?

[Andrew Morton]

On Wed, 29 Jul 2026 21:43:26 -0400 Derek Barbosa <debarbos@redhat.com> wrote:

> Hi,
> 
> > Currently, they're flagged on patch reviews, which can be
> > a bit frustrating for contributors who didn't introduce the problem,
> > and adds a bit of noise for maintainers/reviewers looking at the
> > Sashiko report.
> 
> Unfortunately, you aren't the first to report a problem with these.
> 
> Currently, in the 11th stage instruction in the code, pre-existing issues are
> propagated in the report like so:
> 
> <snip>
> 
> CRITICAL RULE: If a finding is flagged as pre-existing (`\"preexisting\":
> true`), you MUST explicitly state in your inline comment that this issue is
> pre-existing and was not introduced by the patch under review. Use phrasing like
> \"This isn't a bug introduced by this patch, but...\" or \"This is a
> pre-existing issue, but...\" to start the comment.
> 
> </snip>
> 
> Instead of them replying inline with "This isn't a bug introduced by this patch,
> but by..." or "This is a pre-existing issue, but does..." how would you like to
> see this information presented/conveyed?
> 
> I created an issue here [2]
>
> [2] https://github.com/sashiko-dev/sashiko/issues/381

"To a separate section" is OK.

I wouldn't want to lose the pre-existing issue reporting.  I don't see
that an author is obligated to address these things (although they
often do).  But I like to see that the pre-existing things are brought
to the official maintainer's attention.


And yes, it's all very disorganized and error-prone.  How nice would it
be for Maintainer to think "hm, I have a few hours to spare - what bug
reports are there against my stuff".  Then click on a link.

Re: Full subsystem codebase scan for identifying pre-existing issues?

[Roman Gushchin]

Andrew Morton <akpm@linux-foundation.org> writes:

> On Wed, 29 Jul 2026 21:43:26 -0400 Derek Barbosa <debarbos@redhat.com> wrote:
>
>> Hi,
>> 
>> > Currently, they're flagged on patch reviews, which can be
>> > a bit frustrating for contributors who didn't introduce the problem,
>> > and adds a bit of noise for maintainers/reviewers looking at the
>> > Sashiko report.
>> 
>> Unfortunately, you aren't the first to report a problem with these.
>> 
>> Currently, in the 11th stage instruction in the code, pre-existing issues are
>> propagated in the report like so:
>> 
>> <snip>
>> 
>> CRITICAL RULE: If a finding is flagged as pre-existing (`\"preexisting\":
>> true`), you MUST explicitly state in your inline comment that this issue is
>> pre-existing and was not introduced by the patch under review. Use phrasing like
>> \"This isn't a bug introduced by this patch, but...\" or \"This is a
>> pre-existing issue, but...\" to start the comment.
>> 
>> </snip>
>> 
>> Instead of them replying inline with "This isn't a bug introduced by this patch,
>> but by..." or "This is a pre-existing issue, but does..." how would you like to
>> see this information presented/conveyed?
>> 
>> I created an issue here [2]
>>
>> [2] https://github.com/sashiko-dev/sashiko/issues/381
>
> "To a separate section" is OK.
>
> I wouldn't want to lose the pre-existing issue reporting.  I don't see
> that an author is obligated to address these things (although they
> often do).  But I like to see that the pre-existing things are brought
> to the official maintainer's attention.
>
>
> And yes, it's all very disorganized and error-prone.  How nice would it
> be for Maintainer to think "hm, I have a few hours to spare - what bug
> reports are there against my stuff".  Then click on a link.

As Chris mentioned in the thread, we're working on it: a dashboard
of corresponding issues available to all maintainers.

Thanks

Re: Full subsystem codebase scan for identifying pre-existing issues?

[Chris Mason]

On Wed, Jul 29, 2026 at 9:43 PM Derek Barbosa <debarbos@redhat.com> wrote:
>
> Hi,
>
> On Wed, Jul 29, 2026 at 01:34:10PM -0700, Joanne Koong wrote:
> > Hi,
> >
> > Is there a way for Sashiko to run a full scan of a subsystem/tree to
> > identify all the pre-existing issues so that they can be fixed
> > proactively?
>
> At the moment, no. At least nothing that I am aware of :)
>
> There have been some small conversations about such functionality, but nothing
> has really taken shape yet. One of the concerns was trying to validate the sheer
> number of potential-false-positives, and how to present such information in an
> actionable manner. IIRC syzbot/syzkaller's way of storing such data was used as
> a comparison point here.
>
> FWIW, Chris Mason has a repository for a tool that runs such scans on codebases
> -- paired with tools like semcode. His prompts are the same ones that we
> frequently sync back to Sashiko. It won't solve the other point you made, but it
> may be worth looking into. [0][1]

Hi everyone,

Joanne was an early tester of the review prompts last output fall, and
also of the kres scans (thanks Joanne!).  Unfortunately (fortunately?)
I haven't made much progress in June/July because I was out on PTO,
but I'm ramping back up now.  I had promised Roman I'd try and
integrate with his databases, and I'm picking all of that up again
now.

As Derek mentioned, it won't really solve the question of how sashiko
should present existing issues, but hopefully we can knock off some of
the rough edges.

-chris

Re: Full subsystem codebase scan for identifying pre-existing issues?

[Joanne Koong]

Hi Derek,

On Wed, Jul 29, 2026 at 6:43 PM Derek Barbosa <debarbos@redhat.com> wrote:
>
> Hi,
>
> On Wed, Jul 29, 2026 at 01:34:10PM -0700, Joanne Koong wrote:
> > Hi,
> >
> > Is there a way for Sashiko to run a full scan of a subsystem/tree to
> > identify all the pre-existing issues so that they can be fixed
> > proactively?
>
> At the moment, no. At least nothing that I am aware of :)
>
> There have been some small conversations about such functionality, but nothing
> has really taken shape yet. One of the concerns was trying to validate the sheer
> number of potential-false-positives, and how to present such information in an

Gotcha, thanks for the context :) That makes sense, that sounds tough
balancing potential false positives.

Just thinking out loud here... if those false positives are currently
getting reported/flagged by Sashiko on contributor submissions
anyways, maybe it'd still be helpful for them to show up in the
subsystem codebase scan so maintainers can course-correct the rules
for them in Sashiko (and hopefully save contributors not as familiar
with the subsystem some confusion :)).

> actionable manner. IIRC syzbot/syzkaller's way of storing such data was used as
> a comparison point here.
>
> FWIW, Chris Mason has a repository for a tool that runs such scans on codebases
> -- paired with tools like semcode. His prompts are the same ones that we
> frequently sync back to Sashiko. It won't solve the other point you made, but it

Awesome! I'm a huge fan of Chris's work. I wasn't sure if there were
differences in the analysis between Sashiko and Chris's tools, but
it's useful to know that the prompts are synced from Chris's repo.
Anecdotally, when I ran a couple of my series through Chris's AI
systems a couple months ago, there were some things it pointed out
that Sashiko didn't and vice versa, but I think that's probably more
due to the underlying LLM being used (eg claude vs gemini).

> may be worth looking into. [0][1]
>
> > Currently, they're flagged on patch reviews, which can be
> > a bit frustrating for contributors who didn't introduce the problem,
> > and adds a bit of noise for maintainers/reviewers looking at the
> > Sashiko report.
>
> Unfortunately, you aren't the first to report a problem with these.
>
> Currently, in the 11th stage instruction in the code, pre-existing issues are
> propagated in the report like so:
>
> <snip>
>
> CRITICAL RULE: If a finding is flagged as pre-existing (`\"preexisting\":
> true`), you MUST explicitly state in your inline comment that this issue is
> pre-existing and was not introduced by the patch under review. Use phrasing like
> \"This isn't a bug introduced by this patch, but...\" or \"This is a
> pre-existing issue, but...\" to start the comment.
>
> </snip>
>
> Instead of them replying inline with "This isn't a bug introduced by this patch,
> but by..." or "This is a pre-existing issue, but does..." how would you like to
> see this information presented/conveyed?

I (like Andrew) also find the pre-existing issue reporting really
useful and definitely wouldn't want to lose them. Maybe something that
could help is if they were marked even more obviously as pre-existing
issues? (eg prefaced with a bold "PRE-EXISTING ISSUE - " or displayed
in a different font color or enclosed within parentheses). I usually
look at the web ui and sometimes when glancing at a series (eg [1]),
it would be faster to differentiate between bugs I caused vs
pre-existing ones if they were visually marked differently. This is
not a big deal at all though. Thanks for your work on Sashiko!

Thanks,
Joanne

[1] https://sashiko.dev/#/patchset/20260715174305.336261-1-joannelkoong%40gmail.com

Re: Full subsystem codebase scan for identifying pre-existing issues?

[Derek Barbosa]

Hi Joanne,

On Thu, Jul 30, 2026 at 05:34:10PM -0700, Joanne Koong wrote:
> Hi Derek,
> 
> On Wed, Jul 29, 2026 at 6:43 PM Derek Barbosa <debarbos@redhat.com> wrote:
> >
> > Hi,
> >
> > On Wed, Jul 29, 2026 at 01:34:10PM -0700, Joanne Koong wrote:
> > > Hi,
> > >
> > > Is there a way for Sashiko to run a full scan of a subsystem/tree to
> > > identify all the pre-existing issues so that they can be fixed
> > > proactively?
> >
> > At the moment, no. At least nothing that I am aware of :)
> >
> > There have been some small conversations about such functionality, but nothing
> > has really taken shape yet. One of the concerns was trying to validate the sheer
> > number of potential-false-positives, and how to present such information in an
> 
> Gotcha, thanks for the context :) That makes sense, that sounds tough
> balancing potential false positives.

We're actively working on trying to get more data (such as semcode) to act as
some sort of backing "evidence" of a finding. That, in theory, should at least
filter out the totally bogus claims. But it is still all a WIP.

> 
> Just thinking out loud here... if those false positives are currently
> getting reported/flagged by Sashiko on contributor submissions
> anyways, maybe it'd still be helpful for them to show up in the
> subsystem codebase scan so maintainers can course-correct the rules
> for them in Sashiko (and hopefully save contributors not as familiar
> with the subsystem some confusion :)).

I see what you mean. I think there may be a way to reach a "good enough"
solution in the interim: https://github.com/sashiko-dev/sashiko/issues/385

> 
> > actionable manner. IIRC syzbot/syzkaller's way of storing such data was used as
> > a comparison point here.
> >
> > FWIW, Chris Mason has a repository for a tool that runs such scans on codebases
> > -- paired with tools like semcode. His prompts are the same ones that we
> > frequently sync back to Sashiko. It won't solve the other point you made, but it
> 
> Awesome! I'm a huge fan of Chris's work. I wasn't sure if there were
> differences in the analysis between Sashiko and Chris's tools, but
> it's useful to know that the prompts are synced from Chris's repo.
> Anecdotally, when I ran a couple of my series through Chris's AI
> systems a couple months ago, there were some things it pointed out
> that Sashiko didn't and vice versa, but I think that's probably more
> due to the underlying LLM being used (eg claude vs gemini).

Right. From what I've seen, different harnesses (claude code, opencode,
gemini-cli, etc) have their own set of underlying prompts and housekeeping logic
that result in an interesting set of permutations when you throw custom prompts
and tool calls in the mix. Sashiko would be an example of a harness (one
tailored to work in a more "distributed" fashion) that calls out to Gemini in
the current "prod" instance).

FWIW I have also seen considerable differences in said prompts across different
harnesses, etc.

> 
> > may be worth looking into. [0][1]
> >
> > > Currently, they're flagged on patch reviews, which can be
> > > a bit frustrating for contributors who didn't introduce the problem,
> > > and adds a bit of noise for maintainers/reviewers looking at the
> > > Sashiko report.
> >
> > Unfortunately, you aren't the first to report a problem with these.
> >
> > Currently, in the 11th stage instruction in the code, pre-existing issues are
> > propagated in the report like so:
> >
> > <snip>
> >
> > CRITICAL RULE: If a finding is flagged as pre-existing (`\"preexisting\":
> > true`), you MUST explicitly state in your inline comment that this issue is
> > pre-existing and was not introduced by the patch under review. Use phrasing like
> > \"This isn't a bug introduced by this patch, but...\" or \"This is a
> > pre-existing issue, but...\" to start the comment.
> >
> > </snip>
> >
> > Instead of them replying inline with "This isn't a bug introduced by this patch,
> > but by..." or "This is a pre-existing issue, but does..." how would you like to
> > see this information presented/conveyed?
> 
> I (like Andrew) also find the pre-existing issue reporting really
> useful and definitely wouldn't want to lose them. Maybe something that
> could help is if they were marked even more obviously as pre-existing
> issues? (eg prefaced with a bold "PRE-EXISTING ISSUE - " or displayed
> in a different font color or enclosed within parentheses). I usually
> look at the web ui and sometimes when glancing at a series (eg [1]),
> it would be faster to differentiate between bugs I caused vs
> pre-existing ones if they were visually marked differently. This is
> not a big deal at all though. Thanks for your work on Sashiko!

I'll take note of that and try to run an experiment or two with your
suggestions. Changing prompts can feel like alchemy at times! :)

> 
> Thanks,
> Joanne
> 
> [1] https://sashiko.dev/#/patchset/20260715174305.336261-1-joannelkoong%40gmail.com
> 

Thanks for the suggestions,

-- 
Derek <debarbos@redhat.com>

Re: Full subsystem codebase scan for identifying pre-existing issues?

[Roman Gushchin]

Joanne Koong <joannelkoong@gmail.com> writes:

> Hi Derek,
>
> On Wed, Jul 29, 2026 at 6:43 PM Derek Barbosa <debarbos@redhat.com> wrote:
>>
>> Hi,
>>
>> On Wed, Jul 29, 2026 at 01:34:10PM -0700, Joanne Koong wrote:
>> > Hi,
>> >
>> > Is there a way for Sashiko to run a full scan of a subsystem/tree to
>> > identify all the pre-existing issues so that they can be fixed
>> > proactively?
>>
>> At the moment, no. At least nothing that I am aware of :)
>>
>> There have been some small conversations about such functionality, but nothing
>> has really taken shape yet. One of the concerns was trying to validate the sheer
>> number of potential-false-positives, and how to present such information in an
>
> Gotcha, thanks for the context :) That makes sense, that sounds tough
> balancing potential false positives.
>
> Just thinking out loud here... if those false positives are currently
> getting reported/flagged by Sashiko on contributor submissions
> anyways, maybe it'd still be helpful for them to show up in the
> subsystem codebase scan so maintainers can course-correct the rules
> for them in Sashiko (and hopefully save contributors not as familiar
> with the subsystem some confusion :)).

This is in work too and I'm even somewhat optimistic that in many cases
Sashiko can generate these prompts automatically based on human
feedback. The tricky part is that human are often wrong too, so Sashiko
can't blindly trust human feedback either.

Re: Full subsystem codebase scan for identifying pre-existing issues?

[Roman Gushchin]

Joanne Koong <joannelkoong@gmail.com> writes:

> Hi,

Hi Joanne!

> Is there a way for Sashiko to run a full scan of a subsystem/tree to
> identify all the pre-existing issues so that they can be fixed
> proactively? Currently, they're flagged on patch reviews, which can be
> a bit frustrating for contributors who didn't introduce the problem,
> and adds a bit of noise for maintainers/reviewers looking at the
> Sashiko report.

Well, I doubt there is a completely non-frustrating path here and the
current way of doing things is clearly working in a sense that there are
over 1000 upstream fixes now mentioning Sashiko. Somebody needs to spend
time on these findings and nobody obviously has this time freely
available.

That said, as Chris said, we're working on a more systematic way to
handle these findings, I hope to roll it out relatively soon.

Thanks