Re: How to fix problems with the sashiko review model

[Roman Gushchin <roman.gushchin@linux.dev>]

> On Jul 22, 2026, at 8:10 AM, James Bottomley <James.Bottomley@hansenpartnership.com> wrote:
> 
> On Wed, 2026-07-22 at 06:43 -0700, Guenter Roeck wrote:
> [...]
>> For Sashiko, fork and clone
>> https://github.com/sashiko-dev/sashiko.git,
>> submit your changes into the clone, and send a pull request. Or at
>> least that is how I do it.
>> 
>> In the Sashiko source, per-subsystem prompts are located in directory
>> third_party/prompts/kernel/subsystem.
> 
> OK, now I'm confused.  I thought
> 
> https://github.com/masoncl/review-prompts
> 
> was the source of truth for this, but that project seems to have been
> copied into sashiko rather than pulled in as a module.  If you want to
> improve the review prompts do you now have to submit to both (which is
> getting to be a bit wasteful)?

It’s a bit messy now, but Chris and me are working on something more sane.
As of now, Sashiko is using per-subsystem prompts from Chris’s repository
with some changes around more generic prompts. I periodically pull updates
from Chris’s repository. Also several folks sent prompts changes directly to
Sashiko repository, Idk if Chris is merging them back, probably not.

I also experiment with auto-generating prompts based on human feedback
to sashiko reviews, initial results are quite promising.

So as of now, I’d say send a pull request to Chris or me directly, we’ll figure out
a better model going forward.

The hard part is testing: it’s quite easy to confuse a LLM with an unlucky prompt change.
We need a stronger guidance and maybe LLM review for prompts too (sashiko for sashiko?!).
E.g. Sashiko implements a much smaller set of tools (intentionally) than a generic coding agent,
so if a prompt contains something like “run checkpatch.pl” or “search lore for …” it might be
very confusing to the LLM. When I’m pulling significant prompt updates from Chris’s repository,
I run the full benchmark to make sure we don’t get regressions.

Thanks!