Re: [RFC] Mitigating Sashiko-driven iteration noise on the main MPTCP list

[Shardul Bankar <shardulsb08@gmail.com>]

Hi Matt, Gang,

On 04/09/2026 12:32, Matthieu Baerts wrote:
> So at the end, it seems like a good thing to have access to various
> models to cross checks the reviews.

Agreed, and that is where I ran into something worth passing on. If the
plan is for people to run reviews locally, and more than one model at
that, then what a local run costs decides whether anyone actually keeps
doing it.

I ran Sashiko locally against my Claude subscription, using the Claude
Code CLI provider rather than an API key. One patch took about 45
minutes and consumed essentially a whole usage window. That window is
five hours, so it works out at roughly one patch per window on that
setup, which was enough to have me scheduling runs around the reset
rather than just running them when I wanted a review. That timing was in
August, on a build from before the declarative workflow engine landed,
so I cannot say the 45 minutes still holds. What has not changed is the
shape of it: the CLI provider still starts a separate claude process per
stage, with no session carried between them, so the context is built and
sent again for each one.

The part I did not know at the time is that there are two quite
different things here. "sashiko init" then "sashiko review" runs the
full staged pipeline, one process per stage, and that is the expensive
one. The Sashiko repository also ships, in the prompt bundle it vendors,
a /kreview command that runs the review as a single Claude Code session,
so the context is read once instead of once per stage.

/kreview is not mentioned in Sashiko's README, its maintainers guide, or
anything under docs/. It is documented only in
third_party/prompts/README.md, inside the vendored copy of the review
prompts, with the command itself in
third_party/prompts/kernel/slash-commands/kreview.md. So anyone
following the setup instructions gets the expensive path and has no
particular reason to learn the other one exists. That seems worth fixing
if the goal is more people running checks before they post.

A few of the things that make a local run costlier than it needs to be
look fixable, and I have sent patches for them to the Sashiko project
rather than describe them here, since that is where they belong. One is
worth a line because of where it does and does not apply: Sashiko can
cache model responses so a repeated request is answered from disk rather
than sent again, but that setting has no effect on the local path, so
retrying a failed run pays for everything a second time. The reviews you
see arriving on this list are unaffected, since the daemon that posts
them does use the cache.

One other thing, unrelated to cost and possibly of wider interest. Of
the findings Sashiko has published for this list, the ones it marks as
pre-existing are graded higher than the ones it attributes to the patch
under review. Roughly 95 percent of the pre-existing ones are High or
Critical and not one is Low, against roughly half for the findings
attributed to the patch. I had been treating "this was already broken"
as a reason to downgrade, and that is not how the tool treats it. The
reasoning holds up, since a pre-existing bug is live in the shipping
kernel while one introduced by a patch under review will most likely be
caught before it merges.

Gang, you mentioned deploying Sashiko locally. Did you see anything like
the same cost, and what were you running it against? If this turns out
to be particular to a fixed-quota subscription, I would rather find that
out than have people avoid running reviews locally over a cost that does
not apply to them.

Thanks,
Shardul