Re: Stop false review statements

["Theodore Tso" <tytso@mit.edu>]

On Mon, May 18, 2026 at 11:04:29AM +0300, Jani Nikula wrote:
> > Sashiko is supporting various LLMs, including open models - it’s just a practical
> > choice: to my knowledge the quality of open models is not on par with frontier closed
> > models and it would require a non-trivial amount of hardware and infrastructure to run
> > an open model at the required scale.
> 
> In the context of the "Reviewed-by: Sashiko" discussion, this actually
> makes it really hard to assess the quality of those reviews.

Agreed.  There's a reason why the coding-assistants.rst specifies the
model which is used:

  Assisted-by: AGENT_NAME:MODEL_VERSION [TOOL1] [TOOL2]

The problem is that (as Jon has pointed out) coding-assistants.rst was
intended for use when the tool was beging used to help create the code
--- that is, "Coding Assistants".  What we're doing here is more of a
reviewer assistance.  Something like:

  Scanned-by: AGENT_NAME:MODEL_VERSION [TOOL1] [TOOL2]

Would be more interesting, but it doesn't actually tell us anything
about what the results were of the scan.

One of the problems here is that there is a distinction between the
infrastructure and review prompts in the Sashiko github repository,
and the reviews that are being published by Sashiko the web service
being run by Google that is being lost by some folks.  So I wonder if
for now, we should just do something like:

Link: https://sashiko.dev/#/patchset/20260515091829.194810-1-me%40linux.beauty

Or just have a link to lore where the review has responded to the
Sashiko review stating where the Sashiko review reported a
pre-existing condition (perhaps one that we don't care about because
races in readahead logic is really Not A Big Deal, etc.)  We go for
this strategy, it would actually be better for the Shashiko.dev review
to get cc'ed to the mailing list.

Personally, I think that's probably be best way to go.  We already
don't insert into the git commit an explanation of why some bullsh*t
review by some wannabe human reviewer should be ignored, or why a
discussion of some problem discovered by a human review in the source
of the review would be handled in a future patch set.  That's what the
discussion on lore.kernel.org is for.  And we shouldn't treat AI
reviews any different from how we deal with human reviews.  So if we
want to give credit to an AI review, then let's go with the
Scanned-by.  Or we can just let people look at the mailing list, and
if people want to have statistics, we can ask people to use a script
running against public inbox to figure things out.

						- Ted