Re: Stop false review statements

[Roman Gushchin <roman.gushchin@linux.dev>]

> On May 17, 2026, at 9:40 AM, Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> 
> On Sun, 17 May 2026 12:12:00 +0200
> Greg KH <gregkh@linuxfoundation.org> wrote:
> 
>>> On Sun, May 17, 2026 at 12:05:56PM +0200, Mauro Carvalho Chehab wrote:
>>> On Sat, 16 May 2026 14:59:44 -0700
>>> Roman Gushchin <roman.gushchin@linux.dev> wrote:
>>> 
>>>>> On May 16, 2026, at 2:33 PM, Krzysztof Kozlowski <krzk@kernel.org> wrote:
>>>>> 
>>>>> I find it opposite: clogging commits with useless information, because
>>>>> some arbitrary and completely closed-source tool did analysis means
>>>>> nothing to me one year later when I look at the commit in the Git history.    
>>>> 
>>>> This is simple not true: Sashiko is fully open-source, under Apache 2.0 license
>>>> and the code belongs to LF.   
>>> 
>>>> Yes, the instance behind sashiko.dev is using
>>>> Gemini 3.1 Pro LLM, which is not open-source, but it’s not a fundamental limitation -
>>>> Sashiko is supporting various LLMs, including open models - it’s just a practical
>>>> choice: to my knowledge the quality of open models is not on par with frontier closed
>>>> models   
>>> 
>>> I would very much prefer using an open source LLM, even if not in pair
>>> with latest paid models.
>>> 
>>>> and it would require a non-trivial amount of hardware and infrastructure to run
>>>> an open model at the required scale.  
>>> 
>>> IMHO the best would be to have them running on some infra that would accept
>>> open source models (*). If there aren't enough resources to have our own
>>> infra, there are offers out there which allows running open source models
>>> like https://ollama.com/pricing (I never used myself).
>>> 
>>> (*) For instance, Qwen3.6 is brand new and licensed under apache-2.0.
>>>    Not bad on my tests running it locally.  
>> 
>> You can run the tool locally, with whatever model you want, if you want
>> to.
>> 
>> But for now, let's just take the free credits that Google is willing to
>> throw at this thing and let it give us reviews IF the maintainer of the
>> subsystem feels it is something they want to do.  No one is forcing
>> maintainers to do this.
> 
> If Google and/or others are willing to give free credits on their cloud,
> they could instead or in addition give free credits to run ollama
> there, allowing us to use different models.
> 
> From my side, while I won't personally object getting reviews from
> Sashiko/Gemini, this is something I can't reproduce locally. I would
> very much want something where I can select my LLM preferred model
> and run on my ollama docker container on my own GPU, in a way that
> I could run it locally before even sending a patch series.

2 thoughts here:
1) I actually tried to run it with ollama on my personal framework 13. Adding nominal support is trivial,
but the whole thing is not really useful: I can get maybe few hundreds tokens per second using
a quantified model with reduced quality; an average sashiko review is consuming 3.5 millions tokens
(with Gemini 3.1 pro, it’s also model-dependent).
I’m personally all in on having the entire thing as open as possible and I believe Sashiko is what 
is realistically the best at this moment - a fully open-source harness and set of prompts which 
can work with a variety of models.
I’m happy to merge a support for any LLM model which can produce decent review results.

2) Due to probabilistic nature of LLMs, nothing is reproducible in a strict sense of the word.
Even with exactly the same model/harness/prompts you’ll get different results every time you run it.
It’s unfortunate, but it is what it is at the moment.