[ANN] Sashiko reviews for Linux media

[ANN] Sashiko reviews for Linux media

[Mauro Carvalho Chehab]

Hi all,

1. Introduction

There have been lots of discuss and efforts over the last years with regards
to improving the quality of media patches.

In the past, Mauro had my own set of scripts to check for regressions and 
do media compiling per patch while reviewing submitted patches. Later on,
Hans came up with his set of scripts as well. Both approaches rely not
only on kernel compilation but they also run two static analyzer tools:
sparse and smatch.

Media ML also receives since a long time ago mails from bots like Syzbot, 
kernel test robot - and there were some other bots as well that sent us 
reviews in the past.

We also added the first CI using Jenkins, which we still use for userspace
tools.

There was also Coverity which is a paid tool which usage is free for
open source developers, which provides more sophisticated analysis of
the Kernel code. Its last run was 8 months ago, so probably nobody is
using it anymore.

Starting on Jan, 2024, Ricardo started writing the media-ci code which is
now used to review all Linux Kernel media patches and merges.

There is a common pattern on *all* such tools: they produce false
positives. Last week we just addressed one such issue on media-CI:
if, for whatever reason, a CI job is interrupted (system error), it
was reporting it the same way as if the patch had a high relevance
issue.

Btw, media-ci also have on its tree a list of false-positive issues
to be ignored, which needs to be reviewed and eventually updated on
every new kernel release.

So, 2,5 years after the start of process, we're still fine-tuning
media CI to reduce false-positives. There have been more than 300 merge
requests for Media-CI since v0, and the rate if MR is not slowing down.

That's perfectly fine, as having bots doing their work properly requires 
hard work. 

Ricardo and other media-ci contributors: thank you for your work there!

So, maybe despite common sense, all CI bots have issues and can
generate false positives. In doubt, authors should use their own good
sense, and, in doubt, check with a more experienced developer to
double-check CI findings.

2. Sashiko

Maybe to cover the space left by Coverity, there is now a new
project aiming to do some static analysis of the kernel code
using LLM. Version 0.1.0 is from March, 2026, so this is
a brand new approach. As such, it can also generate false
positives, and, more importantly, as it is a new tool, it
will certainly need some time to become mature.

One of the strengths of using static analysis tools is that they
are usually better to review error paths that may slip on normal
reviews.

With LLM-based tools, the hardest part is to have a set of
prompts that teach the tool what it should produce, what kind
of reviews are important and what it can consider as false
positives.

During the last Media Summit, we reached a consensus about
start testing it. There was some doubts about who should be
receiving Sashiko's return. So, there was an inquiry with
Media Summit participants later on to double check how
Sashiko tests would work on media. There was non consensus,
so we opted to be conservative: Sashiko's e-mails will be
sent to media-ci@linuxtv.org mailing list without c/c the
patch author. Our patchwork instance is picking such e-mails
and adding them as checks at https://patchwork.linuxtv.org.

Subscribing to it can be done via mailman3 Postorious interface:
	https://linuxtv.org/mailman3/postorius/lists/media-ci.linuxtv.org/

(to avoid SPAM, only members can post to it without moderation)

Archives are available at:
	https://linuxtv.org/mailman3/hyperkitty/list/media-ci@linuxtv.org/
and:
	https://sashiko.dev/#/?list=org.kernel.vger.linux-media

Yesterday, Sashiko v.0.2.5 was tagged(*), containing IMHO two 
relevant set of changes:

1. Initial support for linux-media was added. This includes
   a new prompt meant to cover media specifics:
	https://github.com/sashiko-dev/sashiko/blob/main/third_party/prompts/kernel/subsystem/media.md
   Those prompts were generated after a feedback we gave to
   Sashiko maintainers, reflected on this issue:

	https://github.com/sashiko-dev/sashiko/issues/334

2. It should be possible now to use Ollama to run it locally
   using open-weight models. 
   This may help media developers to test how changes at the 
   media prompt may affect the tool. Still, running it locally
   takes maybe 10-20 minutes per patch, and most models are bad
   honoring JSON format, which is currently needed by the Ollama
   integration.

(*) https://github.com/sashiko-dev/sashiko/releases/tag/v.0.2.5

With that in place, we guess we can now start testing the tool.

For such purpose, we kindly ask all contributors (patch authors and
reviewers) to subscribe to media-ci@linuxtv.org, check Sashiko output
and in the event of finding false positives share with media-ci ML
so we can improve the prompts or create a new issue at:

	https://github.com/sashiko-dev/sashiko/issues

This way, Sashiko developers can work to improve linux-media
specifics.

Btw, the same applies to media CI: if you find false positive
issues there, feel free to open/update issues at:

	https://gitlab.freedesktop.org/linux-media/media-ci/-/work_items

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Hans Verkuil <hverkuil@kernel.org>
Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>