Re: [RFC] VEGA: a syzbot-like workflow for LLM-found kernel bugs

[Laurent Pinchart <laurent.pinchart@ideasonboard.com>]

On Wed, Jul 08, 2026 at 02:22:47AM -0700, Yuan Tan wrote:
> Hi all,
> 
> We would like to ask for feedback on a proposed workflow for reporting Linux
> kernel bugs found by an LLM-assisted code auditing tool that we have
> been developing since earlier this year.
> 
> Since February, we have been developing an LLM-driven kernel code auditing
> tool called VEGA. It started as a side project, but the results became much
> substantial than we expected: VEGA has found hundreds of valid bugs in Linux
> kernel.
> 
> That immediately created a practical problem: we do not want to dump a large
> pile of bug reports onto mail lists and annoy the maintainers.
> 
> The first thing we tried was to fix as many as we could ourselves. We
> started working with a group of student volunteers. Most of them are
> college students, so we have been training them, reviewing their patches,
> and trying to build an internal review process before anything is sent to
> the mailing list. The goal is to turn these findings into useful fixes, and
> also to help new contributors grow into people who can reduce maintainer
> workload instead of adding to it.
> 
> The process was not perfect. Some patches were not good enough, and we also
> made some mistakes early on when deciding what should be called a security
> issue.  Our internal review process has been improving with the help of the
> community.
> 
> Since March, we picked up non-root triggerable bug first and have worked on
> fixes for more than 100 validated kernel bugs. we especially want to thank
> the students and professor who have helped a lot with this effort.
> 
> But the remaining queue is still too large for us to handle.
> 
> Recently Jamal pointed out problems around our tags. That made me realize
> that we should probably stop treating this as an ad-hoc patch effort and
> build something closer to syzbot: public, reproducible, trackable,
> deduplicated, and useful to maintainers.
> 
> So this mail is an RFC for a VEGA reporting workflow.
> 
> The rough idea
> ==============
> 
> VEGA would have a public dashboard, similar to syzbot, and would
> send selected bug reports to the relevant kernel mailing lists.
> 
> The goal is to send reports that contain enough information for maintainers
> or other developers to pick up, understand, reproduce and fix the issue.
> 
> For each public report, we expect to include:
> 
>   - a description of the bug
>   - the tested kernel tree and commit
>   - the kernel config and environment
>   - the crash log
>   - a minimized user-space reproducer
>   - the suspected introducing commit
>   - a suggested fix patch
> 
> The suggested fix patch is meant to reduce maintainer burden. It still need
> human review, but hopefully it can save a lot time from building a patch
> from scratch.

Will the information included in the public report (including the
suggested fix) be written by a human or an LLM ? In the latter case I
don't see how you could reasonably claim to reduce maintainer burden, so
that would be a big NACK as far as I'm concerned.

> What will be public
> ===================
> 
> All VEGA findings that we have evaluated as not having major security
> impact can be published on the VEGA dashboard. The dashboard would make it
> possible to see what VEGA found, whether the issue was reproduced, whether
> a fix exists, whether it was reported to a mailing list, and whether it has
> been fixed upstream.
> 
> For issues that we have validated as having possible serious security
> impact, we will not publish it on the public dashboard before going through
> the appropriate kernel security process.
> 
> Dumping everything onto the mailing list may be annoying. During the initial
> stage, reports will be rate-limited and sent manually. We will check for
> duplicates against lore/upstream, and make sure the issue is not already
> fixed or reported.
> 
> Report identity and tags
> ========================
> 
> Each public VEGA report will have a stable identity, similar to
> syzbot reports.
> 
> One possible format is:
> 
>   Reported-by: VEGA <vega+HASH@DOMAIN>
>   Closes: <public dashboard URL>
> 
> =========
> 
> We would like to hear what maintainers think about this before we start
> sending these reports.
> 
> We do not want VEGA to become another source of mailing list noise. The goal
> is to make LLM-based bug finding transparent and useful, and to make sure
> the reports come with enough context, reproducers, suggested fixes, and
> tracking so that they reduce work rather than create more.

-- 
Regards,

Laurent Pinchart