Re: [ANNOUNCE] A Syzbot-style Platform for Verifying and Fixing LLM-reported Bugs

[Guenter Roeck <linux@roeck-us.net>]

On 8/30/26 04:55, Yuan Tan wrote:
> 
> Hi all,
> 
> A month ago, I posted an RFC[1] to the mailing list proposing an automated
> platform that validates AI-reported bugs and prepares draft fixes, and
> later discussed the idea at the Netdev conference. After further
> development, it is finally ready.
> 
> While preparing to send this email, I noticed that Roman has since started
> a related discussion:
> [MAINTAINERS SUMMIT] The place of AI code review in the Linux Kernel process
> 
> It turns out this platform already addresses several of the needs raised
> there.
> 
> The platform is available at:
> https://bugtracker.nebusec.ai
> 
> It is currently hosted under my company's domain for convenience. I would
> prefer to move it to a neutral, community-oriented domain once the project
> name is settled.
> Access is currently restricted to maintainers whose email addresses are
> listed in the Linux kernel MAINTAINERS file.
> For now, only bug reports from the net subsystem have been fully imported
> and processed.
> 
> -------------------------------------------------------------------------
> 
> LLM-powered tools such as Sashiko and Claskiko produce a significant number

Is "Claskiko" misspelled ? I don't find a reference to it.

> of false positives. Pre-existing bugs uncovered by these tools are also not
> collected in one place and they remain scattered across individual review
> reports.
> 
> To address this, like syzbot for fuzzer-found bugs, this platform provides
> an **automated** tracking layer for AI-reported bugs, but goes further by
> generating PoCs, running them in QEMU to produce crash logs, triaging
> severity, and drafting patches. This requires no extra effort from
> maintainers; instead, it may helps them understand and fix these bugs more
> efficiently.
> 
> 
> The platform offers the following capabilities:
> 
> 1. Collect and Deduplicate
> 
> The platform ingests bug reports from multiple sources, including Sashiko,
> Claskiko, and others. It deduplicates them and monitors mailing lists and
> git history to track whether they have been fixed.
> 
> It also provides visibility into the Sashiko/Claskiko ingestion queue, so
> users can see which reports are waiting to be collected and processed.
> 
> 
> 2. Verify by generating PoC and running it in QEMU
> 
> An agent attempts to generate a proof-of-concept for each bug to determine
> whether it is a false positive. According to paper Patch-to-PoC[2] and
> follow-up research, GPT-5.4 achieves up to a 95% success rate in generating
> PoCs for genuinely exploitable bugs.
> 
> This makes PoC generation a strong signal: if a bug has no PoC, it is very
> likely a false positive. And even in cases where a real bug is missed, the
> difficulty of generating a PoC suggests it is unlikely to be practically
> exploitable.
> 

I think it is a mistake to claim that "Not exploitable / No PoC --> false
positive". Not all bugs are vulnerabilities, much less exploitable ones.

Guenter

> 
> 3. Draft Patch
> 
> The platform produces a draft patch to give maintainers a starting point
> and suggested fix direction. These still require human review. Patches can
> be downloaded via b4 am.
> 
> These patches are not sent to mailing lists to avoid adding AI-generated
> noise.
> 
> 
> 4. Triage
> 
> Based on the PoC, the agent evaluates the conditions required to trigger
> the bug, for example, whether it requires a namespace, root privileges, or
> can be triggered by an unprivileged user.
> 
> Bugs that require root are generally less important, while those reachable
> by unprivileged users are more likely to have real security impact.
> 
> If a bug can be triggered from namespace, it is still worth
> paying attention to.
> In particular, a bug should not be considered root-only merely because
> triggering it requires CAP_NET_ADMIN in a network namespace. On systems
> that allow unprivileged user namespaces, an ordinary user may be able to
> create a user namespace, create a network namespace owned by it, and obtain
> capabilities such as CAP_NET_ADMIN with respect to that namespace. Such
> bugs may therefore still be reachable by an otherwise unprivileged local
> user.
> 
> For example, this configuration has historically been available by default
> on distributions such as Ubuntu 22.04 LTS and earlier.
> 
> Jamal previously offered some suggestions on this severity classification
> scheme which I have not yet had time to implement; that will come in a
> future update.
> 
> 
> 5. Chat with agent
> 
> Each bug page includes a chat interface for discussing the bug and its fix
> with the agent.
> 
> 
> 
> Based on earlier feedback, the system is not fully public. Only email
> addresses listed in the MAINTAINERS file are eligible to register, to
> prevent bugs with potential security impact from being exposed publicly.
> 
> Jason’s idea of delegating fixes could also be implemented on this
> platform. This is essentially what my volunteer bug-fixing team and I have
> been doing over the past six months: anyone interested can pick up an issue
> and try to fix it.
> Each subsystem’s maintainers could choose whether to make its issues
> public. Making them public would also let potential reporters check whether
> an issue is already known before submitting a new report.
> 
> Bugs are also categorized by subsystem, so after logging in, maintainers
> see only the bugs relevant to the modules they maintain.
> 
> 
> Welcome any suggestions:)  I will continue maintaining this system and
> adding more features, not only out of personal interest, but also our
> bug-fixing volunteer team is using it too.
> 
> We periodically burn tokens and run state-of-the-art models against the
> full kernel source code, with the goal of finding security vulnerabilities
> before attackers do. While I am not an expert in the net subsystem and
> cannot review patches myself at this time, I still hope this platform can
> be of help to the community.
> 
> If this system proves genuinely useful, I am happy to transfer project
> ownership to the Linux Foundation or another neutral host.
> 
> Going forward, the platform will expose a public API so that other bug
> finding research teams can submit their findings here for centralized
> processing. We also plan to ingest syzbot-found bugs to provide them with
> the same triage workflow.
> 
> P.S. I have been struggling to come up with a good name for this platform. A
> few candidates I am considering are Palomar, Tengu, FixArc, and Ephemeris.
> If anyone has a preference or a better suggestion, I would love to hear it.
> 
> 
> Current Limitations
> 
> - For a tracked bug, the system currently only knows that a fix exists; it
> does not yet distinguish between a patch that has been posted to the
> mailing list and one that has already been merged.
> 
> - PoC generation and false-positive verification are not yet supported for
> driver-related bugs.
> 
> - Unable to scrape Sashiko/Clashiko review reports that are still under embargo.
> 
> - Only net subsystem bugs from Sashiko and Claskiko have been imported with
> a fully automated fix-detection pipeline so far. Bugs from other subsystems
> are shown but may already be fixed. If other subsystem maintainers are
> interested, I will prioritize adding support.
> 
> 
> [1] https://lore.kernel.org/all/20260708092247.4188498-1-yuantan098@gmail.com/
> [2] Juefei Pu, Xingyu Li, Zhengchuan Liang, et al. "Patch-to-PoC: A
> Systematic Study of Agentic LLM Systems for Linux Kernel N-Day
> Reproduction." arXiv:2602.07287, 2026. https://arxiv.org/abs/2602.07287
> 
> 
> Thanks,
> Yuan