[MAINTAINERS SUMMIT] Deterministic verification gates for security submissions

[MAINTAINERS SUMMIT] Deterministic verification gates for security submissions

[Rajat Gupta]

Hi all,

This builds on Sasha's "Scaling our security process" thread, specifically
point 5: "Can we tell AI assisted quality work from slop without burning out
the people doing triage?" I'd like to propose a concrete answer to that
question.

We discussed this at the netdevconf 0x1A "New Age Tooling" BoF two weeks ago
(chaired by Jamal Hadi Salim). The maintainers in the room were in consensus
that detecting AI involvement is impossible and the wrong goal - what matters
is whether the submission comes with verifiable evidence. The framework I
presented received broad agreement as a practical path forward.

## The Problem

Security submission volume has grown significantly, and the signal-to-noise
ratio has collapsed. Most submissions arrive as prose-only reports:
plausible-sounding RCA, a patch, but no reproducer and no proof the patch was
ever tested. Reviewing these from scratch takes 30-60 minutes each and often
ends with "can't reproduce, closing."

Greg noted in Sasha's thread that LLM-based intake filtering "just doesn't
work" - I agree completely. The answer isn't AI judging AI. The answer is
requiring deterministic, mechanically verifiable evidence from the submitter.
The verification involves no AI.

## Proposed Solution: 4 Verification Gates

Rather than judge prose quality or detect tooling, prioritize submissions by
evidence:

  Gate 1 - Trigger + Impact
    Does a reproducer crash the kernel under a sanitizer?
    Higher impact evidence (controlled corruption, privilege escalation)
    gets higher priority.
    Automatable: YES (build kernel, boot QEMU, run trigger, check output)

  Gate 2 - Root Cause Evidence
    Is there mechanically verifiable evidence (KASAN trace, bpftrace output,
    differential test) showing WHY the bug occurs - not just WHERE it crashes?
    Automatable: PARTIALLY (sanitizer output is automatic; understanding
    causality still needs human judgment)

  Gate 3 - Patch Verification
    Does the trigger crash before the patch and pass after?
    Automatable: YES (two kernel builds, one trigger, compare output)

  Gate 4 - Regression
    Do subsystem selftests pass with the patch applied?
    Automatable: YES (same QEMU environment, run selftests)

Submissions are prioritized by evidence depth. All 4 gates pass -> top of the
queue. Missing a trigger -> bottom of the queue. Not rejected - deprioritized.

## How This Helps

For reviewers: A submission that passes all 4 gates will take less time to
review. The evidence is pre-verified - the reviewer confirms it, not
investigates from scratch. Unverified submissions (prose + patch, no trigger)
still take 30-60 minutes. The gates surface the verified work first.

For submitters: Clear requirements. If you show up with a trigger + RCA trace
+ before/after proof + selftests, your submission gets priority regardless of
whether AI helped you find it. The incentive shifts from "write convincing
prose" to "produce evidence." In effect, we would be encouraging people to
use AI to produce concrete, verifiable evidence.

For the process: 3 of 4 gates are fully automatable. This can run as CI
infrastructure that assigns priority scores to incoming submissions before a
human ever looks at them.

## Honest Limitations

This cannot distinguish a symptom-fix from a root-cause-fix. A NULL check that
silences a crash will pass gates 1, 3, and 4. Only gate 2 (root cause
evidence) helps the reviewer spot this, and that still requires human
judgment.

What it does eliminate is AI slop: submissions where the bug doesn't exist,
the RCA describes an impossible code path, and the patch was never tested.
That covers the majority of current noise.

## Potential Discussion Points at the Summit

1. Should a working trigger become the minimum bar for security-tagged
   submissions? Or remain advisory with prioritization?
2. Where should this CI infrastructure live - kernel.org, per-subsystem, or a
   separate service?
3. How do we handle legitimate bugs found by code inspection that are hard to
   trigger? (Hardware-dependent, narrow races, error-path-only.) Proposal:
   lower priority, not rejection.

## Background

I work on AI assisted offensive security research. I have found and
responsibly disclosed multiple privilege escalation vulnerabilities in the
Linux kernel with full evidence chains (trigger + root cause + patch + working
root exploit). I use AI-assisted tooling in my research workflow with
deterministic verification at every stage. I presented this framework at
netdevconf 0x1A in Rome and received positive feedback from networking
maintainers.

Slides from the netdevconf 0x1A BoF, for reference:
https://github.com/rjt-gupta/conferences-slides/blob/main/netdevconf-0x1A/slides.pptx

Best regards,
Rajat Gupta

Re: [MAINTAINERS SUMMIT] Deterministic verification gates for security submissions

[Greg KH]

On Thu, Jul 23, 2026 at 08:35:28PM -0700, Rajat Gupta wrote:
> ## Proposed Solution: 4 Verification Gates
> 
> Rather than judge prose quality or detect tooling, prioritize submissions by
> evidence:
> 
>   Gate 1 - Trigger + Impact
>     Does a reproducer crash the kernel under a sanitizer?
>     Higher impact evidence (controlled corruption, privilege escalation)
>     gets higher priority.
>     Automatable: YES (build kernel, boot QEMU, run trigger, check output)
> 
>   Gate 2 - Root Cause Evidence
>     Is there mechanically verifiable evidence (KASAN trace, bpftrace output,
>     differential test) showing WHY the bug occurs - not just WHERE it crashes?
>     Automatable: PARTIALLY (sanitizer output is automatic; understanding
>     causality still needs human judgment)
> 
>   Gate 3 - Patch Verification
>     Does the trigger crash before the patch and pass after?
>     Automatable: YES (two kernel builds, one trigger, compare output)
> 
>   Gate 4 - Regression
>     Do subsystem selftests pass with the patch applied?
>     Automatable: YES (same QEMU environment, run selftests)
> 
> Submissions are prioritized by evidence depth. All 4 gates pass -> top of the
> queue. Missing a trigger -> bottom of the queue. Not rejected - deprioritized.
> 
> ## How This Helps
> 
> For reviewers: A submission that passes all 4 gates will take less time to
> review. The evidence is pre-verified - the reviewer confirms it, not
> investigates from scratch. Unverified submissions (prose + patch, no trigger)
> still take 30-60 minutes. The gates surface the verified work first.
> 
> For submitters: Clear requirements. If you show up with a trigger + RCA trace
> + before/after proof + selftests, your submission gets priority regardless of
> whether AI helped you find it. The incentive shifts from "write convincing
> prose" to "produce evidence." In effect, we would be encouraging people to
> use AI to produce concrete, verifiable evidence.
> 
> For the process: 3 of 4 gates are fully automatable. This can run as CI
> infrastructure that assigns priority scores to incoming submissions before a
> human ever looks at them.
> 
> ## Honest Limitations
> 
> This cannot distinguish a symptom-fix from a root-cause-fix. A NULL check that
> silences a crash will pass gates 1, 3, and 4. Only gate 2 (root cause
> evidence) helps the reviewer spot this, and that still requires human
> judgment.
> 
> What it does eliminate is AI slop: submissions where the bug doesn't exist,
> the RCA describes an impossible code path, and the patch was never tested.
> That covers the majority of current noise.

I like this, BUT I will note that this would only work for parts of the
kernel that we all can emulate/run.  For networking, this would be
great, but for almost everything else, specific hardware would be needed
to verify anything.  Just look at some of the recent DRM bugfixes for
specific examples of that.

However, for the network developers, this would be nice to have.

> ## Potential Discussion Points at the Summit
> 
> 1. Should a working trigger become the minimum bar for security-tagged
>    submissions? Or remain advisory with prioritization?

We don't have any such "trigger" to meet the bar of any random person
emailing security@k.o, so I don't know what you mean by this.  It could
drive the decision of "do we talk about this on a public list or not",
and "which issue should I work on now", but it's not going to gate
anyone telling us about issues.

> 2. Where should this CI infrastructure live - kernel.org, per-subsystem, or a
>    separate service?

This MUST be something we all can run on individual machines as
security@k.o reports can NOT be sent to any infrastructure run by anyone
else other than the developers involved in the report.  So if you can
build the framework, great, odds are we can all run it ourselves as most
security@k.o participants have a random box sitting around somewhere
behind their private networks.

> 3. How do we handle legitimate bugs found by code inspection that are hard to
>    trigger? (Hardware-dependent, narrow races, error-path-only.) Proposal:
>    lower priority, not rejection.

That's a huge number of bug reports we normally deal with on a
day-by-day basis on the normal mailing lists.  So not a really big deal
here, if it comes with a patch, and it seems sane, we take the patch
like normal.

thanks,

greg k-h

Re: [MAINTAINERS SUMMIT] Deterministic verification gates for security submissions

[Rajat Gupta]

On 7/24/2026 7:26 AM, Greg KH wrote:
> On Thu, Jul 23, 2026 at 08:35:28PM -0700, Rajat Gupta wrote:
>> ## Proposed Solution: 4 Verification Gates
>>
>> Rather than judge prose quality or detect tooling, prioritize submissions by
>> evidence:
>>
>>   Gate 1 - Trigger + Impact
>>     Does a reproducer crash the kernel under a sanitizer?
>>     Higher impact evidence (controlled corruption, privilege escalation)
>>     gets higher priority.
>>     Automatable: YES (build kernel, boot QEMU, run trigger, check output)
>>
>>   Gate 2 - Root Cause Evidence
>>     Is there mechanically verifiable evidence (KASAN trace, bpftrace output,
>>     differential test) showing WHY the bug occurs - not just WHERE it crashes?
>>     Automatable: PARTIALLY (sanitizer output is automatic; understanding
>>     causality still needs human judgment)
>>
>>   Gate 3 - Patch Verification
>>     Does the trigger crash before the patch and pass after?
>>     Automatable: YES (two kernel builds, one trigger, compare output)
>>
>>   Gate 4 - Regression
>>     Do subsystem selftests pass with the patch applied?
>>     Automatable: YES (same QEMU environment, run selftests)
>>
>> Submissions are prioritized by evidence depth. All 4 gates pass -> top of the
>> queue. Missing a trigger -> bottom of the queue. Not rejected - deprioritized.
>>
>> ## How This Helps
>>
>> For reviewers: A submission that passes all 4 gates will take less time to
>> review. The evidence is pre-verified - the reviewer confirms it, not
>> investigates from scratch. Unverified submissions (prose + patch, no trigger)
>> still take 30-60 minutes. The gates surface the verified work first.
>>
>> For submitters: Clear requirements. If you show up with a trigger + RCA trace
>> + before/after proof + selftests, your submission gets priority regardless of
>> whether AI helped you find it. The incentive shifts from "write convincing
>> prose" to "produce evidence." In effect, we would be encouraging people to
>> use AI to produce concrete, verifiable evidence.
>>
>> For the process: 3 of 4 gates are fully automatable. This can run as CI
>> infrastructure that assigns priority scores to incoming submissions before a
>> human ever looks at them.
>>
>> ## Honest Limitations
>>
>> This cannot distinguish a symptom-fix from a root-cause-fix. A NULL check that
>> silences a crash will pass gates 1, 3, and 4. Only gate 2 (root cause
>> evidence) helps the reviewer spot this, and that still requires human
>> judgment.
>>
>> What it does eliminate is AI slop: submissions where the bug doesn't exist,
>> the RCA describes an impossible code path, and the patch was never tested.
>> That covers the majority of current noise.
> 
> I like this, BUT I will note that this would only work for parts of the
> kernel that we all can emulate/run.  For networking, this would be
> great, but for almost everything else, specific hardware would be needed
> to verify anything.  Just look at some of the recent DRM bugfixes for
> specific examples of that.
> 
> However, for the network developers, this would be nice to have.

Agree. Networking is a natural first target. > 
>> ## Potential Discussion Points at the Summit
>>
>> 1. Should a working trigger become the minimum bar for security-tagged
>>    submissions? Or remain advisory with prioritization?
> 
> We don't have any such "trigger" to meet the bar of any random person
> emailing security@k.o, so I don't know what you mean by this.  It could
> drive the decision of "do we talk about this on a public list or not",
> and "which issue should I work on now", but it's not going to gate
> anyone telling us about issues.

Right, these checks (not "gates") are just prioritization signal for
the reviewer's queue.
> 
>> 2. Where should this CI infrastructure live - kernel.org, per-subsystem, or a
>>    separate service?
> 
> This MUST be something we all can run on individual machines as
> security@k.o reports can NOT be sent to any infrastructure run by anyone
> else other than the developers involved in the report.  So if you can
> build the framework, great, odds are we can all run it ourselves as most
> security@k.o participants have a random box sitting around somewhere
> behind their private networks.
> 
Agree.

>> 3. How do we handle legitimate bugs found by code inspection that are hard to
>>    trigger? (Hardware-dependent, narrow races, error-path-only.) Proposal:
>>    lower priority, not rejection.
> 
> That's a huge number of bug reports we normally deal with on a
> day-by-day basis on the normal mailing lists.  So not a really big deal
> here, if it comes with a patch, and it seems sane, we take the patch
> like normal.

Makes sense. 

Thanks,
Rajat
> 
> thanks,
> 
> greg k-h