RFC: GitLab issues for security disclosures

RFC: GitLab issues for security disclosures

[Daniel P. Berrangé]

The qemu-security mailing list was created several years back now and
traditionally saw 1-2 disclosures a month at worst. This was manageable.

Since approx March 1st, the new normal is to see as many as 20 disclosures
in one single day, more than 200 in total now. This is unsustainable.
I was thinking we needed more people on qemu-security to triage, but IMHO
this won't really fix the problem.

This needs an issue tracker to cope with & email is not an issue tracker.
We faked an issue tracker with a shared spreadsheet to prevent us drowning
these past few months, but this is still not sustainable & probably won't
ever be.

The traditional POV was that security disclosures must be dealt with on
a strict "need to know" basis until there was a concious decision to make
them public. Typically that happened when a patch was published by the
maintainer on qemu-devel. Rarely have we applied an embargo period after
a maintainer had a patch ready to post, as most bugs were not severe
enough. Also in typical usage Libvirt has SELinux/AppArmour to mitigate
impact of a guest ecsape into QEMU.

QEMU is not alone in this chaos, to quote Linus

   https://lkml.org/lkml/2026/5/17/896

[quote]
  the continued flood of AI reports has basically made the security list
  almost entirely unmanageable, with enormous duplication due to
  different people finding the same things with the same tools. People
  spend all their time just forwarding things to the right people or
  saying "that was already fixed a week/month ago" and pointing to the
  public discussion.
[/quote]

They have explicitly changed their policy for disclosure now:

  https://github.com/torvalds/linux/commit/a03ef333fbd6cd861c8457c3d055ee3643a9baad

[quote]
  If you resorted to AI assistance to identify a bug, you must treat
  it as public. While you may have valid reasons to believe it is not,
  the security team's experience shows that bugs discovered this way
  systematically surface simultaneously across multiple researchers,
  often on the same day
[/quote]

IME for qemu-security we've definitely seen the same bugs reported by
different people, within days of each other, a couple of times even
the same day. Dupes are not the majority  - QEMU has so much low
hanging fruit - but they're a sizeable chunk.


Because we don't have a good automated way to publish disclosures
from qemu-security, we don't provide contributors any way to check
for duplicates before submission. The burden of checking for dupes
thus falls on the qemu-security triage people, or the maintainers
we forward issues to.

For disclosures in the non-virtualization use case, we ask the
reporter to file a gitlab issue or send a patch. They almost never
do this. We're lucky to have any reply to emails at all.


IMHO we need to move to an issue tracker. GitLab was originally
discounted in favour of qemu-security list since "Confidential"
issues cannot have visibility managed in a fine grained way.
They are visible to everyone with "Reporter" role or higher,
which is 70+ people for QEMU.

If we accept the kernel's view that AI discovered issues should be
considered public immediately because of the high chance of parallel
"discovery", then GitLab's limitations don't seem to bad.

It is also of note that we've had some people ignoring qemu-security
policy and directly filing public gitlab issues for stuff discovered
via AI/LLM agents already. Those are not getting attention for CVE
assignment, should it be needed which and can't be cross-referenced
by general maintaniers to stuff we have received via qemu-security.

We have some options IMHO

 1. Move all security disclosure to GitLab confidential issues
    no disclosures via email

 2. Move AI/fuzzer assisted disclosures to GitLab confidential
    issues, keep human discovered issues on qemu-security list

 3. Move AI/fuzzer assisted disclosures to GitLab public
    issues, keep human discovered issues on qemu-security list

In all three cases, if a GitLab issue is determined to be security
relevant, any maintainer could send a request to qemu-security list
to get Red Hat product security to allocate a CVE.

Out of those I'd probably lean towards (2) which is slightly
more restrictive than the kernel new policy but not by much.

Some key benefits of using GitLab for security disclosures

 * We can trivially make disclosures public if we classify them
   as a non-virtualization use case, or when the fix is ready.

 * We can formally track the lifecycle of disclosures through to
   the final fix, for both virtualization & non-virtualization
   use cases. The only difference will be that the former can
   request a CVE assignment

 * We can do reports/queries of outstanding issues
 
 * We can more easily use automation to process issues

 * Maintainers can see bugs without waiting for someone to triage
   and forward it on their way.

 * The small number of security bug triage people are no a bottle
   neck anymore

Some downsides/implications

 * Every disclosure in a confidential issue will be visible to every
   maintainer who has joined the qemu-project repo on GitLab. IOW
   that is treating every maintainer as equally trusted.

   We do have qemu-security though we could be mailed if someone
   considered their disclosure to be severely impactful but the triage
   team can't make that decision.

 * We must NOT grant membership to qemu-project at a Reporter level
   for anyone whom is not an active maintainer. They must be limited
   to the "Guest" role at most.

 * No one is formally responsible for GitLab issue triage. We have
   had Thomas do it in the past periodically with script assistance.
   We have Alex doing some of it now with bot assistance. The danger
   is security disclosures get ignored as "somebody else's problem"
   no one has accountability.

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: RFC: GitLab issues for security disclosures

[Michael S. Tsirkin]

On Tue, May 19, 2026 at 03:26:51PM +0100, Daniel P. Berrangé wrote:
> The qemu-security mailing list was created several years back now and
> traditionally saw 1-2 disclosures a month at worst. This was manageable.
> 
> Since approx March 1st, the new normal is to see as many as 20 disclosures
> in one single day, more than 200 in total now. This is unsustainable.
> I was thinking we needed more people on qemu-security to triage, but IMHO
> this won't really fix the problem.
> 
> This needs an issue tracker to cope with & email is not an issue tracker.
> We faked an issue tracker with a shared spreadsheet to prevent us drowning
> these past few months, but this is still not sustainable & probably won't
> ever be.
> 
> The traditional POV was that security disclosures must be dealt with on
> a strict "need to know" basis until there was a concious decision to make
> them public. Typically that happened when a patch was published by the
> maintainer on qemu-devel. Rarely have we applied an embargo period after
> a maintainer had a patch ready to post, as most bugs were not severe
> enough. Also in typical usage Libvirt has SELinux/AppArmour to mitigate
> impact of a guest ecsape into QEMU.
> 
> QEMU is not alone in this chaos, to quote Linus
> 
>    https://lkml.org/lkml/2026/5/17/896
> 
> [quote]
>   the continued flood of AI reports has basically made the security list
>   almost entirely unmanageable, with enormous duplication due to
>   different people finding the same things with the same tools. People
>   spend all their time just forwarding things to the right people or
>   saying "that was already fixed a week/month ago" and pointing to the
>   public discussion.
> [/quote]
> 
> They have explicitly changed their policy for disclosure now:
> 
>   https://github.com/torvalds/linux/commit/a03ef333fbd6cd861c8457c3d055ee3643a9baad
> 
> [quote]
>   If you resorted to AI assistance to identify a bug, you must treat
>   it as public. While you may have valid reasons to believe it is not,
>   the security team's experience shows that bugs discovered this way
>   systematically surface simultaneously across multiple researchers,
>   often on the same day
> [/quote]
> 
> IME for qemu-security we've definitely seen the same bugs reported by
> different people, within days of each other, a couple of times even
> the same day. Dupes are not the majority  - QEMU has so much low
> hanging fruit - but they're a sizeable chunk.
> 
> 
> Because we don't have a good automated way to publish disclosures
> from qemu-security, we don't provide contributors any way to check
> for duplicates before submission. The burden of checking for dupes
> thus falls on the qemu-security triage people, or the maintainers
> we forward issues to.
> 
> For disclosures in the non-virtualization use case, we ask the
> reporter to file a gitlab issue or send a patch. They almost never
> do this. We're lucky to have any reply to emails at all.
> 
> 
> IMHO we need to move to an issue tracker. GitLab was originally
> discounted in favour of qemu-security list since "Confidential"
> issues cannot have visibility managed in a fine grained way.
> They are visible to everyone with "Reporter" role or higher,
> which is 70+ people for QEMU.
> 
> If we accept the kernel's view that AI discovered issues should be
> considered public immediately because of the high chance of parallel
> "discovery", then GitLab's limitations don't seem to bad.
> 
> It is also of note that we've had some people ignoring qemu-security
> policy and directly filing public gitlab issues for stuff discovered
> via AI/LLM agents already. Those are not getting attention for CVE
> assignment, should it be needed which and can't be cross-referenced
> by general maintaniers to stuff we have received via qemu-security.
> 
> We have some options IMHO
> 
>  1. Move all security disclosure to GitLab confidential issues
>     no disclosures via email
> 
>  2. Move AI/fuzzer assisted disclosures to GitLab confidential
>     issues, keep human discovered issues on qemu-security list
> 
>  3. Move AI/fuzzer assisted disclosures to GitLab public
>     issues, keep human discovered issues on qemu-security list

or 4. do it like linux did, and move ai assisted disclosured
to the public ML. Fuzzer assisted can stay with human discovered.


> In all three cases, if a GitLab issue is determined to be security
> relevant, any maintainer could send a request to qemu-security list
> to get Red Hat product security to allocate a CVE.
> 
> Out of those I'd probably lean towards (2) which is slightly
> more restrictive than the kernel new policy but not by much.
> 
> Some key benefits of using GitLab for security disclosures
> 
>  * We can trivially make disclosures public if we classify them
>    as a non-virtualization use case, or when the fix is ready.
> 
>  * We can formally track the lifecycle of disclosures through to
>    the final fix, for both virtualization & non-virtualization
>    use cases. The only difference will be that the former can
>    request a CVE assignment
> 
>  * We can do reports/queries of outstanding issues
>  
>  * We can more easily use automation to process issues
> 
>  * Maintainers can see bugs without waiting for someone to triage
>    and forward it on their way.
> 
>  * The small number of security bug triage people are no a bottle
>    neck anymore
> 
> Some downsides/implications
> 
>  * Every disclosure in a confidential issue will be visible to every
>    maintainer who has joined the qemu-project repo on GitLab. IOW
>    that is treating every maintainer as equally trusted.
> 
>    We do have qemu-security though we could be mailed if someone
>    considered their disclosure to be severely impactful but the triage
>    team can't make that decision.
> 
>  * We must NOT grant membership to qemu-project at a Reporter level
>    for anyone whom is not an active maintainer. They must be limited
>    to the "Guest" role at most.
> 
>  * No one is formally responsible for GitLab issue triage. We have
>    had Thomas do it in the past periodically with script assistance.
>    We have Alex doing some of it now with bot assistance. The danger
>    is security disclosures get ignored as "somebody else's problem"
>    no one has accountability.
> 
> With regards,
> Daniel
> -- 
> |: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
> |: https://libvirt.org          ~~          https://entangle-photo.org :|
> |: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: RFC: GitLab issues for security disclosures

[Daniel P. Berrangé]

On Tue, May 19, 2026 at 11:18:31AM -0400, Michael S. Tsirkin wrote:
> On Tue, May 19, 2026 at 03:26:51PM +0100, Daniel P. Berrangé wrote:
> > The qemu-security mailing list was created several years back now and
> > traditionally saw 1-2 disclosures a month at worst. This was manageable.
> > 
> > Since approx March 1st, the new normal is to see as many as 20 disclosures
> > in one single day, more than 200 in total now. This is unsustainable.
> > I was thinking we needed more people on qemu-security to triage, but IMHO
> > this won't really fix the problem.
> > 
> > This needs an issue tracker to cope with & email is not an issue tracker.
> > We faked an issue tracker with a shared spreadsheet to prevent us drowning
> > these past few months, but this is still not sustainable & probably won't
> > ever be.
> > 
> > The traditional POV was that security disclosures must be dealt with on
> > a strict "need to know" basis until there was a concious decision to make
> > them public. Typically that happened when a patch was published by the
> > maintainer on qemu-devel. Rarely have we applied an embargo period after
> > a maintainer had a patch ready to post, as most bugs were not severe
> > enough. Also in typical usage Libvirt has SELinux/AppArmour to mitigate
> > impact of a guest ecsape into QEMU.
> > 
> > QEMU is not alone in this chaos, to quote Linus
> > 
> >    https://lkml.org/lkml/2026/5/17/896
> > 
> > [quote]
> >   the continued flood of AI reports has basically made the security list
> >   almost entirely unmanageable, with enormous duplication due to
> >   different people finding the same things with the same tools. People
> >   spend all their time just forwarding things to the right people or
> >   saying "that was already fixed a week/month ago" and pointing to the
> >   public discussion.
> > [/quote]
> > 
> > They have explicitly changed their policy for disclosure now:
> > 
> >   https://github.com/torvalds/linux/commit/a03ef333fbd6cd861c8457c3d055ee3643a9baad
> > 
> > [quote]
> >   If you resorted to AI assistance to identify a bug, you must treat
> >   it as public. While you may have valid reasons to believe it is not,
> >   the security team's experience shows that bugs discovered this way
> >   systematically surface simultaneously across multiple researchers,
> >   often on the same day
> > [/quote]
> > 
> > IME for qemu-security we've definitely seen the same bugs reported by
> > different people, within days of each other, a couple of times even
> > the same day. Dupes are not the majority  - QEMU has so much low
> > hanging fruit - but they're a sizeable chunk.
> > 
> > 
> > Because we don't have a good automated way to publish disclosures
> > from qemu-security, we don't provide contributors any way to check
> > for duplicates before submission. The burden of checking for dupes
> > thus falls on the qemu-security triage people, or the maintainers
> > we forward issues to.
> > 
> > For disclosures in the non-virtualization use case, we ask the
> > reporter to file a gitlab issue or send a patch. They almost never
> > do this. We're lucky to have any reply to emails at all.
> > 
> > 
> > IMHO we need to move to an issue tracker. GitLab was originally
> > discounted in favour of qemu-security list since "Confidential"
> > issues cannot have visibility managed in a fine grained way.
> > They are visible to everyone with "Reporter" role or higher,
> > which is 70+ people for QEMU.
> > 
> > If we accept the kernel's view that AI discovered issues should be
> > considered public immediately because of the high chance of parallel
> > "discovery", then GitLab's limitations don't seem to bad.
> > 
> > It is also of note that we've had some people ignoring qemu-security
> > policy and directly filing public gitlab issues for stuff discovered
> > via AI/LLM agents already. Those are not getting attention for CVE
> > assignment, should it be needed which and can't be cross-referenced
> > by general maintaniers to stuff we have received via qemu-security.
> > 
> > We have some options IMHO
> > 
> >  1. Move all security disclosure to GitLab confidential issues
> >     no disclosures via email
> > 
> >  2. Move AI/fuzzer assisted disclosures to GitLab confidential
> >     issues, keep human discovered issues on qemu-security list
> > 
> >  3. Move AI/fuzzer assisted disclosures to GitLab public
> >     issues, keep human discovered issues on qemu-security list
> 
> or 4. do it like linux did, and move ai assisted disclosured
> to the public ML. Fuzzer assisted can stay with human discovered.

Linux could do that because of the way they issue CVEs. IIUC, every
bug that goes into stable kernels gets evaluated for whether a CVE
is required. Thus they don't need to do CVE allocation in alignment
with any bug disclosure process.

While our stable branch process for QEMU is healthier than it has
ever been, it is still essentially a 1 person effort looking for
candidate patches, not a strong enough culture of maintainers
sending patches to stable proactively.  So I don't think we're
in a position to replicate Linux's approach to CVE assignment.

So IMHO we still need to have some issue tracking mechanism to
tag incoming disclosures as needing CVE assignment or not. If
all disclosures end up on qemu-devel and untracked we'll just
never end up getting CVEs for most of them. That doesn't feel
like a viable approach


With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: RFC: GitLab issues for security disclosures

[Michael S. Tsirkin]

On Tue, May 19, 2026 at 05:11:23PM +0100, Daniel P. Berrangé wrote:
> On Tue, May 19, 2026 at 11:18:31AM -0400, Michael S. Tsirkin wrote:
> > On Tue, May 19, 2026 at 03:26:51PM +0100, Daniel P. Berrangé wrote:
> > > The qemu-security mailing list was created several years back now and
> > > traditionally saw 1-2 disclosures a month at worst. This was manageable.
> > > 
> > > Since approx March 1st, the new normal is to see as many as 20 disclosures
> > > in one single day, more than 200 in total now. This is unsustainable.
> > > I was thinking we needed more people on qemu-security to triage, but IMHO
> > > this won't really fix the problem.
> > > 
> > > This needs an issue tracker to cope with & email is not an issue tracker.
> > > We faked an issue tracker with a shared spreadsheet to prevent us drowning
> > > these past few months, but this is still not sustainable & probably won't
> > > ever be.
> > > 
> > > The traditional POV was that security disclosures must be dealt with on
> > > a strict "need to know" basis until there was a concious decision to make
> > > them public. Typically that happened when a patch was published by the
> > > maintainer on qemu-devel. Rarely have we applied an embargo period after
> > > a maintainer had a patch ready to post, as most bugs were not severe
> > > enough. Also in typical usage Libvirt has SELinux/AppArmour to mitigate
> > > impact of a guest ecsape into QEMU.
> > > 
> > > QEMU is not alone in this chaos, to quote Linus
> > > 
> > >    https://lkml.org/lkml/2026/5/17/896
> > > 
> > > [quote]
> > >   the continued flood of AI reports has basically made the security list
> > >   almost entirely unmanageable, with enormous duplication due to
> > >   different people finding the same things with the same tools. People
> > >   spend all their time just forwarding things to the right people or
> > >   saying "that was already fixed a week/month ago" and pointing to the
> > >   public discussion.
> > > [/quote]
> > > 
> > > They have explicitly changed their policy for disclosure now:
> > > 
> > >   https://github.com/torvalds/linux/commit/a03ef333fbd6cd861c8457c3d055ee3643a9baad
> > > 
> > > [quote]
> > >   If you resorted to AI assistance to identify a bug, you must treat
> > >   it as public. While you may have valid reasons to believe it is not,
> > >   the security team's experience shows that bugs discovered this way
> > >   systematically surface simultaneously across multiple researchers,
> > >   often on the same day
> > > [/quote]
> > > 
> > > IME for qemu-security we've definitely seen the same bugs reported by
> > > different people, within days of each other, a couple of times even
> > > the same day. Dupes are not the majority  - QEMU has so much low
> > > hanging fruit - but they're a sizeable chunk.
> > > 
> > > 
> > > Because we don't have a good automated way to publish disclosures
> > > from qemu-security, we don't provide contributors any way to check
> > > for duplicates before submission. The burden of checking for dupes
> > > thus falls on the qemu-security triage people, or the maintainers
> > > we forward issues to.
> > > 
> > > For disclosures in the non-virtualization use case, we ask the
> > > reporter to file a gitlab issue or send a patch. They almost never
> > > do this. We're lucky to have any reply to emails at all.
> > > 
> > > 
> > > IMHO we need to move to an issue tracker. GitLab was originally
> > > discounted in favour of qemu-security list since "Confidential"
> > > issues cannot have visibility managed in a fine grained way.
> > > They are visible to everyone with "Reporter" role or higher,
> > > which is 70+ people for QEMU.
> > > 
> > > If we accept the kernel's view that AI discovered issues should be
> > > considered public immediately because of the high chance of parallel
> > > "discovery", then GitLab's limitations don't seem to bad.
> > > 
> > > It is also of note that we've had some people ignoring qemu-security
> > > policy and directly filing public gitlab issues for stuff discovered
> > > via AI/LLM agents already. Those are not getting attention for CVE
> > > assignment, should it be needed which and can't be cross-referenced
> > > by general maintaniers to stuff we have received via qemu-security.
> > > 
> > > We have some options IMHO
> > > 
> > >  1. Move all security disclosure to GitLab confidential issues
> > >     no disclosures via email
> > > 
> > >  2. Move AI/fuzzer assisted disclosures to GitLab confidential
> > >     issues, keep human discovered issues on qemu-security list
> > > 
> > >  3. Move AI/fuzzer assisted disclosures to GitLab public
> > >     issues, keep human discovered issues on qemu-security list
> > 
> > or 4. do it like linux did, and move ai assisted disclosured
> > to the public ML. Fuzzer assisted can stay with human discovered.
> 
> Linux could do that because of the way they issue CVEs. IIUC, every
> bug that goes into stable kernels gets evaluated for whether a CVE
> is required. Thus they don't need to do CVE allocation in alignment
> with any bug disclosure process.
> 
> While our stable branch process for QEMU is healthier than it has
> ever been, it is still essentially a 1 person effort looking for
> candidate patches, not a strong enough culture of maintainers
> sending patches to stable proactively.  So I don't think we're
> in a position to replicate Linux's approach to CVE assignment.
> 
> So IMHO we still need to have some issue tracking mechanism to
> tag incoming disclosures as needing CVE assignment or not. If
> all disclosures end up on qemu-devel and untracked we'll just
> never end up getting CVEs for most of them. That doesn't feel
> like a viable approach
> 
> 
> With regards,
> Daniel
> -- 
> |: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
> |: https://libvirt.org          ~~          https://entangle-photo.org :|
> |: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|



Oh I certainly do not object to that! can be public gitlab+public ML.

-- 
MST

Re: RFC: GitLab issues for security disclosures

[Mauro Matteo Cascella]

On Tue, May 19, 2026 at 4:31 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> The qemu-security mailing list was created several years back now and
> traditionally saw 1-2 disclosures a month at worst. This was manageable.
>
> Since approx March 1st, the new normal is to see as many as 20 disclosures
> in one single day, more than 200 in total now. This is unsustainable.
> I was thinking we needed more people on qemu-security to triage, but IMHO
> this won't really fix the problem.
>
> This needs an issue tracker to cope with & email is not an issue tracker.
> We faked an issue tracker with a shared spreadsheet to prevent us drowning
> these past few months, but this is still not sustainable & probably won't
> ever be.
>
> The traditional POV was that security disclosures must be dealt with on
> a strict "need to know" basis until there was a concious decision to make
> them public. Typically that happened when a patch was published by the
> maintainer on qemu-devel. Rarely have we applied an embargo period after
> a maintainer had a patch ready to post, as most bugs were not severe
> enough. Also in typical usage Libvirt has SELinux/AppArmour to mitigate
> impact of a guest ecsape into QEMU.
>
> QEMU is not alone in this chaos, to quote Linus
>
>    https://lkml.org/lkml/2026/5/17/896
>
> [quote]
>   the continued flood of AI reports has basically made the security list
>   almost entirely unmanageable, with enormous duplication due to
>   different people finding the same things with the same tools. People
>   spend all their time just forwarding things to the right people or
>   saying "that was already fixed a week/month ago" and pointing to the
>   public discussion.
> [/quote]
>
> They have explicitly changed their policy for disclosure now:
>
>   https://github.com/torvalds/linux/commit/a03ef333fbd6cd861c8457c3d055ee3643a9baad
>
> [quote]
>   If you resorted to AI assistance to identify a bug, you must treat
>   it as public. While you may have valid reasons to believe it is not,
>   the security team's experience shows that bugs discovered this way
>   systematically surface simultaneously across multiple researchers,
>   often on the same day
> [/quote]
>
> IME for qemu-security we've definitely seen the same bugs reported by
> different people, within days of each other, a couple of times even
> the same day. Dupes are not the majority  - QEMU has so much low
> hanging fruit - but they're a sizeable chunk.
>
>
> Because we don't have a good automated way to publish disclosures
> from qemu-security, we don't provide contributors any way to check
> for duplicates before submission. The burden of checking for dupes
> thus falls on the qemu-security triage people, or the maintainers
> we forward issues to.
>
> For disclosures in the non-virtualization use case, we ask the
> reporter to file a gitlab issue or send a patch. They almost never
> do this. We're lucky to have any reply to emails at all.
>
>
> IMHO we need to move to an issue tracker. GitLab was originally
> discounted in favour of qemu-security list since "Confidential"
> issues cannot have visibility managed in a fine grained way.
> They are visible to everyone with "Reporter" role or higher,
> which is 70+ people for QEMU.
>
> If we accept the kernel's view that AI discovered issues should be
> considered public immediately because of the high chance of parallel
> "discovery", then GitLab's limitations don't seem to bad.
>
> It is also of note that we've had some people ignoring qemu-security
> policy and directly filing public gitlab issues for stuff discovered
> via AI/LLM agents already. Those are not getting attention for CVE
> assignment, should it be needed which and can't be cross-referenced
> by general maintaniers to stuff we have received via qemu-security.
>
> We have some options IMHO
>
>  1. Move all security disclosure to GitLab confidential issues
>     no disclosures via email
>
>  2. Move AI/fuzzer assisted disclosures to GitLab confidential
>     issues, keep human discovered issues on qemu-security list
>
>  3. Move AI/fuzzer assisted disclosures to GitLab public
>     issues, keep human discovered issues on qemu-security list
>
> In all three cases, if a GitLab issue is determined to be security
> relevant, any maintainer could send a request to qemu-security list
> to get Red Hat product security to allocate a CVE.
>
> Out of those I'd probably lean towards (2) which is slightly
> more restrictive than the kernel new policy but not by much.
>
> Some key benefits of using GitLab for security disclosures
>
>  * We can trivially make disclosures public if we classify them
>    as a non-virtualization use case, or when the fix is ready.
>
>  * We can formally track the lifecycle of disclosures through to
>    the final fix, for both virtualization & non-virtualization
>    use cases. The only difference will be that the former can
>    request a CVE assignment
>
>  * We can do reports/queries of outstanding issues
>
>  * We can more easily use automation to process issues
>
>  * Maintainers can see bugs without waiting for someone to triage
>    and forward it on their way.
>
>  * The small number of security bug triage people are no a bottle
>    neck anymore
>
> Some downsides/implications
>
>  * Every disclosure in a confidential issue will be visible to every
>    maintainer who has joined the qemu-project repo on GitLab. IOW
>    that is treating every maintainer as equally trusted.
>
>    We do have qemu-security though we could be mailed if someone
>    considered their disclosure to be severely impactful but the triage
>    team can't make that decision.
>
>  * We must NOT grant membership to qemu-project at a Reporter level
>    for anyone whom is not an active maintainer. They must be limited
>    to the "Guest" role at most.
>
>  * No one is formally responsible for GitLab issue triage. We have
>    had Thomas do it in the past periodically with script assistance.
>    We have Alex doing some of it now with bot assistance. The danger
>    is security disclosures get ignored as "somebody else's problem"
>    no one has accountability.

The benefits of moving to an issue tracker clearly outweigh the
downsides IMHO. Given the current industry landscape and the
escalating volume of disclosures in RH Product Security, I am no
longer in a position to effectively support the QEMU triage process.
The move to GitLab seems necessary to ensure sustainability going
forward. Option #2 sounds like the best compromise, though I'd be
totally fine with #1 (no disclosures via email) as well.

Thanks,

> With regards,
> Daniel
> --
> |: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
> |: https://libvirt.org          ~~          https://entangle-photo.org :|
> |: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|
>

-- 
Mauro Matteo Cascella
Red Hat Product Security
PGP-Key ID: BB3410B0

Re: RFC: GitLab issues for security disclosures

[Pierrick Bouvier]

Hi Daniel,

On 5/19/2026 9:26 AM, Daniel P. Berrangé wrote:
> The qemu-security mailing list was created several years back now and
> traditionally saw 1-2 disclosures a month at worst. This was manageable.
> 
> Since approx March 1st, the new normal is to see as many as 20 disclosures
> in one single day, more than 200 in total now. This is unsustainable.
> I was thinking we needed more people on qemu-security to triage, but IMHO
> this won't really fix the problem.
>

Considering the increase in number of issues, would that be possible to
make stricter rules about what is expected?

For instance, asking for a working exploit and optionally a VM image +
instructions to reproduce it. I am not expert on the topic, but what I
see is that if we have this, all duplicates would be eliminated at once.

This would force security researchers to spend time (and tokens) on
providing a concrete test and environment to reproduce, which really
helps with fixing and testing. Else, simply ignore and close those issues.

> This needs an issue tracker to cope with & email is not an issue tracker.
> We faked an issue tracker with a shared spreadsheet to prevent us drowning
> these past few months, but this is still not sustainable & probably won't
> ever be.
>

Overall, you're right.
However, changing the tool won't solve the number of issues sent, and
for that, something additional is needed.

I wonder also what is the percentage of duplicates there is from what
you observed in the last 2 months. Any rough idea of the number?

> The traditional POV was that security disclosures must be dealt with on
> a strict "need to know" basis until there was a concious decision to make
> them public. Typically that happened when a patch was published by the
> maintainer on qemu-devel. Rarely have we applied an embargo period after
> a maintainer had a patch ready to post, as most bugs were not severe
> enough. Also in typical usage Libvirt has SELinux/AppArmour to mitigate
> impact of a guest ecsape into QEMU.
> 
> QEMU is not alone in this chaos, to quote Linus
> 
>    https://lkml.org/lkml/2026/5/17/896
> 
> [quote]
>   the continued flood of AI reports has basically made the security list
>   almost entirely unmanageable, with enormous duplication due to
>   different people finding the same things with the same tools. People
>   spend all their time just forwarding things to the right people or
>   saying "that was already fixed a week/month ago" and pointing to the
>   public discussion.
> [/quote]
> 
> They have explicitly changed their policy for disclosure now:
> 
>   https://github.com/torvalds/linux/commit/a03ef333fbd6cd861c8457c3d055ee3643a9baad
> 
> [quote]
>   If you resorted to AI assistance to identify a bug, you must treat
>   it as public. While you may have valid reasons to believe it is not,
>   the security team's experience shows that bugs discovered this way
>   systematically surface simultaneously across multiple researchers,
>   often on the same day
> [/quote]
> 
> IME for qemu-security we've definitely seen the same bugs reported by
> different people, within days of each other, a couple of times even
> the same day. Dupes are not the majority  - QEMU has so much low
> hanging fruit - but they're a sizeable chunk.
> 
> 
> Because we don't have a good automated way to publish disclosures
> from qemu-security, we don't provide contributors any way to check
> for duplicates before submission. The burden of checking for dupes
> thus falls on the qemu-security triage people, or the maintainers
> we forward issues to.
> 
> For disclosures in the non-virtualization use case, we ask the
> reporter to file a gitlab issue or send a patch. They almost never
> do this. We're lucky to have any reply to emails at all.
> 
> 
> IMHO we need to move to an issue tracker. GitLab was originally
> discounted in favour of qemu-security list since "Confidential"
> issues cannot have visibility managed in a fine grained way.
> They are visible to everyone with "Reporter" role or higher,
> which is 70+ people for QEMU.
> 
> If we accept the kernel's view that AI discovered issues should be
> considered public immediately because of the high chance of parallel
> "discovery", then GitLab's limitations don't seem to bad.
> 
> It is also of note that we've had some people ignoring qemu-security
> policy and directly filing public gitlab issues for stuff discovered
> via AI/LLM agents already. Those are not getting attention for CVE
> assignment, should it be needed which and can't be cross-referenced
> by general maintaniers to stuff we have received via qemu-security.
> 
> We have some options IMHO
> 
>  1. Move all security disclosure to GitLab confidential issues
>     no disclosures via email
> 
>  2. Move AI/fuzzer assisted disclosures to GitLab confidential
>     issues, keep human discovered issues on qemu-security list
> 
>  3. Move AI/fuzzer assisted disclosures to GitLab public
>     issues, keep human discovered issues on qemu-security list
> 
> In all three cases, if a GitLab issue is determined to be security
> relevant, any maintainer could send a request to qemu-security list
> to get Red Hat product security to allocate a CVE.
> 
> Out of those I'd probably lean towards (2) which is slightly
> more restrictive than the kernel new policy but not by much.
> 
> Some key benefits of using GitLab for security disclosures
> 
>  * We can trivially make disclosures public if we classify them
>    as a non-virtualization use case, or when the fix is ready.
> 
>  * We can formally track the lifecycle of disclosures through to
>    the final fix, for both virtualization & non-virtualization
>    use cases. The only difference will be that the former can
>    request a CVE assignment
> 
>  * We can do reports/queries of outstanding issues
>  
>  * We can more easily use automation to process issues
> 
>  * Maintainers can see bugs without waiting for someone to triage
>    and forward it on their way.
> 
>  * The small number of security bug triage people are no a bottle
>    neck anymore
> 
> Some downsides/implications
> 
>  * Every disclosure in a confidential issue will be visible to every
>    maintainer who has joined the qemu-project repo on GitLab. IOW
>    that is treating every maintainer as equally trusted.
> 
>    We do have qemu-security though we could be mailed if someone
>    considered their disclosure to be severely impactful but the triage
>    team can't make that decision.
> 
>  * We must NOT grant membership to qemu-project at a Reporter level
>    for anyone whom is not an active maintainer. They must be limited
>    to the "Guest" role at most.
> 
>  * No one is formally responsible for GitLab issue triage. We have
>    had Thomas do it in the past periodically with script assistance.
>    We have Alex doing some of it now with bot assistance. The danger
>    is security disclosures get ignored as "somebody else's problem"
>    no one has accountability.
> 
> With regards,
> Daniel

Regards,
Pierrick

Re: RFC: GitLab issues for security disclosures

[Daniel P. Berrangé]

On Wed, May 20, 2026 at 10:01:14AM -0500, Pierrick Bouvier wrote:
> Hi Daniel,
> 
> On 5/19/2026 9:26 AM, Daniel P. Berrangé wrote:
> > The qemu-security mailing list was created several years back now and
> > traditionally saw 1-2 disclosures a month at worst. This was manageable.
> > 
> > Since approx March 1st, the new normal is to see as many as 20 disclosures
> > in one single day, more than 200 in total now. This is unsustainable.
> > I was thinking we needed more people on qemu-security to triage, but IMHO
> > this won't really fix the problem.
> >
> 
> Considering the increase in number of issues, would that be possible to
> make stricter rules about what is expected?
> 
> For instance, asking for a working exploit and optionally a VM image +
> instructions to reproduce it. I am not expert on the topic, but what I
> see is that if we have this, all duplicates would be eliminated at once.

With the new crop of AI assisted disclosures there is absolutely no
lack of data provided.

Most come with reproducible exploits, detailed descriptions and analysis,
and more - everything you could conceivably need to triage the disclosure.
Reading and interpreting this takes significant mental effort and there's
too much data to quickly/easily eliminate dupes.

> > This needs an issue tracker to cope with & email is not an issue tracker.
> > We faked an issue tracker with a shared spreadsheet to prevent us drowning
> > these past few months, but this is still not sustainable & probably won't
> > ever be.
> >
> 
> Overall, you're right.
> However, changing the tool won't solve the number of issues sent, and
> for that, something additional is needed.

I don't expect there to be any change in submission rate. The proposal
is based on the expectation that the submission rate will continue at
a high level for a long time. Primarily the goal is to reduce the
tracking and triage work overhead and to eliminate/reduce single person
bottlenecks in the process

> I wonder also what is the percentage of duplicates there is from what
> you observed in the last 2 months. Any rough idea of the number?

Definitely at least 10%, probably closer to 15%.


With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: RFC: GitLab issues for security disclosures

[Pierrick Bouvier]

On 5/20/2026 10:09 AM, Daniel P. Berrangé wrote:
> On Wed, May 20, 2026 at 10:01:14AM -0500, Pierrick Bouvier wrote:
>> Hi Daniel,
>>
>> On 5/19/2026 9:26 AM, Daniel P. Berrangé wrote:
>>> The qemu-security mailing list was created several years back now and
>>> traditionally saw 1-2 disclosures a month at worst. This was manageable.
>>>
>>> Since approx March 1st, the new normal is to see as many as 20 disclosures
>>> in one single day, more than 200 in total now. This is unsustainable.
>>> I was thinking we needed more people on qemu-security to triage, but IMHO
>>> this won't really fix the problem.
>>>
>>
>> Considering the increase in number of issues, would that be possible to
>> make stricter rules about what is expected?
>>
>> For instance, asking for a working exploit and optionally a VM image +
>> instructions to reproduce it. I am not expert on the topic, but what I
>> see is that if we have this, all duplicates would be eliminated at once.
> 
> With the new crop of AI assisted disclosures there is absolutely no
> lack of data provided.
> 
> Most come with reproducible exploits, detailed descriptions and analysis,
> and more - everything you could conceivably need to triage the disclosure.
> Reading and interpreting this takes significant mental effort and there's
> too much data to quickly/easily eliminate dupes.
>

Maybe we need to "standardize" this part then.
Or do something like asking a (GitLab) CI pipeline to be written to
expose the issue. If we can just run this with a specific qemu
remote/branch, it becomes trivial to rerun it when fixes are pushed.

It definitely does not solve the original scaling issue, but maybe can
help to absorb it, and spend time where it's useful: writing and
upstreaming a fix, and check it "broke" the exploit.

>>> This needs an issue tracker to cope with & email is not an issue tracker.
>>> We faked an issue tracker with a shared spreadsheet to prevent us drowning
>>> these past few months, but this is still not sustainable & probably won't
>>> ever be.
>>>
>>
>> Overall, you're right.
>> However, changing the tool won't solve the number of issues sent, and
>> for that, something additional is needed.
> 
> I don't expect there to be any change in submission rate. The proposal
> is based on the expectation that the submission rate will continue at
> a high level for a long time. Primarily the goal is to reduce the
> tracking and triage work overhead and to eliminate/reduce single person
> bottlenecks in the process
> 
>> I wonder also what is the percentage of duplicates there is from what
>> you observed in the last 2 months. Any rough idea of the number?
> 
> Definitely at least 10%, probably closer to 15%.
>

Ok, interesting number, thanks. I was expecting much more, but I'm
biased having heard Linus this morning talk about this for Linux kernel.

> 
> With regards,
> Daniel

Regards,
Pierrick

Re: RFC: GitLab issues for security disclosures

[Daniel P. Berrangé]

On Wed, May 20, 2026 at 12:33:16PM -0500, Pierrick Bouvier wrote:
> On 5/20/2026 10:09 AM, Daniel P. Berrangé wrote:
> > On Wed, May 20, 2026 at 10:01:14AM -0500, Pierrick Bouvier wrote:
> >> Hi Daniel,
> >>
> >> On 5/19/2026 9:26 AM, Daniel P. Berrangé wrote:
> >>> The qemu-security mailing list was created several years back now and
> >>> traditionally saw 1-2 disclosures a month at worst. This was manageable.
> >>>
> >>> Since approx March 1st, the new normal is to see as many as 20 disclosures
> >>> in one single day, more than 200 in total now. This is unsustainable.
> >>> I was thinking we needed more people on qemu-security to triage, but IMHO
> >>> this won't really fix the problem.
> >>>
> >>
> >> Considering the increase in number of issues, would that be possible to
> >> make stricter rules about what is expected?
> >>
> >> For instance, asking for a working exploit and optionally a VM image +
> >> instructions to reproduce it. I am not expert on the topic, but what I
> >> see is that if we have this, all duplicates would be eliminated at once.
> > 
> > With the new crop of AI assisted disclosures there is absolutely no
> > lack of data provided.
> > 
> > Most come with reproducible exploits, detailed descriptions and analysis,
> > and more - everything you could conceivably need to triage the disclosure.
> > Reading and interpreting this takes significant mental effort and there's
> > too much data to quickly/easily eliminate dupes.
> >
> 
> Maybe we need to "standardize" this part then.
> Or do something like asking a (GitLab) CI pipeline to be written to
> expose the issue. If we can just run this with a specific qemu
> remote/branch, it becomes trivial to rerun it when fixes are pushed.
> 
> It definitely does not solve the original scaling issue, but maybe can
> help to absorb it, and spend time where it's useful: writing and
> upstreaming a fix, and check it "broke" the exploit.
> 
> >>> This needs an issue tracker to cope with & email is not an issue tracker.
> >>> We faked an issue tracker with a shared spreadsheet to prevent us drowning
> >>> these past few months, but this is still not sustainable & probably won't
> >>> ever be.
> >>>
> >>
> >> Overall, you're right.
> >> However, changing the tool won't solve the number of issues sent, and
> >> for that, something additional is needed.
> > 
> > I don't expect there to be any change in submission rate. The proposal
> > is based on the expectation that the submission rate will continue at
> > a high level for a long time. Primarily the goal is to reduce the
> > tracking and triage work overhead and to eliminate/reduce single person
> > bottlenecks in the process
> > 
> >> I wonder also what is the percentage of duplicates there is from what
> >> you observed in the last 2 months. Any rough idea of the number?
> > 
> > Definitely at least 10%, probably closer to 15%.
> >
> 
> Ok, interesting number, thanks. I was expecting much more, but I'm
> biased having heard Linus this morning talk about this for Linux kernel.

I expect the dupes to increase over time as more people run the
same analysis across QEMU, especially given that most of the bugs
are not yet fixed. 

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: RFC: GitLab issues for security disclosures

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 5238 bytes --]

On Wed, May 20, 2026 at 11:40 AM Daniel P. Berrangé <berrange@redhat.com>
wrote:

> On Wed, May 20, 2026 at 12:33:16PM -0500, Pierrick Bouvier wrote:
> > On 5/20/2026 10:09 AM, Daniel P. Berrangé wrote:
> > > On Wed, May 20, 2026 at 10:01:14AM -0500, Pierrick Bouvier wrote:
> > >> Hi Daniel,
> > >>
> > >> On 5/19/2026 9:26 AM, Daniel P. Berrangé wrote:
> > >>> The qemu-security mailing list was created several years back now and
> > >>> traditionally saw 1-2 disclosures a month at worst. This was
> manageable.
> > >>>
> > >>> Since approx March 1st, the new normal is to see as many as 20
> disclosures
> > >>> in one single day, more than 200 in total now. This is unsustainable.
> > >>> I was thinking we needed more people on qemu-security to triage, but
> IMHO
> > >>> this won't really fix the problem.
> > >>>
> > >>
> > >> Considering the increase in number of issues, would that be possible
> to
> > >> make stricter rules about what is expected?
> > >>
> > >> For instance, asking for a working exploit and optionally a VM image +
> > >> instructions to reproduce it. I am not expert on the topic, but what I
> > >> see is that if we have this, all duplicates would be eliminated at
> once.
> > >
> > > With the new crop of AI assisted disclosures there is absolutely no
> > > lack of data provided.
> > >
> > > Most come with reproducible exploits, detailed descriptions and
> analysis,
> > > and more - everything you could conceivably need to triage the
> disclosure.
> > > Reading and interpreting this takes significant mental effort and
> there's
> > > too much data to quickly/easily eliminate dupes.
> > >
> >
> > Maybe we need to "standardize" this part then.
> > Or do something like asking a (GitLab) CI pipeline to be written to
> > expose the issue. If we can just run this with a specific qemu
> > remote/branch, it becomes trivial to rerun it when fixes are pushed.
> >
> > It definitely does not solve the original scaling issue, but maybe can
> > help to absorb it, and spend time where it's useful: writing and
> > upstreaming a fix, and check it "broke" the exploit.
> >
> > >>> This needs an issue tracker to cope with & email is not an issue
> tracker.
> > >>> We faked an issue tracker with a shared spreadsheet to prevent us
> drowning
> > >>> these past few months, but this is still not sustainable & probably
> won't
> > >>> ever be.
> > >>>
> > >>
> > >> Overall, you're right.
> > >> However, changing the tool won't solve the number of issues sent, and
> > >> for that, something additional is needed.
> > >
> > > I don't expect there to be any change in submission rate. The proposal
> > > is based on the expectation that the submission rate will continue at
> > > a high level for a long time. Primarily the goal is to reduce the
> > > tracking and triage work overhead and to eliminate/reduce single person
> > > bottlenecks in the process
> > >
> > >> I wonder also what is the percentage of duplicates there is from what
> > >> you observed in the last 2 months. Any rough idea of the number?
> > >
> > > Definitely at least 10%, probably closer to 15%.
> > >
> >
> > Ok, interesting number, thanks. I was expecting much more, but I'm
> > biased having heard Linus this morning talk about this for Linux kernel.
>
> I expect the dupes to increase over time as more people run the
> same analysis across QEMU, especially given that most of the bugs
> are not yet fixed.
>

So this isn't security related, but I think it's informative. I started
upstreaming
a large amount (8k lines) of bsd-user code in March. Claude did the moving
and author chasing. Then I asked claude to review that code, and refined
the reviews into a checklist. I iterated for 6 weeks or so, on and off, and
I wound up fixing 100 "bugs," 10 of which were 'real' the rest were logical
or only if certain options were enabled. I went ahead and fixed them all
since CONFIG_REMAP_DEBUG should have a chance of working and
it's useful to be careful between host arch values and target arch values,
even if we know they are the same. But it takes a while to come up with
these, and has an annoying habit of reporting no more than 8-12 on any
run and always seems to find something new (sometimes a new critical
thing).

But I just stopped running the reviews after a while. The details likely
don't matter...  I just did another review, and claude found more, so
that suggests that even with this flood, we're still in the sampling phase
and we'll see dupes, but we won't hit bottom for quite some time since
I think my experience is at least suggestive about what to expect.

But all this suggests that we're going to have a huge volume of reports,
most of which will be mediocre in terms of effect, even if they come
with big mountains of data. AI produces the data in this regard
very well, but is crap for know what's a real problem and what
isn't.

Warner

P.S. For "yucks" I asked claude to review my code again, and it
flagged 20 more things. Most of which look at least semi-legit,
though claude has about a high false positive hit rate when it
found the 100 bugs, it flagged about 125 or so.

[-- Attachment #2: Type: text/html, Size: 7601 bytes --]

Re: RFC: GitLab issues for security disclosures

[Michael S. Tsirkin]

On Wed, May 20, 2026 at 06:39:14PM +0100, Daniel P. Berrangé wrote:
> On Wed, May 20, 2026 at 12:33:16PM -0500, Pierrick Bouvier wrote:
> > On 5/20/2026 10:09 AM, Daniel P. Berrangé wrote:
> > > On Wed, May 20, 2026 at 10:01:14AM -0500, Pierrick Bouvier wrote:
> > >> Hi Daniel,
> > >>
> > >> On 5/19/2026 9:26 AM, Daniel P. Berrangé wrote:
> > >>> The qemu-security mailing list was created several years back now and
> > >>> traditionally saw 1-2 disclosures a month at worst. This was manageable.
> > >>>
> > >>> Since approx March 1st, the new normal is to see as many as 20 disclosures
> > >>> in one single day, more than 200 in total now. This is unsustainable.
> > >>> I was thinking we needed more people on qemu-security to triage, but IMHO
> > >>> this won't really fix the problem.
> > >>>
> > >>
> > >> Considering the increase in number of issues, would that be possible to
> > >> make stricter rules about what is expected?
> > >>
> > >> For instance, asking for a working exploit and optionally a VM image +
> > >> instructions to reproduce it. I am not expert on the topic, but what I
> > >> see is that if we have this, all duplicates would be eliminated at once.
> > > 
> > > With the new crop of AI assisted disclosures there is absolutely no
> > > lack of data provided.
> > > 
> > > Most come with reproducible exploits, detailed descriptions and analysis,
> > > and more - everything you could conceivably need to triage the disclosure.
> > > Reading and interpreting this takes significant mental effort and there's
> > > too much data to quickly/easily eliminate dupes.
> > >
> > 
> > Maybe we need to "standardize" this part then.
> > Or do something like asking a (GitLab) CI pipeline to be written to
> > expose the issue. If we can just run this with a specific qemu
> > remote/branch, it becomes trivial to rerun it when fixes are pushed.
> > 
> > It definitely does not solve the original scaling issue, but maybe can
> > help to absorb it, and spend time where it's useful: writing and
> > upstreaming a fix, and check it "broke" the exploit.
> > 
> > >>> This needs an issue tracker to cope with & email is not an issue tracker.
> > >>> We faked an issue tracker with a shared spreadsheet to prevent us drowning
> > >>> these past few months, but this is still not sustainable & probably won't
> > >>> ever be.
> > >>>
> > >>
> > >> Overall, you're right.
> > >> However, changing the tool won't solve the number of issues sent, and
> > >> for that, something additional is needed.
> > > 
> > > I don't expect there to be any change in submission rate. The proposal
> > > is based on the expectation that the submission rate will continue at
> > > a high level for a long time. Primarily the goal is to reduce the
> > > tracking and triage work overhead and to eliminate/reduce single person
> > > bottlenecks in the process
> > > 
> > >> I wonder also what is the percentage of duplicates there is from what
> > >> you observed in the last 2 months. Any rough idea of the number?
> > > 
> > > Definitely at least 10%, probably closer to 15%.
> > >
> > 
> > Ok, interesting number, thanks. I was expecting much more, but I'm
> > biased having heard Linus this morning talk about this for Linux kernel.
> 
> I expect the dupes to increase over time as more people run the
> same analysis across QEMU, especially given that most of the bugs
> are not yet fixed. 
> 
> With regards,
> Daniel
> -- 
> |: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
> |: https://libvirt.org          ~~          https://entangle-photo.org :|
> |: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|


something I was unaware of previously, is that gitlab is a CNA:
https://about.gitlab.com/security/cve/

so using gitlab issues means assigning CVE #s should be super easy.


-- 
MST

Re: RFC: GitLab issues for security disclosures

[Daniel P. Berrangé]

On Wed, May 20, 2026 at 07:14:41PM -0400, Michael S. Tsirkin wrote:
> On Wed, May 20, 2026 at 06:39:14PM +0100, Daniel P. Berrangé wrote:
> > On Wed, May 20, 2026 at 12:33:16PM -0500, Pierrick Bouvier wrote:
> > > On 5/20/2026 10:09 AM, Daniel P. Berrangé wrote:
> > > > On Wed, May 20, 2026 at 10:01:14AM -0500, Pierrick Bouvier wrote:
> > > >> Hi Daniel,
> > > >>
> > > >> On 5/19/2026 9:26 AM, Daniel P. Berrangé wrote:
> > > >>> The qemu-security mailing list was created several years back now and
> > > >>> traditionally saw 1-2 disclosures a month at worst. This was manageable.
> > > >>>
> > > >>> Since approx March 1st, the new normal is to see as many as 20 disclosures
> > > >>> in one single day, more than 200 in total now. This is unsustainable.
> > > >>> I was thinking we needed more people on qemu-security to triage, but IMHO
> > > >>> this won't really fix the problem.
> > > >>>
> > > >>
> > > >> Considering the increase in number of issues, would that be possible to
> > > >> make stricter rules about what is expected?
> > > >>
> > > >> For instance, asking for a working exploit and optionally a VM image +
> > > >> instructions to reproduce it. I am not expert on the topic, but what I
> > > >> see is that if we have this, all duplicates would be eliminated at once.
> > > > 
> > > > With the new crop of AI assisted disclosures there is absolutely no
> > > > lack of data provided.
> > > > 
> > > > Most come with reproducible exploits, detailed descriptions and analysis,
> > > > and more - everything you could conceivably need to triage the disclosure.
> > > > Reading and interpreting this takes significant mental effort and there's
> > > > too much data to quickly/easily eliminate dupes.
> > > >
> > > 
> > > Maybe we need to "standardize" this part then.
> > > Or do something like asking a (GitLab) CI pipeline to be written to
> > > expose the issue. If we can just run this with a specific qemu
> > > remote/branch, it becomes trivial to rerun it when fixes are pushed.
> > > 
> > > It definitely does not solve the original scaling issue, but maybe can
> > > help to absorb it, and spend time where it's useful: writing and
> > > upstreaming a fix, and check it "broke" the exploit.
> > > 
> > > >>> This needs an issue tracker to cope with & email is not an issue tracker.
> > > >>> We faked an issue tracker with a shared spreadsheet to prevent us drowning
> > > >>> these past few months, but this is still not sustainable & probably won't
> > > >>> ever be.
> > > >>>
> > > >>
> > > >> Overall, you're right.
> > > >> However, changing the tool won't solve the number of issues sent, and
> > > >> for that, something additional is needed.
> > > > 
> > > > I don't expect there to be any change in submission rate. The proposal
> > > > is based on the expectation that the submission rate will continue at
> > > > a high level for a long time. Primarily the goal is to reduce the
> > > > tracking and triage work overhead and to eliminate/reduce single person
> > > > bottlenecks in the process
> > > > 
> > > >> I wonder also what is the percentage of duplicates there is from what
> > > >> you observed in the last 2 months. Any rough idea of the number?
> > > > 
> > > > Definitely at least 10%, probably closer to 15%.
> > > >
> > > 
> > > Ok, interesting number, thanks. I was expecting much more, but I'm
> > > biased having heard Linus this morning talk about this for Linux kernel.
> > 
> > I expect the dupes to increase over time as more people run the
> > same analysis across QEMU, especially given that most of the bugs
> > are not yet fixed. 
> 
> something I was unaware of previously, is that gitlab is a CNA:
> https://about.gitlab.com/security/cve/
> 
> so using gitlab issues means assigning CVE #s should be super easy.

That's interesting, but I'm not sure it is a match for what we want
in practice if you see their process & timeframes...

  "We will acknowledge receipt of CVE requests the next business
   day and strive to send regular updates about our progress. Our
   goal is to determine if the vulnerability is valid and communicate
   back to the submitter within 30 business days."

We don't really need someone to spend time on additional analysis,
as we'll already know the bug is valid at this point and just need
a CVE assignment, ideally the same day.



With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: RFC: GitLab issues for security disclosures

[Mauro Matteo Cascella]

On Thu, May 21, 2026 at 1:14 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Wed, May 20, 2026 at 06:39:14PM +0100, Daniel P. Berrangé wrote:
> > On Wed, May 20, 2026 at 12:33:16PM -0500, Pierrick Bouvier wrote:
> > > On 5/20/2026 10:09 AM, Daniel P. Berrangé wrote:
> > > > On Wed, May 20, 2026 at 10:01:14AM -0500, Pierrick Bouvier wrote:
> > > >> Hi Daniel,
> > > >>
> > > >> On 5/19/2026 9:26 AM, Daniel P. Berrangé wrote:
> > > >>> The qemu-security mailing list was created several years back now and
> > > >>> traditionally saw 1-2 disclosures a month at worst. This was manageable.
> > > >>>
> > > >>> Since approx March 1st, the new normal is to see as many as 20 disclosures
> > > >>> in one single day, more than 200 in total now. This is unsustainable.
> > > >>> I was thinking we needed more people on qemu-security to triage, but IMHO
> > > >>> this won't really fix the problem.
> > > >>>
> > > >>
> > > >> Considering the increase in number of issues, would that be possible to
> > > >> make stricter rules about what is expected?
> > > >>
> > > >> For instance, asking for a working exploit and optionally a VM image +
> > > >> instructions to reproduce it. I am not expert on the topic, but what I
> > > >> see is that if we have this, all duplicates would be eliminated at once.
> > > >
> > > > With the new crop of AI assisted disclosures there is absolutely no
> > > > lack of data provided.
> > > >
> > > > Most come with reproducible exploits, detailed descriptions and analysis,
> > > > and more - everything you could conceivably need to triage the disclosure.
> > > > Reading and interpreting this takes significant mental effort and there's
> > > > too much data to quickly/easily eliminate dupes.
> > > >
> > >
> > > Maybe we need to "standardize" this part then.
> > > Or do something like asking a (GitLab) CI pipeline to be written to
> > > expose the issue. If we can just run this with a specific qemu
> > > remote/branch, it becomes trivial to rerun it when fixes are pushed.
> > >
> > > It definitely does not solve the original scaling issue, but maybe can
> > > help to absorb it, and spend time where it's useful: writing and
> > > upstreaming a fix, and check it "broke" the exploit.
> > >
> > > >>> This needs an issue tracker to cope with & email is not an issue tracker.
> > > >>> We faked an issue tracker with a shared spreadsheet to prevent us drowning
> > > >>> these past few months, but this is still not sustainable & probably won't
> > > >>> ever be.
> > > >>>
> > > >>
> > > >> Overall, you're right.
> > > >> However, changing the tool won't solve the number of issues sent, and
> > > >> for that, something additional is needed.
> > > >
> > > > I don't expect there to be any change in submission rate. The proposal
> > > > is based on the expectation that the submission rate will continue at
> > > > a high level for a long time. Primarily the goal is to reduce the
> > > > tracking and triage work overhead and to eliminate/reduce single person
> > > > bottlenecks in the process
> > > >
> > > >> I wonder also what is the percentage of duplicates there is from what
> > > >> you observed in the last 2 months. Any rough idea of the number?
> > > >
> > > > Definitely at least 10%, probably closer to 15%.
> > > >
> > >
> > > Ok, interesting number, thanks. I was expecting much more, but I'm
> > > biased having heard Linus this morning talk about this for Linux kernel.
> >
> > I expect the dupes to increase over time as more people run the
> > same analysis across QEMU, especially given that most of the bugs
> > are not yet fixed.
> >
> > With regards,
> > Daniel
> > --
> > |: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
> > |: https://libvirt.org          ~~          https://entangle-photo.org :|
> > |: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|
>
>
> something I was unaware of previously, is that gitlab is a CNA:
> https://about.gitlab.com/security/cve/
>
> so using gitlab issues means assigning CVE #s should be super easy.

Red Hat is a CNA too, QEMU CVEs are currently being reserved by Red Hat.

In fact, Red Hat is a root CNA:
https://www.cve.org/Media/News/item/blog/2023/01/10/Why-Red-Hat-Became-Root

Projects like glibc, postgresql and curl now operate as independent
CNAs under Red Hat, retaining complete end-to-end ownership over CVE
assignment. Is it time for QEMU to follow suit?

> --
> MST
>

--
Mauro Matteo Cascella
Red Hat Product Security
PGP-Key ID: BB3410B0

Re: RFC: GitLab issues for security disclosures

[Daniel P. Berrangé]

On Thu, May 21, 2026 at 02:45:08PM +0200, Mauro Matteo Cascella wrote:
> On Thu, May 21, 2026 at 1:14 AM Michael S. Tsirkin <mst@redhat.com> wrote:
> > something I was unaware of previously, is that gitlab is a CNA:
> > https://about.gitlab.com/security/cve/
> >
> > so using gitlab issues means assigning CVE #s should be super easy.
> 
> Red Hat is a CNA too, QEMU CVEs are currently being reserved by Red Hat.
> 
> In fact, Red Hat is a root CNA:
> https://www.cve.org/Media/News/item/blog/2023/01/10/Why-Red-Hat-Became-Root
> 
> Projects like glibc, postgresql and curl now operate as independent
> CNAs under Red Hat, retaining complete end-to-end ownership over CVE
> assignment. Is it time for QEMU to follow suit?

Is there any guidance on the process & implications of taking that
route, specifically as an OSS project ?

I've found this:

  https://github.com/ossf/wg-vulnerability-disclosures/blob/main/docs/guides/becoming-a-cna-as-an-open-source-org-or-project.md

And there are obligations/requirements there which I would not be
very comfortable with agreeing to with my QEMU hat on.

 * CNA must provide a phone number for the primary POC.

I'm guessing the phone number is intended for someone/org to escalate
urgent issues ?

Related to this I see

 * CNA either should or must publish CVE Records within 24 hours
   of publication of a CVE ID.

and similarly in

  https://www.cve.org/ResourcesSupport/AllResources/CNARules#section_2_sub_cnas

   "3.2.2.2 The administrative POC MUST include both email
    addresses and phone numbers and MAY include additional
    contact methods."
    
   "3.2.2.5 The administrative POC MUST respond in a timely manner.
    Automated responses do not qualify as “a timely manner.”"

As a co-operative volunteer project, my view is that we do not owe
anyone a guranteed or timely response. Yes, many of us are employed
by vendors to work on QEMU, but our work in the upstream community
context is still on a best effort basis. 

If someone requires an urgent or guaranteed response, whether to a CVE
or any other kind of issue, then the obligation needs to fall on the
paid vendors who distribute QEMU rather than any individual upstream
maintainers.

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: RFC: GitLab issues for security disclosures

[Mauro Matteo Cascella]

On Thu, May 21, 2026 at 3:07 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> On Thu, May 21, 2026 at 02:45:08PM +0200, Mauro Matteo Cascella wrote:
> > On Thu, May 21, 2026 at 1:14 AM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > something I was unaware of previously, is that gitlab is a CNA:
> > > https://about.gitlab.com/security/cve/
> > >
> > > so using gitlab issues means assigning CVE #s should be super easy.
> >
> > Red Hat is a CNA too, QEMU CVEs are currently being reserved by Red Hat.
> >
> > In fact, Red Hat is a root CNA:
> > https://www.cve.org/Media/News/item/blog/2023/01/10/Why-Red-Hat-Became-Root
> >
> > Projects like glibc, postgresql and curl now operate as independent
> > CNAs under Red Hat, retaining complete end-to-end ownership over CVE
> > assignment. Is it time for QEMU to follow suit?
>
> Is there any guidance on the process & implications of taking that
> route, specifically as an OSS project ?
>
> I've found this:
>
>   https://github.com/ossf/wg-vulnerability-disclosures/blob/main/docs/guides/becoming-a-cna-as-an-open-source-org-or-project.md
>
> And there are obligations/requirements there which I would not be
> very comfortable with agreeing to with my QEMU hat on.
>
>  * CNA must provide a phone number for the primary POC.
>
> I'm guessing the phone number is intended for someone/org to escalate
> urgent issues ?
>
> Related to this I see
>
>  * CNA either should or must publish CVE Records within 24 hours
>    of publication of a CVE ID.
>
> and similarly in
>
>   https://www.cve.org/ResourcesSupport/AllResources/CNARules#section_2_sub_cnas
>
>    "3.2.2.2 The administrative POC MUST include both email
>     addresses and phone numbers and MAY include additional
>     contact methods."
>
>    "3.2.2.5 The administrative POC MUST respond in a timely manner.
>     Automated responses do not qualify as “a timely manner.”"
>
> As a co-operative volunteer project, my view is that we do not owe
> anyone a guranteed or timely response. Yes, many of us are employed
> by vendors to work on QEMU, but our work in the upstream community
> context is still on a best effort basis.
>
> If someone requires an urgent or guaranteed response, whether to a CVE
> or any other kind of issue, then the obligation needs to fall on the
> paid vendors who distribute QEMU rather than any individual upstream
> maintainers.

Pedro, Yogesh, can you help answer Daniel's questions?

Thanks,

> With regards,
> Daniel
> --
> |: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
> |: https://libvirt.org          ~~          https://entangle-photo.org :|
> |: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|
>

-- 
Mauro Matteo Cascella
Red Hat Product Security
PGP-Key ID: BB3410B0

Re: RFC: GitLab issues for security disclosures

[Alex Bennée]

Daniel P. Berrangé <berrange@redhat.com> writes:

> The qemu-security mailing list was created several years back now and
> traditionally saw 1-2 disclosures a month at worst. This was manageable.
>
> Since approx March 1st, the new normal is to see as many as 20 disclosures
> in one single day, more than 200 in total now. This is unsustainable.
> I was thinking we needed more people on qemu-security to triage, but IMHO
> this won't really fix the problem.
>
<snip>
>
> Some key benefits of using GitLab for security disclosures
>
>  * We can trivially make disclosures public if we classify them
>    as a non-virtualization use case, or when the fix is ready.
>
>  * We can formally track the lifecycle of disclosures through to
>    the final fix, for both virtualization & non-virtualization
>    use cases. The only difference will be that the former can
>    request a CVE assignment
>
>  * We can do reports/queries of outstanding issues
>  
>  * We can more easily use automation to process issues
>
>  * Maintainers can see bugs without waiting for someone to triage
>    and forward it on their way.
>
>  * The small number of security bug triage people are no a bottle
>    neck anymore
>
> Some downsides/implications
>
>  * Every disclosure in a confidential issue will be visible to every
>    maintainer who has joined the qemu-project repo on GitLab. IOW
>    that is treating every maintainer as equally trusted.
>
>    We do have qemu-security though we could be mailed if someone
>    considered their disclosure to be severely impactful but the triage
>    team can't make that decision.
>
>  * We must NOT grant membership to qemu-project at a Reporter level
>    for anyone whom is not an active maintainer. They must be limited
>    to the "Guest" role at most.

We have currently have the following:

"dgibson, dgibson, 20"
"Cleber Rosa, cleber.gnu, 40"
"Stefan Hajnoczi, stefanha, 30"
"Paolo Bonzini, bonzini, 40"
"Michael Roth, mdroth, 30"
"John Snow, jsnow, 20"
"Daniel P. Berrangé, berrange, 20"
"Thomas Huth, thuth, 20"
"Philippe Mathieu-Daudé, philmd, 20"
"Qemu Janitor, qemu-janitor, 20"
"Richard Henderson, rth7680, 40"
"Marc-André Lureau, marcandre.lureau, 20"
"Cornelia Huck, cohuck, 20"
"Stefano Garzarella, sgarzarella, 20"
"Dr. David Alan Gilbert, dagrh, 20"
"Alexander Bulekov, a1xndr, 20"
"Greg Kurz, gkurz, 20"
"Laurent Vivier, lvivier, 20"
"Klaus Jensen, birkelund, 20"
"Hanna Czenczek, hreitz, 20"
"Stefan Weil, stweil, 20"
"Vladimir Sementsov-Ogievskiy, vsementsov, 20"
"Mark Cave-Ayland, mcayland, 20"
"Jason Wang, jasowang, 20"
"Gerd Hoffmann, kraxel, 20"
"Joaquin de Andres, xcancerberox, 20"
"Paul Zimmerman, pauldzim, 20"
"Warner Losh, bsdimp, 20"
"Eduardo Habkost, ehabkost, 20"
"Ani Sinha, anisinha, 20"
"Lars D, lars.dunemark, 20"
"Daniel Henrique Barboza, danielhb, 20"
"Christian Borntraeger, cborntra, 20"
"Alexander Graf, agraf, 20"
"Fam Zheng, famzheng, 20"
"Igor Mammedov, imammedo, 20"
"Cédric Le Goater, legoater, 20"
"Michael Tokarev, mjt0k, 40"
"Marc-André Lureau, marcandre.lureau-rh, 20"
"Alistair Francis, alistair23, 20"
"Emilio Cota, cota_, 20"
"David Woodhouse, dwmw2, 20"
"Eldon, eldondev, 40"
"Bastian Koppelmann, kbastian-qemu, 20"
"Cédric Le Goater, clegoate, 20"
"David Hildenbrand, davidhildenbrand, 20"
"Bin Meng, lbmeng, 20"
"Stefan Berger, stefanberger, 20"
"Alex Williamson, alex.williamson, 20"
"Eric Blake, ebblake, 20"
"Juan Quintela, juan.quintela, 20"
"MST, mstredhat, 20"
"Christian Schoenebeck, schoenebeck, 20"
"npiggin, npiggin, 20"
"Kostiantyn Kostiuk, kostyanf14, 20"
"Kevin Wolf, kmwolf, 30"
"Aihua Liang, aliang1, 20"
"Helge Deller, hdeller, 20"
"Fabiano Rosas, farosas, 20"
"Gustavo Romero, gusbromero, 20"
"Peter Krempa (work), pkrempa, 20"
"Harsh Prateek Bora, harshpb, 30"
"Jim MacArthur, jmacarthur, 20"
"Manos Pitsidianakis, epilys, 20"
"Brian Cain, brian-cain, 20"
"Anthony Roberts, anthony-linaro, 20"
"Pierrick Bouvier, pierrick.bouvier, 20"

>
>  * No one is formally responsible for GitLab issue triage. We have
>    had Thomas do it in the past periodically with script assistance.
>    We have Alex doing some of it now with bot assistance. The danger
>    is security disclosures get ignored as "somebody else's problem"
>    no one has accountability.
>
> With regards,
> Daniel

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: RFC: GitLab issues for security disclosures

[Daniel P. Berrangé]

On Tue, May 19, 2026 at 03:26:51PM +0100, Daniel P. Berrangé wrote:
> This needs an issue tracker to cope with & email is not an issue tracker.
> We faked an issue tracker with a shared spreadsheet to prevent us drowning
> these past few months, but this is still not sustainable & probably won't
> ever be.

snip

> We have some options IMHO
> 
>  1. Move all security disclosure to GitLab confidential issues
>     no disclosures via email
> 
>  2. Move AI/fuzzer assisted disclosures to GitLab confidential
>     issues, keep human discovered issues on qemu-security list
> 
>  3. Move AI/fuzzer assisted disclosures to GitLab public
>     issues, keep human discovered issues on qemu-security list

snip

> Some downsides/implications
> 
>  * Every disclosure in a confidential issue will be visible to every
>    maintainer who has joined the qemu-project repo on GitLab. IOW
>    that is treating every maintainer as equally trusted.
> 
>    We do have qemu-security though we could be mailed if someone
>    considered their disclosure to be severely impactful but the triage
>    team can't make that decision.
> 
>  * We must NOT grant membership to qemu-project at a Reporter level
>    for anyone whom is not an active maintainer. They must be limited
>    to the "Guest" role at most.

I did a query

  $ glab api --paginate /projects/11167699/members/all | jq '.[].name' | sort > members.txt
  $ IFS="                   
  " ;  for line in $(cat members.txt | sed -e 's/"//g' ) ; do echo -n "$line: " ; grep $line MAINTAINERS | wc -l ; done | grep ': 0'

Of the results without a match as a maintainer I see

 * Qemu Janitor
 * stsquad-gitlab-api-access

   Bot accounts


 * dgibson
 * Hanna Czenczek
 * MST

   False negatives - just name mismatches between gitlab account
   and MAINTAINERS files


 * Eduardo Habkost
 * Juan Quintela

   Former maintainers, no longer active in QEMU AFAIK


 * Peter Krempa
 * Peter Krempa (work)

   Libvirt maintainer, added to enable to move bugs between projects


 * Anthony Roberts
 * Bastian Koppelmann
 * Emilio Cota
 * Jim MacArthur
 * Joaquin de Andres
 * Paul Zimmerman

   At least 1 code commit, but not maintainers


 * Eldon

   Provided us some CI hardware for a period of time

 * Aihua Liang
 * Lars D

   Unclear


The former maintainers can probably be removed at this point, given
the length of time that's passed.

If we want to use "Confidential" issues in any way, the question is
whether the rest of the non-maintainers / non-bot accounts should
retain "Reporter" role or be moved to "Guest" role ?

Some contributors may be active enough that they're effectively
maintainers, even if not listed in MAINTAINERS.

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|