Re: [PATCH 1/4] docs/devel: relax policy on AI-generated contributions

["Daniel P. Berrangé" <berrange@redhat.com>]

On Thu, Oct 01, 2026 at 01:21:43PM +0100, Peter Maydell wrote:
> On Wed, 30 Sept 2026 at 09:54, Paolo Bonzini <pbonzini@redhat.com> wrote:
> > +LLM-assisted and LLM-created contributions
> > +''''''''''''''''''''''''''''''''''''''''''
> > +
> > +Use of generative AI can be classified in the following four buckets:
> > +
> > +- "background" assistance, such as completion and code review.  This does
> > +  not need disclosure.
> > +
> > +- small LLM-assisted bugfixes (as a rule of thumb, below 10 lines of code)
> > +  are allowed with disclosure.  Splitting a submission in multiple parts
> > +  is **not** a valid way to bypass this rule; repeated, frequent posting
> > +  of such fixes must follow the rule for full contributions.
> > +
> > +- use of LLMs to help generating parts of a larger patch---test cases,
> > +  adaptations of existing code, boilerplate code for a new API, a tool to
> > +  help performing mechanical changes, etc.  These are generally allowed;
> > +  disclosure is highly recommended for non-trivial, functional code.
> 
> I think we can get or have consensus on these first three buckets...
> 
> > +- large, heavily LLM-assisted contributions where LLMs write large parts
> > +  of functional code.  These are only allowed if *pre-arranged*,
> > +  *high-quality* and *well-tested*.
> > +
> > +The last case (*"LLM-created contribution"*) is the one that requires
> > +a more careful approach.  In particular:
> > +
> > +- "Pre-arranged" means that a maintainer has communicated ahead of time,
> > +  for example on `IRC or the QEMU mailing
> > +  list <https://www.qemu.org/contribute/>`__, that they are willing
> > +  to review an LLM-created contribution.
> > +
> > +- "High-quality" means that the contributor must apply the same judgment
> > +  that would be applied to other code changes.  Contributors must invest
> > +  substantial time reviewing their contributions, curating them, and
> > +  understanding them in depth.
> > +
> > +- "Well-tested" means the LLM-created contributions will be held to a
> > +  higher standard than human-created ones, because LLMs make it easier
> > +  to write tests.  There are no exceptions for "writing the tests seems
> > +  hard" or for `yak shaving <https://en.wiktionary.org/wiki/yak_shaving>`__.
> 
> ...but I think fundamentally I am not in favour of having this
> fourth category of permitted uses. It seems even to have lost the
> idea of it being an "experimental" approach in this revision of the
> series. The first three categories are fine, and I have no particular
> problem with our relaxing our current policy to explicitly permit them.

I suggested previously to remove the "experimental" tag, because it
felt rather misleading wrt what the policy was actually proposing to
permit in practice.


When we're saying that a maintainer and contributor can agree to
use AI for *anything* and will merge it, then that work has ceased
to be for "experiments". It is a fully accepted use case for writing
and merging code. All the policy is doing at that point is attempting
to rate-limit work by requiring prior agreement to use AI.


> Our problem as a project does not seem to be on the "writing
> code" side of things, but has always been on the "code review"
> side. This policy relaxation makes it much easier to write more
> code. Even when there's a "pre-arranged reviewer" that isn't
> necessarily helpful -- if you're one of the primary maintainers
> for an area of QEMU then the fact that somebody else is reviewing
> it doesn't make a big AI-generated code infusion any more palatable.

Yes, that is my concern with anything that ends up spreading outside
the bounds of a single subsystem & its maintainer(s).


> I look at our issue tracker where we see AI assisted contributions
> that run the whole range from "an human making intelligent use of
> the tool to provide high-quality bug reports with good analysis"
> through to "absolute slop dumped on us with no apparent human
> thought or intervention". I don't really want to expand that
> from our issue tracker to our code review and codebase, please.

One of the problems I have with the AI contribs in the bug tracker
is that the quality/accuracy is just about high enough that I am
not comfortable rejecting (closing) most of the bugs, as there's
usually some nugget of a real problem in there. At the same time
the quality is not high enough to make it pleasant to deal with
the issues, which is amplified by the volume of bugs.

I expect the same thing will happen with code. The patches will be
good enough, such that you'll have a really hard time to justify
rejecting them outright. They can even be as good as an initial
patch from a human. The difference is that when you point out all
the mistakes to the human, they'll remember most of it and not
make the same mistakes again in future contributions. For bots,
it is a potluck whether your feedback has any effect long time
on future contributions. So we'll end up suffering through the
review of the work regardless.

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|