Re: on ai generated and code provenance

[Kevin Wolf <kwolf@redhat.com>]

Am 27.05.2026 um 12:43 hat Alex Bennée geschrieben:
> Paolo Bonzini <pbonzini@redhat.com> writes:
> 
> > On 5/27/26 10:41, Kevin Wolf wrote:
> >> Am 26.05.2026 um 21:52 hat Warner Losh geschrieben:
> >>> The QEMU Project currently may accept limited uses of AI that produce
> >>> high quality patches that are limited in the creative content added.
> >>> While maintainers will ultimately decide, changes like the following
> >>> fall within this policy
> >>> 1. Fixing obvious warnings in the obvious ways suggested by the tool
> >>> 2. Tree wide API changes, and other similar mechanical changes done
> >>>     today with perl/python/sed/coccinelle
> >> As I said in the paragraph you quoted below, I don't think we should
> >> encourage using AI for tasks that a deterministic tool could do.
> >
> > In some cases such a tool does not exist.  Much to my surprise, there
> > is no tool to do static type inference on Python code, but AI is very
> > good at doing it.
> >
> >> Letting AI perform the change directly instead may be an acceptable
> >> shortcut for a one-man hobby project that nobody else will ever look at,
> >> but in the context of a community project like QEMU in which your
> >> changes have to be reviewed and understood by others, it matters a lot
> >> that the output of the tool is reproducible. Otherwise, you're creating
> >> unnecessary work for others, and that isn't acceptable.
> >
> > When applicable, going through coccinelle (with the aid of AI if
> > needed! is indeed a good middle ground as it helps reviewers for large
> > changes. If you have many slightly different but easily separated
> > changes (e.g. you can split the patch by struct field), it may make
> > things worse.
> >
> > Its also worth noting that in other cases even sed or coccinelle,
> > while deterministic, cannot produce 100% of the patch.
> >
> >> So maybe we should even explicitly mention a recommendation like the
> >> following:
> >>      If you can use a deterministic tool, don't use AI instead. If
> >> you
> >>      don't know how to use the deterministic tool, use the AI to tell you
> >>      how to use it instead of trying to replace it.
> >
> > I like it.
> >
> >>> 3. Limited, small changes to fix bugs or add a small new feature whose
> >>>     scope is less than about 100 lines and the originator can explain
> >>>     them all or the meta issues about the patch.
> >> Not sure if mentioning a number of lines is wise. 100 lines can be
> >> mostly boilerplate and simple sequential code or they can be a deeply
> >> nested complex algorithm.
> >
> > I'd put the threshold at 20-50 at most.
> >
> >> I think I would see more use in a tag like (better name welcome):
> >>     AI-used-for: [code|tests|docs|commit message]...
> >
> > I like this *a lot*.  No need for free advertisement, but some
> > traceability is useful.
> >
> > For tools such as sed or coccinelle, having the exact script in the
> > patch or commit message useful.  Plus, the execution of the script
> > more or lesss delimits the commit by itself (or 90%+ of it).  For LLMs
> > it's a bit less clear cut because separating docs makes little sense.
> > And the exact model is pointless, it will be obsolete in 6 months and
> > provide no useful information.
> >
> > So, something like:
> >
> > ------------------- 8< -------------------
> > Use of AI-generated content
> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >
> > The QEMU project currently allows using AI/LLM tools to produce
> > patches in scenarios with limited creative content:
> >
> > Mechanical changes
> >   If you can use a deterministic tool or a script, don't use AI instead.
> >   If you don't know how to do the change deterministically, you may
> >   ask the AI for help, rather than having it stand in for the tools.
> 
> I like the idea of pointing people towards tools but I wouldn't be quite
> so prescriptive. The series MST referred to was easily eyeball-able and
> I suspect the extra steps would generate friction for contributions.
> That said the wider the change to the code base the more likely a random
> hallucination can get lost in the noise.
> 
> Maybe:
> 
>   Mechanical changes
>     Using AI tools to make simple mechanical changes is allowed. For larger
>     tree-wide changes it is strongly recommended to use a deterministic
>     tool like `sed` or `coccinelle`. You can use AI to help you craft the
>     invocation for you.

I think we do want to discourage the direct use of AI in such cases,
while not outright banning it. So maybe just a minor tweak to Paolo's
wording?

    Mechanical changes
      If you can use a deterministic tool or a script, it is preferred
      that you use it and not replace it with AI. If you don't know how
      to do the change deterministically, you can ask the AI for help,
      rather than having it stand in for the tools.

Kevin