Re: [PATCH v2] docs/devel: relax policy on AI-generated contributions

["Alex Bennée" <alex.bennee@linaro.org>]

Paolo Bonzini <pbonzini@redhat.com> writes:

> Until now QEMU's code provenance policy declined any contribution
> believed to include or derive from AI-generated content.  A blanket ban
> was easy to maintain while LLM output was rarely usable on its own, but
> as the tools improved an absolute prohibition has become harder to
> justify.
>
<snip>
>  
> -TL;DR:
> +.. warning::
>  
> -  **Current QEMU project policy is to DECLINE any contributions which are
> -  believed to include or derive from AI generated content. This includes
> -  ChatGPT, Claude, Copilot, Llama and similar tools.**
> +   Please read the below policy before using AI to contribute code or
> +   documentation to QEMU.  This applies to ChatGPT, Claude, Copilot,
> +   Llama, and similar tools.**
>

Stray **, also extra space after QEMU.

> -  **This policy does not apply to other uses of AI, such as researching APIs
> -  or algorithms, static analysis, or debugging, provided their output is not
> -  included in contributions.**
> +The increasing prevalence of AI-assisted software development,
> +and especially the use of content generated by `Large Language Models
> +<https://en.wikipedia.org/wiki/Large_language_model>`__ (LLMs),
> +poses a number of difficult questions.
>  
> -The increasing prevalence of AI-assisted software development results in a
> -number of difficult legal questions and risks for software projects, including
> -QEMU.  Of particular concern is content generated by `Large Language Models
> -<https://en.wikipedia.org/wiki/Large_language_model>`__ (LLMs).
> +Risks to open source projects include maintainer burnout from an
> +increased number of contributions, as well as the risk to the project
> +from unintentional inclusion of copyrighted material in the LLM's output.
> +In order to mitigate these risks, the QEMU project currently allows
> +using AI/LLM tools to produce patches in a limited set of scenarios:
>  
> -The QEMU community requires that contributors certify their patch submissions
> -are made in accordance with the rules of the `Developer's Certificate of
> -Origin (DCO) <dco>`.
> +**Mechanical changes**
> +  If you can use a deterministic tool, it is preferred that you use
> it

deterministic tool or script,?

> +  and not replace it with AI. If you don't know how to do the change
> +  deterministically, you can ask the AI for help.
>  
> -To satisfy the DCO, the patch contributor has to fully understand the
> -copyright and license status of content they are contributing to QEMU. With AI
> -content generators, the copyright and license status of the output is
> -ill-defined with no generally accepted, settled legal foundation.
> +**Small bug fixes**
> +  These should be limited to 20 lines of code or less, not including
> +  tests.  You are still expected to :ref:`understand and explain your changes
> +  <write_a_meaningful_commit_message>` and the rationale behind them.
>  
> -Where the training material is known, it is common for it to include large
> -volumes of material under restrictive licensing/copyright terms. Even where
> -the training material is all known to be under open source licenses, it is
> -likely to be under a variety of terms, not all of which will be compatible
> -with QEMU's licensing requirements.
> +**Documentation and code comments**
> +  While AI can help draft text, it still requires significant human
> +  oversight.  Pay attention to the organization and flow of the generated
> +  text, and strictly fact-check all technical details as LLMs are prone
> +  to being confidently wrong.
>  
> -How contributors could comply with DCO terms (b) or (c) for the output of AI
> -content generators commonly available today is unclear.  The QEMU project is
> -not willing or able to accept the legal risks of non-compliance.
> +**Tests**
> +  Note that you must still confirm that each test actually exercises
> +  the intended behavior including, for regression tests, that it
> +  fails without the code under test and passes for the right reason.
>  
> -The QEMU project thus requires that contributors refrain from using AI content
> -generators on patches intended to be submitted to the project, and will
> -decline any contribution if use of AI is either known or suspected.
> +These boundaries do not apply to other uses of AI, such as researching
> +APIs or algorithms, static analysis, or debugging, provided the model's
> +output is not included in contributions.
>  
> -Examples of tools impacted by this policy includes GitHub's CoPilot, OpenAI's
> -ChatGPT, Anthropic's Claude, and Meta's Code Llama, and code/content
> -generation agents which are built on top of such tools.
> +If you wish to send large amounts of AI-generated changes, or any other
> +contribution not in the above categories, please get in touch with the
> +maintainer beforehand.  These can be treated as experiments, at the
> +discretion of the maintainer and the community, with no obligation
> +to accept them.
>  
> -This policy may evolve as AI tools mature and the legal situation is
> -clarified.
> +**Use of AI does not remove the need for authors to comply with all
> +other requirements for contribution.**  In particular, the
> +``Signed-off-by`` label in a patch submission is a statement that
> +the author takes responsibility for the entire contents of the patch,
> +certifying that their patch submission is made in accordance with the
> +rules of the `Developer's Certificate of Origin (DCO) <dco>`.
>  
> -Exceptions
> -^^^^^^^^^^
> +Commit messages for AI-assisted changes
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>

In my v2 I added:

  AI tools **should not be used to write commit messages**. The act of
  summarising and explaining the reasoning for the changes is an
  important demonstration of the human authors understanding of the
  commit.


> -The QEMU project welcomes discussion on any exceptions to this policy,
> -or more general revisions. This can be done by contacting the qemu-devel
> -mailing list with details of a proposed tool, model, usage scenario, etc.
> -that is beneficial to QEMU, while still mitigating issues around compliance
> -with the DCO.  After discussion, any exception will be listed below.
> +When AI/LLM tools produce or substantively shape your patch, add an
> +``AI-used-for:`` line before ``Signed-off-by``, as a reminder of your
> +DCO obligations and a guide to reviewers.  The text is one or more of
> +``code``, ``tests``, ``docs``, ``research``, possibly followed by an
> +explanation in parentheses:
>  
> -Exceptions do not remove the need for authors to comply with all other
> -requirements for contribution.  In particular, the "Signed-off-by"
> -label in a patch submission is a statement that the author takes
> -responsibility for the entire contents of the patch, including any parts
> -that were generated or assisted by AI tools or other tools.
> +.. code-block:: none
> +
> +     AI-used-for: tests, docs
> +     AI-used-for: code
> +     AI-used-for: code (refactoring)
> +     AI-used-for: code (prototype)
> +     AI-used-for: research
> +
> +``AI-used-for`` should not be included for "background" usage such as
> +autocomplete or obtaining a pre-review of the patch.
> +
> +There is no requirement to include your prompts or summarize the
> +conversation in the commit message or cover letter, but you may do so
> +if you think it helps a reviewer judge the result.  For example:
> +
> +**Helpful prompts**
> +  These describe concrete constraints or instructions, making it easy for a
> +  reviewer to see how the tool's output was guided:
> +
> +  * "move field ``foo`` from ``struct aa`` to ``struct bb``.  If a
> +    function already has a local variable or parameter of type ``struct
> +    bb``, use it instead of accessing ``aa.bb``"
> +
> +  * "add an implementation of the trait for ``Mutex<T: MyTrait>``; it
> +    takes the lock around the calls and forwards to ``T``"
> +
> +**Unhelpful prompts**
> +  These are too generic to provide meaningful context.  You can of course
> +  use them in the context of a complex interaction with the LLM, but they
> +  should not be included in the commit message:
> +
> +  * "write user-facing documentation for the new tool"
> +
> +  * "write testcases for the new functions"
> +
> +QEMU does *not* use ``Assisted-by``, ``Co-authored-by`` or ``Generated-by``
> +trailers to indicate AI usage.  In particular, it is not necessary to
> +specify the exact AI model or tool used to create the commit.
> +
> +Deterministic tooling (sed, coccinelle, formatters) is out of scope for
> +the trailer, but should be mentioned in the commit message.

The other changes in my v2 where just different wordings for the same concept.

With those have a:

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro