Re: [PATCH 1/4] docs/devel: relax policy on AI-generated contributions

[Kevin Wolf <kwolf@redhat.com>]

Am 06.10.2026 um 11:56 hat Daniel P. Berrangé geschrieben:
> On Tue, Oct 06, 2026 at 10:44:37AM +0100, Alex Bennée wrote:
> > Kevin Wolf <kwolf@redhat.com> writes:
> > 
> > > Am 05.10.2026 um 23:09 hat BALATON Zoltan geschrieben:
> > >> On Fri, 2 Oct 2026, Paolo Bonzini wrote:
> > >> > I agree completely with how you phrased the first and third.  I agree
> > >> > that the second bullet is a risk too, but it's mitigated by "LLM-created
> > >> > contributions will be held to a higher standard than human-created ones,
> > >> > because LLMs make it easier to write tests" (and by maintainers being
> > >> > able to
> > >> 
> > >> Tests need to be reviewed too to be able to trust them so that does not help
> > >> against overloading reviewers with generated patches but burdens them even
> > >> more: they will also have to review the tests before the code.
> > >
> > > The required level of trust for tests is comparatively low, so it's much
> > > easier to review tests to a sufficient level than say device emulation
> > > code.
> > >
> > > If a new test looks plausible, fails before the fix and passes
> > > afterwards, it's more useful that what we had before, even if it's
> > > technically buggy in some corner case.
> > 
> > And also easier to rip out in the unlikely event it infringes anything.
> > 
> > But broadly having a test* is better than having no test.
> > 
> > * modulo the effects on our coverage.
> 
> The maintability of test code is really important too, just as much
> as the code being tested. Every test potentially constrains how we
> can change code in future as tests themselves are never perfect or
> exhaustive and often take short-cuts in their design to suit the
> current testing needs. As we enhance the code under test, the test
> suite needs to evolve with it and that requires that we understand
> the test code and why it is doing what it is doing.
> 
> A test spat out by AI which is a string of 20 undocumented qtest
> commands may fail before the fix and pass afterwards, but that
> test is not automatically a win if we don't understand why the test
> is doing what it is doing - it becomes a burden for long term maint.

That's the "looks plausible" part. It's really not a lot of work to
recognise that a test is just 20 undocumented qtest commands.

Kevin