Re: [PATCH 0/6] tracetool: add mypy --strict checking [AI discussion ahead!]

[Paolo Bonzini <pbonzini@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 3009 bytes --]

Il mer 8 ott 2025, 19:23 Stefan Hajnoczi <stefanha@redhat.com> ha scritto:

> On Wed, Oct 08, 2025 at 08:35:39AM +0200, Paolo Bonzini wrote:
> > In this version, the types were added mostly with the RightTyper tool
> > (https://github.com/RightTyper/RightTyper), which uses profiling to
> detect
> > the types of arguments and return types at run time.  However, because
> > adding type annotations is such a narrow and verifiable task, I also
> developed
> > a parallel version using an LLM, to provide some data on topics such as:
> >
> > - how much choice/creativity is there in writing type annotations?
> >   Is it closer to writing functional code or to refactoring?
> >
> > - how does AI output for such mechanical transformations compare to other
> >   automated tools, whose output is known not to be copyrightable?
> >
> > - what is the kind of time saving that the tool can provide?
> >
> > - how effective is an LLM for this task?  Is the required human help a
> >   pain to provide, or can the human keep the fun part for itself?
>
> Here are my thoughts:
>
> - Type annotations are probably not copyrightable. Although I think AI
>   output contributions should be allowed for non-copyrightable work, I
>   understand the slippery slope argument. Also, at QEMU Summit the
>   consensus was to give it some time and then revisit the AI policy next
>   year. I think we should stick to that so that the QEMU community has
>   time to consider other scenarios involving AI.


I agree. At the same time, rushing a policy change is bad but making an
uninformed one is too. Submitting the non-AI change, recounting the
experience with the AI tool, and explaining how they differed for me,
hopefully helps informing decisions. In fact this is probably at the upper
end of the complexity, for what can still be described as mechanical
transformation.

In other words, I want to be clear that, even though I did rush a bit the
discussion on the policy, I don't want to rush the changes but rather be
prepared for them.

- Markus pointed out why generating them automatically may not result in
>   the same quality as manually-written annotations because the AI
>   doesn't have the context that is missing from the code itself. [...] The
> difference between
>   manually-written type hints and AI-generated ones is probably less
>   significant than with API documentation though, so I think they are
>   sufficiently valuable and high quality enough to merge.
>

Indeed. And also, the nuances are more important for something where type
hints describe an API than if you're doing it only for some extra static
checking.

You can always start with a tool and refine, too. In fact I expect that
*if* an exception is granted for mechanical changes, submissions would
almost never consist of AI-generated changes only. Full isolation may not
always be practical, but separating AI- or tool-generated commits from
human refinements is something we'd ask for anyway, for the sake of
reviewability.

Paolo

>

[-- Attachment #2: Type: text/html, Size: 4269 bytes --]