RE: [RFC PATCH 0/6] KVM: arm64: Errata management for VM Live migration

[Cornelia Huck <cohuck@redhat.com>]

On Fri, Oct 18 2024, Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com> wrote:

>> -----Original Message-----
>> From: Eric Auger <eric.auger@redhat.com>
>> Sent: Thursday, October 17, 2024 6:16 PM
>> To: Cornelia Huck <cohuck@redhat.com>; Shameerali Kolothum Thodi
>> <shameerali.kolothum.thodi@huawei.com>; Oliver Upton
>> <oliver.upton@linux.dev>; Marc Zyngier <maz@kernel.org>
>> Cc: kvmarm@lists.linux.dev; catalin.marinas@arm.com; will@kernel.org;
>> mark.rutland@arm.com; yuzenghui <yuzenghui@huawei.com>; Wangzhou
>> (B) <wangzhou1@hisilicon.com>; jiangkunkun
>> <jiangkunkun@huawei.com>; Jonathan Cameron
>> <jonathan.cameron@huawei.com>; Anthony Jebson
>> <anthony.jebson@huawei.com>; linux-arm-kernel@lists.infradead.org;
>> Linuxarm <linuxarm@huawei.com>
>> Subject: Re: [RFC PATCH 0/6] KVM: arm64: Errata management for VM Live
>> migration
>> 
>> Hi Shameer,
>> 
>> On 10/17/24 17:49, Cornelia Huck wrote:
>> > On Fri, Oct 11 2024, Shameerali Kolothum Thodi
>> <shameerali.kolothum.thodi@huawei.com> wrote:
>> >
>> >>> -----Original Message-----
>> >>> From: Oliver Upton <oliver.upton@linux.dev>
>> >>> Sent: Friday, October 11, 2024 4:11 PM
>> >>> To: Marc Zyngier <maz@kernel.org>
>> >>> Cc: Shameerali Kolothum Thodi
>> >>> <shameerali.kolothum.thodi@huawei.com>; kvmarm@lists.linux.dev;
>> >>> catalin.marinas@arm.com; will@kernel.org; mark.rutland@arm.com;
>> >>> cohuck@redhat.com; eric.auger@redhat.com; yuzenghui
>> >>> <yuzenghui@huawei.com>; Wangzhou (B)
>> <wangzhou1@hisilicon.com>;
>> >>> jiangkunkun <jiangkunkun@huawei.com>; Jonathan Cameron
>> >>> <jonathan.cameron@huawei.com>; Anthony Jebson
>> >>> <anthony.jebson@huawei.com>; linux-arm-kernel@lists.infradead.org;
>> >>> Linuxarm <linuxarm@huawei.com>
>> >>> Subject: Re: [RFC PATCH 0/6] KVM: arm64: Errata management for VM
>> Live
>> >>> migration
>> >>>
>> >>> On Fri, Oct 11, 2024 at 12:43:28PM +0100, Marc Zyngier wrote:
>> >>>> On Fri, 11 Oct 2024 11:57:10 +0100, Shameerali Kolothum Thodi
>> >>> <shameerali.kolothum.thodi@huawei.com> wrote:
>> >>>>>>> Please take a look and let me know your thoughts.
>> >>>>>> Having eyeballed this very superficially, I think we can do
>> something
>> >>>>>> simpler, and maybe more future-proof:
>> >>>>> Thanks Marc for taking a look and the quick feedback.
>> >>>> No worries, that's the least I could do given that you put the effort
>> >>>> implementing my silly ideas! ;-)
>> >>>>
>> >>>>>> - I don't think KVM should be concerned about the description of
>> the
>> >>>>>>   target CPUs. The hypercall you defined is the right thing to do,
>> >>>>>>   but the VMM should completely handle it. That's an
>> implementation
>> >>>>>>   detail, but it would make things much simpler.
>> >>>>> Ok. So does that mean the hypercall will use some sort of shared
>> >>> memory
>> >>>>> to retrieve the list of target CPUs from VMM?
>> >>>> Two possibilities:
>> >>>>
>> >>>> - either shared memory, in which case the hypercall would require the
>> >>>>   guest to give an IPA and size for the VMM to write its stuff into
>> >>>>   the guest memory,
>> >>>>
>> >>>> - or more simply return the data as an MIDR/REVIDR pair in registers,
>> >>>>   the guest requesting an index, and getting an error when out of
>> >>>>   range, leaving it with the freedom to organise the storage.
>> >>>>
>> >>>> The second option is a bit slower, but way simpler, and it only
>> >>>> happens once per guest boot, so it would probably be my preferred
>> >>>> option unless this is proved to be impractical.
>> >>> Also worth noting there's existing UAPI [*] for allowing userspace to
>> >>> register range(s) of hypercalls that it services directly. It's a bit
>> >>> weird that we'd allow userspace to do stuff in KVM's own hypercall
>> >>> range, but I don't think it really matters at this point since this is
>> >>> all prototyping.
>> >>>
>> >>> [*]: https://docs.kernel.org/virt/kvm/devices/vm.html#attribute-kvm-
>> arm-
>> >>> vm-smccc-filter-w-o
>> >> Thanks. Yes and there are attempts to add that handling in Qemu[*] in
>> the context
>> >> of vCPU hotplug support(PSCI related ones though).  Will take a look.
>> >>
>> >> Thanks,
>> >> Shameer
>> >> [1] https://lore.kernel.org/qemu-devel/20241009033704.250287-1-
>> salil.mehta@huawei.com/
>> > Speaking of QEMU: Do you maybe already have some prototype code that
>> > tries to do something with the setup here? (I don't think QEMU currently
>> > mucks around with MIDR and friends when running with KVM; I wonder
>> what
>> > it should provide to the guest and if it should care to set something as
>> > a base level that gives guests not using the hypercall a chance to work
>> > properly.)
>
> I had a hacked Qemu to test this RFC. Nothing fancy but just to verify the  RFC.
> It basically adds a named CPU to virt as below,
>
> 1. Checks the host MIDR and REVIDR and if it is one of the supported platforms,
>    use the writable register interface to update the ID registers with a common
>    minimum    feature list.
> 2. Use the IOCTL in this RFC to update the target CPU list info.
> 3. Ignore the invariant register SET errors.

Thanks for sharing this information (no need to share the code, I
usually do not want to share my hacked-together-for-testing code, either
:)

>
>> >
>> As discussed during the KVM forum we are working on a qemu integration
>> for writable ID regs. The first goal is to be able to specialize the
>> host passthrough model (custom host model). Maybe this will trigger more
>> discussions on named models too. This is complementary to the
>> MIDR/REVIDR problematic and I hope we will be able to consolidate our
>> works at some point.
>
> Cool!. Happy to know that it is making progress. Once I rework this RFC based on the
> suggestions, I will rework the Qemu to have a hacked prototype to test it as well and
> probably can share that.

The reason I asked was mostly to get a feel about how you wanted to end
up using the interface. Did I understand correctly that you derive your
custom set of features from whatever comes as common features with the
MIDR/REVIDR, and do not treat "frankencpu with features common across
all possible targets" and "list of MIDR/REVIDR for all possible targets"
separately and tack them together in the end? It would also be
interesting to discuss what actual named models could look like (models
we could use for baselining etc.) But that's probably something that
needs to be discussed on qemu-devel anyway, and I certainly don't want
to derail the discussions about the KVM changes. (Agreeing on QEMU
interfaces is going to take a lot of work anyway :)

>
> Please keep me in loop if you have plans to share your Qemu work soon. That will
> be very helpful.
>
> Thanks,
> Shameer