RE: [RFC PATCH 0/6] KVM: arm64: Errata management for VM Live migration

[Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>]

> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: Thursday, October 17, 2024 6:16 PM
> To: Cornelia Huck <cohuck@redhat.com>; Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com>; Oliver Upton
> <oliver.upton@linux.dev>; Marc Zyngier <maz@kernel.org>
> Cc: kvmarm@lists.linux.dev; catalin.marinas@arm.com; will@kernel.org;
> mark.rutland@arm.com; yuzenghui <yuzenghui@huawei.com>; Wangzhou
> (B) <wangzhou1@hisilicon.com>; jiangkunkun
> <jiangkunkun@huawei.com>; Jonathan Cameron
> <jonathan.cameron@huawei.com>; Anthony Jebson
> <anthony.jebson@huawei.com>; linux-arm-kernel@lists.infradead.org;
> Linuxarm <linuxarm@huawei.com>
> Subject: Re: [RFC PATCH 0/6] KVM: arm64: Errata management for VM Live
> migration
> 
> Hi Shameer,
> 
> On 10/17/24 17:49, Cornelia Huck wrote:
> > On Fri, Oct 11 2024, Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com> wrote:
> >
> >>> -----Original Message-----
> >>> From: Oliver Upton <oliver.upton@linux.dev>
> >>> Sent: Friday, October 11, 2024 4:11 PM
> >>> To: Marc Zyngier <maz@kernel.org>
> >>> Cc: Shameerali Kolothum Thodi
> >>> <shameerali.kolothum.thodi@huawei.com>; kvmarm@lists.linux.dev;
> >>> catalin.marinas@arm.com; will@kernel.org; mark.rutland@arm.com;
> >>> cohuck@redhat.com; eric.auger@redhat.com; yuzenghui
> >>> <yuzenghui@huawei.com>; Wangzhou (B)
> <wangzhou1@hisilicon.com>;
> >>> jiangkunkun <jiangkunkun@huawei.com>; Jonathan Cameron
> >>> <jonathan.cameron@huawei.com>; Anthony Jebson
> >>> <anthony.jebson@huawei.com>; linux-arm-kernel@lists.infradead.org;
> >>> Linuxarm <linuxarm@huawei.com>
> >>> Subject: Re: [RFC PATCH 0/6] KVM: arm64: Errata management for VM
> Live
> >>> migration
> >>>
> >>> On Fri, Oct 11, 2024 at 12:43:28PM +0100, Marc Zyngier wrote:
> >>>> On Fri, 11 Oct 2024 11:57:10 +0100, Shameerali Kolothum Thodi
> >>> <shameerali.kolothum.thodi@huawei.com> wrote:
> >>>>>>> Please take a look and let me know your thoughts.
> >>>>>> Having eyeballed this very superficially, I think we can do
> something
> >>>>>> simpler, and maybe more future-proof:
> >>>>> Thanks Marc for taking a look and the quick feedback.
> >>>> No worries, that's the least I could do given that you put the effort
> >>>> implementing my silly ideas! ;-)
> >>>>
> >>>>>> - I don't think KVM should be concerned about the description of
> the
> >>>>>>   target CPUs. The hypercall you defined is the right thing to do,
> >>>>>>   but the VMM should completely handle it. That's an
> implementation
> >>>>>>   detail, but it would make things much simpler.
> >>>>> Ok. So does that mean the hypercall will use some sort of shared
> >>> memory
> >>>>> to retrieve the list of target CPUs from VMM?
> >>>> Two possibilities:
> >>>>
> >>>> - either shared memory, in which case the hypercall would require the
> >>>>   guest to give an IPA and size for the VMM to write its stuff into
> >>>>   the guest memory,
> >>>>
> >>>> - or more simply return the data as an MIDR/REVIDR pair in registers,
> >>>>   the guest requesting an index, and getting an error when out of
> >>>>   range, leaving it with the freedom to organise the storage.
> >>>>
> >>>> The second option is a bit slower, but way simpler, and it only
> >>>> happens once per guest boot, so it would probably be my preferred
> >>>> option unless this is proved to be impractical.
> >>> Also worth noting there's existing UAPI [*] for allowing userspace to
> >>> register range(s) of hypercalls that it services directly. It's a bit
> >>> weird that we'd allow userspace to do stuff in KVM's own hypercall
> >>> range, but I don't think it really matters at this point since this is
> >>> all prototyping.
> >>>
> >>> [*]: https://docs.kernel.org/virt/kvm/devices/vm.html#attribute-kvm-
> arm-
> >>> vm-smccc-filter-w-o
> >> Thanks. Yes and there are attempts to add that handling in Qemu[*] in
> the context
> >> of vCPU hotplug support(PSCI related ones though).  Will take a look.
> >>
> >> Thanks,
> >> Shameer
> >> [1] https://lore.kernel.org/qemu-devel/20241009033704.250287-1-
> salil.mehta@huawei.com/
> > Speaking of QEMU: Do you maybe already have some prototype code that
> > tries to do something with the setup here? (I don't think QEMU currently
> > mucks around with MIDR and friends when running with KVM; I wonder
> what
> > it should provide to the guest and if it should care to set something as
> > a base level that gives guests not using the hypercall a chance to work
> > properly.)

I had a hacked Qemu to test this RFC. Nothing fancy but just to verify the  RFC.
It basically adds a named CPU to virt as below,

1. Checks the host MIDR and REVIDR and if it is one of the supported platforms,
   use the writable register interface to update the ID registers with a common
   minimum    feature list.
2. Use the IOCTL in this RFC to update the target CPU list info.
3. Ignore the invariant register SET errors.

> >
> As discussed during the KVM forum we are working on a qemu integration
> for writable ID regs. The first goal is to be able to specialize the
> host passthrough model (custom host model). Maybe this will trigger more
> discussions on named models too. This is complementary to the
> MIDR/REVIDR problematic and I hope we will be able to consolidate our
> works at some point.

Cool!. Happy to know that it is making progress. Once I rework this RFC based on the
suggestions, I will rework the Qemu to have a hacked prototype to test it as well and
probably can share that.

Please keep me in loop if you have plans to share your Qemu work soon. That will
be very helpful.

Thanks,
Shameer