Re: [RFC 1/3] target/arm/cpu: Add new CPU property for KVM regs to hide

[Cornelia Huck <cohuck@redhat.com>]

On Tue, Oct 14 2025, Eric Auger <eric.auger@redhat.com> wrote:

> On 10/8/25 3:49 PM, Cornelia Huck wrote:
>> On Fri, Oct 03 2025, Eric Auger <eric.auger@redhat.com> wrote:
>>
>>> Hi Sebastian,
>>>
>>> On 9/18/25 6:16 PM, Sebastian Ott wrote:
>>>> On Thu, 11 Sep 2025, Eric Auger wrote:
>>>>> New kernels sometimes expose new registers in an unconditionnal
>>>>> manner.  This situation breaks backward migration as qemu notices
>>>>> there are more registers to store on guest than supported in the
>>>>> destination kerenl. This leads to a "failed to load
>>>>> cpu:cpreg_vmstate_array_len" error.
>>>>>
>>>>> A good example is the introduction of KVM_REG_ARM_VENDOR_HYP_BMAP_2
>>>>> pseudo FW register in v6.16 by commit C0000e58c74e (“KVM: arm64:
>>>>> Introduce KVM_REG_ARM_VENDOR_HYP_BMAP_2”). Trying to do backward
>>>>> migration from a host kernel which features the commit to a destination
>>>>> host that doesn't fail.
>>>>>
>>>>> Currently QEMU is not using that feature so ignoring this latter
>>>>> is not a problem. An easy way to fix the migration issue is to teach
>>>>> qemu we don't care about that register and we can simply ignore it,
>>>>> including its state migration.
>>>>>
>>>>> This patch introduces a CPU property, under the form of an array of
>>>>> reg indices which indicates which registers can be ignored.
>>>>>
>>>>> The goal then is to set this property in machine type compats such
>>>>> as:
>>>>> static GlobalProperty arm_virt_kernel_compat_10_1[] = {
>>>>>    /* KVM_REG_ARM_VENDOR_HYP_BMAP_2 */
>>>>>    { TYPE_ARM_CPU, "kvm-hidden-regs", "0x6030000000160003" },
>>>>> }
>>>> One thing worth noting - once this series lands:
>>>> https://lore.kernel.org/qemu-devel/20250801074730.28329-1-shameerkolothum@gmail.com/
>>>>
>>>> we might need to add a bit more logic here. Either using the kvm
>>>> interfaces (only ignore KVM_REG_ARM_VENDOR_HYP_BMAP_2 when the register
>>>> value is 0) or qemu knowledge (only ignore KVM_REG_ARM_VENDOR_HYP_BMAP_2
>>>> when the impl-cpu property is not used).
>>> Effectively if we "hide" KVM_REG_ARM_VENDOR_HYP_BMAP_2 on save/restore
>>> we must enforce the reg is not used by userspace.
>>>
>>> One way would be to test whether KVM_REG_ARM_VENDOR_HYP_BMAP_2 is hidden
>>> in kvm_arm_target_impl_cpus_supported() and if it is, report false.
>>> However for every new functionality in qemu it does not sound sensible
>>> to check whether new KVM regs being used are anonymously hidden.
>>>
>>> Another way could be to fail kvm_set_one_reg/kvm_get_one_reg in case the
>>> register is hidden. That way in Shameer's series, kvm_arch_init_vcpu()
>>> would fail if BMAP_2 is hidden, ie. in our example for all machines
>>> types before 10.2. By the way adding Shameer.
>> I think tying this to the state of the reg (hidden or not) is less
>> error-prone (I'd assume we'd have different ways of detecting whether
>> something is used for future cases, and "is the reg hidden?" would work
>> in all cases.) We'd need to tie migration to matching machine versions
>> anyway, I think.
>>
> I guess you suggest to check the hidden/fake state in
>
> kvm_set_one_reg/kvm_get_one_reg too. One issue is those helpers are arch agnostic. I would need to either introduce a callback in the CPU class to check the actual status or add the props in the parent CPU object. Or introduce a KVM IOTCL to teach KVM a reg shall never be accessed.

Aren't they always called from arch code, though? We could easily wrap
them for arm, I think. (Unless there's some more generic interest in
conditional ONE_REG accesses.)