Re: [RFC 1/3] target/arm/cpu: Add new CPU property for KVM regs to hide

[Eric Auger <eric.auger@redhat.com>]

On 10/8/25 3:49 PM, Cornelia Huck wrote:
> On Fri, Oct 03 2025, Eric Auger <eric.auger@redhat.com> wrote:
>
>> Hi Sebastian,
>>
>> On 9/18/25 6:16 PM, Sebastian Ott wrote:
>>> On Thu, 11 Sep 2025, Eric Auger wrote:
>>>> New kernels sometimes expose new registers in an unconditionnal
>>>> manner.  This situation breaks backward migration as qemu notices
>>>> there are more registers to store on guest than supported in the
>>>> destination kerenl. This leads to a "failed to load
>>>> cpu:cpreg_vmstate_array_len" error.
>>>>
>>>> A good example is the introduction of KVM_REG_ARM_VENDOR_HYP_BMAP_2
>>>> pseudo FW register in v6.16 by commit C0000e58c74e (“KVM: arm64:
>>>> Introduce KVM_REG_ARM_VENDOR_HYP_BMAP_2”). Trying to do backward
>>>> migration from a host kernel which features the commit to a destination
>>>> host that doesn't fail.
>>>>
>>>> Currently QEMU is not using that feature so ignoring this latter
>>>> is not a problem. An easy way to fix the migration issue is to teach
>>>> qemu we don't care about that register and we can simply ignore it,
>>>> including its state migration.
>>>>
>>>> This patch introduces a CPU property, under the form of an array of
>>>> reg indices which indicates which registers can be ignored.
>>>>
>>>> The goal then is to set this property in machine type compats such
>>>> as:
>>>> static GlobalProperty arm_virt_kernel_compat_10_1[] = {
>>>>    /* KVM_REG_ARM_VENDOR_HYP_BMAP_2 */
>>>>    { TYPE_ARM_CPU, "kvm-hidden-regs", "0x6030000000160003" },
>>>> }
>>> One thing worth noting - once this series lands:
>>> https://lore.kernel.org/qemu-devel/20250801074730.28329-1-shameerkolothum@gmail.com/
>>>
>>> we might need to add a bit more logic here. Either using the kvm
>>> interfaces (only ignore KVM_REG_ARM_VENDOR_HYP_BMAP_2 when the register
>>> value is 0) or qemu knowledge (only ignore KVM_REG_ARM_VENDOR_HYP_BMAP_2
>>> when the impl-cpu property is not used).
>> Effectively if we "hide" KVM_REG_ARM_VENDOR_HYP_BMAP_2 on save/restore
>> we must enforce the reg is not used by userspace.
>>
>> One way would be to test whether KVM_REG_ARM_VENDOR_HYP_BMAP_2 is hidden
>> in kvm_arm_target_impl_cpus_supported() and if it is, report false.
>> However for every new functionality in qemu it does not sound sensible
>> to check whether new KVM regs being used are anonymously hidden.
>>
>> Another way could be to fail kvm_set_one_reg/kvm_get_one_reg in case the
>> register is hidden. That way in Shameer's series, kvm_arch_init_vcpu()
>> would fail if BMAP_2 is hidden, ie. in our example for all machines
>> types before 10.2. By the way adding Shameer.
> I think tying this to the state of the reg (hidden or not) is less
> error-prone (I'd assume we'd have different ways of detecting whether
> something is used for future cases, and "is the reg hidden?" would work
> in all cases.) We'd need to tie migration to matching machine versions
> anyway, I think.
>
I guess you suggest to check the hidden/fake state in

kvm_set_one_reg/kvm_get_one_reg too. One issue is those helpers are arch agnostic. I would need to either introduce a callback in the CPU class to check the actual status or add the props in the parent CPU object. Or introduce a KVM IOTCL to teach KVM a reg shall never be accessed.

Thoughts?

Thanks

Eric