Re: [RFC 1/3] target/arm/cpu: Add new CPU property for KVM regs to hide

[Eric Auger <eric.auger@redhat.com>]

Hi Connie,

On 10/15/25 3:12 PM, Cornelia Huck wrote:
> On Tue, Oct 14 2025, Eric Auger <eric.auger@redhat.com> wrote:
>
>> On 10/8/25 3:49 PM, Cornelia Huck wrote:
>>> On Fri, Oct 03 2025, Eric Auger <eric.auger@redhat.com> wrote:
>>>
>>>> Hi Sebastian,
>>>>
>>>> On 9/18/25 6:16 PM, Sebastian Ott wrote:
>>>>> On Thu, 11 Sep 2025, Eric Auger wrote:
>>>>>> New kernels sometimes expose new registers in an unconditionnal
>>>>>> manner.  This situation breaks backward migration as qemu notices
>>>>>> there are more registers to store on guest than supported in the
>>>>>> destination kerenl. This leads to a "failed to load
>>>>>> cpu:cpreg_vmstate_array_len" error.
>>>>>>
>>>>>> A good example is the introduction of KVM_REG_ARM_VENDOR_HYP_BMAP_2
>>>>>> pseudo FW register in v6.16 by commit C0000e58c74e (“KVM: arm64:
>>>>>> Introduce KVM_REG_ARM_VENDOR_HYP_BMAP_2”). Trying to do backward
>>>>>> migration from a host kernel which features the commit to a destination
>>>>>> host that doesn't fail.
>>>>>>
>>>>>> Currently QEMU is not using that feature so ignoring this latter
>>>>>> is not a problem. An easy way to fix the migration issue is to teach
>>>>>> qemu we don't care about that register and we can simply ignore it,
>>>>>> including its state migration.
>>>>>>
>>>>>> This patch introduces a CPU property, under the form of an array of
>>>>>> reg indices which indicates which registers can be ignored.
>>>>>>
>>>>>> The goal then is to set this property in machine type compats such
>>>>>> as:
>>>>>> static GlobalProperty arm_virt_kernel_compat_10_1[] = {
>>>>>>    /* KVM_REG_ARM_VENDOR_HYP_BMAP_2 */
>>>>>>    { TYPE_ARM_CPU, "kvm-hidden-regs", "0x6030000000160003" },
>>>>>> }
>>>>> One thing worth noting - once this series lands:
>>>>> https://lore.kernel.org/qemu-devel/20250801074730.28329-1-shameerkolothum@gmail.com/
>>>>>
>>>>> we might need to add a bit more logic here. Either using the kvm
>>>>> interfaces (only ignore KVM_REG_ARM_VENDOR_HYP_BMAP_2 when the register
>>>>> value is 0) or qemu knowledge (only ignore KVM_REG_ARM_VENDOR_HYP_BMAP_2
>>>>> when the impl-cpu property is not used).
>>>> Effectively if we "hide" KVM_REG_ARM_VENDOR_HYP_BMAP_2 on save/restore
>>>> we must enforce the reg is not used by userspace.
>>>>
>>>> One way would be to test whether KVM_REG_ARM_VENDOR_HYP_BMAP_2 is hidden
>>>> in kvm_arm_target_impl_cpus_supported() and if it is, report false.
>>>> However for every new functionality in qemu it does not sound sensible
>>>> to check whether new KVM regs being used are anonymously hidden.
>>>>
>>>> Another way could be to fail kvm_set_one_reg/kvm_get_one_reg in case the
>>>> register is hidden. That way in Shameer's series, kvm_arch_init_vcpu()
>>>> would fail if BMAP_2 is hidden, ie. in our example for all machines
>>>> types before 10.2. By the way adding Shameer.
>>> I think tying this to the state of the reg (hidden or not) is less
>>> error-prone (I'd assume we'd have different ways of detecting whether
>>> something is used for future cases, and "is the reg hidden?" would work
>>> in all cases.) We'd need to tie migration to matching machine versions
>>> anyway, I think.
>>>
>> I guess you suggest to check the hidden/fake state in
>>
>> kvm_set_one_reg/kvm_get_one_reg too. One issue is those helpers are arch agnostic. I would need to either introduce a callback in the CPU class to check the actual status or add the props in the parent CPU object. Or introduce a KVM IOTCL to teach KVM a reg shall never be accessed.
> Aren't they always called from arch code, though? We could easily wrap
> them for arm, I think. (Unless there's some more generic interest in
> conditional ONE_REG accesses.)

For the time being I chose to add a new CPU class callback to check
whether the reg is hidden in kvm_set/get_one_reg.
Let's wait for other comments ... ;-)

Thanks

Eric
>