Question about heterogeneous VM live migration

Question about heterogeneous VM live migration

[Zhou Wang]

Hi,

We are now trying to do heterogeneous VM live migration among HiSilicon ARM
servers, seems there are problems about disabling a feature in guest.

For a feature, if we disable it in VM by configure related ID register field,
we should make it actually been disabled, e.g. configure related control
register in EL2 or trap EL0/EL1 access to EL2.

Possible problems:

1. Some features can not be disabled actually in EL0/EL1, e.g. FEAT_AFP,
   FEAT_RPRES, FEAT_CSSC, FEAT_LRCPC3...

   Disabling it by ID can not avoid a stupid user to directly use it without ID
   checking, which may bring subtle problem in heterogeneous VM live migration.

2. For some features, it can be trapped, but KVM does not support yet. Not sure
   if we should support them in future.

   E.g. If we disable ID of FEAT_HAFT for guest, we need configure
        HCRX_EL2.TCR2En to 0, so access to TCR2_EL1.HAFT will be trapped.

        For FEAT_PAN3, it instroduces EPAN to SCTLR_EL1，if we disable ID of
	FEAT_PAN3, we need make SCTLR_EL1 to trap by setting HFGRTR_EL2.SCTLR_EL1.

   Seems there are no trap setting in above cases. Just a quick look, maybe I
   miss something.

I am not sure if we already consider above problems, any help will be appreciated.

Best,
Zhou

Re: Question about heterogeneous VM live migration

[Marc Zyngier]

On Thu, 16 Oct 2025 03:00:07 +0100,
Zhou Wang <wangzhou1@hisilicon.com> wrote:
> 
> Hi,
> 
> We are now trying to do heterogeneous VM live migration among HiSilicon ARM
> servers, seems there are problems about disabling a feature in guest.
> 
> For a feature, if we disable it in VM by configure related ID register field,
> we should make it actually been disabled, e.g. configure related control
> register in EL2 or trap EL0/EL1 access to EL2.

This is in general not possible.

> Possible problems:
> 
> 1. Some features can not be disabled actually in EL0/EL1, e.g. FEAT_AFP,
>    FEAT_RPRES, FEAT_CSSC, FEAT_LRCPC3...
> 
>    Disabling it by ID can not avoid a stupid user to directly use it without ID
>    checking, which may bring subtle problem in heterogeneous VM live migration.

Yes. There is no solution to that.

> 2. For some features, it can be trapped, but KVM does not support yet. Not sure
>    if we should support them in future.
> 
>    E.g. If we disable ID of FEAT_HAFT for guest, we need configure
>         HCRX_EL2.TCR2En to 0, so access to TCR2_EL1.HAFT will be trapped.

That's wrong. If you force TCR2EN to 0, none of the TCR2_EL1 controls
will work. For example, you'd lose the POE/PIE controls.

>         For FEAT_PAN3, it instroduces EPAN to SCTLR_EL1，if we disable ID of
> 	FEAT_PAN3, we need make SCTLR_EL1 to trap by setting HFGRTR_EL2.SCTLR_EL1.
> 
>    Seems there are no trap setting in above cases. Just a quick look, maybe I
>    miss something.
> 
> I am not sure if we already consider above problems, any help will be appreciated.

we know about most of these things already. In general, you cannot
hide features that the host has, other than indicating to the guest
that they may not exist. The underlying HW is still there and will act
as expected.

I think you're simply expecting too much from the architecture.

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: Question about heterogeneous VM live migration

[Zhou Wang]

On 2025/10/17 21:12, Marc Zyngier wrote:
> On Thu, 16 Oct 2025 03:00:07 +0100,
> Zhou Wang <wangzhou1@hisilicon.com> wrote:
>>
>> Hi,
>>
>> We are now trying to do heterogeneous VM live migration among HiSilicon ARM
>> servers, seems there are problems about disabling a feature in guest.
>>
>> For a feature, if we disable it in VM by configure related ID register field,
>> we should make it actually been disabled, e.g. configure related control
>> register in EL2 or trap EL0/EL1 access to EL2.
> 
> This is in general not possible.
> 
>> Possible problems:
>>
>> 1. Some features can not be disabled actually in EL0/EL1, e.g. FEAT_AFP,
>>    FEAT_RPRES, FEAT_CSSC, FEAT_LRCPC3...
>>
>>    Disabling it by ID can not avoid a stupid user to directly use it without ID
>>    checking, which may bring subtle problem in heterogeneous VM live migration.
> 
> Yes. There is no solution to that.
> 
>> 2. For some features, it can be trapped, but KVM does not support yet. Not sure
>>    if we should support them in future.
>>
>>    E.g. If we disable ID of FEAT_HAFT for guest, we need configure
>>         HCRX_EL2.TCR2En to 0, so access to TCR2_EL1.HAFT will be trapped.
> 
> That's wrong. If you force TCR2EN to 0, none of the TCR2_EL1 controls
> will work. For example, you'd lose the POE/PIE controls.

Yes, it will impact other control bits, same as SCTLR_EL1 trapping mentioned below.

> 
>>         For FEAT_PAN3, it instroduces EPAN to SCTLR_EL1，if we disable ID of
>> 	FEAT_PAN3, we need make SCTLR_EL1 to trap by setting HFGRTR_EL2.SCTLR_EL1.
>>
>>    Seems there are no trap setting in above cases. Just a quick look, maybe I
>>    miss something.
>>
>> I am not sure if we already consider above problems, any help will be appreciated.
> 
> we know about most of these things already. In general, you cannot
> hide features that the host has, other than indicating to the guest
> that they may not exist. The underlying HW is still there and will act
> as expected.
> 
> I think you're simply expecting too much from the architecture.

Many thanks about above explanation, I get the situation now.

Best,
Zhou

> 
> 	M.
>