[RFC PATCH v1 0/2] KVM: arm64: Making BT Field in ID_AA64PFR1_EL1 writable

[RFC PATCH v1 0/2] KVM: arm64: Making BT Field in ID_AA64PFR1_EL1 writable

[Shaoqin Huang]

Hi guys,

I'm trying to enable migration from MtCollins(Ampere Altra, ARMv8.2+) to
AmpereOne(AmpereOne, ARMv8.6+), the migration always fails when migration from
MtCollins to AmpereOne due to some register fields differing between the
two machines.

In this patch series, we try to make more register fields writable like
ID_AA64PFR1_EL1.BT. This is first step towards making the migration possible.
Some other hurdles need to be overcome. This is not sufficient to make the
migration successful from MtCollins to AmpereOne.

Shaoqin Huang (2):
  KVM: arm64: Allow BT field in ID_AA64PFR1_EL1 writable
  KVM: selftests: aarch64: Add writable test for ID_AA64PFR1_EL1

 arch/arm64/kvm/sys_regs.c                         | 2 +-
 tools/testing/selftests/kvm/aarch64/set_id_regs.c | 6 ++++++
 2 files changed, 7 insertions(+), 1 deletion(-)

-- 
2.40.1

[RFC PATCH v1 2/2] KVM: selftests: aarch64: Add writable test for ID_AA64PFR1_EL1

[Shaoqin Huang]

Add test for the BT field in the ID_AA64PFR1_EL1 register.

Signed-off-by: Shaoqin Huang <shahuang@redhat.com>
---
 tools/testing/selftests/kvm/aarch64/set_id_regs.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/tools/testing/selftests/kvm/aarch64/set_id_regs.c b/tools/testing/selftests/kvm/aarch64/set_id_regs.c
index 16e2338686c1..5381b8ec5562 100644
--- a/tools/testing/selftests/kvm/aarch64/set_id_regs.c
+++ b/tools/testing/selftests/kvm/aarch64/set_id_regs.c
@@ -133,6 +133,11 @@ static const struct reg_ftr_bits ftr_id_aa64pfr0_el1[] = {
 	REG_FTR_END,
 };
 
+static const struct reg_ftr_bits ftr_id_aa64pfr1_el1[] = {
+	REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64PFR1_EL1, BT, 0),
+	REG_FTR_END,
+};
+
 static const struct reg_ftr_bits ftr_id_aa64mmfr0_el1[] = {
 	REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64MMFR0_EL1, ECV, 0),
 	REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64MMFR0_EL1, EXS, 0),
@@ -199,6 +204,7 @@ static struct test_feature_reg test_regs[] = {
 	TEST_REG(SYS_ID_AA64ISAR1_EL1, ftr_id_aa64isar1_el1),
 	TEST_REG(SYS_ID_AA64ISAR2_EL1, ftr_id_aa64isar2_el1),
 	TEST_REG(SYS_ID_AA64PFR0_EL1, ftr_id_aa64pfr0_el1),
+	TEST_REG(SYS_ID_AA64PFR1_EL1, ftr_id_aa64pfr1_el1),
 	TEST_REG(SYS_ID_AA64MMFR0_EL1, ftr_id_aa64mmfr0_el1),
 	TEST_REG(SYS_ID_AA64MMFR1_EL1, ftr_id_aa64mmfr1_el1),
 	TEST_REG(SYS_ID_AA64MMFR2_EL1, ftr_id_aa64mmfr2_el1),
-- 
2.40.1

Re: [RFC PATCH v1 0/2] KVM: arm64: Making BT Field in ID_AA64PFR1_EL1 writable

[Oliver Upton]

Hi Shaoqin,

On Tue, Jun 11, 2024 at 10:35:50PM -0400, Shaoqin Huang wrote:
> Hi guys,
> 
> I'm trying to enable migration from MtCollins(Ampere Altra, ARMv8.2+) to
> AmpereOne(AmpereOne, ARMv8.6+), the migration always fails when migration from
> MtCollins to AmpereOne due to some register fields differing between the
> two machines.
> 
> In this patch series, we try to make more register fields writable like
> ID_AA64PFR1_EL1.BT. This is first step towards making the migration possible.
> Some other hurdles need to be overcome. This is not sufficient to make the
> migration successful from MtCollins to AmpereOne.

It isn't possible to transparently migrate between these systems. The
former has a cntfrq of 25MHz, and the latter has a cntfrq of 1GHz. There
isn't a mechanism for scaling the counter frequency, and I have zero
appetite for a paravirt interface.

On top of that, erratum AC03_CPU_38 seems to make a migration from
Neoverse-N1 to AmpereOne quite perilous, unless you hide FEAT_HAFDBS on
the source.

These issues are separate, though, from any possible changes to the
writability of ID_AA64PFR1_EL1, which still may be useful to userspace.

-- 
Thanks,
Oliver

Re: [RFC PATCH v1 0/2] KVM: arm64: Making BT Field in ID_AA64PFR1_EL1 writable

[Shaoqin Huang]

Hi Oliver,

On 6/12/24 13:30, Oliver Upton wrote:
> Hi Shaoqin,
> 
> On Tue, Jun 11, 2024 at 10:35:50PM -0400, Shaoqin Huang wrote:
>> Hi guys,
>>
>> I'm trying to enable migration from MtCollins(Ampere Altra, ARMv8.2+) to
>> AmpereOne(AmpereOne, ARMv8.6+), the migration always fails when migration from
>> MtCollins to AmpereOne due to some register fields differing between the
>> two machines.
>>
>> In this patch series, we try to make more register fields writable like
>> ID_AA64PFR1_EL1.BT. This is first step towards making the migration possible.
>> Some other hurdles need to be overcome. This is not sufficient to make the
>> migration successful from MtCollins to AmpereOne.
> 
> It isn't possible to transparently migrate between these systems. The
> former has a cntfrq of 25MHz, and the latter has a cntfrq of 1GHz. There
> isn't a mechanism for scaling the counter frequency, and I have zero
> appetite for a paravirt interface.

Thanks for letting me know the cntfrq will block the migration between 
the two machine. And we don't have the solution for it, which means it's 
impossible to migrate between these systems which have different cntfrq.

> 
> On top of that, erratum AC03_CPU_38 seems to make a migration from
> Neoverse-N1 to AmpereOne quite perilous, unless you hide FEAT_HAFDBS on
> the source.
> 
> These issues are separate, though, from any possible changes to the
> writability of ID_AA64PFR1_EL1, which still may be useful to userspace.
> 
I think I can still making the ID_AA64PFR1_EL1 writable to enable 
migration on some other machines.

Thanks,
Shaoqin

-- 
Shaoqin

Re: [RFC PATCH v1 0/2] KVM: arm64: Making BT Field in ID_AA64PFR1_EL1 writable

[Marc Zyngier]

On Wed, 12 Jun 2024 06:30:51 +0100,
Oliver Upton <oliver.upton@linux.dev> wrote:
> 
> Hi Shaoqin,
> 
> On Tue, Jun 11, 2024 at 10:35:50PM -0400, Shaoqin Huang wrote:
> > Hi guys,
> > 
> > I'm trying to enable migration from MtCollins(Ampere Altra, ARMv8.2+) to
> > AmpereOne(AmpereOne, ARMv8.6+), the migration always fails when migration from
> > MtCollins to AmpereOne due to some register fields differing between the
> > two machines.
> > 
> > In this patch series, we try to make more register fields writable like
> > ID_AA64PFR1_EL1.BT. This is first step towards making the migration possible.
> > Some other hurdles need to be overcome. This is not sufficient to make the
> > migration successful from MtCollins to AmpereOne.
> 
> It isn't possible to transparently migrate between these systems. The
> former has a cntfrq of 25MHz, and the latter has a cntfrq of 1GHz. There
> isn't a mechanism for scaling the counter frequency, and I have zero
> appetite for a paravirt interface.

Note that there *is* an architectural workaround in the form of
FEAT_CNTSC. But of course:

- it is optional (and likely not implemented)
- it is global (hence affecting all SW running on the machine)
- it invalidates the requirements of ARMv8.6 (who cares?)
- KVM has nothing to do with it (yay!)

So if the two systems (from the same manufacturer) were ever designed
to allow migration between the two, they would have at least baked
some of that in.

As for the paravirt interface, I agree that this is a non-starter
(been there, done that, dumped it in the bin).

The patch itself is interesting and may be of use once it has been put
to a compiler and not just dumped on the list without any testing.

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [RFC PATCH v1 0/2] KVM: arm64: Making BT Field in ID_AA64PFR1_EL1 writable

[Shaoqin Huang]

Hi Marc,

On 6/12/24 18:07, Marc Zyngier wrote:
> On Wed, 12 Jun 2024 06:30:51 +0100,
> Oliver Upton <oliver.upton@linux.dev> wrote:
>>
>> Hi Shaoqin,
>>
>> On Tue, Jun 11, 2024 at 10:35:50PM -0400, Shaoqin Huang wrote:
>>> Hi guys,
>>>
>>> I'm trying to enable migration from MtCollins(Ampere Altra, ARMv8.2+) to
>>> AmpereOne(AmpereOne, ARMv8.6+), the migration always fails when migration from
>>> MtCollins to AmpereOne due to some register fields differing between the
>>> two machines.
>>>
>>> In this patch series, we try to make more register fields writable like
>>> ID_AA64PFR1_EL1.BT. This is first step towards making the migration possible.
>>> Some other hurdles need to be overcome. This is not sufficient to make the
>>> migration successful from MtCollins to AmpereOne.
>>
>> It isn't possible to transparently migrate between these systems. The
>> former has a cntfrq of 25MHz, and the latter has a cntfrq of 1GHz. There
>> isn't a mechanism for scaling the counter frequency, and I have zero
>> appetite for a paravirt interface.
> 
> Note that there *is* an architectural workaround in the form of
> FEAT_CNTSC. But of course:
> 
> - it is optional (and likely not implemented)
> - it is global (hence affecting all SW running on the machine)
> - it invalidates the requirements of ARMv8.6 (who cares?)
> - KVM has nothing to do with it (yay!)
> 
> So if the two systems (from the same manufacturer) were ever designed
> to allow migration between the two, they would have at least baked
> some of that in.
> 
> As for the paravirt interface, I agree that this is a non-starter
> (been there, done that, dumped it in the bin).
> 
> The patch itself is interesting and may be of use once it has been put
> to a compiler and not just dumped on the list without any testing.
> 
> 	M.

Thanks for putting your comments here.

If we don't care about the FEAT_CNTSC right now. Could I fix the compile 
issue and respin this again without the background of enabling migration 
between MtCollins and AmpereOne, and just keep the information of the 
different BT field between different machine?

Thanks,
Shaoqin

> 

-- 
Shaoqin

Re: [RFC PATCH v1 0/2] KVM: arm64: Making BT Field in ID_AA64PFR1_EL1 writable

[Oliver Upton]

On Thu, Jun 13, 2024 at 04:31:45PM +0800, Shaoqin Huang wrote:
> Hi Marc,
> 
> On 6/12/24 18:07, Marc Zyngier wrote:
> > On Wed, 12 Jun 2024 06:30:51 +0100,
> > Oliver Upton <oliver.upton@linux.dev> wrote:
> > > 
> > > Hi Shaoqin,
> > > 
> > > On Tue, Jun 11, 2024 at 10:35:50PM -0400, Shaoqin Huang wrote:
> > > > Hi guys,
> > > > 
> > > > I'm trying to enable migration from MtCollins(Ampere Altra, ARMv8.2+) to
> > > > AmpereOne(AmpereOne, ARMv8.6+), the migration always fails when migration from
> > > > MtCollins to AmpereOne due to some register fields differing between the
> > > > two machines.
> > > > 
> > > > In this patch series, we try to make more register fields writable like
> > > > ID_AA64PFR1_EL1.BT. This is first step towards making the migration possible.
> > > > Some other hurdles need to be overcome. This is not sufficient to make the
> > > > migration successful from MtCollins to AmpereOne.
> > > 
> > > It isn't possible to transparently migrate between these systems. The
> > > former has a cntfrq of 25MHz, and the latter has a cntfrq of 1GHz. There
> > > isn't a mechanism for scaling the counter frequency, and I have zero
> > > appetite for a paravirt interface.
> > 
> > Note that there *is* an architectural workaround in the form of
> > FEAT_CNTSC. But of course:

Heh, I should've further specified a per-CPU mechanism :)

> > - it is optional (and likely not implemented)
> > - it is global (hence affecting all SW running on the machine)
> > - it invalidates the requirements of ARMv8.6 (who cares?)
> > - KVM has nothing to do with it (yay!)
> > 
> > So if the two systems (from the same manufacturer) were ever designed
> > to allow migration between the two, they would have at least baked
> > some of that in.
> > 
> > As for the paravirt interface, I agree that this is a non-starter
> > (been there, done that, dumped it in the bin).
> > 
> > The patch itself is interesting and may be of use once it has been put
> > to a compiler and not just dumped on the list without any testing.
> > 
> > 	M.
> 
> Thanks for putting your comments here.
> 
> If we don't care about the FEAT_CNTSC right now. Could I fix the compile
> issue and respin this again without the background of enabling migration
> between MtCollins and AmpereOne, and just keep the information of the
> different BT field between different machine?

I don't think cross-platform migration is relevant for the KVM UAPI
you're trying to augment. We want to give userspace the ability to control
the visible feature set for a VM, which *could* be used by userspace in such
a way to transparently migrate VMs.

So if you could focus the changelog purely as an improvement upon the
existing controls we expose to the VMM then the patch is more obviously
justified.

Please do respin.

-- 
Thanks,
Oliver

Re: [RFC PATCH v1 0/2] KVM: arm64: Making BT Field in ID_AA64PFR1_EL1 writable

[Marc Zyngier]

On Thu, 13 Jun 2024 09:31:45 +0100,
Shaoqin Huang <shahuang@redhat.com> wrote:
> 
> If we don't care about the FEAT_CNTSC right now. Could I fix the
> compile issue and respin this again without the background of enabling
> migration between MtCollins and AmpereOne, and just keep the
> information of the different BT field between different machine?

As I said, I think this patch is valuable. But maybe you should
consider tackling the full register, rather than only addressing a
single field.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [RFC PATCH v1 0/2] KVM: arm64: Making BT Field in ID_AA64PFR1_EL1 writable

[Shaoqin Huang]

Hi Marc,

On 6/13/24 16:42, Marc Zyngier wrote:
> On Thu, 13 Jun 2024 09:31:45 +0100,
> Shaoqin Huang <shahuang@redhat.com> wrote:
>>
>> If we don't care about the FEAT_CNTSC right now. Could I fix the
>> compile issue and respin this again without the background of enabling
>> migration between MtCollins and AmpereOne, and just keep the
>> information of the different BT field between different machine?
> 
> As I said, I think this patch is valuable. But maybe you should
> consider tackling the full register, rather than only addressing a
> single field.

Yes, it would be better to tackling the full register. I will put more 
time on other fields in the register and try to making more field to be 
writable. But currently I just respin the series with deleting the 
machine specific information and fixing the compilation issue.

Thanks,
Shaoqin

> 
> Thanks,
> 
> 	M.
> 

-- 
Shaoqin