[RFC] Exposing arm64 ID-register safe_rules semantics to userspace?

[RFC] Exposing arm64 ID-register safe_rules semantics to userspace?

[Khushit Shah]

Hi all,

On KVM_SET_ONE_REG for ID registers, KVM validates the requested
field values against the host's values using the per-field rules
in arm64_ftr_bits[](arch/arm64/kernel/cpufeature.c) (and, some
KVM specific overrides on top);

In our QEMU RFC for named ARM CPU models [1], we use the safe_rules to:
- pre-validate a guest ID register configuration before KVM write back,
- report blockers in QMP query-cpu-definitions for a given named
model,
- enumerate supported values per field for QMP/libvirt/upper layers.

RFC has a hand-maintained copy of safe_rules in QEMU that mirrors
the kernel.

Any advice for VMMs on how to track the safe_rules used by KVM to
validate ID-register field values written by userspace?

There is no API that exposes those rules to VMMs. Until a VMM issues
KVM_SET_ONE_REG, it cannot know whether the requested set of values
will be accepted. On failure, VMM still cannot know the faulting field.

We'd like to know what the right approach is:
- Should there be a KVM ioctl that exposes the safe_rules to
userspace?
Maybe Something like KVM_GET_SAFE_RULE (that takes reg_idx and
start_bit_pos)
or a bulk ioctl like KVM_GET_REG_SAFE_RULES (that returns (reg_idx,
start_bit, width, safe_rule, safe_vals) for all the ID registers?
- Is it acceptable to keep mirroring ftr_bits[] in userspace?
- Or is this simply not meant to be exposed to userspace VMMs?

Happy to prototype whichever direction maintainers prefer.

[1] https://lore.kernel.org/qemu-arm/20260605083358.1320563-1-khushit.shah@nutanix.com/T/#m71ce71d5a652edd205e2b5c200cb75292f57c0fc

Warm Regards,
Khushit

Re: [RFC] Exposing arm64 ID-register safe_rules semantics to userspace?

[Marc Zyngier]

On Mon, 08 Jun 2026 10:43:11 +0100,
Khushit Shah <khushit.shah@nutanix.com> wrote:
> 
> Hi all,
> 
> On KVM_SET_ONE_REG for ID registers, KVM validates the requested
> field values against the host's values using the per-field rules
> in arm64_ftr_bits[](arch/arm64/kernel/cpufeature.c) (and, some
> KVM specific overrides on top);
>
> In our QEMU RFC for named ARM CPU models [1], we use the safe_rules to:
> - pre-validate a guest ID register configuration before KVM write back,

Pre-validate *how*?

> - report blockers in QMP query-cpu-definitions for a given named
> model,

Sorry, but I have no idea what this is.

> - enumerate supported values per field for QMP/libvirt/upper layers.
>
> RFC has a hand-maintained copy of safe_rules in QEMU that mirrors
> the kernel.
> 
> Any advice for VMMs on how to track the safe_rules used by KVM to
> validate ID-register field values written by userspace?

Why should the VMM care? Either a field is exposed as writable, and in
general it is safe to expose something "lower" than the host value to
the guest, or it isn't, and only that particular value is possible.

There is a few exceptions to this rule, but I expect this to be the
minority.

> 
> There is no API that exposes those rules to VMMs. Until a VMM issues
> KVM_SET_ONE_REG, it cannot know whether the requested set of values
> will be accepted. On failure, VMM still cannot know the faulting field.
>
> We'd like to know what the right approach is:
> - Should there be a KVM ioctl that exposes the safe_rules to
> userspace?
> Maybe Something like KVM_GET_SAFE_RULE (that takes reg_idx and
> start_bit_pos)
> or a bulk ioctl like KVM_GET_REG_SAFE_RULES (that returns (reg_idx,
> start_bit, width, safe_rule, safe_vals) for all the ID registers?
> - Is it acceptable to keep mirroring ftr_bits[] in userspace?
> - Or is this simply not meant to be exposed to userspace VMMs?

My question above still stands: why isn't knowing that a field is
writable or not, together with the max value of that field, enough to
decide what is possible to be set for that field?

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [RFC] Exposing arm64 ID-register safe_rules semantics to userspace?

[Khushit Shah]

> On 8 Jun 2026, at 4:40 PM, Marc Zyngier <maz@kernel.org> wrote:
> 
> On Mon, 08 Jun 2026 10:43:11 +0100,
> Khushit Shah <khushit.shah@nutanix.com> wrote:
>> 
>> Hi all,
>> 
>> On KVM_SET_ONE_REG for ID registers, KVM validates the requested
>> field values against the host's values using the per-field rules
>> in arm64_ftr_bits[](arch/arm64/kernel/cpufeature.c) (and, some
>> KVM specific overrides on top);
>> 
>> In our QEMU RFC for named ARM CPU models [1], we use the safe_rules to:
>> - pre-validate a guest ID register configuration before KVM write back,
> 
> Pre-validate *how*?
> 
>> - report blockers in QMP query-cpu-definitions for a given named
>> model,
> 
> Sorry, but I have no idea what this is.

Sorry, QEMU jargon. 
We check that the value named model request from KVM is valid and if not,
what is blocking from a model to be realizable, for example a model
may not be realizable on a host because SVE is not available.

> 
>> - enumerate supported values per field for QMP/libvirt/upper layers.
>> 
>> RFC has a hand-maintained copy of safe_rules in QEMU that mirrors
>> the kernel.
>> 
>> Any advice for VMMs on how to track the safe_rules used by KVM to
>> validate ID-register field values written by userspace?
> 
> Why should the VMM care? Either a field is exposed as writable, and in
> general it is safe to expose something "lower" than the host value to
> the guest, or it isn't, and only that particular value is possible.
> 
> There is a few exceptions to this rule, but I expect this to be the
> minority.
> 
>> 
>> There is no API that exposes those rules to VMMs. Until a VMM issues
>> KVM_SET_ONE_REG, it cannot know whether the requested set of values
>> will be accepted. On failure, VMM still cannot know the faulting field.
>> 
>> We'd like to know what the right approach is:
>> - Should there be a KVM ioctl that exposes the safe_rules to
>> userspace?
>> Maybe Something like KVM_GET_SAFE_RULE (that takes reg_idx and
>> start_bit_pos)
>> or a bulk ioctl like KVM_GET_REG_SAFE_RULES (that returns (reg_idx,
>> start_bit, width, safe_rule, safe_vals) for all the ID registers?
>> - Is it acceptable to keep mirroring ftr_bits[] in userspace?
>> - Or is this simply not meant to be exposed to userspace VMMs?
> 
> My question above still stands: why isn't knowing that a field is
> writable or not, together with the max value of that field, enough to
> decide what is possible to be set for that field?

Okay, makes sense, there are 57 fields in cpufeatures.c that are not
unsigned FTR_LOWER_SAFE. 

Right now, only 21 of those are writable. 8 of those are single bit,
with FTR_EXACT and safe_value as 0, so no special case for them.

So, only special handling for 13 fields are needed for now:

EXACT: 4 (TGRAN*_2, L1Ip)
SIGNED_LOWER_SAFE: 7 (TGRAN4, TGRAN64, InnerShr, OuterShr, DoubleLock, PMUVer, PerfMon)
HIGHER_SAFE: 2 (SpecSEI)

Hardcoding these 13 works for today’s kernel. But when new fields are made
writable or are added to ftr_bits, older VMM will by default make assumption
of them as LOWER_SAFE, which might be wrong.

> Thanks,
> 
> M.
> 
> -- 
> Without deviation from the norm, progress is not possible.

Thanks,
Khushit

Re: [RFC] Exposing arm64 ID-register safe_rules semantics to userspace?

[Marc Zyngier]

On Mon, 08 Jun 2026 14:28:13 +0100,
Khushit Shah <khushit.shah@nutanix.com> wrote:
> > On 8 Jun 2026, at 4:40 PM, Marc Zyngier <maz@kernel.org> wrote:
> > 
> > My question above still stands: why isn't knowing that a field is
> > writable or not, together with the max value of that field, enough to
> > decide what is possible to be set for that field?
> 
> Okay, makes sense, there are 57 fields in cpufeatures.c that are not
> unsigned FTR_LOWER_SAFE. 
> 
> Right now, only 21 of those are writable. 8 of those are single bit,
> with FTR_EXACT and safe_value as 0, so no special case for them.
> 
> So, only special handling for 13 fields are needed for now:
> 
> EXACT: 4 (TGRAN*_2, L1Ip)
> SIGNED_LOWER_SAFE: 7 (TGRAN4, TGRAN64, InnerShr, OuterShr, DoubleLock, PMUVer, PerfMon)
> HIGHER_SAFE: 2 (SpecSEI)
> 
> Hardcoding these 13 works for today’s kernel. But when new fields are made
> writable or are added to ftr_bits, older VMM will by default make assumption
> of them as LOWER_SAFE, which might be wrong.

Hardcoding *anything* feels very wrong. Each of these features is
handled differently for a number of reasons that are related to the
architecture itself and the way the kernel is using it *for itself*.
This information is not supposed to be consumed by userspace, and is
not exposed to it.

The problem you seem to have here is that you want to deal with these
features without capturing any understanding of what these actually
mean in the context of the architecture. I can't see a good outcome of
that sort of thing in the long run. The VMM is, for better or worse,
part of the overall hypervisor. It cannot treat the features as a bag
of bits.

This is also why the whole "CPU model" thing is IMO a bad idea: it is
syntactic sugar that falls at the first hurdle, simply because the
architecture is not fully virtualisable. For example you can't take a
Neoverse V2 description and apply it to a KVM guest running on V2 HW.

I think you need to revisit your approach to this stuff.

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [RFC] Exposing arm64 ID-register safe_rules semantics to userspace?

[Khushit Shah]

> On 8 Jun 2026, at 7:38 PM, Marc Zyngier <maz@kernel.org> wrote:
> 
> On Mon, 08 Jun 2026 14:28:13 +0100,
> Khushit Shah <khushit.shah@nutanix.com> wrote:
>>> On 8 Jun 2026, at 4:40 PM, Marc Zyngier <maz@kernel.org> wrote:
>>> 
>>> My question above still stands: why isn't knowing that a field is
>>> writable or not, together with the max value of that field, enough to
>>> decide what is possible to be set for that field?
>> 
>> Okay, makes sense, there are 57 fields in cpufeatures.c that are not
>> unsigned FTR_LOWER_SAFE. 
>> 
>> Right now, only 21 of those are writable. 8 of those are single bit,
>> with FTR_EXACT and safe_value as 0, so no special case for them.
>> 
>> So, only special handling for 13 fields are needed for now:
>> 
>> EXACT: 4 (TGRAN*_2, L1Ip)
>> SIGNED_LOWER_SAFE: 7 (TGRAN4, TGRAN64, InnerShr, OuterShr, DoubleLock, PMUVer, PerfMon)
>> HIGHER_SAFE: 2 (SpecSEI)
>> 
>> Hardcoding these 13 works for today’s kernel. But when new fields are made
>> writable or are added to ftr_bits, older VMM will by default make assumption
>> of them as LOWER_SAFE, which might be wrong.
> 
> Hardcoding *anything* feels very wrong. Each of these features is
> handled differently for a number of reasons that are related to the
> architecture itself and the way the kernel is using it *for itself*.
> This information is not supposed to be consumed by userspace, and is
> not exposed to it.
> 
> The problem you seem to have here is that you want to deal with these
> features without capturing any understanding of what these actually
> mean in the context of the architecture. I can't see a good outcome of
> that sort of thing in the long run. The VMM is, for better or worse,
> part of the overall hypervisor. It cannot treat the features as a bag
> of bits.
> This is also why the whole "CPU model" thing is IMO a bad idea: it is
> syntactic sugar that falls at the first hurdle, simply because the
> architecture is not fully virtualisable. For example you can't take a
> Neoverse V2 description and apply it to a KVM guest running on V2 HW.

Hi Marc,

I take your point that a named model cannot be a faithful Neoverse V2
(or any other) TRM view: even on V2 hardware, a KVM guest only gets
the subset that the architecture and KVM can actually virtualise. We
are not trying to present a TRM-accurate Neoverse V2 to the guest.

What we are after is a migration ABI: a fixed guest-visible ID register
contract that stays identical across all hosts capable of running it.

In my research so far, I have identified the following pitfalls with
this approach:

- Guest errata: errata handling in the guest is driven by MIDR/REVIDR,
  so if those do not match the real host, the guest can pick the wrong
  workarounds or skip needed ones.

  Shameer's errata-management work, where the VMM gives the guest the
  list of target implementations could address this.

- PMU: events and counters are core-specific, so they cannot be
  supported reliably across heterogeneous hosts.

  We currently leave PMU out of named models for that reason.

- IMP-DEF registers: ACTLR_EL1, AMAIR_EL1 and AFSR0/1_EL1 are the
  trickiest in principle. A named model should only be allowed to boot
  on hosts where these registers have the same semantics as on the
  implementation the model's MIDR describes.

  Would it make sense for KVM to make them RAZ/WI instead? AFAIU, they
  are rarely used by guests; mainline Linux does not touch them at all.

That said, on the hardware I currently have access to (Neoverse V2 and
Neoverse V3), those IMP-DEF registers are RES0 (as per TRM). I was able
to migrate a guest between V2 and V3 hosts using a "Neoverse-V2-like”
model (with PAuth disabled), and between two different V2 SoCs with
different feature sets. In both cases, the guest's ID-register view
stayed unchanged across the migration.

This reflects what I have encountered and tested so far. Given your
experience, are there other parts that I am overlooking?

Thanks,
Khushit

> I think you need to revisit your approach to this stuff.
> 
> M.
> 
> -- 
> Without deviation from the norm, progress is not possible.

Re: [RFC] Exposing arm64 ID-register safe_rules semantics to userspace?

[Marc Zyngier]

On Fri, 16 Oct 2026 10:07:37 +0100,
Khushit Shah <khushit.shah@nutanix.com> wrote:
> 
> I take your point that a named model cannot be a faithful Neoverse V2
> (or any other) TRM view: even on V2 hardware, a KVM guest only gets
> the subset that the architecture and KVM can actually virtualise. We
> are not trying to present a TRM-accurate Neoverse V2 to the guest.
> 
> What we are after is a migration ABI: a fixed guest-visible ID register
> contract that stays identical across all hosts capable of running it.

That ABI exists. It is probably buggy as hell, but it is there at
least in principles.

> In my research so far, I have identified the following pitfalls with
> this approach:
> 
> - Guest errata: errata handling in the guest is driven by MIDR/REVIDR,
>   so if those do not match the real host, the guest can pick the wrong
>   workarounds or skip needed ones.
> 
>   Shameer's errata-management work, where the VMM gives the guest the
>   list of target implementations could address this.
> 
> - PMU: events and counters are core-specific, so they cannot be
>   supported reliably across heterogeneous hosts.
> 
>   We currently leave PMU out of named models for that reason.

This is a deal breaker IMO. Most interesting events are *architected*,
and SW that relies on non-architectural events has bigger problems
anyway (i.e. it is never able to migrate correctly). Removing PMU
support from named models on these grounds is yet another nail on that
coffin.

The "better" thing to do is to list the events you want to support and
use the event filter to reject anything else.

> - IMP-DEF registers: ACTLR_EL1, AMAIR_EL1 and AFSR0/1_EL1 are the
>   trickiest in principle. A named model should only be allowed to boot
>   on hosts where these registers have the same semantics as on the
>   implementation the model's MIDR describes.
> 
>   Would it make sense for KVM to make them RAZ/WI instead? AFAIU, they
>   are rarely used by guests; mainline Linux does not touch them at all.

ACTLR_EL1 is treated as WI already, but will report the value the
physical CPU has. We don't know if it is safe to have anything else.
It may have been yet another early misdesign to do so, but we can't
change this behaviour now.

Trapping the other registers to make them RAZ/WI requires trapping all
VM related registers on CPUs that do not have FEAT_FGT (see the
registers trapped by HCR_EL2.TVM to get an idea). The cost of that is
simply not acceptable.

And to be clear: an implementation that associates non-architectural
behaviours to these registers is not virtualisable, and will break
under migration. There is nothing that SW can do about it, because the
hardware is fundamentally misdesigned when it comes to virtualisation.

But we're pretty far from named models anyway.

	M.

-- 
Without deviation from the norm, progress is not possible.