Re: [RFC 18/21] arm/cpu: Introduce a customizable kvm host cpu model

[Cornelia Huck <cohuck@redhat.com>]

On Tue, Nov 12 2024, Eric Auger <eric.auger@redhat.com> wrote:
> I am scared about the overall implementation/test matrix. Can we afford
> supporting that many setups, revision based models, named models with
> all kinds of features optim-in/opt-out. We should also keep in mind what
> is our current goal, ie. supporting migration between different machines
> but still with sufficient commonalities.
>
> Shouldn't we try to collect community needs in that regard. I don't know
> if people will/can tell us what they want to migrate from/to. At least
> do the machines comply with the same spec rev or how large is the gap.
> How many ID regs/features do mismatch in general. Are those ID regs
> currently writable? What would be the best grouping model for them?
>
> Also many ID regs are not writable, because we have not turned them yet
> as writable or because they are not that much a problem at the moment.
> Maybe some features are not fully turned as writable. I mean some ID
> regs they depend on are but not all of them.
>
> So maybe we should reduce the complexity by reducing the scope to the 16
> writable regs we have atm (~ 126 writable ID reg fields).

Reducing scope is definetely needed here, I agree. Probably best to put
the design of named models on the backburner for now, until we've
figured out what we actually need/want to support.

In order to figure out how difficult/problematic mapping ID register
contents to features would be, I've been looking at some real world
examples, i.e. looking at register dumps on some servers that should be
reasonably close in capabilities and where we would like to be able to
migrate between them. "Reasonably close" is mostly "not more than two
handfuls of differences"; sometimes derived from a common base
(e.g. Neoverse-V2.)

The good news is that in many cases we only have differences in bits
that map to a feature (and are actually writable in current KVM.) The
bad news is that we have a number of exceptions.

Comparison #1:

ID_AA64DFR0
f010307009	#of breakpoints:7
f010305009	#of breakpoints:5

BRPs does not match to any feature (but has a different meaning when we
have FEAT_Debugv8p9 and 16+ breakpoints)
[this is a whole can of worms in general]

ID_AA64MMFR0
2100022200101026	FEAT_ECV, FEAT_FGT, 4PB
0000022200101125	mixed endian, 256TB

FEAT_ECV -> may be 1 or 2 in ECV, with different capabilities (I guess
we would need to allow something like FEAT_ECV=2 to expess this?)
support for mixed endian -> indicated in BigEnd, no feature (how
relevant is this in practice?)
PARange (52 bits/4PB vs 48 bits/256TB) -> no feature, but some values
depend on other features -- we care about this when creating a cpu, but
migrating to another system with a mismatched range would be
problematic, unless configuration outside of the cpu model would take
care of it

Comparison #2:

ID_AA64PFR0
1101011021111111	FEAT_AMUv1, GIC v3.0/4.0
1101001020111111

GIC == 1 indicates GIC CPU sysreg interface for 3.0/4.0, but no feature
(I'm not quite sure how we handle this in QEMU)

ID_AA64MMFR1
1000000010312122	FEAT_ECBHB, !FEAT_ETS2, FEAT_PAN3, FEAT_HPDS2, FEAT_HAFDBS
0000001010211120	!FEAT_ETS2, FEAT_PAN2, FEAT_HPDS

both ETS == 0 and ETS == 1 indicate that FEAT_ETS2 is not implememented
(ETS == 2 would indicate FEAT_ETS2) -- I guess we would want to
standardize on ETS == 0
FEAT_PAN3 implies FEAT_PAN2, and FEAT_HPDS2 implies FEAT_HPDS2, so
probably fine

ID_AA64MMFR2
1221011112001011	FEAT_BBM=2
1211011112011011	FEAT_BBM=1, FEAT_LVA

FEAT_BBM can have different values -- so not a neat boolean

So in summary, we have
(a) a few features that look a bit funky, but should be managable,
    especially if we accept non-boolean values
(b) things that are indicated in ID registers but do not map to any
    feature like mixed endian support
(c) things in (b) that we already know to be problematic, like the
    number of breakpoints
(d) things in (b) that actually interact with other configurations we
    might do via the commandline (PARange, probably GIC?)

and that's just from a sample of existing systems I was able to get data
from; I think we don't expect that there aren't any more of these in the
remainder of the architecture. (I'm excluding not-yet writable bits, as
I assume we'll eventually render everything writable that is feasible
and needed, although I suspect that there are problems lurking in there
as well.)

[I completely ignored aarch32 registers.]

My current plan is to push off features vs id regs into the future and
do a new version of this patch set that keeps id regs, but drops the
'custom' model, and that can be used to try migration between some hand
picked systems (to find out what other problems we will need to deal
with.)