Re: Questions on setting guest CPU models with ARM kvm enabled

[Andrew Jones <drjones@redhat.com>]

On Fri, Sep 10, 2021 at 10:40:30AM +0800, Jingyi Wang wrote:
> Hi all,
>     On ARM hardware, as qemu docs describes, "Named CPU models generally
> do not work with KVM." May I ask what is the main obstacle to setting a
> guest CPU model on later host hardware?

Currently KVM does not give the user (QEMU) control over ID registers.
You cannot migrate a model using KVM to another host because you cannot
write the model's ID registers to KVM if they do not exactly match the
destination host's ID registers. Changing that behavior is just a matter
of coding, but at least one register, MIDR, will still cause problems
with guests that use it to modify their behavior, e.g. to enable
workarounds for errata. A general solution to that problem is not
really possible. See [1] for where I wrote more about that.

> 
>     For example, if I want to start Cortex-A57 guest on Cortex-A72 host,
> I noticed that there is not much big difference between these two CPU
> models(Please correct me if I'm wrong), what work should I do?

You may be able to find two systems that you deem "close enough" to
attempt to migrate. [1] has some proposals for attempting to support
"close enough" migrations.

Also, Peng Liang was working on attempting to make ID registers user
controlled [2],[3]. However, I think the approach was trying to do too
much too fast. We need to take baby steps towards CPU models. Also,
I heard somebody from Google might be trying to give users control
over ID registers, but I haven't seen patches.

IMO, a first baby step towards CPU models would be changing the
reject list we currently have in KVM for what features we expose
to the guest (see arch/arm64/kvm/sys_regs.c:read_id_reg) to an allow
list. The second step would be to give user control to ID registers,
which would mean each ID register set_user function would need to
learn how to sanity check its register value, which would probably
need to be done in a some vcpu finalize function, since there can be
inter-register dependencies. KVM will also have to give up control
over registers like MIDR, because the fields are implementation
defined, so there's no way to sanity check them.

Once KVM has these changes, then we'd need to build models in QEMU,
advertise the model capabilities to upper layers, etc. in order for
them to be used. But, we'd still leave a big burden on the users to
determine what safe migrations are, due to the errata problem I
mentioned above.

The last sentence of the last paragraph is why we've never really
considered all the work to be worth it. However, I would welcome
some baby steps, particularly the KVM reject-list to allow-list
change, which would ensure safer execution of guests in general,
not just for migration.

Thanks,
drew

[1] https://yhbt.net/lore/all/CABgNM92-PHWg3X41tCVfErmV9-Hnu5GYpz_pa9-iXfbScFUCZg@mail.gmail.com/T/#ma3475a71f6ad34853470c8b38cd4ff43815b79d9
[2] https://www.mail-archive.com/qemu-devel@nongnu.org/msg756323.html
[3] https://lists.cs.columbia.edu/pipermail/kvmarm/2020-November/043011.html