Re: [PATCH v8 0/6] Support writable CPU ID registers from userspace

From: Marc Zyngier <maz@kernel.org>
To: Cornelia Huck <cohuck@redhat.com>
Cc: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>,
	Jing Zhang <jingzhangos@google.com>, KVM <kvm@vger.kernel.org>,
	KVMARM <kvmarm@lists.linux.dev>,
	ARMLinux <linux-arm-kernel@lists.infradead.org>,
	Oliver Upton <oupton@google.com>, Will Deacon <will@kernel.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	James Morse <james.morse@arm.com>,
	Alexandru Elisei <alexandru.elisei@arm.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	Fuad Tabba <tabba@google.com>, Reiji Watanabe <reijiw@google.com>,
	Raghavendra Rao Ananta <rananta@google.com>
Subject: Re: [PATCH v8 0/6] Support writable CPU ID registers from userspace
Date: Wed, 17 May 2023 16:53:42 +0100	[thread overview]
Message-ID: <86o7mjkl89.wl-maz@kernel.org> (raw)
In-Reply-To: <877ct7x94e.fsf@redhat.com>

On Wed, 17 May 2023 16:36:49 +0100,
Cornelia Huck <cohuck@redhat.com> wrote:
> 
> On Tue, May 16 2023, Marc Zyngier <maz@kernel.org> wrote:
> 
> > On Tue, 16 May 2023 15:19:00 +0100,
> > Cornelia Huck <cohuck@redhat.com> wrote:
> >> 
> >> On Tue, May 16 2023, Marc Zyngier <maz@kernel.org> wrote:
> >> 
> >> > On Tue, 16 May 2023 12:55:14 +0100,
> >> > Cornelia Huck <cohuck@redhat.com> wrote:
> >> >> 
> >> >> Do you have more concrete ideas for QEMU CPU models already? Asking
> >> >> because I wanted to talk about this at KVM Forum, so collecting what
> >> >> others would like to do seems like a good idea :)
> >> >
> >> > I'm not being asked, but I'll share my thoughts anyway! ;-)
> >> >
> >> > I don't think CPU models are necessarily the most important thing.
> >> > Specially when you look at the diversity of the ecosystem (and even
> >> > the same CPU can be configured in different ways at integration
> >> > time). Case in point, Neoverse N1 which can have its I/D caches made
> >> > coherent or not. And the guest really wants to know which one it is
> >> > (you can only lie in one direction).
> >> >
> >> > But being able to control the feature set exposed to the guest from
> >> > userspace is a huge benefit in terms of migration.
> >> 
> >> Certainly; the important part is that we can keep the guest ABI
> >> stable... which parts match to a "CPU model" in the way other
> >> architectures use it is an interesting question. It almost certainly
> >> will look different from e.g. s390, where we only have to deal with a
> >> single manufacturer.
> >> 
> >> I'm wondering whether we'll end up building frankenmonster CPUs.
> >
> > We already do. KVM hides a bunch of things we don't want the guest to
> > see, either because we don't support the feature, or that we want to
> > present it with a different shape (cache topology, for example), and
> > these combination don't really exist in any physical implementation.
> >
> > Which is why I don't really buy the "CPU model" concept as defined by
> > x86 and s390. We already are in a vastly different place.
> 
> Yes, I agree that the "named cpu models" approach probably won't work on
> Arm (especially if you add other accelerators into the mix -- cpu 'foo'
> with tcg is unlikely to be 100% identical to cpu 'foo' with KVM.) OTOH,
> "these two cpus are not that different from each other, so we support
> migration between them with a least common denominator feature/behaviour
> set" seems more reasonable.

It does, assuming the platform is "compatible". I realise I didn't
mention the small issue of the counter frequency, which cannot be
scaled. ARMv8.6 tried to specify a 1GHz counter frequency, but I still
haven't see a single system having adopted it, and everyone is doing
something different.

We *could* always trap/emulate the counters/timers, but that quickly
becomes ridiculously bad. Isn't ARM virtualisation fun?

> > The way I see it, you get a bunch of architectural features that can
> > be enabled/disabled depending on the underlying HW, hypervisor's
> > capabilities and userspace input. On top of that, there is a layer of
> > paint that tells you what is the overall implementation you could be
> > running on (that's what MIDR+REVIDR+AIDR tell you) so that you can
> > apply some unspeakable, uarch-specific hacks that keep the machine
> > going (got to love these CPU errata).
> >
> >> Another interesting aspect is how KVM ends up influencing what the guest
> >> sees on the CPU level, as in the case where we migrate across matching
> >> CPUs, but with a different software level. I think we want userspace to
> >> control that to some extent, but I'm not sure if this fully matches the
> >> CPU model context.
> >
> > I'm not sure I get the "different software level" part. Do you mean
> > VMM revisions?
> 
> Yes. Basically, two (for migration purposes) identical machines with
> different kernel/QEMU versions, but using the same QEMU compat
> machine. Migrate from old to new, get more regs: works. Migrate from
> new to old, get less regs: boom. Expectation would be for this to
> work, and handling it from machine compat code seems very awkward.

Old to new, fine. I'm not sure how we make the other way work.
Actually, scratch that. I'm pretty sure we can't make it work. You'd
have constraint your guest at creation time so that it doesn't use any
feature that isn't available to older VMM/kernel revisions.

I sense some interesting discussions next month! :D

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel