Re: [PATCH RFC] KVM: arm64: PMU: Use multiple host PMUs

[Marc Zyngier <maz@kernel.org>]

On Thu, 20 Mar 2025 09:52:59 +0000,
Akihiko Odaki <akihiko.odaki@daynix.com> wrote:
> 
> On 2025/03/20 18:10, Marc Zyngier wrote:
> > On Thu, 20 Mar 2025 06:03:35 +0000,
> > Akihiko Odaki <akihiko.odaki@daynix.com> wrote:
> >> 
> >> On 2025/03/20 3:51, Oliver Upton wrote:
> >>> On Wed, Mar 19, 2025 at 06:38:38PM +0000, Marc Zyngier wrote:
> >>>> On Wed, 19 Mar 2025 11:51:21 +0000, Akihiko Odaki <akihiko.odaki@daynix.com> wrote:
> >>>>> What about setting the flag automatically when a user fails to pin
> >>>>> vCPUs to CPUs that are covered by one PMU? There would be no change if
> >>>>> a user correctly pins vCPUs as it is. Otherwise, they will see a
> >>>>> correct feature set advertised to the guest and the cycle counter
> >>>>> working.
> >>>> 
> >>>> How do you know that the affinity is "correct"? VCPU affinity can be
> >>>> changed at any time. I, for one, do not want my VMs to change
> >>>> behaviour because I let the vcpus bounce around as the scheduler sees
> >>>> fit.
> >> 
> >> Checking the affinity when picking the default PMU; the vCPU affinity
> >> is the only thing that rules the choice of the default PMU even now.
> >> 
> >> Perhaps we may model the API as follows: introduce another "composite"
> >> PMU that works on any core but only exposes the cycle counter. Robust
> >> VMMs will choose it or one of hardware PMUs with
> >> KVM_ARM_VCPU_PMU_V3_SET_PMU. KVM will choose the default PMU according
> >> to the vCPU affinity at the point of KVM_ARM_VCPU_INIT otherwise. If
> >> the affinity is covered by one hardware PMU, that PMU will be chosen
> >> as the default. The "composite" PMU will be the default otherwise.
> > 
> > This makes no sense to me. A VCPU is always affine to a PMU, because
> > we do not support configurations where only some CPUs have a PMU. This
> > is an all-or-nothing situation.
> 
> At least isn't it fine to have the composite PMU with a new value for
> KVM_ARM_VCPU_PMU_V3_SET_PMU?

Not sure KVM_ARM_VCPU_PMU_V3_SET_PMU is the right hook (it takes a PMU
'type', which is under control of the perf subsystem). But if we can
find a value that is guaranteed to be unique, why not.

> > More importantly, you keep suggesting the same "new default", and I
> > keep saying NO.
> > 
> > My position is clear: if you want a *new* behaviour, you *must* add a
> > new flag that the VMM explicitly provides to enable this CC-only PMU.
> > No change in default behaviour at all.
> > 
> > I'm not going to move from that.
> 
> Why not? It will not break anything guaranteed to work in the past.

It doesn't have to be guaranteed. It just has to *exist*. That's the
Linux ABI for you.

> Currently KVM only guarantees that the emulated PMU correctly counts
> only when
> 1) the vCPU affinity is contained by one PMU and
> 2) it will not expand
> 
> Breaking these conditions will make the behavior of the emulated PMU
> undefined. Now I'm proposing to remove 1).

And I'm saying no to that. I'm also getting tired of arguing the same
point on and on.

We currently have two different behaviours:

- either you explicitly set a PMU, and the affinity of this PMU
  constraints the affinity of the vcpus. The vcpus are not allowed to
  run outside of this affinity. Everything counts all the time.

- or you don't explicitly set a PMU, and a default PMU will be picked
  from the current affinity of the first vcpu. Your vcpus will be able
  to run anywhere, but the virtual PMU will *only* count when the
  vcpus are affine to the default PMU. When the vcpus are not affine
  to default PMU, *nothing* counts.

These two behaviours are ABI. They are not changing. They don't get
relaxed, they don't get tightened, they stay just as they are,
forever.

You want a *third* behaviour, go ahead. Define it the way you want.
But the behaviours described above will stay unchanged.

I'm looking forward to your patches implementing it, but I am also
done arguing on it.

	M.

-- 
Without deviation from the norm, progress is not possible.