From: Oliver Upton <oliver.upton@linux.dev>
To: Marc Zyngier <maz@kernel.org>
Cc: Reiji Watanabe <reijiw@google.com>,
kvmarm@lists.linux.dev, kvm@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
James Morse <james.morse@arm.com>,
Alexandru Elisei <alexandru.elisei@arm.com>,
Zenghui Yu <yuzenghui@huawei.com>,
Suzuki K Poulose <suzuki.poulose@arm.com>,
Paolo Bonzini <pbonzini@redhat.com>,
Ricardo Koller <ricarkol@google.com>,
Jing Zhang <jingzhangos@google.com>,
Raghavendra Rao Anata <rananta@google.com>
Subject: Re: [PATCH v2 3/8] KVM: arm64: PMU: Preserve vCPU's PMCR_EL0.N value on vCPU reset
Date: Fri, 20 Jan 2023 18:04:55 +0000 [thread overview]
Message-ID: <Y8rXx+7EUob7qPXh@google.com> (raw)
In-Reply-To: <86pmb9mmkv.wl-maz@kernel.org>
Hey Marc,
On Fri, Jan 20, 2023 at 12:12:32PM +0000, Marc Zyngier wrote:
> On Fri, 20 Jan 2023 00:30:33 +0000, Oliver Upton <oliver.upton@linux.dev> wrote:
> > I think we need to derive a sanitised value for PMCR_EL0.N, as I believe
> > nothing in the architecture prevents implementers from gluing together
> > cores with varying numbers of PMCs. We probably haven't noticed it yet
> > since it would appear all Arm designs have had 6 PMCs.
>
> This brings back the question of late onlining. How do you cope with
> with the onlining of such a CPU that has a smaller set of counters
> than its online counterparts? This is at odds with the way the PMU
> code works.
You're absolutely right, any illusion we derived from the online set of
CPUs could fall apart with a late onlining of a different core.
> If you have a different set of counters, you are likely to have a
> different PMU altogether:
>
> [ 1.192606] hw perfevents: enabled with armv8_cortex_a57 PMU driver, 7 counters available
> [ 1.201254] hw perfevents: enabled with armv8_cortex_a53 PMU driver, 7 counters available
>
> This isn't a broken system, but it has two set of cores which are
> massively different, and two PMUs.
>
> This really should tie back to the PMU type we're counting on, and to
> the set of CPUs that implements it. We already have some
> infrastructure to check for the affinity of the PMU vs the CPU we're
> running on, and this is already visible to userspace.
>
> Can't we just leave this responsibility to userspace?
Believe me, I'm always a fan of offloading things to userspace :)
If the VMM is privy to the details of the system it is on then the
differing PMUs can be passed through to the guest w/ pinned vCPU
threads. I just worry about the case of a naive VMM that assumes a
homogenous system. I don't think I could entirely blame the VMM in this
case either as we've gone to lengths to sanitise the feature set
exposed to userspace.
What happens when a vCPU gets scheduled on a core where the vPMU
doesn't match? Ignoring other incongruences, it is not possible to
virtualize more counters than are supported by the vPMU of the core.
Stopping short of any major hacks in the kernel to fudge around the
problem, I believe we may need to provide better documentation of how
heterogeneous CPUs are handled in KVM and what userspace can do about
it.
--
Thanks,
Oliver
next prev parent reply other threads:[~2023-01-20 18:05 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-17 1:35 [PATCH v2 0/8] KVM: arm64: PMU: Allow userspace to limit the number of PMCs on vCPU Reiji Watanabe
2023-01-17 1:35 ` [PATCH v2 1/8] KVM: arm64: PMU: Have reset_pmu_reg() to clear a register Reiji Watanabe
2023-01-20 14:04 ` Marc Zyngier
2023-01-20 14:11 ` Marc Zyngier
2023-01-21 5:18 ` Reiji Watanabe
2023-01-17 1:35 ` [PATCH v2 2/8] KVM: arm64: PMU: Use reset_pmu_reg() for PMUSERENR_EL0 and PMCCFILTR_EL0 Reiji Watanabe
2023-01-17 1:35 ` [PATCH v2 3/8] KVM: arm64: PMU: Preserve vCPU's PMCR_EL0.N value on vCPU reset Reiji Watanabe
2023-01-20 0:30 ` Oliver Upton
2023-01-20 12:12 ` Marc Zyngier
2023-01-20 18:04 ` Oliver Upton [this message]
2023-01-20 18:53 ` Reiji Watanabe
2023-01-17 1:35 ` [PATCH v2 4/8] KVM: arm64: PMU: Disallow userspace to set PMCR.N greater than the host value Reiji Watanabe
2023-01-20 14:18 ` Marc Zyngier
2023-01-17 1:35 ` [PATCH v2 5/8] tools: arm64: Import perf_event.h Reiji Watanabe
2023-01-17 1:35 ` [PATCH v2 6/8] KVM: selftests: aarch64: Introduce vpmu_counter_access test Reiji Watanabe
2023-01-17 1:35 ` [PATCH v2 7/8] KVM: selftests: aarch64: vPMU register test for implemented counters Reiji Watanabe
2023-01-18 7:47 ` Shaoqin Huang
2023-01-19 3:02 ` Reiji Watanabe
2023-01-17 1:35 ` [PATCH v2 8/8] KVM: selftests: aarch64: vPMU register test for unimplemented counters Reiji Watanabe
2023-01-18 7:49 ` Shaoqin Huang
2023-01-19 3:04 ` Reiji Watanabe
2023-01-17 7:25 ` [PATCH v2 0/8] KVM: arm64: PMU: Allow userspace to limit the number of PMCs on vCPU Shaoqin Huang
2023-01-18 5:53 ` Reiji Watanabe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Y8rXx+7EUob7qPXh@google.com \
--to=oliver.upton@linux.dev \
--cc=alexandru.elisei@arm.com \
--cc=james.morse@arm.com \
--cc=jingzhangos@google.com \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.linux.dev \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=maz@kernel.org \
--cc=pbonzini@redhat.com \
--cc=rananta@google.com \
--cc=reijiw@google.com \
--cc=ricarkol@google.com \
--cc=suzuki.poulose@arm.com \
--cc=yuzenghui@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox