Linux Perf Users
 help / color / mirror / Atom feed
From: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>
To: Alexandru Elisei <alexandru.elisei@arm.com>
Cc: maz@kernel.org, oupton@kernel.org, fuad.tabba@linux.dev,
	 joey.gouly@arm.com, seiden@linux.ibm.com,
	suzuki.poulose@arm.com,  yuzenghui@huawei.com,
	linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev,
	 will@kernel.org, mark.rutland@arm.com,
	linux-perf-users@vger.kernel.org,  catalin.marinas@arm.com,
	james.clark@linaro.org
Subject: Re: [RFC PATCH v7 00/28] KVM: arm64: Add Statistical Profiling Extension (SPE) support
Date: Mon, 16 Nov 2026 17:51:25 +0000	[thread overview]
Message-ID: <avs9oxfRNGiIi8an@gremlin> (raw)
In-Reply-To: <au29ACRXFEOXaWPV@raptor>

On Fri, Nov 06, 2026 at 11:41:52AM +0000, Alexandru Elisei wrote:
> Hi Lorenzo,
>
> On Thu, Nov 05, 2026 at 05:47:45PM +0000, Lorenzo Stoakes (ARM) wrote:
> > Hi Alex,
> >
> > I've gone through all of the patches now, thank you for your patience with
> > the review :)
>
> Many thanks for the stellar review, will start replying soonish.
>
> >
> > So there are various dependencies/architectural decisions that need to be
> > made here to go from RFC -> non-RFC:
> >
> > 1. The 'Ignore MMU notifiers for guest_memfd-only memslots' patch [0] is a
> >    hard dependency for the series. I chased over there, hopefully we can
> >    get something landed there.
>
> Like you said, Sean is planning to send my original version of that patch with
> an updated commit message, thanks for the ping!

No worries! :)

>
> >
> > 2. non-VHE, pKVM, NV disallowed for SPE - As I mentioned off-list, I spoke
> >    to Marc, and he confirmed that he's more than happy for these to be off
> >    the table for a v1. So I think you can drop the HACK: patches and just
> >    make these conditions of using SPE.
>
> Oliver, some time ago, expressed the opposite preference: VHE-only is OK for an
> RFC, but he was concerned about the discrepancy between VHE and nVHE
> configurations. Maybe the direction for KVM changed in the meantime.
>
> Fine by me to drop the HACK! patches, of course, that's just something to keep
> in mind.

Ack, Oliver - let us know if this is a problem (Marc suggested he was fine with
requiring VHE), I think certainly it's useful most of all to move forwards
without the 'HACK' patches, and with the various architectural points raised
here addressed we can look to a non-RFC series.

And obviously we can update as necessary as we go based on maintainer
requirements.

>
> >
> > 3. The entire VM needs to be pre-faulted, but who should do it? - I do
> >    think that relying on userland to do this is a little iffy. So I wonder
>
> There's plenty of iffiness that userspace already handles when it comes to
> virtual machines.

Yup, does seem to be a fair bit of implicit assumptions of sanity :)

>
> Or, how Oliver put it [1] (regarding migration, but it fits):
>
> 'My take is that there's no amount of idiot-proofing we can do in KVM to prevent
> the VMM from messing this up somehow'.

Haha, right.

I guess fine to require userland to set things up like that, and can always add
an option to have it automatically set up for them in future as a nice-to-have.

I want to separate out the gmem bit though, replying below:

>
> [1] https://lore.kernel.org/kvm/alE3jRLtYeq-WesA@kernel.org/
>
> >    whether it might be possible to, on SPE first run, prefault the ranges
> >    there, and make that a condition of using it? Relatedly, KVM could
> >    reject non-guest_memfd memslots for an SPE VM rather than trusting
> >    userspace there too.
>
> I'm not sure that's the best idea. Mainly two reasons:
>
> 1. KVM will be imposing the condition that *all* writable memslots be backed by
> guest_memfd, and *all* memory backing these memslots be pre-faulted and
> effectively pinned.

I mean this really is a requirement for SPE to work at all, since the guest is
effectively deciding which memory is being used for SPE (I think it's perf AUX
memory right?)

I think given that it's guest-designated _and_ the guest has no idea which parts
of memory are 'SPE safe' and 'SPE unsafe', this is a requirement.

>
> There are use cases for writable memslots to describe guest memory that won't be
> used by SPE. First thing that comes to mind would be paravirtualized interfaces
> between host userspace and the guest. For example, pvtime - kvmtool creates a
> RAM memslot for pvtime (i.e, non-read-only memslot), and it looks to me that
> qemu does the same (but my knowledge of qemu is severely limited, so I might be
> completely wrong).

I don't think any of this is disallowed by the gmem requirement.

>
> Then there's also the possibility that a guest might want to assign a slice of
> RAM for use by SPE, to avoid pre-faulting and pinning the entire VM memory. In
> this case, userspace will want to have *only* the memslot describing that slice
> pre-faulted and pinned, and not the rest of system memory.

As above, there's just no way for userspace to know which bits of memory are SPE
safe and which are not. So I don't think this works.

>
> 2. Perfomance - userspace can pre-fault in parallel, KVM cannot easily do that.
>
> arm64 now has support for KVM_PRE_FAULT_MEMORY, I say let's use that :)

:) right.

OK, so maybe the right path then is to keep the prefaulting with userspace, but
I really do think we have to have KVM reject a writable non-gmem memslot once
SPE is enabled for the VM (and reject enabling SPE if one already exists).

That should be relatively straightforward and avoids at least the most broken
userland mistakes.

>
> >
> > 4. Dirty page logging - patch 27/28 relies on series [1] being landed,
> >    which is also an RFC, and allows migration of SPE VMs. So we need to
> >    decide - do we want this migration? Additionally, I think it's
> >    reasonable to, if we want it, send both [1] and patch 27/28 as a follow
> >    up separate from the v1 non-RFC of the series.
>
> For background, I started this conversation in the LOG_DIRTY_PAGES support for
> guest_memfd series [2]. Oliver had some concerns, but he didn't follow up on my
> explanations, so I decided to send migration support as part of SPE so people
> can have a more complete picture of how everything fits together.
>
> I would be very happy to get basic support for SPE merged in KVM, and, if we
> decide it's useful, migration implemented as a follow up, as that involves
> landing dirty page logging support for guest_memfd.

Ack, thanks, let's treat that as a follow-up then, I think best thing here is
definitely to focus on getting the fundamental bits in for SPE first.

>
> [2] https://lore.kernel.org/kvm/alAFmiAsNf6Xm49_@kernel.org/
>
> >
> > 5. Errata handling - the default already disallows SPE on the affected
> >    silicon, which seems right. I wonder whether kvm-arm.ignore_spe_errata
> >    should exist at all though, other than in some sort of debug mode.
>
> It is intended as a debug/development option, to be used only when the VMs are
> trusted. As to why it's there, it's there because without it, SPE would be
> disabled on all the machines that I have access to, and I suspect systems not
> affected are few and rare between.

Ack ok that clarifies that :) let's then clearly mark that as such
e.g. 'debug/development only, trusted VMs only' in the documentation, and
tainting the kernel if it's enabled, I think, would be sensible.

>
> >
> > So 1 is a known dependency, I think there's no getting around that, and
> > thus that makes merging [0] or something like it a top priority.
> >
> > It seems to me 2 is pretty clear-cut, and 4 can reasonably (if we want it)
> > be sent as a follow up.
> >
> > So really 3 and 5 are the decisions to make here!
>
> I really appreciate your enthusiasm, thanks!

No worries :) am happy to put in some work on my side to help move things
forwards (and obviously more than happy to help with review on respins of
course :)

>
> Alex

--
Cheers, Lorenzo

      reply	other threads:[~2026-11-16 17:51 UTC|newest]

Thread overview: 115+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-03 16:05 [RFC PATCH v7 00/28] KVM: arm64: Add Statistical Profiling Extension (SPE) support Alexandru Elisei
2026-09-03 16:05 ` [RFC PATCH v7 01/28] arm64/sysreg: Add the nVM field to PMBLIMITR_EL1 Alexandru Elisei
2026-09-03 16:14   ` sashiko-bot
2026-10-15 13:23   ` Lorenzo Stoakes (ARM)
2026-09-03 16:05 ` [RFC PATCH v7 02/28] arm64/sysreg: Define MDCR_EL2.E2PB values Alexandru Elisei
2026-09-03 16:12   ` sashiko-bot
2026-10-15 13:52   ` Lorenzo Stoakes (ARM)
2026-11-11 17:14     ` Alexandru Elisei
2026-11-16 18:00       ` Lorenzo Stoakes (ARM)
2026-09-03 16:05 ` [RFC PATCH v7 03/28] KVM: arm64: Add CONFIG_KVM_ARM_SPE Kconfig option Alexandru Elisei
2026-09-03 16:13   ` sashiko-bot
2026-10-15 14:15   ` Lorenzo Stoakes (ARM)
2026-11-11 17:16     ` Alexandru Elisei
2026-11-16 18:00       ` Lorenzo Stoakes (ARM)
2026-09-03 16:05 ` [RFC PATCH v7 04/28] perf: arm_spe_pmu: Move struct arm_spe_pmu to a separate header file Alexandru Elisei
2026-09-03 16:11   ` sashiko-bot
2026-10-15 14:40   ` Lorenzo Stoakes (ARM)
2026-11-11 17:17     ` Alexandru Elisei
2026-11-16 18:03       ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 05/28] perf: arm_spe_pmu: Add PMBIDR_EL1 and PMSIDR_EL1 to struct arm_spe_pmu Alexandru Elisei
2026-09-03 16:11   ` sashiko-bot
2026-10-15 15:12   ` Lorenzo Stoakes (ARM)
2026-11-11 17:20     ` Alexandru Elisei
2026-11-16 17:56       ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 06/28] KVM: arm64: Add KVM_CAP_ARM_SPE capability Alexandru Elisei
2026-09-03 16:15   ` sashiko-bot
2026-10-15 16:51   ` Lorenzo Stoakes (ARM)
2026-11-12 15:34     ` Alexandru Elisei
2026-11-12 16:13       ` Alexandru Elisei
2026-11-16 18:29       ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 07/28] KVM: arm64: Add KVM_ARM_VCPU_SPE VCPU feature Alexandru Elisei
2026-09-03 16:21   ` sashiko-bot
2026-10-16 14:23   ` Lorenzo Stoakes (ARM)
2026-11-12 17:18     ` Alexandru Elisei
2026-11-16 17:59       ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 08/28] HACK! KVM: arm64: Disable SPE virtualization if protected KVM is enabled Alexandru Elisei
2026-09-03 16:21   ` sashiko-bot
2026-10-16 14:30   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 09/28] HACK! KVM: arm64: Enable SPE virtualization only in VHE mode Alexandru Elisei
2026-09-03 16:15   ` sashiko-bot
2026-09-03 16:06 ` [RFC PATCH v7 10/28] HACK! KVM: arm64: Disable SPE virtualization if nested virt is enabled Alexandru Elisei
2026-09-03 16:20   ` sashiko-bot
2026-09-03 16:06 ` [RFC PATCH v7 11/28] KVM: arm64: Add a new VCPU device control group for SPE Alexandru Elisei
2026-09-03 16:22   ` sashiko-bot
2026-10-16 14:44   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 12/28] KVM: arm64: Add SPE VCPU device attribute to set the interrupt number Alexandru Elisei
2026-09-03 16:27   ` sashiko-bot
2026-10-20 17:20   ` Lorenzo Stoakes (ARM)
2026-10-28 16:52   ` Suzuki K Poulose
2026-09-03 16:06 ` [RFC PATCH v7 13/28] KVM: arm64: Add SPE VCPU device attribute to set the SPE device Alexandru Elisei
2026-09-03 16:39   ` sashiko-bot
2026-09-04  9:32     ` Alexandru Elisei
2026-10-28 13:11   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 14/28] KVM: arm64: Add SPE VCPU device attribute to initialize SPE Alexandru Elisei
2026-09-03 16:28   ` sashiko-bot
2026-10-28 13:45   ` Lorenzo Stoakes (ARM)
2026-10-29 18:03   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 15/28] KVM: arm64: Use PMSVer from the assigned SPE instance Alexandru Elisei
2026-09-03 16:41   ` sashiko-bot
2026-09-04 10:26     ` Alexandru Elisei
2026-10-28 16:41   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 16/28] KVM: arm64: Add SPE system registers to VCPU context Alexandru Elisei
2026-09-03 16:32   ` sashiko-bot
2026-09-04 10:28     ` Alexandru Elisei
2026-10-29 15:42   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 17/28] KVM: arm64: Apply a RES0 mask to PMBLIMITR_EL1 writes Alexandru Elisei
2026-09-03 16:37   ` sashiko-bot
2026-09-04 10:41     ` Alexandru Elisei
2026-10-29 17:23   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 18/28] KVM: arm64: config: Use functions from spe.c to test FEAT_SPE_{FnE,FDS} Alexandru Elisei
2026-09-03 16:40   ` sashiko-bot
2026-10-29 17:47   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 19/28] KVM: arm64: VHE: Context switch SPE state Alexandru Elisei
2026-09-03 16:43   ` sashiko-bot
2026-09-04 11:35     ` Alexandru Elisei
2026-11-03 14:10       ` Lorenzo Stoakes (ARM)
2026-11-03 16:00   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 20/28] KVM: arm64: Allow guest SPE physical timestamps only if kernel allows it Alexandru Elisei
2026-09-03 16:48   ` sashiko-bot
2026-09-04 13:45     ` Alexandru Elisei
2026-11-03 17:04   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 21/28] KVM: arm64: Handle SPE maintenance interrupts Alexandru Elisei
2026-09-03 16:58   ` sashiko-bot
2026-09-04 14:04     ` Alexandru Elisei
2026-11-05 16:05   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 22/28] arm64: errata: Disable SPE in KVM Alexandru Elisei
2026-09-03 16:50   ` sashiko-bot
2026-11-03 20:12   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 23/28] KVM: arm64: Add kvm-arm.ignore_spe_errata kernel parameter Alexandru Elisei
2026-09-03 16:46   ` sashiko-bot
2026-11-03 19:37   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 24/28] arm64: errata: Don't enable guest buffer if misprogrammed Alexandru Elisei
2026-09-03 17:00   ` sashiko-bot
2026-11-05 16:51   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 25/28] KVM: arm64: at: Use callback for reading descriptor Alexandru Elisei
2026-09-03 16:51   ` sashiko-bot
2026-11-03 19:55   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 26/28] KVM: arm64: Map memory on a SPE stage 2 fault Alexandru Elisei
2026-09-03 17:08   ` sashiko-bot
2026-11-05 17:10   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 27/28] KVM: arm64: Handle dirty page logging when SPE feature is set Alexandru Elisei
2026-09-03 17:06   ` sashiko-bot
2026-09-04 14:41     ` Alexandru Elisei
2026-11-05 17:35   ` Lorenzo Stoakes (ARM)
2026-09-03 16:06 ` [RFC PATCH v7 28/28] KVM: arm64: Allow the creation of a SPE enabled VM Alexandru Elisei
2026-09-03 16:59   ` sashiko-bot
2026-09-04 14:09     ` Alexandru Elisei
2026-11-03 19:49   ` Lorenzo Stoakes (ARM)
2026-10-15 10:25 ` [RFC PATCH v7 00/28] KVM: arm64: Add Statistical Profiling Extension (SPE) support Alexandru Elisei
2026-10-15 12:50   ` Lorenzo Stoakes (ARM)
2026-10-15 13:22     ` Alexandru Elisei
2026-11-05 17:47 ` Lorenzo Stoakes (ARM)
2026-11-05 19:28   ` Lorenzo Stoakes (ARM)
2026-11-06 11:41   ` Alexandru Elisei
2026-11-16 17:51     ` Lorenzo Stoakes (ARM) [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=avs9oxfRNGiIi8an@gremlin \
    --to=ljs@kernel.org \
    --cc=alexandru.elisei@arm.com \
    --cc=catalin.marinas@arm.com \
    --cc=fuad.tabba@linux.dev \
    --cc=james.clark@linaro.org \
    --cc=joey.gouly@arm.com \
    --cc=kvmarm@lists.linux.dev \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=maz@kernel.org \
    --cc=oupton@kernel.org \
    --cc=seiden@linux.ibm.com \
    --cc=suzuki.poulose@arm.com \
    --cc=will@kernel.org \
    --cc=yuzenghui@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox