* Re: [PATCH 2/6] x86/sev: add support for enabling RMPOPT
From: Dave Hansen @ 2026-02-18 22:56 UTC (permalink / raw)
To: Kalra, Ashish, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <bab6afd4-b197-4a7b-b9d1-f518d192524f@amd.com>
On 2/18/26 14:17, Kalra, Ashish wrote:
> Yes, by default Venice platform has the NPS2 configuration enabled by default,
> so we have 'X' nodes per socket and we have to consider this NPSx configuration
> and optimize for those groups.
Why, though?
You keep saying: "We have NPS so we must configure sockets". But not *why*.
I suspect this is another premature optimization. Nodes are a bit too
small so if you configure via nodes, the later nodes will have RMPOPT
tables that cover empty address space off the end of system memory.
Honestly, I think this is all just done wrong. It doesn't need to even
consider sockets. Sockets might even be the wrong thing to look at.
Basically, RMPOPT gives you a 2TB window of potentially "fast" memory.
The rest of memory is "slow". If you're lucky, the memory that's fast
because of RMPOPT is also in a low-distance NUMA node.
Sockets are a good thing to use, for sure. But they're not even optimal!
Just imagine what's going to happen if you have more than 2TB in a
socket. You just turn off the per-socket optimization. If that happens,
the last node in the socket will end up with an RMPOPT table that has
itself at the beginning, but probably a nonzero amount of off-socket memory.
I'd probably just do something like this:
Given a NUMA node, go through each 1GB of memory in the system and see
what the average NUMA distance of that 2TB window of memory is. Find the
2TB window with the lowest average distance. That'll give you a more or
less optimal RMPOPT window. It'll work with NPS or regular NUMA or
whatever bonkers future fancy thing shows up.
But that's all optimization territory. Please squirrel that away to go
look at in 6 months once you get the rest of this merged.
^ permalink raw reply
* [Invitation] bi-weekly guest_memfd upstream call on 2026-02-19
From: David Hildenbrand (Arm) @ 2026-02-18 22:46 UTC (permalink / raw)
To: linux-coco@lists.linux.dev, linux-mm@kvack.org, KVM
Hi,
Our next guest_memfd upstream call is scheduled for Thursday,
2026-02-19 at 8:00 - 9:00am (GMT-08:00) Pacific Time - Vancouver.
We'll be using the following Google meet:
http://meet.google.com/wxp-wtju-jzw
The meeting notes can be found at [1], where we also link recordings and
collect current guest_memfd upstream proposals. If you want an google
calendar invitation that also covers all future meetings, just write me
or Ackerley a mail.
In this meeting, we'll discuss how to best handle private MMIO, and I'm
sure some other topics will pop up :)
To put something to discuss onto the agenda, reply to this mail or add
them to the "Topics/questions for next meeting(s)" section in the
meeting notes as a comment.
[1]
https://docs.google.com/document/d/1M6766BzdY1Lhk7LiR5IqVR8B8mG3cr-cxTxOrAosPOk/edit?usp=sharing
[2] https://lore.kernel.org/all/20251203144159.6131-1-itazur@amazon.com/
--
Cheers,
David
^ permalink raw reply
* Re: [PATCH 2/6] x86/sev: add support for enabling RMPOPT
From: Kalra, Ashish @ 2026-02-18 22:17 UTC (permalink / raw)
To: Dave Hansen, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <10baddd3-add6-4771-a1ce-f759d3ec69d2@intel.com>
Hello Dave,
On 2/17/2026 4:06 PM, Dave Hansen wrote:
>> +#define RMPOPT_TABLE_MAX_LIMIT_IN_TB 2
>> +#define NUM_TB(pfn_min, pfn_max) \
>> + (((pfn_max) - (pfn_min)) / (1 << (40 - PAGE_SHIFT)))
>
> IMNHO, you should just keep these in bytes. No reason to keep them in TB.
>
>> +struct rmpopt_socket_config {
>> + unsigned long start_pfn, end_pfn;
>> + cpumask_var_t cpulist;
>> + int *node_id;
>> + int current_node_idx;
>> +};
>
> This looks like optimization complexity before the groundwork is in
> place. Also, don't we *have* CPU lists for NUMA nodes? This seems rather
> redundant.
>
Yes, we do have CPU lists for NUMA nodes, but we need a socket specific
cpumask, let me explain more about that below.
>> +/*
>> + * Build a cpumask of online primary threads, accounting for primary threads
>> + * that have been offlined while their secondary threads are still online.
>> + */
>> +static void get_cpumask_of_primary_threads(cpumask_var_t cpulist)
>> +{
>> + cpumask_t cpus;
>> + int cpu;
>> +
>> + cpumask_copy(&cpus, cpu_online_mask);
>> + for_each_cpu(cpu, &cpus) {
>> + cpumask_set_cpu(cpu, cpulist);
>> + cpumask_andnot(&cpus, &cpus, cpu_smt_mask(cpu));
>> + }
>> +}
>
> Don't we have a primary thread mask already? I thought we did.
>
Already discussed this.
>> +static void __configure_rmpopt(void *val)
>> +{
>> + u64 rmpopt_base = ((u64)val & PUD_MASK) | MSR_AMD64_RMPOPT_ENABLE;
>> +
>> + wrmsrq(MSR_AMD64_RMPOPT_BASE, rmpopt_base);
>> +}
>
> I'd honestly just make the callers align the address..
>
>> +static void configure_rmpopt_non_numa(cpumask_var_t primary_threads_cpulist)
>> +{
>> + on_each_cpu_mask(primary_threads_cpulist, __configure_rmpopt, (void *)0, true);
>> +}
>> +
>> +static void free_rmpopt_socket_config(struct rmpopt_socket_config *socket)
>> +{
>> + int i;
>> +
>> + if (!socket)
>> + return;
>> +
>> + for (i = 0; i < topology_max_packages(); i++) {
>> + free_cpumask_var(socket[i].cpulist);
>> + kfree(socket[i].node_id);
>> + }
>> +
>> + kfree(socket);
>> +}
>> +DEFINE_FREE(free_rmpopt_socket_config, struct rmpopt_socket_config *, free_rmpopt_socket_config(_T))
>
> Looking at all this, I really think you need a more organized series.
>
> Make something that's _functional_ and works for all <2TB configs. Then,
> go add all this NUMA complexity in a follow-on patch or patches. There's
> too much going on here.
Sure.
>
>> +static void configure_rmpopt_large_physmem(cpumask_var_t primary_threads_cpulist)
>> +{
>> + struct rmpopt_socket_config *socket __free(free_rmpopt_socket_config) = NULL;
>> + int max_packages = topology_max_packages();
>> + struct rmpopt_socket_config *sc;
>> + int cpu, i;
>> +
>> + socket = kcalloc(max_packages, sizeof(struct rmpopt_socket_config), GFP_KERNEL);
>> + if (!socket)
>> + return;
>> +
>> + for (i = 0; i < max_packages; i++) {
>> + sc = &socket[i];
>> + if (!zalloc_cpumask_var(&sc->cpulist, GFP_KERNEL))
>> + return;
>> + sc->node_id = kcalloc(nr_node_ids, sizeof(int), GFP_KERNEL);
>> + if (!sc->node_id)
>> + return;
>> + sc->current_node_idx = -1;
>> + }
>> +
>> + /*
>> + * Handle case of virtualized NUMA software domains, such as AMD Nodes Per Socket(NPS)
>> + * configurations. The kernel does not have an abstraction for physical sockets,
>> + * therefore, enumerate the physical sockets and Nodes Per Socket(NPS) information by
>> + * walking the online CPU list.
>> + */
>
> By this point, I've forgotten why sockets are important here.
>
> Why are they important?
Because, Nodes per Socket (NPS) configuration is enabled by default, therefore, we have to
look at Sockets instead of simply NUMA nodes, and collect/aggregate all the Node data per Socket
and then accordingly setup the RMPOPT tables, so that the 2TB limit of RMPOPT tables is covered
appropriately and we try to map the maximum possible memory in RMPOPT tables per-Socket rather
than per-Node.
And as there is no per-Socket information available in kernel, we walk through the online
CPU list and collect all this per-Socket information (including socket's start, end addresses,
NUMA nodes in the socket, cpumask of the socket, etc.)
>
>> + for_each_cpu(cpu, primary_threads_cpulist) {
>> + int socket_id, nid;
>> +
>> + socket_id = topology_logical_package_id(cpu);
>> + nid = cpu_to_node(cpu);
>> + sc = &socket[socket_id];
>> +
>> + /*
>> + * For each socket, determine the corresponding nodes and the socket's start
>> + * and end PFNs.
>> + * Record the node and the start and end PFNs of the first node found on the
>> + * socket, then record each subsequent node and update the end PFN for that
>> + * socket as additional nodes are found.
>> + */
>> + if (sc->current_node_idx == -1) {
>> + sc->current_node_idx = 0;
>> + sc->node_id[sc->current_node_idx] = nid;
>> + sc->start_pfn = node_start_pfn(nid);
>> + sc->end_pfn = node_end_pfn(nid);
>> + } else if (sc->node_id[sc->current_node_idx] != nid) {
>> + sc->current_node_idx++;
>> + sc->node_id[sc->current_node_idx] = nid;
>> + sc->end_pfn = node_end_pfn(nid);
>> + }
>> +
>> + cpumask_set_cpu(cpu, sc->cpulist);
>> + }
>> +
>> + /*
>> + * If the "physical" socket has up to 2TB of memory, the per-CPU RMPOPT tables are
>> + * configured to the starting physical address of the socket, otherwise the tables
>> + * are configured per-node.
>> + */
>> + for (i = 0; i < max_packages; i++) {
>> + int num_tb_socket;
>> + phys_addr_t pa;
>> + int j;
>> +
>> + sc = &socket[i];
>> + num_tb_socket = NUM_TB(sc->start_pfn, sc->end_pfn) + 1;
>> +
>> + pr_debug("socket start_pfn 0x%lx, end_pfn 0x%lx, socket cpu mask %*pbl\n",
>> + sc->start_pfn, sc->end_pfn, cpumask_pr_args(sc->cpulist));
>> +
>> + if (num_tb_socket <= RMPOPT_TABLE_MAX_LIMIT_IN_TB) {
>> + pa = PFN_PHYS(sc->start_pfn);
>> + on_each_cpu_mask(sc->cpulist, __configure_rmpopt, (void *)pa, true);
>> + continue;
>> + }
>> +
>> + for (j = 0; j <= sc->current_node_idx; j++) {
>> + int nid = sc->node_id[j];
>> + struct cpumask node_mask;
>> +
>> + cpumask_and(&node_mask, cpumask_of_node(nid), sc->cpulist);
>> + pa = PFN_PHYS(node_start_pfn(nid));
>> +
>> + pr_debug("RMPOPT_BASE MSR on nodeid %d cpu mask %*pbl set to 0x%llx\n",
>> + nid, cpumask_pr_args(&node_mask), pa);
>> + on_each_cpu_mask(&node_mask, __configure_rmpopt, (void *)pa, true);
>> + }
>> + }
>> +}
>
> Ahh, so you're not optimizing by NUMA itself: you're assuming that there
> are groups of NUMA nodes in a socket and then optimizing for those groups.
>
Yes, by default Venice platform has the NPS2 configuration enabled by default,
so we have 'X' nodes per socket and we have to consider this NPSx configuration
and optimize for those groups.
> It would have been nice to say that. It would make great material for
> the changelog for your broken out patches.
Ok.
>
> I have the feeling that the structure here could be one of these in a patch:
>
> 1. Support systems with <2TB of memory
> 2. Support a RMPOPT range per NUMA node
> 3. Group NUMA nodes at socket boundaries and have them share a common
> RMPOPT config.
>
> Right?
Yes, sure.
>
>> +static __init void configure_and_enable_rmpopt(void)
>> +{
>> + cpumask_var_t primary_threads_cpulist;
>> + int num_tb;
>> +
>> + if (!cpu_feature_enabled(X86_FEATURE_RMPOPT)) {
>> + pr_debug("RMPOPT not supported on this platform\n");
>> + return;
>> + }
>> +
>> + if (!cc_platform_has(CC_ATTR_HOST_SEV_SNP)) {
>> + pr_debug("RMPOPT optimizations not enabled as SNP support is not enabled\n");
>> + return;
>> + }
>> +
>> + if (!(rmp_cfg & MSR_AMD64_SEG_RMP_ENABLED)) {
>> + pr_info("RMPOPT optimizations not enabled, segmented RMP required\n");
>> + return;
>> + }
>> +
>> + if (!zalloc_cpumask_var(&primary_threads_cpulist, GFP_KERNEL))
>> + return;
>> +
>> + num_tb = NUM_TB(min_low_pfn, max_pfn) + 1;
>> + pr_debug("NUM_TB pages in system %d\n", num_tb);
>
> This looks wrong. Earlier, you program 0 as the base RMPOPT address into
> the MSR. But this uses 'min_low_pfn'. Why not 0?
You are right, we should have used min_low_pfn earlier to program the
base RMPOPT address into the MSR.
>
>> + /* Only one thread per core needs to set RMPOPT_BASE MSR as it is per-core */
>> + get_cpumask_of_primary_threads(primary_threads_cpulist);
>> +
>> + /*
>> + * Per-CPU RMPOPT tables support at most 2 TB of addressable memory for RMP optimizations.
>> + *
>> + * Fastpath RMPOPT configuration and setup:
>> + * For systems with <= 2 TB of RAM, configure each per-core RMPOPT base to 0,
>> + * ensuring all system RAM is RMP-optimized on all CPUs.
>> + */
>> + if (num_tb <= RMPOPT_TABLE_MAX_LIMIT_IN_TB)
>> + configure_rmpopt_non_numa(primary_threads_cpulist);
>
> this part:
>
>> + else
>> + configure_rmpopt_large_physmem(primary_threads_cpulist);
>
> ^^ needs to be broken out into a separate optimization patch.
>
Ok.
Thanks,
Ashish
^ permalink raw reply
* Re: [PATCH 0/6] Add RMPOPT support.
From: Kalra, Ashish @ 2026-02-18 21:09 UTC (permalink / raw)
To: Dave Hansen, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <2398df04-082d-4d98-beca-f85de385941a@intel.com>
Hello Dave,
On 2/18/2026 11:15 AM, Dave Hansen wrote:
> On 2/18/26 09:03, Kalra, Ashish wrote:
>>> They are known not to contain any SEV-SNP guest memory at the
>>> moment snp_rmptable_init() finishes, no?
>> Yes, but RMP checks are still performed and they affect performance.
>>
>> Testing a bit in the per‑CPU RMPOPT table to avoid RMP checks
>> significantly improves performance.
>
> Sorry, Ashish, I don't think I'm explaining myself very well. Let me try
> again, please.
>
> First, my goal here is to ensure that the system has a whole has good
> performance, with minimal kernel code, and in the most common
> configurations.
>
> I would wager that the most common SEV-SNP configuration in the whole
> world is a system that has booted, enabled SEV-SNP, and has never run an
> SEV-SNP guest. If it's not *the* most common, it's certainly going to be
> common enough to care about deeply.
>
> Do you agree?
Yes.
>
> If you agree, I hope we can also agree that a "SNP enabled but never ran
> a guest" state is deserving of good performance with minimal kernel code.
>
> My assumption (which is maybe a bad one) is that there is a natural
> point when SEV-SNP is enabled on the system when the system as a whole
> can easily assert that no SEV-SNP guest has ever run. I'm assuming that
> there is *a* point where, for instance, the RMP table gets atomically
> flipped from being unprotected to being protected. At that point, its
> state *must* be known. It must also be naturally obvious that no guest
> has had a chance to run at this point.
>
> If that point can be leveraged, and the RMPOPT optimization can be
> applied at SEV-SNP enabled time, then an important SEV-SNP configuration
> would be optimized by default and with zero or little kernel code needed
> to drive it.
>
> To me, that seems like a valuable goal.
>
> Do you agree?
Now, RMP gets protected at the *same* point where SNP is enabled and then
RMP checking is started. And this is the same point at which RMPOPT
optimizations are enabled with this patch.
I believe you are talking about the hardware doing it as part of SNP enablement,
but that isn't how it is implemented and the reasons for that are it would take
a long time (in CPU terms) for a single WRMSR, and we don't support that.
And if RMP has been allocated means that you are going to be running SNP guests,
otherwise you wouldn't have allocated the RMP and enabled SNP in BIOS.
The RMPOPT feature address the RMP checks associated with non-SNP guests and the
hypervisor itself, theoretically, a cloud provider has good memory placement for
guests and can benefit even when launching/running SNP guests.
We can simplify this initial series to just using this RMPOPT feature and enabling
RMP optimizations for 0 to 2TB across the system and then do the optimizations
for/or supporting larger systems as a follow on series.
That will address your concerns of performing the RMPOPT optimizations at
SEV-SNP enabled time, and having the important SEV-SNP configuration
optimized by default and with little kernel code needed to drive it.
Thanks,
Ashish
^ permalink raw reply
* Re: [PATCH v3 00/16] x86/msr: Inline rdmsr/wrmsr instructions
From: H. Peter Anvin @ 2026-02-18 20:37 UTC (permalink / raw)
To: Juergen Gross, linux-kernel, x86, linux-coco, kvm, linux-hyperv,
virtualization, llvm
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
Kiryl Shutsemau, Rick Edgecombe, Sean Christopherson,
Paolo Bonzini, K. Y. Srinivasan, Haiyang Zhang, Wei Liu,
Dexuan Cui, Long Li, Vitaly Kuznetsov, Boris Ostrovsky, xen-devel,
Ajay Kaher, Alexey Makhalov, Broadcom internal kernel review list,
Andy Lutomirski, Peter Zijlstra, Xin Li, Nathan Chancellor,
Nick Desaulniers, Bill Wendling, Justin Stitt, Josh Poimboeuf,
andy.cooper
In-Reply-To: <20260218082133.400602-1-jgross@suse.com>
On February 18, 2026 12:21:17 AM PST, Juergen Gross <jgross@suse.com> wrote:
>When building a kernel with CONFIG_PARAVIRT_XXL the paravirt
>infrastructure will always use functions for reading or writing MSRs,
>even when running on bare metal.
>
>Switch to inline RDMSR/WRMSR instructions in this case, reducing the
>paravirt overhead.
>
>The first patch is a prerequisite fix for alternative patching. Its
>is needed due to the initial indirect call needs to be padded with
>NOPs in some cases with the following patches.
>
>In order to make this less intrusive, some further reorganization of
>the MSR access helpers is done in the patches 1-6.
>
>The next 4 patches are converting the non-paravirt case to use direct
>inlining of the MSR access instructions, including the WRMSRNS
>instruction and the immediate variants of RDMSR and WRMSR if possible.
>
>Patches 11-13 are some further preparations for making the real switch
>to directly patch in the native MSR instructions easier.
>
>Patch 14 is switching the paravirt MSR function interface from normal
>call ABI to one more similar to the native MSR instructions.
>
>Patch 15 is a little cleanup patch.
>
>Patch 16 is the final step for patching in the native MSR instructions
>when not running as a Xen PV guest.
>
>This series has been tested to work with Xen PV and on bare metal.
>
>Note that there is more room for improvement. This series is sent out
>to get a first impression how the code will basically look like.
Does that mean you are considering this patchset an RFC? If so, you should put that in the subject header.
>Right now the same problem is solved differently for the paravirt and
>the non-paravirt cases. In case this is not desired, there are two
>possibilities to merge the two implementations. Both solutions have
>the common idea to have rather similar code for paravirt and
>non-paravirt variants, but just use a different main macro for
>generating the respective code. For making the code of both possible
>scenarios more similar, the following variants are possible:
>
>1. Remove the micro-optimizations of the non-paravirt case, making
> it similar to the paravirt code in my series. This has the
> advantage of being more simple, but might have a very small
> negative performance impact (probably not really detectable).
>
>2. Add the same micro-optimizations to the paravirt case, requiring
> to enhance paravirt patching to support a to be patched indirect
> call in the middle of the initial code snipplet.
>
>In both cases the native MSR function variants would no longer be
>usable in the paravirt case, but this would mostly affect Xen, as it
>would need to open code the WRMSR/RDMSR instructions to be used
>instead the native_*msr*() functions.
>
>Changes since V2:
>- switch back to the paravirt approach
>
>Changes since V1:
>- Use Xin Li's approach for inlining
>- Several new patches
>
>Juergen Gross (16):
> x86/alternative: Support alt_replace_call() with instructions after
> call
> coco/tdx: Rename MSR access helpers
> x86/sev: Replace call of native_wrmsr() with native_wrmsrq()
> KVM: x86: Remove the KVM private read_msr() function
> x86/msr: Minimize usage of native_*() msr access functions
> x86/msr: Move MSR trace calls one function level up
> x86/opcode: Add immediate form MSR instructions
> x86/extable: Add support for immediate form MSR instructions
> x86/msr: Use the alternatives mechanism for WRMSR
> x86/msr: Use the alternatives mechanism for RDMSR
> x86/alternatives: Add ALTERNATIVE_4()
> x86/paravirt: Split off MSR related hooks into new header
> x86/paravirt: Prepare support of MSR instruction interfaces
> x86/paravirt: Switch MSR access pv_ops functions to instruction
> interfaces
> x86/msr: Reduce number of low level MSR access helpers
> x86/paravirt: Use alternatives for MSR access with paravirt
>
> arch/x86/coco/sev/internal.h | 7 +-
> arch/x86/coco/tdx/tdx.c | 8 +-
> arch/x86/hyperv/ivm.c | 2 +-
> arch/x86/include/asm/alternative.h | 6 +
> arch/x86/include/asm/fred.h | 2 +-
> arch/x86/include/asm/kvm_host.h | 10 -
> arch/x86/include/asm/msr.h | 345 ++++++++++++++++------
> arch/x86/include/asm/paravirt-msr.h | 148 ++++++++++
> arch/x86/include/asm/paravirt.h | 67 -----
> arch/x86/include/asm/paravirt_types.h | 57 ++--
> arch/x86/include/asm/qspinlock_paravirt.h | 4 +-
> arch/x86/kernel/alternative.c | 5 +-
> arch/x86/kernel/cpu/mshyperv.c | 7 +-
> arch/x86/kernel/kvmclock.c | 2 +-
> arch/x86/kernel/paravirt.c | 42 ++-
> arch/x86/kvm/svm/svm.c | 16 +-
> arch/x86/kvm/vmx/tdx.c | 2 +-
> arch/x86/kvm/vmx/vmx.c | 8 +-
> arch/x86/lib/x86-opcode-map.txt | 5 +-
> arch/x86/mm/extable.c | 35 ++-
> arch/x86/xen/enlighten_pv.c | 52 +++-
> arch/x86/xen/pmu.c | 4 +-
> tools/arch/x86/lib/x86-opcode-map.txt | 5 +-
> tools/objtool/check.c | 1 +
> 24 files changed, 576 insertions(+), 264 deletions(-)
> create mode 100644 arch/x86/include/asm/paravirt-msr.h
>
Could you clarify *on the high design level* what "go back to the paravirt approach" means, and the motivation for that?
Note that for Xen *most* MSRs fall in one of two categories: those that are dropped entirely and those that are just passed straight on to the hardware.
I don't know if anyone cares about optimizing PV Xen anymore, but at least in theory Xen can un-paravirtualize most sites.
^ permalink raw reply
* Re: [RFC PATCH v5 05/45] KVM: TDX: Drop kvm_x86_ops.link_external_spt(), use .set_external_spte() for all
From: Edgecombe, Rick P @ 2026-02-18 19:37 UTC (permalink / raw)
To: seanjc@google.com, Zhao, Yan Y
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev, Li, Xiaoyao,
Huang, Kai, dave.hansen@linux.intel.com, kas@kernel.org,
binbin.wu@linux.intel.com, mingo@redhat.com, pbonzini@redhat.com,
ackerleytng@google.com, linux-kernel@vger.kernel.org,
Yamahata, Isaku, sagis@google.com, tglx@kernel.org, bp@alien8.de,
Annapurve, Vishal, x86@kernel.org
In-Reply-To: <aYJU8Som706YkIEO@google.com>
On Tue, 2026-02-03 at 20:05 +0000, Sean Christopherson wrote:
> > And mirror_spte --> new_spte?
>
> Hmm, ya, I made that change later, but it can probably be shifted here.
Sorry for the late comment on the tiny detail, but things seemed to have calmed
down enough to attempt to merge these discussions into the snarl.
It doesn't quite fit in this patch because the set_external_spte() op also uses
the mirror_pte name. So then you need to either expand the scope of the patch to
change "mirror" to "new" across the callchain, or creating a small mismatch
between tdx_sept_set_private_spte() and tdx_sept_link_private_spt().
The patch where it happens in this series needs to add the old_pte, forcing
mirror_spte to grow some new nomenclature. So on balance I think it fits better
there, and we should leave it alone here. We can update it in
tdx_sept_link_private_spt() in "KVM: x86/mmu: Plumb the old_spte into
kvm_x86_ops.set_external_spte()".
^ permalink raw reply
* Re: [PATCH 2/6] x86/sev: add support for enabling RMPOPT
From: Dave Hansen @ 2026-02-18 17:17 UTC (permalink / raw)
To: Kalra, Ashish, K Prateek Nayak, tglx, mingo, bp, dave.hansen, x86,
hpa, seanjc, peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, Tycho.Andersen, Nathan.Fontenot,
jackyli, pgonda, rientjes, jacobhxu, xin, pawan.kumar.gupta,
babu.moger, dyoung, nikunj, john.allen, darwi, linux-kernel,
linux-crypto, kvm, linux-coco
In-Reply-To: <e774a45a-c31d-4051-93f7-3848de539888@amd.com>
On 2/18/26 09:07, Kalra, Ashish wrote:
> On 2/18/2026 11:01 AM, Dave Hansen wrote:
>> On 2/18/26 08:55, Kalra, Ashish wrote:
>>> Because, setting RMPOPT_BASE MSR (which is a per-core MSR) and
>>> RMPOPT instruction need to be issued on only one thread per core. If
>>> the primary thread is offlined and secondary thread is not
>>> considered, we will miss/skip setting either the RMPOPT_BASE MSR or
>>> not issuing the RMPOPT instruction for that physical CPU, which
>>> means no RMP optimizations enabled for that physical CPU.
>> What is the harm of issuing it twice per core?
> Why to issue it if we can avoid it.
>
> It is not that complex to setup a cpumask containing the online primary
> or secondary thread and then issue the RMPOPT instruction only once per
> thread.
It's a non-zero amount of error-prone kernel code. It *is* complex. It
has to be reviewed and maintained.
Please remove this unnecessary optimization from the series. If you
would like to add it back, please do it in a patch at the end so it can
be evaluated on its own. Include performance numbers so the code
complexity can be balanced against the performance gain.
^ permalink raw reply
* Re: [PATCH 0/6] Add RMPOPT support.
From: Dave Hansen @ 2026-02-18 17:15 UTC (permalink / raw)
To: Kalra, Ashish, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <cc930514-b0c0-4b9f-8287-aaee2878e668@amd.com>
On 2/18/26 09:03, Kalra, Ashish wrote:
>> They are known not to contain any SEV-SNP guest memory at the
>> moment snp_rmptable_init() finishes, no?
> Yes, but RMP checks are still performed and they affect performance.
>
> Testing a bit in the per‑CPU RMPOPT table to avoid RMP checks
> significantly improves performance.
Sorry, Ashish, I don't think I'm explaining myself very well. Let me try
again, please.
First, my goal here is to ensure that the system has a whole has good
performance, with minimal kernel code, and in the most common
configurations.
I would wager that the most common SEV-SNP configuration in the whole
world is a system that has booted, enabled SEV-SNP, and has never run an
SEV-SNP guest. If it's not *the* most common, it's certainly going to be
common enough to care about deeply.
Do you agree?
If you agree, I hope we can also agree that a "SNP enabled but never ran
a guest" state is deserving of good performance with minimal kernel code.
My assumption (which is maybe a bad one) is that there is a natural
point when SEV-SNP is enabled on the system when the system as a whole
can easily assert that no SEV-SNP guest has ever run. I'm assuming that
there is *a* point where, for instance, the RMP table gets atomically
flipped from being unprotected to being protected. At that point, its
state *must* be known. It must also be naturally obvious that no guest
has had a chance to run at this point.
If that point can be leveraged, and the RMPOPT optimization can be
applied at SEV-SNP enabled time, then an important SEV-SNP configuration
would be optimized by default and with zero or little kernel code needed
to drive it.
To me, that seems like a valuable goal.
Do you agree?
^ permalink raw reply
* Re: [PATCH 2/6] x86/sev: add support for enabling RMPOPT
From: Kalra, Ashish @ 2026-02-18 17:07 UTC (permalink / raw)
To: Dave Hansen, K Prateek Nayak, tglx, mingo, bp, dave.hansen, x86,
hpa, seanjc, peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, Tycho.Andersen, Nathan.Fontenot,
jackyli, pgonda, rientjes, jacobhxu, xin, pawan.kumar.gupta,
babu.moger, dyoung, nikunj, john.allen, darwi, linux-kernel,
linux-crypto, kvm, linux-coco
In-Reply-To: <eaa17a3f-e481-49dc-8d7b-bd5247d3ed57@intel.com>
On 2/18/2026 11:01 AM, Dave Hansen wrote:
> On 2/18/26 08:55, Kalra, Ashish wrote:
>> Because, setting RMPOPT_BASE MSR (which is a per-core MSR) and
>> RMPOPT instruction need to be issued on only one thread per core. If
>> the primary thread is offlined and secondary thread is not
>> considered, we will miss/skip setting either the RMPOPT_BASE MSR or
>> not issuing the RMPOPT instruction for that physical CPU, which
>> means no RMP optimizations enabled for that physical CPU.
> What is the harm of issuing it twice per core?
Why to issue it if we can avoid it.
It is not that complex to setup a cpumask containing the online primary
or secondary thread and then issue the RMPOPT instruction only once per
thread.
Thanks,
Ashish
^ permalink raw reply
* Re: [PATCH 0/6] Add RMPOPT support.
From: Kalra, Ashish @ 2026-02-18 17:03 UTC (permalink / raw)
To: Dave Hansen, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <31b42ba3-dd0c-42e7-ad1e-800c5cd2bcf8@intel.com>
On 2/18/2026 9:03 AM, Dave Hansen wrote:
> On 2/17/26 20:12, Kalra, Ashish wrote:
>>> That's not awful.
>>>
>>> To be honest, though, I think this is misdesigned. Shouldn't the CPU
>>> *boot* in a state where it is optimized? Why should software have to
>>> tell it that coming out of reset, there is no SEV-SNP memory?
>> When the CPU boots, the RMP checks are not done and therefore the CPU
>> is booting in a state where it is optimized.
>>
>> The RMP checks are not enabled till SEV-SNP is enabled and SNP is enabled
>> during kernel boot (as part of iommu_snp_enable() -> snp_rmptable_init()).
>>
>> Once SNP is enabled as part of kernel boot, hypervisor and non-SNP guests are
>> subject to RMP checks on writes to provide integrity of SEV-SNP guest memory.
>>
>> Therefore, we need to enable these RMP optimizations after SNP has been
>> enabled to indicate which 1GB regions of memory are known to not contain any
>> SEV-SNP guest memory.
>
> They are known not to contain any SEV-SNP guest memory at the moment
> snp_rmptable_init() finishes, no?
Yes, but RMP checks are still performed and they affect performance.
Testing a bit in the per‑CPU RMPOPT table to avoid RMP checks significantly improves performance.
Thanks,
Ashish
^ permalink raw reply
* Re: [PATCH 2/6] x86/sev: add support for enabling RMPOPT
From: Dave Hansen @ 2026-02-18 17:01 UTC (permalink / raw)
To: Kalra, Ashish, K Prateek Nayak, tglx, mingo, bp, dave.hansen, x86,
hpa, seanjc, peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, Tycho.Andersen, Nathan.Fontenot,
jackyli, pgonda, rientjes, jacobhxu, xin, pawan.kumar.gupta,
babu.moger, dyoung, nikunj, john.allen, darwi, linux-kernel,
linux-crypto, kvm, linux-coco
In-Reply-To: <4e912046-8ae0-4cb2-b2cb-11c754df7536@amd.com>
On 2/18/26 08:55, Kalra, Ashish wrote:
> Because, setting RMPOPT_BASE MSR (which is a per-core MSR) and
> RMPOPT instruction need to be issued on only one thread per core. If
> the primary thread is offlined and secondary thread is not
> considered, we will miss/skip setting either the RMPOPT_BASE MSR or
> not issuing the RMPOPT instruction for that physical CPU, which
> means no RMP optimizations enabled for that physical CPU.
What is the harm of issuing it twice per core?
^ permalink raw reply
* Re: [PATCH 2/6] x86/sev: add support for enabling RMPOPT
From: Kalra, Ashish @ 2026-02-18 16:55 UTC (permalink / raw)
To: Dave Hansen, K Prateek Nayak, tglx, mingo, bp, dave.hansen, x86,
hpa, seanjc, peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, Tycho.Andersen, Nathan.Fontenot,
jackyli, pgonda, rientjes, jacobhxu, xin, pawan.kumar.gupta,
babu.moger, dyoung, nikunj, john.allen, darwi, linux-kernel,
linux-crypto, kvm, linux-coco
In-Reply-To: <59c0b0f0-26b0-4311-82a9-a5f8392ec4c6@intel.com>
On 2/18/2026 8:59 AM, Dave Hansen wrote:
> On 2/17/26 19:08, K Prateek Nayak wrote:
>> Hello Dave,
>>
>> On 2/18/2026 3:36 AM, Dave Hansen wrote:
>>>> +/*
>>>> + * Build a cpumask of online primary threads, accounting for primary threads
>>>> + * that have been offlined while their secondary threads are still online.
>>>> + */
>>>> +static void get_cpumask_of_primary_threads(cpumask_var_t cpulist)
>>>> +{
>>>> + cpumask_t cpus;
>>>> + int cpu;
>>>> +
>>>> + cpumask_copy(&cpus, cpu_online_mask);
>>>> + for_each_cpu(cpu, &cpus) {
>>>> + cpumask_set_cpu(cpu, cpulist);
>>>> + cpumask_andnot(&cpus, &cpus, cpu_smt_mask(cpu));
>>>> + }
>>>> +}
>>>
>>> Don't we have a primary thread mask already? I thought we did.
>>
>> If you are referring to cpu_primary_thread_mask(), the CPUs are set on it
>> based on the LSB of APICID, specifically:
>>
>> !(apicid & (__max_threads_per_core - 1))
>>
>> It can so happen, the primary thread ((apicid & 1) == 0) of the core is
>> offline while the secondary thread ((apicid & 1) == 1) is online but the
>> traversal of (cpu_primary_thread_mask() & cpu_online_mask()) will simply
>> skip these cores.
>>
>> Is there an equivalent mask that sets the first online CPU of each core?
>
> No I don't think we have that sitting around.
>
> But, stepping back, why is this even necessary? Is it just saving a few
> IPIs in the super rare case that someone has offlined the primary thread
> but not a secondary one?
>
> Why bother?
Because, setting RMPOPT_BASE MSR (which is a per-core MSR) and RMPOPT instruction
need to be issued on only one thread per core. If the primary thread is offlined
and secondary thread is not considered, we will miss/skip setting either the
RMPOPT_BASE MSR or not issuing the RMPOPT instruction for that physical CPU, which means
no RMP optimizations enabled for that physical CPU.
Thanks,
Ashish
^ permalink raw reply
* Re: [PATCH 3/6] x86/sev: add support for RMPOPT instruction
From: Uros Bizjak @ 2026-02-18 16:28 UTC (permalink / raw)
To: Ashish Kalra, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <66348e8ad761a1b0ccb26c8027efedf46329db54.1771321114.git.ashish.kalra@amd.com>
On 2/17/26 21:10, Ashish Kalra wrote:
> From: Ashish Kalra <ashish.kalra@amd.com>
>
> As SEV-SNP is enabled by default on boot when an RMP table is
> allocated by BIOS, the hypervisor and non-SNP guests are subject to
> RMP write checks to provide integrity of SNP guest memory.
>
> RMPOPT is a new instruction that minimizes the performance overhead of
> RMP checks on the hypervisor and on non-SNP guests by allowing RMP
> checks to be skipped for 1GB regions of memory that are known not to
> contain any SEV-SNP guest memory.
>
> Enable RMPOPT optimizations globally for all system RAM at RMP
> initialization time. RMP checks can initially be skipped for 1GB memory
> ranges that do not contain SEV-SNP guest memory (excluding preassigned
> pages such as the RMP table and firmware pages). As SNP guests are
> launched, RMPUPDATE will disable the corresponding RMPOPT optimizations.
>
> Suggested-by: Thomas Lendacky <thomas.lendacky@amd.com>
> Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
> ---
> arch/x86/virt/svm/sev.c | 84 +++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 84 insertions(+)
>
> diff --git a/arch/x86/virt/svm/sev.c b/arch/x86/virt/svm/sev.c
> index e6b784d26c33..a0d38fc50698 100644
> --- a/arch/x86/virt/svm/sev.c
> +++ b/arch/x86/virt/svm/sev.c
> @@ -19,6 +19,7 @@
> #include <linux/iommu.h>
> #include <linux/amd-iommu.h>
> #include <linux/nospec.h>
> +#include <linux/kthread.h>
>
> #include <asm/sev.h>
> #include <asm/processor.h>
> @@ -127,10 +128,17 @@ static DEFINE_SPINLOCK(snp_leaked_pages_list_lock);
>
> static unsigned long snp_nr_leaked_pages;
>
> +enum rmpopt_function {
> + RMPOPT_FUNC_VERIFY_AND_REPORT_STATUS,
> + RMPOPT_FUNC_REPORT_STATUS
> +};
> +
> #define RMPOPT_TABLE_MAX_LIMIT_IN_TB 2
> #define NUM_TB(pfn_min, pfn_max) \
> (((pfn_max) - (pfn_min)) / (1 << (40 - PAGE_SHIFT)))
>
> +static struct task_struct *rmpopt_task;
> +
> struct rmpopt_socket_config {
> unsigned long start_pfn, end_pfn;
> cpumask_var_t cpulist;
> @@ -527,6 +535,66 @@ static void get_cpumask_of_primary_threads(cpumask_var_t cpulist)
> }
> }
>
> +/*
> + * 'val' is a system physical address aligned to 1GB OR'ed with
> + * a function selection. Currently supported functions are 0
> + * (verify and report status) and 1 (report status).
> + */
> +static void rmpopt(void *val)
> +{
> + asm volatile(".byte 0xf2, 0x0f, 0x01, 0xfc\n\t"
There is no need for \n\t instruction delimiter with single instruction
in the asm template, it will just confuse compiler's insn count estimator.
Uros.
> + : : "a" ((u64)val & PUD_MASK), "c" ((u64)val & 0x1)
> + : "memory", "cc");
> +}
> +
> +static int rmpopt_kthread(void *__unused)
> +{
> + phys_addr_t pa_start, pa_end;
> + cpumask_var_t cpus;
> +
> + if (!zalloc_cpumask_var(&cpus, GFP_KERNEL))
> + return -ENOMEM;
> +
> + pa_start = ALIGN_DOWN(PFN_PHYS(min_low_pfn), PUD_SIZE);
> + pa_end = ALIGN(PFN_PHYS(max_pfn), PUD_SIZE);
> +
> + while (!kthread_should_stop()) {
> + phys_addr_t pa;
> +
> + pr_info("RMP optimizations enabled on physical address range @1GB alignment [0x%016llx - 0x%016llx]\n",
> + pa_start, pa_end);
> +
> + /* Only one thread per core needs to issue RMPOPT instruction */
> + get_cpumask_of_primary_threads(cpus);
> +
> + /*
> + * RMPOPT optimizations skip RMP checks at 1GB granularity if this range of
> + * memory does not contain any SNP guest memory.
> + */
> + for (pa = pa_start; pa < pa_end; pa += PUD_SIZE) {
> + /* Bit zero passes the function to the RMPOPT instruction. */
> + on_each_cpu_mask(cpus, rmpopt,
> + (void *)(pa | RMPOPT_FUNC_VERIFY_AND_REPORT_STATUS),
> + true);
> +
> + /* Give a chance for other threads to run */
> + cond_resched();
> + }
> +
> + set_current_state(TASK_INTERRUPTIBLE);
> + schedule();
> + }
> +
> + free_cpumask_var(cpus);
> + return 0;
> +}
> +
> +static void rmpopt_all_physmem(void)
> +{
> + if (rmpopt_task)
> + wake_up_process(rmpopt_task);
> +}
> +
> static void __configure_rmpopt(void *val)
> {
> u64 rmpopt_base = ((u64)val & PUD_MASK) | MSR_AMD64_RMPOPT_ENABLE;
> @@ -687,6 +755,22 @@ static __init void configure_and_enable_rmpopt(void)
> else
> configure_rmpopt_large_physmem(primary_threads_cpulist);
>
> + rmpopt_task = kthread_create(rmpopt_kthread, NULL, "rmpopt_kthread");
> + if (IS_ERR(rmpopt_task)) {
> + pr_warn("Unable to start RMPOPT kernel thread\n");
> + rmpopt_task = NULL;
> + goto free_cpumask;
> + }
> +
> + pr_info("RMPOPT worker thread created with PID %d\n", task_pid_nr(rmpopt_task));
> +
> + /*
> + * Once all per-CPU RMPOPT tables have been configured, enable RMPOPT
> + * optimizations on all physical memory.
> + */
> + rmpopt_all_physmem();
> +
> +free_cpumask:
> free_cpumask_var(primary_threads_cpulist);
> }
>
^ permalink raw reply
* Re: [PATCH 5/6] x86/sev: Use configfs to re-enable RMP optimizations.
From: Dave Hansen @ 2026-02-18 15:10 UTC (permalink / raw)
To: Kalra, Ashish, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <e72165ed-c65d-4d21-bff6-9981b46311cf@amd.com>
On 2/17/26 19:34, Kalra, Ashish wrote:
...
> As currently, i.e, as part of this patch series, there is no
> mechanism to re-issue RMPOPT automatically as part of SNP guest
> cleanup, therefore this support exists to doing it manually at
> runtime via configfs.
I think you need a mechanism that re-enable RMP optimizations
automatically for this feature to go upstream. It's just dead code
otherwise, and we don't merge dead code.
A configfs hack doesn't really count.
^ permalink raw reply
* Re: [PATCH 0/6] Add RMPOPT support.
From: Dave Hansen @ 2026-02-18 15:03 UTC (permalink / raw)
To: Kalra, Ashish, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <65986f9e-59e8-4f1c-aaa7-1edf45af24d8@amd.com>
On 2/17/26 20:12, Kalra, Ashish wrote:
>> That's not awful.
>>
>> To be honest, though, I think this is misdesigned. Shouldn't the CPU
>> *boot* in a state where it is optimized? Why should software have to
>> tell it that coming out of reset, there is no SEV-SNP memory?
> When the CPU boots, the RMP checks are not done and therefore the CPU
> is booting in a state where it is optimized.
>
> The RMP checks are not enabled till SEV-SNP is enabled and SNP is enabled
> during kernel boot (as part of iommu_snp_enable() -> snp_rmptable_init()).
>
> Once SNP is enabled as part of kernel boot, hypervisor and non-SNP guests are
> subject to RMP checks on writes to provide integrity of SEV-SNP guest memory.
>
> Therefore, we need to enable these RMP optimizations after SNP has been
> enabled to indicate which 1GB regions of memory are known to not contain any
> SEV-SNP guest memory.
They are known not to contain any SEV-SNP guest memory at the moment
snp_rmptable_init() finishes, no?
^ permalink raw reply
* Re: [PATCH 2/6] x86/sev: add support for enabling RMPOPT
From: Dave Hansen @ 2026-02-18 14:59 UTC (permalink / raw)
To: K Prateek Nayak, Ashish Kalra, tglx, mingo, bp, dave.hansen, x86,
hpa, seanjc, peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, Tycho.Andersen, Nathan.Fontenot,
jackyli, pgonda, rientjes, jacobhxu, xin, pawan.kumar.gupta,
babu.moger, dyoung, nikunj, john.allen, darwi, linux-kernel,
linux-crypto, kvm, linux-coco
In-Reply-To: <b860e5f4-4111-4de7-acc7-aec4a3f23908@amd.com>
On 2/17/26 19:08, K Prateek Nayak wrote:
> Hello Dave,
>
> On 2/18/2026 3:36 AM, Dave Hansen wrote:
>>> +/*
>>> + * Build a cpumask of online primary threads, accounting for primary threads
>>> + * that have been offlined while their secondary threads are still online.
>>> + */
>>> +static void get_cpumask_of_primary_threads(cpumask_var_t cpulist)
>>> +{
>>> + cpumask_t cpus;
>>> + int cpu;
>>> +
>>> + cpumask_copy(&cpus, cpu_online_mask);
>>> + for_each_cpu(cpu, &cpus) {
>>> + cpumask_set_cpu(cpu, cpulist);
>>> + cpumask_andnot(&cpus, &cpus, cpu_smt_mask(cpu));
>>> + }
>>> +}
>>
>> Don't we have a primary thread mask already? I thought we did.
>
> If you are referring to cpu_primary_thread_mask(), the CPUs are set on it
> based on the LSB of APICID, specifically:
>
> !(apicid & (__max_threads_per_core - 1))
>
> It can so happen, the primary thread ((apicid & 1) == 0) of the core is
> offline while the secondary thread ((apicid & 1) == 1) is online but the
> traversal of (cpu_primary_thread_mask() & cpu_online_mask()) will simply
> skip these cores.
>
> Is there an equivalent mask that sets the first online CPU of each core?
No I don't think we have that sitting around.
But, stepping back, why is this even necessary? Is it just saving a few
IPIs in the super rare case that someone has offlined the primary thread
but not a secondary one?
Why bother?
^ permalink raw reply
* Re: [PATCH v3 04/16] KVM: x86: Remove the KVM private read_msr() function
From: Sean Christopherson @ 2026-02-18 14:29 UTC (permalink / raw)
To: Rick P Edgecombe
Cc: kvm@vger.kernel.org, jgross@suse.com, linux-coco@lists.linux.dev,
linux-kernel@vger.kernel.org, x86@kernel.org, bp@alien8.de,
kas@kernel.org, hpa@zytor.com, mingo@redhat.com,
dave.hansen@linux.intel.com, tglx@kernel.org, pbonzini@redhat.com
In-Reply-To: <330ed8d57ee5c7574d7bc1b637598bbef5325ee4.camel@intel.com>
On Wed, Feb 18, 2026, Rick P Edgecombe wrote:
> On Wed, 2026-02-18 at 09:21 +0100, Juergen Gross wrote:
> > Instead of having a KVM private read_msr() function, just use
> > rdmsrq().
>
> Might be nice to include a little bit more on the "why", but the patch
> is pretty simple.
Eh, the why is basically "KVM is old and crusty". I'm a-ok without a history
lesson on how we got here :-)
> > Signed-off-by: Juergen Gross <jgross@suse.com>
> > Reviewed-by: H. Peter Anvin (Intel) <hpa@zytor.com>
>
> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Acked-by: Sean Christopherson <seanjc@google.com>
^ permalink raw reply
* Re: [PATCH v3 04/16] KVM: x86: Remove the KVM private read_msr() function
From: Edgecombe, Rick P @ 2026-02-18 14:21 UTC (permalink / raw)
To: kvm@vger.kernel.org, jgross@suse.com, linux-coco@lists.linux.dev,
linux-kernel@vger.kernel.org, x86@kernel.org
Cc: seanjc@google.com, bp@alien8.de, kas@kernel.org, hpa@zytor.com,
mingo@redhat.com, dave.hansen@linux.intel.com, tglx@kernel.org,
pbonzini@redhat.com
In-Reply-To: <20260218082133.400602-5-jgross@suse.com>
On Wed, 2026-02-18 at 09:21 +0100, Juergen Gross wrote:
> Instead of having a KVM private read_msr() function, just use
> rdmsrq().
Might be nice to include a little bit more on the "why", but the patch
is pretty simple.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Reviewed-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
^ permalink raw reply
* Re: [PATCH v3 02/16] coco/tdx: Rename MSR access helpers
From: Edgecombe, Rick P @ 2026-02-18 14:11 UTC (permalink / raw)
To: kvm@vger.kernel.org, jgross@suse.com, linux-coco@lists.linux.dev,
linux-kernel@vger.kernel.org, x86@kernel.org
Cc: hpa@zytor.com, mingo@redhat.com, tglx@kernel.org, kas@kernel.org,
bp@alien8.de, dave.hansen@linux.intel.com
In-Reply-To: <20260218082133.400602-3-jgross@suse.com>
On Wed, 2026-02-18 at 09:21 +0100, Juergen Gross wrote:
> In order to avoid a name clash with some general MSR access helpers
> after a future MSR infrastructure rework, rename the TDX specific
> helpers.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Reviewed-by: Kiryl Shutsemau <kas@kernel.org>
> Reviewed-by: H. Peter Anvin (Intel) <hpa@zytor.com>
> ---
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
^ permalink raw reply
* [PATCH v3 04/16] KVM: x86: Remove the KVM private read_msr() function
From: Juergen Gross @ 2026-02-18 8:21 UTC (permalink / raw)
To: linux-kernel, x86, kvm, linux-coco
Cc: Juergen Gross, Sean Christopherson, Paolo Bonzini,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Kiryl Shutsemau, Rick Edgecombe
In-Reply-To: <20260218082133.400602-1-jgross@suse.com>
Instead of having a KVM private read_msr() function, just use rdmsrq().
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: H. Peter Anvin (Intel) <hpa@zytor.com>
---
V2:
- remove the helper and use rdmsrq() directly (Sean Christopherson)
---
arch/x86/include/asm/kvm_host.h | 10 ----------
arch/x86/kvm/vmx/tdx.c | 2 +-
arch/x86/kvm/vmx/vmx.c | 6 +++---
3 files changed, 4 insertions(+), 14 deletions(-)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index ff07c45e3c73..9034222a96e8 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -2347,16 +2347,6 @@ static inline void kvm_load_ldt(u16 sel)
asm("lldt %0" : : "rm"(sel));
}
-#ifdef CONFIG_X86_64
-static inline unsigned long read_msr(unsigned long msr)
-{
- u64 value;
-
- rdmsrq(msr, value);
- return value;
-}
-#endif
-
static inline void kvm_inject_gp(struct kvm_vcpu *vcpu, u32 error_code)
{
kvm_queue_exception_e(vcpu, GP_VECTOR, error_code);
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 5df9d32d2058..d9e371e39853 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -801,7 +801,7 @@ void tdx_prepare_switch_to_guest(struct kvm_vcpu *vcpu)
if (likely(is_64bit_mm(current->mm)))
vt->msr_host_kernel_gs_base = current->thread.gsbase;
else
- vt->msr_host_kernel_gs_base = read_msr(MSR_KERNEL_GS_BASE);
+ rdmsrq(MSR_KERNEL_GS_BASE, vt->msr_host_kernel_gs_base);
vt->guest_state_loaded = true;
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 967b58a8ab9d..3799cbbb4577 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -1403,8 +1403,8 @@ void vmx_prepare_switch_to_guest(struct kvm_vcpu *vcpu)
} else {
savesegment(fs, fs_sel);
savesegment(gs, gs_sel);
- fs_base = read_msr(MSR_FS_BASE);
- vt->msr_host_kernel_gs_base = read_msr(MSR_KERNEL_GS_BASE);
+ rdmsrq(MSR_FS_BASE, fs_base);
+ rdmsrq(MSR_KERNEL_GS_BASE, vt->msr_host_kernel_gs_base);
}
wrmsrq(MSR_KERNEL_GS_BASE, vmx->msr_guest_kernel_gs_base);
@@ -1463,7 +1463,7 @@ static u64 vmx_read_guest_host_msr(struct vcpu_vmx *vmx, u32 msr, u64 *cache)
{
preempt_disable();
if (vmx->vt.guest_state_loaded)
- *cache = read_msr(msr);
+ rdmsrq(msr, *cache);
preempt_enable();
return *cache;
}
--
2.53.0
^ permalink raw reply related
* [PATCH v3 02/16] coco/tdx: Rename MSR access helpers
From: Juergen Gross @ 2026-02-18 8:21 UTC (permalink / raw)
To: linux-kernel, x86, linux-coco, kvm
Cc: Juergen Gross, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, H. Peter Anvin, Kiryl Shutsemau, Rick Edgecombe
In-Reply-To: <20260218082133.400602-1-jgross@suse.com>
In order to avoid a name clash with some general MSR access helpers
after a future MSR infrastructure rework, rename the TDX specific
helpers.
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Kiryl Shutsemau <kas@kernel.org>
Reviewed-by: H. Peter Anvin (Intel) <hpa@zytor.com>
---
arch/x86/coco/tdx/tdx.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/x86/coco/tdx/tdx.c b/arch/x86/coco/tdx/tdx.c
index 7b2833705d47..500166c1a161 100644
--- a/arch/x86/coco/tdx/tdx.c
+++ b/arch/x86/coco/tdx/tdx.c
@@ -468,7 +468,7 @@ static void __cpuidle tdx_safe_halt(void)
raw_local_irq_enable();
}
-static int read_msr(struct pt_regs *regs, struct ve_info *ve)
+static int tdx_read_msr(struct pt_regs *regs, struct ve_info *ve)
{
struct tdx_module_args args = {
.r10 = TDX_HYPERCALL_STANDARD,
@@ -489,7 +489,7 @@ static int read_msr(struct pt_regs *regs, struct ve_info *ve)
return ve_instr_len(ve);
}
-static int write_msr(struct pt_regs *regs, struct ve_info *ve)
+static int tdx_write_msr(struct pt_regs *regs, struct ve_info *ve)
{
struct tdx_module_args args = {
.r10 = TDX_HYPERCALL_STANDARD,
@@ -842,9 +842,9 @@ static int virt_exception_kernel(struct pt_regs *regs, struct ve_info *ve)
case EXIT_REASON_HLT:
return handle_halt(ve);
case EXIT_REASON_MSR_READ:
- return read_msr(regs, ve);
+ return tdx_read_msr(regs, ve);
case EXIT_REASON_MSR_WRITE:
- return write_msr(regs, ve);
+ return tdx_write_msr(regs, ve);
case EXIT_REASON_CPUID:
return handle_cpuid(regs, ve);
case EXIT_REASON_EPT_VIOLATION:
--
2.53.0
^ permalink raw reply related
* [PATCH v3 00/16] x86/msr: Inline rdmsr/wrmsr instructions
From: Juergen Gross @ 2026-02-18 8:21 UTC (permalink / raw)
To: linux-kernel, x86, linux-coco, kvm, linux-hyperv, virtualization,
llvm
Cc: Juergen Gross, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, H. Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
Sean Christopherson, Paolo Bonzini, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Vitaly Kuznetsov,
Boris Ostrovsky, xen-devel, Ajay Kaher, Alexey Makhalov,
Broadcom internal kernel review list, Andy Lutomirski,
Peter Zijlstra, Xin Li, Nathan Chancellor, Nick Desaulniers,
Bill Wendling, Justin Stitt, Josh Poimboeuf
When building a kernel with CONFIG_PARAVIRT_XXL the paravirt
infrastructure will always use functions for reading or writing MSRs,
even when running on bare metal.
Switch to inline RDMSR/WRMSR instructions in this case, reducing the
paravirt overhead.
The first patch is a prerequisite fix for alternative patching. Its
is needed due to the initial indirect call needs to be padded with
NOPs in some cases with the following patches.
In order to make this less intrusive, some further reorganization of
the MSR access helpers is done in the patches 1-6.
The next 4 patches are converting the non-paravirt case to use direct
inlining of the MSR access instructions, including the WRMSRNS
instruction and the immediate variants of RDMSR and WRMSR if possible.
Patches 11-13 are some further preparations for making the real switch
to directly patch in the native MSR instructions easier.
Patch 14 is switching the paravirt MSR function interface from normal
call ABI to one more similar to the native MSR instructions.
Patch 15 is a little cleanup patch.
Patch 16 is the final step for patching in the native MSR instructions
when not running as a Xen PV guest.
This series has been tested to work with Xen PV and on bare metal.
Note that there is more room for improvement. This series is sent out
to get a first impression how the code will basically look like.
Right now the same problem is solved differently for the paravirt and
the non-paravirt cases. In case this is not desired, there are two
possibilities to merge the two implementations. Both solutions have
the common idea to have rather similar code for paravirt and
non-paravirt variants, but just use a different main macro for
generating the respective code. For making the code of both possible
scenarios more similar, the following variants are possible:
1. Remove the micro-optimizations of the non-paravirt case, making
it similar to the paravirt code in my series. This has the
advantage of being more simple, but might have a very small
negative performance impact (probably not really detectable).
2. Add the same micro-optimizations to the paravirt case, requiring
to enhance paravirt patching to support a to be patched indirect
call in the middle of the initial code snipplet.
In both cases the native MSR function variants would no longer be
usable in the paravirt case, but this would mostly affect Xen, as it
would need to open code the WRMSR/RDMSR instructions to be used
instead the native_*msr*() functions.
Changes since V2:
- switch back to the paravirt approach
Changes since V1:
- Use Xin Li's approach for inlining
- Several new patches
Juergen Gross (16):
x86/alternative: Support alt_replace_call() with instructions after
call
coco/tdx: Rename MSR access helpers
x86/sev: Replace call of native_wrmsr() with native_wrmsrq()
KVM: x86: Remove the KVM private read_msr() function
x86/msr: Minimize usage of native_*() msr access functions
x86/msr: Move MSR trace calls one function level up
x86/opcode: Add immediate form MSR instructions
x86/extable: Add support for immediate form MSR instructions
x86/msr: Use the alternatives mechanism for WRMSR
x86/msr: Use the alternatives mechanism for RDMSR
x86/alternatives: Add ALTERNATIVE_4()
x86/paravirt: Split off MSR related hooks into new header
x86/paravirt: Prepare support of MSR instruction interfaces
x86/paravirt: Switch MSR access pv_ops functions to instruction
interfaces
x86/msr: Reduce number of low level MSR access helpers
x86/paravirt: Use alternatives for MSR access with paravirt
arch/x86/coco/sev/internal.h | 7 +-
arch/x86/coco/tdx/tdx.c | 8 +-
arch/x86/hyperv/ivm.c | 2 +-
arch/x86/include/asm/alternative.h | 6 +
arch/x86/include/asm/fred.h | 2 +-
arch/x86/include/asm/kvm_host.h | 10 -
arch/x86/include/asm/msr.h | 345 ++++++++++++++++------
arch/x86/include/asm/paravirt-msr.h | 148 ++++++++++
arch/x86/include/asm/paravirt.h | 67 -----
arch/x86/include/asm/paravirt_types.h | 57 ++--
arch/x86/include/asm/qspinlock_paravirt.h | 4 +-
arch/x86/kernel/alternative.c | 5 +-
arch/x86/kernel/cpu/mshyperv.c | 7 +-
arch/x86/kernel/kvmclock.c | 2 +-
arch/x86/kernel/paravirt.c | 42 ++-
arch/x86/kvm/svm/svm.c | 16 +-
arch/x86/kvm/vmx/tdx.c | 2 +-
arch/x86/kvm/vmx/vmx.c | 8 +-
arch/x86/lib/x86-opcode-map.txt | 5 +-
arch/x86/mm/extable.c | 35 ++-
arch/x86/xen/enlighten_pv.c | 52 +++-
arch/x86/xen/pmu.c | 4 +-
tools/arch/x86/lib/x86-opcode-map.txt | 5 +-
tools/objtool/check.c | 1 +
24 files changed, 576 insertions(+), 264 deletions(-)
create mode 100644 arch/x86/include/asm/paravirt-msr.h
--
2.53.0
^ permalink raw reply
* Re: [PATCH 5/6] x86/sev: Use configfs to re-enable RMP optimizations.
From: Kalra, Ashish @ 2026-02-18 4:39 UTC (permalink / raw)
To: Dave Hansen, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <e72165ed-c65d-4d21-bff6-9981b46311cf@amd.com>
On 2/17/2026 9:34 PM, Kalra, Ashish wrote:
> Hello Dave,
>
> On 2/17/2026 4:19 PM, Dave Hansen wrote:
>> On 2/17/26 12:11, Ashish Kalra wrote:
>>> From: Ashish Kalra <ashish.kalra@amd.com>
>>>
>>> Use configfs as an interface to re-enable RMP optimizations at runtime
>>>
>>> When SNP guests are launched, RMPUPDATE disables the corresponding
>>> RMPOPT optimizations. Therefore, an interface is required to manually
>>> re-enable RMP optimizations, as no mechanism currently exists to do so
>>> during SNP guest cleanup.
>>
>> Is this like a proof-of-concept to poke the hardware and show it works?
>> Or, is this intended to be the way that folks actually interact with
>> SEV-SNP optimization in real production scenarios?
>>
>> Shouldn't freeing SEV-SNP memory back to the system do this
>> automatically? Worst case, keep a 1-bit-per-GB bitmap of memory that's
>> been freed and schedule_work() to run in 1 or 10 or 100 seconds. That
>> should batch things up nicely enough. No?
And there is a cost associated with re-enabling the optimizations for all
system RAM (even though it runs as a background kernel thread executing RMPOPT
on different 1GB regions in parallel and with inline cond_resched()'s),
we don't want to run this periodically.
In case of running SNP guests, this scheduled/periodic run will conflict with
RMPUPDATE(s) being executed for assigning the guest pages and marking them as private.
Even though the hardware takes care of handling such race conditions where
one CPU is doing RMPOPT on it while another is changing one of the pages in that
region to be assigned via RMPUPDATE. In this case, the hardware ensures that after
the RMPUPDATE completes, the CPU that did RMPOPT will see the region as un-optimized.
Once 1GB hugetlb support (for guest_memfd) has been merged, however it will be
straightforward to plumb it into the 1GB hugetlb cleanup path.
Thanks,
Ashish
>
> Actually, the RMPOPT implementation is going to be a multi-phased development.
>
> In the first phase (which is this patch-series) we enable RMPOPT globally, and let RMPUPDATE(s)
> slowly switch it off over time as SNP guest spin up, and then in phase#2 once 1GB hugetlb is in place,
> we enable re-issuing of RMPOPT during 1GB page cleanup.
>
> So automatic re-issuing of RMPOPT will be done when SNP guests are shutdown and as part of
> SNP guest cleanup once 1GB hugetlb support (for guest_memfd) has been merged.
>
> As currently, i.e, as part of this patch series, there is no mechanism to re-issue RMPOPT
> automatically as part of SNP guest cleanup, therefore this support exists to doing it
> manually at runtime via configfs.
>
> I will describe this multi-phased RMPOPT implementation plan in the cover letter for
> next revision of this patch series.
>
>
>>
>> I can't fathom that users don't want this to be done automatically for them.
>>
>> Is the optimization scan really expensive or something? 1GB of memory
>> should have a small number of megabytes of metadata to scan.
^ permalink raw reply
* Re: [PATCH 0/6] Add RMPOPT support.
From: Kalra, Ashish @ 2026-02-18 4:12 UTC (permalink / raw)
To: Dave Hansen, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <9c77e206-442d-4891-bb29-295bc8bffe20@intel.com>
Hello Dave,
On 2/17/2026 4:11 PM, Dave Hansen wrote:
> On 2/17/26 12:09, Ashish Kalra wrote:
>> RMPOPT is a new instruction designed to minimize the performance
>> overhead of RMP checks for the hypervisor and non-SNP guests.
>
> This needs a little theory of operation for the new instruction. It
> seems like it will enable optimizations all by itself. You just call it,
> and it figures out when the CPU can optimize things. The CPU also
> figures out when the optimization must be flipped off.
Yes, i will add more theory of operation for the new instruction.
RMPOPT instruction with the verify and report status operation, in this operation
the CPU will read the RMP contents, verify the entire 1GB region starting
at the provided SPA is HV-owned. For the entire 1GB region it checks that all RMP
entries in this region are HV-owned (i.e, not in assigned state) and then
accordingly update the RMPOPT table to indicate if optimization has been enabled
and provide indication to software if the optimization was successful.
RMPUPDATE instruction that mark new pages as assigned will automatically clear the
optimizations and the appropriate bit in the RMPOPT table.
The RMPOPT table is managed by a combination of software and hardware. Software uses
the RMPOPT instruction to set bits in the table, indicating that regions of memory are
entirely HV-owned. Hardware automatically clears bits in the RMPOPT table when RMP contents
are changed during RMPUPDATE instruction.
>
> That's not awful.
>
> To be honest, though, I think this is misdesigned. Shouldn't the CPU
> *boot* in a state where it is optimized? Why should software have to
> tell it that coming out of reset, there is no SEV-SNP memory?
When the CPU boots, the RMP checks are not done and therefore the CPU
is booting in a state where it is optimized.
The RMP checks are not enabled till SEV-SNP is enabled and SNP is enabled
during kernel boot (as part of iommu_snp_enable() -> snp_rmptable_init()).
Once SNP is enabled as part of kernel boot, hypervisor and non-SNP guests are
subject to RMP checks on writes to provide integrity of SEV-SNP guest memory.
Therefore, we need to enable these RMP optimizations after SNP has been
enabled to indicate which 1GB regions of memory are known to not contain any
SEV-SNP guest memory.
I will add the above details to the cover letter for the next revision of this
patch series.
Thanks,
Ashish
^ permalink raw reply
* Re: [PATCH 5/6] x86/sev: Use configfs to re-enable RMP optimizations.
From: Kalra, Ashish @ 2026-02-18 3:34 UTC (permalink / raw)
To: Dave Hansen, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <21250a3e-536c-4348-bf4c-a7356a13939b@intel.com>
Hello Dave,
On 2/17/2026 4:19 PM, Dave Hansen wrote:
> On 2/17/26 12:11, Ashish Kalra wrote:
>> From: Ashish Kalra <ashish.kalra@amd.com>
>>
>> Use configfs as an interface to re-enable RMP optimizations at runtime
>>
>> When SNP guests are launched, RMPUPDATE disables the corresponding
>> RMPOPT optimizations. Therefore, an interface is required to manually
>> re-enable RMP optimizations, as no mechanism currently exists to do so
>> during SNP guest cleanup.
>
> Is this like a proof-of-concept to poke the hardware and show it works?
> Or, is this intended to be the way that folks actually interact with
> SEV-SNP optimization in real production scenarios?
>
> Shouldn't freeing SEV-SNP memory back to the system do this
> automatically? Worst case, keep a 1-bit-per-GB bitmap of memory that's
> been freed and schedule_work() to run in 1 or 10 or 100 seconds. That
> should batch things up nicely enough. No?
Actually, the RMPOPT implementation is going to be a multi-phased development.
In the first phase (which is this patch-series) we enable RMPOPT globally, and let RMPUPDATE(s)
slowly switch it off over time as SNP guest spin up, and then in phase#2 once 1GB hugetlb is in place,
we enable re-issuing of RMPOPT during 1GB page cleanup.
So automatic re-issuing of RMPOPT will be done when SNP guests are shutdown and as part of
SNP guest cleanup once 1GB hugetlb support (for guest_memfd) has been merged.
As currently, i.e, as part of this patch series, there is no mechanism to re-issue RMPOPT
automatically as part of SNP guest cleanup, therefore this support exists to doing it
manually at runtime via configfs.
I will describe this multi-phased RMPOPT implementation plan in the cover letter for
next revision of this patch series.
Thanks,
Ashish
>
> I can't fathom that users don't want this to be done automatically for them.
>
> Is the optimization scan really expensive or something? 1GB of memory
> should have a small number of megabytes of metadata to scan.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox