* Re: [PATCH v2 1/7] x86/cpufeatures: Add X86_FEATURE_AMD_RMPOPT feature flag
From: Borislav Petkov @ 2026-03-05 12:36 UTC (permalink / raw)
To: Ashish Kalra, Sean Christopherson
Cc: tglx, mingo, dave.hansen, x86, hpa, peterz, thomas.lendacky,
herbert, davem, ardb, pbonzini, aik, Michael.Roth, KPrateek.Nayak,
Tycho.Andersen, Nathan.Fontenot, jackyli, pgonda, rientjes,
jacobhxu, xin, pawan.kumar.gupta, babu.moger, dyoung, nikunj,
john.allen, darwi, linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <219ebbd57ac1d99fc5ea055431f7a8396021c2c2.1772486459.git.ashish.kalra@amd.com>
On Mon, Mar 02, 2026 at 09:35:19PM +0000, Ashish Kalra wrote:
> From: Ashish Kalra <ashish.kalra@amd.com>
>
> Add a flag indicating whether RMPOPT instruction is supported.
>
> RMPOPT is a new instruction designed to minimize the performance
> overhead of RMP checks on the hypervisor and on non-SNP guests by
> allowing RMP checks to be skipped when 1G regions of memory are known
> not to contain any SEV-SNP guest memory.
>
> For more information on the RMPOPT instruction, see the AMD64 RMPOPT
> technical documentation. [1]
>
> Link: https://docs.amd.com/v/u/en-US/69201_1.00_AMD64_RMPOPT_PUB [1]
Please do not add URLs to documents on corporate sites because latter change
notoriously fast, resulting in dead links. Instead, quote the document title
so that anyone looking for it, can find it after a web search engine has
indexed it.
> Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
> ---
> arch/x86/include/asm/cpufeatures.h | 2 +-
> arch/x86/kernel/cpu/scattered.c | 1 +
> 2 files changed, 2 insertions(+), 1 deletion(-)
Btw, looking further in the set, the first several patches are for tip and
then KVM ones come.
I'm thinking, when the time comes, I'll give you, Sean, an immutable branch
which you can merge.
Right?
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply
* Re: [PATCH net-next v3 0/2] dma-buf: heaps: system: add an option to allocate explicitly decrypted memory
From: Jiri Pirko @ 2026-03-05 12:40 UTC (permalink / raw)
To: dri-devel, linaro-mm-sig, iommu, linux-media
Cc: sumit.semwal, benjamin.gaignard, Brian.Starkey, jstultz,
tjmercier, christian.koenig, m.szyprowski, robin.murphy, jgg,
leon, sean.anderson, ptesarik, catalin.marinas, aneesh.kumar,
suzuki.poulose, steven.price, thomas.lendacky, john.allen,
ashish.kalra, suravee.suthikulpanit, linux-coco
In-Reply-To: <20260305123641.164164-1-jiri@resnulli.us>
The "net-next" in [PATCH] brackets is obviously incorrect, ignore
this bad string please.
^ permalink raw reply
* Re: [PATCH v2] KVM: x86: synthesize CPUID bits only if CPU capability is set
From: Sean Christopherson @ 2026-03-05 17:07 UTC (permalink / raw)
To: Sean Christopherson, bp, kvm, Carlos López
Cc: linux-coco, jmattson, binbin.wu, Paolo Bonzini, Thomas Gleixner,
Ingo Molnar, Dave Hansen, x86, H. Peter Anvin, linux-kernel
In-Reply-To: <20260209153108.70667-2-clopez@suse.de>
On Mon, 09 Feb 2026 16:31:09 +0100, Carlos López wrote:
> KVM incorrectly synthesizes CPUID bits for KVM-only leaves, as the
> following branch in kvm_cpu_cap_init() is never taken:
>
> if (leaf < NCAPINTS)
> kvm_cpu_caps[leaf] &= kernel_cpu_caps[leaf];
>
> This means that bits set via SYNTHESIZED_F() for KVM-only leaves are
> unconditionally set. This for example can cause issues for SEV-SNP
> guests running on Family 19h CPUs, as TSA_SQ_NO and TSA_L1_NO are
> always enabled by KVM in 80000021[ECX]. When userspace issues a
> SNP_LAUNCH_UPDATE command to update the CPUID page for the guest, SNP
> firmware will explicitly reject the command if the page sets sets these
> bits on vulnerable CPUs.
>
> [...]
Applied to kvm-x86 fixes, thanks!
[1/1] KVM: x86: synthesize CPUID bits only if CPU capability is set
https://github.com/kvm-x86/linux/commit/6a5028d8f9f4
--
https://github.com/kvm-x86/linux/tree/next
^ permalink raw reply
* Re: [PATCH 00/14] KVM: x86: Emulator MMIO fix and cleanups
From: Sean Christopherson @ 2026-03-05 17:07 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
On Tue, 24 Feb 2026 17:20:35 -0800, Sean Christopherson wrote:
> Fix a UAF stack bug where KVM references a stack pointer around an exit to
> userspace, and then clean up the related code to try to make it easier to
> maintain (not necessarily "easy", but "easier").
>
> The SEV-ES and TDX changes are compile-tested only.
>
> Sean Christopherson (14):
> KVM: x86: Use scratch field in MMIO fragment to hold small write
> values
> KVM: x86: Open code handling of completed MMIO reads in
> emulator_read_write()
> KVM: x86: Trace unsatisfied MMIO reads on a per-page basis
> KVM: x86: Use local MMIO fragment variable to clean up
> emulator_read_write()
> KVM: x86: Open code read vs. write userspace MMIO exits in
> emulator_read_write()
> KVM: x86: Move MMIO write tracing into vcpu_mmio_write()
> KVM: x86: Harden SEV-ES MMIO against on-stack use-after-free
> KVM: x86: Dedup kvm_sev_es_mmio_{read,write}()
> KVM: x86: Consolidate SEV-ES MMIO emulation into a single public API
> KVM: x86: Bury emulator read/write ops in
> emulator_{read,write}_emulated()
> KVM: x86: Fold emulator_write_phys() into write_emulate()
> KVM: x86: Rename .read_write_emulate() to .read_write_guest()
> KVM: x86: Don't panic the kernel if completing userspace I/O / MMIO
> goes sideways
> KVM: x86: Add helpers to prepare kvm_run for userspace MMIO exit
>
> [...]
Applied to kvm-x86 mmio, with the hardened version of the helper in patch 14.
Thanks for the testing!
[01/14] KVM: x86: Use scratch field in MMIO fragment to hold small write values
https://github.com/kvm-x86/linux/commit/0b16e69d17d8
[02/14] KVM: x86: Open code handling of completed MMIO reads in emulator_read_write()
https://github.com/kvm-x86/linux/commit/4046823e78b0
[03/14] KVM: x86: Trace unsatisfied MMIO reads on a per-page basis
https://github.com/kvm-x86/linux/commit/4f11fded5381
[04/14] KVM: x86: Use local MMIO fragment variable to clean up emulator_read_write()
https://github.com/kvm-x86/linux/commit/523b6269f700
[05/14] KVM: x86: Open code read vs. write userspace MMIO exits in emulator_read_write()
https://github.com/kvm-x86/linux/commit/cbbf8228c071
[06/14] KVM: x86: Move MMIO write tracing into vcpu_mmio_write()
https://github.com/kvm-x86/linux/commit/72f36f99072c
[07/14] KVM: x86: Harden SEV-ES MMIO against on-stack use-after-free
https://github.com/kvm-x86/linux/commit/144089f5c394
[08/14] KVM: x86: Dedup kvm_sev_es_mmio_{read,write}()
https://github.com/kvm-x86/linux/commit/33e09e2f9735
[09/14] KVM: x86: Consolidate SEV-ES MMIO emulation into a single public API
https://github.com/kvm-x86/linux/commit/326e810eaaa5
[10/14] KVM: x86: Bury emulator read/write ops in emulator_{read,write}_emulated()
https://github.com/kvm-x86/linux/commit/3517193ef9c2
[11/14] KVM: x86: Fold emulator_write_phys() into write_emulate()
https://github.com/kvm-x86/linux/commit/929613b3cd1a
[12/14] KVM: x86: Rename .read_write_emulate() to .read_write_guest()
https://github.com/kvm-x86/linux/commit/216729846603
[13/14] KVM: x86: Don't panic the kernel if completing userspace I/O / MMIO goes sideways
https://github.com/kvm-x86/linux/commit/4f09e62afcd6
[14/14] KVM: x86: Add helpers to prepare kvm_run for userspace MMIO exit
https://github.com/kvm-x86/linux/commit/e2138c4a5be1
--
https://github.com/kvm-x86/linux/tree/next
^ permalink raw reply
* Re: [PATCH v3 00/16] KVM: x86/tdx: Have TDX handle VMXON during bringup
From: Sean Christopherson @ 2026-03-05 17:08 UTC (permalink / raw)
To: Sean Christopherson, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, x86, Kiryl Shutsemau,
Peter Zijlstra, Arnaldo Carvalho de Melo, Namhyung Kim,
Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, linux-perf-users, Chao Gao,
Xu Yilun, Dan Williams
In-Reply-To: <20260214012702.2368778-1-seanjc@google.com>
On Fri, 13 Feb 2026 17:26:46 -0800, Sean Christopherson wrote:
> Assuming I didn't break anything between v2 and v3, I think this is ready to
> rip. Given the scope of the KVM changes, and that they extend outside of x86,
> my preference is to take this through the KVM tree. But a stable topic branch
> in tip would work too, though I think we'd want it sooner than later so that
> it can be used as a base.
>
> Chao, I deliberately omitted your Tested-by, as I shuffled things around enough
> while splitting up the main patch that I'm not 100% positive I didn't regress
> anything relative to v2.
>
> [...]
Applied to kvm-x86 vmxon, with the minor fixups. I'll make sure not to touch
the hashes at this point, but holler if anyone wants an "official" stable tag.
[01/16] KVM: x86: Move kvm_rebooting to x86
https://github.com/kvm-x86/linux/commit/4059172b2a78
[02/16] KVM: VMX: Move architectural "vmcs" and "vmcs_hdr" structures to public vmx.h
https://github.com/kvm-x86/linux/commit/3c75e6a5da3c
[03/16] KVM: x86: Move "kvm_rebooting" to kernel as "virt_rebooting"
https://github.com/kvm-x86/linux/commit/a1450a8156c6
[04/16] KVM: VMX: Unconditionally allocate root VMCSes during boot CPU bringup
https://github.com/kvm-x86/linux/commit/405b7c27934e
[05/16] x86/virt: Force-clear X86_FEATURE_VMX if configuring root VMCS fails
https://github.com/kvm-x86/linux/commit/95e4adb24ff6
[06/16] KVM: VMX: Move core VMXON enablement to kernel
https://github.com/kvm-x86/linux/commit/920da4f75519
[07/16] KVM: SVM: Move core EFER.SVME enablement to kernel
https://github.com/kvm-x86/linux/commit/32d76cdfa122
[08/16] KVM: x86: Move bulk of emergency virtualizaton logic to virt subsystem
https://github.com/kvm-x86/linux/commit/428afac5a8ea
[09/16] x86/virt: Add refcounting of VMX/SVM usage to support multiple in-kernel users
https://github.com/kvm-x86/linux/commit/8528a7f9c91d
[10/16] x86/virt/tdx: Drop the outdated requirement that TDX be enabled in IRQ context
https://github.com/kvm-x86/linux/commit/0efe5dc16169
[11/16] KVM: x86/tdx: Do VMXON and TDX-Module initialization during subsys init
https://github.com/kvm-x86/linux/commit/165e77353831
[12/16] x86/virt/tdx: Tag a pile of functions as __init, and globals as __ro_after_init
https://github.com/kvm-x86/linux/commit/9900400e20c0
[13/16] x86/virt/tdx: KVM: Consolidate TDX CPU hotplug handling
https://github.com/kvm-x86/linux/commit/eac90a5ba0aa
[14/16] x86/virt/tdx: Use ida_is_empty() to detect if any TDs may be running
https://github.com/kvm-x86/linux/commit/afe31de159bf
[15/16] KVM: Bury kvm_{en,dis}able_virtualization() in kvm_main.c once more
https://github.com/kvm-x86/linux/commit/d30372d0b7e6
[16/16] KVM: TDX: Fold tdx_bringup() into tdx_hardware_setup()
https://github.com/kvm-x86/linux/commit/f630de1f8d70
--
https://github.com/kvm-x86/linux/tree/next
^ permalink raw reply
* Re: [PATCH v3 00/16] KVM: x86/tdx: Have TDX handle VMXON during bringup
From: dan.j.williams @ 2026-03-05 18:50 UTC (permalink / raw)
To: Sean Christopherson, Sean Christopherson, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Dave Hansen, x86, Kiryl Shutsemau,
Peter Zijlstra, Arnaldo Carvalho de Melo, Namhyung Kim,
Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, linux-perf-users, Chao Gao,
Xu Yilun, Dan Williams
In-Reply-To: <177272960351.1566277.2741684808536756847.b4-ty@google.com>
Sean Christopherson wrote:
> On Fri, 13 Feb 2026 17:26:46 -0800, Sean Christopherson wrote:
> > Assuming I didn't break anything between v2 and v3, I think this is ready to
> > rip. Given the scope of the KVM changes, and that they extend outside of x86,
> > my preference is to take this through the KVM tree. But a stable topic branch
> > in tip would work too, though I think we'd want it sooner than later so that
> > it can be used as a base.
> >
> > Chao, I deliberately omitted your Tested-by, as I shuffled things around enough
> > while splitting up the main patch that I'm not 100% positive I didn't regress
> > anything relative to v2.
> >
> > [...]
>
> Applied to kvm-x86 vmxon, with the minor fixups. I'll make sure not to touch
> the hashes at this point, but holler if anyone wants an "official" stable tag.
Thanks, Sean!
Please do make an official stable tag that I can use for coordinating
the initial TDX Connect enabling series. While there is no strict
dependency I do not want it to be the case that a bisect of TDX Connect
bounces between a world where you need to load kvm_intel before the PCI
layer can do link encryption operations and keep it loaded etc.
My proposal, unless you or Dave holler, is to take the first round of
TDX Connect enabling through the tsm.git tree with acks. This round does
not have kvm entanglements, i.e. IOMMU coordination and device
assignment come later. It also does not have much in the way of core x86
entanglements beyond new seamcall exports.
^ permalink raw reply
* Re: [PATCH v3 00/16] KVM: x86/tdx: Have TDX handle VMXON during bringup
From: Dave Hansen @ 2026-03-05 18:54 UTC (permalink / raw)
To: dan.j.williams, Sean Christopherson, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, x86, Kiryl Shutsemau,
Peter Zijlstra, Arnaldo Carvalho de Melo, Namhyung Kim,
Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, linux-perf-users, Chao Gao,
Xu Yilun
In-Reply-To: <69a9d0645bc31_6423c1006@dwillia2-mobl4.notmuch>
On 3/5/26 10:50, dan.j.williams@intel.com wrote:
> My proposal, unless you or Dave holler, is to take the first round of
> TDX Connect enabling through the tsm.git tree with acks. This round does
> not have kvm entanglements, i.e. IOMMU coordination and device
> assignment come later. It also does not have much in the way of core x86
> entanglements beyond new seamcall exports.
Sounds sane to me.
^ permalink raw reply
* Re: [PATCH v3 00/16] KVM: x86/tdx: Have TDX handle VMXON during bringup
From: Sean Christopherson @ 2026-03-05 19:07 UTC (permalink / raw)
To: Dave Hansen
Cc: dan.j.williams, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, Kiryl Shutsemau, Peter Zijlstra,
Arnaldo Carvalho de Melo, Namhyung Kim, Paolo Bonzini,
linux-kernel, linux-coco, kvm, linux-perf-users, Chao Gao,
Xu Yilun
In-Reply-To: <00406192-932a-4cce-a579-48fe18b9f777@intel.com>
On Thu, Mar 05, 2026, Dave Hansen wrote:
> On 3/5/26 10:50, dan.j.williams@intel.com wrote:
> > My proposal, unless you or Dave holler, is to take the first round of
> > TDX Connect enabling through the tsm.git tree with acks. This round does
> > not have kvm entanglements, i.e. IOMMU coordination and device
> > assignment come later. It also does not have much in the way of core x86
> > entanglements beyond new seamcall exports.
>
> Sounds sane to me.
+1. If there aren't any KVM changes, ignorance is bliss :-)
^ permalink raw reply
* Re: [PATCH v3 00/16] KVM: x86/tdx: Have TDX handle VMXON during bringup
From: Sean Christopherson @ 2026-03-05 19:08 UTC (permalink / raw)
To: dan.j.williams
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Peter Zijlstra, Arnaldo Carvalho de Melo,
Namhyung Kim, Paolo Bonzini, linux-kernel, linux-coco, kvm,
linux-perf-users, Chao Gao, Xu Yilun
In-Reply-To: <69a9d0645bc31_6423c1006@dwillia2-mobl4.notmuch>
On Thu, Mar 05, 2026, dan.j.williams@intel.com wrote:
> Sean Christopherson wrote:
> > On Fri, 13 Feb 2026 17:26:46 -0800, Sean Christopherson wrote:
> > > Assuming I didn't break anything between v2 and v3, I think this is ready to
> > > rip. Given the scope of the KVM changes, and that they extend outside of x86,
> > > my preference is to take this through the KVM tree. But a stable topic branch
> > > in tip would work too, though I think we'd want it sooner than later so that
> > > it can be used as a base.
> > >
> > > Chao, I deliberately omitted your Tested-by, as I shuffled things around enough
> > > while splitting up the main patch that I'm not 100% positive I didn't regress
> > > anything relative to v2.
> > >
> > > [...]
> >
> > Applied to kvm-x86 vmxon, with the minor fixups. I'll make sure not to touch
> > the hashes at this point, but holler if anyone wants an "official" stable tag.
>
> Thanks, Sean!
>
> Please do make an official stable tag that I can use for coordinating
> the initial TDX Connect enabling series. While there is no strict
> dependency I do not want it to be the case that a bisect of TDX Connect
> bounces between a world where you need to load kvm_intel before the PCI
> layer can do link encryption operations and keep it loaded etc.
With a timestamp, in case fixups on top are needed:
kvm-x86-vmxon-2026.03.05
^ permalink raw reply
* Re: [PATCH v2 3/7] x86/sev: add support for RMPOPT instruction
From: Kalra, Ashish @ 2026-03-05 19:22 UTC (permalink / raw)
To: Dave Hansen, Sean Christopherson
Cc: tglx, mingo, bp, dave.hansen, x86, hpa, peterz, thomas.lendacky,
herbert, davem, ardb, pbonzini, aik, Michael.Roth, KPrateek.Nayak,
Tycho.Andersen, Nathan.Fontenot, jackyli, pgonda, rientjes,
jacobhxu, xin, pawan.kumar.gupta, babu.moger, dyoung, nikunj,
john.allen, darwi, linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <0fbb94ad-bfcf-4fbe-bf40-d79051d67ad8@amd.com>
An update on performance data:
>
> RMPOPT after SNP guest shutdown:
> ...
> [ 298.746893] SEV-SNP: RMPOPT max. CPU cycles 248083620
> [ 298.746898] SEV-SNP: RMPOPT min. CPU cycles 60
> [ 298.746900] SEV-SNP: RMPOPT average cycles 127859
>
>
A single RMPOPT instruction should not be taking 248M cycles, so i looked at
my performance measurement code :
I was not disabling interrupts around my measurement code, so probably this
measurement code was getting interrupted/preempted and causing this discrepancy:
I am now measuring with interrupts disabled around this code:
static void rmpopt(void *val)
{
bool optimized;
u64 start, end;
local_irq_disable();
start = rdtsc_ordered();
asm volatile(".byte 0xf2, 0x0f, 0x01, 0xfc"
: "=@ccc" (optimized)
: "a" ((u64)val & PUD_MASK), "c" ((u64)val & 0x1)
: "memory", "cc");
end = rdtsc_ordered();
local_irq_enable();
total_cycles += (end - start);
++iteration;
if ((end - start) > largest_cycle_rmpopt) {
pr_info("RMPOPT max cycle on cpu %d, addr 0x%llx, cycles %llu, prev largest %llu\n",
smp_processor_id(), ((u64)val & PUD_MASK), end - start, largest_cycle_rmpopt);
largest_cycle_rmpopt = end - start;
}
...
...
But, the following is interesting, if I invoke rmpopt() using smp_call_on_cpu() which issues
RMPOPT on each CPU serially compared to using on_each_cpu_mask() above which will execute rmpopt()
function and RMPOPT instruction in parallel on multiple CPUs (by sending IPIs in parallel),
I observe a significant difference and improvement in "individual" RMPOPT instruction performance:
rmpopt() executing serially using smp_call_on_cpu():
[ 244.518677] SEV-SNP: RMPOPT instruction cycles 3300
[ 244.518716] SEV-SNP: RMPOPT instruction cycles 2840
[ 244.518758] SEV-SNP: RMPOPT instruction cycles 3260
[ 244.518800] SEV-SNP: RMPOPT instruction cycles 3640
[ 244.518838] SEV-SNP: RMPOPT instruction cycles 1980
[ 244.518878] SEV-SNP: RMPOPT instruction cycles 3420
[ 244.518919] SEV-SNP: RMPOPT instruction cycles 3620
[ 244.518958] SEV-SNP: RMPOPT instruction cycles 3120
[ 244.518997] SEV-SNP: RMPOPT instruction cycles 2160
[ 244.519038] SEV-SNP: RMPOPT instruction cycles 3040
[ 244.519078] SEV-SNP: RMPOPT instruction cycles 3700
[ 244.519119] SEV-SNP: RMPOPT instruction cycles 3960
[ 244.519158] SEV-SNP: RMPOPT instruction cycles 3420
[ 244.519211] SEV-SNP: RMPOPT instruction cycles 5080
[ 244.519254] SEV-SNP: RMPOPT instruction cycles 3000
[ 244.519295] SEV-SNP: RMPOPT instruction cycles 3420
[ 244.527150] SEV-SNP: RMPOPT max cycle on cpu 256, addr 0x40000000, cycles 34680, prev largest 22100
[ 244.529622] SEV-SNP: RMPOPT max cycle on cpu 320, addr 0x40000000, cycles 36800, prev largest 34680
[ 244.559314] SEV-SNP: RMPOPT max cycle on cpu 256, addr 0x80000000, cycles 39740, prev largest 36800
[ 244.561718] SEV-SNP: RMPOPT max cycle on cpu 320, addr 0x80000000, cycles 41840, prev largest 39740
[ 244.562837] SEV-SNP: RMPOPT max cycle on cpu 352, addr 0x80000000, cycles 42160, prev largest 41840
[ 244.886705] SEV-SNP: RMPOPT max cycle on cpu 384, addr 0x300000000, cycles 42300, prev largest 42160
[ 247.701377] SEV-SNP: RMPOPT max cycle on cpu 384, addr 0x1980000000, cycles 42400, prev largest 42300
[ 250.322355] SEV-SNP: RMPOPT max cycle on cpu 384, addr 0x2ec0000000, cycles 42420, prev largest 42400
[ 250.755457] SEV-SNP: RMPOPT max cycle on cpu 384, addr 0x3240000000, cycles 42540, prev largest 42420
[ 264.271293] SEV-SNP: RMPOPT max cycle on cpu 32, addr 0xa040000000, cycles 50400, prev largest 42540
[ 264.333739] SEV-SNP: RMPOPT max cycle on cpu 32, addr 0xa0c0000000, cycles 50940, prev largest 50400
[ 264.395521] SEV-SNP: RMPOPT max cycle on cpu 32, addr 0xa140000000, cycles 51240, prev largest 50940
[ 264.733133] SEV-SNP: RMPOPT max cycle on cpu 32, addr 0xa400000000, cycles 51480, prev largest 51240
[ 269.500891] SEV-SNP: RMPOPT max cycle on cpu 0, addr 0xcac0000000, cycles 66080, prev largest 51480
[ 273.507009] SEV-SNP: RMPOPT max cycle on cpu 320, addr 0xeb40000000, cycles 83680, prev largest 66080
[ 276.435091] SEV-SNP: RMPOPT largest cycles 83680
[ 276.435096] SEV-SNP: RMPOPT smallest cycles 60
[ 276.435097] SEV-SNP: RMPOPT average cycles 5658
[ 276.435098] SEV-SNP: RMPOPT cycles taken for physical address range 0x0000000000000000 - 0x0000010380000000 on all cpus 63815935380 cycles
Compare this to executing rmpopt() in parallel:
[ 1238.809183] SEV-SNP: RMPOPT average cycles 114372
So, looks like executing RMPOPT in parallel is causing performance degradation, which we will investigate.
But, these are the performance numbers you should be considering :
RMPOPT during boot:
[ 49.913402] SEV-SNP: RMPOPT largest cycles 1143020
[ 49.913407] SEV-SNP: RMPOPT smallest cycles 60
[ 49.913408] SEV-SNP: RMPOPT average cycles 5226
RMPOPT after SNP guest shutdown:
[ 276.435091] SEV-SNP: RMPOPT largest cycles 83680
[ 276.435096] SEV-SNP: RMPOPT smallest cycles 60
[ 276.435097] SEV-SNP: RMPOPT average cycles 5658
Thanks,
Ashish
^ permalink raw reply
* Re: [PATCH v2 3/7] x86/sev: add support for RMPOPT instruction
From: Dave Hansen @ 2026-03-05 19:40 UTC (permalink / raw)
To: Kalra, Ashish, Sean Christopherson
Cc: tglx, mingo, bp, dave.hansen, x86, hpa, peterz, thomas.lendacky,
herbert, davem, ardb, pbonzini, aik, Michael.Roth, KPrateek.Nayak,
Tycho.Andersen, Nathan.Fontenot, jackyli, pgonda, rientjes,
jacobhxu, xin, pawan.kumar.gupta, babu.moger, dyoung, nikunj,
john.allen, darwi, linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <6a4f4ecf-ffc0-43a9-98d4-06235b42063e@amd.com>
On 3/5/26 11:22, Kalra, Ashish wrote:
> But, these are the performance numbers you should be considering :
>
> RMPOPT during boot:
>
> [ 49.913402] SEV-SNP: RMPOPT largest cycles 1143020
> [ 49.913407] SEV-SNP: RMPOPT smallest cycles 60
> [ 49.913408] SEV-SNP: RMPOPT average cycles 5226
>
> RMPOPT after SNP guest shutdown:
>
> [ 276.435091] SEV-SNP: RMPOPT largest cycles 83680
> [ 276.435096] SEV-SNP: RMPOPT smallest cycles 60
> [ 276.435097] SEV-SNP: RMPOPT average cycles 5658
First of all, I'd really appreciate wall clock measurements on these.
It's just less math and guesswork. Cycles are easy to measure but hard
to read. Please make these easier to read. Also, the per-RMPOPT numbers
don't mean much. You have to scale it by the number of CPUs and memory
(or 2TB) to get to a real, useful number.
The thing that matters is how long this loop takes:
for (pa = pa_start; pa < pa_end; pa += PUD_SIZE)
and *especially* how long it takes per-cpu and when the system has a
full 2TB load of memory.
That will tell us how many resources this RMPOPT thing is going to take,
which is the _real_ thing we need to know.
Also, to some degree, the thing we care about here the *most* is the
worst case scenario. I think the worst possible case is that there's one
4k private page in each 1GB of memory, and that it's the last 4k page.
I'd like to see numbers for something close to *that*, not when there
are no private pages.
The two things you measured above are interesting, but they're only part
of the story.
^ permalink raw reply
* [PATCH v4 0/2] Extend KVM_HC_MAP_GPA_RANGE api to allow retry
From: Sagi Shahar @ 2026-03-05 22:26 UTC (permalink / raw)
To: Vishal Annapurve, Sean Christopherson, Paolo Bonzini, Dave Hansen,
Kiryl Shutsemau, Rick Edgecombe
Cc: Thomas Gleixner, Borislav Petkov, H. Peter Anvin, Michael Roth,
Tom Lendacky, x86, kvm, linux-kernel, linux-coco, Sagi Shahar
In some cases, userspace might decide to split MAP_GPA requests and
retry them the next time the guest runs. One common case is MAP_GPA
requests received right before intrahost migration when userspace
might decide to complete the request after the migration is complete
to reduce blackout time.
This is v4 of the series.
Changes from v3[1]:
* Rebased on top of v7.0-rc2.
* Switch "if" statement to switch-case in tdx_complete_vmcall_map_gpa()
as suggested by Michael Roth.
[1] https://lore.kernel.org/lkml/20260206222829.3758171-1-sagis@google.com/
Sagi Shahar (1):
KVM: SEV: Restrict userspace return codes for KVM_HC_MAP_GPA_RANGE
Vishal Annapurve (1):
KVM: TDX: Allow userspace to return errors to guest for MAPGPA
Documentation/virt/kvm/api.rst | 3 +++
arch/x86/kvm/svm/sev.c | 12 ++++++++++--
arch/x86/kvm/vmx/tdx.c | 28 +++++++++++++++++++++-------
arch/x86/kvm/x86.h | 6 ++++++
4 files changed, 40 insertions(+), 9 deletions(-)
--
2.53.0.473.g4a7958ca14-goog
^ permalink raw reply
* [PATCH v4 1/2] KVM: TDX: Allow userspace to return errors to guest for MAPGPA
From: Sagi Shahar @ 2026-03-05 22:26 UTC (permalink / raw)
To: Vishal Annapurve, Sean Christopherson, Paolo Bonzini, Dave Hansen,
Kiryl Shutsemau, Rick Edgecombe
Cc: Thomas Gleixner, Borislav Petkov, H. Peter Anvin, Michael Roth,
Tom Lendacky, x86, kvm, linux-kernel, linux-coco, Sagi Shahar
In-Reply-To: <20260305222627.4193305-1-sagis@google.com>
From: Vishal Annapurve <vannapurve@google.com>
MAPGPA request from TDX VMs gets split into chunks by KVM using a loop
of userspace exits until the complete range is handled.
In some cases userspace VMM might decide to break the MAPGPA operation
and continue it later. For example: in the case of intrahost migration
userspace might decide to continue the MAPGPA operation after the
migration is completed.
Allow userspace to signal to TDX guests that the MAPGPA operation should
be retried the next time the guest is scheduled.
This is potentially a breaking change since if userspace sets
hypercall.ret to a value other than EBUSY or EINVAL an EINVAL error code
will be returned to userspace. As of now QEMU never sets hypercall.ret
to a non-zero value after handling KVM_EXIT_HYPERCALL so this change
should be safe.
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Vishal Annapurve <vannapurve@google.com>
Co-developed-by: Sagi Shahar <sagis@google.com>
Signed-off-by: Sagi Shahar <sagis@google.com>
---
Documentation/virt/kvm/api.rst | 3 +++
arch/x86/kvm/vmx/tdx.c | 28 +++++++++++++++++++++-------
arch/x86/kvm/x86.h | 6 ++++++
3 files changed, 30 insertions(+), 7 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 6f85e1b321dd..027f7fadd757 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -8808,6 +8808,9 @@ block sizes is exposed in KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES as a
This capability, if enabled, will cause KVM to exit to userspace
with KVM_EXIT_HYPERCALL exit reason to process some hypercalls.
+Userspace may fail the hypercall by setting hypercall.ret to EINVAL
+or may request the hypercall to be retried the next time the guest run
+by setting hypercall.ret to EAGAIN.
Calling KVM_CHECK_EXTENSION for this capability will return a bitmask
of hypercalls that can be configured to exit to userspace.
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index c5065f84b78b..f47d5e34f3fc 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -1186,12 +1186,22 @@ static void __tdx_map_gpa(struct vcpu_tdx *tdx);
static int tdx_complete_vmcall_map_gpa(struct kvm_vcpu *vcpu)
{
+ u64 hypercall_ret = READ_ONCE(vcpu->run->hypercall.ret);
struct vcpu_tdx *tdx = to_tdx(vcpu);
+ long rc;
- if (vcpu->run->hypercall.ret) {
- tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND);
- tdx->vp_enter_args.r11 = tdx->map_gpa_next;
- return 1;
+ switch (hypercall_ret) {
+ case 0:
+ break;
+ case EAGAIN:
+ rc = TDVMCALL_STATUS_RETRY;
+ goto propagate_error;
+ case EINVAL:
+ rc = TDVMCALL_STATUS_INVALID_OPERAND;
+ goto propagate_error;
+ default:
+ WARN_ON_ONCE(kvm_is_valid_map_gpa_range_ret(hypercall_ret));
+ return -EINVAL;
}
tdx->map_gpa_next += TDX_MAP_GPA_MAX_LEN;
@@ -1204,13 +1214,17 @@ static int tdx_complete_vmcall_map_gpa(struct kvm_vcpu *vcpu)
* TDVMCALL_MAP_GPA, see comments in tdx_protected_apic_has_interrupt().
*/
if (kvm_vcpu_has_events(vcpu)) {
- tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_RETRY);
- tdx->vp_enter_args.r11 = tdx->map_gpa_next;
- return 1;
+ rc = TDVMCALL_STATUS_RETRY;
+ goto propagate_error;
}
__tdx_map_gpa(tdx);
return 0;
+
+propagate_error:
+ tdvmcall_set_return_code(vcpu, rc);
+ tdx->vp_enter_args.r11 = tdx->map_gpa_next;
+ return 1;
}
static void __tdx_map_gpa(struct vcpu_tdx *tdx)
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index 94d4f07aaaa0..9dc6da955c2a 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -720,6 +720,12 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size,
unsigned int port, void *data, unsigned int count,
int in);
+static inline bool kvm_is_valid_map_gpa_range_ret(u64 hypercall_ret)
+{
+ return !hypercall_ret || hypercall_ret == EINVAL ||
+ hypercall_ret == EAGAIN;
+}
+
static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
{
return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
--
2.53.0.473.g4a7958ca14-goog
^ permalink raw reply related
* [PATCH v4 2/2] KVM: SEV: Restrict userspace return codes for KVM_HC_MAP_GPA_RANGE
From: Sagi Shahar @ 2026-03-05 22:26 UTC (permalink / raw)
To: Vishal Annapurve, Sean Christopherson, Paolo Bonzini, Dave Hansen,
Kiryl Shutsemau, Rick Edgecombe
Cc: Thomas Gleixner, Borislav Petkov, H. Peter Anvin, Michael Roth,
Tom Lendacky, x86, kvm, linux-kernel, linux-coco, Sagi Shahar
In-Reply-To: <20260305222627.4193305-1-sagis@google.com>
To align with the updated TDX api that allows userspace to request
that guests retry MAP_GPA operations, make sure that userspace is only
returning EINVAL or EAGAIN as possible error codes.
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sagi Shahar <sagis@google.com>
---
arch/x86/kvm/svm/sev.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 3f9c1aa39a0a..04076262f087 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -3718,9 +3718,13 @@ static int snp_rmptable_psmash(kvm_pfn_t pfn)
static int snp_complete_psc_msr(struct kvm_vcpu *vcpu)
{
+ u64 hypercall_ret = READ_ONCE(vcpu->run->hypercall.ret);
struct vcpu_svm *svm = to_svm(vcpu);
- if (vcpu->run->hypercall.ret)
+ if (!kvm_is_valid_map_gpa_range_ret(hypercall_ret))
+ return -EINVAL;
+
+ if (hypercall_ret)
set_ghcb_msr(svm, GHCB_MSR_PSC_RESP_ERROR);
else
set_ghcb_msr(svm, GHCB_MSR_PSC_RESP);
@@ -3811,10 +3815,14 @@ static void __snp_complete_one_psc(struct vcpu_svm *svm)
static int snp_complete_one_psc(struct kvm_vcpu *vcpu)
{
+ u64 hypercall_ret = READ_ONCE(vcpu->run->hypercall.ret);
struct vcpu_svm *svm = to_svm(vcpu);
struct psc_buffer *psc = svm->sev_es.ghcb_sa;
- if (vcpu->run->hypercall.ret) {
+ if (!kvm_is_valid_map_gpa_range_ret(hypercall_ret))
+ return -EINVAL;
+
+ if (hypercall_ret) {
snp_complete_psc(svm, VMGEXIT_PSC_ERROR_GENERIC);
return 1; /* resume guest */
}
--
2.53.0.473.g4a7958ca14-goog
^ permalink raw reply related
* Re: [PATCH v3 1/2] KVM: TDX: Allow userspace to return errors to guest for MAPGPA
From: Sagi Shahar @ 2026-03-05 22:27 UTC (permalink / raw)
To: Sean Christopherson
Cc: Michael Roth, Tom Lendacky, Paolo Bonzini, Dave Hansen,
Kiryl Shutsemau, Rick Edgecombe, Thomas Gleixner, Borislav Petkov,
H. Peter Anvin, x86, kvm, linux-kernel, linux-coco,
Vishal Annapurve
In-Reply-To: <aZS_ePUyLcTyZ4Am@google.com>
On Tue, Feb 17, 2026 at 1:20 PM Sean Christopherson <seanjc@google.com> wrote:
>
> On Tue, Feb 17, 2026, Michael Roth wrote:
> > On Tue, Feb 17, 2026 at 12:45:52PM -0600, Tom Lendacky wrote:
> > > On 2/17/26 12:05, Michael Roth wrote:
> > > >> diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> > > >> index 2d7a4d52ccfb..056a44b9d78b 100644
> > > >> --- a/arch/x86/kvm/vmx/tdx.c
> > > >> +++ b/arch/x86/kvm/vmx/tdx.c
> > > >> @@ -1186,10 +1186,21 @@ static void __tdx_map_gpa(struct vcpu_tdx *tdx);
> > > >>
> > > >> static int tdx_complete_vmcall_map_gpa(struct kvm_vcpu *vcpu)
> > > >> {
> > > >> + u64 hypercall_ret = READ_ONCE(vcpu->run->hypercall.ret);
> > > >> struct vcpu_tdx *tdx = to_tdx(vcpu);
> > > >>
> > > >> - if (vcpu->run->hypercall.ret) {
> > > >> - tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND);
> > > >> + if (hypercall_ret) {
> > > >> + if (hypercall_ret == EAGAIN) {
> > > >> + tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_RETRY);
> > > >> + } else if (vcpu->run->hypercall.ret == EINVAL) {
> > > >> + tdvmcall_set_return_code(
> > > >> + vcpu, TDVMCALL_STATUS_INVALID_OPERAND);
> > > >> + } else {
> > > >> + WARN_ON_ONCE(
> > > >> + kvm_is_valid_map_gpa_range_ret(hypercall_ret));
> > > >> + return -EINVAL;
> > > >> + }
> > > >> +
> > > >> tdx->vp_enter_args.r11 = tdx->map_gpa_next;
> > > >> return 1;
> > > >> }
> > > >
> > > > Maybe slightly more readable?
> > > >
> > > > switch (hypercall_ret) {
> > > > case EAGAIN:
> > > > tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_RETRY);
> > > > /* fallthrough */
> > >
> > > I think you want a break here, not a fallthrough, so that you don't set
> > > the return code twice with the last one not being correct for EAGAIN.
> >
> > Doh, thanks for the catch. I guess a break for the EINVAL case as well would
> > be more consistent then.
> >
> > switch (hypercall_ret) {
> > case EAGAIN:
> > tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_RETRY);
> > break;
> > case EINVAL:
> > tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND);
> > break;
> > case 0:
> > break;
> > case default:
> > WARN_ON_ONCE(kvm_is_valid_map_gpa_range_ret(hypercall_ret));
> > return -EINVAL;
> > }
> >
> > tdx->vp_enter_args.r11 = tdx->map_gpa_next;
> > return 1;
>
> Heh, except then KVM will fail to handle the next chunk on success. I like the
> idea of a switch statement, so what if we add that and dedup the error handling?
>
> static int tdx_complete_vmcall_map_gpa(struct kvm_vcpu *vcpu)
> {
> u64 hypercall_ret = READ_ONCE(vcpu->run->hypercall.ret);
> struct vcpu_tdx *tdx = to_tdx(vcpu);
> long rc;
>
> switch (hypercall_ret) {
> case 0:
> break;
> case EAGAIN:
> rc = TDVMCALL_STATUS_RETRY;
> goto propagate_error;
> case EINVAL:
> rc = TDVMCALL_STATUS_INVALID_OPERAND;
> goto propagate_error;
> default:
> WARN_ON_ONCE(kvm_is_valid_map_gpa_range_ret(hypercall_ret));
> return -EINVAL;
> }
>
> tdx->map_gpa_next += TDX_MAP_GPA_MAX_LEN;
> if (tdx->map_gpa_next >= tdx->map_gpa_end)
> return 1;
>
> /*
> * Stop processing the remaining part if there is a pending interrupt,
> * which could be qualified to deliver. Skip checking pending RVI for
> * TDVMCALL_MAP_GPA, see comments in tdx_protected_apic_has_interrupt().
> */
> if (kvm_vcpu_has_events(vcpu)) {
> rc = TDVMCALL_STATUS_RETRY;
> goto propagate_error;
> }
>
> __tdx_map_gpa(tdx);
> return 0;
>
> propagate_error:
> tdvmcall_set_return_code(vcpu, rc);
> tdx->vp_enter_args.r11 = tdx->map_gpa_next;
> return 1;
> }
Thanks for the review. I updated the code and sent out v4 for review.
^ permalink raw reply
* Re: [PATCH v4 02/24] coco/tdx-host: Introduce a "tdx_host" device
From: Chao Gao @ 2026-03-06 2:13 UTC (permalink / raw)
To: Binbin Wu
Cc: linux-coco, linux-kernel, kvm, x86, reinette.chatre, ira.weiny,
kai.huang, dan.j.williams, yilun.xu, sagis, vannapurve, paulmck,
nik.borisov, zhenzhong.duan, seanjc, rick.p.edgecombe, kas,
dave.hansen, vishal.l.verma, tony.lindgren, Jonathan Cameron,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin
In-Reply-To: <1be33429-25ee-4e99-b795-18f77f6cbc34@linux.intel.com>
>> The call to tdx_get_sysinfo() ensures that the TDX Module is ready to
>
>Nit:
>There are "TDX module", "TDX-module" and "TDX Module" in the cover letter.
>Better to align the style.
You are right. The terminology is inconsistent and confusing.
Different Intel specifications use different formats: the CPU Architectural
Extensions spec uses "TDX module" (lowercase 'm'), while the Module Base
Architecture Specification uses "TDX Module" (capital 'M'). I'm not sure where
"TDX-module" comes from, and Sean's VMXON series [*] adds to the confusion by
using "TDX-Module" in log messages.
*: https://lore.kernel.org/kvm/20260214012702.2368778-12-seanjc@google.com/
I don't have a strong preference, but I'll standardize on "TDX Module" since it
matches the Base Architecture Specification, which I think is the most
authoritative source about TDX Module features/terms.
^ permalink raw reply
* Re: [PATCH v4 02/24] coco/tdx-host: Introduce a "tdx_host" device
From: Dave Hansen @ 2026-03-06 4:17 UTC (permalink / raw)
To: Chao Gao, Binbin Wu
Cc: linux-coco, linux-kernel, kvm, x86, reinette.chatre, ira.weiny,
kai.huang, dan.j.williams, yilun.xu, sagis, vannapurve, paulmck,
nik.borisov, zhenzhong.duan, seanjc, rick.p.edgecombe, kas,
dave.hansen, vishal.l.verma, tony.lindgren, Jonathan Cameron,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin
In-Reply-To: <aao4VunqChU5ZTOE@intel.com>
On 3/5/26 18:13, Chao Gao wrote:
> I don't have a strong preference, but I'll standardize on "TDX
> Module" since it matches the Base Architecture Specification, which
> I think is the most authoritative source about TDX Module features/
> terms.
How about doing what the Linux kernel does -- and has been doing --
instead of trying to pick a new policy a few years into the kernel
dealing with TDX?
"TDX module" was the first and it's 20x more common in the history than
the next closest one:
$ git log -p arch/x86/ | grep -i -o 'tdx[- ]module' | sort | uniq -c |
sort -n
2 TDX-module
21 TDX-Module
26 TDX Module
501 TDX module
If you don't have a strong preference, why are you arguing for change now?
^ permalink raw reply
* Re: [PATCH v4 02/24] coco/tdx-host: Introduce a "tdx_host" device
From: Chao Gao @ 2026-03-06 5:12 UTC (permalink / raw)
To: Dave Hansen
Cc: Binbin Wu, linux-coco, linux-kernel, kvm, x86, reinette.chatre,
ira.weiny, kai.huang, dan.j.williams, yilun.xu, sagis, vannapurve,
paulmck, nik.borisov, zhenzhong.duan, seanjc, rick.p.edgecombe,
kas, dave.hansen, vishal.l.verma, tony.lindgren, Jonathan Cameron,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin
In-Reply-To: <635e5c2d-9b4d-4c2b-8e7d-b9b6b3ec538f@intel.com>
On Thu, Mar 05, 2026 at 08:17:34PM -0800, Dave Hansen wrote:
>On 3/5/26 18:13, Chao Gao wrote:
>> I don't have a strong preference, but I'll standardize on "TDX
>> Module" since it matches the Base Architecture Specification, which
>> I think is the most authoritative source about TDX Module features/
>> terms.
>How about doing what the Linux kernel does -- and has been doing --
>instead of trying to pick a new policy a few years into the kernel
>dealing with TDX?
Makes sense to me.
>
>"TDX module" was the first and it's 20x more common in the history than
>the next closest one:
>
>$ git log -p arch/x86/ | grep -i -o 'tdx[- ]module' | sort | uniq -c |
>sort -n
> 2 TDX-module
> 21 TDX-Module
> 26 TDX Module
> 501 TDX module
>
>If you don't have a strong preference, why are you arguing for change now?
I was just explaining what I would do for this series and my reasoning (if no
one had a strong preference and no one responded). I wasn't arguing that "TDX
module" is worse in any way.
^ permalink raw reply
* Re: [PATCH v4 13/24] x86/virt/seamldr: Shut down the current TDX module
From: Chao Gao @ 2026-03-06 8:14 UTC (permalink / raw)
To: Huang, Kai
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev,
linux-kernel@vger.kernel.org, x86@kernel.org,
dave.hansen@linux.intel.com, tony.lindgren@linux.intel.com,
binbin.wu@linux.intel.com, seanjc@google.com, kas@kernel.org,
Chatre, Reinette, Verma, Vishal L, nik.borisov@suse.com,
mingo@redhat.com, Weiny, Ira, hpa@zytor.com, Annapurve, Vishal,
sagis@google.com, Duan, Zhenzhong, Edgecombe, Rick P,
paulmck@kernel.org, tglx@kernel.org, yilun.xu@linux.intel.com,
Williams, Dan J, bp@alien8.de
In-Reply-To: <0c86d95449543dee0369bd83740b25aae595a5d1.camel@intel.com>
>> Ideally, the kernel needs to retrieve the handoff versions supported by
>> the current module and the new module and select a version supported by
>> both. But, since the Linux kernel only supports module upgrades, simply
>
>Nit:
>
>Again, ".. the Linux kernel only supports module upgrades ..." sounds like
>describing the behaviour of the current kernel, but for now runtime update
>is not supported yet.
ack.
>
>I would change to " .. this implementation chooses to only support module
>upgrades".
looks good to me. Will do.
>> --- a/arch/x86/virt/vmx/tdx/tdx.h
>> +++ b/arch/x86/virt/vmx/tdx/tdx.h
>> @@ -46,6 +46,7 @@
>> #define TDH_PHYMEM_PAGE_WBINVD 41
>> #define TDH_VP_WR 43
>> #define TDH_SYS_CONFIG 45
>> +#define TDH_SYS_SHUTDOWN 52
>>
>> /*
>> * SEAMCALL leaf:
>> @@ -118,4 +119,6 @@ struct tdmr_info_list {
>> int max_tdmrs; /* How many 'tdmr_info's are allocated */
>> };
>>
>> +int tdx_module_shutdown(void);
>
>This (and future patches) makes couple of tdx_xx() functions visible out of
>tdx.c. The alternative is to move the main "module update" function out of
>seamldr.c to tdx.c, but that would require making couple of seamldr_xx()s
>(and data structures probably) visible to tdx.c too.
Yes. I'll keep this organization unless someone strongly prefers moving the
main "module update" function and related data structures to tdx.c.
If neither approach is acceptable, a third option would be to remove seamldr.c
entirely and merge it into tdx.c. This would mean adding ~360 LoC to an
existing file that already has ~1900 LoC.
^ permalink raw reply
* Re: [PATCH v4 19/24] x86/virt/tdx: Update tdx_sysinfo and check features post-update
From: Chao Gao @ 2026-03-06 8:32 UTC (permalink / raw)
To: Huang, Kai
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev,
linux-kernel@vger.kernel.org, x86@kernel.org,
dave.hansen@linux.intel.com, tony.lindgren@linux.intel.com,
binbin.wu@linux.intel.com, seanjc@google.com, kas@kernel.org,
Chatre, Reinette, Verma, Vishal L, nik.borisov@suse.com,
mingo@redhat.com, Weiny, Ira, hpa@zytor.com, Annapurve, Vishal,
sagis@google.com, Duan, Zhenzhong, Edgecombe, Rick P,
paulmck@kernel.org, tglx@kernel.org, yilun.xu@linux.intel.com,
Williams, Dan J, bp@alien8.de
In-Reply-To: <1aaa74b61001b45261701aa73fc085a14473919d.camel@intel.com>
>>
>> +/*
>> + * Update tdx_sysinfo and check if any TDX module features changed after
>> + * updates
>
>s/updates/update? I don't see more than one update.
By "updates" I meant the update process in general, not multiple updates. Since
"update" is countable, it needs an article. I'll change it to "after an update"
for clarity.
>
>And it's more than "check module features being changed" since there are
>other metadata fields which may have different values after update, right?
Good point.
>
>I would just remove this comment since I don't see it says more than just
>repeating the code below (which also has comments saying the same thing, in
>a more elaborated way).
I added the comment because the function name isn't immediately clear about
what it does. I'd like people to understand the function's purpose without
reading the implementation (I also couldn't find a self-explanatory name for
the function). So I'd prefer to revise the comment rather than remove it.
Thanks.
^ permalink raw reply
* Re: SVSM Development Call March 4, 2026
From: Jörg Rödel @ 2026-03-06 8:45 UTC (permalink / raw)
To: coconut-svsm, linux-coco
In-Reply-To: <bvfkpoqw2bz6ryy3kblt4vrqweomutyypvn7tfqqokyje6yoc3@dqwyss5eb6qy>
Meeting minutes are now posted here:
https://github.com/coconut-svsm/governance/pull/98
-Joerg
^ permalink raw reply
* Re: [PATCH v4 06/24] coco/tdx-host: Expose P-SEAMLDR information via sysfs
From: Binbin Wu @ 2026-03-06 9:29 UTC (permalink / raw)
To: Chao Gao
Cc: linux-coco, linux-kernel, kvm, x86, reinette.chatre, ira.weiny,
kai.huang, dan.j.williams, yilun.xu, sagis, vannapurve, paulmck,
nik.borisov, zhenzhong.duan, seanjc, rick.p.edgecombe, kas,
dave.hansen, vishal.l.verma, tony.lindgren, Farrah Chen
In-Reply-To: <20260212143606.534586-7-chao.gao@intel.com>
On 2/12/2026 10:35 PM, Chao Gao wrote:
> TDX Module updates require userspace to select the appropriate module
> to load. Expose necessary information to facilitate this decision. Two
> values are needed:
>
> - P-SEAMLDR version: for compatibility checks between TDX Module and
> P-SEAMLDR
> - num_remaining_updates: indicates how many updates can be performed
>
> Expose them as tdx-host device attributes.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
> Tested-by: Farrah Chen <farrah.chen@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Some nits below.
> ---
> v4:
> - Make seamldr attribute permission "0400" [Dave]
> - Don't include implementation details in OS ABI docs [Dave]
> - Tag tdx_host_group as static [Kai]
>
> v3:
> - use #ifdef rather than .is_visible() to control P-SEAMLDR sysfs
> visibility [Yilun]
> ---
> .../ABI/testing/sysfs-devices-faux-tdx-host | 23 +++++++
> drivers/virt/coco/tdx-host/tdx-host.c | 63 ++++++++++++++++++-
> 2 files changed, 85 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/ABI/testing/sysfs-devices-faux-tdx-host b/Documentation/ABI/testing/sysfs-devices-faux-tdx-host
> index 901abbae2e61..88a9c0b2bdfe 100644
> --- a/Documentation/ABI/testing/sysfs-devices-faux-tdx-host
> +++ b/Documentation/ABI/testing/sysfs-devices-faux-tdx-host
> @@ -4,3 +4,26 @@ Description: (RO) Report the version of the loaded TDX Module. The TDX Module
> version is formatted as x.y.z, where "x" is the major version,
> "y" is the minor version and "z" is the update version. Versions
> are used for bug reporting, TDX Module updates and etc.
> +
> +What: /sys/devices/faux/tdx_host/seamldr/version
> +Contact: linux-coco@lists.linux.dev
> +Description: (RO) Report the version of the loaded SEAM loader. The SEAM
> + loader version is formatted as x.y.z, where "x" is the major
> + version, "y" is the minor version and "z" is the update version.
> + Versions are used for bug reporting and compatibility checks.
> +
> +What: /sys/devices/faux/tdx_host/seamldr/num_remaining_updates
> +Contact: linux-coco@lists.linux.dev
> +Description: (RO) Report the number of remaining updates. TDX maintains a
> + log about each TDX Module which has been loaded. This log has
^
that
> + a finite size which limits the number of TDX Module updates
^
,
> + which can be performed.
^
that
> +
> + After each successful update, the number reduces by one. Once it
> + reaches zero, further updates will fail until next reboot. The
> + number is always zero if the P-SEAMLDR doesn't support updates.
> +
> + See Intel® Trust Domain Extensions - SEAM Loader (SEAMLDR)
> + Interface Specification, Revision 343755-003, Chapter 3.3
> + "SEAMLDR_INFO" and Chapter 4.2 "SEAMLDR.INSTALL" for more
> + information.
> diff --git a/drivers/virt/coco/tdx-host/tdx-host.c b/drivers/virt/coco/tdx-host/tdx-host.c
> index 0424933b2560..fd6ffb4f2ff1 100644
> --- a/drivers/virt/coco/tdx-host/tdx-host.c
> +++ b/drivers/virt/coco/tdx-host/tdx-host.c
> @@ -11,6 +11,7 @@
> #include <linux/sysfs.h>
>
> #include <asm/cpu_device_id.h>
> +#include <asm/seamldr.h>
> #include <asm/tdx.h>
>
> static const struct x86_cpu_id tdx_host_ids[] = {
> @@ -40,7 +41,67 @@ static struct attribute *tdx_host_attrs[] = {
> &dev_attr_version.attr,
> NULL,
> };
> -ATTRIBUTE_GROUPS(tdx_host);
> +
> +static struct attribute_group tdx_host_group = {
> + .attrs = tdx_host_attrs,
> +};
> +
> +static ssize_t seamldr_version_show(struct device *dev, struct device_attribute *attr,
> + char *buf)
> +{
> + struct seamldr_info info;
> + int ret;
> +
> + ret = seamldr_get_info(&info);
> + if (ret)
> + return ret;
> +
> + return sysfs_emit(buf, "%u.%u.%02u\n", info.major_version,
> + info.minor_version,
> + info.update_version);
> +}
> +
> +static ssize_t num_remaining_updates_show(struct device *dev,
> + struct device_attribute *attr,
> + char *buf)
> +{
> + struct seamldr_info info;
> + int ret;
> +
> + ret = seamldr_get_info(&info);
> + if (ret)
> + return ret;
> +
> + return sysfs_emit(buf, "%u\n", info.num_remaining_updates);
> +}
> +
> +/*
> + * Open-code DEVICE_ATTR_ADMIN_RO to specify a different 'show' function
> + * for P-SEAMLDR version as version_show() is used for TDX Module version.
> + *
> + * admin-only readable as reading these attributes calls into P-SEAMLDR,
^
Admin-only
> + * which may have potential performance and system impact.
> + */
> +static struct device_attribute dev_attr_seamldr_version =
> + __ATTR(version, 0400, seamldr_version_show, NULL);
> +static DEVICE_ATTR_ADMIN_RO(num_remaining_updates);
> +
> +static struct attribute *seamldr_attrs[] = {
> + &dev_attr_seamldr_version.attr,
> + &dev_attr_num_remaining_updates.attr,
> + NULL,
> +};
> +
> +static struct attribute_group seamldr_group = {
> + .name = "seamldr",
> + .attrs = seamldr_attrs,
> +};
> +
> +static const struct attribute_group *tdx_host_groups[] = {
> + &tdx_host_group,
> + &seamldr_group,
> + NULL,
> +};
>
> static struct faux_device *fdev;
>
^ permalink raw reply
* Re: [PATCH v4 19/24] x86/virt/tdx: Update tdx_sysinfo and check features post-update
From: Huang, Kai @ 2026-03-06 9:35 UTC (permalink / raw)
To: Gao, Chao
Cc: tony.lindgren@linux.intel.com, linux-coco@lists.linux.dev,
kvm@vger.kernel.org, dave.hansen@linux.intel.com, bp@alien8.de,
kas@kernel.org, mingo@redhat.com, Chatre, Reinette, Weiny, Ira,
seanjc@google.com, Verma, Vishal L, nik.borisov@suse.com,
binbin.wu@linux.intel.com, hpa@zytor.com, Annapurve, Vishal,
sagis@google.com, Duan, Zhenzhong, Edgecombe, Rick P,
linux-kernel@vger.kernel.org, paulmck@kernel.org, tglx@kernel.org,
yilun.xu@linux.intel.com, x86@kernel.org, Williams, Dan J
In-Reply-To: <aaqRN62xGuOwkvGN@intel.com>
> >
> > I would just remove this comment since I don't see it says more than just
> > repeating the code below (which also has comments saying the same thing, in
> > a more elaborated way).
>
> I added the comment because the function name isn't immediately clear about
> what it does.
>
It says "post_update", so it is clear to me that it is for "all last steps"
need to be done in the kernel after module is updated.
> I'd like people to understand the function's purpose without
> reading the implementation (I also couldn't find a self-explanatory name for
> the function). So I'd prefer to revise the comment rather than remove it.
>
We can see exactly what is done as "post update" in the code.
And if you add more functionalities in the future to this function, you
don't have to modify the comment of the function to expand.
Anyway, up to you :-)
^ permalink raw reply
* Re: [PATCH v2 2/7] x86/sev: add support for enabling RMPOPT
From: Borislav Petkov @ 2026-03-06 15:18 UTC (permalink / raw)
To: Ashish Kalra
Cc: tglx, mingo, dave.hansen, x86, hpa, seanjc, peterz,
thomas.lendacky, herbert, davem, ardb, pbonzini, aik,
Michael.Roth, KPrateek.Nayak, Tycho.Andersen, Nathan.Fontenot,
jackyli, pgonda, rientjes, jacobhxu, xin, pawan.kumar.gupta,
babu.moger, dyoung, nikunj, john.allen, darwi, linux-kernel,
linux-crypto, kvm, linux-coco
In-Reply-To: <85aec55af41957678d214e9629eb6249b064fa87.1772486459.git.ashish.kalra@amd.com>
On Mon, Mar 02, 2026 at 09:35:55PM +0000, Ashish Kalra wrote:
> From: Ashish Kalra <ashish.kalra@amd.com>
>
> The new RMPOPT instruction sets bits in a per-CPU RMPOPT table, which
> indicates whether specific 1GB physical memory regions contain SEV-SNP
"... which indicate... "
> guest memory.
>
> Per-CPU RMPOPT tables support at most 2 TB of addressable memory for
> RMP optimizations.
>
> Initialize the per-CPU RMPOPT table base to the starting physical
> address. This enables RMP optimization for up to 2 TB of system RAM on
> all CPUs.
...
> +static void __configure_rmpopt(void *val)
> +{
> + u64 rmpopt_base = ((u64)val & PUD_MASK) | MSR_AMD64_RMPOPT_ENABLE;
> +
> + wrmsrq(MSR_AMD64_RMPOPT_BASE, rmpopt_base);
> +}
> +
> +static __init void configure_and_enable_rmpopt(void)
If the sub-helper is called __configure_rmpopt() then this should be called
"configure_rmpopt", without the prepended underscores.
> + phys_addr_t pa_start = ALIGN_DOWN(PFN_PHYS(min_low_pfn), PUD_SIZE);
> +
> + if (!cpu_feature_enabled(X86_FEATURE_RMPOPT)) {
> + pr_debug("RMPOPT not supported on this platform\n");
> + return;
> + }
> +
> + if (!cc_platform_has(CC_ATTR_HOST_SEV_SNP)) {
> + pr_debug("RMPOPT optimizations not enabled as SNP support is not enabled\n");
> + return;
> + }
Zap this one - snp_rmptable_init() already checked it.
Also, zap those pr_debugs - you have that information elsewhere already.
> +
> + if (!(rmp_cfg & MSR_AMD64_SEG_RMP_ENABLED)) {
You can't test this one - you need to test the result of
setup_segmented_rmptable() and whether it did set up the segmented RMP
properly. Only then you can continue here.
> + pr_info("RMPOPT optimizations not enabled, segmented RMP required\n");
This looks like pr_notice() to me.
> + return;
> + }
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply
* Re: [PATCH v2 2/7] x86/sev: add support for enabling RMPOPT
From: Tom Lendacky @ 2026-03-06 15:33 UTC (permalink / raw)
To: Borislav Petkov, Ashish Kalra
Cc: tglx, mingo, dave.hansen, x86, hpa, seanjc, peterz, herbert,
davem, ardb, pbonzini, aik, Michael.Roth, KPrateek.Nayak,
Tycho.Andersen, Nathan.Fontenot, jackyli, pgonda, rientjes,
jacobhxu, xin, pawan.kumar.gupta, babu.moger, dyoung, nikunj,
john.allen, darwi, linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <20260306151849.GJaarwWSaWnnRh9ffB@fat_crate.local>
On 3/6/26 09:18, Borislav Petkov wrote:
> On Mon, Mar 02, 2026 at 09:35:55PM +0000, Ashish Kalra wrote:
>> From: Ashish Kalra <ashish.kalra@amd.com>
>>
>> The new RMPOPT instruction sets bits in a per-CPU RMPOPT table, which
>> indicates whether specific 1GB physical memory regions contain SEV-SNP
>
> "... which indicate... "
>
>> guest memory.
>>
>> Per-CPU RMPOPT tables support at most 2 TB of addressable memory for
>> RMP optimizations.
>>
>> Initialize the per-CPU RMPOPT table base to the starting physical
>> address. This enables RMP optimization for up to 2 TB of system RAM on
>> all CPUs.
>
> ...
>
>> +static void __configure_rmpopt(void *val)
>> +{
>> + u64 rmpopt_base = ((u64)val & PUD_MASK) | MSR_AMD64_RMPOPT_ENABLE;
>> +
>> + wrmsrq(MSR_AMD64_RMPOPT_BASE, rmpopt_base);
>> +}
>> +
>> +static __init void configure_and_enable_rmpopt(void)
>
> If the sub-helper is called __configure_rmpopt() then this should be called
> "configure_rmpopt", without the prepended underscores.
>
>> + phys_addr_t pa_start = ALIGN_DOWN(PFN_PHYS(min_low_pfn), PUD_SIZE);
>> +
>> + if (!cpu_feature_enabled(X86_FEATURE_RMPOPT)) {
>> + pr_debug("RMPOPT not supported on this platform\n");
>> + return;
>> + }
>> +
>> + if (!cc_platform_has(CC_ATTR_HOST_SEV_SNP)) {
>> + pr_debug("RMPOPT optimizations not enabled as SNP support is not enabled\n");
>> + return;
>> + }
>
> Zap this one - snp_rmptable_init() already checked it.
>
> Also, zap those pr_debugs - you have that information elsewhere already.
>
>> +
>> + if (!(rmp_cfg & MSR_AMD64_SEG_RMP_ENABLED)) {
>
> You can't test this one - you need to test the result of
> setup_segmented_rmptable() and whether it did set up the segmented RMP
> properly. Only then you can continue here.
If the segmented RMP setup fails, then CC_ATTR_HOST_SEV_SNP gets cleared,
so it looks like the above check needs to remain then.
Thanks,
Tom
>
>> + pr_info("RMPOPT optimizations not enabled, segmented RMP required\n");
>
> This looks like pr_notice() to me.
>
>> + return;
>> + }
>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox