* Re: [PATCH v4 12/24] x86/virt/seamldr: Abort updates if errors occurred midway
From: Huang, Kai @ 2026-03-04 22:38 UTC (permalink / raw)
To: kvm@vger.kernel.org, linux-coco@lists.linux.dev,
linux-kernel@vger.kernel.org, Gao, Chao, x86@kernel.org
Cc: dave.hansen@linux.intel.com, tony.lindgren@linux.intel.com,
binbin.wu@linux.intel.com, seanjc@google.com, kas@kernel.org,
Chatre, Reinette, Verma, Vishal L, nik.borisov@suse.com,
mingo@redhat.com, Weiny, Ira, hpa@zytor.com, Annapurve, Vishal,
sagis@google.com, Duan, Zhenzhong, Edgecombe, Rick P,
paulmck@kernel.org, tglx@kernel.org, yilun.xu@linux.intel.com,
Williams, Dan J, bp@alien8.de
In-Reply-To: <20260212143606.534586-13-chao.gao@intel.com>
On Thu, 2026-02-12 at 06:35 -0800, Chao Gao wrote:
> The TDX Module update process has multiple steps, each of which may
> encounter failures.
>
> The current state machine of updates proceeds to the next step regardless
> of errors. But continuing updates when errors occur midway is pointless.
>
> Abort the update by setting a flag to indicate that a CPU has encountered
> an error, forcing all CPUs to exit the execution loop. Note that failing
> CPUs do not acknowledge the current step. This keeps all other CPUs waiting
> in the current step (since advancing to the next step requires all CPUs to
> acknowledge the current step) until they detect the fault flag and exit the
> loop.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
>
Reviewed-by: Kai Huang <kai.huang@intel.com>
^ permalink raw reply
* [Invitation] bi-weekly guest_memfd upstream call on 2026-03-05
From: David Hildenbrand (Arm) @ 2026-03-04 20:11 UTC (permalink / raw)
To: linux-coco@lists.linux.dev, linux-mm@kvack.org, KVM
Cc: ackerleytng, amit, aneeshkumar.kizhakeveetil, ashish.kalra,
dan.j.williams, eberman, fvdl, gshan, jackmanb, jackyli,
jthoughton, kalyazin, kevinloughlin, liruxin, michael.day,
michael.roth, mike.rapoport, mvaralar, pankaj.gupta, papaluri,
patrick.roy, Peter Xu, pheragu, pkondeti, psalian, seanjc,
shan.gavin, shivankg, sidtelang, suzuki.poulose, tabba, tatashin,
vannapurve, vbabka, wyihan
Hi,
Our next guest_memfd upstream call is scheduled for tomorrow, Thursday,
2026-03-05 at 8:00 - 9:00am (GMT-08:00) Pacific Time - Vancouver.
We'll be using the following Google meet:
http://meet.google.com/wxp-wtju-jzw
The meeting notes can be found at [1], where we also link recordings and
collect current guest_memfd upstream proposals. If you want an google
calendar invitation that also covers all future meetings, just write me
or Ackerley a mail.
For some reason, google seems to randomly drop all invitees from the
calendar entry ... so expect some annoying invitation mails until that
seems to stop.
In this meeting, Ackerley will give us an update on guest_memfd HugeTLB
support, which we were not able to squeeze into the last meeting. I'm
sure, other things will pop up around HugeTLB support and
expected/desired timelines.
To put something to discuss onto the agenda, reply to this mail or add
them to the "Topics/questions for next meeting(s)" section in the
meeting notes as a comment.
[1]
https://docs.google.com/document/d/1M6766BzdY1Lhk7LiR5IqVR8B8mG3cr-cxTxOrAosPOk/edit?usp=sharing
[2] https://lore.kernel.org/all/20251203144159.6131-1-itazur@amazon.com/
--
Cheers,
David
^ permalink raw reply
* Re: [PATCH v2 09/19] PCI/TSM: Support creating encrypted MMIO descriptors via TDISP Report
From: dan.j.williams @ 2026-03-04 17:14 UTC (permalink / raw)
To: Dan Williams, linux-coco, linux-pci
Cc: gregkh, aik, aneesh.kumar, yilun.xu, bhelgaas, alistair23, lukas,
jgg, Arnd Bergmann
In-Reply-To: <20260303000207.1836586-10-dan.j.williams@intel.com>
Dan Williams wrote:
> After pci_tsm_bind() and pci_tsm_lock() the low level TSM driver is
> expected to populate PCI_TSM_EVIDENCE_TYPE_REPORT in its evidence store.
> This report is defined by the TDISP GET_DEVICE_INTERFACE_REPORT response
> payload.
>
> Add a helper to create encrypted MMIO descriptors from that report
> data. With those descriptors the TSM driver can use pci_tsm_mmio_setup() to
> inform ioremap() how to map the device per the device's expectations. The
> VM is expected to validate the interface with the relying party before
> accepting the device for operation.
>
> The helper also provides the obfuscated starting address for each
> encrypted MMIO range as the VM is never disclosed on the hpa that
> correlates to the gpa of the device's mmio. The obfuscated address is BAR
> relative.
>
> Based on an original patch by Aneesh [1]
>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Link: https://lore.kernel.org/linux-coco/20251117140007.122062-8-aneesh.kumar@kernel.org/
> Co-developed-by: Xu Yilun <yilun.xu@linux.intel.com>
> Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
[..]
> +/**
> + * pci_tsm_mmio_alloc() - allocate encrypted MMIO range descriptor
> + * @pdev: device owner of MMIO ranges
> + * @report_data: TDISP Device Interface (DevIf) Report blob
> + * @report_sz: DevIf Report size
> + *
> + * Return: the encrypted MMIO range descriptor on success, NULL on failure
> + *
> + * Assumes that this is called within the live lifetime of a PCI device's
> + * association with a low level TSM.
> + */
> +struct pci_tsm_mmio *pci_tsm_mmio_alloc(struct pci_dev *pdev)
> +{
> + struct pci_tsm *tsm = pdev->tsm;
> + struct pci_tsm_evidence *evidence = &tsm->evidence;
> + struct pci_tsm_evidence_object *report_obj = &evidence->obj[PCI_TSM_EVIDENCE_TYPE_REPORT];
> + struct tsm_dev *tsm_dev = tsm->tsm_dev;
> + u64 reporting_bar_base, last_reporting_end;
> + const struct pci_tsm_devif_report *report;
> + u32 mmio_range_count;
> + int last_bar = -1;
> + int i;
> +
> + guard(rwsem_read)(&evidence->lock);
> + if (report_obj->len < sizeof(struct pci_tsm_devif_report))
> + return NULL;
> +
> + if (dev_WARN_ONCE(&tsm_dev->dev, !IS_ALIGNED((unsigned long) report_obj->data, 8),
> + "misaligned report data\n"))
> + return NULL;
Is this going to cause any implementation to need to copy the buffer
received from the low-level TSM? If so I would just mark 'struct
pci_tsm_devif_report' and 'struct pci_tsm_mmio_entry' as __packed and
drop this check.
> +
> + report = report_obj->data;
> + mmio_range_count = __le32_to_cpu(report->mmio_range_count);
> +
> + /* check that the report object is self-consistent on mmio entries */
> + if (report_obj->len < struct_size(report, mmio, mmio_range_count))
> + return NULL;
> +
> + /* create pci_tsm_mmio descriptors from the report data */
> + struct pci_tsm_mmio *mmio __free(kfree) =
> + kzalloc(struct_size(mmio, mmio, mmio_range_count), GFP_KERNEL);
> + if (!mmio)
> + return NULL;
> +
> + for (i = 0; i < mmio_range_count; i++) {
> + u64 range_off;
> + struct range range;
> + const struct pci_tsm_devif_mmio *mmio_data = &report->mmio[i];
> + struct pci_tsm_mmio_entry *entry =
> + pci_tsm_mmio_entry(mmio, mmio->nr);
> + /* report values in are in terms of 4K pages */
> + u64 tsm_offset = __le64_to_cpu(mmio_data->pfn) * SZ_4K;
> + u64 size = __le32_to_cpu(mmio_data->nr_pfns) * SZ_4K;
> + u32 attr = __le32_to_cpu(mmio_data->attributes);
> + int bar = FIELD_GET(PCI_TSM_DEVIF_REPORT_MMIO_ATTR_RANGE_ID,
> + attr);
> +
> + tsm_offset *= SZ_4K;
> + size *= SZ_4K;
Whoops, these pfn to absolute address conversions were already performed above, will fix.
^ permalink raw reply
* Re: [PATCH v2 3/7] x86/sev: add support for RMPOPT instruction
From: Dave Hansen @ 2026-03-04 16:03 UTC (permalink / raw)
To: Andrew Cooper
Cc: Ashish.Kalra, KPrateek.Nayak, Michael.Roth, Nathan.Fontenot,
Tycho.Andersen, aik, ardb, babu.moger, bp, darwi, dave.hansen,
davem, dyoung, herbert, hpa, jackyli, jacobhxu, john.allen, kvm,
linux-coco, linux-crypto, linux-kernel, mingo, nikunj,
pawan.kumar.gupta, pbonzini, peterz, pgonda, rientjes, seanjc,
tglx, thomas.lendacky, x86, xin
In-Reply-To: <4ec520a1-68c7-4833-9e8f-edc610e5fdfa@citrix.com>
On 3/4/26 07:56, Andrew Cooper wrote:
> Logic asking hardware to optimise a 1G region because of no guest memory
> should at least WARN() if hardware comes back and says "well hang on now..."
It would be _nice_ to have a system where we can do a WARN(). But for
something that's just a lowly optimization, I'd rather that RMPOPT lose
the occasional race with a shared=>private conversion than have it take
a lock and _block_ those conversions.
^ permalink raw reply
* Re: [PATCH v2 3/7] x86/sev: add support for RMPOPT instruction
From: Andrew Cooper @ 2026-03-04 15:56 UTC (permalink / raw)
To: dave.hansen
Cc: Andrew Cooper, Ashish.Kalra, KPrateek.Nayak, Michael.Roth,
Nathan.Fontenot, Tycho.Andersen, aik, ardb, babu.moger, bp, darwi,
dave.hansen, davem, dyoung, herbert, hpa, jackyli, jacobhxu,
john.allen, kvm, linux-coco, linux-crypto, linux-kernel, mingo,
nikunj, pawan.kumar.gupta, pbonzini, peterz, pgonda, rientjes,
seanjc, tglx, thomas.lendacky, x86, xin
In-Reply-To: <a1701ab4-d80f-496c-bdb3-5d94d2d2f673@intel.com>
>> +/* + * 'val' is a system physical address aligned to 1GB OR'ed with
>> + * a function selection. Currently supported functions are 0 + *
>> (verify and report status) and 1 (report status). + */ +static void
>> rmpopt(void *val) +{ + asm volatile(".byte 0xf2, 0x0f, 0x01, 0xfc" +
>> : : "a" ((u64)val & PUD_MASK), "c" ((u64)val & 0x1) + : "memory",
>> "cc"); +}
> Doesn't this belong in:
>
> arch/x86/include/asm/special_insns.h
>
> Also, it's not reporting *any* status here, right? So why even talk
> about it if the kernel isn't doing any status checks? It just makes it
> more confusing.
The "c" (val & 0x1) constraint encodes whether this is a query or a
mutation, but both forms produce an answer via the carry flag.
Because it's void, it's a useless helper, and the overloading via one
parameter makes specifically poor code generation.
It should be:
static inline bool __rmpopt(unsigned long addr, unsigned int fn)
{
bool res;
asm volatile (".byte 0xf2, 0x0f, 0x01, 0xfc"
: "=ccc" (res)
: "a" (addr), "c" (fn));
return res;
}
with:
static inline bool rmpopt_query(unsigned long addr)
static inline bool rmpopt_set(unsigned long addr)
built on top.
Logic asking hardware to optimise a 1G region because of no guest memory
should at least WARN() if hardware comes back and says "well hang on now..."
The memory barrier isn't necessary and hinders the optimiser.
~Andrew
^ permalink raw reply
* Re: [PATCH v2 3/7] x86/sev: add support for RMPOPT instruction
From: Dave Hansen @ 2026-03-04 15:32 UTC (permalink / raw)
To: Sean Christopherson, Ashish Kalra
Cc: tglx, mingo, bp, dave.hansen, x86, hpa, peterz, thomas.lendacky,
herbert, davem, ardb, pbonzini, aik, Michael.Roth, KPrateek.Nayak,
Tycho.Andersen, Nathan.Fontenot, jackyli, pgonda, rientjes,
jacobhxu, xin, pawan.kumar.gupta, babu.moger, dyoung, nikunj,
john.allen, darwi, linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <7ab8d3af-b4f5-481c-ab2e-059ddd7e718e@intel.com>
On 3/4/26 07:25, Dave Hansen wrote:
> BTW, I assume that the RMPOPT disable machinery is driven from the
> INVLPGB-like TLB invalidations that are a part of the SNP
> shared=>private conversions. It's a darn shame that RMPOPT wasn't
> broadcast in the same way. It would save the poor OS a lot of work. The
> RMPOPT table is per-cpu of course, but I'm not sure what keeps *a* CPU
> from broadcasting its success finding an SNP-free physical region to
> other CPUs.
I guess the other dirt simple optimization would be to have one CPU to
the RMPOPT scan and then only IPI more CPUs if that first one succeeds.
That wouldn't be awful.
^ permalink raw reply
* Re: [PATCH v2 3/7] x86/sev: add support for RMPOPT instruction
From: Dave Hansen @ 2026-03-04 15:25 UTC (permalink / raw)
To: Sean Christopherson, Ashish Kalra
Cc: tglx, mingo, bp, dave.hansen, x86, hpa, peterz, thomas.lendacky,
herbert, davem, ardb, pbonzini, aik, Michael.Roth, KPrateek.Nayak,
Tycho.Andersen, Nathan.Fontenot, jackyli, pgonda, rientjes,
jacobhxu, xin, pawan.kumar.gupta, babu.moger, dyoung, nikunj,
john.allen, darwi, linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <aahH4XARlftClMrQ@google.com>
On 3/4/26 07:01, Sean Christopherson wrote:
> I don't see any performance data in either posted version. Bluntly, this series
> isn't going anywhere without data to guide us. E.g. comments like this from v1
>
> : And there is a cost associated with re-enabling the optimizations for all
> : system RAM (even though it runs as a background kernel thread executing RMPOPT
> : on different 1GB regions in parallel and with inline cond_resched()'s),
> : we don't want to run this periodically.
>
> suggest there is meaningful cost associated with the scan.
Well the RMP is 0.4% of the size of system memory, and I assume that you
need to scan the whole table. There are surely shortcuts for 2M pages,
but with 4k, that's ~8.5GB of RMP table for 2TB of memory. That's an
awful lot of memory traffic for each CPU.
It'll be annoying to keep a refcount per 1GB of paddr space.
One other way to do it would be to loosely mirror the RMPOPT bitmap and
keep our own bitmap of 1GB regions that _need_ RMPOPT run on them. Any
private=>shared conversion sets a bit in the bitmap and schedules some
work out in the future.
It could also be less granular than that. Instead of any private=>shared
conversion, the RMPOPT scan could be triggered on VM destruction which
is much more likely to result in RMPOPT doing anything useful.
BTW, I assume that the RMPOPT disable machinery is driven from the
INVLPGB-like TLB invalidations that are a part of the SNP
shared=>private conversions. It's a darn shame that RMPOPT wasn't
broadcast in the same way. It would save the poor OS a lot of work. The
RMPOPT table is per-cpu of course, but I'm not sure what keeps *a* CPU
from broadcasting its success finding an SNP-free physical region to
other CPUs.
tl;dr: I agree with you. The cost of these scans is going to be
annoying, and it's going to need OS help to optimize it.
^ permalink raw reply
* Re: [PATCH v4 0/4] x86/tdx: Clean up the definitions of TDX TD ATTRIBUTES
From: Nikolay Borisov @ 2026-03-04 15:02 UTC (permalink / raw)
To: Xiaoyao Li, Dave Hansen, Kiryl Shutsemau, Sean Christopherson,
Paolo Bonzini
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
linux-coco, kvm, linux-kernel, x86, Rick Edgecombe, Kai Huang,
binbin.wu, Tony Lindgren
In-Reply-To: <20260303030335.766779-1-xiaoyao.li@intel.com>
On 3.03.26 г. 5:03 ч., Xiaoyao Li wrote:
> The main purpose of this series was to remove redundant macros between
> core TDX and KVM, along with a typo fix. They were implemented as patch 1
> and patch 2.
>
> During the review of v1 and v2, there was encouragement to refine the
> names of the macros related to TD attributes to clarify their scope.
> Thus patch 3 and patch 4 are added.
>
> Note, Binbin suggested to rename tdx_attributes[] to tdx_td_attributes[]
> during v3 review. However, this v4 doesn't do it but leaves it for future
> cleanup to avoid making it more complicated because it also looks like
> it needs to rename "Attributes" to "TD Attributes" in tdx_dump_attributes(),
> which has user visibility change.
>
Reviewed-by: Nikolay Borisov <nik.borisov@suse.com>
^ permalink raw reply
* Re: [PATCH v2 3/7] x86/sev: add support for RMPOPT instruction
From: Sean Christopherson @ 2026-03-04 15:01 UTC (permalink / raw)
To: Ashish Kalra
Cc: tglx, mingo, bp, dave.hansen, x86, hpa, peterz, thomas.lendacky,
herbert, davem, ardb, pbonzini, aik, Michael.Roth, KPrateek.Nayak,
Tycho.Andersen, Nathan.Fontenot, jackyli, pgonda, rientjes,
jacobhxu, xin, pawan.kumar.gupta, babu.moger, dyoung, nikunj,
john.allen, darwi, linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <8dc0198f1261f5ae4b16388fc1ffad5ddb3895f9.1772486459.git.ashish.kalra@amd.com>
On Mon, Mar 02, 2026, Ashish Kalra wrote:
> @@ -500,6 +508,61 @@ static bool __init setup_rmptable(void)
> +/*
> + * 'val' is a system physical address aligned to 1GB OR'ed with
> + * a function selection. Currently supported functions are 0
> + * (verify and report status) and 1 (report status).
> + */
> +static void rmpopt(void *val)
> +{
> + asm volatile(".byte 0xf2, 0x0f, 0x01, 0xfc"
> + : : "a" ((u64)val & PUD_MASK), "c" ((u64)val & 0x1)
> + : "memory", "cc");
> +}
> +
> +static int rmpopt_kthread(void *__unused)
> +{
> + phys_addr_t pa_start, pa_end;
> +
> + pa_start = ALIGN_DOWN(PFN_PHYS(min_low_pfn), PUD_SIZE);
> + pa_end = ALIGN(PFN_PHYS(max_pfn), PUD_SIZE);
> +
> + /* Limit memory scanning to the first 2 TB of RAM */
> + pa_end = (pa_end - pa_start) <= SZ_2T ? pa_end : pa_start + SZ_2T;
> +
> + while (!kthread_should_stop()) {
> + phys_addr_t pa;
> +
> + pr_info("RMP optimizations enabled on physical address range @1GB alignment [0x%016llx - 0x%016llx]\n",
> + pa_start, pa_end);
> +
> + /*
> + * RMPOPT optimizations skip RMP checks at 1GB granularity if this range of
> + * memory does not contain any SNP guest memory.
> + */
> + for (pa = pa_start; pa < pa_end; pa += PUD_SIZE) {
> + /* Bit zero passes the function to the RMPOPT instruction. */
> + on_each_cpu_mask(cpu_online_mask, rmpopt,
> + (void *)(pa | RMPOPT_FUNC_VERIFY_AND_REPORT_STATUS),
> + true);
> +
> + /* Give a chance for other threads to run */
I'm not terribly concerned with other threads, but I am most definitely concerned
about other CPUs. IIUC, *every* time a guest_memfd file is destroyed, the kernel
will process *every* 2MiB chunk of memory, interrupting *every* CPU in the process.
Given that the whole point of RMPOPT is to allow running non-SNP and SNP VMs
side-by-side, inducing potentially significant jitter when stopping SNP VMs seems
like a dealbreaker.
Even using a kthread seems flawed, e.g. if all CPUs in the system are being used
to run VMs, then the kernel could be stealing cycles from an arbitrary VM/vCPU to
process RMPOPT. Contrast that with KVM's NX hugepage recovery thread, which is
spawned in the context of a specific VM so that recovering steady state performance
at the cost of periodically consuming CPU cycles is bound entirely to that VM.
I don't see any performance data in either posted version. Bluntly, this series
isn't going anywhere without data to guide us. E.g. comments like this from v1
: And there is a cost associated with re-enabling the optimizations for all
: system RAM (even though it runs as a background kernel thread executing RMPOPT
: on different 1GB regions in parallel and with inline cond_resched()'s),
: we don't want to run this periodically.
suggest there is meaningful cost associated with the scan.
^ permalink raw reply
* Re: [PATCH kernel 4/9] dma/swiotlb: Stop forcing SWIOTLB for TDISP devices
From: Jason Gunthorpe @ 2026-03-04 12:43 UTC (permalink / raw)
To: Alexey Kardashevskiy
Cc: dan.j.williams, Robin Murphy, x86, linux-kernel, kvm, linux-pci,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Sean Christopherson, Paolo Bonzini,
Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas, Marek Szyprowski,
Andrew Morton, Catalin Marinas, Michael Ellerman, Mike Rapoport,
Tom Lendacky, Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra,
Stefano Garzarella, Melody Wang, Seongman Lee, Joerg Roedel,
Nikunj A Dadhania, Michael Roth, Suravee Suthikulpanit,
Andi Kleen, Kuppuswamy Sathyanarayanan, Tony Luck,
David Woodhouse, Greg Kroah-Hartman, Denis Efremov, Geliang Tang,
Piotr Gregor, Michael S. Tsirkin, Alex Williamson, Arnd Bergmann,
Jesse Barnes, Jacob Pan, Yinghai Lu, Kevin Brodsky,
Jonathan Cameron, Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu,
Kim Phillips, Konrad Rzeszutek Wilk, Stefano Stabellini,
Claire Chang, linux-coco, iommu
In-Reply-To: <5d669086-a5c8-4e55-8108-a9fff41cf094@amd.com>
On Wed, Mar 04, 2026 at 05:45:31PM +1100, Alexey Kardashevskiy wrote:
> > I suspect AMD needs to use their vTOM feature to allow shared memory
> > to remain available to TDISP RUN with a high/low address split.
>
> I could probably do something about it bit I wonder what is the real
> live use case which requires leaking SME mask, have a live example
> which I could try recreating?
We need shared memory allocated through a DMABUF heap:
https://lore.kernel.org/all/20260223095136.225277-1-jiri@resnulli.us/
To work with all PCI devices in the system, TDISP or not.
Without this the ability for a TDISP device to ingest (encrypted) data
requires all kinds of memcpy..
So the DMA API should see the DMA_ATTR_CC_DECRYPTED and setup the
correct dma_dddr_t either by choosing the shared alias for the TDISP
device's vTOM, or setting the C bit in a vIOMMU S1.
Jason
^ permalink raw reply
* Re: [PATCH v12 06/46] arm64: RMI: Define the user ABI
From: Steven Price @ 2026-03-04 12:08 UTC (permalink / raw)
To: Marc Zyngier
Cc: kvm, kvmarm, Catalin Marinas, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <86h5qx83df.wl-maz@kernel.org>
On 03/03/2026 13:11, Marc Zyngier wrote:
> On Mon, 02 Mar 2026 15:23:44 +0000,
> Steven Price <steven.price@arm.com> wrote:
>>
>> Hi Marc,
>>
>> On 02/03/2026 14:25, Marc Zyngier wrote:
>>> On Wed, 17 Dec 2025 10:10:43 +0000,
>>> Steven Price <steven.price@arm.com> wrote:
>>>>
>>>> There is one CAP which identified the presence of CCA, and two ioctls.
>>>> One ioctl is used to populate memory and the other is used when user
>>>> space is providing the PSCI implementation to identify the target of the
>>>> operation.
>>>>
>>>> Signed-off-by: Steven Price <steven.price@arm.com>
>>>> ---
>>>> Changes since v11:
>>>> * Completely reworked to be more implicit. Rather than having explicit
>>>> CAP operations to progress the realm construction these operations
>>>> are done when needed (on populating and on first vCPU run).
>>>> * Populate and PSCI complete are promoted to proper ioctls.
>>>> Changes since v10:
>>>> * Rename symbols from RME to RMI.
>>>> Changes since v9:
>>>> * Improvements to documentation.
>>>> * Bump the magic number for KVM_CAP_ARM_RME to avoid conflicts.
>>>> Changes since v8:
>>>> * Minor improvements to documentation following review.
>>>> * Bump the magic numbers to avoid conflicts.
>>>> Changes since v7:
>>>> * Add documentation of new ioctls
>>>> * Bump the magic numbers to avoid conflicts
>>>> Changes since v6:
>>>> * Rename some of the symbols to make their usage clearer and avoid
>>>> repetition.
>>>> Changes from v5:
>>>> * Actually expose the new VCPU capability (KVM_ARM_VCPU_REC) by bumping
>>>> KVM_VCPU_MAX_FEATURES - note this also exposes KVM_ARM_VCPU_HAS_EL2!
>>>> ---
>>>> Documentation/virt/kvm/api.rst | 57 ++++++++++++++++++++++++++++++++++
>>>> include/uapi/linux/kvm.h | 23 ++++++++++++++
>>>> 2 files changed, 80 insertions(+)
>>>>
>>>> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
>>>> index 01a3abef8abb..2d5dc7e48954 100644
>>>> --- a/Documentation/virt/kvm/api.rst
>>>> +++ b/Documentation/virt/kvm/api.rst
>>>> @@ -6517,6 +6517,54 @@ the capability to be present.
>>>>
>>>> `flags` must currently be zero.
>>>>
>>>> +4.144 KVM_ARM_VCPU_RMI_PSCI_COMPLETE
>>>> +------------------------------------
>>>> +
>>>> +:Capability: KVM_CAP_ARM_RMI
>>>> +:Architectures: arm64
>>>> +:Type: vcpu ioctl
>>>> +:Parameters: struct kvm_arm_rmi_psci_complete (in)
>>>> +:Returns: 0 if successful, < 0 on error
>>>> +
>>>> +::
>>>> +
>>>> + struct kvm_arm_rmi_psci_complete {
>>>> + __u64 target_mpidr;
>>>> + __u32 psci_status;
>>>> + __u32 padding[3];
>>>> + };
>>>> +
>>>> +Where PSCI functions are handled by user space, the RMM needs to be informed of
>>>> +the target of the operation using `target_mpidr`, along with the status
>>>> +(`psci_status`). The RMM v1.0 specification defines two functions that require
>>>> +this call: PSCI_CPU_ON and PSCI_AFFINITY_INFO.
>>>> +
>>>> +If the kernel is handling PSCI then this is done automatically and the VMM
>>>> +doesn't need to call this ioctl.
>>>
>>> Shouldn't we make handling of PSCI mandatory for VMMs that deal with
>>> CCA? I suspect it would simplify the implementation significantly.
>>
>> What do you mean by making it "mandatory for VMMs"? If you mean PSCI is
>> always forwarded to user space then I don't think it's going to make
>> much difference. Patch 27 handles the PSCI changes (72 lines added), and
>> some of that is adding this uAPI for the VMM to handle it.
>>
>> Removing the functionality to allow the VMM to handle it would obviously
>> simplify things a bit (we can drop this uAPI), but I think the desire is
>> to push this onto user space.
>
> And that's what I'm asking for. I do not want this to be optional. CCA
> should implies PSCI in userspace, and that's it.
>
>>
>>> What vcpu fd does this apply to? The vcpu calling the PSCI function?
>>> Or the target? This is pretty important for PSCI_ON. My guess is that
>>> this is setting the return value for the caller?
>>
>> Yes the fd is the vcpu calling PSCI. As you say, this is for the return
>> value to be set correctly.
>
> Another question is why do we need the ioctl at all? Why can't it be
> done on the first run of the target vcpu? If no PSCI call was issued
> to run it, then the run fails.
So my concern is the ordering of operations for PSCI_CPU_ON. As things
stand the RMM needs to know the MPIDR mapping to look up the REC object
before either VCPU runs again.
If we do this on the first run of the target VCPU, then how is the VMM
to tell that the target VCPU has executed "long enough" that it is safe
to do the return on the initial VCPU? Since the VCPUs are different
threads this becomes tricky. Options I can see are:
a) The VMM has to wait for the target VCPU to exit - we'd probably want
to trigger an artificial early exit in this case to unblock things.
b) The kernel blocks the initial VCPU from running until the target VCPU
has completed this "first run" logic. I think waiting in the kernel is
probably problematic, so this implies return some sort of "retry later"
response to the VMM.
c) The kernel handles the "PSCI_COMPLETE" dance on whichever VCPU runs
first, blocking the other until the dance is complete. A disadvantage
here is that behaviour can differ (in error conditions) depending on
which VCPU thread wins the race.
All these options also involve the kernel keeping track of the PSCI
sequence, in particular:
1. Tracking that the exit was due to a PSCI_CPU_ON.
2. Treating attempting to run the target VCPU as an implicit success
return from the PSCI call.
3. Recognising the next run on the initial VCPU as containing the PSCI
result - if 2, above, has happened then the kernel will need to handle
this (by killing the guest).
TLDR; Yes this is possible but I don't think it's pretty, and I'm not
convinced it's an improved uAPI.
Of course the above all assumes that the RMM can't just track things
internally. My preference is to kill RMI_PSCI_COMPLETE altogether, but
I'm not sure how possible that is within the context of the RMM.
>>
>>> Assuming this is indeed for the caller, why do we have a different
>>> flow from anything else that returns a result from a hypercall?
>>
>> I'm not entirely sure what you are suggesting. Do you mean why are we
>> not just writing to the GPRS that would contain the result? The issue
>> here is that the RMM needs to know the PA of the target REC structure -
>> this isn't a return to the guest, but information for the RMM itself to
>> complete the PSCI call.
>
> PSCI is a SMC call. SMC calls are routed to userspace as such. For odd
> reasons, the RMM treats PSCI differently from any other SMC call.
>
> That seems a very bizarre behaviour to me.
The RMM generally treats SMC specially. We have the RSI_HOST_CALL as a
proxy for "general SMC-like" behaviour which is forwarded to the VMM. I
believe the intention here is to ensure that SMCs (from the realm guest)
are handled by a trusted agent (i.e. the RMM). PSCI is a corner case
because it requires some coordination and buy-in from the VMM.
I'm not sure I fully understand the security pros and cons of the design
here and what impact it would have if PSCI was well trusted.
Thanks,
Steve
>>
>> Ultimately even in the case where the VMM is handling PSCI, it's
>> actually a combination of the VMM and the RMM - with the RMM validating
>> the responses.
>
> I don't see why PSCI is singled out here, irrespective of the tracking
> that the RMM wants to do.
>
> M.
>
^ permalink raw reply
* Re: [PATCH kernel 4/9] dma/swiotlb: Stop forcing SWIOTLB for TDISP devices
From: Alexey Kardashevskiy @ 2026-03-04 6:45 UTC (permalink / raw)
To: Jason Gunthorpe, dan.j.williams
Cc: Robin Murphy, x86, linux-kernel, kvm, linux-pci, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Dave Hansen, H. Peter Anvin,
Sean Christopherson, Paolo Bonzini, Andy Lutomirski,
Peter Zijlstra, Bjorn Helgaas, Marek Szyprowski, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu
In-Reply-To: <20260303124306.GA1002356@nvidia.com>
On 3/3/26 23:43, Jason Gunthorpe wrote:
> On Mon, Mar 02, 2026 at 08:19:11PM -0400, Jason Gunthorpe wrote:
>>> Oh, I thought SEV-TIO had trouble with this, if this is indeed the case,
>>> great, ignore my first comment.
>>
>> Alexey?
>>
>> I think it is really important that shared mappings continue to be
>> reachable by TDISP device.
>
> I think Alexey has clarified this in the other thread, and probably
> AMD has some work to do here.
>
> The issue is AMD does not have seperate address spaces for
> shared/private like ARM does, instead it relies on a C bit in the *PTE*
> to determine shared/private.
>
> The S2 IOMMU page table *does* have the full mapping of all shared &
> private pages but the HW requires a matching C bit to permit access.
>
> If there is a S1 IOMMU then the IOPTEs of the VM can provide the C
> bit, so no problem.
>
> If there is no S1 then the sDTE of the hypervisor controls the C bit,
sDTE is controlled by the FW (not the HV) on the VM behalf - the VM chooses whether to enable sDTE and therefore vTOM.
> and it sounds like currently AMD sets this globally which effectively
> locks TDISP RUN devices to *only* access private memory.
Right. The assumption is that if the guest wants finer control - there is secure vIOMMU (in the works).
> I suspect AMD needs to use their vTOM feature to allow shared memory
> to remain available to TDISP RUN with a high/low address split.
I could probably do something about it bit I wonder what is the real live use case which requires leaking SME mask, have a live example which I could try recreating?
> Alexey, did I capture this properly?
Yes, with the correction about sDTE above. Thanks,
>
> Jason
--
Alexey
^ permalink raw reply
* Re: [PATCH v3 00/16] KVM: x86/tdx: Have TDX handle VMXON during bringup
From: Sagi Shahar @ 2026-03-04 0:06 UTC (permalink / raw)
To: Sean Christopherson
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Peter Zijlstra, Arnaldo Carvalho de Melo,
Namhyung Kim, Paolo Bonzini, linux-kernel, linux-coco, kvm,
linux-perf-users, Chao Gao, Xu Yilun, Dan Williams
In-Reply-To: <CAAhR5DF5BAcFO2tj0H63ZoRCcdpDS4Jw9XzqC=L2xWMW0M=0QQ@mail.gmail.com>
On Tue, Mar 3, 2026 at 3:39 PM Sagi Shahar <sagis@google.com> wrote:
>
> On Fri, Feb 13, 2026 at 7:27 PM Sean Christopherson <seanjc@google.com> wrote:
> >
> > Assuming I didn't break anything between v2 and v3, I think this is ready to
> > rip. Given the scope of the KVM changes, and that they extend outside of x86,
> > my preference is to take this through the KVM tree. But a stable topic branch
> > in tip would work too, though I think we'd want it sooner than later so that
> > it can be used as a base.
> >
> > Chao, I deliberately omitted your Tested-by, as I shuffled things around enough
> > while splitting up the main patch that I'm not 100% positive I didn't regress
> > anything relative to v2.
>
> Tested running TDs and TDX module update using "Runtime TDX Module
> update support" patches [1]
> Tested-by: Sagi Shahar <sagishah@gmail.com>
>
> [1] https://lore.kernel.org/lkml/20260123145645.90444-1-chao.gao@intel.com/
Actually, looking at the "Runtime TDX Module update support" patches I
don't think I ran those with this version of the patches since the
"tdx_module_status" changes are incompatible. So it was just the
patches in this patchset.
^ permalink raw reply
* Re: [PATCH v2 00/19] PCI/TSM: TEE I/O infrastructure
From: dan.j.williams @ 2026-03-03 22:01 UTC (permalink / raw)
To: Aneesh Kumar K.V, Dan Williams, linux-coco, linux-pci
Cc: gregkh, aik, yilun.xu, bhelgaas, alistair23, lukas, jgg,
Andy Lutomirski, Arnd Bergmann, Borislav Petkov,
Christoph Hellwig, Danilo Krummrich, Dave Hansen, Donald Hunter,
H. Peter Anvin, Ingo Molnar, Jakub Kicinski, Jason Gunthorpe,
Luis Chamberlain, Marek Szyprowski, Peter Zijlstra,
Rafael J. Wysocki, Robin Murphy, Roman Kisel, Samuel Ortiz,
Saravana Kannan, Suzuki K Poulose, Thomas Gleixner,
Thomas Gleixner
In-Reply-To: <yq5aecm19shr.fsf@kernel.org>
Aneesh Kumar K.V wrote:
> Dan Williams <dan.j.williams@intel.com> writes:
>
> ....
>
> To support devices without IDE/DOE support we need something similar.
>
> modified drivers/pci/tsm/core.c
> @@ -1236,12 +1236,14 @@ int pci_tsm_pf0_constructor(struct pci_dev *pdev, struct pci_tsm_pf0 *tsm,
> struct tsm_dev *tsm_dev)
> {
> mutex_init(&tsm->lock);
> - tsm->doe_mb = pci_find_doe_mailbox(pdev, PCI_VENDOR_ID_PCI_SIG,
> - PCI_DOE_FEATURE_CMA);
> - if (!tsm->doe_mb) {
> - pci_warn(pdev, "TSM init failure, no CMA mailbox\n");
> - return -ENODEV;
> - }
> +
> + /*
> + * Note, low-level TSM driver responsible for determining if it wants to
> + * proceed with a device that has no DOE mailbox. TSM may have an
> + * alternate method for coordinating TDISP.
> + */
> + if (!tsm->doe_mb)
> + pci_dbg(pdev, "no CMA mailbox\n");
A patch like patch can go upstream now. Care to send?
^ permalink raw reply
* Re: [PATCH v3 00/16] KVM: x86/tdx: Have TDX handle VMXON during bringup
From: Sagi Shahar @ 2026-03-03 21:39 UTC (permalink / raw)
To: Sean Christopherson
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Peter Zijlstra, Arnaldo Carvalho de Melo,
Namhyung Kim, Paolo Bonzini, linux-kernel, linux-coco, kvm,
linux-perf-users, Chao Gao, Xu Yilun, Dan Williams
In-Reply-To: <20260214012702.2368778-1-seanjc@google.com>
On Fri, Feb 13, 2026 at 7:27 PM Sean Christopherson <seanjc@google.com> wrote:
>
> Assuming I didn't break anything between v2 and v3, I think this is ready to
> rip. Given the scope of the KVM changes, and that they extend outside of x86,
> my preference is to take this through the KVM tree. But a stable topic branch
> in tip would work too, though I think we'd want it sooner than later so that
> it can be used as a base.
>
> Chao, I deliberately omitted your Tested-by, as I shuffled things around enough
> while splitting up the main patch that I'm not 100% positive I didn't regress
> anything relative to v2.
Tested running TDs and TDX module update using "Runtime TDX Module
update support" patches [1]
Tested-by: Sagi Shahar <sagishah@gmail.com>
[1] https://lore.kernel.org/lkml/20260123145645.90444-1-chao.gao@intel.com/
^ permalink raw reply
* Re: [PATCH 14/14] KVM: x86: Add helpers to prepare kvm_run for userspace MMIO exit
From: Edgecombe, Rick P @ 2026-03-03 19:51 UTC (permalink / raw)
To: seanjc@google.com
Cc: x86@kernel.org, zhangjiaji1@huawei.com, kas@kernel.org,
Li, Xiaoyao, linux-kernel@vger.kernel.org,
thomas.lendacky@amd.com, pbonzini@redhat.com,
linux-coco@lists.linux.dev, michael.roth@amd.com,
kvm@vger.kernel.org, binbin.wu@linux.intel.com
In-Reply-To: <aac6DGISvMX4krhb@google.com>
On Tue, 2026-03-03 at 11:44 -0800, Sean Christopherson wrote:
> >
> > Seems ok and an improvement over the patch. But looking at the other
> > callers, there is quite a bit of min(8u, len) logic spread around. Might be
> > worth a wider cleanup someday.
>
> LOL, "might". :-)
Trying to not set you off... :)
>
> Definitely a project for the future though, especially given how subtle and
> brittle this all is.
^ permalink raw reply
* Re: [PATCH 14/14] KVM: x86: Add helpers to prepare kvm_run for userspace MMIO exit
From: Sean Christopherson @ 2026-03-03 19:44 UTC (permalink / raw)
To: Rick P Edgecombe
Cc: x86@kernel.org, zhangjiaji1@huawei.com, kas@kernel.org,
Xiaoyao Li, linux-kernel@vger.kernel.org, thomas.lendacky@amd.com,
binbin.wu@linux.intel.com, pbonzini@redhat.com,
linux-coco@lists.linux.dev, kvm@vger.kernel.org,
michael.roth@amd.com
In-Reply-To: <4574be9a29d75d565e553579ef6ce915ef33b19b.camel@intel.com>
On Tue, Mar 03, 2026, Rick P Edgecombe wrote:
> On Mon, 2026-03-02 at 18:24 -0800, Sean Christopherson wrote:
> > Ooh, better idea. Since TDX is the only direct user of
> > __kvm_prepare_emulated_mmio_exit() and it only supports lenths of 1, 2, 4, and 8,
> > kvm_prepare_emulated_mmio_exit() is the only path that actually needs to cap the
> > length. Then the inner helper can assert a valid length. Doesn't change anything
> > in practice, but I like the idea of making the caller be aware of the limitation
> > (even if that caller is itself a helper).
>
> Seems ok and an improvement over the patch. But looking at the other callers,
> there is quite a bit of min(8u, len) logic spread around. Might be worth a wider
> cleanup someday.
LOL, "might". :-)
Definitely a project for the future though, especially given how subtle and brittle
this all is.
^ permalink raw reply
* Re: [PATCH 14/14] KVM: x86: Add helpers to prepare kvm_run for userspace MMIO exit
From: Edgecombe, Rick P @ 2026-03-03 19:21 UTC (permalink / raw)
To: seanjc@google.com
Cc: x86@kernel.org, zhangjiaji1@huawei.com, kas@kernel.org,
Li, Xiaoyao, linux-kernel@vger.kernel.org,
thomas.lendacky@amd.com, binbin.wu@linux.intel.com,
pbonzini@redhat.com, linux-coco@lists.linux.dev,
kvm@vger.kernel.org, michael.roth@amd.com
In-Reply-To: <aaZGTY3CzhaCb1lc@google.com>
On Mon, 2026-03-02 at 18:24 -0800, Sean Christopherson wrote:
> Ooh, better idea. Since TDX is the only direct user of
> __kvm_prepare_emulated_mmio_exit() and it only supports lenths of 1, 2, 4, and 8,
> kvm_prepare_emulated_mmio_exit() is the only path that actually needs to cap the
> length. Then the inner helper can assert a valid length. Doesn't change anything
> in practice, but I like the idea of making the caller be aware of the limitation
> (even if that caller is itself a helper).
Seems ok and an improvement over the patch. But looking at the other callers,
there is quite a bit of min(8u, len) logic spread around. Might be worth a wider
cleanup someday.
^ permalink raw reply
* SVSM Development Call March 4, 2026
From: Jörg Rödel @ 2026-03-03 17:07 UTC (permalink / raw)
To: coconut-svsm, linux-coco
Hi,
Here is the call for agenda items for this weeks SVSM development call. Please
send any agenda items you have in mind as a reply to this email or raise them
in the meeting.
We will use the LF Zoom instance. Details of the meeting can be found in our
governance repository at:
https://github.com/coconut-svsm/governance
The link to the COCONUT-SVSM calendar is:
https://zoom-lfx.platform.linuxfoundation.org/meetings/coconut-svsm?view=week
The meeting will be recorded and the recording eventually published.
Regards,
Jörg
^ permalink raw reply
* Re: [PATCH v2 08/19] PCI/TSM: Add "evidence" support
From: Aneesh Kumar K.V @ 2026-03-03 16:38 UTC (permalink / raw)
To: Dan Williams, linux-coco, linux-pci
Cc: gregkh, aik, yilun.xu, bhelgaas, alistair23, lukas, jgg,
Donald Hunter, Jakub Kicinski
In-Reply-To: <20260303000207.1836586-9-dan.j.williams@intel.com>
Dan Williams <dan.j.williams@intel.com> writes:
> Once one accepts the threat model that devices may be adversarial the
> process of establishing trust in the device identity, the integrity +
> confidentiality of its link, and the integrity + confidentiality of its
> MMIO interface requires multiple evidence objects from the device. The
> device's certificate chain, measurements and interface report need to be
> retrieved by the host, validated by the TSM and transmitted to the guest
> all while mitigating TOCTOU races.
>
> All TSM implementations share the same fundamental objects, but vary in how
> the TSM conveys its trust in the objects. Some TSM implementations expect
> the full documents to be conveyed over untrustworthy channels while the TSM
> securely conveys a digest. Others transmit full objects with signed SPDM
> transcripts of requester provided nonces. Some offer a single transcript
> to convey the version, capabilities, and algorithms (VCA) data and
> measurements in one blob while others split VCA as a separate signed blob.
>
> Introduce a netlink interface to dump all these objects in a common way
> across TSM implementations and across host and guest environments.
> Userspace is responsible for handling the variance of "TSM provides combo
> measurements + VCA + nonce + signature, vs TSM provides a digest over a
> secure channel of the same".
>
> The implementation adheres to the guideline from:
> Documentation/userspace-api/netlink/genetlink-legacy.rst
>
> New Netlink families should never respond to a DO operation with
> multiple replies, with ``NLM_F_MULTI`` set. Use a filtered dump
> instead.
>
> Per SPDM, transcripts may grow to be 16MB in size. Large PCI/TSM netlink
> blobs are handled via a sequence of dump messages that userspace must
> concatenate.
>
Should we also expose evidence->generation to userspace so it can be
used during accept()? This would allow us to ensure that the device is
accepted using the same evidence generation observed by userspace.
-aneesh
^ permalink raw reply
* Re: [PATCH v12 06/46] arm64: RMI: Define the user ABI
From: Suzuki K Poulose @ 2026-03-03 16:02 UTC (permalink / raw)
To: Marc Zyngier
Cc: Steven Price, kvm, kvmarm, Catalin Marinas, Will Deacon,
James Morse, Oliver Upton, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <86ecm17zeb.wl-maz@kernel.org>
On 03/03/2026 14:37, Marc Zyngier wrote:
> On Tue, 03 Mar 2026 14:23:08 +0000,
> Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
>>
>> On 03/03/2026 13:13, Marc Zyngier wrote:
>>> On Mon, 02 Mar 2026 17:13:41 +0000,
>>> Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
>>>>
>>>> More importantly, we have to make sure that the "RMI_PSCI_COMPLETE" is
>>>> invoked before both of the following:
>>>> 1. The "source" vCPU is run again
>>>> 2. More importantly the "target" vCPU is run.
>>>
>>> I don't understand why (1) is required. Once the VMM gets the request,
>>
>> The underlying issue is, the RMM doesn't have the VCPU object for the
>> "target" VCPU, to make the book keeping. Also, please note that for a
>> Realm, PSCI is emulated by the "RMM". Host is obviously notified of the
>> "PSCI" changes via EXIT_PSCI (note, it is not SMCCC exit)
>> so that it can be in sync with the real state. And does have a say in
>> CPU_ON. So, before we return to running the "source" CPU,
>> Host must provide the target VCPU object and its consent (via
>> psci_status) to the RMM. This allows the RMM to emulate the PSCI
>> request correctly and also at the same time keep its book keeping
>> in tact (i.e., marking the Target VCPU as runnable or not).
>>
>> When a "source" VCPU exits to the host with a PSCI_EXIT, the RMM
>> marks the source VCPU has a pending PSCI operation, and
>> RMI_PSCI_COMPLETE request ticks that off, making it runnable again.
>
> Sure. What I don't get is what this has to happen on the source vcpu
> thread. The RMM has absolutely no clue about that, and there should be
> no impediment to letting the target vcpu do it as it starts.
Because, the RMM wants to make that the state is consistent. i.e,
Host cannot lie to the "source" VCPU and RMM (e.g., CPU_ON denied) and
then run the "target" VCPU.
In other words, the response to the CPU_ON must be recorded by the RMM
in the target VCPU state, to make sure the target VCPU state is
consistent with the response.
The only way to fix this would be RMM keeping track of "mpidr" to REC
object mapping (VCPU object), which impacts the scalability. With that
in place, RMM can update the target VCPU state from the return of the
REC_ENTER after a PSCI_EXIT.
That said, will explore the options to address this
Thanks
Suzuki
>
> Even better, you should be able to do that on the first thread that
> reenters the guest, completely removing any RMM knowledge from the
> PSCI handling in userspace.
>
> If you can't do that, then please consider fixing the RMM to allow it.
>
> Thanks,
>
> M.
>
^ permalink raw reply
* Re: [PATCH v12 06/46] arm64: RMI: Define the user ABI
From: Marc Zyngier @ 2026-03-03 14:37 UTC (permalink / raw)
To: Suzuki K Poulose
Cc: Steven Price, kvm, kvmarm, Catalin Marinas, Will Deacon,
James Morse, Oliver Upton, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <d87ee902-3b5e-4cf9-8b97-d83f8da02a5a@arm.com>
On Tue, 03 Mar 2026 14:23:08 +0000,
Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
>
> On 03/03/2026 13:13, Marc Zyngier wrote:
> > On Mon, 02 Mar 2026 17:13:41 +0000,
> > Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
> >>
> >> More importantly, we have to make sure that the "RMI_PSCI_COMPLETE" is
> >> invoked before both of the following:
> >> 1. The "source" vCPU is run again
> >> 2. More importantly the "target" vCPU is run.
> >
> > I don't understand why (1) is required. Once the VMM gets the request,
>
> The underlying issue is, the RMM doesn't have the VCPU object for the
> "target" VCPU, to make the book keeping. Also, please note that for a
> Realm, PSCI is emulated by the "RMM". Host is obviously notified of the
> "PSCI" changes via EXIT_PSCI (note, it is not SMCCC exit)
> so that it can be in sync with the real state. And does have a say in
> CPU_ON. So, before we return to running the "source" CPU,
> Host must provide the target VCPU object and its consent (via
> psci_status) to the RMM. This allows the RMM to emulate the PSCI
> request correctly and also at the same time keep its book keeping
> in tact (i.e., marking the Target VCPU as runnable or not).
>
> When a "source" VCPU exits to the host with a PSCI_EXIT, the RMM
> marks the source VCPU has a pending PSCI operation, and
> RMI_PSCI_COMPLETE request ticks that off, making it runnable again.
Sure. What I don't get is what this has to happen on the source vcpu
thread. The RMM has absolutely no clue about that, and there should be
no impediment to letting the target vcpu do it as it starts.
Even better, you should be able to do that on the first thread that
reenters the guest, completely removing any RMM knowledge from the
PSCI handling in userspace.
If you can't do that, then please consider fixing the RMM to allow it.
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply
* Re: [PATCH v12 06/46] arm64: RMI: Define the user ABI
From: Suzuki K Poulose @ 2026-03-03 14:23 UTC (permalink / raw)
To: Marc Zyngier
Cc: Steven Price, kvm, kvmarm, Catalin Marinas, Will Deacon,
James Morse, Oliver Upton, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <86fr6h838s.wl-maz@kernel.org>
On 03/03/2026 13:13, Marc Zyngier wrote:
> On Mon, 02 Mar 2026 17:13:41 +0000,
> Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
>>
>> On 02/03/2026 15:23, Steven Price wrote:
>>> Hi Marc,
>>>
>>> On 02/03/2026 14:25, Marc Zyngier wrote:
>>>> On Wed, 17 Dec 2025 10:10:43 +0000,
>>>> Steven Price <steven.price@arm.com> wrote:
>>>>>
>>>>> There is one CAP which identified the presence of CCA, and two ioctls.
>>>>> One ioctl is used to populate memory and the other is used when user
>>>>> space is providing the PSCI implementation to identify the target of the
>>>>> operation.
>>>>>
>>>>> Signed-off-by: Steven Price <steven.price@arm.com>
>>>>> ---
>>>>> Changes since v11:
>>>>> * Completely reworked to be more implicit. Rather than having explicit
>>>>> CAP operations to progress the realm construction these operations
>>>>> are done when needed (on populating and on first vCPU run).
>>>>> * Populate and PSCI complete are promoted to proper ioctls.
>>>>> Changes since v10:
>>>>> * Rename symbols from RME to RMI.
>>>>> Changes since v9:
>>>>> * Improvements to documentation.
>>>>> * Bump the magic number for KVM_CAP_ARM_RME to avoid conflicts.
>>>>> Changes since v8:
>>>>> * Minor improvements to documentation following review.
>>>>> * Bump the magic numbers to avoid conflicts.
>>>>> Changes since v7:
>>>>> * Add documentation of new ioctls
>>>>> * Bump the magic numbers to avoid conflicts
>>>>> Changes since v6:
>>>>> * Rename some of the symbols to make their usage clearer and avoid
>>>>> repetition.
>>>>> Changes from v5:
>>>>> * Actually expose the new VCPU capability (KVM_ARM_VCPU_REC) by bumping
>>>>> KVM_VCPU_MAX_FEATURES - note this also exposes KVM_ARM_VCPU_HAS_EL2!
>>>>> ---
>>>>> Documentation/virt/kvm/api.rst | 57 ++++++++++++++++++++++++++++++++++
>>>>> include/uapi/linux/kvm.h | 23 ++++++++++++++
>>>>> 2 files changed, 80 insertions(+)
>>>>>
>>>>> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
>>>>> index 01a3abef8abb..2d5dc7e48954 100644
>>>>> --- a/Documentation/virt/kvm/api.rst
>>>>> +++ b/Documentation/virt/kvm/api.rst
>>>>> @@ -6517,6 +6517,54 @@ the capability to be present.
>>>>> `flags` must currently be zero.
>>>>> +4.144 KVM_ARM_VCPU_RMI_PSCI_COMPLETE
>>>>> +------------------------------------
>>>>> +
>>>>> +:Capability: KVM_CAP_ARM_RMI
>>>>> +:Architectures: arm64
>>>>> +:Type: vcpu ioctl
>>>>> +:Parameters: struct kvm_arm_rmi_psci_complete (in)
>>>>> +:Returns: 0 if successful, < 0 on error
>>>>> +
>>>>> +::
>>>>> +
>>>>> + struct kvm_arm_rmi_psci_complete {
>>>>> + __u64 target_mpidr;
>>>>> + __u32 psci_status;
>>>>> + __u32 padding[3];
>>>>> + };
>>>>> +
>>>>> +Where PSCI functions are handled by user space, the RMM needs to be informed of
>>>>> +the target of the operation using `target_mpidr`, along with the status
>>>>> +(`psci_status`). The RMM v1.0 specification defines two functions that require
>>>>> +this call: PSCI_CPU_ON and PSCI_AFFINITY_INFO.
>>>>> +
>>>>> +If the kernel is handling PSCI then this is done automatically and the VMM
>>>>> +doesn't need to call this ioctl.
>>>>
>>>> Shouldn't we make handling of PSCI mandatory for VMMs that deal with
>>>> CCA? I suspect it would simplify the implementation significantly.
>>>
>>> What do you mean by making it "mandatory for VMMs"? If you mean PSCI is
>>> always forwarded to user space then I don't think it's going to make
>>> much difference. Patch 27 handles the PSCI changes (72 lines added), and
>>> some of that is adding this uAPI for the VMM to handle it.
>>>
>>> Removing the functionality to allow the VMM to handle it would obviously
>>> simplify things a bit (we can drop this uAPI), but I think the desire is
>>> to push this onto user space.
>>>
>>>> What vcpu fd does this apply to? The vcpu calling the PSCI function?
>>>> Or the target? This is pretty important for PSCI_ON. My guess is that
>>>> this is setting the return value for the caller?
>>>
>>> Yes the fd is the vcpu calling PSCI. As you say, this is for the return
>>> value to be set correctly.
>>>
>>>> Assuming this is indeed for the caller, why do we have a different
>>>> flow from anything else that returns a result from a hypercall?
>>>
>>> I'm not entirely sure what you are suggesting. Do you mean why are we
>>> not just writing to the GPRS that would contain the result? The issue
>>> here is that the RMM needs to know the PA of the target REC structure -
>>> this isn't a return to the guest, but information for the RMM itself to
>>> complete the PSCI call.
>>>
>>> Ultimately even in the case where the VMM is handling PSCI, it's
>>> actually a combination of the VMM and the RMM - with the RMM validating
>>> the responses.
>>>
>>
>> More importantly, we have to make sure that the "RMI_PSCI_COMPLETE" is
>> invoked before both of the following:
>> 1. The "source" vCPU is run again
>> 2. More importantly the "target" vCPU is run.
>
> I don't understand why (1) is required. Once the VMM gets the request,
The underlying issue is, the RMM doesn't have the VCPU object for the
"target" VCPU, to make the book keeping. Also, please note that for a
Realm, PSCI is emulated by the "RMM". Host is obviously notified of the
"PSCI" changes via EXIT_PSCI (note, it is not SMCCC exit)
so that it can be in sync with the real state. And does have a say in
CPU_ON. So, before we return to running the "source" CPU,
Host must provide the target VCPU object and its consent (via
psci_status) to the RMM. This allows the RMM to emulate the PSCI
request correctly and also at the same time keep its book keeping
in tact (i.e., marking the Target VCPU as runnable or not).
When a "source" VCPU exits to the host with a PSCI_EXIT, the RMM
marks the source VCPU has a pending PSCI operation, and
RMI_PSCI_COMPLETE request ticks that off, making it runnable again.
Suzuki
> the target vcpu can run, and can itself do the completion, without any
> additional userspace involvement.
>
> M.
>
^ permalink raw reply
* Re: [PATCH v12 06/46] arm64: RMI: Define the user ABI
From: Marc Zyngier @ 2026-03-03 13:13 UTC (permalink / raw)
To: Suzuki K Poulose
Cc: Steven Price, kvm, kvmarm, Catalin Marinas, Will Deacon,
James Morse, Oliver Upton, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <9d702666-72a8-43e4-8ab3-548d8154a529@arm.com>
On Mon, 02 Mar 2026 17:13:41 +0000,
Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
>
> On 02/03/2026 15:23, Steven Price wrote:
> > Hi Marc,
> >
> > On 02/03/2026 14:25, Marc Zyngier wrote:
> >> On Wed, 17 Dec 2025 10:10:43 +0000,
> >> Steven Price <steven.price@arm.com> wrote:
> >>>
> >>> There is one CAP which identified the presence of CCA, and two ioctls.
> >>> One ioctl is used to populate memory and the other is used when user
> >>> space is providing the PSCI implementation to identify the target of the
> >>> operation.
> >>>
> >>> Signed-off-by: Steven Price <steven.price@arm.com>
> >>> ---
> >>> Changes since v11:
> >>> * Completely reworked to be more implicit. Rather than having explicit
> >>> CAP operations to progress the realm construction these operations
> >>> are done when needed (on populating and on first vCPU run).
> >>> * Populate and PSCI complete are promoted to proper ioctls.
> >>> Changes since v10:
> >>> * Rename symbols from RME to RMI.
> >>> Changes since v9:
> >>> * Improvements to documentation.
> >>> * Bump the magic number for KVM_CAP_ARM_RME to avoid conflicts.
> >>> Changes since v8:
> >>> * Minor improvements to documentation following review.
> >>> * Bump the magic numbers to avoid conflicts.
> >>> Changes since v7:
> >>> * Add documentation of new ioctls
> >>> * Bump the magic numbers to avoid conflicts
> >>> Changes since v6:
> >>> * Rename some of the symbols to make their usage clearer and avoid
> >>> repetition.
> >>> Changes from v5:
> >>> * Actually expose the new VCPU capability (KVM_ARM_VCPU_REC) by bumping
> >>> KVM_VCPU_MAX_FEATURES - note this also exposes KVM_ARM_VCPU_HAS_EL2!
> >>> ---
> >>> Documentation/virt/kvm/api.rst | 57 ++++++++++++++++++++++++++++++++++
> >>> include/uapi/linux/kvm.h | 23 ++++++++++++++
> >>> 2 files changed, 80 insertions(+)
> >>>
> >>> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
> >>> index 01a3abef8abb..2d5dc7e48954 100644
> >>> --- a/Documentation/virt/kvm/api.rst
> >>> +++ b/Documentation/virt/kvm/api.rst
> >>> @@ -6517,6 +6517,54 @@ the capability to be present.
> >>> `flags` must currently be zero.
> >>> +4.144 KVM_ARM_VCPU_RMI_PSCI_COMPLETE
> >>> +------------------------------------
> >>> +
> >>> +:Capability: KVM_CAP_ARM_RMI
> >>> +:Architectures: arm64
> >>> +:Type: vcpu ioctl
> >>> +:Parameters: struct kvm_arm_rmi_psci_complete (in)
> >>> +:Returns: 0 if successful, < 0 on error
> >>> +
> >>> +::
> >>> +
> >>> + struct kvm_arm_rmi_psci_complete {
> >>> + __u64 target_mpidr;
> >>> + __u32 psci_status;
> >>> + __u32 padding[3];
> >>> + };
> >>> +
> >>> +Where PSCI functions are handled by user space, the RMM needs to be informed of
> >>> +the target of the operation using `target_mpidr`, along with the status
> >>> +(`psci_status`). The RMM v1.0 specification defines two functions that require
> >>> +this call: PSCI_CPU_ON and PSCI_AFFINITY_INFO.
> >>> +
> >>> +If the kernel is handling PSCI then this is done automatically and the VMM
> >>> +doesn't need to call this ioctl.
> >>
> >> Shouldn't we make handling of PSCI mandatory for VMMs that deal with
> >> CCA? I suspect it would simplify the implementation significantly.
> >
> > What do you mean by making it "mandatory for VMMs"? If you mean PSCI is
> > always forwarded to user space then I don't think it's going to make
> > much difference. Patch 27 handles the PSCI changes (72 lines added), and
> > some of that is adding this uAPI for the VMM to handle it.
> >
> > Removing the functionality to allow the VMM to handle it would obviously
> > simplify things a bit (we can drop this uAPI), but I think the desire is
> > to push this onto user space.
> >
> >> What vcpu fd does this apply to? The vcpu calling the PSCI function?
> >> Or the target? This is pretty important for PSCI_ON. My guess is that
> >> this is setting the return value for the caller?
> >
> > Yes the fd is the vcpu calling PSCI. As you say, this is for the return
> > value to be set correctly.
> >
> >> Assuming this is indeed for the caller, why do we have a different
> >> flow from anything else that returns a result from a hypercall?
> >
> > I'm not entirely sure what you are suggesting. Do you mean why are we
> > not just writing to the GPRS that would contain the result? The issue
> > here is that the RMM needs to know the PA of the target REC structure -
> > this isn't a return to the guest, but information for the RMM itself to
> > complete the PSCI call.
> >
> > Ultimately even in the case where the VMM is handling PSCI, it's
> > actually a combination of the VMM and the RMM - with the RMM validating
> > the responses.
> >
>
> More importantly, we have to make sure that the "RMI_PSCI_COMPLETE" is
> invoked before both of the following:
> 1. The "source" vCPU is run again
> 2. More importantly the "target" vCPU is run.
I don't understand why (1) is required. Once the VMM gets the request,
the target vcpu can run, and can itself do the completion, without any
additional userspace involvement.
M.
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply
* Re: [PATCH v12 06/46] arm64: RMI: Define the user ABI
From: Marc Zyngier @ 2026-03-03 13:11 UTC (permalink / raw)
To: Steven Price
Cc: kvm, kvmarm, Catalin Marinas, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <33053e22-6cc6-4d55-bc7f-01f873a15d28@arm.com>
On Mon, 02 Mar 2026 15:23:44 +0000,
Steven Price <steven.price@arm.com> wrote:
>
> Hi Marc,
>
> On 02/03/2026 14:25, Marc Zyngier wrote:
> > On Wed, 17 Dec 2025 10:10:43 +0000,
> > Steven Price <steven.price@arm.com> wrote:
> >>
> >> There is one CAP which identified the presence of CCA, and two ioctls.
> >> One ioctl is used to populate memory and the other is used when user
> >> space is providing the PSCI implementation to identify the target of the
> >> operation.
> >>
> >> Signed-off-by: Steven Price <steven.price@arm.com>
> >> ---
> >> Changes since v11:
> >> * Completely reworked to be more implicit. Rather than having explicit
> >> CAP operations to progress the realm construction these operations
> >> are done when needed (on populating and on first vCPU run).
> >> * Populate and PSCI complete are promoted to proper ioctls.
> >> Changes since v10:
> >> * Rename symbols from RME to RMI.
> >> Changes since v9:
> >> * Improvements to documentation.
> >> * Bump the magic number for KVM_CAP_ARM_RME to avoid conflicts.
> >> Changes since v8:
> >> * Minor improvements to documentation following review.
> >> * Bump the magic numbers to avoid conflicts.
> >> Changes since v7:
> >> * Add documentation of new ioctls
> >> * Bump the magic numbers to avoid conflicts
> >> Changes since v6:
> >> * Rename some of the symbols to make their usage clearer and avoid
> >> repetition.
> >> Changes from v5:
> >> * Actually expose the new VCPU capability (KVM_ARM_VCPU_REC) by bumping
> >> KVM_VCPU_MAX_FEATURES - note this also exposes KVM_ARM_VCPU_HAS_EL2!
> >> ---
> >> Documentation/virt/kvm/api.rst | 57 ++++++++++++++++++++++++++++++++++
> >> include/uapi/linux/kvm.h | 23 ++++++++++++++
> >> 2 files changed, 80 insertions(+)
> >>
> >> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
> >> index 01a3abef8abb..2d5dc7e48954 100644
> >> --- a/Documentation/virt/kvm/api.rst
> >> +++ b/Documentation/virt/kvm/api.rst
> >> @@ -6517,6 +6517,54 @@ the capability to be present.
> >>
> >> `flags` must currently be zero.
> >>
> >> +4.144 KVM_ARM_VCPU_RMI_PSCI_COMPLETE
> >> +------------------------------------
> >> +
> >> +:Capability: KVM_CAP_ARM_RMI
> >> +:Architectures: arm64
> >> +:Type: vcpu ioctl
> >> +:Parameters: struct kvm_arm_rmi_psci_complete (in)
> >> +:Returns: 0 if successful, < 0 on error
> >> +
> >> +::
> >> +
> >> + struct kvm_arm_rmi_psci_complete {
> >> + __u64 target_mpidr;
> >> + __u32 psci_status;
> >> + __u32 padding[3];
> >> + };
> >> +
> >> +Where PSCI functions are handled by user space, the RMM needs to be informed of
> >> +the target of the operation using `target_mpidr`, along with the status
> >> +(`psci_status`). The RMM v1.0 specification defines two functions that require
> >> +this call: PSCI_CPU_ON and PSCI_AFFINITY_INFO.
> >> +
> >> +If the kernel is handling PSCI then this is done automatically and the VMM
> >> +doesn't need to call this ioctl.
> >
> > Shouldn't we make handling of PSCI mandatory for VMMs that deal with
> > CCA? I suspect it would simplify the implementation significantly.
>
> What do you mean by making it "mandatory for VMMs"? If you mean PSCI is
> always forwarded to user space then I don't think it's going to make
> much difference. Patch 27 handles the PSCI changes (72 lines added), and
> some of that is adding this uAPI for the VMM to handle it.
>
> Removing the functionality to allow the VMM to handle it would obviously
> simplify things a bit (we can drop this uAPI), but I think the desire is
> to push this onto user space.
And that's what I'm asking for. I do not want this to be optional. CCA
should implies PSCI in userspace, and that's it.
>
> > What vcpu fd does this apply to? The vcpu calling the PSCI function?
> > Or the target? This is pretty important for PSCI_ON. My guess is that
> > this is setting the return value for the caller?
>
> Yes the fd is the vcpu calling PSCI. As you say, this is for the return
> value to be set correctly.
Another question is why do we need the ioctl at all? Why can't it be
done on the first run of the target vcpu? If no PSCI call was issued
to run it, then the run fails.
>
> > Assuming this is indeed for the caller, why do we have a different
> > flow from anything else that returns a result from a hypercall?
>
> I'm not entirely sure what you are suggesting. Do you mean why are we
> not just writing to the GPRS that would contain the result? The issue
> here is that the RMM needs to know the PA of the target REC structure -
> this isn't a return to the guest, but information for the RMM itself to
> complete the PSCI call.
PSCI is a SMC call. SMC calls are routed to userspace as such. For odd
reasons, the RMM treats PSCI differently from any other SMC call.
That seems a very bizarre behaviour to me.
>
> Ultimately even in the case where the VMM is handling PSCI, it's
> actually a combination of the VMM and the RMM - with the RMM validating
> the responses.
I don't see why PSCI is singled out here, irrespective of the tracking
that the RMM wants to do.
M.
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox