* Re: [PATCH kernel 2/9] pci/tsm: Add tsm_tdi_status
From: dan.j.williams @ 2026-02-25 6:33 UTC (permalink / raw)
To: Alexey Kardashevskiy, x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
In-Reply-To: <20260225053806.3311234-3-aik@amd.com>
Alexey Kardashevskiy wrote:
> Define a structure with all info about a TDI such as TDISP status,
> bind state, used START_INTERFACE options and the report digest.
>
> This will be extended and shared to the userspace.
>
> Signed-off-by: Alexey Kardashevskiy <aik@amd.com>
> ---
>
> Make it uapi? We might want a sysfs node per a field so probably not.
> For now its only user is AMD SEV TIO with a plan to expose this struct
> as a whole via sysfs.
Say more about what this uapi when sysfs already has lock+accept
indications?
Or are you just talking about exporting the TDISP report as a binary
blob?
I think the kernel probably wants a generic abstraction for asserting
that the tsm layer believes the report remains valid between fetch and
run. In other words I am not sure arch features like intf_report_counter
ever show up anywhere in uapi outside of debugfs.
^ permalink raw reply
* Re: [PATCH kernel 1/9] pci/tsm: Add TDISP report blob and helpers to parse it
From: dan.j.williams @ 2026-02-25 6:16 UTC (permalink / raw)
To: Alexey Kardashevskiy, x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
In-Reply-To: <20260225053806.3311234-2-aik@amd.com>
Alexey Kardashevskiy wrote:
> The TDI interface report is defined in PCIe r7.0,
> chapter "11.3.11 DEVICE_INTERFACE_REPORT". The report enumerates
> MMIO resources and their properties which will take effect upon
> transitioning to the RUN state.
>
> Store the report in pci_tsm.
>
> Define macros and helpers to parse the binary blob.
>
> Signed-off-by: Alexey Kardashevskiy <aik@amd.com>
> ---
>
> Probably pci_tsm::report could be struct tdi_report_header*?
[..]
> +struct tdi_report_header {
> + __u16 interface_info; /* TSM_TDI_REPORT_xxx */
> + __u16 reserved2;
> + __u16 msi_x_message_control;
> + __u16 lnr_control;
> + __u32 tph_control;
> + __u32 mmio_range_count;
> +} __packed;
> +
> +/*
> + * Each MMIO Range of the TDI is reported with the MMIO reporting offset added.
> + * Base and size in units of 4K pages
> + */
> +#define TSM_TDI_REPORT_MMIO_MSIX_TABLE BIT(0)
> +#define TSM_TDI_REPORT_MMIO_PBA BIT(1)
> +#define TSM_TDI_REPORT_MMIO_IS_NON_TEE BIT(2)
> +#define TSM_TDI_REPORT_MMIO_IS_UPDATABLE BIT(3)
> +#define TSM_TDI_REPORT_MMIO_RESERVED GENMASK(15, 4)
> +#define TSM_TDI_REPORT_MMIO_RANGE_ID GENMASK(31, 16)
> +
> +struct tdi_report_mmio_range {
> + __u64 first_page; /* First 4K page with offset added */
> + __u32 num; /* Number of 4K pages in this range */
> + __u32 range_attributes; /* TSM_TDI_REPORT_MMIO_xxx */
Those should be __le64 and le32, right? But see below for another
option...
> +} __packed;
> +
> +struct tdi_report_footer {
> + __u32 device_specific_info_len;
> + __u8 device_specific_info[];
> +} __packed;
> +
> +#define TDI_REPORT_HDR(rep) ((struct tdi_report_header *) ((rep)->data))
> +#define TDI_REPORT_MR_NUM(rep) (TDI_REPORT_HDR(rep)->mmio_range_count)
> +#define TDI_REPORT_MR_OFF(rep) ((struct tdi_report_mmio_range *) (TDI_REPORT_HDR(rep) + 1))
> +#define TDI_REPORT_MR(rep, rangeid) TDI_REPORT_MR_OFF(rep)[rangeid]
> +#define TDI_REPORT_FTR(rep) ((struct tdi_report_footer *) &TDI_REPORT_MR((rep), \
> + TDI_REPORT_MR_NUM(rep)))
> +
So we all have a version of a patch like this and the general style
suggestion I have is to just parse this layout with typical
offsets+bitfield definitions.
This follows the precedent, admittedly tiny, of the DOE definitions in
pci_regs.h. See:
/* DOE Data Object - note not actually registers */
I have a patch that parses the TDISP report with these defines:
/*
* PCIe ECN TEE Device Interface Security Protocol (TDISP)
*
* Device Interface Report data object layout as defined by PCIe r7.0 section
* 11.3.11
*/
#define PCI_TSM_DEVIF_REPORT_INFO 0
#define PCI_TSM_DEVIF_REPORT_MSIX 4
#define PCI_TSM_DEVIF_REPORT_LNR 6
#define PCI_TSM_DEVIF_REPORT_TPH 8
#define PCI_TSM_DEVIF_REPORT_MMIO_COUNT 12
#define PCI_TSM_DEVIF_REPORT_MMIO_PFN 0 /* An interface report 'pfn' is 4K in size */
#define PCI_TSM_DEVIF_REPORT_MMIO_NR_PFNS 8
#define PCI_TSM_DEVIF_REPORT_MMIO_ATTR 12
#define PCI_TSM_DEVIF_REPORT_MMIO_ATTR_MSIX_TABLE BIT(0)
#define PCI_TSM_DEVIF_REPORT_MMIO_ATTR_MSIX_PBA BIT(1)
#define PCI_TSM_DEVIF_REPORT_MMIO_ATTR_IS_NON_TEE BIT(2)
#define PCI_TSM_DEVIF_REPORT_MMIO_ATTR_IS_UPDATABLE BIT(3)
#define PCI_TSM_DEVIF_REPORT_MMIO_ATTR_RANGE_ID GENMASK(31, 16)
#define PCI_TSM_DEVIF_REPORT_MMIO_SIZE (16)
#define PCI_TSM_DEVIF_REPORT_BASE_SIZE(nr_mmio) (16 + nr_mmio * PCI_TSM_DEVIF_REPORT_MMIO_SIZE)
Any strong feelings one way or the other? I have a mild preference for
this offset+bitfields approach.
^ permalink raw reply
* Re: [PATCH kernel 7/9] coco/sev-guest: Implement the guest support for SEV TIO (phase2)
From: Borislav Petkov @ 2026-02-25 6:00 UTC (permalink / raw)
To: Alexey Kardashevskiy, x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Dave Hansen, H. Peter Anvin, Sean Christopherson, Paolo Bonzini,
Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas, Dan Williams,
Marek Szyprowski, Robin Murphy, Andrew Morton, Catalin Marinas,
Michael Ellerman, Mike Rapoport, Tom Lendacky, Ard Biesheuvel,
Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella, Melody Wang,
Seongman Lee, Joerg Roedel, Nikunj A Dadhania, Michael Roth,
Suravee Suthikulpanit, Andi Kleen, Kuppuswamy Sathyanarayanan,
Tony Luck, David Woodhouse, Greg Kroah-Hartman, Denis Efremov,
Geliang Tang, Piotr Gregor, Michael S. Tsirkin, Alex Williamson,
Arnd Bergmann, Jesse Barnes, Jacob Pan, Yinghai Lu, Kevin Brodsky,
Jonathan Cameron, Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu,
Kim Phillips, Konrad Rzeszutek Wilk, Stefano Stabellini,
Claire Chang, linux-coco, iommu
In-Reply-To: <20260225053806.3311234-8-aik@amd.com>
On February 25, 2026 5:37:50 AM UTC, Alexey Kardashevskiy <aik@amd.com> wrote:
>Implement the SEV-TIO (Trusted I/O) support in for AMD SEV-SNP guests.
>
>The implementation includes Device Security Manager (DSM) operations
>for:
>- binding a PCI function (GHCB extension) to a VM and locking
>the device configuration;
>- receiving TDI report and configuring MMIO and DMA/sDTE;
>- accepting the device into the guest TCB.
>
>Detect the SEV-TIO support (reported via GHCB HV features) and install
>the SEV-TIO TSM ops.
>
>Implement lock/accept/unlock TSM ops.
>
>Define 2 new VMGEXIT codes for GHCB:
>- TIO Guest Request to provide secure communication between a VM and
>the FW (for configuring MMIO and DMA);
>- TIO Op for requesting the HV to bind a TDI to the VM and for
>starting/stopping a TDI.
Just from staring at that huuuge diff, those bullets and things above are basically begging to be separate patches...
--
Small device. Typos and formatting crap
^ permalink raw reply
* [PATCH kernel 9/9] pci: Allow encrypted MMIO mapping via sysfs
From: Alexey Kardashevskiy @ 2026-02-25 5:37 UTC (permalink / raw)
To: x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
In-Reply-To: <20260225053806.3311234-1-aik@amd.com>
Add another resource#d_enc to allow mapping MMIO as
an encrypted/private region.
Unlike resourceN_wc, the node is added always as ability to
map MMIO as private depends on negotiation with the TSM which
happens quite late.
Signed-off-by: Alexey Kardashevskiy <aik@amd.com>
---
include/linux/pci.h | 2 +-
drivers/pci/mmap.c | 11 +++++++-
drivers/pci/pci-sysfs.c | 27 +++++++++++++++-----
drivers/pci/proc.c | 2 +-
4 files changed, 32 insertions(+), 10 deletions(-)
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 1a31353dc109..6e258b793278 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -2217,7 +2217,7 @@ pci_alloc_irq_vectors(struct pci_dev *dev, unsigned int min_vecs,
*/
int pci_mmap_resource_range(struct pci_dev *dev, int bar,
struct vm_area_struct *vma,
- enum pci_mmap_state mmap_state, int write_combine);
+ enum pci_mmap_state mmap_state, int write_combine, int enc);
#ifndef arch_can_pci_mmap_wc
#define arch_can_pci_mmap_wc() 0
diff --git a/drivers/pci/mmap.c b/drivers/pci/mmap.c
index 8da3347a95c4..90a8ab4753b8 100644
--- a/drivers/pci/mmap.c
+++ b/drivers/pci/mmap.c
@@ -23,7 +23,7 @@ static const struct vm_operations_struct pci_phys_vm_ops = {
int pci_mmap_resource_range(struct pci_dev *pdev, int bar,
struct vm_area_struct *vma,
- enum pci_mmap_state mmap_state, int write_combine)
+ enum pci_mmap_state mmap_state, int write_combine, int enc)
{
unsigned long size;
int ret;
@@ -46,6 +46,15 @@ int pci_mmap_resource_range(struct pci_dev *pdev, int bar,
vma->vm_ops = &pci_phys_vm_ops;
+ /*
+ * Calling remap_pfn_range() directly as io_remap_pfn_range()
+ * enforces shared mapping.
+ */
+ if (enc)
+ return remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff,
+ vma->vm_end - vma->vm_start,
+ pgprot_encrypted(vma->vm_page_prot));
+
return io_remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff,
vma->vm_end - vma->vm_start,
vma->vm_page_prot);
diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c
index 7f9237a926c2..715407eb8b15 100644
--- a/drivers/pci/pci-sysfs.c
+++ b/drivers/pci/pci-sysfs.c
@@ -1104,7 +1104,7 @@ void pci_remove_legacy_files(struct pci_bus *b)
* Use the regular PCI mapping routines to map a PCI resource into userspace.
*/
static int pci_mmap_resource(struct kobject *kobj, const struct bin_attribute *attr,
- struct vm_area_struct *vma, int write_combine)
+ struct vm_area_struct *vma, int write_combine, int enc)
{
struct pci_dev *pdev = to_pci_dev(kobj_to_dev(kobj));
int bar = (unsigned long)attr->private;
@@ -1124,21 +1124,28 @@ static int pci_mmap_resource(struct kobject *kobj, const struct bin_attribute *a
mmap_type = res->flags & IORESOURCE_MEM ? pci_mmap_mem : pci_mmap_io;
- return pci_mmap_resource_range(pdev, bar, vma, mmap_type, write_combine);
+ return pci_mmap_resource_range(pdev, bar, vma, mmap_type, write_combine, enc);
}
static int pci_mmap_resource_uc(struct file *filp, struct kobject *kobj,
const struct bin_attribute *attr,
struct vm_area_struct *vma)
{
- return pci_mmap_resource(kobj, attr, vma, 0);
+ return pci_mmap_resource(kobj, attr, vma, 0, 0);
}
static int pci_mmap_resource_wc(struct file *filp, struct kobject *kobj,
const struct bin_attribute *attr,
struct vm_area_struct *vma)
{
- return pci_mmap_resource(kobj, attr, vma, 1);
+ return pci_mmap_resource(kobj, attr, vma, 1, 0);
+}
+
+static int pci_mmap_resource_enc(struct file *filp, struct kobject *kobj,
+ const struct bin_attribute *attr,
+ struct vm_area_struct *vma)
+{
+ return pci_mmap_resource(kobj, attr, vma, 0, 1);
}
static ssize_t pci_resource_io(struct file *filp, struct kobject *kobj,
@@ -1232,7 +1239,7 @@ static void pci_remove_resource_files(struct pci_dev *pdev)
}
}
-static int pci_create_attr(struct pci_dev *pdev, int num, int write_combine)
+static int pci_create_attr(struct pci_dev *pdev, int num, int write_combine, int enc)
{
/* allocate attribute structure, piggyback attribute name */
int name_len = write_combine ? 13 : 10;
@@ -1250,6 +1257,9 @@ static int pci_create_attr(struct pci_dev *pdev, int num, int write_combine)
if (write_combine) {
sprintf(res_attr_name, "resource%d_wc", num);
res_attr->mmap = pci_mmap_resource_wc;
+ } else if (enc) {
+ sprintf(res_attr_name, "resource%d_enc", num);
+ res_attr->mmap = pci_mmap_resource_enc;
} else {
sprintf(res_attr_name, "resource%d", num);
if (pci_resource_flags(pdev, num) & IORESOURCE_IO) {
@@ -1310,11 +1320,14 @@ static int pci_create_resource_files(struct pci_dev *pdev)
if (!pci_resource_len(pdev, i))
continue;
- retval = pci_create_attr(pdev, i, 0);
+ retval = pci_create_attr(pdev, i, 0, 0);
/* for prefetchable resources, create a WC mappable file */
if (!retval && arch_can_pci_mmap_wc() &&
pdev->resource[i].flags & IORESOURCE_PREFETCH)
- retval = pci_create_attr(pdev, i, 1);
+ retval = pci_create_attr(pdev, i, 1, 0);
+ /* Add node for private MMIO mapping */
+ if (!retval)
+ retval = pci_create_attr(pdev, i, 0, 1);
if (retval) {
pci_remove_resource_files(pdev);
return retval;
diff --git a/drivers/pci/proc.c b/drivers/pci/proc.c
index 9348a0fb8084..e0c0ece7f3f5 100644
--- a/drivers/pci/proc.c
+++ b/drivers/pci/proc.c
@@ -288,7 +288,7 @@ static int proc_bus_pci_mmap(struct file *file, struct vm_area_struct *vma)
/* Adjust vm_pgoff to be the offset within the resource */
vma->vm_pgoff -= start >> PAGE_SHIFT;
ret = pci_mmap_resource_range(dev, i, vma,
- fpriv->mmap_state, write_combine);
+ fpriv->mmap_state, write_combine, 0);
if (ret < 0)
return ret;
--
2.52.0
^ permalink raw reply related
* [PATCH kernel 8/9] RFC: PCI: Avoid needless touching of Command register
From: Alexey Kardashevskiy @ 2026-02-25 5:37 UTC (permalink / raw)
To: x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
In-Reply-To: <20260225053806.3311234-1-aik@amd.com>
Once locked, a TDI's MSE and BME are not allowed to be cleared.
Skip INTx test as TEE-capable PCI functions are most likely IOV VFs
anyway and those do not support INTx at all.
Add a quirk preventing the probing code from disabling MSE when
updating 64bit BAR (which cannot be done atomically).
Note that normally this happens too early and likely not really
needed for the device attestation happening long after PCI probing.
Signed-off-by: Alexey Kardashevskiy <aik@amd.com>
---
This is also handled in QEMU - it will block clearing BME and MSE
(normally happening on modprobe/rmmod) as long as the TDI is
CONFIG_LOCKED or RUN.
This only patch is not enough but reduces the number of unwanted
writes to MSE/BME.
Also, SRIOV cannot have INTx so pci_intx_mask_broken() could skip
VFs too, should it?
---
drivers/pci/probe.c | 5 +++++
drivers/pci/quirks.c | 9 +++++++++
2 files changed, 14 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 4c3aec1fd53e..cc0613e7c905 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1930,6 +1930,11 @@ static int pci_intx_mask_broken(struct pci_dev *dev)
{
u16 orig, toggle, new;
+ if (dev->devcap & PCI_EXP_DEVCAP_TEE) {
+ pci_warn_once(dev, "(TIO) Disable check for broken INTX");
+ return 1;
+ }
+
pci_read_config_word(dev, PCI_COMMAND, &orig);
toggle = orig ^ PCI_COMMAND_INTX_DISABLE;
pci_write_config_word(dev, PCI_COMMAND, toggle);
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 214ed060ca1b..b875859699ba 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -316,6 +316,15 @@ static void quirk_mmio_always_on(struct pci_dev *dev)
DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_ANY_ID, PCI_ANY_ID,
PCI_CLASS_BRIDGE_HOST, 8, quirk_mmio_always_on);
+static void quirk_mmio_tio_always_on(struct pci_dev *dev)
+{
+ if (dev->devcap & PCI_EXP_DEVCAP_TEE) {
+ pci_info(dev, "(TIO) quirk: MMIO always On");
+ dev->mmio_always_on = 1;
+ }
+}
+DECLARE_PCI_FIXUP_EARLY(PCI_ANY_ID, PCI_ANY_ID, quirk_mmio_tio_always_on);
+
/*
* The Mellanox Tavor device gives false positive parity errors. Disable
* parity error reporting.
--
2.52.0
^ permalink raw reply related
* [PATCH kernel 7/9] coco/sev-guest: Implement the guest support for SEV TIO (phase2)
From: Alexey Kardashevskiy @ 2026-02-25 5:37 UTC (permalink / raw)
To: x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
In-Reply-To: <20260225053806.3311234-1-aik@amd.com>
Implement the SEV-TIO (Trusted I/O) support in for AMD SEV-SNP guests.
The implementation includes Device Security Manager (DSM) operations
for:
- binding a PCI function (GHCB extension) to a VM and locking
the device configuration;
- receiving TDI report and configuring MMIO and DMA/sDTE;
- accepting the device into the guest TCB.
Detect the SEV-TIO support (reported via GHCB HV features) and install
the SEV-TIO TSM ops.
Implement lock/accept/unlock TSM ops.
Define 2 new VMGEXIT codes for GHCB:
- TIO Guest Request to provide secure communication between a VM and
the FW (for configuring MMIO and DMA);
- TIO Op for requesting the HV to bind a TDI to the VM and for
starting/stopping a TDI.
Signed-off-by: Alexey Kardashevskiy <aik@amd.com>
---
drivers/virt/coco/sev-guest/Kconfig | 1 +
drivers/virt/coco/sev-guest/Makefile | 3 +
arch/x86/include/asm/sev-common.h | 1 +
arch/x86/include/asm/sev.h | 13 +
arch/x86/include/uapi/asm/svm.h | 13 +
drivers/virt/coco/sev-guest/sev-guest.h | 4 +
include/linux/psp-sev.h | 31 +
include/uapi/linux/sev-guest.h | 43 ++
arch/x86/coco/sev/core.c | 53 ++
drivers/virt/coco/sev-guest/sev-guest.c | 13 +
drivers/virt/coco/sev-guest/tio.c | 707 ++++++++++++++++++++
11 files changed, 882 insertions(+)
diff --git a/drivers/virt/coco/sev-guest/Kconfig b/drivers/virt/coco/sev-guest/Kconfig
index a6405ab6c2c3..4255072dfa1a 100644
--- a/drivers/virt/coco/sev-guest/Kconfig
+++ b/drivers/virt/coco/sev-guest/Kconfig
@@ -3,6 +3,7 @@ config SEV_GUEST
default m
depends on AMD_MEM_ENCRYPT
select TSM_REPORTS
+ select PCI_TSM if PCI
help
SEV-SNP firmware provides the guest a mechanism to communicate with
the PSP without risk from a malicious hypervisor who wishes to read,
diff --git a/drivers/virt/coco/sev-guest/Makefile b/drivers/virt/coco/sev-guest/Makefile
index 9604792e0095..b4766289c85f 100644
--- a/drivers/virt/coco/sev-guest/Makefile
+++ b/drivers/virt/coco/sev-guest/Makefile
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: GPL-2.0-only
obj-$(CONFIG_SEV_GUEST) += sev_guest.o
sev_guest-y += sev-guest.o
+ifeq ($(CONFIG_PCI_TSM),y)
+sev_guest-y += tio.o
+endif
diff --git a/arch/x86/include/asm/sev-common.h b/arch/x86/include/asm/sev-common.h
index 01a6e4dbe423..ff763c3c5d63 100644
--- a/arch/x86/include/asm/sev-common.h
+++ b/arch/x86/include/asm/sev-common.h
@@ -137,6 +137,7 @@ enum psc_op {
#define GHCB_HV_FT_SNP BIT_ULL(0)
#define GHCB_HV_FT_SNP_AP_CREATION BIT_ULL(1)
#define GHCB_HV_FT_SNP_MULTI_VMPL BIT_ULL(5)
+#define GHCB_HV_FT_SNP_SEV_TIO BIT_ULL(7)
/*
* SNP Page State Change NAE event
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index 0e6c0940100f..f6e1a2f96d47 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -149,6 +149,8 @@ struct snp_req_data {
unsigned long resp_gpa;
unsigned long data_gpa;
unsigned int data_npages;
+ unsigned int guest_rid;
+ unsigned long param;
};
#define MAX_AUTHTAG_LEN 32
@@ -179,6 +181,14 @@ enum msg_type {
SNP_MSG_TSC_INFO_REQ = 17,
SNP_MSG_TSC_INFO_RSP,
+ TIO_MSG_TDI_INFO_REQ = 19,
+ TIO_MSG_TDI_INFO_RSP = 20,
+ TIO_MSG_MMIO_VALIDATE_REQ = 21,
+ TIO_MSG_MMIO_VALIDATE_RSP = 22,
+ TIO_MSG_MMIO_CONFIG_REQ = 23,
+ TIO_MSG_MMIO_CONFIG_RSP = 24,
+ TIO_MSG_SDTE_WRITE_REQ = 25,
+ TIO_MSG_SDTE_WRITE_RSP = 26,
SNP_MSG_TYPE_MAX
};
@@ -597,6 +607,9 @@ static inline void sev_evict_cache(void *va, int npages)
}
}
+bool sev_tio_ghcb_supported(void);
+int sev_tio_op(u32 guest_rid, unsigned int op, u64 *fw_err, u64 *tdi_id);
+
#else /* !CONFIG_AMD_MEM_ENCRYPT */
#define snp_vmpl 0
diff --git a/arch/x86/include/uapi/asm/svm.h b/arch/x86/include/uapi/asm/svm.h
index 650e3256ea7d..c4b735d0aa1e 100644
--- a/arch/x86/include/uapi/asm/svm.h
+++ b/arch/x86/include/uapi/asm/svm.h
@@ -122,6 +122,17 @@
#define SVM_VMGEXIT_SAVIC_REGISTER_GPA 0
#define SVM_VMGEXIT_SAVIC_UNREGISTER_GPA 1
#define SVM_VMGEXIT_SAVIC_SELF_GPA ~0ULL
+#define SVM_VMGEXIT_SEV_TIO_GUEST_REQUEST 0x80000020
+#define SVM_VMGEXIT_SEV_TIO_GUEST_REQUEST_PARAM_STATE BIT(0)
+#define SVM_VMGEXIT_SEV_TIO_GUEST_REQUEST_PARAM_REPORT BIT(3)
+#define SVM_VMGEXIT_SEV_TIO_OP 0x80000021
+#define SVM_VMGEXIT_SEV_TIO_OP_PARAM(guest_id, action) ((u64)(action)<<32|(guest_id))
+#define SVM_VMGEXIT_SEV_TIO_OP_ACTION(exitinfo1) ((exitinfo1)>>32)
+#define SVM_VMGEXIT_SEV_TIO_OP_GUEST_ID(exitinfo1) ((exitinfo1) & 0xFFFFFFFF)
+#define SVM_VMGEXIT_SEV_TIO_OP_BIND 0
+#define SVM_VMGEXIT_SEV_TIO_OP_UNBIND 1
+#define SVM_VMGEXIT_SEV_TIO_OP_RUN 2
+#define SVM_VMGEXIT_SEV_TIO_OP_STOP 3
#define SVM_VMGEXIT_HV_FEATURES 0x8000fffd
#define SVM_VMGEXIT_TERM_REQUEST 0x8000fffe
#define SVM_VMGEXIT_TERM_REASON(reason_set, reason_code) \
@@ -245,6 +256,8 @@
{ SVM_VMGEXIT_GUEST_REQUEST, "vmgexit_guest_request" }, \
{ SVM_VMGEXIT_EXT_GUEST_REQUEST, "vmgexit_ext_guest_request" }, \
{ SVM_VMGEXIT_AP_CREATION, "vmgexit_ap_creation" }, \
+ { SVM_VMGEXIT_SEV_TIO_GUEST_REQUEST, "vmgexit_sev_tio_guest_request" }, \
+ { SVM_VMGEXIT_SEV_TIO_OP, "vmgexit_sev_tio_op" }, \
{ SVM_VMGEXIT_HV_FEATURES, "vmgexit_hypervisor_feature" }, \
{ SVM_EXIT_ERR, "invalid_guest_state" }
diff --git a/drivers/virt/coco/sev-guest/sev-guest.h b/drivers/virt/coco/sev-guest/sev-guest.h
index b2a97778e635..c823a782739f 100644
--- a/drivers/virt/coco/sev-guest/sev-guest.h
+++ b/drivers/virt/coco/sev-guest/sev-guest.h
@@ -11,6 +11,10 @@ struct snp_guest_dev {
struct miscdevice misc;
struct snp_msg_desc *msg_desc;
+
+ struct tsm_dev *tsmdev;
};
+void sev_guest_tsm_set_ops(bool set, struct snp_guest_dev *snp_dev);
+
#endif /* __SEV_GUEST_H__ */
diff --git a/include/linux/psp-sev.h b/include/linux/psp-sev.h
index cce864dbf281..dc2932953abc 100644
--- a/include/linux/psp-sev.h
+++ b/include/linux/psp-sev.h
@@ -1050,4 +1050,35 @@ static inline bool sev_is_snp_ciphertext_hiding_supported(void) { return false;
#endif /* CONFIG_CRYPTO_DEV_SP_PSP */
+/*
+ * Status codes from TIO_MSG_MMIO_VALIDATE_REQ
+ */
+enum mmio_validate_status {
+ MMIO_VALIDATE_SUCCESS = 0,
+ MMIO_VALIDATE_INVALID_TDI = 1,
+ MMIO_VALIDATE_TDI_UNBOUND = 2,
+ MMIO_VALIDATE_NOT_ASSIGNED = 3,
+ MMIO_VALIDATE_NOT_IO = 4,
+ MMIO_VALIDATE_NOT_UNIFORM = 5, /* The Validated bit is not uniformly set for
+ the MMIO subrange */
+ MMIO_VALIDATE_NOT_IMMUTABLE = 6,/* At least one page does not have immutable bit set
+ when validated bit is clear */
+ MMIO_VALIDATE_NOT_MAPPED = 7, /* At least one page is not mapped to the expected GPA */
+ MMIO_VALIDATE_NOT_REPORTED = 8, /* The provided MMIO range ID is not reported in
+ the interface report */
+ MMIO_VALIDATE_OUT_OF_RANGE = 9, /* The subrange is out the MMIO range in
+ the interface report */
+ MMIO_VALIDATE_NOT_4K = 10, /* At least one page is not 4K page size */
+};
+
+/*
+ * Status codes from TIO_MSG_SDTE_WRITE_REQ
+ */
+enum sdte_write_status {
+ SDTE_WRITE_SUCCESS = 0,
+ SDTE_WRITE_INVALID_TDI = 1,
+ SDTE_WRITE_TDI_NOT_BOUND = 2,
+ SDTE_WRITE_RESERVED = 3,
+};
+
#endif /* __PSP_SEV_H__ */
diff --git a/include/uapi/linux/sev-guest.h b/include/uapi/linux/sev-guest.h
index fcdfea767fca..5015160254f4 100644
--- a/include/uapi/linux/sev-guest.h
+++ b/include/uapi/linux/sev-guest.h
@@ -13,6 +13,7 @@
#define __UAPI_LINUX_SEV_GUEST_H_
#include <linux/types.h>
+#include <linux/uuid.h>
#define SNP_REPORT_USER_DATA_SIZE 64
@@ -96,4 +97,46 @@ struct snp_ext_report_req {
#define SNP_GUEST_VMM_ERR_INVALID_LEN 1
#define SNP_GUEST_VMM_ERR_BUSY 2
+/*
+ * TIO_GUEST_REQUEST's TIO_MSG_MMIO_VALIDATE_REQ
+ * encoding for MMIO in RDX:
+ *
+ * ........ ....GGGG GGGGGGGG GGGGGGGG GGGGGGGG GGGGGGGG GGGGOOOO OOOOTrrr
+ * Where:
+ * G - guest physical address
+ * O - order of 4K pages
+ * T - TEE (valid for TIO_MSG_MMIO_CONFIG_REQ)
+ * r - range id == BAR
+ */
+#define MMIO_VALIDATE_GPA(r) ((r) & 0x000FFFFFFFFFF000ULL)
+#define MMIO_VALIDATE_LEN(r) (1ULL << (12 + (((r) >> 4) & 0xFF)))
+#define MMIO_VALIDATE_RANGEID(r) ((r) & 0x7)
+#define MMIO_VALIDATE_RESERVED(r) ((r) & 0xFFF0000000000000ULL)
+#define MMIO_VALIDATE_PRIVATE(r) (!!((r) & BIT(3)))
+
+#define MMIO_MK_VALIDATE(start, size, range_id, private) \
+ (MMIO_VALIDATE_GPA(start) | \
+ (get_order(size) << 4) | \
+ ((private) ? BIT(3) : 0) | \
+ ((range_id) & 7) )
+
+#define SDTE_VALIDATE 1
+
+/* Optional Certificates/measurements/report data from TIO_GUEST_REQUEST */
+struct tio_blob_table_entry {
+ guid_t guid;
+ __u32 offset;
+ __u32 length;
+} __packed;
+
+/* Measurement’s blob: 5caa80c6-12ef-401a-b364-ec59a93abe3f */
+#define TIO_GUID_MEASUREMENTS \
+ GUID_INIT(0x5caa80c6, 0x12ef, 0x401a, 0xb3, 0x64, 0xec, 0x59, 0xa9, 0x3a, 0xbe, 0x3f)
+/* Certificates blob: 078ccb75-2644-49e8-afe7-5686c5cf72f1 */
+#define TIO_GUID_CERTIFICATES \
+ GUID_INIT(0x078ccb75, 0x2644, 0x49e8, 0xaf, 0xe7, 0x56, 0x86, 0xc5, 0xcf, 0x72, 0xf1)
+/* Attestation report: 70dc5b0e-0cc0-4cd5-97bb-ff0ba25bf320 */
+#define TIO_GUID_REPORT \
+ GUID_INIT(0x70dc5b0e, 0x0cc0, 0x4cd5, 0x97, 0xbb, 0xff, 0x0b, 0xa2, 0x5b, 0xf3, 0x20)
+
#endif /* __UAPI_LINUX_SEV_GUEST_H_ */
diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index 9ae3b11754e6..1f2e34367772 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -136,6 +136,43 @@ static unsigned long snp_tsc_freq_khz __ro_after_init;
DEFINE_PER_CPU(struct sev_es_runtime_data*, runtime_data);
DEFINE_PER_CPU(struct sev_es_save_area *, sev_vmsa);
+bool sev_tio_ghcb_supported(void)
+{
+ return !!(sev_hv_features & GHCB_HV_FT_SNP_SEV_TIO);
+}
+EXPORT_SYMBOL_GPL(sev_tio_ghcb_supported);
+
+int sev_tio_op(u32 guest_rid, unsigned int op, u64 *fw_err, u64 *tdi_id)
+{
+ struct ghcb_state state;
+ struct es_em_ctxt ctxt;
+ struct ghcb *ghcb;
+ int ret;
+
+ /* __sev_get_ghcb() needs IRQs disabled because it uses per-CPU GHCB. */
+ guard(irqsave)();
+
+ ghcb = __sev_get_ghcb(&state);
+ if (!ghcb)
+ return -EIO;
+
+ vc_ghcb_invalidate(ghcb);
+ ret = sev_es_ghcb_hv_call(ghcb, &ctxt, SVM_VMGEXIT_SEV_TIO_OP,
+ SVM_VMGEXIT_SEV_TIO_OP_PARAM(guest_rid, op), 0);
+
+ *fw_err = ghcb->save.sw_exit_info_2;
+ if (*fw_err)
+ ret = -EIO;
+
+ if (!ret && op == SVM_VMGEXIT_SEV_TIO_OP_BIND && tdi_id)
+ *tdi_id = ghcb_get_rcx(ghcb);
+
+ __sev_put_ghcb(&state);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(sev_tio_op);
+
/*
* SVSM related information:
* When running under an SVSM, the VMPL that Linux is executing at must be
@@ -1666,6 +1703,11 @@ static int snp_issue_guest_request(struct snp_guest_req *req)
if (req->exit_code == SVM_VMGEXIT_EXT_GUEST_REQUEST) {
ghcb_set_rax(ghcb, input->data_gpa);
ghcb_set_rbx(ghcb, input->data_npages);
+ } else if (req->exit_code == SVM_VMGEXIT_SEV_TIO_GUEST_REQUEST) {
+ ghcb_set_rax(ghcb, input->data_gpa);
+ ghcb_set_rbx(ghcb, input->data_npages);
+ ghcb_set_rcx(ghcb, input->guest_rid);
+ ghcb_set_rdx(ghcb, input->param);
}
ret = sev_es_ghcb_hv_call(ghcb, &ctxt, req->exit_code, input->req_gpa, input->resp_gpa);
@@ -1675,6 +1717,8 @@ static int snp_issue_guest_request(struct snp_guest_req *req)
req->exitinfo2 = ghcb->save.sw_exit_info_2;
switch (req->exitinfo2) {
case 0:
+ if (req->exit_code == SVM_VMGEXIT_SEV_TIO_GUEST_REQUEST)
+ input->param = ghcb_get_rdx(ghcb);
break;
case SNP_GUEST_VMM_ERR(SNP_GUEST_VMM_ERR_BUSY):
@@ -1687,6 +1731,10 @@ static int snp_issue_guest_request(struct snp_guest_req *req)
input->data_npages = ghcb_get_rbx(ghcb);
ret = -ENOSPC;
break;
+ } else if (req->exit_code == SVM_VMGEXIT_SEV_TIO_GUEST_REQUEST) {
+ input->data_npages = ghcb_get_rbx(ghcb);
+ ret = -ENOSPC;
+ break;
}
fallthrough;
default:
@@ -2176,6 +2224,11 @@ static int __handle_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_r
rc = snp_issue_guest_request(req);
switch (rc) {
case -ENOSPC:
+ if (req->exit_code == SVM_VMGEXIT_SEV_TIO_GUEST_REQUEST) {
+ pr_warn("SVM_VMGEXIT_SEV_TIO_GUEST_REQUEST => -ENOSPC");
+ break;
+ }
+
/*
* If the extended guest request fails due to having too
* small of a certificate data buffer, retry the same
diff --git a/drivers/virt/coco/sev-guest/sev-guest.c b/drivers/virt/coco/sev-guest/sev-guest.c
index e1ceeab54a21..41072ece79a8 100644
--- a/drivers/virt/coco/sev-guest/sev-guest.c
+++ b/drivers/virt/coco/sev-guest/sev-guest.c
@@ -46,6 +46,10 @@ static int vmpck_id = -1;
module_param(vmpck_id, int, 0444);
MODULE_PARM_DESC(vmpck_id, "The VMPCK ID to use when communicating with the PSP.");
+static bool tsm_enable = true;
+module_param(tsm_enable, bool, 0644);
+MODULE_PARM_DESC(tsm_enable, "Enable SEV TIO");
+
static inline struct snp_guest_dev *to_snp_dev(struct file *file)
{
struct miscdevice *dev = file->private_data;
@@ -667,6 +671,13 @@ static int __init sev_guest_probe(struct platform_device *pdev)
snp_dev->msg_desc = mdesc;
dev_info(dev, "Initialized SEV guest driver (using VMPCK%d communication key)\n",
mdesc->vmpck_id);
+
+ if (!sev_tio_ghcb_supported())
+ tsm_enable = false;
+
+ if (tsm_enable)
+ sev_guest_tsm_set_ops(true, snp_dev);
+
return 0;
e_msg_init:
@@ -680,6 +691,8 @@ static void __exit sev_guest_remove(struct platform_device *pdev)
struct snp_guest_dev *snp_dev = platform_get_drvdata(pdev);
snp_msg_free(snp_dev->msg_desc);
+ if (tsm_enable)
+ sev_guest_tsm_set_ops(false, snp_dev);
misc_deregister(&snp_dev->misc);
}
diff --git a/drivers/virt/coco/sev-guest/tio.c b/drivers/virt/coco/sev-guest/tio.c
new file mode 100644
index 000000000000..6739b1f49e0e
--- /dev/null
+++ b/drivers/virt/coco/sev-guest/tio.c
@@ -0,0 +1,707 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+#include <linux/bitfield.h>
+#include <linux/bitops.h>
+#include <linux/pci.h>
+#include <linux/psp-sev.h>
+#include <linux/tsm.h>
+#include <linux/pci-tsm.h>
+#include <crypto/gcm.h>
+#include <uapi/linux/sev-guest.h>
+
+#include <asm/svm.h>
+#include <asm/sev.h>
+#include <asm/sev-internal.h>
+
+#include "sev-guest.h"
+
+#define TIO_MESSAGE_VERSION 1
+
+ulong tsm_vtom = 0x7fffffff;
+module_param(tsm_vtom, ulong, 0644);
+MODULE_PARM_DESC(tsm_vtom, "SEV TIO vTOM value");
+
+#define tsm_dev_to_snp_dev(t) ((struct snp_guest_dev *)dev_get_drvdata((t)->dev.parent))
+#define pdev_to_tdi(p) container_of((p)->tsm, struct tio_guest_tdi, ds.base_tsm)
+
+struct tio_guest_tdi {
+ struct pci_tsm_devsec ds;
+ struct snp_guest_dev *snp_dev;
+ u64 tdi_id; /* Runtime FW generated TDI id */
+};
+
+static int handle_tio_guest_request(struct snp_guest_dev *snp_dev, u8 type,
+ void *req_buf, size_t req_sz, void *resp_buf, u32 resp_sz,
+ void *pt, u64 *npages, u64 *bdfn, u64 *param, u64 *fw_err)
+{
+ struct snp_msg_desc *mdesc = snp_dev->msg_desc;
+ struct snp_guest_req req = {
+ .msg_version = TIO_MESSAGE_VERSION,
+ };
+ u64 exitinfo2 = 0;
+ int ret;
+
+ req.msg_type = type;
+ req.vmpck_id = mdesc->vmpck_id;
+ req.req_buf = kmemdup(req_buf, req_sz, GFP_KERNEL);
+ req.req_sz = req_sz;
+ req.resp_buf = kmalloc(resp_sz, GFP_KERNEL);
+ req.resp_sz = resp_sz;
+ req.exit_code = SVM_VMGEXIT_SEV_TIO_GUEST_REQUEST;
+
+ req.input.guest_rid = 0;
+ req.input.param = 0;
+
+ if (pt && npages) {
+ req.certs_data = pt;
+ req.input.data_npages = *npages;
+ }
+ if (bdfn)
+ req.input.guest_rid = *bdfn;
+ req.input.param = *param;
+
+ ret = snp_send_guest_request(mdesc, &req);
+
+ memcpy(resp_buf, req.resp_buf, resp_sz);
+
+ *param = req.input.param;
+
+ *fw_err = exitinfo2;
+
+ kfree(req.resp_buf);
+ kfree(req.req_buf);
+
+ return ret;
+}
+
+static void free_shared_pages(void *buf, size_t sz)
+{
+ unsigned int npages = PAGE_ALIGN(sz) >> PAGE_SHIFT;
+ int ret;
+
+ if (!buf)
+ return;
+
+ ret = set_memory_encrypted((unsigned long)buf, npages);
+ if (ret) {
+ WARN_ONCE(ret, "failed to restore encryption mask (leak it)\n");
+ return;
+ }
+
+ __free_pages(virt_to_page(buf), get_order(sz));
+}
+
+static void *alloc_shared_pages(size_t sz)
+{
+ unsigned int npages = PAGE_ALIGN(sz) >> PAGE_SHIFT;
+ struct page *page;
+ int ret;
+
+ page = alloc_pages(GFP_KERNEL_ACCOUNT, get_order(sz));
+ if (!page)
+ return NULL;
+
+ ret = set_memory_decrypted((unsigned long)page_address(page), npages);
+ if (ret) {
+ pr_err("failed to mark page shared, ret=%d\n", ret);
+ __free_pages(page, get_order(sz));
+ return NULL;
+ }
+
+ return page_address(page);
+}
+
+static int guest_request_tio_data(struct snp_guest_dev *snp_dev, u8 type,
+ void *req_buf, size_t req_sz, void *resp_buf, u32 resp_sz,
+ u64 bdfn, enum tsm_tdisp_state *state,
+ struct tsm_blob **report, u64 *fw_err)
+{
+#define TIO_DATA_PAGES (SZ_32K >> PAGE_SHIFT)
+ u64 npages = TIO_DATA_PAGES, param = 0;
+ struct tio_blob_table_entry *pt;
+ int rc;
+
+ pt = alloc_shared_pages(TIO_DATA_PAGES << PAGE_SHIFT);
+ if (!pt)
+ return -ENOMEM;
+
+ if (state)
+ param |= SVM_VMGEXIT_SEV_TIO_GUEST_REQUEST_PARAM_STATE;
+ if (report)
+ param |= SVM_VMGEXIT_SEV_TIO_GUEST_REQUEST_PARAM_REPORT;
+
+ rc = handle_tio_guest_request(snp_dev, type, req_buf, req_sz, resp_buf, resp_sz,
+ pt, &npages, &bdfn, ¶m, fw_err);
+ if (npages > TIO_DATA_PAGES) {
+ free_shared_pages(pt, TIO_DATA_PAGES << PAGE_SHIFT);
+ pt = alloc_shared_pages(npages << PAGE_SHIFT);
+ if (!pt)
+ return -ENOMEM;
+
+ rc = handle_tio_guest_request(snp_dev, type, req_buf, req_sz, resp_buf, resp_sz,
+ pt, &npages, &bdfn, ¶m, fw_err);
+ }
+ if (rc)
+ return rc;
+
+ if (report) {
+ tsm_blob_free(*report);
+ *report = NULL;
+ }
+
+ for (unsigned int i = 0; i < 3; ++i) {
+ u8 *ptr = ((u8 *) pt) + pt[i].offset;
+ size_t len = pt[i].length;
+ struct tsm_blob *b;
+
+ if (guid_is_null(&pt[i].guid))
+ break;
+
+ if (!len)
+ continue;
+
+ b = tsm_blob_new(ptr, len);
+ if (!b)
+ break;
+
+ if (guid_equal(&pt[i].guid, &TIO_GUID_REPORT) && report)
+ *report = b;
+ }
+ free_shared_pages(pt, npages);
+
+ if (state)
+ *state = param;
+
+ return 0;
+}
+
+struct tio_msg_tdi_info_req {
+ u16 guest_device_id;
+ u8 reserved[14];
+} __packed;
+
+enum {
+ TIO_MSG_TDI_INFO_RSP_STATUS_BOUND = 0,
+ TIO_MSG_TDI_INFO_RSP_STATUS_INVALID = 1,
+ TIO_MSG_TDI_INFO_RSP_STATUS_UNBOUND = 2,
+};
+
+struct tio_msg_tdi_info_rsp {
+ u16 guest_device_id;
+ u16 status; /* TIO_MSG_TDI_INFO_RSP_STATUS_xxx */
+ u8 reserved1[12];
+
+ u32 meas_digest_valid:1;
+ u32 meas_digest_fresh:1;
+ u32 reserved2:30;
+
+ /* These are TDISP's LOCK_INTERFACE_REQUEST flags */
+ u32 no_fw_update:1;
+ u32 cache_line_size:1;
+ u32 lock_msix:1;
+ u32 bind_p2p:1;
+ u32 all_request_redirect:1;
+ u32 reserved3:27;
+
+ u64 spdm_algos;
+ u8 certs_digest[48];
+ u8 meas_digest[48];
+ u8 interface_report_digest[48];
+ u64 tdi_report_count;
+ u64 reserved4;
+} __packed;
+
+/* Passing pci_tsm explicitly as it may not be set in pci_dev just yet */
+static int tio_tdi_status(struct pci_dev *pdev, struct snp_guest_dev *snp_dev,
+ struct tsm_tdi_status *ts, struct tsm_blob **report)
+{
+ enum tsm_tdisp_state state = TDISP_STATE_CONFIG_UNLOCKED;
+ struct snp_msg_desc *mdesc = snp_dev->msg_desc;
+ size_t resp_len = sizeof(struct tio_msg_tdi_info_rsp) + mdesc->ctx->authsize;
+ struct tio_msg_tdi_info_rsp *rsp __free(kfree_sensitive) = kzalloc(resp_len, GFP_KERNEL);
+ struct tio_msg_tdi_info_req req = {
+ .guest_device_id = pci_dev_id(pdev),
+ };
+ u64 fw_err = 0;
+ int rc;
+
+ pci_notice(pdev, "TDI info");
+ if (!rsp)
+ return -ENOMEM;
+
+ rc = guest_request_tio_data(snp_dev, TIO_MSG_TDI_INFO_REQ, &req,
+ sizeof(req), rsp, resp_len,
+ req.guest_device_id, &state,
+ report, &fw_err);
+ if (rc)
+ return rc;
+
+ ts->no_fw_update = rsp->no_fw_update;
+ ts->cache_line_size = rsp->cache_line_size == 0 ? 64 : 128;
+ ts->lock_msix = rsp->lock_msix;
+ ts->bind_p2p = rsp->bind_p2p;
+ ts->all_request_redirect = rsp->all_request_redirect;
+ memcpy(ts->interface_report_digest, rsp->interface_report_digest,
+ sizeof(ts->interface_report_digest));
+ ts->intf_report_counter = rsp->tdi_report_count;
+
+ switch (rsp->status) {
+ case TIO_MSG_TDI_INFO_RSP_STATUS_BOUND:
+ ts->status = TDISP_STATE_BOUND;
+ break;
+ case TIO_MSG_TDI_INFO_RSP_STATUS_UNBOUND:
+ ts->status = TDISP_STATE_UNBOUND;
+ break;
+ default:
+ ts->status = TDISP_STATE_INVALID;
+ break;
+ }
+ ts->state = state;
+
+ return 0;
+}
+
+struct tio_msg_mmio_validate_req {
+ u16 guest_device_id;
+ u16 reserved1;
+ u8 reserved2[12];
+ u64 subrange_base;
+ u32 subrange_page_count;
+ u32 range_offset;
+
+ u16 validated:1; /* Desired value to set RMP.Validated for the range */
+ /*
+ * Force validated:
+ * 0: If subrange does not have RMP.Validated set uniformly, fail.
+ * 1: If subrange does not have RMP.Validated set uniformly, force
+ * to requested value
+ */
+ u16 force_validated:1;
+ u16 reserved3:14;
+
+ u16 range_id;
+ u8 reserved4[12];
+} __packed;
+
+struct tio_msg_mmio_validate_rsp {
+ u16 guest_interface_id;
+ u16 status; /* MMIO_VALIDATE_xxx */
+ u8 reserved1[12];
+ u64 subrange_base;
+ u32 subrange_page_count;
+ u32 range_offset;
+
+ u16 changed:1; /* Validated bit has changed due to this operation */
+ u16 reserved2:15;
+
+ u16 range_id;
+ u8 reserved3[12];
+} __packed;
+
+static int mmio_validate_range(struct snp_guest_dev *snp_dev, struct pci_dev *pdev,
+ unsigned int range_id,
+ resource_size_t start, resource_size_t size,
+ bool invalidate, u64 *fw_err, u16 *status)
+{
+ struct snp_msg_desc *mdesc = snp_dev->msg_desc;
+ size_t resp_len = sizeof(struct tio_msg_mmio_validate_rsp) + mdesc->ctx->authsize;
+ struct tio_msg_mmio_validate_rsp *rsp __free(kfree_sensitive) = kzalloc(resp_len, GFP_KERNEL);
+ struct tio_msg_mmio_validate_req req = {
+ .guest_device_id = pci_dev_id(pdev),
+ .subrange_base = start,
+ .subrange_page_count = size >> PAGE_SHIFT,
+ .range_offset = 0,
+ .validated = !invalidate, /* Desired value to set RMP.Validated for the range */
+ .force_validated = 0,
+ .range_id = range_id,
+ };
+ u64 bdfn = pci_dev_id(pdev);
+ u64 mmio_val = MMIO_MK_VALIDATE(start, size, range_id, !invalidate);
+ int rc;
+
+ if (!rsp)
+ return -ENOMEM;
+
+ rc = handle_tio_guest_request(snp_dev, TIO_MSG_MMIO_VALIDATE_REQ,
+ &req, sizeof(req), rsp, resp_len,
+ NULL, NULL, &bdfn, &mmio_val, fw_err);
+ if (rc)
+ return rc;
+
+ *status = rsp->status;
+
+ return 0;
+}
+
+static bool get_range(struct pci_dev *pdev, struct tsm_blob *report, unsigned int index,
+ unsigned int *range_id, resource_size_t *start, resource_size_t *size)
+{
+ struct tdi_report_mmio_range mr = TDI_REPORT_MR(report, index);
+ unsigned int rangeid = FIELD_GET(TSM_TDI_REPORT_MMIO_RANGE_ID, mr.range_attributes);
+ struct resource *r = pci_resource_n(pdev, rangeid);
+ u64 first, offset;
+ unsigned int i;
+
+ if (FIELD_GET(TSM_TDI_REPORT_MMIO_IS_NON_TEE, mr.range_attributes)) {
+ pci_info(pdev, "Skipping non-TEE range [%d] #%d %d pages, %llx..%llx\n",
+ index, rangeid, mr.num, r->start, r->end);
+ return false;
+ }
+
+ /* Currently not supported */
+ if (FIELD_GET(TSM_TDI_REPORT_MMIO_MSIX_TABLE, mr.range_attributes) ||
+ FIELD_GET(TSM_TDI_REPORT_MMIO_PBA, mr.range_attributes)) {
+ pci_info(pdev, "Skipping MSIX (%ld/%ld) range [%d] #%d %d pages, %llx..%llx\n",
+ FIELD_GET(TSM_TDI_REPORT_MMIO_MSIX_TABLE, mr.range_attributes),
+ FIELD_GET(TSM_TDI_REPORT_MMIO_PBA, mr.range_attributes),
+ index, rangeid, mr.num, r->start, r->end);
+ return false;
+ }
+
+ /*
+ * First the first subregion of BAR, i.e. with the smallest .first_page.
+ * This assumes that the same MMIO_REPORTING_OFFSET is applied to all regions.
+ * */
+ for (i = 0, first = mr.first_page; i < TDI_REPORT_MR_NUM(report); ++i) {
+ struct tdi_report_mmio_range mrtmp = TDI_REPORT_MR(report, i);
+
+ if (rangeid != FIELD_GET(TSM_TDI_REPORT_MMIO_RANGE_ID, mrtmp.range_attributes))
+ continue;
+
+ first = min(mrtmp.first_page, first);
+ }
+
+ offset = mr.first_page - first;
+ if (((offset + mr.num) << PAGE_SHIFT) > (r->end - r->start + 1)) {
+ pci_warn(pdev, "Skipping broken range [%d] BAR%d off=%llx %d pages, %llx..%llx %llx %llx\n",
+ index, rangeid, offset, mr.num, r->start, r->end, mr.first_page, first);
+ return false;
+ }
+
+ *range_id = rangeid;
+ *start = r->start + offset;
+ *size = mr.num << PAGE_SHIFT;
+
+ return true;
+}
+
+static int tio_tdi_mmio_validate(struct pci_dev *pdev, struct snp_guest_dev *snp_dev)
+{
+ struct pci_tsm *tsm = pdev->tsm;
+ u16 mmio_status;
+ u64 fw_err = 0;
+ int i = 0, rc = 0;
+ struct pci_tsm_mmio *mmio __free(kfree) =
+ kzalloc(struct_size(mmio, res, PCI_NUM_RESOURCES), GFP_KERNEL);
+
+ if (!mmio)
+ return -ENOMEM;
+
+ if (WARN_ON_ONCE(!tsm || !tsm->report))
+ return -ENODEV;
+
+ pci_notice(pdev, "MMIO validate");
+
+ for (i = 0; i < TDI_REPORT_MR_NUM(tsm->report); ++i) {
+ unsigned int range_id;
+ resource_size_t start = 0, size = 0, end;
+
+ if (!get_range(pdev, tsm->report, i, &range_id, &start, &size))
+ continue;
+
+ end = start + size - 1;
+ mmio_status = 0;
+ rc = mmio_validate_range(snp_dev, pdev, range_id, start, size,
+ false, &fw_err,
+ &mmio_status);
+ if (rc || fw_err != SEV_RET_SUCCESS || mmio_status != MMIO_VALIDATE_SUCCESS) {
+ pci_err(pdev, "MMIO #%d %llx..%llx validation failed 0x%llx %d\n",
+ range_id, start, end, fw_err, mmio_status);
+ continue;
+ }
+
+ mmio->res[mmio->nr] = DEFINE_RES_NAMED_DESC(start, size, "PCI MMIO Encrypted",
+ pci_resource_flags(pdev, range_id), IORES_DESC_ENCRYPTED);
+ ++mmio->nr;
+
+ pci_notice(pdev, "MMIO #%d %llx..%llx validated\n",range_id, start, end);
+ }
+
+ if (!rc) {
+ rc = pci_tsm_mmio_setup(pdev, mmio);
+ if (!rc) {
+ struct pci_tsm_devsec *devsec_tsm = to_pci_tsm_devsec(tsm);
+
+ devsec_tsm->mmio = no_free_ptr(mmio);
+ }
+ }
+
+ return rc;
+}
+
+static void tio_tdi_mmio_invalidate(struct pci_dev *pdev, struct snp_guest_dev *snp_dev)
+{
+ struct pci_tsm *tsm = pdev->tsm;
+ u16 mmio_status;
+ u64 fw_err = 0;
+ int i = 0, rc = 0;
+ struct pci_tsm_devsec *devsec_tsm = to_pci_tsm_devsec(tsm);
+ struct pci_tsm_mmio *mmio = devsec_tsm->mmio;
+
+ if (!mmio)
+ return;
+
+ pci_notice(pdev, "MMIO invalidate");
+
+ for (i = 0; i < TDI_REPORT_MR_NUM(tsm->report); ++i) {
+ unsigned int range_id;
+ resource_size_t start = 0, size = 0, end;
+
+ if (!get_range(pdev, tsm->report, i, &range_id, &start, &size))
+ continue;
+
+ end = start + size - 1;
+ mmio_status = 0;
+ rc = mmio_validate_range(snp_dev, pdev, range_id,
+ start, size, true, &fw_err,
+ &mmio_status);
+ if (rc || fw_err != SEV_RET_SUCCESS || mmio_status != MMIO_VALIDATE_SUCCESS) {
+ pci_err(pdev, "MMIO #%d %llx..%llx validation failed 0x%llx %d\n",
+ range_id, start, end, fw_err, mmio_status);
+ continue;
+ }
+
+ pci_notice(pdev, "MMIO #%d %llx..%llx invalidated\n", range_id, start, end);
+ }
+
+ pci_tsm_mmio_teardown(devsec_tsm->mmio);
+ kfree(devsec_tsm->mmio);
+ devsec_tsm->mmio = NULL;
+}
+
+struct sdte {
+ u64 v : 1;
+ u64 reserved : 3;
+ u64 cxlio : 3;
+ u64 reserved1 : 45;
+ u64 ppr : 1;
+ u64 reserved2 : 1;
+ u64 giov : 1;
+ u64 gv : 1;
+ u64 glx : 2;
+ u64 gcr3_tbl_rp0 : 3;
+ u64 ir : 1;
+ u64 iw : 1;
+ u64 reserved3 : 1;
+ u16 domain_id;
+ u16 gcr3_tbl_rp1;
+ u32 interrupt : 1;
+ u32 reserved4 : 5;
+ u32 ex : 1;
+ u32 sd : 1;
+ u32 reserved5 : 2;
+ u32 sats : 1;
+ u32 gcr3_tbl_rp2 : 21;
+ u64 giv : 1;
+ u64 gint_tbl_len : 4;
+ u64 reserved6 : 1;
+ u64 gint_tbl : 46;
+ u64 reserved7 : 2;
+ u64 gpm : 2;
+ u64 reserved8 : 3;
+ u64 hpt_mode : 1;
+ u64 reserved9 : 4;
+ u32 asid : 12;
+ u32 reserved10 : 3;
+ u32 viommu_en : 1;
+ u32 guest_device_id : 16;
+ u32 guest_id : 15;
+ u32 guest_id_mbo : 1;
+ u32 reserved11 : 1;
+ u32 vmpl : 2;
+ u32 reserved12 : 3;
+ u32 attrv : 1;
+ u32 reserved13 : 1;
+ u32 sa : 8;
+ u8 ide_stream_id[8];
+ u32 vtom_en : 1;
+ u32 vtom : 31;
+ u32 rp_id : 5;
+ u32 reserved14 : 27;
+ u8 reserved15[0x40-0x30];
+} __packed;
+
+struct tio_msg_sdte_write_req {
+ u16 guest_device_id;
+ u8 reserved[14];
+ struct sdte sdte;
+} __packed;
+
+struct tio_msg_sdte_write_rsp {
+ u16 guest_device_id;
+ u16 status; /* SDTE_WRITE_xxx */
+ u8 reserved[12];
+} __packed;
+
+static int tio_tdi_sdte_write(struct pci_dev *pdev, struct snp_guest_dev *snp_dev, bool invalidate)
+{
+ struct snp_msg_desc *mdesc = snp_dev->msg_desc;
+ size_t resp_len = sizeof(struct tio_msg_sdte_write_rsp) + mdesc->ctx->authsize;
+ struct tio_msg_sdte_write_rsp *rsp __free(kfree_sensitive) = kzalloc(resp_len, GFP_KERNEL);
+ struct tio_msg_sdte_write_req req;
+ u64 fw_err = 0;
+ u64 bdfn = pci_dev_id(pdev);
+ u64 flags = invalidate ? 0 : SDTE_VALIDATE;
+ int rc;
+
+ BUILD_BUG_ON(sizeof(struct sdte) * 8 != 512);
+
+ if (!invalidate)
+ req = (struct tio_msg_sdte_write_req) {
+ .guest_device_id = bdfn,
+ .sdte.vmpl = 0,
+ .sdte.vtom = tsm_vtom,
+ .sdte.vtom_en = 1,
+ .sdte.iw = 1,
+ .sdte.ir = 1,
+ .sdte.v = 1,
+ };
+ else
+ req = (struct tio_msg_sdte_write_req) {
+ .guest_device_id = bdfn,
+ };
+
+ pci_notice(pdev, "SDTE write vTOM=%lx", (unsigned long) req.sdte.vtom << 21);
+
+ if (!rsp)
+ return -ENOMEM;
+
+ rc = handle_tio_guest_request(snp_dev, TIO_MSG_SDTE_WRITE_REQ,
+ &req, sizeof(req), rsp, resp_len,
+ NULL, NULL, &bdfn, &flags, &fw_err);
+ if (rc) {
+ pci_err(pdev, "SDTE write failed with 0x%llx\n", fw_err);
+ return rc;
+ }
+
+ return 0;
+}
+
+static struct pci_tsm *sev_guest_lock(struct tsm_dev *tsmdev, struct pci_dev *pdev)
+{
+ struct tio_guest_tdi *gtdi __free(kfree) = kzalloc(sizeof(*gtdi), GFP_KERNEL);
+ struct tsm_blob *report = NULL;
+ struct tsm_tdi_status ts = {};
+ u64 fw_err = 0, tdi_id = 0;
+ int rc;
+
+ if (!gtdi)
+ return ERR_PTR(-ENOMEM);
+
+ /* Enabling device tells the HV to register MMIO as memory slots */
+ rc = pci_enable_device_mem(pdev);
+ if (rc)
+ return ERR_PTR(rc);
+
+ rc = pci_tsm_devsec_constructor(pdev, >di->ds, tsmdev);
+ if (rc)
+ return ERR_PTR(rc);
+
+ pci_dbg(pdev, "TSM enabled\n");
+
+ gtdi->snp_dev = tsm_dev_to_snp_dev(tsmdev);
+
+ rc = sev_tio_op(pci_dev_id(pdev), SVM_VMGEXIT_SEV_TIO_OP_BIND, &fw_err, &tdi_id);
+ if (rc) {
+ pci_err(pdev, "TDI bind CONFIG_LOCKED failed rc=%d fw=0x%llx\n",
+ rc, fw_err);
+ return ERR_PTR(rc);
+ }
+ pci_dbg(pdev, "New TDI ID=%llx\n", tdi_id);
+
+ rc = tio_tdi_status(pdev, gtdi->snp_dev, &ts, &report);
+ if (rc)
+ return ERR_PTR(rc);
+ if (!report)
+ return ERR_PTR(-ENODEV);
+
+ gtdi->tdi_id = tdi_id;
+ gtdi->ds.base_tsm.report = report;
+
+ return &no_free_ptr(gtdi)->ds.base_tsm;
+}
+
+static void sev_guest_unlock(struct pci_tsm *tsm)
+{
+ struct pci_dev *pdev = tsm->pdev;
+ struct tio_guest_tdi *gtdi = pdev_to_tdi(pdev);
+ struct snp_guest_dev *snp_dev = gtdi->snp_dev;
+ u64 fw_err = 0;
+ int rc;
+
+ /* Quiesce DMA */
+ sev_tio_op(pci_dev_id(pdev), SVM_VMGEXIT_SEV_TIO_OP_STOP, &fw_err, NULL);
+
+ /* Disable encrypted DMA but the HV is unable to restart it as MMIO is still blocked for HV */
+ rc = tio_tdi_sdte_write(pdev, snp_dev, true);
+ if (rc || fw_err)
+ pr_err("SDTE_WRITE did not go through, ret=%d fw=0x%llx\n", rc, fw_err);
+
+ tio_tdi_mmio_invalidate(pdev, snp_dev);
+
+ sev_tio_op(pci_dev_id(pdev), SVM_VMGEXIT_SEV_TIO_OP_UNBIND, &fw_err, NULL);
+
+ tsm->pdev->tsm = NULL;
+ kvfree(tsm);
+}
+
+static int sev_guest_accept(struct pci_dev *pdev)
+{
+ struct tio_guest_tdi *gtdi = pdev_to_tdi(pdev);
+ struct snp_guest_dev *snp_dev = gtdi->snp_dev;
+ struct pci_tsm *tsm = pdev->tsm;
+ u64 fw_err = 0;
+ int ret;
+
+ if (!tsm->report) {
+ pci_warn_once(pdev, "Cannot accept without the report");
+ return -ENODEV;
+ }
+
+ ret = sev_tio_op(pci_dev_id(pdev), SVM_VMGEXIT_SEV_TIO_OP_RUN, &fw_err, NULL);
+ if (ret)
+ return ret;
+
+ ret = tio_tdi_sdte_write(pdev, snp_dev, false);
+ if (ret)
+ return ret;
+
+ ret = tio_tdi_mmio_validate(pdev, snp_dev);
+
+ return ret;
+}
+
+struct pci_tsm_ops sev_guest_tsm_ops = {
+ .lock = sev_guest_lock,
+ .unlock = sev_guest_unlock,
+ .accept = sev_guest_accept,
+};
+
+void sev_guest_tsm_set_ops(bool set, struct snp_guest_dev *snp_dev)
+{
+ if (set) {
+ struct tsm_dev *tsmdev;
+
+ tsmdev = tsm_register(snp_dev->dev, &sev_guest_tsm_ops);
+ if (IS_ERR(tsmdev))
+ return;
+
+ snp_dev->tsmdev = tsmdev;
+ return;
+ }
+
+ if (snp_dev->tsmdev) {
+ tsm_unregister(snp_dev->tsmdev);
+ snp_dev->tsmdev = NULL;
+ }
+}
--
2.52.0
^ permalink raw reply related
* [PATCH kernel 6/9] x86/dma-direct: Stop changing encrypted page state for TDISP devices
From: Alexey Kardashevskiy @ 2026-02-25 5:37 UTC (permalink / raw)
To: x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
In-Reply-To: <20260225053806.3311234-1-aik@amd.com>
TDISP devices operate in CoCo VMs only and capable of accessing
encrypted guest memory.
Currently when SME is on, the DMA subsystem forces the SME mask in
DMA handles in phys_to_dma() which assumes IOMMU pass through
which is never the case with CoCoVM running with a TDISP device.
Define X86's version of phys_to_dma() to skip leaking SME mask to
the device.
Signed-off-by: Alexey Kardashevskiy <aik@amd.com>
---
Doing this in the generic version breaks ARM which uses
the SME mask in DMA handles, hence ARCH_HAS_PHYS_TO_DMA.
pci_device_add() enforces the FFFF_FFFF coherent DMA mask so
dma_alloc_coherent() fails when SME=on, this is how I ended up fixing
phys_to_dma() and not quite sure it is the right fix.
---
arch/x86/Kconfig | 1 +
arch/x86/include/asm/dma-direct.h | 39 ++++++++++++++++++++
2 files changed, 40 insertions(+)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index fa3b616af03a..c46283064518 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -112,6 +112,7 @@ config X86
select ARCH_HAS_UBSAN
select ARCH_HAS_DEBUG_WX
select ARCH_HAS_ZONE_DMA_SET if EXPERT
+ select ARCH_HAS_PHYS_TO_DMA
select ARCH_HAVE_NMI_SAFE_CMPXCHG
select ARCH_HAVE_EXTRA_ELF_NOTES
select ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE
diff --git a/arch/x86/include/asm/dma-direct.h b/arch/x86/include/asm/dma-direct.h
new file mode 100644
index 000000000000..f50e03d643c1
--- /dev/null
+++ b/arch/x86/include/asm/dma-direct.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef ASM_X86_DMA_DIRECT_H
+#define ASM_X86_DMA_DIRECT_H 1
+
+static inline dma_addr_t __phys_to_dma(struct device *dev, phys_addr_t paddr)
+{
+ if (dev->dma_range_map)
+ return translate_phys_to_dma(dev, paddr);
+ return paddr;
+}
+
+static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr)
+{
+ /*
+ * TDISP devices only work in CoCoVMs and rely on IOMMU to
+ * decide on the memory encryption.
+ * Stop leaking the SME mask in DMA handles and return
+ * the real address.
+ */
+ if (device_cc_accepted(dev))
+ return dma_addr_unencrypted(__phys_to_dma(dev, paddr));
+
+ return dma_addr_encrypted(__phys_to_dma(dev, paddr));
+}
+
+static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
+{
+ return daddr;
+}
+
+static inline dma_addr_t phys_to_dma_unencrypted(struct device *dev,
+ phys_addr_t paddr)
+{
+ return dma_addr_unencrypted(__phys_to_dma(dev, paddr));
+}
+
+#define phys_to_dma_unencrypted phys_to_dma_unencrypted
+
+#endif /* ASM_X86_DMA_DIRECT_H */
--
2.52.0
^ permalink raw reply related
* [PATCH kernel 5/9] x86/mm: Stop forcing decrypted page state for TDISP devices
From: Alexey Kardashevskiy @ 2026-02-25 5:37 UTC (permalink / raw)
To: x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
In-Reply-To: <20260225053806.3311234-1-aik@amd.com>
The DMA subsystem does is forcing private-to-shared
page conversion in force_dma_unencrypted().
Return false from force_dma_unencrypted() for TDISP devices.
Signed-off-by: Alexey Kardashevskiy <aik@amd.com>
---
arch/x86/mm/mem_encrypt.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 95bae74fdab2..8daa6482b080 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -20,10 +20,11 @@
bool force_dma_unencrypted(struct device *dev)
{
/*
- * For SEV, all DMA must be to unencrypted addresses.
+ * dma_direct_alloc() forces page state change if private memory is
+ * allocated for DMA. Skip conversion if the TDISP device is accepted.
*/
if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
- return true;
+ return !device_cc_accepted(dev);
/*
* For SME, all DMA must be to unencrypted addresses if the
--
2.52.0
^ permalink raw reply related
* [PATCH kernel 4/9] dma/swiotlb: Stop forcing SWIOTLB for TDISP devices
From: Alexey Kardashevskiy @ 2026-02-25 5:37 UTC (permalink / raw)
To: x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
In-Reply-To: <20260225053806.3311234-1-aik@amd.com>
SWIOTLB is enforced when encrypted guest memory is detected
in pci_swiotlb_detect() which is required for legacy devices.
Skip SWIOTLB for TDISP devices.
Signed-off-by: Alexey Kardashevskiy <aik@amd.com>
---
include/linux/swiotlb.h | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 3dae0f592063..119c25d639a7 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -173,6 +173,15 @@ static inline bool is_swiotlb_force_bounce(struct device *dev)
{
struct io_tlb_mem *mem = dev->dma_io_tlb_mem;
+ /*
+ * CC_ATTR_GUEST_MEM_ENCRYPT enforces SWIOTLB_FORCE in
+ * swiotlb_init_remap() to allow legacy devices access arbitrary
+ * VM encrypted memory.
+ * Skip it for TDISP devices capable of DMA-ing the encrypted memory.
+ */
+ if (device_cc_accepted(dev))
+ return false;
+
return mem && mem->force_bounce;
}
--
2.52.0
^ permalink raw reply related
* [PATCH kernel 3/9] coco/sev-guest: Allow multiple source files in the driver
From: Alexey Kardashevskiy @ 2026-02-25 5:37 UTC (permalink / raw)
To: x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
In-Reply-To: <20260225053806.3311234-1-aik@amd.com>
Prepare for SEV-TIO support as it is going to equal or bigger
than the existing sev_guest.c which is already 700 lines and
keeps growing.
No behavioural change expected.
Signed-off-by: Alexey Kardashevskiy <aik@amd.com>
---
drivers/virt/coco/sev-guest/Makefile | 3 ++-
drivers/virt/coco/sev-guest/sev-guest.h | 16 ++++++++++++++++
drivers/virt/coco/sev-guest/sev-guest.c | 10 ++--------
3 files changed, 20 insertions(+), 9 deletions(-)
diff --git a/drivers/virt/coco/sev-guest/Makefile b/drivers/virt/coco/sev-guest/Makefile
index 63d67c27723a..9604792e0095 100644
--- a/drivers/virt/coco/sev-guest/Makefile
+++ b/drivers/virt/coco/sev-guest/Makefile
@@ -1,2 +1,3 @@
# SPDX-License-Identifier: GPL-2.0-only
-obj-$(CONFIG_SEV_GUEST) += sev-guest.o
+obj-$(CONFIG_SEV_GUEST) += sev_guest.o
+sev_guest-y += sev-guest.o
diff --git a/drivers/virt/coco/sev-guest/sev-guest.h b/drivers/virt/coco/sev-guest/sev-guest.h
new file mode 100644
index 000000000000..b2a97778e635
--- /dev/null
+++ b/drivers/virt/coco/sev-guest/sev-guest.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef __SEV_GUEST_H__
+#define __SEV_GUEST_H__
+
+#include <linux/miscdevice.h>
+#include <asm/sev.h>
+
+struct snp_guest_dev {
+ struct device *dev;
+ struct miscdevice misc;
+
+ struct snp_msg_desc *msg_desc;
+};
+
+#endif /* __SEV_GUEST_H__ */
diff --git a/drivers/virt/coco/sev-guest/sev-guest.c b/drivers/virt/coco/sev-guest/sev-guest.c
index b01ec99106cd..e1ceeab54a21 100644
--- a/drivers/virt/coco/sev-guest/sev-guest.c
+++ b/drivers/virt/coco/sev-guest/sev-guest.c
@@ -28,19 +28,13 @@
#include <uapi/linux/psp-sev.h>
#include <asm/svm.h>
-#include <asm/sev.h>
+
+#include "sev-guest.h"
#define DEVICE_NAME "sev-guest"
#define SVSM_MAX_RETRIES 3
-struct snp_guest_dev {
- struct device *dev;
- struct miscdevice misc;
-
- struct snp_msg_desc *msg_desc;
-};
-
/*
* The VMPCK ID represents the key used by the SNP guest to communicate with the
* SEV firmware in the AMD Secure Processor (ASP, aka PSP). By default, the key
--
2.52.0
^ permalink raw reply related
* [PATCH kernel 2/9] pci/tsm: Add tsm_tdi_status
From: Alexey Kardashevskiy @ 2026-02-25 5:37 UTC (permalink / raw)
To: x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
In-Reply-To: <20260225053806.3311234-1-aik@amd.com>
Define a structure with all info about a TDI such as TDISP status,
bind state, used START_INTERFACE options and the report digest.
This will be extended and shared to the userspace.
Signed-off-by: Alexey Kardashevskiy <aik@amd.com>
---
Make it uapi? We might want a sysfs node per a field so probably not.
For now its only user is AMD SEV TIO with a plan to expose this struct
as a whole via sysfs.
---
include/linux/pci-tsm.h | 26 ++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/include/linux/pci-tsm.h b/include/linux/pci-tsm.h
index 7987ede76914..8086f358c51b 100644
--- a/include/linux/pci-tsm.h
+++ b/include/linux/pci-tsm.h
@@ -355,4 +355,30 @@ struct tdi_report_footer {
#define TDI_REPORT_FTR(rep) ((struct tdi_report_footer *) &TDI_REPORT_MR((rep), \
TDI_REPORT_MR_NUM(rep)))
+enum tsm_tdisp_state {
+ TDISP_STATE_CONFIG_UNLOCKED = 0,
+ TDISP_STATE_CONFIG_LOCKED = 1,
+ TDISP_STATE_RUN = 2,
+ TDISP_STATE_ERROR = 3,
+};
+
+enum tsm_tdisp_status {
+ TDISP_STATE_BOUND = 0,
+ TDISP_STATE_INVALID = 1,
+ TDISP_STATE_UNBOUND = 2,
+};
+
+struct tsm_tdi_status {
+ __u8 status; /* enum tsm_tdisp_status */
+ __u8 state; /* enum tsm_tdisp_state */
+ __u8 all_request_redirect;
+ __u8 bind_p2p;
+ __u8 lock_msix;
+ __u8 no_fw_update;
+ __u16 cache_line_size;
+ __u8 interface_report_digest[48];
+ __u64 intf_report_counter;
+ struct tdisp_interface_id id;
+} __packed;
+
#endif /*__PCI_TSM_H */
--
2.52.0
^ permalink raw reply related
* [PATCH kernel 1/9] pci/tsm: Add TDISP report blob and helpers to parse it
From: Alexey Kardashevskiy @ 2026-02-25 5:37 UTC (permalink / raw)
To: x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
In-Reply-To: <20260225053806.3311234-1-aik@amd.com>
The TDI interface report is defined in PCIe r7.0,
chapter "11.3.11 DEVICE_INTERFACE_REPORT". The report enumerates
MMIO resources and their properties which will take effect upon
transitioning to the RUN state.
Store the report in pci_tsm.
Define macros and helpers to parse the binary blob.
Signed-off-by: Alexey Kardashevskiy <aik@amd.com>
---
Probably pci_tsm::report could be struct tdi_report_header*?
---
include/linux/pci-tsm.h | 84 ++++++++++++++++++++
drivers/virt/coco/tsm-core.c | 19 +++++
2 files changed, 103 insertions(+)
diff --git a/include/linux/pci-tsm.h b/include/linux/pci-tsm.h
index b984711fa91f..7987ede76914 100644
--- a/include/linux/pci-tsm.h
+++ b/include/linux/pci-tsm.h
@@ -10,6 +10,18 @@ struct tsm_dev;
struct kvm;
enum pci_tsm_req_scope;
+/* Data object for measurements/certificates/attestationreport */
+struct tsm_blob {
+ void *data;
+ size_t len;
+};
+
+struct tsm_blob *tsm_blob_new(void *data, size_t len);
+static inline void tsm_blob_free(struct tsm_blob *b)
+{
+ kfree(b);
+}
+
/*
* struct pci_tsm_ops - manage confidential links and security state
* @link_ops: Coordinate PCIe SPDM and IDE establishment via a platform TSM.
@@ -123,6 +135,7 @@ struct pci_tsm {
struct pci_dev *dsm_dev;
struct tsm_dev *tsm_dev;
struct pci_tdi *tdi;
+ struct tsm_blob *report;
};
/**
@@ -271,4 +284,75 @@ static inline ssize_t pci_tsm_guest_req(struct pci_dev *pdev,
return -ENXIO;
}
#endif
+
+/*
+ * struct tdisp_interface_id - TDISP INTERFACE_ID Definition
+ *
+ * @function_id: Identifies the function of the device hosting the TDI
+ * 15:0: @rid: Requester ID
+ * 23:16: @rseg: Requester Segment (Reserved if Requester Segment Valid is Clear)
+ * 24: @rseg_valid: Requester Segment Valid
+ * 31:25 – Reserved
+ * 8B - Reserved
+ */
+#define TSM_TDISP_IID_REQUESTER_ID GENMASK(15, 0)
+#define TSM_TDISP_IID_RSEG GENMASK(23, 16)
+#define TSM_TDISP_IID_RSEG_VALID BIT(24)
+
+struct tdisp_interface_id {
+ __u32 function_id; /* TSM_TDISP_IID_xxxx */
+ __u8 reserved[8];
+} __packed;
+
+#define SPDM_MEASUREMENTS_NONCE_LEN 32
+typedef __u8 spdm_measurements_nonce_t[SPDM_MEASUREMENTS_NONCE_LEN];
+
+/*
+ * TDI Report Structure as defined in TDISP.
+ */
+#define _BITSH(x) (1 << (x))
+#define TSM_TDI_REPORT_NO_FW_UPDATE _BITSH(0) /* not updates in CONFIG_LOCKED or RUN */
+#define TSM_TDI_REPORT_DMA_NO_PASID _BITSH(1) /* TDI generates DMA requests without PASID */
+#define TSM_TDI_REPORT_DMA_PASID _BITSH(2) /* TDI generates DMA requests with PASID */
+#define TSM_TDI_REPORT_ATS _BITSH(3) /* ATS supported and enabled for the TDI */
+#define TSM_TDI_REPORT_PRS _BITSH(4) /* PRS supported and enabled for the TDI */
+
+struct tdi_report_header {
+ __u16 interface_info; /* TSM_TDI_REPORT_xxx */
+ __u16 reserved2;
+ __u16 msi_x_message_control;
+ __u16 lnr_control;
+ __u32 tph_control;
+ __u32 mmio_range_count;
+} __packed;
+
+/*
+ * Each MMIO Range of the TDI is reported with the MMIO reporting offset added.
+ * Base and size in units of 4K pages
+ */
+#define TSM_TDI_REPORT_MMIO_MSIX_TABLE BIT(0)
+#define TSM_TDI_REPORT_MMIO_PBA BIT(1)
+#define TSM_TDI_REPORT_MMIO_IS_NON_TEE BIT(2)
+#define TSM_TDI_REPORT_MMIO_IS_UPDATABLE BIT(3)
+#define TSM_TDI_REPORT_MMIO_RESERVED GENMASK(15, 4)
+#define TSM_TDI_REPORT_MMIO_RANGE_ID GENMASK(31, 16)
+
+struct tdi_report_mmio_range {
+ __u64 first_page; /* First 4K page with offset added */
+ __u32 num; /* Number of 4K pages in this range */
+ __u32 range_attributes; /* TSM_TDI_REPORT_MMIO_xxx */
+} __packed;
+
+struct tdi_report_footer {
+ __u32 device_specific_info_len;
+ __u8 device_specific_info[];
+} __packed;
+
+#define TDI_REPORT_HDR(rep) ((struct tdi_report_header *) ((rep)->data))
+#define TDI_REPORT_MR_NUM(rep) (TDI_REPORT_HDR(rep)->mmio_range_count)
+#define TDI_REPORT_MR_OFF(rep) ((struct tdi_report_mmio_range *) (TDI_REPORT_HDR(rep) + 1))
+#define TDI_REPORT_MR(rep, rangeid) TDI_REPORT_MR_OFF(rep)[rangeid]
+#define TDI_REPORT_FTR(rep) ((struct tdi_report_footer *) &TDI_REPORT_MR((rep), \
+ TDI_REPORT_MR_NUM(rep)))
+
#endif /*__PCI_TSM_H */
diff --git a/drivers/virt/coco/tsm-core.c b/drivers/virt/coco/tsm-core.c
index e65ab3461d14..3929176b8d3b 100644
--- a/drivers/virt/coco/tsm-core.c
+++ b/drivers/virt/coco/tsm-core.c
@@ -16,6 +16,25 @@ static struct class *tsm_class;
static DECLARE_RWSEM(tsm_rwsem);
static DEFINE_IDA(tsm_ida);
+struct tsm_blob *tsm_blob_new(void *data, size_t len)
+{
+ struct tsm_blob *b;
+
+ if (!len || !data)
+ return NULL;
+
+ b = kzalloc(sizeof(*b) + len, GFP_KERNEL);
+ if (!b)
+ return NULL;
+
+ b->data = (void *)b + sizeof(*b);
+ b->len = len;
+ memcpy(b->data, data, len);
+
+ return b;
+}
+EXPORT_SYMBOL_GPL(tsm_blob_new);
+
static int match_id(struct device *dev, const void *data)
{
struct tsm_dev *tsm_dev = container_of(dev, struct tsm_dev, dev);
--
2.52.0
^ permalink raw reply related
* [PATCH kernel 0/9] PCI/TSM: coco/sev-guest: Implement SEV-TIO PCIe TDISP (phase2)
From: Alexey Kardashevskiy @ 2026-02-25 5:37 UTC (permalink / raw)
To: x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
Here are some patches to continue enabling SEV-TIO on AMD.
SEV-TIO allows guests to establish trust in a device that supports TEE
Device Interface Security Protocol (TDISP, defined in PCIe r6.0+) and
then interact with the device via private memory.
In order to streamline upstreaming process, a common TSM infrastructure
is being developed in collaboration with Intel+ARM+RiscV. There is
Documentation/driver-api/pci/tsm.rst with proposed phases:
1. IDE: encrypt PCI, host only
2. TDISP: lock + accept flow, host and guest, interface report
3. Enable secure MMIO + DMA: IOMMUFD, KVM changes
4. Device attestation: certificates, measurements
This is phase2 == basic guest support allowing TDISP CONFIG_LOCKED and RUN states, and unlocking as well.
Acronyms:
TEE - Trusted Execution Environments, a concept of managing trust between the host and devices
TSM - TEE Security Manager (TSM), an entity which ensures security on the host
PSP - AMD platform secure processor (also "ASP", "AMD-SP"), acts as TSM on AMD.
SEV TIO - the TIO protocol implemented by the PSP and used by the host, extension to SEV-SNP
GHCB - guest/host communication block - a protocol for guest-to-host communication via a shared page
TDISP - TEE Device Interface Security Protocol (PCIe).
Flow:
- Boot guest OS, load sev-guest.ko which registers itself as a TSM
- PCI TSM creates sysfs nodes under "tsm" subdirectory in for all
TDISP-capable devices
- lock the device via:
echo tsm0 > "/sys/bus/pci/devices/0000:01:00.0/tsm/lock"
- accept the device via:
echo 1 > "/sys/bus/pci/devices/0000:01:00.0/tsm/accept"
- load the device driver:
- DMA to encrypted memory should work right away
- MMIO regions reported in TDISP interface report will be mapped as encrypted
Since one of my test devices does not use private MMIO for the main function,
there is 9/9 which allows https://github.com/billfarrow/pcimem.git mapping MMIO as private.
The previous conversation is here:
https://lore.kernel.org/r/20250218111017.491719-1-aik@amd.com
This is based on sha1
4fe8662d1a9c Dan Williams PCI/TSM: Documentation: Add Maturity Map
from
https://git.kernel.org/pub/scm/linux/kernel/git/devsec/tsm.git/log/?h=staging
and 3 cherrypicks on top, please find the exact tree at:
https://github.com/AMDESE/linux-kvm/commits/tsm-staging
The host support is pushed here:
https://github.com/AMDESE/linux-kvm/commits/tsm
The SEV TIO spec:
https://www.amd.com/content/dam/amd/en/documents/epyc-technical-docs/specifications/58271.pdf
Individual patches have extra "---" comments (could have been "RFC"?)
Please comment. Thanks.
ps: quite a cc list from get_maintainers.pl.
Alexey Kardashevskiy (9):
pci/tsm: Add TDISP report blob and helpers to parse it
pci/tsm: Add tsm_tdi_status
coco/sev-guest: Allow multiple source files in the driver
dma/swiotlb: Stop forcing SWIOTLB for TDISP devices
x86/mm: Stop forcing decrypted page state for TDISP devices
x86/dma-direct: Stop changing encrypted page state for TDISP devices
coco/sev-guest: Implement the guest support for SEV TIO (phase2)
RFC: PCI: Avoid needless touching of Command register
pci: Allow encrypted MMIO mapping via sysfs
arch/x86/Kconfig | 1 +
drivers/virt/coco/sev-guest/Kconfig | 1 +
drivers/virt/coco/sev-guest/Makefile | 6 +-
arch/x86/include/asm/dma-direct.h | 39 ++
arch/x86/include/asm/sev-common.h | 1 +
arch/x86/include/asm/sev.h | 13 +
arch/x86/include/uapi/asm/svm.h | 13 +
drivers/virt/coco/sev-guest/sev-guest.h | 20 +
include/linux/pci-tsm.h | 110 +++
include/linux/pci.h | 2 +-
include/linux/psp-sev.h | 31 +
include/linux/swiotlb.h | 9 +
include/uapi/linux/sev-guest.h | 43 ++
arch/x86/coco/sev/core.c | 53 ++
arch/x86/mm/mem_encrypt.c | 5 +-
drivers/pci/mmap.c | 11 +-
drivers/pci/pci-sysfs.c | 27 +-
drivers/pci/probe.c | 5 +
drivers/pci/proc.c | 2 +-
drivers/pci/quirks.c | 9 +
drivers/virt/coco/sev-guest/sev-guest.c | 23 +-
drivers/virt/coco/sev-guest/tio.c | 707 ++++++++++++++++++++
drivers/virt/coco/tsm-core.c | 19 +
23 files changed, 1129 insertions(+), 21 deletions(-)
create mode 100644 arch/x86/include/asm/dma-direct.h
create mode 100644 drivers/virt/coco/sev-guest/sev-guest.h
create mode 100644 drivers/virt/coco/sev-guest/tio.c
--
2.52.0
^ permalink raw reply
* Re: [PATCH] KVM: TDX: Set SIGNIFCANT_INDEX flag for supported CPUIDs
From: Binbin Wu @ 2026-02-25 3:23 UTC (permalink / raw)
To: Sean Christopherson
Cc: Rick P Edgecombe, Xiaoyao Li, changyuanl@google.com,
pbonzini@redhat.com, Binbin Wu, Isaku Yamahata, bp@alien8.de,
x86@kernel.org, kas@kernel.org, hpa@zytor.com, mingo@redhat.com,
linux-kernel@vger.kernel.org, dave.hansen@linux.intel.com,
tglx@kernel.org, kvm@vger.kernel.org, linux-coco@lists.linux.dev
In-Reply-To: <aZ3LxD5XMepnU8jh@google.com>
On 2/25/2026 12:03 AM, Sean Christopherson wrote:
> On Tue, Feb 24, 2026, Binbin Wu wrote:
>> On 2/24/2026 9:57 AM, Edgecombe, Rick P wrote:
>>>> diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
>>>> index 2d7a4d52ccfb4..0c524f9a94a6c 100644
>>>> --- a/arch/x86/kvm/vmx/tdx.c
>>>> +++ b/arch/x86/kvm/vmx/tdx.c
>>>> @@ -172,9 +172,15 @@ static void td_init_cpuid_entry2(struct
>>>> kvm_cpuid_entry2 *entry, unsigned char i
>>>> entry->ecx = (u32)td_conf->cpuid_config_values[idx][1];
>>>> entry->edx = td_conf->cpuid_config_values[idx][1] >> 32;
>>>>
>>>> - if (entry->index == KVM_TDX_CPUID_NO_SUBLEAF)
>>>> + if (entry->index == KVM_TDX_CPUID_NO_SUBLEAF) {
>>>> entry->index = 0;
>>>> + entry->flags &= ~KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
>>>
>>> There are two callers of this. One is already zeroed, and the other has
>>> stack garbage in flags. But that second caller doesn't look at the
>>> flags so it is harmless. Maybe it would be simpler and clearer to just
>>> zero init the entry struct in that caller. Then you don't need to clear
>>> it here. Or alternatively set flags to zero above, and then add
>>> KVM_CPUID_FLAG_SIGNIFCANT_INDEX if needed. Rather than manipulating a
>>> single bit in a field of garbage, which seems weird.
>
> +1, td_init_cpuid_entry2() should initialize flags to '0' and then set
> KVM_CPUID_FLAG_SIGNIFCANT_INDEX as appropriate.
>
>>>> + } else {
>>>> + entry->flags |= KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
>>>> + }
>>>>
>>>> + WARN_ON_ONCE(cpuid_function_is_indexed(entry->function) !=
>>>> + !!(entry->flags &
>>>> KVM_CPUID_FLAG_SIGNIFCANT_INDEX));
>>>
>>> It warns on leaf 0x23 for me. Is it intentional?
>>
>> I guess because the list in cpuid_function_is_indexed() is hard-coded
>> and 0x23 is not added into the list yet.
>
> Yeah, I was anticipating that we'd run afoul of leaves that aren't known to
> the kernel. FWIW, it looks like 0x24 is also indexed.
>
>> It's fine for existing KVM code because cpuid_function_is_indexed() is
>> only used to check that if a CPUID entry is queried without index, it
>> shouldn't be included in the indexed list.
>>
>> But adding the consistency check here would cause compatibility issue.
>> Generally, if a new CPUID indexed function is added for some new CPU and
>> the TDX module reports it, KVM versions without the CPUID function in
>> the list will trigger the warning.
>
> IMO, that's a good thing and working as intended. WARNs aren't inherently evil.
> While the goal is to be WARN-free, in this case triggering the WARN if the TDX
> Module is updated (or new silicon arrives) is desirable, because it alerts us to
> that new behavior, so that we can go update KVM.
So it effectively leverages the TDX module's interface to retrieve the hardware
information to validate the hard-coded list.
Do we need to consider the panic_on_warn case? I guess the option will not be
enabled in a production environment?
>
> But we should "fix" 0x23 and 0x24 before landing this patch.
>
^ permalink raw reply
* [PATCH 14/14] KVM: x86: Add helpers to prepare kvm_run for userspace MMIO exit
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Add helpers to fill kvm_run for userspace MMIO exits to deduplicate a
variety of code, and to allow for a cleaner return path in
emulator_read_write().
No functional change intended.
Cc: Rick Edgecombe <rick.p.edgecombe@intel.com>
Cc: Binbin Wu <binbin.wu@linux.intel.com>
Cc: Xiaoyao Li <xiaoyao.li@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/vmx/tdx.c | 14 ++++----------
arch/x86/kvm/x86.c | 42 ++++++++----------------------------------
arch/x86/kvm/x86.h | 24 ++++++++++++++++++++++++
3 files changed, 36 insertions(+), 44 deletions(-)
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 5df9d32d2058..a813c502336c 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -1467,17 +1467,11 @@ static int tdx_emulate_mmio(struct kvm_vcpu *vcpu)
/* Request the device emulation to userspace device model. */
vcpu->mmio_is_write = write;
- if (!write)
+
+ __kvm_prepare_emulated_mmio_exit(vcpu, gpa, size, &val, write);
+
+ if (!write) {
vcpu->arch.complete_userspace_io = tdx_complete_mmio_read;
-
- vcpu->run->mmio.phys_addr = gpa;
- vcpu->run->mmio.len = size;
- vcpu->run->mmio.is_write = write;
- vcpu->run->exit_reason = KVM_EXIT_MMIO;
-
- if (write) {
- memcpy(vcpu->run->mmio.data, &val, size);
- } else {
vcpu->mmio_fragments[0].gpa = gpa;
vcpu->mmio_fragments[0].len = size;
trace_kvm_mmio(KVM_TRACE_MMIO_READ_UNSATISFIED, size, gpa, NULL);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 5376b370b4db..889a9098403c 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8210,7 +8210,6 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
const struct read_write_emulator_ops *ops)
{
struct kvm_vcpu *vcpu = emul_to_vcpu(ctxt);
- struct kvm_mmio_fragment *frag;
int rc;
if (WARN_ON_ONCE((bytes > 8u || !ops->write) && object_is_on_stack(val)))
@@ -8268,12 +8267,9 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
vcpu->mmio_needed = 1;
vcpu->mmio_cur_fragment = 0;
+ vcpu->mmio_is_write = ops->write;
- frag = &vcpu->mmio_fragments[0];
- vcpu->run->mmio.len = min(8u, frag->len);
- vcpu->run->mmio.is_write = vcpu->mmio_is_write = ops->write;
- vcpu->run->exit_reason = KVM_EXIT_MMIO;
- vcpu->run->mmio.phys_addr = frag->gpa;
+ kvm_prepare_emulated_mmio_exit(vcpu, &vcpu->mmio_fragments[0]);
/*
* For MMIO reads, stop emulating and immediately exit to userspace, as
@@ -8283,11 +8279,7 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
* after completing emulation (see the check on vcpu->mmio_needed in
* x86_emulate_instruction()).
*/
- if (!ops->write)
- return X86EMUL_IO_NEEDED;
-
- memcpy(vcpu->run->mmio.data, frag->data, min(8u, frag->len));
- return X86EMUL_CONTINUE;
+ return ops->write ? X86EMUL_CONTINUE : X86EMUL_IO_NEEDED;
}
static int emulator_read_emulated(struct x86_emulate_ctxt *ctxt,
@@ -11884,12 +11876,7 @@ static int complete_emulated_mmio(struct kvm_vcpu *vcpu)
return complete_emulated_io(vcpu);
}
- run->exit_reason = KVM_EXIT_MMIO;
- run->mmio.phys_addr = frag->gpa;
- if (vcpu->mmio_is_write)
- memcpy(run->mmio.data, frag->data, min(8u, frag->len));
- run->mmio.len = min(8u, frag->len);
- run->mmio.is_write = vcpu->mmio_is_write;
+ kvm_prepare_emulated_mmio_exit(vcpu, frag);
vcpu->arch.complete_userspace_io = complete_emulated_mmio;
return 0;
}
@@ -14296,15 +14283,8 @@ static int complete_sev_es_emulated_mmio(struct kvm_vcpu *vcpu)
}
// More MMIO is needed
- run->mmio.phys_addr = frag->gpa;
- run->mmio.len = min(8u, frag->len);
- run->mmio.is_write = vcpu->mmio_is_write;
- if (run->mmio.is_write)
- memcpy(run->mmio.data, frag->data, min(8u, frag->len));
- run->exit_reason = KVM_EXIT_MMIO;
-
+ kvm_prepare_emulated_mmio_exit(vcpu, frag);
vcpu->arch.complete_userspace_io = complete_sev_es_emulated_mmio;
-
return 0;
}
@@ -14333,23 +14313,17 @@ int kvm_sev_es_mmio(struct kvm_vcpu *vcpu, bool is_write, gpa_t gpa,
* requests that split a page boundary.
*/
frag = vcpu->mmio_fragments;
- vcpu->mmio_nr_fragments = 1;
frag->len = bytes;
frag->gpa = gpa;
frag->data = data;
vcpu->mmio_needed = 1;
vcpu->mmio_cur_fragment = 0;
+ vcpu->mmio_nr_fragments = 1;
+ vcpu->mmio_is_write = is_write;
- vcpu->run->mmio.phys_addr = gpa;
- vcpu->run->mmio.len = min(8u, frag->len);
- vcpu->run->mmio.is_write = is_write;
- if (is_write)
- memcpy(vcpu->run->mmio.data, frag->data, min(8u, frag->len));
- vcpu->run->exit_reason = KVM_EXIT_MMIO;
-
+ kvm_prepare_emulated_mmio_exit(vcpu, frag);
vcpu->arch.complete_userspace_io = complete_sev_es_emulated_mmio;
-
return 0;
}
EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_sev_es_mmio);
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index 1d0f0edd31b3..d66f1c53d2b5 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -718,6 +718,30 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size,
unsigned int port, void *data, unsigned int count,
int in);
+static inline void __kvm_prepare_emulated_mmio_exit(struct kvm_vcpu *vcpu,
+ gpa_t gpa, unsigned int len,
+ const void *data,
+ bool is_write)
+{
+ struct kvm_run *run = vcpu->run;
+
+ run->mmio.len = min(8u, len);
+ run->mmio.is_write = is_write;
+ run->exit_reason = KVM_EXIT_MMIO;
+ run->mmio.phys_addr = gpa;
+ if (is_write)
+ memcpy(run->mmio.data, data, min(8u, len));
+}
+
+static inline void kvm_prepare_emulated_mmio_exit(struct kvm_vcpu *vcpu,
+ struct kvm_mmio_fragment *frag)
+{
+ WARN_ON_ONCE(!vcpu->mmio_needed || !vcpu->mmio_nr_fragments);
+
+ __kvm_prepare_emulated_mmio_exit(vcpu, frag->gpa, frag->len, frag->data,
+ vcpu->mmio_is_write);
+}
+
static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
{
return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 13/14] KVM: x86: Don't panic the kernel if completing userspace I/O / MMIO goes sideways
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Kill the VM instead of the host kernel if KVM botches I/O and/or MMIO
handling. There is zero danger to the host or guest, i.e. panicking the
host isn't remotely justified.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index f3e2ec7e1828..5376b370b4db 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -9710,7 +9710,8 @@ static int complete_fast_pio_in(struct kvm_vcpu *vcpu)
unsigned long val;
/* We should only ever be called with arch.pio.count equal to 1 */
- BUG_ON(vcpu->arch.pio.count != 1);
+ if (KVM_BUG_ON(vcpu->arch.pio.count != 1, vcpu->kvm))
+ return -EIO;
if (unlikely(!kvm_is_linear_rip(vcpu, vcpu->arch.cui_linear_rip))) {
vcpu->arch.pio.count = 0;
@@ -11820,7 +11821,8 @@ static inline int complete_emulated_io(struct kvm_vcpu *vcpu)
static int complete_emulated_pio(struct kvm_vcpu *vcpu)
{
- BUG_ON(!vcpu->arch.pio.count);
+ if (KVM_BUG_ON(!vcpu->arch.pio.count, vcpu->kvm))
+ return -EIO;
return complete_emulated_io(vcpu);
}
@@ -11849,7 +11851,8 @@ static int complete_emulated_mmio(struct kvm_vcpu *vcpu)
struct kvm_mmio_fragment *frag;
unsigned len;
- BUG_ON(!vcpu->mmio_needed);
+ if (KVM_BUG_ON(!vcpu->mmio_needed, vcpu->kvm))
+ return -EIO;
/* Complete previous fragment */
frag = &vcpu->mmio_fragments[vcpu->mmio_cur_fragment];
@@ -14262,7 +14265,8 @@ static int complete_sev_es_emulated_mmio(struct kvm_vcpu *vcpu)
struct kvm_mmio_fragment *frag;
unsigned int len;
- BUG_ON(!vcpu->mmio_needed);
+ if (KVM_BUG_ON(!vcpu->mmio_needed, vcpu->kvm))
+ return -EIO;
/* Complete previous fragment */
frag = &vcpu->mmio_fragments[vcpu->mmio_cur_fragment];
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 12/14] KVM: x86: Rename .read_write_emulate() to .read_write_guest()
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Rename the ops and helpers to read/write guest memory to clarify that they
do exactly that, i.e. aren't generic emulation flows and don't do anything
related to emulated MMIO.
Opportunistically add comments to explain the flow, e.g. it's not exactly
obvious that KVM deliberately treats "failed" accesses to guest memory as
emulated MMIO.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 38 ++++++++++++++++++++++++++++----------
1 file changed, 28 insertions(+), 10 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 5a3ba161db7b..f3e2ec7e1828 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8103,21 +8103,21 @@ static int vcpu_mmio_gva_to_gpa(struct kvm_vcpu *vcpu, unsigned long gva,
}
struct read_write_emulator_ops {
- int (*read_write_emulate)(struct kvm_vcpu *vcpu, gpa_t gpa,
- void *val, int bytes);
+ int (*read_write_guest)(struct kvm_vcpu *vcpu, gpa_t gpa,
+ void *val, int bytes);
int (*read_write_mmio)(struct kvm_vcpu *vcpu, gpa_t gpa,
int bytes, void *val);
bool write;
};
-static int read_emulate(struct kvm_vcpu *vcpu, gpa_t gpa,
- void *val, int bytes)
+static int emulator_read_guest(struct kvm_vcpu *vcpu, gpa_t gpa,
+ void *val, int bytes)
{
return !kvm_vcpu_read_guest(vcpu, gpa, val, bytes);
}
-static int write_emulate(struct kvm_vcpu *vcpu, gpa_t gpa,
- void *val, int bytes)
+static int emulator_write_guest(struct kvm_vcpu *vcpu, gpa_t gpa,
+ void *val, int bytes)
{
int ret;
@@ -8157,11 +8157,22 @@ static int emulator_read_write_onepage(unsigned long addr, void *val,
return X86EMUL_PROPAGATE_FAULT;
}
- if (!ret && ops->read_write_emulate(vcpu, gpa, val, bytes))
+ /*
+ * If the memory is not _known_ to be emulated MMIO, attempt to access
+ * guest memory. If accessing guest memory fails, e.g. because there's
+ * no memslot, then handle the access as MMIO. Note, treating the
+ * access as emulated MMIO is technically wrong if there is a memslot,
+ * i.e. if accessing host user memory failed, but this has been KVM's
+ * historical ABI for decades.
+ */
+ if (!ret && ops->read_write_guest(vcpu, gpa, val, bytes))
return X86EMUL_CONTINUE;
/*
- * Is this MMIO handled locally?
+ * Attempt to handle emulated MMIO within the kernel, e.g. for accesses
+ * to an in-kernel local or I/O APIC, or to an ioeventfd range attached
+ * to MMIO bus. If the access isn't fully resolved, insert an MMIO
+ * fragment with the relevant details.
*/
handled = ops->read_write_mmio(vcpu, gpa, bytes, val);
if (handled == bytes)
@@ -8182,6 +8193,13 @@ static int emulator_read_write_onepage(unsigned long addr, void *val,
frag->data = val;
}
frag->len = bytes;
+
+ /*
+ * Continue emulating, even though KVM needs to (eventually) do an MMIO
+ * exit to userspace. If the access splits multiple pages, then KVM
+ * needs to exit to userspace only after emulating both parts of the
+ * access.
+ */
return X86EMUL_CONTINUE;
}
@@ -8279,7 +8297,7 @@ static int emulator_read_emulated(struct x86_emulate_ctxt *ctxt,
struct x86_exception *exception)
{
static const struct read_write_emulator_ops ops = {
- .read_write_emulate = read_emulate,
+ .read_write_guest = emulator_read_guest,
.read_write_mmio = vcpu_mmio_read,
.write = false,
};
@@ -8294,7 +8312,7 @@ static int emulator_write_emulated(struct x86_emulate_ctxt *ctxt,
struct x86_exception *exception)
{
static const struct read_write_emulator_ops ops = {
- .read_write_emulate = write_emulate,
+ .read_write_guest = emulator_write_guest,
.read_write_mmio = vcpu_mmio_write,
.write = true,
};
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 11/14] KVM: x86: Fold emulator_write_phys() into write_emulate()
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Fold emulator_write_phys() into write_emulate() to drop a superfluous
wrapper, and to provide more symmetry between the read and write paths.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/kvm_host.h | 3 ---
arch/x86/kvm/x86.c | 20 +++++++-------------
2 files changed, 7 insertions(+), 16 deletions(-)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index ff07c45e3c73..aa030fbd669d 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -2097,9 +2097,6 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end);
int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long cr3);
-int emulator_write_phys(struct kvm_vcpu *vcpu, gpa_t gpa,
- const void *val, int bytes);
-
extern bool tdp_enabled;
u64 vcpu_tsc_khz(struct kvm_vcpu *vcpu);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 5fb5259c208f..5a3ba161db7b 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8102,18 +8102,6 @@ static int vcpu_mmio_gva_to_gpa(struct kvm_vcpu *vcpu, unsigned long gva,
return vcpu_is_mmio_gpa(vcpu, gva, *gpa, write);
}
-int emulator_write_phys(struct kvm_vcpu *vcpu, gpa_t gpa,
- const void *val, int bytes)
-{
- int ret;
-
- ret = kvm_vcpu_write_guest(vcpu, gpa, val, bytes);
- if (ret < 0)
- return 0;
- kvm_page_track_write(vcpu, gpa, val, bytes);
- return 1;
-}
-
struct read_write_emulator_ops {
int (*read_write_emulate)(struct kvm_vcpu *vcpu, gpa_t gpa,
void *val, int bytes);
@@ -8131,7 +8119,13 @@ static int read_emulate(struct kvm_vcpu *vcpu, gpa_t gpa,
static int write_emulate(struct kvm_vcpu *vcpu, gpa_t gpa,
void *val, int bytes)
{
- return emulator_write_phys(vcpu, gpa, val, bytes);
+ int ret;
+
+ ret = kvm_vcpu_write_guest(vcpu, gpa, val, bytes);
+ if (ret < 0)
+ return 0;
+ kvm_page_track_write(vcpu, gpa, val, bytes);
+ return 1;
}
static int emulator_read_write_onepage(unsigned long addr, void *val,
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 10/14] KVM: x86: Bury emulator read/write ops in emulator_{read,write}_emulated()
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Now that SEV-ES invokes vcpu_mmio_{read,write}() directly, bury the
read vs. write emulator ops in the dedicated emulator callbacks so that
they are colocated with their usage, and to make it harder for
non-emulator code to use hooks that are intended for the emulator.
Opportunistically rename the structures to get rid of the long-standing
"emultor" typo.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 29 ++++++++++++++---------------
1 file changed, 14 insertions(+), 15 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 022e953906f7..5fb5259c208f 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8134,17 +8134,6 @@ static int write_emulate(struct kvm_vcpu *vcpu, gpa_t gpa,
return emulator_write_phys(vcpu, gpa, val, bytes);
}
-static const struct read_write_emulator_ops read_emultor = {
- .read_write_emulate = read_emulate,
- .read_write_mmio = vcpu_mmio_read,
-};
-
-static const struct read_write_emulator_ops write_emultor = {
- .read_write_emulate = write_emulate,
- .read_write_mmio = vcpu_mmio_write,
- .write = true,
-};
-
static int emulator_read_write_onepage(unsigned long addr, void *val,
unsigned int bytes,
struct x86_exception *exception,
@@ -8295,8 +8284,13 @@ static int emulator_read_emulated(struct x86_emulate_ctxt *ctxt,
unsigned int bytes,
struct x86_exception *exception)
{
- return emulator_read_write(ctxt, addr, val, bytes,
- exception, &read_emultor);
+ static const struct read_write_emulator_ops ops = {
+ .read_write_emulate = read_emulate,
+ .read_write_mmio = vcpu_mmio_read,
+ .write = false,
+ };
+
+ return emulator_read_write(ctxt, addr, val, bytes, exception, &ops);
}
static int emulator_write_emulated(struct x86_emulate_ctxt *ctxt,
@@ -8305,8 +8299,13 @@ static int emulator_write_emulated(struct x86_emulate_ctxt *ctxt,
unsigned int bytes,
struct x86_exception *exception)
{
- return emulator_read_write(ctxt, addr, (void *)val, bytes,
- exception, &write_emultor);
+ static const struct read_write_emulator_ops ops = {
+ .read_write_emulate = write_emulate,
+ .read_write_mmio = vcpu_mmio_write,
+ .write = true,
+ };
+
+ return emulator_read_write(ctxt, addr, (void *)val, bytes, exception, &ops);
}
#define emulator_try_cmpxchg_user(t, ptr, old, new) \
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 09/14] KVM: x86: Consolidate SEV-ES MMIO emulation into a single public API
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Dedup kvm_sev_es_mmio_{read,write}() into a single API, as the "cost" of
plumbing in a boolean is largely negligible since KVM pulls out a boolean
for ops->write anyways, and consolidating the APIs will allow for
additional cleanups.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/svm/sev.c | 20 ++++++--------------
arch/x86/kvm/x86.c | 29 +++++++++--------------------
arch/x86/kvm/x86.h | 6 ++----
3 files changed, 17 insertions(+), 38 deletions(-)
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index ea515cf41168..723f4452302a 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -4434,25 +4434,17 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
switch (control->exit_code) {
case SVM_VMGEXIT_MMIO_READ:
- ret = setup_vmgexit_scratch(svm, true, control->exit_info_2);
- if (ret)
- break;
+ case SVM_VMGEXIT_MMIO_WRITE: {
+ bool is_write = control->exit_code == SVM_VMGEXIT_MMIO_WRITE;
- ret = kvm_sev_es_mmio_read(vcpu,
- control->exit_info_1,
- control->exit_info_2,
- svm->sev_es.ghcb_sa);
- break;
- case SVM_VMGEXIT_MMIO_WRITE:
- ret = setup_vmgexit_scratch(svm, false, control->exit_info_2);
+ ret = setup_vmgexit_scratch(svm, !is_write, control->exit_info_2);
if (ret)
break;
- ret = kvm_sev_es_mmio_write(vcpu,
- control->exit_info_1,
- control->exit_info_2,
- svm->sev_es.ghcb_sa);
+ ret = kvm_sev_es_mmio(vcpu, is_write, control->exit_info_1,
+ control->exit_info_2, svm->sev_es.ghcb_sa);
break;
+ }
case SVM_VMGEXIT_NMI_COMPLETE:
++vcpu->stat.nmi_window_exits;
svm->nmi_masked = false;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index f93f0f8961af..022e953906f7 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -14293,9 +14293,8 @@ static int complete_sev_es_emulated_mmio(struct kvm_vcpu *vcpu)
return 0;
}
-static int kvm_sev_es_do_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
- unsigned int bytes, void *data,
- const struct read_write_emulator_ops *ops)
+int kvm_sev_es_mmio(struct kvm_vcpu *vcpu, bool is_write, gpa_t gpa,
+ unsigned int bytes, void *data)
{
struct kvm_mmio_fragment *frag;
int handled;
@@ -14303,7 +14302,10 @@ static int kvm_sev_es_do_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
if (!data || WARN_ON_ONCE(object_is_on_stack(data)))
return -EINVAL;
- handled = ops->read_write_mmio(vcpu, gpa, bytes, data);
+ if (is_write)
+ handled = vcpu_mmio_write(vcpu, gpa, bytes, data);
+ else
+ handled = vcpu_mmio_read(vcpu, gpa, bytes, data);
if (handled == bytes)
return 1;
@@ -14326,8 +14328,8 @@ static int kvm_sev_es_do_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
vcpu->run->mmio.phys_addr = gpa;
vcpu->run->mmio.len = min(8u, frag->len);
- vcpu->run->mmio.is_write = ops->write;
- if (ops->write)
+ vcpu->run->mmio.is_write = is_write;
+ if (is_write)
memcpy(vcpu->run->mmio.data, frag->data, min(8u, frag->len));
vcpu->run->exit_reason = KVM_EXIT_MMIO;
@@ -14335,20 +14337,7 @@ static int kvm_sev_es_do_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
return 0;
}
-
-int kvm_sev_es_mmio_write(struct kvm_vcpu *vcpu, gpa_t gpa, unsigned int bytes,
- void *data)
-{
- return kvm_sev_es_do_mmio(vcpu, gpa, bytes, data, &write_emultor);
-}
-EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_sev_es_mmio_write);
-
-int kvm_sev_es_mmio_read(struct kvm_vcpu *vcpu, gpa_t gpa, unsigned int bytes,
- void *data)
-{
- return kvm_sev_es_do_mmio(vcpu, gpa, bytes, data, &read_emultor);
-}
-EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_sev_es_mmio_read);
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_sev_es_mmio);
static void advance_sev_es_emulated_pio(struct kvm_vcpu *vcpu, unsigned count, int size)
{
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index 94d4f07aaaa0..1d0f0edd31b3 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -712,10 +712,8 @@ static inline bool __kvm_is_valid_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)
__reserved_bits; \
})
-int kvm_sev_es_mmio_write(struct kvm_vcpu *vcpu, gpa_t src, unsigned int bytes,
- void *dst);
-int kvm_sev_es_mmio_read(struct kvm_vcpu *vcpu, gpa_t src, unsigned int bytes,
- void *dst);
+int kvm_sev_es_mmio(struct kvm_vcpu *vcpu, bool is_write, gpa_t gpa,
+ unsigned int bytes, void *data);
int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size,
unsigned int port, void *data, unsigned int count,
int in);
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 08/14] KVM: x86: Dedup kvm_sev_es_mmio_{read,write}()
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Dedup the SEV-ES emulated MMIO code by using the read vs. write emulator
ops to handle the few differences between reads and writes.
Opportunistically tweak the comment about fragments to call out that KVM
should verify that userspace can actually handle MMIO requests that cross
page boundaries. Unlike emulated MMIO, the request is made in the GPA
space, not the GVA space, i.e. emulation across page boundaries can work
generically, at least in theory.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 110 +++++++++++++++++++--------------------------
1 file changed, 45 insertions(+), 65 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 2db0bf738d2d..f93f0f8961af 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -14293,80 +14293,60 @@ static int complete_sev_es_emulated_mmio(struct kvm_vcpu *vcpu)
return 0;
}
+static int kvm_sev_es_do_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
+ unsigned int bytes, void *data,
+ const struct read_write_emulator_ops *ops)
+{
+ struct kvm_mmio_fragment *frag;
+ int handled;
+
+ if (!data || WARN_ON_ONCE(object_is_on_stack(data)))
+ return -EINVAL;
+
+ handled = ops->read_write_mmio(vcpu, gpa, bytes, data);
+ if (handled == bytes)
+ return 1;
+
+ bytes -= handled;
+ gpa += handled;
+ data += handled;
+
+ /*
+ * TODO: Determine whether or not userspace plays nice with MMIO
+ * requests that split a page boundary.
+ */
+ frag = vcpu->mmio_fragments;
+ vcpu->mmio_nr_fragments = 1;
+ frag->len = bytes;
+ frag->gpa = gpa;
+ frag->data = data;
+
+ vcpu->mmio_needed = 1;
+ vcpu->mmio_cur_fragment = 0;
+
+ vcpu->run->mmio.phys_addr = gpa;
+ vcpu->run->mmio.len = min(8u, frag->len);
+ vcpu->run->mmio.is_write = ops->write;
+ if (ops->write)
+ memcpy(vcpu->run->mmio.data, frag->data, min(8u, frag->len));
+ vcpu->run->exit_reason = KVM_EXIT_MMIO;
+
+ vcpu->arch.complete_userspace_io = complete_sev_es_emulated_mmio;
+
+ return 0;
+}
+
int kvm_sev_es_mmio_write(struct kvm_vcpu *vcpu, gpa_t gpa, unsigned int bytes,
void *data)
{
- int handled;
- struct kvm_mmio_fragment *frag;
-
- if (!data || WARN_ON_ONCE(object_is_on_stack(data)))
- return -EINVAL;
-
- handled = write_emultor.read_write_mmio(vcpu, gpa, bytes, data);
- if (handled == bytes)
- return 1;
-
- bytes -= handled;
- gpa += handled;
- data += handled;
-
- /*TODO: Check if need to increment number of frags */
- frag = vcpu->mmio_fragments;
- vcpu->mmio_nr_fragments = 1;
- frag->len = bytes;
- frag->gpa = gpa;
- frag->data = data;
-
- vcpu->mmio_needed = 1;
- vcpu->mmio_cur_fragment = 0;
-
- vcpu->run->mmio.phys_addr = gpa;
- vcpu->run->mmio.len = min(8u, frag->len);
- vcpu->run->mmio.is_write = 1;
- memcpy(vcpu->run->mmio.data, frag->data, min(8u, frag->len));
- vcpu->run->exit_reason = KVM_EXIT_MMIO;
-
- vcpu->arch.complete_userspace_io = complete_sev_es_emulated_mmio;
-
- return 0;
+ return kvm_sev_es_do_mmio(vcpu, gpa, bytes, data, &write_emultor);
}
EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_sev_es_mmio_write);
int kvm_sev_es_mmio_read(struct kvm_vcpu *vcpu, gpa_t gpa, unsigned int bytes,
void *data)
{
- int handled;
- struct kvm_mmio_fragment *frag;
-
- if (!data || WARN_ON_ONCE(object_is_on_stack(data)))
- return -EINVAL;
-
- handled = read_emultor.read_write_mmio(vcpu, gpa, bytes, data);
- if (handled == bytes)
- return 1;
-
- bytes -= handled;
- gpa += handled;
- data += handled;
-
- /*TODO: Check if need to increment number of frags */
- frag = vcpu->mmio_fragments;
- vcpu->mmio_nr_fragments = 1;
- frag->len = bytes;
- frag->gpa = gpa;
- frag->data = data;
-
- vcpu->mmio_needed = 1;
- vcpu->mmio_cur_fragment = 0;
-
- vcpu->run->mmio.phys_addr = gpa;
- vcpu->run->mmio.len = min(8u, frag->len);
- vcpu->run->mmio.is_write = 0;
- vcpu->run->exit_reason = KVM_EXIT_MMIO;
-
- vcpu->arch.complete_userspace_io = complete_sev_es_emulated_mmio;
-
- return 0;
+ return kvm_sev_es_do_mmio(vcpu, gpa, bytes, data, &read_emultor);
}
EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_sev_es_mmio_read);
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 07/14] KVM: x86: Harden SEV-ES MMIO against on-stack use-after-free
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Add a sanity check to ensure KVM doesn't use an on-stack variable when
handling an MMIO request for an SEV-ES guest. The source/destination
for SEV-ES MMIO should _always_ be the #VMGEXIT scratch area.
Opportunistically update the comment in the completion side of things
to clarify that frag->data doesn't need to be copied anywhere, and the
VMEGEXIT is trap-like (the current comment doesn't clarify *how* RIP is
advanced).
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 7abd6f93c386..2db0bf738d2d 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -14273,8 +14273,10 @@ static int complete_sev_es_emulated_mmio(struct kvm_vcpu *vcpu)
if (vcpu->mmio_cur_fragment >= vcpu->mmio_nr_fragments) {
vcpu->mmio_needed = 0;
- // VMG change, at this point, we're always done
- // RIP has already been advanced
+ /*
+ * All done, as frag->data always points at the GHCB scratch
+ * area and VMGEXIT is trap-like (RIP is advanced by hardware).
+ */
return 1;
}
@@ -14297,7 +14299,7 @@ int kvm_sev_es_mmio_write(struct kvm_vcpu *vcpu, gpa_t gpa, unsigned int bytes,
int handled;
struct kvm_mmio_fragment *frag;
- if (!data)
+ if (!data || WARN_ON_ONCE(object_is_on_stack(data)))
return -EINVAL;
handled = write_emultor.read_write_mmio(vcpu, gpa, bytes, data);
@@ -14336,7 +14338,7 @@ int kvm_sev_es_mmio_read(struct kvm_vcpu *vcpu, gpa_t gpa, unsigned int bytes,
int handled;
struct kvm_mmio_fragment *frag;
- if (!data)
+ if (!data || WARN_ON_ONCE(object_is_on_stack(data)))
return -EINVAL;
handled = read_emultor.read_write_mmio(vcpu, gpa, bytes, data);
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 06/14] KVM: x86: Move MMIO write tracing into vcpu_mmio_write()
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Move the invocation of MMIO write tracepoint into vcpu_mmio_write() and
drop its largely-useless wrapper to cull pointless code and to make the
code symmetrical with respect to vcpu_mmio_read().
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 13 +++++--------
1 file changed, 5 insertions(+), 8 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 5fde5bb010e7..7abd6f93c386 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -7769,11 +7769,14 @@ static void kvm_init_msr_lists(void)
}
static int vcpu_mmio_write(struct kvm_vcpu *vcpu, gpa_t addr, int len,
- const void *v)
+ void *__v)
{
+ const void *v = __v;
int handled = 0;
int n;
+ trace_kvm_mmio(KVM_TRACE_MMIO_WRITE, len, addr, __v);
+
do {
n = min(len, 8);
if (!(lapic_in_kernel(vcpu) &&
@@ -8131,12 +8134,6 @@ static int write_emulate(struct kvm_vcpu *vcpu, gpa_t gpa,
return emulator_write_phys(vcpu, gpa, val, bytes);
}
-static int write_mmio(struct kvm_vcpu *vcpu, gpa_t gpa, int bytes, void *val)
-{
- trace_kvm_mmio(KVM_TRACE_MMIO_WRITE, bytes, gpa, val);
- return vcpu_mmio_write(vcpu, gpa, bytes, val);
-}
-
static const struct read_write_emulator_ops read_emultor = {
.read_write_emulate = read_emulate,
.read_write_mmio = vcpu_mmio_read,
@@ -8144,7 +8141,7 @@ static const struct read_write_emulator_ops read_emultor = {
static const struct read_write_emulator_ops write_emultor = {
.read_write_emulate = write_emulate,
- .read_write_mmio = write_mmio,
+ .read_write_mmio = vcpu_mmio_write,
.write = true,
};
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 05/14] KVM: x86: Open code read vs. write userspace MMIO exits in emulator_read_write()
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Open code the differences in read vs. write userspace MMIO exits instead
of burying three lines of code behind indirect callbacks, as splitting the
logic makes it extremely hard to track that KVM's handling of reads vs.
write is _significantly_ different. Add a comment to explain why the
semantics are different, and how on earth an MMIO write ends up triggering
an exit to userspace.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 33 +++++++++++++--------------------
1 file changed, 13 insertions(+), 20 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 7cbd6f7d8578..5fde5bb010e7 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8116,8 +8116,6 @@ struct read_write_emulator_ops {
void *val, int bytes);
int (*read_write_mmio)(struct kvm_vcpu *vcpu, gpa_t gpa,
int bytes, void *val);
- int (*read_write_exit_mmio)(struct kvm_vcpu *vcpu, gpa_t gpa,
- void *val, int bytes);
bool write;
};
@@ -8139,31 +8137,14 @@ static int write_mmio(struct kvm_vcpu *vcpu, gpa_t gpa, int bytes, void *val)
return vcpu_mmio_write(vcpu, gpa, bytes, val);
}
-static int read_exit_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
- void *val, int bytes)
-{
- return X86EMUL_IO_NEEDED;
-}
-
-static int write_exit_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
- void *val, int bytes)
-{
- struct kvm_mmio_fragment *frag = &vcpu->mmio_fragments[0];
-
- memcpy(vcpu->run->mmio.data, frag->data, min(8u, frag->len));
- return X86EMUL_CONTINUE;
-}
-
static const struct read_write_emulator_ops read_emultor = {
.read_write_emulate = read_emulate,
.read_write_mmio = vcpu_mmio_read,
- .read_write_exit_mmio = read_exit_mmio,
};
static const struct read_write_emulator_ops write_emultor = {
.read_write_emulate = write_emulate,
.read_write_mmio = write_mmio,
- .read_write_exit_mmio = write_exit_mmio,
.write = true,
};
@@ -8296,7 +8277,19 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
vcpu->run->exit_reason = KVM_EXIT_MMIO;
vcpu->run->mmio.phys_addr = frag->gpa;
- return ops->read_write_exit_mmio(vcpu, frag->gpa, val, bytes);
+ /*
+ * For MMIO reads, stop emulating and immediately exit to userspace, as
+ * KVM needs the value to correctly emulate the instruction. For MMIO
+ * writes, continue emulating as the write to MMIO is a side effect for
+ * all intents and purposes. KVM will still exit to userspace, but
+ * after completing emulation (see the check on vcpu->mmio_needed in
+ * x86_emulate_instruction()).
+ */
+ if (!ops->write)
+ return X86EMUL_IO_NEEDED;
+
+ memcpy(vcpu->run->mmio.data, frag->data, min(8u, frag->len));
+ return X86EMUL_CONTINUE;
}
static int emulator_read_emulated(struct x86_emulate_ctxt *ctxt,
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 04/14] KVM: x86: Use local MMIO fragment variable to clean up emulator_read_write()
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Grab the MMIO fragment used by emulator_read_write() to initiate an exit
to userspace in a local variable to make the code easier to read.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index a74ae3a81076..7cbd6f7d8578 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8231,7 +8231,7 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
const struct read_write_emulator_ops *ops)
{
struct kvm_vcpu *vcpu = emul_to_vcpu(ctxt);
- gpa_t gpa;
+ struct kvm_mmio_fragment *frag;
int rc;
if (WARN_ON_ONCE((bytes > 8u || !ops->write) && object_is_on_stack(val)))
@@ -8287,17 +8287,16 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
if (!vcpu->mmio_nr_fragments)
return X86EMUL_CONTINUE;
- gpa = vcpu->mmio_fragments[0].gpa;
-
vcpu->mmio_needed = 1;
vcpu->mmio_cur_fragment = 0;
- vcpu->run->mmio.len = min(8u, vcpu->mmio_fragments[0].len);
+ frag = &vcpu->mmio_fragments[0];
+ vcpu->run->mmio.len = min(8u, frag->len);
vcpu->run->mmio.is_write = vcpu->mmio_is_write = ops->write;
vcpu->run->exit_reason = KVM_EXIT_MMIO;
- vcpu->run->mmio.phys_addr = gpa;
+ vcpu->run->mmio.phys_addr = frag->gpa;
- return ops->read_write_exit_mmio(vcpu, gpa, val, bytes);
+ return ops->read_write_exit_mmio(vcpu, frag->gpa, val, bytes);
}
static int emulator_read_emulated(struct x86_emulate_ctxt *ctxt,
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox