* [PATCH kernel 3/9] coco/sev-guest: Allow multiple source files in the driver
From: Alexey Kardashevskiy @ 2026-02-25 5:37 UTC (permalink / raw)
To: x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
In-Reply-To: <20260225053806.3311234-1-aik@amd.com>
Prepare for SEV-TIO support as it is going to equal or bigger
than the existing sev_guest.c which is already 700 lines and
keeps growing.
No behavioural change expected.
Signed-off-by: Alexey Kardashevskiy <aik@amd.com>
---
drivers/virt/coco/sev-guest/Makefile | 3 ++-
drivers/virt/coco/sev-guest/sev-guest.h | 16 ++++++++++++++++
drivers/virt/coco/sev-guest/sev-guest.c | 10 ++--------
3 files changed, 20 insertions(+), 9 deletions(-)
diff --git a/drivers/virt/coco/sev-guest/Makefile b/drivers/virt/coco/sev-guest/Makefile
index 63d67c27723a..9604792e0095 100644
--- a/drivers/virt/coco/sev-guest/Makefile
+++ b/drivers/virt/coco/sev-guest/Makefile
@@ -1,2 +1,3 @@
# SPDX-License-Identifier: GPL-2.0-only
-obj-$(CONFIG_SEV_GUEST) += sev-guest.o
+obj-$(CONFIG_SEV_GUEST) += sev_guest.o
+sev_guest-y += sev-guest.o
diff --git a/drivers/virt/coco/sev-guest/sev-guest.h b/drivers/virt/coco/sev-guest/sev-guest.h
new file mode 100644
index 000000000000..b2a97778e635
--- /dev/null
+++ b/drivers/virt/coco/sev-guest/sev-guest.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef __SEV_GUEST_H__
+#define __SEV_GUEST_H__
+
+#include <linux/miscdevice.h>
+#include <asm/sev.h>
+
+struct snp_guest_dev {
+ struct device *dev;
+ struct miscdevice misc;
+
+ struct snp_msg_desc *msg_desc;
+};
+
+#endif /* __SEV_GUEST_H__ */
diff --git a/drivers/virt/coco/sev-guest/sev-guest.c b/drivers/virt/coco/sev-guest/sev-guest.c
index b01ec99106cd..e1ceeab54a21 100644
--- a/drivers/virt/coco/sev-guest/sev-guest.c
+++ b/drivers/virt/coco/sev-guest/sev-guest.c
@@ -28,19 +28,13 @@
#include <uapi/linux/psp-sev.h>
#include <asm/svm.h>
-#include <asm/sev.h>
+
+#include "sev-guest.h"
#define DEVICE_NAME "sev-guest"
#define SVSM_MAX_RETRIES 3
-struct snp_guest_dev {
- struct device *dev;
- struct miscdevice misc;
-
- struct snp_msg_desc *msg_desc;
-};
-
/*
* The VMPCK ID represents the key used by the SNP guest to communicate with the
* SEV firmware in the AMD Secure Processor (ASP, aka PSP). By default, the key
--
2.52.0
^ permalink raw reply related
* [PATCH kernel 2/9] pci/tsm: Add tsm_tdi_status
From: Alexey Kardashevskiy @ 2026-02-25 5:37 UTC (permalink / raw)
To: x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
In-Reply-To: <20260225053806.3311234-1-aik@amd.com>
Define a structure with all info about a TDI such as TDISP status,
bind state, used START_INTERFACE options and the report digest.
This will be extended and shared to the userspace.
Signed-off-by: Alexey Kardashevskiy <aik@amd.com>
---
Make it uapi? We might want a sysfs node per a field so probably not.
For now its only user is AMD SEV TIO with a plan to expose this struct
as a whole via sysfs.
---
include/linux/pci-tsm.h | 26 ++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/include/linux/pci-tsm.h b/include/linux/pci-tsm.h
index 7987ede76914..8086f358c51b 100644
--- a/include/linux/pci-tsm.h
+++ b/include/linux/pci-tsm.h
@@ -355,4 +355,30 @@ struct tdi_report_footer {
#define TDI_REPORT_FTR(rep) ((struct tdi_report_footer *) &TDI_REPORT_MR((rep), \
TDI_REPORT_MR_NUM(rep)))
+enum tsm_tdisp_state {
+ TDISP_STATE_CONFIG_UNLOCKED = 0,
+ TDISP_STATE_CONFIG_LOCKED = 1,
+ TDISP_STATE_RUN = 2,
+ TDISP_STATE_ERROR = 3,
+};
+
+enum tsm_tdisp_status {
+ TDISP_STATE_BOUND = 0,
+ TDISP_STATE_INVALID = 1,
+ TDISP_STATE_UNBOUND = 2,
+};
+
+struct tsm_tdi_status {
+ __u8 status; /* enum tsm_tdisp_status */
+ __u8 state; /* enum tsm_tdisp_state */
+ __u8 all_request_redirect;
+ __u8 bind_p2p;
+ __u8 lock_msix;
+ __u8 no_fw_update;
+ __u16 cache_line_size;
+ __u8 interface_report_digest[48];
+ __u64 intf_report_counter;
+ struct tdisp_interface_id id;
+} __packed;
+
#endif /*__PCI_TSM_H */
--
2.52.0
^ permalink raw reply related
* [PATCH kernel 1/9] pci/tsm: Add TDISP report blob and helpers to parse it
From: Alexey Kardashevskiy @ 2026-02-25 5:37 UTC (permalink / raw)
To: x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
In-Reply-To: <20260225053806.3311234-1-aik@amd.com>
The TDI interface report is defined in PCIe r7.0,
chapter "11.3.11 DEVICE_INTERFACE_REPORT". The report enumerates
MMIO resources and their properties which will take effect upon
transitioning to the RUN state.
Store the report in pci_tsm.
Define macros and helpers to parse the binary blob.
Signed-off-by: Alexey Kardashevskiy <aik@amd.com>
---
Probably pci_tsm::report could be struct tdi_report_header*?
---
include/linux/pci-tsm.h | 84 ++++++++++++++++++++
drivers/virt/coco/tsm-core.c | 19 +++++
2 files changed, 103 insertions(+)
diff --git a/include/linux/pci-tsm.h b/include/linux/pci-tsm.h
index b984711fa91f..7987ede76914 100644
--- a/include/linux/pci-tsm.h
+++ b/include/linux/pci-tsm.h
@@ -10,6 +10,18 @@ struct tsm_dev;
struct kvm;
enum pci_tsm_req_scope;
+/* Data object for measurements/certificates/attestationreport */
+struct tsm_blob {
+ void *data;
+ size_t len;
+};
+
+struct tsm_blob *tsm_blob_new(void *data, size_t len);
+static inline void tsm_blob_free(struct tsm_blob *b)
+{
+ kfree(b);
+}
+
/*
* struct pci_tsm_ops - manage confidential links and security state
* @link_ops: Coordinate PCIe SPDM and IDE establishment via a platform TSM.
@@ -123,6 +135,7 @@ struct pci_tsm {
struct pci_dev *dsm_dev;
struct tsm_dev *tsm_dev;
struct pci_tdi *tdi;
+ struct tsm_blob *report;
};
/**
@@ -271,4 +284,75 @@ static inline ssize_t pci_tsm_guest_req(struct pci_dev *pdev,
return -ENXIO;
}
#endif
+
+/*
+ * struct tdisp_interface_id - TDISP INTERFACE_ID Definition
+ *
+ * @function_id: Identifies the function of the device hosting the TDI
+ * 15:0: @rid: Requester ID
+ * 23:16: @rseg: Requester Segment (Reserved if Requester Segment Valid is Clear)
+ * 24: @rseg_valid: Requester Segment Valid
+ * 31:25 – Reserved
+ * 8B - Reserved
+ */
+#define TSM_TDISP_IID_REQUESTER_ID GENMASK(15, 0)
+#define TSM_TDISP_IID_RSEG GENMASK(23, 16)
+#define TSM_TDISP_IID_RSEG_VALID BIT(24)
+
+struct tdisp_interface_id {
+ __u32 function_id; /* TSM_TDISP_IID_xxxx */
+ __u8 reserved[8];
+} __packed;
+
+#define SPDM_MEASUREMENTS_NONCE_LEN 32
+typedef __u8 spdm_measurements_nonce_t[SPDM_MEASUREMENTS_NONCE_LEN];
+
+/*
+ * TDI Report Structure as defined in TDISP.
+ */
+#define _BITSH(x) (1 << (x))
+#define TSM_TDI_REPORT_NO_FW_UPDATE _BITSH(0) /* not updates in CONFIG_LOCKED or RUN */
+#define TSM_TDI_REPORT_DMA_NO_PASID _BITSH(1) /* TDI generates DMA requests without PASID */
+#define TSM_TDI_REPORT_DMA_PASID _BITSH(2) /* TDI generates DMA requests with PASID */
+#define TSM_TDI_REPORT_ATS _BITSH(3) /* ATS supported and enabled for the TDI */
+#define TSM_TDI_REPORT_PRS _BITSH(4) /* PRS supported and enabled for the TDI */
+
+struct tdi_report_header {
+ __u16 interface_info; /* TSM_TDI_REPORT_xxx */
+ __u16 reserved2;
+ __u16 msi_x_message_control;
+ __u16 lnr_control;
+ __u32 tph_control;
+ __u32 mmio_range_count;
+} __packed;
+
+/*
+ * Each MMIO Range of the TDI is reported with the MMIO reporting offset added.
+ * Base and size in units of 4K pages
+ */
+#define TSM_TDI_REPORT_MMIO_MSIX_TABLE BIT(0)
+#define TSM_TDI_REPORT_MMIO_PBA BIT(1)
+#define TSM_TDI_REPORT_MMIO_IS_NON_TEE BIT(2)
+#define TSM_TDI_REPORT_MMIO_IS_UPDATABLE BIT(3)
+#define TSM_TDI_REPORT_MMIO_RESERVED GENMASK(15, 4)
+#define TSM_TDI_REPORT_MMIO_RANGE_ID GENMASK(31, 16)
+
+struct tdi_report_mmio_range {
+ __u64 first_page; /* First 4K page with offset added */
+ __u32 num; /* Number of 4K pages in this range */
+ __u32 range_attributes; /* TSM_TDI_REPORT_MMIO_xxx */
+} __packed;
+
+struct tdi_report_footer {
+ __u32 device_specific_info_len;
+ __u8 device_specific_info[];
+} __packed;
+
+#define TDI_REPORT_HDR(rep) ((struct tdi_report_header *) ((rep)->data))
+#define TDI_REPORT_MR_NUM(rep) (TDI_REPORT_HDR(rep)->mmio_range_count)
+#define TDI_REPORT_MR_OFF(rep) ((struct tdi_report_mmio_range *) (TDI_REPORT_HDR(rep) + 1))
+#define TDI_REPORT_MR(rep, rangeid) TDI_REPORT_MR_OFF(rep)[rangeid]
+#define TDI_REPORT_FTR(rep) ((struct tdi_report_footer *) &TDI_REPORT_MR((rep), \
+ TDI_REPORT_MR_NUM(rep)))
+
#endif /*__PCI_TSM_H */
diff --git a/drivers/virt/coco/tsm-core.c b/drivers/virt/coco/tsm-core.c
index e65ab3461d14..3929176b8d3b 100644
--- a/drivers/virt/coco/tsm-core.c
+++ b/drivers/virt/coco/tsm-core.c
@@ -16,6 +16,25 @@ static struct class *tsm_class;
static DECLARE_RWSEM(tsm_rwsem);
static DEFINE_IDA(tsm_ida);
+struct tsm_blob *tsm_blob_new(void *data, size_t len)
+{
+ struct tsm_blob *b;
+
+ if (!len || !data)
+ return NULL;
+
+ b = kzalloc(sizeof(*b) + len, GFP_KERNEL);
+ if (!b)
+ return NULL;
+
+ b->data = (void *)b + sizeof(*b);
+ b->len = len;
+ memcpy(b->data, data, len);
+
+ return b;
+}
+EXPORT_SYMBOL_GPL(tsm_blob_new);
+
static int match_id(struct device *dev, const void *data)
{
struct tsm_dev *tsm_dev = container_of(dev, struct tsm_dev, dev);
--
2.52.0
^ permalink raw reply related
* [PATCH kernel 0/9] PCI/TSM: coco/sev-guest: Implement SEV-TIO PCIe TDISP (phase2)
From: Alexey Kardashevskiy @ 2026-02-25 5:37 UTC (permalink / raw)
To: x86
Cc: linux-kernel, kvm, linux-pci, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Sean Christopherson,
Paolo Bonzini, Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas,
Dan Williams, Marek Szyprowski, Robin Murphy, Andrew Morton,
Catalin Marinas, Michael Ellerman, Mike Rapoport, Tom Lendacky,
Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra, Stefano Garzarella,
Melody Wang, Seongman Lee, Joerg Roedel, Nikunj A Dadhania,
Michael Roth, Suravee Suthikulpanit, Andi Kleen,
Kuppuswamy Sathyanarayanan, Tony Luck, David Woodhouse,
Greg Kroah-Hartman, Denis Efremov, Geliang Tang, Piotr Gregor,
Michael S. Tsirkin, Alex Williamson, Arnd Bergmann, Jesse Barnes,
Jacob Pan, Yinghai Lu, Kevin Brodsky, Jonathan Cameron,
Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu, Kim Phillips,
Konrad Rzeszutek Wilk, Stefano Stabellini, Claire Chang,
linux-coco, iommu, Alexey Kardashevskiy
Here are some patches to continue enabling SEV-TIO on AMD.
SEV-TIO allows guests to establish trust in a device that supports TEE
Device Interface Security Protocol (TDISP, defined in PCIe r6.0+) and
then interact with the device via private memory.
In order to streamline upstreaming process, a common TSM infrastructure
is being developed in collaboration with Intel+ARM+RiscV. There is
Documentation/driver-api/pci/tsm.rst with proposed phases:
1. IDE: encrypt PCI, host only
2. TDISP: lock + accept flow, host and guest, interface report
3. Enable secure MMIO + DMA: IOMMUFD, KVM changes
4. Device attestation: certificates, measurements
This is phase2 == basic guest support allowing TDISP CONFIG_LOCKED and RUN states, and unlocking as well.
Acronyms:
TEE - Trusted Execution Environments, a concept of managing trust between the host and devices
TSM - TEE Security Manager (TSM), an entity which ensures security on the host
PSP - AMD platform secure processor (also "ASP", "AMD-SP"), acts as TSM on AMD.
SEV TIO - the TIO protocol implemented by the PSP and used by the host, extension to SEV-SNP
GHCB - guest/host communication block - a protocol for guest-to-host communication via a shared page
TDISP - TEE Device Interface Security Protocol (PCIe).
Flow:
- Boot guest OS, load sev-guest.ko which registers itself as a TSM
- PCI TSM creates sysfs nodes under "tsm" subdirectory in for all
TDISP-capable devices
- lock the device via:
echo tsm0 > "/sys/bus/pci/devices/0000:01:00.0/tsm/lock"
- accept the device via:
echo 1 > "/sys/bus/pci/devices/0000:01:00.0/tsm/accept"
- load the device driver:
- DMA to encrypted memory should work right away
- MMIO regions reported in TDISP interface report will be mapped as encrypted
Since one of my test devices does not use private MMIO for the main function,
there is 9/9 which allows https://github.com/billfarrow/pcimem.git mapping MMIO as private.
The previous conversation is here:
https://lore.kernel.org/r/20250218111017.491719-1-aik@amd.com
This is based on sha1
4fe8662d1a9c Dan Williams PCI/TSM: Documentation: Add Maturity Map
from
https://git.kernel.org/pub/scm/linux/kernel/git/devsec/tsm.git/log/?h=staging
and 3 cherrypicks on top, please find the exact tree at:
https://github.com/AMDESE/linux-kvm/commits/tsm-staging
The host support is pushed here:
https://github.com/AMDESE/linux-kvm/commits/tsm
The SEV TIO spec:
https://www.amd.com/content/dam/amd/en/documents/epyc-technical-docs/specifications/58271.pdf
Individual patches have extra "---" comments (could have been "RFC"?)
Please comment. Thanks.
ps: quite a cc list from get_maintainers.pl.
Alexey Kardashevskiy (9):
pci/tsm: Add TDISP report blob and helpers to parse it
pci/tsm: Add tsm_tdi_status
coco/sev-guest: Allow multiple source files in the driver
dma/swiotlb: Stop forcing SWIOTLB for TDISP devices
x86/mm: Stop forcing decrypted page state for TDISP devices
x86/dma-direct: Stop changing encrypted page state for TDISP devices
coco/sev-guest: Implement the guest support for SEV TIO (phase2)
RFC: PCI: Avoid needless touching of Command register
pci: Allow encrypted MMIO mapping via sysfs
arch/x86/Kconfig | 1 +
drivers/virt/coco/sev-guest/Kconfig | 1 +
drivers/virt/coco/sev-guest/Makefile | 6 +-
arch/x86/include/asm/dma-direct.h | 39 ++
arch/x86/include/asm/sev-common.h | 1 +
arch/x86/include/asm/sev.h | 13 +
arch/x86/include/uapi/asm/svm.h | 13 +
drivers/virt/coco/sev-guest/sev-guest.h | 20 +
include/linux/pci-tsm.h | 110 +++
include/linux/pci.h | 2 +-
include/linux/psp-sev.h | 31 +
include/linux/swiotlb.h | 9 +
include/uapi/linux/sev-guest.h | 43 ++
arch/x86/coco/sev/core.c | 53 ++
arch/x86/mm/mem_encrypt.c | 5 +-
drivers/pci/mmap.c | 11 +-
drivers/pci/pci-sysfs.c | 27 +-
drivers/pci/probe.c | 5 +
drivers/pci/proc.c | 2 +-
drivers/pci/quirks.c | 9 +
drivers/virt/coco/sev-guest/sev-guest.c | 23 +-
drivers/virt/coco/sev-guest/tio.c | 707 ++++++++++++++++++++
drivers/virt/coco/tsm-core.c | 19 +
23 files changed, 1129 insertions(+), 21 deletions(-)
create mode 100644 arch/x86/include/asm/dma-direct.h
create mode 100644 drivers/virt/coco/sev-guest/sev-guest.h
create mode 100644 drivers/virt/coco/sev-guest/tio.c
--
2.52.0
^ permalink raw reply
* Re: [PATCH] KVM: TDX: Set SIGNIFCANT_INDEX flag for supported CPUIDs
From: Binbin Wu @ 2026-02-25 3:23 UTC (permalink / raw)
To: Sean Christopherson
Cc: Rick P Edgecombe, Xiaoyao Li, changyuanl@google.com,
pbonzini@redhat.com, Binbin Wu, Isaku Yamahata, bp@alien8.de,
x86@kernel.org, kas@kernel.org, hpa@zytor.com, mingo@redhat.com,
linux-kernel@vger.kernel.org, dave.hansen@linux.intel.com,
tglx@kernel.org, kvm@vger.kernel.org, linux-coco@lists.linux.dev
In-Reply-To: <aZ3LxD5XMepnU8jh@google.com>
On 2/25/2026 12:03 AM, Sean Christopherson wrote:
> On Tue, Feb 24, 2026, Binbin Wu wrote:
>> On 2/24/2026 9:57 AM, Edgecombe, Rick P wrote:
>>>> diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
>>>> index 2d7a4d52ccfb4..0c524f9a94a6c 100644
>>>> --- a/arch/x86/kvm/vmx/tdx.c
>>>> +++ b/arch/x86/kvm/vmx/tdx.c
>>>> @@ -172,9 +172,15 @@ static void td_init_cpuid_entry2(struct
>>>> kvm_cpuid_entry2 *entry, unsigned char i
>>>> entry->ecx = (u32)td_conf->cpuid_config_values[idx][1];
>>>> entry->edx = td_conf->cpuid_config_values[idx][1] >> 32;
>>>>
>>>> - if (entry->index == KVM_TDX_CPUID_NO_SUBLEAF)
>>>> + if (entry->index == KVM_TDX_CPUID_NO_SUBLEAF) {
>>>> entry->index = 0;
>>>> + entry->flags &= ~KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
>>>
>>> There are two callers of this. One is already zeroed, and the other has
>>> stack garbage in flags. But that second caller doesn't look at the
>>> flags so it is harmless. Maybe it would be simpler and clearer to just
>>> zero init the entry struct in that caller. Then you don't need to clear
>>> it here. Or alternatively set flags to zero above, and then add
>>> KVM_CPUID_FLAG_SIGNIFCANT_INDEX if needed. Rather than manipulating a
>>> single bit in a field of garbage, which seems weird.
>
> +1, td_init_cpuid_entry2() should initialize flags to '0' and then set
> KVM_CPUID_FLAG_SIGNIFCANT_INDEX as appropriate.
>
>>>> + } else {
>>>> + entry->flags |= KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
>>>> + }
>>>>
>>>> + WARN_ON_ONCE(cpuid_function_is_indexed(entry->function) !=
>>>> + !!(entry->flags &
>>>> KVM_CPUID_FLAG_SIGNIFCANT_INDEX));
>>>
>>> It warns on leaf 0x23 for me. Is it intentional?
>>
>> I guess because the list in cpuid_function_is_indexed() is hard-coded
>> and 0x23 is not added into the list yet.
>
> Yeah, I was anticipating that we'd run afoul of leaves that aren't known to
> the kernel. FWIW, it looks like 0x24 is also indexed.
>
>> It's fine for existing KVM code because cpuid_function_is_indexed() is
>> only used to check that if a CPUID entry is queried without index, it
>> shouldn't be included in the indexed list.
>>
>> But adding the consistency check here would cause compatibility issue.
>> Generally, if a new CPUID indexed function is added for some new CPU and
>> the TDX module reports it, KVM versions without the CPUID function in
>> the list will trigger the warning.
>
> IMO, that's a good thing and working as intended. WARNs aren't inherently evil.
> While the goal is to be WARN-free, in this case triggering the WARN if the TDX
> Module is updated (or new silicon arrives) is desirable, because it alerts us to
> that new behavior, so that we can go update KVM.
So it effectively leverages the TDX module's interface to retrieve the hardware
information to validate the hard-coded list.
Do we need to consider the panic_on_warn case? I guess the option will not be
enabled in a production environment?
>
> But we should "fix" 0x23 and 0x24 before landing this patch.
>
^ permalink raw reply
* [PATCH 14/14] KVM: x86: Add helpers to prepare kvm_run for userspace MMIO exit
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Add helpers to fill kvm_run for userspace MMIO exits to deduplicate a
variety of code, and to allow for a cleaner return path in
emulator_read_write().
No functional change intended.
Cc: Rick Edgecombe <rick.p.edgecombe@intel.com>
Cc: Binbin Wu <binbin.wu@linux.intel.com>
Cc: Xiaoyao Li <xiaoyao.li@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/vmx/tdx.c | 14 ++++----------
arch/x86/kvm/x86.c | 42 ++++++++----------------------------------
arch/x86/kvm/x86.h | 24 ++++++++++++++++++++++++
3 files changed, 36 insertions(+), 44 deletions(-)
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 5df9d32d2058..a813c502336c 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -1467,17 +1467,11 @@ static int tdx_emulate_mmio(struct kvm_vcpu *vcpu)
/* Request the device emulation to userspace device model. */
vcpu->mmio_is_write = write;
- if (!write)
+
+ __kvm_prepare_emulated_mmio_exit(vcpu, gpa, size, &val, write);
+
+ if (!write) {
vcpu->arch.complete_userspace_io = tdx_complete_mmio_read;
-
- vcpu->run->mmio.phys_addr = gpa;
- vcpu->run->mmio.len = size;
- vcpu->run->mmio.is_write = write;
- vcpu->run->exit_reason = KVM_EXIT_MMIO;
-
- if (write) {
- memcpy(vcpu->run->mmio.data, &val, size);
- } else {
vcpu->mmio_fragments[0].gpa = gpa;
vcpu->mmio_fragments[0].len = size;
trace_kvm_mmio(KVM_TRACE_MMIO_READ_UNSATISFIED, size, gpa, NULL);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 5376b370b4db..889a9098403c 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8210,7 +8210,6 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
const struct read_write_emulator_ops *ops)
{
struct kvm_vcpu *vcpu = emul_to_vcpu(ctxt);
- struct kvm_mmio_fragment *frag;
int rc;
if (WARN_ON_ONCE((bytes > 8u || !ops->write) && object_is_on_stack(val)))
@@ -8268,12 +8267,9 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
vcpu->mmio_needed = 1;
vcpu->mmio_cur_fragment = 0;
+ vcpu->mmio_is_write = ops->write;
- frag = &vcpu->mmio_fragments[0];
- vcpu->run->mmio.len = min(8u, frag->len);
- vcpu->run->mmio.is_write = vcpu->mmio_is_write = ops->write;
- vcpu->run->exit_reason = KVM_EXIT_MMIO;
- vcpu->run->mmio.phys_addr = frag->gpa;
+ kvm_prepare_emulated_mmio_exit(vcpu, &vcpu->mmio_fragments[0]);
/*
* For MMIO reads, stop emulating and immediately exit to userspace, as
@@ -8283,11 +8279,7 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
* after completing emulation (see the check on vcpu->mmio_needed in
* x86_emulate_instruction()).
*/
- if (!ops->write)
- return X86EMUL_IO_NEEDED;
-
- memcpy(vcpu->run->mmio.data, frag->data, min(8u, frag->len));
- return X86EMUL_CONTINUE;
+ return ops->write ? X86EMUL_CONTINUE : X86EMUL_IO_NEEDED;
}
static int emulator_read_emulated(struct x86_emulate_ctxt *ctxt,
@@ -11884,12 +11876,7 @@ static int complete_emulated_mmio(struct kvm_vcpu *vcpu)
return complete_emulated_io(vcpu);
}
- run->exit_reason = KVM_EXIT_MMIO;
- run->mmio.phys_addr = frag->gpa;
- if (vcpu->mmio_is_write)
- memcpy(run->mmio.data, frag->data, min(8u, frag->len));
- run->mmio.len = min(8u, frag->len);
- run->mmio.is_write = vcpu->mmio_is_write;
+ kvm_prepare_emulated_mmio_exit(vcpu, frag);
vcpu->arch.complete_userspace_io = complete_emulated_mmio;
return 0;
}
@@ -14296,15 +14283,8 @@ static int complete_sev_es_emulated_mmio(struct kvm_vcpu *vcpu)
}
// More MMIO is needed
- run->mmio.phys_addr = frag->gpa;
- run->mmio.len = min(8u, frag->len);
- run->mmio.is_write = vcpu->mmio_is_write;
- if (run->mmio.is_write)
- memcpy(run->mmio.data, frag->data, min(8u, frag->len));
- run->exit_reason = KVM_EXIT_MMIO;
-
+ kvm_prepare_emulated_mmio_exit(vcpu, frag);
vcpu->arch.complete_userspace_io = complete_sev_es_emulated_mmio;
-
return 0;
}
@@ -14333,23 +14313,17 @@ int kvm_sev_es_mmio(struct kvm_vcpu *vcpu, bool is_write, gpa_t gpa,
* requests that split a page boundary.
*/
frag = vcpu->mmio_fragments;
- vcpu->mmio_nr_fragments = 1;
frag->len = bytes;
frag->gpa = gpa;
frag->data = data;
vcpu->mmio_needed = 1;
vcpu->mmio_cur_fragment = 0;
+ vcpu->mmio_nr_fragments = 1;
+ vcpu->mmio_is_write = is_write;
- vcpu->run->mmio.phys_addr = gpa;
- vcpu->run->mmio.len = min(8u, frag->len);
- vcpu->run->mmio.is_write = is_write;
- if (is_write)
- memcpy(vcpu->run->mmio.data, frag->data, min(8u, frag->len));
- vcpu->run->exit_reason = KVM_EXIT_MMIO;
-
+ kvm_prepare_emulated_mmio_exit(vcpu, frag);
vcpu->arch.complete_userspace_io = complete_sev_es_emulated_mmio;
-
return 0;
}
EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_sev_es_mmio);
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index 1d0f0edd31b3..d66f1c53d2b5 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -718,6 +718,30 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size,
unsigned int port, void *data, unsigned int count,
int in);
+static inline void __kvm_prepare_emulated_mmio_exit(struct kvm_vcpu *vcpu,
+ gpa_t gpa, unsigned int len,
+ const void *data,
+ bool is_write)
+{
+ struct kvm_run *run = vcpu->run;
+
+ run->mmio.len = min(8u, len);
+ run->mmio.is_write = is_write;
+ run->exit_reason = KVM_EXIT_MMIO;
+ run->mmio.phys_addr = gpa;
+ if (is_write)
+ memcpy(run->mmio.data, data, min(8u, len));
+}
+
+static inline void kvm_prepare_emulated_mmio_exit(struct kvm_vcpu *vcpu,
+ struct kvm_mmio_fragment *frag)
+{
+ WARN_ON_ONCE(!vcpu->mmio_needed || !vcpu->mmio_nr_fragments);
+
+ __kvm_prepare_emulated_mmio_exit(vcpu, frag->gpa, frag->len, frag->data,
+ vcpu->mmio_is_write);
+}
+
static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
{
return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 13/14] KVM: x86: Don't panic the kernel if completing userspace I/O / MMIO goes sideways
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Kill the VM instead of the host kernel if KVM botches I/O and/or MMIO
handling. There is zero danger to the host or guest, i.e. panicking the
host isn't remotely justified.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index f3e2ec7e1828..5376b370b4db 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -9710,7 +9710,8 @@ static int complete_fast_pio_in(struct kvm_vcpu *vcpu)
unsigned long val;
/* We should only ever be called with arch.pio.count equal to 1 */
- BUG_ON(vcpu->arch.pio.count != 1);
+ if (KVM_BUG_ON(vcpu->arch.pio.count != 1, vcpu->kvm))
+ return -EIO;
if (unlikely(!kvm_is_linear_rip(vcpu, vcpu->arch.cui_linear_rip))) {
vcpu->arch.pio.count = 0;
@@ -11820,7 +11821,8 @@ static inline int complete_emulated_io(struct kvm_vcpu *vcpu)
static int complete_emulated_pio(struct kvm_vcpu *vcpu)
{
- BUG_ON(!vcpu->arch.pio.count);
+ if (KVM_BUG_ON(!vcpu->arch.pio.count, vcpu->kvm))
+ return -EIO;
return complete_emulated_io(vcpu);
}
@@ -11849,7 +11851,8 @@ static int complete_emulated_mmio(struct kvm_vcpu *vcpu)
struct kvm_mmio_fragment *frag;
unsigned len;
- BUG_ON(!vcpu->mmio_needed);
+ if (KVM_BUG_ON(!vcpu->mmio_needed, vcpu->kvm))
+ return -EIO;
/* Complete previous fragment */
frag = &vcpu->mmio_fragments[vcpu->mmio_cur_fragment];
@@ -14262,7 +14265,8 @@ static int complete_sev_es_emulated_mmio(struct kvm_vcpu *vcpu)
struct kvm_mmio_fragment *frag;
unsigned int len;
- BUG_ON(!vcpu->mmio_needed);
+ if (KVM_BUG_ON(!vcpu->mmio_needed, vcpu->kvm))
+ return -EIO;
/* Complete previous fragment */
frag = &vcpu->mmio_fragments[vcpu->mmio_cur_fragment];
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 12/14] KVM: x86: Rename .read_write_emulate() to .read_write_guest()
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Rename the ops and helpers to read/write guest memory to clarify that they
do exactly that, i.e. aren't generic emulation flows and don't do anything
related to emulated MMIO.
Opportunistically add comments to explain the flow, e.g. it's not exactly
obvious that KVM deliberately treats "failed" accesses to guest memory as
emulated MMIO.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 38 ++++++++++++++++++++++++++++----------
1 file changed, 28 insertions(+), 10 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 5a3ba161db7b..f3e2ec7e1828 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8103,21 +8103,21 @@ static int vcpu_mmio_gva_to_gpa(struct kvm_vcpu *vcpu, unsigned long gva,
}
struct read_write_emulator_ops {
- int (*read_write_emulate)(struct kvm_vcpu *vcpu, gpa_t gpa,
- void *val, int bytes);
+ int (*read_write_guest)(struct kvm_vcpu *vcpu, gpa_t gpa,
+ void *val, int bytes);
int (*read_write_mmio)(struct kvm_vcpu *vcpu, gpa_t gpa,
int bytes, void *val);
bool write;
};
-static int read_emulate(struct kvm_vcpu *vcpu, gpa_t gpa,
- void *val, int bytes)
+static int emulator_read_guest(struct kvm_vcpu *vcpu, gpa_t gpa,
+ void *val, int bytes)
{
return !kvm_vcpu_read_guest(vcpu, gpa, val, bytes);
}
-static int write_emulate(struct kvm_vcpu *vcpu, gpa_t gpa,
- void *val, int bytes)
+static int emulator_write_guest(struct kvm_vcpu *vcpu, gpa_t gpa,
+ void *val, int bytes)
{
int ret;
@@ -8157,11 +8157,22 @@ static int emulator_read_write_onepage(unsigned long addr, void *val,
return X86EMUL_PROPAGATE_FAULT;
}
- if (!ret && ops->read_write_emulate(vcpu, gpa, val, bytes))
+ /*
+ * If the memory is not _known_ to be emulated MMIO, attempt to access
+ * guest memory. If accessing guest memory fails, e.g. because there's
+ * no memslot, then handle the access as MMIO. Note, treating the
+ * access as emulated MMIO is technically wrong if there is a memslot,
+ * i.e. if accessing host user memory failed, but this has been KVM's
+ * historical ABI for decades.
+ */
+ if (!ret && ops->read_write_guest(vcpu, gpa, val, bytes))
return X86EMUL_CONTINUE;
/*
- * Is this MMIO handled locally?
+ * Attempt to handle emulated MMIO within the kernel, e.g. for accesses
+ * to an in-kernel local or I/O APIC, or to an ioeventfd range attached
+ * to MMIO bus. If the access isn't fully resolved, insert an MMIO
+ * fragment with the relevant details.
*/
handled = ops->read_write_mmio(vcpu, gpa, bytes, val);
if (handled == bytes)
@@ -8182,6 +8193,13 @@ static int emulator_read_write_onepage(unsigned long addr, void *val,
frag->data = val;
}
frag->len = bytes;
+
+ /*
+ * Continue emulating, even though KVM needs to (eventually) do an MMIO
+ * exit to userspace. If the access splits multiple pages, then KVM
+ * needs to exit to userspace only after emulating both parts of the
+ * access.
+ */
return X86EMUL_CONTINUE;
}
@@ -8279,7 +8297,7 @@ static int emulator_read_emulated(struct x86_emulate_ctxt *ctxt,
struct x86_exception *exception)
{
static const struct read_write_emulator_ops ops = {
- .read_write_emulate = read_emulate,
+ .read_write_guest = emulator_read_guest,
.read_write_mmio = vcpu_mmio_read,
.write = false,
};
@@ -8294,7 +8312,7 @@ static int emulator_write_emulated(struct x86_emulate_ctxt *ctxt,
struct x86_exception *exception)
{
static const struct read_write_emulator_ops ops = {
- .read_write_emulate = write_emulate,
+ .read_write_guest = emulator_write_guest,
.read_write_mmio = vcpu_mmio_write,
.write = true,
};
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 11/14] KVM: x86: Fold emulator_write_phys() into write_emulate()
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Fold emulator_write_phys() into write_emulate() to drop a superfluous
wrapper, and to provide more symmetry between the read and write paths.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/kvm_host.h | 3 ---
arch/x86/kvm/x86.c | 20 +++++++-------------
2 files changed, 7 insertions(+), 16 deletions(-)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index ff07c45e3c73..aa030fbd669d 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -2097,9 +2097,6 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end);
int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long cr3);
-int emulator_write_phys(struct kvm_vcpu *vcpu, gpa_t gpa,
- const void *val, int bytes);
-
extern bool tdp_enabled;
u64 vcpu_tsc_khz(struct kvm_vcpu *vcpu);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 5fb5259c208f..5a3ba161db7b 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8102,18 +8102,6 @@ static int vcpu_mmio_gva_to_gpa(struct kvm_vcpu *vcpu, unsigned long gva,
return vcpu_is_mmio_gpa(vcpu, gva, *gpa, write);
}
-int emulator_write_phys(struct kvm_vcpu *vcpu, gpa_t gpa,
- const void *val, int bytes)
-{
- int ret;
-
- ret = kvm_vcpu_write_guest(vcpu, gpa, val, bytes);
- if (ret < 0)
- return 0;
- kvm_page_track_write(vcpu, gpa, val, bytes);
- return 1;
-}
-
struct read_write_emulator_ops {
int (*read_write_emulate)(struct kvm_vcpu *vcpu, gpa_t gpa,
void *val, int bytes);
@@ -8131,7 +8119,13 @@ static int read_emulate(struct kvm_vcpu *vcpu, gpa_t gpa,
static int write_emulate(struct kvm_vcpu *vcpu, gpa_t gpa,
void *val, int bytes)
{
- return emulator_write_phys(vcpu, gpa, val, bytes);
+ int ret;
+
+ ret = kvm_vcpu_write_guest(vcpu, gpa, val, bytes);
+ if (ret < 0)
+ return 0;
+ kvm_page_track_write(vcpu, gpa, val, bytes);
+ return 1;
}
static int emulator_read_write_onepage(unsigned long addr, void *val,
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 10/14] KVM: x86: Bury emulator read/write ops in emulator_{read,write}_emulated()
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Now that SEV-ES invokes vcpu_mmio_{read,write}() directly, bury the
read vs. write emulator ops in the dedicated emulator callbacks so that
they are colocated with their usage, and to make it harder for
non-emulator code to use hooks that are intended for the emulator.
Opportunistically rename the structures to get rid of the long-standing
"emultor" typo.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 29 ++++++++++++++---------------
1 file changed, 14 insertions(+), 15 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 022e953906f7..5fb5259c208f 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8134,17 +8134,6 @@ static int write_emulate(struct kvm_vcpu *vcpu, gpa_t gpa,
return emulator_write_phys(vcpu, gpa, val, bytes);
}
-static const struct read_write_emulator_ops read_emultor = {
- .read_write_emulate = read_emulate,
- .read_write_mmio = vcpu_mmio_read,
-};
-
-static const struct read_write_emulator_ops write_emultor = {
- .read_write_emulate = write_emulate,
- .read_write_mmio = vcpu_mmio_write,
- .write = true,
-};
-
static int emulator_read_write_onepage(unsigned long addr, void *val,
unsigned int bytes,
struct x86_exception *exception,
@@ -8295,8 +8284,13 @@ static int emulator_read_emulated(struct x86_emulate_ctxt *ctxt,
unsigned int bytes,
struct x86_exception *exception)
{
- return emulator_read_write(ctxt, addr, val, bytes,
- exception, &read_emultor);
+ static const struct read_write_emulator_ops ops = {
+ .read_write_emulate = read_emulate,
+ .read_write_mmio = vcpu_mmio_read,
+ .write = false,
+ };
+
+ return emulator_read_write(ctxt, addr, val, bytes, exception, &ops);
}
static int emulator_write_emulated(struct x86_emulate_ctxt *ctxt,
@@ -8305,8 +8299,13 @@ static int emulator_write_emulated(struct x86_emulate_ctxt *ctxt,
unsigned int bytes,
struct x86_exception *exception)
{
- return emulator_read_write(ctxt, addr, (void *)val, bytes,
- exception, &write_emultor);
+ static const struct read_write_emulator_ops ops = {
+ .read_write_emulate = write_emulate,
+ .read_write_mmio = vcpu_mmio_write,
+ .write = true,
+ };
+
+ return emulator_read_write(ctxt, addr, (void *)val, bytes, exception, &ops);
}
#define emulator_try_cmpxchg_user(t, ptr, old, new) \
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 09/14] KVM: x86: Consolidate SEV-ES MMIO emulation into a single public API
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Dedup kvm_sev_es_mmio_{read,write}() into a single API, as the "cost" of
plumbing in a boolean is largely negligible since KVM pulls out a boolean
for ops->write anyways, and consolidating the APIs will allow for
additional cleanups.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/svm/sev.c | 20 ++++++--------------
arch/x86/kvm/x86.c | 29 +++++++++--------------------
arch/x86/kvm/x86.h | 6 ++----
3 files changed, 17 insertions(+), 38 deletions(-)
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index ea515cf41168..723f4452302a 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -4434,25 +4434,17 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
switch (control->exit_code) {
case SVM_VMGEXIT_MMIO_READ:
- ret = setup_vmgexit_scratch(svm, true, control->exit_info_2);
- if (ret)
- break;
+ case SVM_VMGEXIT_MMIO_WRITE: {
+ bool is_write = control->exit_code == SVM_VMGEXIT_MMIO_WRITE;
- ret = kvm_sev_es_mmio_read(vcpu,
- control->exit_info_1,
- control->exit_info_2,
- svm->sev_es.ghcb_sa);
- break;
- case SVM_VMGEXIT_MMIO_WRITE:
- ret = setup_vmgexit_scratch(svm, false, control->exit_info_2);
+ ret = setup_vmgexit_scratch(svm, !is_write, control->exit_info_2);
if (ret)
break;
- ret = kvm_sev_es_mmio_write(vcpu,
- control->exit_info_1,
- control->exit_info_2,
- svm->sev_es.ghcb_sa);
+ ret = kvm_sev_es_mmio(vcpu, is_write, control->exit_info_1,
+ control->exit_info_2, svm->sev_es.ghcb_sa);
break;
+ }
case SVM_VMGEXIT_NMI_COMPLETE:
++vcpu->stat.nmi_window_exits;
svm->nmi_masked = false;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index f93f0f8961af..022e953906f7 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -14293,9 +14293,8 @@ static int complete_sev_es_emulated_mmio(struct kvm_vcpu *vcpu)
return 0;
}
-static int kvm_sev_es_do_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
- unsigned int bytes, void *data,
- const struct read_write_emulator_ops *ops)
+int kvm_sev_es_mmio(struct kvm_vcpu *vcpu, bool is_write, gpa_t gpa,
+ unsigned int bytes, void *data)
{
struct kvm_mmio_fragment *frag;
int handled;
@@ -14303,7 +14302,10 @@ static int kvm_sev_es_do_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
if (!data || WARN_ON_ONCE(object_is_on_stack(data)))
return -EINVAL;
- handled = ops->read_write_mmio(vcpu, gpa, bytes, data);
+ if (is_write)
+ handled = vcpu_mmio_write(vcpu, gpa, bytes, data);
+ else
+ handled = vcpu_mmio_read(vcpu, gpa, bytes, data);
if (handled == bytes)
return 1;
@@ -14326,8 +14328,8 @@ static int kvm_sev_es_do_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
vcpu->run->mmio.phys_addr = gpa;
vcpu->run->mmio.len = min(8u, frag->len);
- vcpu->run->mmio.is_write = ops->write;
- if (ops->write)
+ vcpu->run->mmio.is_write = is_write;
+ if (is_write)
memcpy(vcpu->run->mmio.data, frag->data, min(8u, frag->len));
vcpu->run->exit_reason = KVM_EXIT_MMIO;
@@ -14335,20 +14337,7 @@ static int kvm_sev_es_do_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
return 0;
}
-
-int kvm_sev_es_mmio_write(struct kvm_vcpu *vcpu, gpa_t gpa, unsigned int bytes,
- void *data)
-{
- return kvm_sev_es_do_mmio(vcpu, gpa, bytes, data, &write_emultor);
-}
-EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_sev_es_mmio_write);
-
-int kvm_sev_es_mmio_read(struct kvm_vcpu *vcpu, gpa_t gpa, unsigned int bytes,
- void *data)
-{
- return kvm_sev_es_do_mmio(vcpu, gpa, bytes, data, &read_emultor);
-}
-EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_sev_es_mmio_read);
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_sev_es_mmio);
static void advance_sev_es_emulated_pio(struct kvm_vcpu *vcpu, unsigned count, int size)
{
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index 94d4f07aaaa0..1d0f0edd31b3 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -712,10 +712,8 @@ static inline bool __kvm_is_valid_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)
__reserved_bits; \
})
-int kvm_sev_es_mmio_write(struct kvm_vcpu *vcpu, gpa_t src, unsigned int bytes,
- void *dst);
-int kvm_sev_es_mmio_read(struct kvm_vcpu *vcpu, gpa_t src, unsigned int bytes,
- void *dst);
+int kvm_sev_es_mmio(struct kvm_vcpu *vcpu, bool is_write, gpa_t gpa,
+ unsigned int bytes, void *data);
int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size,
unsigned int port, void *data, unsigned int count,
int in);
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 08/14] KVM: x86: Dedup kvm_sev_es_mmio_{read,write}()
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Dedup the SEV-ES emulated MMIO code by using the read vs. write emulator
ops to handle the few differences between reads and writes.
Opportunistically tweak the comment about fragments to call out that KVM
should verify that userspace can actually handle MMIO requests that cross
page boundaries. Unlike emulated MMIO, the request is made in the GPA
space, not the GVA space, i.e. emulation across page boundaries can work
generically, at least in theory.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 110 +++++++++++++++++++--------------------------
1 file changed, 45 insertions(+), 65 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 2db0bf738d2d..f93f0f8961af 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -14293,80 +14293,60 @@ static int complete_sev_es_emulated_mmio(struct kvm_vcpu *vcpu)
return 0;
}
+static int kvm_sev_es_do_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
+ unsigned int bytes, void *data,
+ const struct read_write_emulator_ops *ops)
+{
+ struct kvm_mmio_fragment *frag;
+ int handled;
+
+ if (!data || WARN_ON_ONCE(object_is_on_stack(data)))
+ return -EINVAL;
+
+ handled = ops->read_write_mmio(vcpu, gpa, bytes, data);
+ if (handled == bytes)
+ return 1;
+
+ bytes -= handled;
+ gpa += handled;
+ data += handled;
+
+ /*
+ * TODO: Determine whether or not userspace plays nice with MMIO
+ * requests that split a page boundary.
+ */
+ frag = vcpu->mmio_fragments;
+ vcpu->mmio_nr_fragments = 1;
+ frag->len = bytes;
+ frag->gpa = gpa;
+ frag->data = data;
+
+ vcpu->mmio_needed = 1;
+ vcpu->mmio_cur_fragment = 0;
+
+ vcpu->run->mmio.phys_addr = gpa;
+ vcpu->run->mmio.len = min(8u, frag->len);
+ vcpu->run->mmio.is_write = ops->write;
+ if (ops->write)
+ memcpy(vcpu->run->mmio.data, frag->data, min(8u, frag->len));
+ vcpu->run->exit_reason = KVM_EXIT_MMIO;
+
+ vcpu->arch.complete_userspace_io = complete_sev_es_emulated_mmio;
+
+ return 0;
+}
+
int kvm_sev_es_mmio_write(struct kvm_vcpu *vcpu, gpa_t gpa, unsigned int bytes,
void *data)
{
- int handled;
- struct kvm_mmio_fragment *frag;
-
- if (!data || WARN_ON_ONCE(object_is_on_stack(data)))
- return -EINVAL;
-
- handled = write_emultor.read_write_mmio(vcpu, gpa, bytes, data);
- if (handled == bytes)
- return 1;
-
- bytes -= handled;
- gpa += handled;
- data += handled;
-
- /*TODO: Check if need to increment number of frags */
- frag = vcpu->mmio_fragments;
- vcpu->mmio_nr_fragments = 1;
- frag->len = bytes;
- frag->gpa = gpa;
- frag->data = data;
-
- vcpu->mmio_needed = 1;
- vcpu->mmio_cur_fragment = 0;
-
- vcpu->run->mmio.phys_addr = gpa;
- vcpu->run->mmio.len = min(8u, frag->len);
- vcpu->run->mmio.is_write = 1;
- memcpy(vcpu->run->mmio.data, frag->data, min(8u, frag->len));
- vcpu->run->exit_reason = KVM_EXIT_MMIO;
-
- vcpu->arch.complete_userspace_io = complete_sev_es_emulated_mmio;
-
- return 0;
+ return kvm_sev_es_do_mmio(vcpu, gpa, bytes, data, &write_emultor);
}
EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_sev_es_mmio_write);
int kvm_sev_es_mmio_read(struct kvm_vcpu *vcpu, gpa_t gpa, unsigned int bytes,
void *data)
{
- int handled;
- struct kvm_mmio_fragment *frag;
-
- if (!data || WARN_ON_ONCE(object_is_on_stack(data)))
- return -EINVAL;
-
- handled = read_emultor.read_write_mmio(vcpu, gpa, bytes, data);
- if (handled == bytes)
- return 1;
-
- bytes -= handled;
- gpa += handled;
- data += handled;
-
- /*TODO: Check if need to increment number of frags */
- frag = vcpu->mmio_fragments;
- vcpu->mmio_nr_fragments = 1;
- frag->len = bytes;
- frag->gpa = gpa;
- frag->data = data;
-
- vcpu->mmio_needed = 1;
- vcpu->mmio_cur_fragment = 0;
-
- vcpu->run->mmio.phys_addr = gpa;
- vcpu->run->mmio.len = min(8u, frag->len);
- vcpu->run->mmio.is_write = 0;
- vcpu->run->exit_reason = KVM_EXIT_MMIO;
-
- vcpu->arch.complete_userspace_io = complete_sev_es_emulated_mmio;
-
- return 0;
+ return kvm_sev_es_do_mmio(vcpu, gpa, bytes, data, &read_emultor);
}
EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_sev_es_mmio_read);
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 07/14] KVM: x86: Harden SEV-ES MMIO against on-stack use-after-free
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Add a sanity check to ensure KVM doesn't use an on-stack variable when
handling an MMIO request for an SEV-ES guest. The source/destination
for SEV-ES MMIO should _always_ be the #VMGEXIT scratch area.
Opportunistically update the comment in the completion side of things
to clarify that frag->data doesn't need to be copied anywhere, and the
VMEGEXIT is trap-like (the current comment doesn't clarify *how* RIP is
advanced).
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 7abd6f93c386..2db0bf738d2d 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -14273,8 +14273,10 @@ static int complete_sev_es_emulated_mmio(struct kvm_vcpu *vcpu)
if (vcpu->mmio_cur_fragment >= vcpu->mmio_nr_fragments) {
vcpu->mmio_needed = 0;
- // VMG change, at this point, we're always done
- // RIP has already been advanced
+ /*
+ * All done, as frag->data always points at the GHCB scratch
+ * area and VMGEXIT is trap-like (RIP is advanced by hardware).
+ */
return 1;
}
@@ -14297,7 +14299,7 @@ int kvm_sev_es_mmio_write(struct kvm_vcpu *vcpu, gpa_t gpa, unsigned int bytes,
int handled;
struct kvm_mmio_fragment *frag;
- if (!data)
+ if (!data || WARN_ON_ONCE(object_is_on_stack(data)))
return -EINVAL;
handled = write_emultor.read_write_mmio(vcpu, gpa, bytes, data);
@@ -14336,7 +14338,7 @@ int kvm_sev_es_mmio_read(struct kvm_vcpu *vcpu, gpa_t gpa, unsigned int bytes,
int handled;
struct kvm_mmio_fragment *frag;
- if (!data)
+ if (!data || WARN_ON_ONCE(object_is_on_stack(data)))
return -EINVAL;
handled = read_emultor.read_write_mmio(vcpu, gpa, bytes, data);
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 06/14] KVM: x86: Move MMIO write tracing into vcpu_mmio_write()
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Move the invocation of MMIO write tracepoint into vcpu_mmio_write() and
drop its largely-useless wrapper to cull pointless code and to make the
code symmetrical with respect to vcpu_mmio_read().
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 13 +++++--------
1 file changed, 5 insertions(+), 8 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 5fde5bb010e7..7abd6f93c386 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -7769,11 +7769,14 @@ static void kvm_init_msr_lists(void)
}
static int vcpu_mmio_write(struct kvm_vcpu *vcpu, gpa_t addr, int len,
- const void *v)
+ void *__v)
{
+ const void *v = __v;
int handled = 0;
int n;
+ trace_kvm_mmio(KVM_TRACE_MMIO_WRITE, len, addr, __v);
+
do {
n = min(len, 8);
if (!(lapic_in_kernel(vcpu) &&
@@ -8131,12 +8134,6 @@ static int write_emulate(struct kvm_vcpu *vcpu, gpa_t gpa,
return emulator_write_phys(vcpu, gpa, val, bytes);
}
-static int write_mmio(struct kvm_vcpu *vcpu, gpa_t gpa, int bytes, void *val)
-{
- trace_kvm_mmio(KVM_TRACE_MMIO_WRITE, bytes, gpa, val);
- return vcpu_mmio_write(vcpu, gpa, bytes, val);
-}
-
static const struct read_write_emulator_ops read_emultor = {
.read_write_emulate = read_emulate,
.read_write_mmio = vcpu_mmio_read,
@@ -8144,7 +8141,7 @@ static const struct read_write_emulator_ops read_emultor = {
static const struct read_write_emulator_ops write_emultor = {
.read_write_emulate = write_emulate,
- .read_write_mmio = write_mmio,
+ .read_write_mmio = vcpu_mmio_write,
.write = true,
};
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 05/14] KVM: x86: Open code read vs. write userspace MMIO exits in emulator_read_write()
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Open code the differences in read vs. write userspace MMIO exits instead
of burying three lines of code behind indirect callbacks, as splitting the
logic makes it extremely hard to track that KVM's handling of reads vs.
write is _significantly_ different. Add a comment to explain why the
semantics are different, and how on earth an MMIO write ends up triggering
an exit to userspace.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 33 +++++++++++++--------------------
1 file changed, 13 insertions(+), 20 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 7cbd6f7d8578..5fde5bb010e7 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8116,8 +8116,6 @@ struct read_write_emulator_ops {
void *val, int bytes);
int (*read_write_mmio)(struct kvm_vcpu *vcpu, gpa_t gpa,
int bytes, void *val);
- int (*read_write_exit_mmio)(struct kvm_vcpu *vcpu, gpa_t gpa,
- void *val, int bytes);
bool write;
};
@@ -8139,31 +8137,14 @@ static int write_mmio(struct kvm_vcpu *vcpu, gpa_t gpa, int bytes, void *val)
return vcpu_mmio_write(vcpu, gpa, bytes, val);
}
-static int read_exit_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
- void *val, int bytes)
-{
- return X86EMUL_IO_NEEDED;
-}
-
-static int write_exit_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
- void *val, int bytes)
-{
- struct kvm_mmio_fragment *frag = &vcpu->mmio_fragments[0];
-
- memcpy(vcpu->run->mmio.data, frag->data, min(8u, frag->len));
- return X86EMUL_CONTINUE;
-}
-
static const struct read_write_emulator_ops read_emultor = {
.read_write_emulate = read_emulate,
.read_write_mmio = vcpu_mmio_read,
- .read_write_exit_mmio = read_exit_mmio,
};
static const struct read_write_emulator_ops write_emultor = {
.read_write_emulate = write_emulate,
.read_write_mmio = write_mmio,
- .read_write_exit_mmio = write_exit_mmio,
.write = true,
};
@@ -8296,7 +8277,19 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
vcpu->run->exit_reason = KVM_EXIT_MMIO;
vcpu->run->mmio.phys_addr = frag->gpa;
- return ops->read_write_exit_mmio(vcpu, frag->gpa, val, bytes);
+ /*
+ * For MMIO reads, stop emulating and immediately exit to userspace, as
+ * KVM needs the value to correctly emulate the instruction. For MMIO
+ * writes, continue emulating as the write to MMIO is a side effect for
+ * all intents and purposes. KVM will still exit to userspace, but
+ * after completing emulation (see the check on vcpu->mmio_needed in
+ * x86_emulate_instruction()).
+ */
+ if (!ops->write)
+ return X86EMUL_IO_NEEDED;
+
+ memcpy(vcpu->run->mmio.data, frag->data, min(8u, frag->len));
+ return X86EMUL_CONTINUE;
}
static int emulator_read_emulated(struct x86_emulate_ctxt *ctxt,
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 04/14] KVM: x86: Use local MMIO fragment variable to clean up emulator_read_write()
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Grab the MMIO fragment used by emulator_read_write() to initiate an exit
to userspace in a local variable to make the code easier to read.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index a74ae3a81076..7cbd6f7d8578 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8231,7 +8231,7 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
const struct read_write_emulator_ops *ops)
{
struct kvm_vcpu *vcpu = emul_to_vcpu(ctxt);
- gpa_t gpa;
+ struct kvm_mmio_fragment *frag;
int rc;
if (WARN_ON_ONCE((bytes > 8u || !ops->write) && object_is_on_stack(val)))
@@ -8287,17 +8287,16 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
if (!vcpu->mmio_nr_fragments)
return X86EMUL_CONTINUE;
- gpa = vcpu->mmio_fragments[0].gpa;
-
vcpu->mmio_needed = 1;
vcpu->mmio_cur_fragment = 0;
- vcpu->run->mmio.len = min(8u, vcpu->mmio_fragments[0].len);
+ frag = &vcpu->mmio_fragments[0];
+ vcpu->run->mmio.len = min(8u, frag->len);
vcpu->run->mmio.is_write = vcpu->mmio_is_write = ops->write;
vcpu->run->exit_reason = KVM_EXIT_MMIO;
- vcpu->run->mmio.phys_addr = gpa;
+ vcpu->run->mmio.phys_addr = frag->gpa;
- return ops->read_write_exit_mmio(vcpu, gpa, val, bytes);
+ return ops->read_write_exit_mmio(vcpu, frag->gpa, val, bytes);
}
static int emulator_read_emulated(struct x86_emulate_ctxt *ctxt,
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 03/14] KVM: x86: Trace unsatisfied MMIO reads on a per-page basis
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Invoke the "unsatisfied MMIO reads" when KVM first detects that a
particular access "chunk" requires an exit to userspace instead of tracing
the entire access at the time KVM initiates the exit to userspace. I.e.
precisely trace the first and/or second fragments of a page split instead
of tracing the entire access, as the GPA could be wrong on a page split
case.
Leave the completion tracepoint alone, at least for now, as fixing the
completion path would incur significantly complexity to track exactly which
fragment(s) of the overall access actually triggered MMIO, but add a
comment that the tracing for completed reads in is technically wrong.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 8b1f02cc8196..a74ae3a81076 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -7808,6 +7808,9 @@ static int vcpu_mmio_read(struct kvm_vcpu *vcpu, gpa_t addr, int len, void *v)
v += n;
} while (len);
+ if (len)
+ trace_kvm_mmio(KVM_TRACE_MMIO_READ_UNSATISFIED, len, addr, NULL);
+
return handled;
}
@@ -8139,7 +8142,6 @@ static int write_mmio(struct kvm_vcpu *vcpu, gpa_t gpa, int bytes, void *val)
static int read_exit_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
void *val, int bytes)
{
- trace_kvm_mmio(KVM_TRACE_MMIO_READ_UNSATISFIED, bytes, gpa, NULL);
return X86EMUL_IO_NEEDED;
}
@@ -8247,6 +8249,11 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
* is copied to the correct chunk of the correct operand.
*/
if (!ops->write && vcpu->mmio_read_completed) {
+ /*
+ * For simplicity, trace the entire MMIO read in one shot, even
+ * though the GPA might be incorrect if there are two fragments
+ * that aren't contiguous in the GPA space.
+ */
trace_kvm_mmio(KVM_TRACE_MMIO_READ, bytes,
vcpu->mmio_fragments[0].gpa, val);
vcpu->mmio_read_completed = 0;
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 02/14] KVM: x86: Open code handling of completed MMIO reads in emulator_read_write()
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
Open code the handling of completed MMIO reads instead of using an ops
hook, as burying the logic behind a (likely RETPOLINE'd) indirect call,
and with an unintuitive name, makes relatively straightforward code hard
to comprehend.
Opportunistically add comments to explain the dependencies between the
emulator's mem_read cache and the MMIO read completion logic, as it's very
easy to overlook the cache's role in getting the read data into the
correct destination.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/emulate.c | 13 +++++++++++++
arch/x86/kvm/x86.c | 33 ++++++++++++++++-----------------
2 files changed, 29 insertions(+), 17 deletions(-)
diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
index c8e292e9a24d..70850e591350 100644
--- a/arch/x86/kvm/emulate.c
+++ b/arch/x86/kvm/emulate.c
@@ -1297,12 +1297,25 @@ static int read_emulated(struct x86_emulate_ctxt *ctxt,
int rc;
struct read_cache *mc = &ctxt->mem_read;
+ /*
+ * If the read gets a cache hit, simply copy the value from the cache.
+ * A "hit" here means that there is unused data in the cache, i.e. when
+ * re-emulating an instruction to complete a userspace exit, KVM relies
+ * on "no decode" to ensure the instruction is re-emulated in the same
+ * sequence, so that multiple reads are fulfilled in the correct order.
+ */
if (mc->pos < mc->end)
goto read_cached;
if (KVM_EMULATOR_BUG_ON((mc->end + size) >= sizeof(mc->data), ctxt))
return X86EMUL_UNHANDLEABLE;
+ /*
+ * Route all reads to the cache. This allows @dest to be an on-stack
+ * variable without triggering use-after-free if KVM needs to exit to
+ * userspace to handle an MMIO read (the MMIO fragment will point at
+ * the current location in the cache).
+ */
rc = ctxt->ops->read_emulated(ctxt, addr, mc->data + mc->end, size,
&ctxt->exception);
if (rc != X86EMUL_CONTINUE)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index ff3a6f86973f..8b1f02cc8196 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8109,8 +8109,6 @@ int emulator_write_phys(struct kvm_vcpu *vcpu, gpa_t gpa,
}
struct read_write_emulator_ops {
- int (*read_write_prepare)(struct kvm_vcpu *vcpu, void *val,
- int bytes);
int (*read_write_emulate)(struct kvm_vcpu *vcpu, gpa_t gpa,
void *val, int bytes);
int (*read_write_mmio)(struct kvm_vcpu *vcpu, gpa_t gpa,
@@ -8120,18 +8118,6 @@ struct read_write_emulator_ops {
bool write;
};
-static int read_prepare(struct kvm_vcpu *vcpu, void *val, int bytes)
-{
- if (vcpu->mmio_read_completed) {
- trace_kvm_mmio(KVM_TRACE_MMIO_READ, bytes,
- vcpu->mmio_fragments[0].gpa, val);
- vcpu->mmio_read_completed = 0;
- return 1;
- }
-
- return 0;
-}
-
static int read_emulate(struct kvm_vcpu *vcpu, gpa_t gpa,
void *val, int bytes)
{
@@ -8167,7 +8153,6 @@ static int write_exit_mmio(struct kvm_vcpu *vcpu, gpa_t gpa,
}
static const struct read_write_emulator_ops read_emultor = {
- .read_write_prepare = read_prepare,
.read_write_emulate = read_emulate,
.read_write_mmio = vcpu_mmio_read,
.read_write_exit_mmio = read_exit_mmio,
@@ -8250,9 +8235,23 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
if (WARN_ON_ONCE((bytes > 8u || !ops->write) && object_is_on_stack(val)))
return X86EMUL_UNHANDLEABLE;
- if (ops->read_write_prepare &&
- ops->read_write_prepare(vcpu, val, bytes))
+ /*
+ * If the read was already completed via a userspace MMIO exit, there's
+ * nothing left to do except trace the MMIO read. When completing MMIO
+ * reads, KVM re-emulates the instruction to propagate the value into
+ * the correct destination, e.g. into the correct register, but the
+ * value itself has already been copied to the read cache.
+ *
+ * Note! This is *tightly* coupled to read_emulated() satisfying reads
+ * from the emulator's mem_read cache, so that the MMIO fragment data
+ * is copied to the correct chunk of the correct operand.
+ */
+ if (!ops->write && vcpu->mmio_read_completed) {
+ trace_kvm_mmio(KVM_TRACE_MMIO_READ, bytes,
+ vcpu->mmio_fragments[0].gpa, val);
+ vcpu->mmio_read_completed = 0;
return X86EMUL_CONTINUE;
+ }
vcpu->mmio_nr_fragments = 0;
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 01/14] KVM: x86: Use scratch field in MMIO fragment to hold small write values
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
In-Reply-To: <20260225012049.920665-1-seanjc@google.com>
When exiting to userspace to service an emulated MMIO write, copy the
to-be-written value to a scratch field in the MMIO fragment if the size
of the data payload is 8 bytes or less, i.e. can fit in a single chunk,
instead of pointing the fragment directly at the source value.
This fixes a class of use-after-free bugs that occur when the emulator
initiates a write using an on-stack, local variable as the source, the
write splits a page boundary, *and* both pages are MMIO pages. Because
KVM's ABI only allows for physically contiguous MMIO requests, accesses
that split MMIO pages are separated into two fragments, and are sent to
userspace one at a time. When KVM attempts to complete userspace MMIO in
response to KVM_RUN after the first fragment, KVM will detect the second
fragment and generate a second userspace exit, and reference the on-stack
variable.
The issue is most visible if the second KVM_RUN is performed by a separate
task, in which case the stack of the initiating task can show up as truly
freed data.
==================================================================
BUG: KASAN: use-after-free in complete_emulated_mmio+0x305/0x420
Read of size 1 at addr ffff888009c378d1 by task syz-executor417/984
CPU: 1 PID: 984 Comm: syz-executor417 Not tainted 5.10.0-182.0.0.95.h2627.eulerosv2r13.x86_64 #3
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.15.0-0-g2dd4b9b3f840-prebuilt.qemu.org 04/01/2014 Call Trace:
dump_stack+0xbe/0xfd
print_address_description.constprop.0+0x19/0x170
__kasan_report.cold+0x6c/0x84
kasan_report+0x3a/0x50
check_memory_region+0xfd/0x1f0
memcpy+0x20/0x60
complete_emulated_mmio+0x305/0x420
kvm_arch_vcpu_ioctl_run+0x63f/0x6d0
kvm_vcpu_ioctl+0x413/0xb20
__se_sys_ioctl+0x111/0x160
do_syscall_64+0x30/0x40
entry_SYSCALL_64_after_hwframe+0x67/0xd1
RIP: 0033:0x42477d
Code: <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007faa8e6890e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00000000004d7338 RCX: 000000000042477d
RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000005
RBP: 00000000004d7330 R08: 00007fff28d546df R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000004d733c
R13: 0000000000000000 R14: 000000000040a200 R15: 00007fff28d54720
The buggy address belongs to the page:
page:0000000029f6a428 refcount:0 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x9c37
flags: 0xfffffc0000000(node=0|zone=1|lastcpupid=0x1fffff)
raw: 000fffffc0000000 0000000000000000 ffffea0000270dc8 0000000000000000
raw: 0000000000000000 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: kasan: bad access detected
Memory state around the buggy address:
ffff888009c37780: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ffff888009c37800: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>ffff888009c37880: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
^
ffff888009c37900: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ffff888009c37980: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
==================================================================
The bug can also be reproduced with a targeted KVM-Unit-Test by hacking
KVM to fill a large on-stack variable in complete_emulated_mmio(), i.e. by
overwrite the data value with garbage.
Limit the use of the scratch fields to 8-byte or smaller accesses, and to
just writes, as larger accesses and reads are not affected thanks to
implementation details in the emulator, but add a sanity check to ensure
those details don't change in the future. Specifically, KVM never uses
on-stack variables for accesses larger that 8 bytes, e.g. uses an operand
in the emulator context, and *all* reads are buffered through the mem_read
cache.
Note! Using the scratch field for reads is not only unnecessary, it's
also extremely difficult to handle correctly. As above, KVM buffers all
reads through the mem_read cache, and heavily relies on that behavior when
re-emulating the instruction after a userspace MMIO read exit. If a read
splits a page, the first page is NOT an MMIO page, and the second page IS
an MMIO page, then the MMIO fragment needs to point at _just_ the second
chunk of the destination, i.e. its position in the mem_read cache. Taking
the "obvious" approach of copying the fragment value into the destination
when re-emulating the instruction would clobber the first chunk of the
destination, i.e. would clobber the data that was read from guest memory.
Fixes: f78146b0f923 ("KVM: Fix page-crossing MMIO")
Suggested-by: Yashu Zhang <zhangjiaji1@huawei.com>
Reported-by: Yashu Zhang <zhangjiaji1@huawei.com>
Closes: https://lore.kernel.org/all/369eaaa2b3c1425c85e8477066391bc7@huawei.com
Cc: stable@vger.kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 14 +++++++++++++-
include/linux/kvm_host.h | 3 ++-
2 files changed, 15 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index db3f393192d9..ff3a6f86973f 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8226,7 +8226,13 @@ static int emulator_read_write_onepage(unsigned long addr, void *val,
WARN_ON(vcpu->mmio_nr_fragments >= KVM_MAX_MMIO_FRAGMENTS);
frag = &vcpu->mmio_fragments[vcpu->mmio_nr_fragments++];
frag->gpa = gpa;
- frag->data = val;
+ if (write && bytes <= 8u) {
+ frag->val = 0;
+ frag->data = &frag->val;
+ memcpy(&frag->val, val, bytes);
+ } else {
+ frag->data = val;
+ }
frag->len = bytes;
return X86EMUL_CONTINUE;
}
@@ -8241,6 +8247,9 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
gpa_t gpa;
int rc;
+ if (WARN_ON_ONCE((bytes > 8u || !ops->write) && object_is_on_stack(val)))
+ return X86EMUL_UNHANDLEABLE;
+
if (ops->read_write_prepare &&
ops->read_write_prepare(vcpu, val, bytes))
return X86EMUL_CONTINUE;
@@ -11847,6 +11856,9 @@ static int complete_emulated_mmio(struct kvm_vcpu *vcpu)
frag++;
vcpu->mmio_cur_fragment++;
} else {
+ if (WARN_ON_ONCE(frag->data == &frag->val))
+ return -EIO;
+
/* Go forward to the next mmio piece. */
frag->data += len;
frag->gpa += len;
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 2c7d76262898..0bb2a34fb93d 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -320,7 +320,8 @@ static inline bool kvm_vcpu_can_poll(ktime_t cur, ktime_t stop)
struct kvm_mmio_fragment {
gpa_t gpa;
void *data;
- unsigned len;
+ u64 val;
+ unsigned int len;
};
struct kvm_vcpu {
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* [PATCH 00/14] KVM: x86: Emulator MMIO fix and cleanups
From: Sean Christopherson @ 2026-02-25 1:20 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Yashu Zhang, Rick Edgecombe,
Binbin Wu, Xiaoyao Li, Tom Lendacky, Michael Roth
Fix a UAF stack bug where KVM references a stack pointer around an exit to
userspace, and then clean up the related code to try to make it easier to
maintain (not necessarily "easy", but "easier").
The SEV-ES and TDX changes are compile-tested only.
Sean Christopherson (14):
KVM: x86: Use scratch field in MMIO fragment to hold small write
values
KVM: x86: Open code handling of completed MMIO reads in
emulator_read_write()
KVM: x86: Trace unsatisfied MMIO reads on a per-page basis
KVM: x86: Use local MMIO fragment variable to clean up
emulator_read_write()
KVM: x86: Open code read vs. write userspace MMIO exits in
emulator_read_write()
KVM: x86: Move MMIO write tracing into vcpu_mmio_write()
KVM: x86: Harden SEV-ES MMIO against on-stack use-after-free
KVM: x86: Dedup kvm_sev_es_mmio_{read,write}()
KVM: x86: Consolidate SEV-ES MMIO emulation into a single public API
KVM: x86: Bury emulator read/write ops in
emulator_{read,write}_emulated()
KVM: x86: Fold emulator_write_phys() into write_emulate()
KVM: x86: Rename .read_write_emulate() to .read_write_guest()
KVM: x86: Don't panic the kernel if completing userspace I/O / MMIO
goes sideways
KVM: x86: Add helpers to prepare kvm_run for userspace MMIO exit
arch/x86/include/asm/kvm_host.h | 3 -
arch/x86/kvm/emulate.c | 13 ++
arch/x86/kvm/svm/sev.c | 20 +--
arch/x86/kvm/vmx/tdx.c | 14 +-
arch/x86/kvm/x86.c | 287 ++++++++++++++------------------
arch/x86/kvm/x86.h | 30 +++-
include/linux/kvm_host.h | 3 +-
7 files changed, 178 insertions(+), 192 deletions(-)
base-commit: 183bb0ce8c77b0fd1fb25874112bc8751a461e49
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply
* Re: [PATCH] KVM: TDX: Set SIGNIFCANT_INDEX flag for supported CPUIDs
From: Binbin Wu @ 2026-02-25 0:18 UTC (permalink / raw)
To: Sean Christopherson
Cc: Rick P Edgecombe, Xiaoyao Li, changyuanl@google.com,
pbonzini@redhat.com, Binbin Wu, Isaku Yamahata, bp@alien8.de,
x86@kernel.org, kas@kernel.org, hpa@zytor.com, mingo@redhat.com,
linux-kernel@vger.kernel.org, dave.hansen@linux.intel.com,
tglx@kernel.org, kvm@vger.kernel.org, linux-coco@lists.linux.dev
In-Reply-To: <aZ3LxD5XMepnU8jh@google.com>
On 2/25/2026 12:03 AM, Sean Christopherson wrote:
>
>>>> + } else {
>>>> + entry->flags |= KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
>>>> + }
>>>>
>>>> + WARN_ON_ONCE(cpuid_function_is_indexed(entry->function) !=
>>>> + !!(entry->flags &
>>>> KVM_CPUID_FLAG_SIGNIFCANT_INDEX));
>>>
>>> It warns on leaf 0x23 for me. Is it intentional?
>>
>> I guess because the list in cpuid_function_is_indexed() is hard-coded
>> and 0x23 is not added into the list yet.
>
> Yeah, I was anticipating that we'd run afoul of leaves that aren't known to
> the kernel. FWIW, it looks like 0x24 is also indexed.
0x24 is there already.
^ permalink raw reply
* Re: [PATCH] KVM: TDX: Set SIGNIFCANT_INDEX flag for supported CPUIDs
From: Edgecombe, Rick P @ 2026-02-24 21:44 UTC (permalink / raw)
To: seanjc@google.com
Cc: changyuanl@google.com, linux-coco@lists.linux.dev, Li, Xiaoyao,
kvm@vger.kernel.org, dave.hansen@linux.intel.com, Wu, Binbin,
kas@kernel.org, linux-kernel@vger.kernel.org,
binbin.wu@linux.intel.com, pbonzini@redhat.com, mingo@redhat.com,
Yamahata, Isaku, hpa@zytor.com, tglx@kernel.org, bp@alien8.de,
x86@kernel.org
In-Reply-To: <aZ4NUZdbH6PPUr5K@google.com>
On Tue, 2026-02-24 at 12:42 -0800, Sean Christopherson wrote:
> > I'm still not clear on the impact of this one, but assuming it's not too
> > serious, could we discuss the WIP CPUID bit TDX arch stuff in PUCK before
> > doing the change?
>
> Sure, I don't see a rush on the patch.
Should we try for tomorrow or next week?
>
> > We were initially focusing on the problem of CPUID bits that affect host
> > state, but then recently were discussing how many other categories of
> > potential problems we should worry about at this point. So it would be good
> > to understand the impact here.
> >
> > If this warn is a trend towards doubling back on the initial decision to
> > expose the CPUID interface to userspace,
>
> Maybe I'm missing something, but I think you're reading into the WARN waaaay
> too much. I suggested it purely as a paranoid guard against the TDX Module
> doing something bizarre and/or the kernel fat-fingering a CPUID function.
> I.e. there's no ulterior motive here, unless maybe Changyuan is planning world
> domination or something. :-D
Heh, well we are already seeing new CPUID bits that cause problems. Not
suspecting any secret motives, but more trying to glean something on your
thinking. Will be easier to discuss the topic.
>
> > which I think is still doable and worth considering as an alternative, then
> > this also affects how we would want the TDX module changes to work.
^ permalink raw reply
* Re: [PATCH 1/3] cpu/bugs: Fix selecting Automatic IBRS using spectre_v2=eibrs
From: Borislav Petkov @ 2026-02-24 21:40 UTC (permalink / raw)
To: Dave Hansen, Kim Phillips, linux-kernel, kvm, linux-coco, x86
Cc: Sean Christopherson, Paolo Bonzini, K Prateek Nayak,
Nikunj A Dadhania, Tom Lendacky, Michael Roth, Borislav Petkov,
Naveen Rao, David Kaplan, Pawan Gupta, stable
In-Reply-To: <6b3b0c86-99eb-406d-b88d-3d71613bef9e@intel.com>
On February 24, 2026 6:22:36 PM UTC, Dave Hansen <dave.hansen@intel.com> wrote:
>On 2/24/26 10:01, Kim Phillips wrote:
>> @@ -2136,7 +2136,8 @@ static void __init spectre_v2_select_mitigation(void)
>> if ((spectre_v2_cmd == SPECTRE_V2_CMD_EIBRS ||
>> spectre_v2_cmd == SPECTRE_V2_CMD_EIBRS_LFENCE ||
>> spectre_v2_cmd == SPECTRE_V2_CMD_EIBRS_RETPOLINE) &&
>> - !boot_cpu_has(X86_FEATURE_IBRS_ENHANCED)) {
>> + !(boot_cpu_has(X86_FEATURE_IBRS_ENHANCED) ||
>> + boot_cpu_has(X86_FEATURE_AUTOIBRS))) {
>> pr_err("EIBRS selected but CPU doesn't have Enhanced or Automatic IBRS. Switching to AUTO select\n");
>> spectre_v2_cmd = SPECTRE_V2_CMD_AUTO;
>> }
>
>Didn't we agree to just use the "Intel feature" name? See this existing
>code:
>
>> /*
>> * AMD's AutoIBRS is equivalent to Intel's eIBRS - use the Intel feature
>> * flag and protect from vendor-specific bugs via the whitelist.
>> *
>> * Don't use AutoIBRS when SNP is enabled because it degrades host
>> * userspace indirect branch performance.
>> */
>> if ((x86_arch_cap_msr & ARCH_CAP_IBRS_ALL) ||
>> (cpu_has(c, X86_FEATURE_AUTOIBRS) &&
>> !cpu_feature_enabled(X86_FEATURE_SEV_SNP))) {
>> setup_force_cpu_cap(X86_FEATURE_IBRS_ENHANCED);
>> if (!cpu_matches(cpu_vuln_whitelist, NO_EIBRS_PBRSB) &&
>> !(x86_arch_cap_msr & ARCH_CAP_PBRSB_NO))
>> setup_force_cpu_bug(X86_BUG_EIBRS_PBRSB);
>> }
>
>You're probably not seeing X86_FEATURE_IBRS_ENHANCED because it doesn't
>get forced under SNP.
Set the Intel flag somewhere in the SNP init path...?
--
Small device. Typos and formatting crap
^ permalink raw reply
* Re: [PATCH] KVM: TDX: Set SIGNIFCANT_INDEX flag for supported CPUIDs
From: Changyuan Lyu @ 2026-02-24 21:29 UTC (permalink / raw)
To: rick.p.edgecombe
Cc: binbin.wu, bp, changyuanl, dave.hansen, hpa, isaku.yamahata, kas,
kvm, linux-coco, linux-kernel, mingo, pbonzini, seanjc, tglx, x86,
xiaoyao.li
In-Reply-To: <213d614fe73e183a230c8f4e0c8fa1cc3d45df39.camel@intel.com>
Hi Rick!
On Tue, 24 Feb 2026 01:57:46 +0000, Edgecombe, Rick P wrote:
> On Mon, 2026-02-23 at 13:43 -0800, Changyuan Lyu wrote:
> > [...]
> > diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> > index 2d7a4d52ccfb4..0c524f9a94a6c 100644
> > --- a/arch/x86/kvm/vmx/tdx.c
> > +++ b/arch/x86/kvm/vmx/tdx.c
> > @@ -172,9 +172,15 @@ static void td_init_cpuid_entry2(struct
> > kvm_cpuid_entry2 *entry, unsigned char i
> > entry->ecx = (u32)td_conf->cpuid_config_values[idx][1];
> > entry->edx = td_conf->cpuid_config_values[idx][1] >> 32;
> >
> > - if (entry->index == KVM_TDX_CPUID_NO_SUBLEAF)
> > + if (entry->index == KVM_TDX_CPUID_NO_SUBLEAF) {
> > entry->index = 0;
> > + entry->flags &= ~KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
>
> There are two callers of this. One is already zeroed, and the other has
> stack garbage in flags. But that second caller doesn't look at the
> flags so it is harmless. Maybe it would be simpler and clearer to just
> zero init the entry struct in that caller. Then you don't need to clear
> it here. Or alternatively set flags to zero above, and then add
> KVM_CPUID_FLAG_SIGNIFCANT_INDEX if needed. Rather than manipulating a
> single bit in a field of garbage, which seems weird.
Thanks for the suggestion. I agree that initializing entry->flags to 0 at
the start of td_init_cpuid_entry2() is much cleaner.
> > + } else {
> > + entry->flags |= KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
> > + }
> >
> > + WARN_ON_ONCE(cpuid_function_is_indexed(entry->function) !=
> > + !!(entry->flags &
> > KVM_CPUID_FLAG_SIGNIFCANT_INDEX));
>
> It warns on leaf 0x23 for me. Is it intentional?
Leaf 0x23 is not in the list of cpuid_function_is_indexed.
Thanks Binbin for the explanation!
> This warning kind of begs the question of how how much consistency
> there should be between KVM_TDX_CAPABILITIES and
> KVM_GET_SUPPORTED_CPUID. There was quite a bit of debate on this and in
> the end we moved forward with a solution that did the bare minimum
> consistency checking.
>
> We actually have been looking at some potential TDX module changes to
> fix the deficiencies from not enforcing the consistency. But didn't
> consider this pattern. Can you explain more about the failure mode?
The main purpose of this patch was to make the KVM_TDX_GET_CPUID API
more intuitive from userspace VMM's perspective.
Since both KVM_TDX_CAPABILITIES and KVM_GET_SUPPORTED_CPUID return
struct kvm_cpuid_entry2, I expected the semantic of the flag in both APIs
to be the same, as I didn't find any special notes to the contrary in the
TDX documentation Documentation/virt/kvm/x86/intel-tdx.rst .
> > /*
> > * The TDX module doesn't allow configuring the guest phys
> > addr bits
> > * (EAX[23:16]). However, KVM uses it as an interface to
> > the userspace
> > --
Regarding the WARN_ON_ONCE, I understand it touches on the larger
consistency and compatibility questions that require more discussion
as you and Sean mentioned. Since I am new to TDX and lack the full context
on those prior debates, I removed the WARN_ON_ONCE check and focus only on
the KVM_CPUID_FLAG_SIGNIFCANT_INDEX consistency fix, which was the core of
this patch.
Best,
Changyuan
-----------------------
From 18b967b718911c09872c3717d8ab083fa59c4a70 Mon Sep 17 00:00:00 2001
From: Changyuan Lyu <changyuanl@google.com>
Date: Fri, 20 Feb 2026 09:55:28 -0800
Subject: [PATCH] KVM: TDX: Set SIGNIFCANT_INDEX flag for supported CPUIDs
Set the KVM_CPUID_FLAG_SIGNIFCANT_INDEX flag in the kvm_cpuid_entry2
structures returned by KVM_TDX_CAPABILITIES if the CPUID is indexed.
This ensures consistency with the CPUID entries returned by
KVM_GET_SUPPORTED_CPUID.
Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Changyuan Lyu <changyuanl@google.com>
---
arch/x86/kvm/vmx/tdx.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 2d7a4d52ccfb4..1c039eab2f3d8 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -167,6 +167,7 @@ static void td_init_cpuid_entry2(struct kvm_cpuid_entry2 *entry, unsigned char i
entry->function = (u32)td_conf->cpuid_config_leaves[idx];
entry->index = td_conf->cpuid_config_leaves[idx] >> 32;
+ entry->flags = 0;
entry->eax = (u32)td_conf->cpuid_config_values[idx][0];
entry->ebx = td_conf->cpuid_config_values[idx][0] >> 32;
entry->ecx = (u32)td_conf->cpuid_config_values[idx][1];
@@ -174,6 +175,9 @@ static void td_init_cpuid_entry2(struct kvm_cpuid_entry2 *entry, unsigned char i
if (entry->index == KVM_TDX_CPUID_NO_SUBLEAF)
entry->index = 0;
+ else
+ entry->flags |= KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
+
/*
* The TDX module doesn't allow configuring the guest phys addr bits
--
2.53.0.414.gf7e9f6c205-goog
^ permalink raw reply related
* SVSM Development Call February 25, 2026
From: Jörg Rödel @ 2026-02-24 21:02 UTC (permalink / raw)
To: coconut-svsm, linux-coco
Hi,
Here is the call for agenda items for this weeks SVSM development call. Please
send any agenda items you have in mind as a reply to this email or raise them
in the meeting.
We will use the LF Zoom instance. Details of the meeting can be found in our
governance repository at:
https://github.com/coconut-svsm/governance
The link to the COCONUT-SVSM calendar is:
https://zoom-lfx.platform.linuxfoundation.org/meetings/coconut-svsm?view=week
The meeting will be recorded and the recording eventually published.
Regards,
Jörg
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox