* Re: [PATCH v2 03/19] device core: Introduce confidential device acceptance
From: Jason Gunthorpe @ 2026-03-26 19:28 UTC (permalink / raw)
To: Dan Williams
Cc: Greg KH, linux-coco, linux-pci, aik, aneesh.kumar, yilun.xu,
bhelgaas, alistair23, lukas, Christoph Hellwig, Marek Szyprowski,
Robin Murphy, Roman Kisel, Samuel Ortiz, Rafael J. Wysocki,
Danilo Krummrich
In-Reply-To: <69c57b745af0f_7ee31003@dwillia2-mobl4.notmuch>
On Thu, Mar 26, 2026 at 11:31:16AM -0700, Dan Williams wrote:
> Jason Gunthorpe wrote:
> [..]
> > > I assume this would also expect that encrypted MMIO mappings are also
> > > not established while trust is less than "TCB"? That would require some
> > > additional enabling to catch attempts to establish an encrypted mapping
> > > that the hardware is prepared for, but dev->trust is not, all without
> > > needing to modify the driver to worry about this difference. Drivers
> > > would just see ioremap() failure in this case.
> >
> > Hmm.. I don't know if this matters. Once we decide to use the device
> > the MMIO should be mapped in the correct way, whatever that is.
> >
> > If we decide to eventually allow a lower trust while T=1 then that
> > should be taken to mean the user wants all the features protecting the
> > communication channel but also all the IOMMU features restricting what
> > memory the device can access.
>
> The question is whether any part of the kernel would ever track that
> secrets in MMIO writes should not be written to TCB-external devices...
> but that is probably a "trust=0" situation. "trust=1" means "be careful
> what you send to this device whether the transport is protected or not".
Right, the kernel has no idea what is secret or not, it is up to
userspace not to give the device secrets if it doesn't fully trust it.
Jason
^ permalink raw reply
* Re: [PATCH v2 02/16] fs, x86/resctrl: Add architecture routines for kernel mode initialization
From: Babu Moger @ 2026-03-26 19:10 UTC (permalink / raw)
To: Reinette Chatre, corbet, tony.luck, Dave.Martin, james.morse,
tglx, mingo, bp, dave.hansen
Cc: skhan, x86, hpa, peterz, juri.lelli, vincent.guittot,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid, kas,
rick.p.edgecombe, akpm, pmladek, rdunlap, dapeng1.mi, kees, elver,
paulmck, lirongqing, safinaskar, fvdl, seanjc, pawan.kumar.gupta,
xin, tiala, Neeraj.Upadhyay, chang.seok.bae, thomas.lendacky,
elena.reshetova, linux-doc, linux-kernel, linux-coco, kvm,
eranian, peternewman
In-Reply-To: <3ef56c9c-cfe4-4e3c-8598-f2217e538c8c@intel.com>
Hi Reinette,
On 3/24/26 17:53, Reinette Chatre wrote:
> Hi Babu,
>
> On 3/12/26 1:36 PM, Babu Moger wrote:
>> Implement the resctrl kernel mode (kmode) arch initialization.
>>
>> - Add resctrl_arch_get_kmode_cfg() to fill the default kernel mode
>> (INHERIT_CTRL_AND_MON). This can be extended later (e.g. for PLZA) to set
>> additional modes.
> I do not think this is something that the architecture should set, at least
> at this time. Every mode has different requirements and this just lets the arch set
> it without any support for what configurations it implies. For example, if
> arch sets a different default mode than INHERIT_CTRL_AND_MON then PQR_PLZA_ASSOC
> needs to be programmed as the CPUs come online and this does not seem to
> accommodate this. This implementation appears to have significant assumptions on
> what architecture will end up setting since it is only considering PLZA.
Sure. Let the arch report what is supported. Will change it to set the
default in fs code.
Users can change change modes from FS code.
>
>> - Add global resctrl_kcfg and resctrl_kmode_init() to initialize default
>> values.
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>> ---
>> v2: New patch to handle PLZA interfaces with /sys/fs/resctrl/info/ directory.
>> https://lore.kernel.org/lkml/2ab556af-095b-422b-9396-f845c6fd0342@intel.com/
>> ---
>> arch/x86/kernel/cpu/resctrl/core.c | 7 +++++++
>> fs/resctrl/rdtgroup.c | 10 ++++++++++
>> 2 files changed, 17 insertions(+)
>>
>> diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
>> index 7667cf7c4e94..4c3ab2d93909 100644
>> --- a/arch/x86/kernel/cpu/resctrl/core.c
>> +++ b/arch/x86/kernel/cpu/resctrl/core.c
>> @@ -892,6 +892,13 @@ bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt)
>> }
>> }
>>
>> +void resctrl_arch_get_kmode_cfg(struct resctrl_kmode_cfg *kcfg)
>> +{
>> + kcfg->kmode = INHERIT_CTRL_AND_MON;
>> + kcfg->kmode_cur = INHERIT_CTRL_AND_MON;
>> + kcfg->k_rdtgrp = NULL;
>> +}
> I already commented on the arch vs filesystem settings.
>
> When using an arch helper this forces all architectures to support this helper. Is a
> helper required? Is it perhaps possible for arch to set a property instead? For example,
> how enumeration is handled?
> I think the assumption here is that INHERIT_CTRL_AND_MON is the default and expected to
> be supported by all architectures. I do not see why arch should set this as default but
> instead this should be from resctrl fs. At the same time it is expected that the
> architecture supports this mode so there needs to be a failure if an architecture does
> not support this mode?
I will change. Arch sets the supported modes. FS sets the default.
Users can change it to required mode later.
>
> I'm going to stop here. I think the comments so far may result in major changes already
> making further detailed review of patches unnecessary.
Based on my comments below you may need to re-look at the some of the
patches.
https://lore.kernel.org/lkml/47c0db32-d0e0-4c53-90bd-b74863d233dc@amd.com/
I am fine otherwise also. Let continue that discussion.
Thanks
Babu
>
> Reinette
>
^ permalink raw reply
* Re: [PATCH v2 01/16] fs/resctrl: Add kernel mode (kmode) data structures and arch hook
From: Babu Moger @ 2026-03-26 18:41 UTC (permalink / raw)
To: Reinette Chatre, corbet, tony.luck, Dave.Martin, james.morse,
tglx, mingo, bp, dave.hansen
Cc: skhan, x86, hpa, peterz, juri.lelli, vincent.guittot,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid, kas,
rick.p.edgecombe, akpm, pmladek, rdunlap, dapeng1.mi, kees, elver,
paulmck, lirongqing, safinaskar, fvdl, seanjc, pawan.kumar.gupta,
xin, tiala, Neeraj.Upadhyay, chang.seok.bae, thomas.lendacky,
elena.reshetova, linux-doc, linux-kernel, linux-coco, kvm,
eranian, peternewman
In-Reply-To: <57c72d52-e62a-44f6-a08a-891a354058e5@intel.com>
Hi Reinette,
On 3/24/26 17:51, Reinette Chatre wrote:
> Hi Babu,
>
> On 3/12/26 1:36 PM, Babu Moger wrote:
>> Add resctrl_kmode, resctrl_kmode_cfg, kernel mode bit defines, and
>> resctrl_arch_get_kmode_cfg() for resctrl kernel mode (e.g. PLZA) support.
> We should not have to start every series from scratch.
> Documentation/process/maintainer-tip.rst. Always.
Sure. Yea. I did not focus on that aspect of patch submission in this
series. Will do next revision.
>
>> ---
>> include/linux/resctrl.h | 10 ++++++++++
>> include/linux/resctrl_types.h | 30 ++++++++++++++++++++++++++++++
>> 2 files changed, 40 insertions(+)
>>
>> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
>> index 006e57fd7ca5..2c36d1ac392f 100644
>> --- a/include/linux/resctrl.h
>> +++ b/include/linux/resctrl.h
>> @@ -699,6 +699,16 @@ int resctrl_arch_io_alloc_enable(struct rdt_resource *r, bool enable);
>> */
>> bool resctrl_arch_get_io_alloc_enabled(struct rdt_resource *r);
>>
>> +/**
>> + * resctrl_arch_get_kmode_cfg() - Get resctrl kernel mode configuration
>> + * @kcfg: Filled with current kernel mode config (kmode, kmode_cur, k_rdtgrp).
>> + *
>> + * Used by the arch (e.g. x86) to report which kernel mode is active and,
>> + * when a global assign mode is in use, which rdtgroup is assigned to
>> + * kernel work.
>> + */
>> +void resctrl_arch_get_kmode_cfg(struct resctrl_kmode_cfg *kcfg);
> This interface does not look right. Would it not be resctrl fs that determines
> which resource group is assigned? This cannot be set by arch. Why does arch decide
> which mode is active? Is this not also resctrl fs? Should arch not just tell
> resctrl fs what it supports?
Yes. Sure. Let the arch tell what is supported. Let fs decide what is
default.
>
>> +
>> extern unsigned int resctrl_rmid_realloc_threshold;
>> extern unsigned int resctrl_rmid_realloc_limit;
>>
>> diff --git a/include/linux/resctrl_types.h b/include/linux/resctrl_types.h
>> index a5f56faa18d2..6b78b08eab29 100644
>> --- a/include/linux/resctrl_types.h
>> +++ b/include/linux/resctrl_types.h
>> @@ -65,7 +65,37 @@ enum resctrl_event_id {
>> QOS_NUM_EVENTS,
>> };
>>
>> +/**
>> + * struct resctrl_kmode - Resctrl kernel mode descriptor
>> + * @name: Human-readable name of the kernel mode.
>> + * @val: Bitmask value for the kernel mode (e.g. INHERIT_CTRL_AND_MON).
>> + */
>> +struct resctrl_kmode {
>> + char name[32];
>> + u32 val;
>> +};
> There is no reason why this needs to be in a central header exposed to archs. Could
> this not be a static within the only function that uses it? Something like
> rdt_mode_str[]?
Yes. I think so.
>
>> +
>> +/**
>> + * struct resctrl_kmode_cfg - Resctrl kernel mode configuration
>> + * @kmode: Requested kernel mode.
>> + * @kmode_cur: Currently active kernel mode.
>> + * @k_rdtgrp: Resource control structure in use, or NULL otherwise.
>> + */
>> +struct resctrl_kmode_cfg {
>> + u32 kmode;
>> + u32 kmode_cur;
>> + struct rdtgroup *k_rdtgrp;
>> +};
>> +
>> #define QOS_NUM_L3_MBM_EVENTS (QOS_L3_MBM_LOCAL_EVENT_ID - QOS_L3_MBM_TOTAL_EVENT_ID + 1)
>> #define MBM_STATE_IDX(evt) ((evt) - QOS_L3_MBM_TOTAL_EVENT_ID)
>>
>> +/* Resctrl kernel mode bits (e.g. for PLZA). */
>> +#define INHERIT_CTRL_AND_MON BIT(0) /* Kernel uses same CLOSID/RMID as user. */
>> +/* One CLOSID for all kernel work; RMID inherited from user. */
>> +#define GLOBAL_ASSIGN_CTRL_INHERIT_MON BIT(1)
>> +/* One resource group (CLOSID+RMID) for all kernel work. */
>> +#define GLOBAL_ASSIGN_CTRL_ASSIGN_MON BIT(2)
>> +#define RESCTRL_KERNEL_MODES_NUM 3
> I think it will make the code much easier to understand if the different modes are described by an
> enum. For example,
Yes. Sure.
>
> enum resctrl_kernel_modes {
> INHERIT_CTRL_AND_MON,
> GLOBAL_ASSIGN_CTRL_INHERIT_MON,
> GLOBAL_ASSIGN_CTRL_ASSIGN_MON,
> RESCTRL_KMODE_LAST = GLOBAL_ASSIGN_CTRL_ASSIGN_MON
> };
> #define RESCTRL_NUM_KERNEL_MODES (RESCTRL_KMODE_LAST + 1)
>
> The supported kernel modes can still be managed as a bitmap with intuitive API using the
> enum that will make the code easier to read. For example, __set_bit(INHERIT_CTRL_AND_MON, ...)
> or BIT(INHERIT_CTRL_AND_MON). The naming is awkward at the moment though, we should improve here.
>
Sure. Yes. We need to think about naming.. Let me think about it.
Thanks
Babu
^ permalink raw reply
* Re: [PATCH v2 03/19] device core: Introduce confidential device acceptance
From: Dan Williams @ 2026-03-26 18:31 UTC (permalink / raw)
To: Jason Gunthorpe, Dan Williams
Cc: Greg KH, linux-coco, linux-pci, aik, aneesh.kumar, yilun.xu,
bhelgaas, alistair23, lukas, Christoph Hellwig, Marek Szyprowski,
Robin Murphy, Roman Kisel, Samuel Ortiz, Rafael J. Wysocki,
Danilo Krummrich
In-Reply-To: <20260326120046.GG67624@nvidia.com>
Jason Gunthorpe wrote:
[..]
> > I assume this would also expect that encrypted MMIO mappings are also
> > not established while trust is less than "TCB"? That would require some
> > additional enabling to catch attempts to establish an encrypted mapping
> > that the hardware is prepared for, but dev->trust is not, all without
> > needing to modify the driver to worry about this difference. Drivers
> > would just see ioremap() failure in this case.
>
> Hmm.. I don't know if this matters. Once we decide to use the device
> the MMIO should be mapped in the correct way, whatever that is.
>
> If we decide to eventually allow a lower trust while T=1 then that
> should be taken to mean the user wants all the features protecting the
> communication channel but also all the IOMMU features restricting what
> memory the device can access.
The question is whether any part of the kernel would ever track that
secrets in MMIO writes should not be written to TCB-external devices...
but that is probably a "trust=0" situation. "trust=1" means "be careful
what you send to this device whether the transport is protected or not".
> Remember there are two parallel things here, one is T=1 which is
> designed to protect against hypervisor and physical attacks, the other
> is the trust level and iommu which would be able to protect against
> attacks from an attested device itself.
>
> Even if you are in a T=1 environment you may still decide you don't
> really trust the device firmware that much and would prefer to have it
> more restricted.
>
> For example, if you have a system with a NVMe drive then all the data
> on the drive is probably still encrypted and has be CPU-decrypted
> before it can be used. It would be reasonable to run in T=1 and attest
> the drive to limit attack surface but also use the IOMMU to limit NVMe
> access to only the memory used to bounce to the CPU decryption as an
> additional fortification.
>
> This is why I am tending to prefer that the kernel's view of trust
> level and the physical HW capability are somewhat orthogonal
> things. Even if the HW has high security the user may still prefer
> that the kernel distrust.
Sounds workable to me.
^ permalink raw reply
* Re: [PATCH 2/2] x86/tdx: Accept hotplugged memory before online
From: Paolo Bonzini @ 2026-03-26 18:25 UTC (permalink / raw)
To: Edgecombe, Rick P
Cc: Marc-André Lureau, Borislav Petkov, kas, Qiang, Chenyi,
Anvin, H. Peter, Ingo Molnar, Kernel Mailing List, Linux,
Dave Hansen, Thomas Gleixner, kvm, linux-coco,
the arch/x86 maintainers
In-Reply-To: <7802b50589c80a7276a50cc473c1aa579750a30c.camel@intel.com>
Il mer 25 mar 2026, 18:21 Edgecombe, Rick P
<rick.p.edgecombe@intel.com> ha scritto:
>
> Ah, I see now! So the problem is not that the kernel is accidentally
> re-accepting the memory. It's that host userspace is not actually
> removing the memory during unplug. Hmm. Why not fix userspace then? If
> the memory is unplugged it should not be usable anymore by the guest.
> If it is still accessible then it seems kind of like a bug, no?
>
> And! This totally justifies the warning. If the error is ignored, the
> guest would think the memory is zeroed, but it could have old data in
> it. It's exactly the kind of tricks a VMM could play to attack the
> guest.
>
> Another option could be to perform a TDG.MEM.PAGE.RELEASE TDCALL from
> the guest when it unplugs the memory, to put it in an unaccepted state.
> This would be more robust to buggy VMM behavior. But working around
> buggy VM behavior would need a high bar.
Wouldn't it actually be a very low bar? Just from these two paragraphs
of yours, it's clear that the line between buggy and malicious is
fine, in fact I think userspace should not care at all about removing
the memory. Only the guest cares about acceptance state.
Doing a RELEASE TDCALL seems more robust and not hard.
Paolo
^ permalink raw reply
* Re: [PATCH v2 00/16] fs,x86/resctrl: Add kernel-mode (e.g., PLZA) support to the resctrl subsystem
From: Babu Moger @ 2026-03-26 17:12 UTC (permalink / raw)
To: Reinette Chatre, corbet, tony.luck, Dave.Martin, james.morse,
tglx, mingo, bp, dave.hansen
Cc: skhan, x86, hpa, peterz, juri.lelli, vincent.guittot,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid, kas,
rick.p.edgecombe, akpm, pmladek, rdunlap, dapeng1.mi, kees, elver,
paulmck, lirongqing, safinaskar, fvdl, seanjc, pawan.kumar.gupta,
xin, tiala, Neeraj.Upadhyay, chang.seok.bae, thomas.lendacky,
elena.reshetova, linux-doc, linux-kernel, linux-coco, kvm,
eranian, peternewman
In-Reply-To: <14a8ad0a-e842-4268-871a-0762f1169e03@intel.com>
Hi Reinette,
Thanks for the review comments. Will address one by one.
On 3/24/26 17:51, Reinette Chatre wrote:
> Hi Babu,
>
> On 3/12/26 1:36 PM, Babu Moger wrote:
>> This series adds support for Privilege-Level Zero Association (PLZA) to the
>> resctrl subsystem. PLZA is an AMD feature that allows specifying a CLOSID
>> and/or RMID for execution in kernel mode (privilege level zero), so that
>> kernel work is not subject to the same resource constrains as the current
>> user-space task. This avoids kernel operations being aggressively throttled
>> when a task's memory bandwidth is heavily limited.
>>
>> The feature documentation is not yet publicly available, but it is expected
>> to be released in the next few weeks. In the meantime, a brief description
>> of the features is provided below.
>>
>> Privilege Level Zero Association (PLZA)
>>
>> Privilege Level Zero Association (PLZA) allows the hardware to
>> automatically associate execution in Privilege Level Zero (CPL=0) with a
>> specific COS (Class of Service) and/or RMID (Resource Monitoring
>> Identifier). The QoS feature set already has a mechanism to associate
>> execution on each logical processor with an RMID or COS. PLZA allows the
>> system to override this per-thread association for a thread that is
>> executing with CPL=0.
>> ------------------------------------------------------------------------
>>
>> The series introduces the feature in a way that supports the interface in
>> a generic manner to accomodate MPAM or other vendor specific implimentation.
>>
>> Below is the detailed requirements provided by Reinette:
>> https://lore.kernel.org/lkml/2ab556af-095b-422b-9396-f845c6fd0342@intel.com/
> Our discussion considered how resctrl could support PLZA in a generic way while
> also preparing to support MPAM's variants and how PLZA may evolve to have similar
> capabilities when considering the capabilities of its registers.
>
> This does not mean that your work needs to implement everything that was discussed.
> Instead, this work is expected to just support what PLZA is capable of today but
> do so in a way that the future enhancements could be added to.
>
> This series is quite difficult to follow since it appears to implement a full
> featured generic interface while PLZA cannot take advantage of it.
>
> Could you please simplify this work to focus on just enabling PLZA and only
> add interfaces needed to do so?
Sure. Will try. Lets continue the discussion.
>
>> Summary:
>> 1. Kernel-mode/PLZA controls and status should be exposed under the resctrl
>> info directory:/sys/fs/resctrl/info/, not as a separate or arch-specific path.
>>
>> 2. Add two info files
>>
>> a. kernel_mode
>> Purpose: Control how resource allocation and monitoring apply in kernel mode
>> (e.g. inherit from task vs global assign).
>>
>> Read: List supported modes and show current one (e.g. with [brackets]).
>> Write: Set current mode by name (e.g. inherit_ctrl_and_mon, global_assign_ctrl_assign_mon).
>>
>> b. kernel_mode_assignment
>>
>> Purpose: When a “global assign” kernel mode is active, specify which resctrl group
>> (CLOSID/RMID) is used for kernel work.
>>
>> Read: Show the assigned group in a path-like form (e.g. //, ctrl1//, ctrl1/mon1/).
>> Write: Assign or clear the group used for kernel mode (and optionally clear with an empty write).
>>
>> The patches are based on top of commit (v7.0.0-rc3)
>> 839e91ce3f41b (tip/master) Merge branch into tip/master: 'x86/tdx'
>> ------------------------------------------------------------------------
>>
>> Examples: kernel_mode and kernel_mode_assignment
>>
>> All paths below are under /sys/fs/resctrl/ (e.g. info/kernel_mode means
>> /sys/fs/resctrl/info/kernel_mode). Resctrl must be mounted and the platform
>> must support the relevant modes (e.g. AMD with PLZA).
>>
>> 1) kernel_mode — show and set the current kernel mode
>>
>> Read supported modes and which one is active (current in brackets):
>>
>> $ cat info/kernel_mode
>> [inherit_ctrl_and_mon]
>> global_assign_ctrl_inherit_mon
>> global_assign_ctrl_assign_mon
>>
>> Set the active mode (e.g. use one CLOSID+RMID for all kernel work):
>>
>> $ echo "global_assign_ctrl_assign_mon" > info/kernel_mode
>> $ cat info/kernel_mode
>> inherit_ctrl_and_mon
>> global_assign_ctrl_inherit_mon
>> [global_assign_ctrl_assign_mon]
>>
>> Mode meanings:
>> - inherit_ctrl_and_mon: kernel uses same CLOSID/RMID as the current task (default).
>> - global_assign_ctrl_inherit_mon: one CLOSID for all kernel work; RMID inherited from user.
>> - global_assign_ctrl_assign_mon: one resource group (CLOSID+RMID) for all kernel work.
>>
>> 2) kernel_mode_assignment — show and set which group is used for kernel work
>>
>> Only relevant when kernel_mode is not "inherit_ctrl_and_mon". Read the
> To help with future usages please connect visibility of this file with the mode in
> info/kernel_mode. This helps us to support future modes with other resctrl files, possible
> within each resource group.
> Specifically, kernel_mode_assignment is not visible to user space if mode is "inherit_ctrl_and_mon",
> while it is visible when mode is global_assign_ctrl_inherit_mon or global_assign_ctrl_assign_mon.
Sure. Will do.
>
>> currently assigned group (path format is "CTRL_MON/MON/"):
> The format depends on the mode, right? If the mode is "global_assign_ctrl_inherit_mon"
> then it should only contain a control group, alternatively, if the mode is
> "global_assign_ctrl_assign_mon" then it contains control and mon group. This gives
> resctrl future flexibility to change format for future modes.
This can be done both ways. Whole purpose of these groups is to get
CLOSID and RMID to enable PLZA. User can echo CTRL_MON or MON group to
kernel_mode_assignment in any of the modes. We can decide what needs to
be updated in MSR (PQR_PLZA_ASSOC) based on what kernel mode is selected.
>
> We should also consider the scenario when it is a "monitoring only" system, which can
> happen independent from what hardware actually supports, for example, if user boots
> with "rdt=!l3cat,!l2cat,!mba,!smba". In this case I assume CLOS should just always be
> zero and thus only "default control group" is accepted?
Yes. It depends on how we want to implement like we mentioned above.
>
>> $ cat info/kernel_mode_assignment
>> //
>>
>> "//" means the default CTRL_MON group is assigned. Assign a specific
>> group instead (e.g. a CTRL_MON group "ctrl1", or a MON group "mon1" under it):
>>
>> $ echo "ctrl1//" > info/kernel_mode_assignment
>> $ cat info/kernel_mode_assignment
>> ctrl1//
>>
>> $ echo "ctrl1/mon1/" > info/kernel_mode_assignment
>> $ cat info/kernel_mode_assignment
>> ctrl1/mon1/
>>
>> Clear the assignment (no dedicated group for kernel work):
>>
>> $ echo >> info/kernel_mode_assignment
>> $ cat info/kernel_mode_assignment
>> Kmode is not configured
> This does not look right. Would this not create a conflict between info/kernel_mode
> and info/kernel_mode_assignment about what the current mode is? The way I see it
> info/kernel_mode_assignment must always contain a valid group.
Yes. We can do that.
>
>> Errors (e.g. invalid group name or unsupported mode) are reported in
>> info/last_cmd_status.
>>
>> ---
>>
>> v2:
>> This is similar to RFC with new proposal. Names of the some interfaces
>> are not final. Lets fix that later as we move forward.
>>
>> Separated the two features: Global Bandwidth Enforcement (GLBE) and
>> Privilege Level Zero Association (PLZA).
>>
>> This series only adds support for PLZA.
>>
>> Used the name of the feature as kmode instead of PLZA. That can be changed as well.
>>
>> Tony suggested using global variables to store the kernel mode
>> CLOSID and RMID. However, the kernel mode CLOSID and RMID are
>> coming from rdtgroup structure with the new interface. Accessing
>> them requires holding the associated lock, which would make the
>> context switch path unnecessarily expensive. So, dropped the idea.
>> https://lore.kernel.org/lkml/aXuxVSbk1GR2ttzF@agluck-desk3/
>> Let me know if there are other ways to optimize this.
> I do not see why the context switch path needs to be touched at all with this
> implementation. Since PLZA only supports global assignment does it not mean that resctrl
> only needs to update PQR_PLZA_ASSOC when user writes to info/kernel_mode and
> info/kernel_mode_assignment?
Each thread has an MSR to configure whether to associate privilege level
zero execution with a separate COS and/or RMID, and the value of the COS
and/or RMID. PLZA may be enabled or disabled on a per-thread
basis. However, the COS and RMID association and configuration must be
the same for all threads in the QOS Domain.
So, PQR_PLZA_ASSOC is a per thread MSR just like PQR_ASSOC.
Privilege-Level Zero Association (PLZA) allows the user to specify a COS
and/or RMID associated with execution in Privilege-Level Zero. When
enabled on a HW thread, when that thread enters Privilige-Level Zero,
transactions associated with that thread will be associated with the
PLZA COS and/or RMID. Otherwise, the HW thread will be associated with
the COS and RMID identified by PQR_ASSOC.
More below.
>
> Consider some of the scenarios:
>
> resctrl mount with default state:
>
> # cat info/kernel_mode
> [inherit_ctrl_and_mon]
> global_assign_ctrl_inherit_mon
> global_assign_ctrl_assign_mon
> # ls info/kernel_mode_assignment
> ls: cannot access 'info/kernel_mode_assignment': No such file or directory
>
> enable global_assign_ctrl_assign_mon mode:
> # echo "global_assign_ctrl_assign_mon" > info/kernel_mode
>
> Expectation here is that when user space sets this mode as above then resctrl would
> in turn program MSR_IA32_PQR_PLZA_ASSOC on all CPUs to be:
> MSR_IA32_PQR_PLZA_ASSOC.rmid=0
> MSR_IA32_PQR_PLZA_ASSOC.rmid_en=1
> MSR_IA32_PQR_PLZA_ASSOC.closid=0
> MSR_IA32_PQR_PLZA_ASSOC.closid_en=1
> MSR_IA32_PQR_PLZA_ASSOC.plza_en=1
>
> I do not see why it is necessary to maintain any per-CPU or per-task state or needing
> to touch the context switch code. Since PLZA only supports global could it not
> just set MSR_IA32_PQR_PLZA_ASSOC on all online CPUs and be done with it?
> Only caveat is that if a CPU is offline then this setting needs to be stashed
> so that MSR_IA32_PQR_PLZA_ASSOC can be set when new CPU comes online.
>
> The way that rdtgroup_config_kmode() introduced in patch #11 assumes it is dealing
> with RDT_RESOURCE_L3 and traverses the resource domain list and resource group
> CPU mask seems unnecessary to me as well as error prone since the system may only
> have, for example, RDT_RESOURCE_MBA enabled or even just monitoring. Why not just set
> MSR_IA32_PQR_PLZA_ASSOC on all CPUs and be done?
>
> To continue the scenarios ...
>
> After user's setting above related files read:
> # cat info/kernel_mode
> inherit_ctrl_and_mon
> global_assign_ctrl_inherit_mon
> [global_assign_ctrl_assign_mon]
> # cat info/kernel_mode_assignment
> //
>
> Modify group used by global_assign_ctrl_assign_mon mode:
> # echo 'ctrl1/mon1/' > info/kernel_mode_assignment
>
> Expectation here is that when user space sets this then resctrl would
> program MSR_IA32_PQR_PLZA_ASSOC on all CPUs to be:
> MSR_IA32_PQR_PLZA_ASSOC.rmid=<rmid of mon1>
> MSR_IA32_PQR_PLZA_ASSOC.rmid_en=1
> MSR_IA32_PQR_PLZA_ASSOC.closid=<closid of ctrl1>
> MSR_IA32_PQR_PLZA_ASSOC.closid_en=1
> MSR_IA32_PQR_PLZA_ASSOC.plza_en=1
This works correctly when PLZA associations are defined by per CPU. For
example, lets assume that *ctrl1* is assigned *CLOSID 1*.
In this scenario, every task in the system running on a any CPU will use
the limits associated with *CLOSID 1* whenever it enters Privilege-Level
Zero, because the CPU's *PQR_PLZA_ASSOC* register has PLZA enabled and
CLOSID is 1.
Now consider task-based association:
We have two resctrl groups:
* *ctrl1 -> CLOSID 1 -> task1.plza = 1 : *User wants PLZA be enabled
for this task.
* *ctrl2 -> CLOSID 2 -> task2.plza = 0 : *User wants PLZA
disabled for this task.
Suppose *task1* is first scheduled on *CPU 0*. This behaves as expected:
since CPU 0 's *PQR_PLZA_ASSOC* contains *CLOSID 1, plza_en =1*, task1
will use the limits from CLOSID 1 when it enters Privilege-Level Zero.
However, if *task2* later runs on *CPU 0*, we expect it to use *CLOSID
2* in both user mode and kernel mode, because user has PLZA disabled for
this task. But CPU 0 still has *CLOSID 1, **plza_en =1* in its
PQR_PLZA_ASSOC register.
As a result, task2 will incorrectly run with *CLOSID 1* when entering
Privilege-Level Zero something we explicitly want to avoid.
At that point, PLZA must be disabled on CPU 0 to prevent the unintended
association. Hope this explanation makes the issue clear.
Thanks
Babu
>
> Enable global_assign_ctrl_inherit_mon mode:
> # echo "global_assign_ctrl_inherit_mon" > info/kernel_mode
>
> Expectation here is that when user space sets this mode then resctrl would
> program MSR_IA32_PQR_PLZA_ASSOC on all CPUs to be:
> MSR_IA32_PQR_PLZA_ASSOC.rmid=0
> MSR_IA32_PQR_PLZA_ASSOC.rmid_en=0
> MSR_IA32_PQR_PLZA_ASSOC.closid=0
> MSR_IA32_PQR_PLZA_ASSOC.closid_en=1
> MSR_IA32_PQR_PLZA_ASSOC.plza_en=1
>
> # cat info/kernel_mode
> inherit_ctrl_and_mon
> [global_assign_ctrl_inherit_mon]
> global_assign_ctrl_assign_mon
> # cat info/kernel_mode_assignment <==== returns just a ctrl group
> /
>
> Modify group used by global_assign_ctrl_inherit_mon mode:
> # echo ctrl1 > info/kernel_mode_assignment
>
> Expectation here is that when user space sets this then resctrl would
> program MSR_IA32_PQR_PLZA_ASSOC on all CPUs to be:
> MSR_IA32_PQR_PLZA_ASSOC.rmid=0
> MSR_IA32_PQR_PLZA_ASSOC.rmid_en=0
> MSR_IA32_PQR_PLZA_ASSOC.closid=<closid of ctrl1>
> MSR_IA32_PQR_PLZA_ASSOC.closid_en=1
> MSR_IA32_PQR_PLZA_ASSOC.plza_en=1
>
> # cat info/kernel_mode_assignment <==== returns just a ctrl group
> ctrl/
>
> Enable inherit_ctrl_and_mon mode:
> # echo "inherit_ctrl_and_mon" > info/kernel_mode
>
> Expectation here is that when user space sets this mode then resctrl would
> program MSR_IA32_PQR_PLZA_ASSOC on all CPUs to be:
> MSR_IA32_PQR_PLZA_ASSOC.rmid=0
> MSR_IA32_PQR_PLZA_ASSOC.rmid_en=0
> MSR_IA32_PQR_PLZA_ASSOC.closid=0
> MSR_IA32_PQR_PLZA_ASSOC.closid_en=0
> MSR_IA32_PQR_PLZA_ASSOC.plza_en=0
>
> At this point info/kernel_mode_assignment is not visible anymore:
>
> # ls info/kernel_mode_assignment
> ls: cannot access 'info/kernel_mode_assignment': No such file or directory
>
> >From what I understand above exposes and enables full capability of PLZA. All the other
> per-task and per-cpu handling in this series is not something that PLZA can benefit from.
> If this is not the case, what am I missing? Could this series be simplified to just support
> PLZA today? When next hardware with more capability needs to be supported resctrl could be
> enhanced to support it by using the more accurate information about what the hardware is
> capable of.
>
> We also do not really know what use cases users prefer. This may even be sufficient.
>
> Reinette
>
^ permalink raw reply
* Re: [PATCH v2 03/19] device core: Introduce confidential device acceptance
From: Greg KH @ 2026-03-26 15:00 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Dan Williams, linux-coco, linux-pci, aik, aneesh.kumar, yilun.xu,
bhelgaas, alistair23, lukas, Christoph Hellwig, Marek Szyprowski,
Robin Murphy, Roman Kisel, Samuel Ortiz, Rafael J. Wysocki,
Danilo Krummrich
In-Reply-To: <20260326120046.GG67624@nvidia.com>
On Thu, Mar 26, 2026 at 09:00:46AM -0300, Jason Gunthorpe wrote:
> On Wed, Mar 25, 2026 at 06:27:04PM -0700, Dan Williams wrote:
> > Jason Gunthorpe wrote:
> > [..]
> > > > Right, the potential to see in-between states concerns me because TSM
> > > > uAPIs would have fully enabled the device to wreak havoc, meanwhile
> > > > dev->trust is still showing the device at some lower level of trust. So
> > > > I think trust modification needs to be synchronous with privileges
> > > > granted/revoked.
> > >
> > > If an iommu is present then the device will still be blocked even
> > > though it is in RUN, I'm not sure this synchronicity is so important.
> >
> > Oh, maybe we are just quibbling about where the mechanism lives. The
> > "unblock DMA" step in current preliminary patches is currently behind
> > the "struct pci_tsm_ops::accept()" op which also handles transitioning
> > the device to RUN / T=1. It is a bus callback.
> >
> > However, if the IOMMU layer is enlightened to block/unblock DMA on trust
> > setting then the TDISP "unblock DMA" step can be factored out of this bus
> > callback and into the IOMMU trust responder.
>
> Yes, I would prefer this because it makes the whole IOMMU mechanism
> entirely general and not tied to TDISP - which I think is sort of what
> Greg is pushing on too.
That is what I am going to _require_ here :)
> > I assume this would also expect that encrypted MMIO mappings are also
> > not established while trust is less than "TCB"? That would require some
> > additional enabling to catch attempts to establish an encrypted mapping
> > that the hardware is prepared for, but dev->trust is not, all without
> > needing to modify the driver to worry about this difference. Drivers
> > would just see ioremap() failure in this case.
>
> Hmm.. I don't know if this matters. Once we decide to use the device
> the MMIO should be mapped in the correct way, whatever that is.
>
> If we decide to eventually allow a lower trust while T=1 then that
> should be taken to mean the user wants all the features protecting the
> communication channel but also all the IOMMU features restricting what
> memory the device can access.
>
> Remember there are two parallel things here, one is T=1 which is
> designed to protect against hypervisor and physical attacks, the other
> is the trust level and iommu which would be able to protect against
> attacks from an attested device itself.
>
> Even if you are in a T=1 environment you may still decide you don't
> really trust the device firmware that much and would prefer to have it
> more restricted.
>
> For example, if you have a system with a NVMe drive then all the data
> on the drive is probably still encrypted and has be CPU-decrypted
> before it can be used. It would be reasonable to run in T=1 and attest
> the drive to limit attack surface but also use the IOMMU to limit NVMe
> access to only the memory used to bounce to the CPU decryption as an
> additional fortification.
>
> This is why I am tending to prefer that the kernel's view of trust
> level and the physical HW capability are somewhat orthogonal
> things. Even if the HW has high security the user may still prefer
> that the kernel distrust.
I agree, that's a good way of putting this.
greg k-h
^ permalink raw reply
* Re: [PATCH v6 16/22] x86/virt/tdx: Update tdx_sysinfo and check features post-update
From: Chao Gao @ 2026-03-26 13:03 UTC (permalink / raw)
To: linux-kernel, linux-coco, kvm
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260326084448.29947-17-chao.gao@intel.com>
>+int tdx_module_post_update(struct tdx_sys_info *info)
>+{
>+ struct tdx_sys_info_version *old, *new;
>+ int ret;
>+
>+ /* Shouldn't fail as the update has succeeded. */
>+ ret = get_tdx_sys_info(info);
>+ if (WARN_ONCE(ret, "version retrieval failed after update, replace the TDX module\n"))
>+ return ret;
>+
>+ old = &tdx_sysinfo.version;
>+ new = &info->version;
>+ pr_info("version %u.%u.%02u -> %u.%u.%02u\n", old->major_version,
>+ old->minor_version,
>+ old->update_version,
>+ new->major_version,
>+ new->minor_version,
>+ new->update_version);
>+
>+ /*
>+ * Blindly refreshing the entire tdx_sysinfo could disrupt running
>+ * software, as it may subtly rely on the previous state unless
>+ * proven otherwise.
>+ *
>+ * Only refresh version information (including handoff version)
>+ * that does not affect functionality, and ignore all other
>+ * changes.
>+ */
>+ tdx_sysinfo.version = info->version;
>+ tdx_sysinfo.handoff = info->handoff;
Sashiko commented:
"""
Because stop_machine() has already completed in seamldr_install_module(),
other CPUs will have resumed execution by the time this is called.
Since tdx_sysinfo.version and tdx_sysinfo.handoff are multi-byte structures
and are updated here without holding a lock, could concurrent readers observe
torn reads if they access these fields simultaneously?
"""
This is valid. tdx_sysinfo.handoff has no concurrent readers. so, no fix is
needed.
tdx_sysinfo.version may be read by userspace via sysfs. However, major/minor
versions don't change across updates, so only update_version needs
READ/WRITE_ONCE() to prevent torn reads. I will apply this fix:
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 432d80b21ef0..0e7668bf20a1 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -1276,7 +1276,7 @@ int tdx_module_post_update(struct tdx_sys_info *info)
* that does not affect functionality, and ignore all other
* changes.
*/
- tdx_sysinfo.version = info->version;
+ WRITE_ONCE(tdx_sysinfo.version.update_version, info->version.update_version);
tdx_sysinfo.handoff = info->handoff;
if (!memcmp(&tdx_sysinfo, info, sizeof(*info)))
diff --git a/drivers/virt/coco/tdx-host/tdx-host.c b/drivers/virt/coco/tdx-host/tdx-host.c
index d4a552853021..43a55666145c 100644
--- a/drivers/virt/coco/tdx-host/tdx-host.c
+++ b/drivers/virt/coco/tdx-host/tdx-host.c
@@ -40,7 +40,7 @@ static ssize_t version_show(struct device *dev, struct device_attribute *attr,
return sysfs_emit(buf, TDX_VERSION_FMT"\n", ver->major_version,
ver->minor_version,
- ver->update_version);
+ READ_ONCE(ver->update_version));
}
static DEVICE_ATTR_RO(version);
>+
>+ if (!memcmp(&tdx_sysinfo, info, sizeof(*info)))
>+ return 0;
>+
>+ pr_info("TDX module features have changed after updates, but might not take effect.\n");
>+ pr_info("Please consider updating your BIOS to install the TDX module.\n");
>+ return 0;
>+}
>+
> static bool is_pamt_page(unsigned long phys)
> {
> struct tdmr_info_list *tdmr_list = &tdx_tdmr_list;
>diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
>index c62874b87d7a..f8686247c660 100644
>--- a/arch/x86/virt/vmx/tdx/tdx.h
>+++ b/arch/x86/virt/vmx/tdx/tdx.h
>@@ -4,6 +4,8 @@
>
> #include <linux/bits.h>
>
>+#include <asm/tdx_global_metadata.h>
>+
> /*
> * This file contains both macros and data structures defined by the TDX
> * architecture and Linux defined software data structures and functions.
>@@ -122,5 +124,6 @@ struct tdmr_info_list {
>
> int tdx_module_shutdown(void);
> int tdx_module_run_update(void);
>+int tdx_module_post_update(struct tdx_sys_info *info);
>
> #endif
>--
>2.47.3
>
^ permalink raw reply related
* Re: [PATCH v6 12/22] x86/virt/tdx: Reset software states during TDX module shutdown
From: Chao Gao @ 2026-03-26 12:35 UTC (permalink / raw)
To: linux-kernel, linux-coco, kvm
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260326084448.29947-13-chao.gao@intel.com>
> int tdx_module_shutdown(void)
> {
> struct tdx_module_args args = {};
>+ int ret, cpu;
>
> /*
> * Shut down the TDX module and prepare handoff data for the next
>@@ -1188,7 +1189,23 @@ int tdx_module_shutdown(void)
> * modules as new modules likely have higher handoff version.
> */
> args.rcx = tdx_sysinfo.handoff.module_hv;
>- return seamcall_prerr(TDH_SYS_SHUTDOWN, &args);
>+ ret = seamcall_prerr(TDH_SYS_SHUTDOWN, &args);
>+ if (ret)
>+ return ret;
>+
>+ tdx_module_status = TDX_MODULE_UNINITIALIZED;
Sashiko commented:
"""
If a TDX module update fails after this shutdown completes, does this leave
the system in an unrecoverable state?
Since tdx_module_status is left as TDX_MODULE_UNINITIALIZED, if KVM is
later reloaded, it appears it will call tdx_enable(), observe the
uninitialized status, and invoke init_tdx_module() again.
Because init_tdx_module() is not re-entrant, won't it blindly append
duplicate memory regions to the global tdx_memlist and allocate new TDMR
arrays and PAMT memory?
This seems like it would permanently leak the previous allocations and
eventually fail when construct_tdmrs() rejects overlapping TDMRs. Is there
a mechanism to prevent re-initialization if the subsequent update steps fail?
"""
This is a valid issue.
A fix is: set tdx_module_status to TDX_MODULE_ERROR here. Failures preserve
ERROR state; success explicitly transitions to INITIALIZED (patch 15).
Alternatively, we could introduce a dedicated shutdown state.
Note that the VMXON series moves TDX initialization to boot time, eliminating
runtime re-initialization (init_tdx_module() calls) entirely.
^ permalink raw reply
* Re: [PATCH v2 03/19] device core: Introduce confidential device acceptance
From: Jason Gunthorpe @ 2026-03-26 12:00 UTC (permalink / raw)
To: Dan Williams
Cc: Greg KH, linux-coco, linux-pci, aik, aneesh.kumar, yilun.xu,
bhelgaas, alistair23, lukas, Christoph Hellwig, Marek Szyprowski,
Robin Murphy, Roman Kisel, Samuel Ortiz, Rafael J. Wysocki,
Danilo Krummrich
In-Reply-To: <69c48b682e6fe_7ee310068@dwillia2-mobl4.notmuch>
On Wed, Mar 25, 2026 at 06:27:04PM -0700, Dan Williams wrote:
> Jason Gunthorpe wrote:
> [..]
> > > Right, the potential to see in-between states concerns me because TSM
> > > uAPIs would have fully enabled the device to wreak havoc, meanwhile
> > > dev->trust is still showing the device at some lower level of trust. So
> > > I think trust modification needs to be synchronous with privileges
> > > granted/revoked.
> >
> > If an iommu is present then the device will still be blocked even
> > though it is in RUN, I'm not sure this synchronicity is so important.
>
> Oh, maybe we are just quibbling about where the mechanism lives. The
> "unblock DMA" step in current preliminary patches is currently behind
> the "struct pci_tsm_ops::accept()" op which also handles transitioning
> the device to RUN / T=1. It is a bus callback.
>
> However, if the IOMMU layer is enlightened to block/unblock DMA on trust
> setting then the TDISP "unblock DMA" step can be factored out of this bus
> callback and into the IOMMU trust responder.
Yes, I would prefer this because it makes the whole IOMMU mechanism
entirely general and not tied to TDISP - which I think is sort of what
Greg is pushing on too.
> I assume this would also expect that encrypted MMIO mappings are also
> not established while trust is less than "TCB"? That would require some
> additional enabling to catch attempts to establish an encrypted mapping
> that the hardware is prepared for, but dev->trust is not, all without
> needing to modify the driver to worry about this difference. Drivers
> would just see ioremap() failure in this case.
Hmm.. I don't know if this matters. Once we decide to use the device
the MMIO should be mapped in the correct way, whatever that is.
If we decide to eventually allow a lower trust while T=1 then that
should be taken to mean the user wants all the features protecting the
communication channel but also all the IOMMU features restricting what
memory the device can access.
Remember there are two parallel things here, one is T=1 which is
designed to protect against hypervisor and physical attacks, the other
is the trust level and iommu which would be able to protect against
attacks from an attested device itself.
Even if you are in a T=1 environment you may still decide you don't
really trust the device firmware that much and would prefer to have it
more restricted.
For example, if you have a system with a NVMe drive then all the data
on the drive is probably still encrypted and has be CPU-decrypted
before it can be used. It would be reasonable to run in T=1 and attest
the drive to limit attack surface but also use the IOMMU to limit NVMe
access to only the memory used to bounce to the CPU decryption as an
additional fortification.
This is why I am tending to prefer that the kernel's view of trust
level and the physical HW capability are somewhat orthogonal
things. Even if the HW has high security the user may still prefer
that the kernel distrust.
Jason
^ permalink raw reply
* Re: [PATCH v6 09/22] x86/virt/seamldr: Introduce skeleton for TDX module updates
From: Chao Gao @ 2026-03-26 11:47 UTC (permalink / raw)
To: linux-kernel, linux-coco, kvm
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260326084448.29947-10-chao.gao@intel.com>
>+static void set_target_state(enum module_update_state state)
>+{
>+ /* Reset ack counter. */
>+ update_data.thread_ack = num_online_cpus();
...
>+static void ack_state(void)
>+{
>+ guard(raw_spinlock)(&update_data.lock);
>+ update_data.thread_ack--;
>+ if (!update_data.thread_ack)
>+ set_target_state(update_data.state + 1);
>+}
>+
>+/*
>+ * See multi_cpu_stop() from where this multi-cpu state-machine was
>+ * adopted, and the rationale for touch_nmi_watchdog().
>+ */
>+static int do_seamldr_install_module(void *seamldr_params)
>+{
>+ enum module_update_state newstate, curstate = MODULE_UPDATE_START;
>+ int ret = 0;
>+
>+ do {
>+ /* Chill out and re-read update_data. */
>+ cpu_relax();
>+ newstate = READ_ONCE(update_data.state);
>+
>+ if (newstate != curstate) {
>+ curstate = newstate;
>+ switch (curstate) {
>+ /* TODO: add the update steps. */
>+ default:
>+ break;
>+ }
>+
>+ ack_state();
>+ } else {
>+ touch_nmi_watchdog();
>+ rcu_momentary_eqs();
>+ }
>+ } while (curstate != MODULE_UPDATE_DONE);
>+
>+ return ret;
>+}
>+
> DEFINE_FREE(free_seamldr_params, struct seamldr_params *,
> if (!IS_ERR_OR_NULL(_T)) free_seamldr_params(_T))
>
>@@ -197,7 +270,7 @@ int seamldr_install_module(const u8 *data, u32 size)
> if (IS_ERR(params))
> return PTR_ERR(params);
>
>- /* TODO: Update TDX module here */
>- return 0;
>+ set_target_state(MODULE_UPDATE_START + 1);
>+ return stop_machine(do_seamldr_install_module, params, cpu_online_mask);
I'm reviewing feedback from sashiko:
https://sashiko.dev/#/patchset/20260326084448.29947-1-chao.gao%40intel.com
It identifies a valid race between CPU hotplug and state machine management. If
a CPU goes offline after set_target_state() but before stop_machine(),
thread_ack never reaches zero, causing all CPUs to spin indefinitely with
interrupts disabled.
The fix is: acquire cpus_read_lock() before set_target_state() and use
stop_machine_cpuslocked(). i.e.,
diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
index ed6a092b11e2..6f9d80a3a76f 100644
--- a/arch/x86/virt/vmx/tdx/seamldr.c
+++ b/arch/x86/virt/vmx/tdx/seamldr.c
@@ -270,7 +270,8 @@ int seamldr_install_module(const u8 *data, u32 size)
if (IS_ERR(params))
return PTR_ERR(params);
+ guard(cpus_read_lock)();
set_target_state(MODULE_UPDATE_START + 1);
- return stop_machine(do_seamldr_install_module, params, cpu_online_mask);
+ return stop_machine_cpuslocked(do_seamldr_install_module, params, cpu_online_mask);
}
EXPORT_SYMBOL_FOR_MODULES(seamldr_install_module, "tdx-host");
^ permalink raw reply related
* Re: [PATCH v13 00/48] arm64: Support for Arm CCA in KVM
From: Suzuki K Poulose @ 2026-03-26 11:22 UTC (permalink / raw)
To: Gavin Shan, Steven Price, Mathieu Poirier
Cc: kvm, kvmarm, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Shanker Donthineni,
Alper Gun, Aneesh Kumar K . V, Emi Kisanuki, Vishal Annapurve
In-Reply-To: <807b844c-32f6-4094-8c24-15d7eb1d3638@redhat.com>
Hi Gavin,
On 26/03/2026 00:48, Gavin Shan wrote:
> Hi Suzuki,
>
> On 3/25/26 8:16 PM, Suzuki K Poulose wrote:
>> On 25/03/2026 06:37, Gavin Shan wrote:
>>> On 3/21/26 2:45 AM, Steven Price wrote:
>
> [...]
>
>>>
>>> In upstream TF-A repository [1], I don't see the config option
>>> 'RMM_V1_COMPAT'.
>>> would it be something else?
>>>
>>> [1] git@github.com:ARM-software/arm-trusted-firmware.git (branch:
>>> master)
>>>
>>
>> suzuki@ewhatever:trusted-firmware-a$ git grep RMM_V1_COMPAT
>> Makefile: RMM_V1_COMPAT \
>> Makefile: RMM_V1_COMPAT \
>> docs/getting_started/build-options.rst:- ``RMM_V1_COMPAT``: Boolean
>> flag to enable support for RMM v1.x compatibility
>> include/services/rmmd_svc.h:#if RMM_V1_COMPAT
>> include/services/rmmd_svc.h:#endif /* RMM_V1_COMPAT */
>> make_helpers/defaults.mk:RMM_V1_COMPAT := 1
>> services/std_svc/rmmd/rmmd_main.c:#if RMM_V1_COMPAT
>> services/std_svc/rmmd/rmmd_main.c:#if RMM_V1_COMPAT
>> services/std_svc/rmmd/rmmd_main.c:#if !RMM_V1_COMPAT
>> services/std_svc/rmmd/rmmd_main.c:#if RMM_V1_COMPAT
>> services/std_svc/rmmd/rmmd_main.c:#if RMM_V1_COMPAT
>> services/std_svc/rmmd/rmmd_main.c:#if RMM_V1_COMPAT
>> services/std_svc/rmmd/rmmd_rmm_lfa.c:#if RMM_V1_COMPAT
>> services/std_svc/rmmd/rmmd_rmm_lfa.c:#if RMM_V1_COMPAT
>> services/std_svc/rmmd/rmmd_rmm_lfa.c:#if RMM_V1_COMPAT
>> services/std_svc/rmmd/rmmd_rmm_lfa.c:#if RMM_V1_COMPAT
>> suzuki@ewhatever:trusted-firmware-a$ git log --oneline -1
>> 8dae0862c (HEAD, origin/master, origin/integration, origin/HEAD) Merge
>> changes from topic "qti_lemans_evk" into integration
>> suzuki@ewhatever:trusted-firmware-a$ git remote get-url origin
>> https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git
>>
>
> Thanks for the details. It turned out that I used the wrong TF-A
> repository. In
> the proposed repository, I'm able to see the option 'RMM_V1_COMPAT' and
> the EL3-RMM
> interface compatible issue disappears. However, there are more issues
> popped up.
>
> I build everything manually where the host is emulated by QEMU instead
> of shrinkwrap
> and FVP model. It's used to work well before. Maybe it's time to switch
> to shinkwrap
> and FVP model since device assignment (DA) isn't supported by an
> emulated host
> by QEMU and shrinkwrap and FVP model seems the only option. I need to
> learn how
> to do that later.
Thanks for the update. Yes, QEMU TF-RMM support is in progress, @Mathieu
Poirier is looking into it
>
> There are two issues I can see with the following combinations. Details
> are provided
> like below.
>
> QEMU: https://git.qemu.org/git/
> qemu.git (branch: stable-9.2)
> TF-RMM: https://git.trustedfirmware.org/TF-RMM/tf-
> rmm.git (branch: topics/rmm-v2.0-poc)
> EDK2: git@github.com:tianocore/
> edk2.git (tag: edk2-stable202411)
> TF-A: https://git.trustedfirmware.org/TF-A/trusted-firmware-
> a.git (branch: master)
> HOST: https://git.gitlab.arm.com/linux-arm/linux-
> cca.git (branch: cca-host/v13)
> BUILDROOT: https://github.com/buildroot/
> buildroot (branch: master)
> KVMTOOL: https://gitlab.arm.com/linux-arm/kvmtool-
> cca (branch: cca/v11)
> GUEST: https://github.com/torvalds/
> linux.git (branch: master)
>
> (1) The emulated host is started by the following command lines.
>
> sudo /home/gshan/sandbox/cca/host/qemu/build/qemu-system-
> aarch64 \
> -M virt,virtualization=on,secure=on,gic-
> version=3,acpi=off \
> -cpu max,x-rme=on -m 8G -smp
> 8 \
> -serial mon:stdio -monitor none -nographic -
> nodefaults \
> -bios /home/gshan/sandbox/cca/host/tf-a/
> flash.bin \
> -kernel /home/gshan/sandbox/cca/host/linux/arch/arm64/boot/
> Image \
> -initrd /home/gshan/sandbox/cca/host/buildroot/output/images/
> rootfs.cpio.xz \
> -device pcie-root-
> port,bus=pcie.0,chassis=1,id=pcie.1 \
> -device pcie-root-
> port,bus=pcie.0,chassis=2,id=pcie.2 \
> -device pcie-root-
> port,bus=pcie.0,chassis=3,id=pcie.3 \
> -device pcie-root-
> port,bus=pcie.0,chassis=4,id=pcie.4 \
> -device virtio-9p-
> device,fsdev=shr0,mount_tag=shr0 \
> -fsdev local,security_model=none,path=/home/gshan/sandbox/cca/
> guest,id=shr0 \
> -netdev tap,id=tap1,script=/etc/qemu-ifup-gshan,downscript=/etc/
> qemu-ifdown-gshan \
> -device virtio-net-pci,bus=pcie.2,netdev=tap1,mac=b8:3f:d2:1d:3e:f1
>
> (2) Issue-1: TF-RMM complains about the root complex list is invalid.
> This error is
> raised in TF-RMM::setup_root_complex_list() where the error code is
> still set to
> 0 (SUCCESS) in this failing case. The TF-RMM initialization is
> terminated early,
> but TF-A still thinks the initialization has been completely done.
>
> INFO: BL31: Initializing RMM
> INFO: RMM init start.
> RMM EL3 compat memory reservation enabled.
> Dynamic VA pool base address: 0xc0000000
> Reserved 20 pages. Remaining: 3615 pages
> Reserve mem: 20 pages at PA: 0x401f2000 (alignment 0x1000)
> Static Low VA initialized. xlat tables allocated: 20 used: 7
> Reserved 514 pages. Remaining: 3101 pages
> Reserve mem: 514 pages at PA: 0x40206000 (alignment 0x1000)
> Dynamic Low VA initialized. xlat tables allocated: 514 used: 514
> Invalid: Root Complex list
> <<<<< ERROR
> INFO: RMM init end.
>
> (3) Issue-2: The host kernel gets stuck in rmi_check_version() where
> SMC_RMI_VERSION
> is issued to TF-A, but it can't be forwarded to TF-RMM because its
> initialization
> isn't completely done (issue-1).
>
> [ 37.438253] Unpacking initramfs...
> [ 37.563460] kvm [1]: nv: 570 coarse grained trap handlers
> [ 37.581139] kvm [1]: nv: 664 fine grained trap handlers
> <... system becomes stuck here ...>
>
> So my workaround is to skip fetching root complex list from the EL3-RMM
> manifest data
> in TF-RMM::setup_root_complex_list() since it's not provided for the
> qemu platform by
^^ This may have to do with the RMM<->TF-A Manifest changes
> TF-A. With this workaround, the host can boot up into shell prompt and
> the guest can
> be started by kvmtool.
>
> host$ uname -r
> 7.0.0-rc1-gavin-gd62aa44b2590
> host$ lkvm run --realm -c 2 -m 256 \
> -k /mnt/linux/arch/arm64/boot/Image \
> -i /mnt/buildroot/output/images/rootfs.cpio.xz
> -p earlycon=uart,mmio,0x101000000
> Info: # lkvm run -k /mnt/linux/arch/arm64/boot/Image -m 256 -c 2 --
> name guest-163
> Info: Enabling Guest memfd for confidential guest
> Warning: The maximum recommended amount of VCPUs is 1
> [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x000f0510]
> [ 0.000000] Linux version 7.0.0-rc2-gavin-g0031c06807cf
> (gshan@nvidia-grace-hopper-01.khw.eng.bos2.dc.redhat.com) (gcc (GCC)
> 14.3.1 20251022 (Red Hat 14.3.1-4), GNU ld version 2.41-64.el10) #2 SMP
> PREEMPT Wed Mar 25 20:28:05 EDT 2026
> [ 0.000000] KASLR enabled
> :
> [ 267.578060] Freeing initrd memory: 4728K
> [ 267.921865] Warning: unable to open an initial console.
> [ 270.327960] Freeing unused kernel memory: 1792K
> [ 270.669368] Run /init as init process
>
Cool, thanks!
Suzuki
> Thanks,
> Gavin
>
^ permalink raw reply
* Re: [PATCH v6 00/22] Runtime TDX module update support
From: Chao Gao @ 2026-03-26 8:52 UTC (permalink / raw)
To: kvm, linux-coco, linux-doc, linux-kernel, linux-rt-devel, x86
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Borislav Petkov, Clark Williams,
H. Peter Anvin, Ingo Molnar, Jonathan Corbet,
Sebastian Andrzej Siewior, Shuah Khan, Steven Rostedt,
Thomas Gleixner
In-Reply-To: <20260326084448.29947-1-chao.gao@intel.com>
On Thu, Mar 26, 2026 at 01:43:51AM -0700, Chao Gao wrote:
>Hi Reviewers,
>
>Please review patches 6 and 17; others already have 2+ RB tags.
>
>Patch 6 was reworked to use is_visible() for attribute visibility (which is
>the standard practice), so previous RB tags were dropped. Patch 17 has
>fewer reviews so far and needs another look.
>
>I believe this series is quite mature and also self-contained (no impact to
>the rest of kernel unless an update is triggered through the dedicated
>sysfs ABIs). I'm hoping it can be merged for 7.1.
>
>Changelog:
>v5->v6:
Below is the diff between v5 and v6:
diff --git a/Documentation/ABI/testing/sysfs-devices-faux-tdx-host b/Documentation/ABI/testing/sysfs-devices-faux-tdx-host
index 97840db794c0..e1a2f3b2ea65 100644
--- a/Documentation/ABI/testing/sysfs-devices-faux-tdx-host
+++ b/Documentation/ABI/testing/sysfs-devices-faux-tdx-host
@@ -24,9 +24,8 @@ Description: (RO) Report the number of remaining updates. TDX maintains a
number is always zero if the P-SEAMLDR doesn't support updates.
See Intel® Trust Domain Extensions - SEAM Loader (SEAMLDR)
- Interface Specification, Revision 343755-003, Chapter 3.3
- "SEAMLDR_INFO" and Chapter 4.2 "SEAMLDR.INSTALL" for more
- information.
+ Interface Specification, Chapter "SEAMLDR_INFO" and Chapter
+ "SEAMLDR.INSTALL" for more information.
What: /sys/devices/faux/tdx_host/firmware/tdx_module
Contact: linux-coco@lists.linux.dev
@@ -58,14 +57,15 @@ Description: (RO) See Documentation/ABI/testing/sysfs-class-firmware for
baseline expectations for this file. The <ERROR> part in the
<STATUS>:<ERROR> format can be:
- "device-busy": Compatibility checks failed.
+ "device-busy": Conflicting operations are in progress, e.g., TD
+ build or TD migration.
"read-write-error": Memory allocation failed.
- "hw-error": Cannot communicate with P-SEAMLDR or TDX module.
+ "hw-error": Communication with P-SEAMLDR or TDX module failed
+ or update limit exhausted.
"firmware-invalid": The provided TDX module update is invalid,
- or the number of updates reached the limit,
or other unexpected errors occurred.
"hw-error" or "firmware-invalid" may be fatal, causing all TDs
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index 386097b2e01b..6351d2c21513 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -116,11 +116,6 @@ static inline bool tdx_supports_runtime_update(const struct tdx_sys_info *sysinf
return sysinfo->features.tdx_features0 & TDX_FEATURES0_TD_PRESERVING;
}
-static inline bool tdx_supports_update_compatibility(const struct tdx_sys_info *sysinfo)
-{
- return sysinfo->features.tdx_features0 & TDX_FEATURES0_UPDATE_COMPAT;
-}
-
int tdx_guest_keyid_alloc(void);
u32 tdx_get_nr_guest_keyids(void);
void tdx_guest_keyid_free(unsigned int keyid);
diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
index 4e1ad06506cc..276330179783 100644
--- a/arch/x86/virt/vmx/tdx/seamldr.c
+++ b/arch/x86/virt/vmx/tdx/seamldr.c
@@ -51,7 +51,8 @@ static_assert(sizeof(struct seamldr_params) == 4096);
/*
* Serialize P-SEAMLDR calls since the hardware only allows a single CPU to
* interact with P-SEAMLDR simultaneously. Use raw version as the calls can
- * be made with interrupts disabled.
+ * be made with interrupts disabled, where plain spinlocks are prohibited in
+ * PREEMPT_RT kernels as they become sleeping locks.
*/
static DEFINE_RAW_SPINLOCK(seamldr_lock);
@@ -73,6 +74,13 @@ int seamldr_get_info(struct seamldr_info *seamldr_info)
}
EXPORT_SYMBOL_FOR_MODULES(seamldr_get_info, "tdx-host");
+static int seamldr_install(const struct seamldr_params *params)
+{
+ struct tdx_module_args args = { .rcx = __pa(params) };
+
+ return seamldr_call(P_SEAMLDR_INSTALL, &args);
+}
+
static void free_seamldr_params(struct seamldr_params *params)
{
free_page((unsigned long)params);
@@ -109,8 +117,9 @@ static struct seamldr_params *alloc_seamldr_params(const void *module, unsigned
ptr = sig;
for (i = 0; i < sig_size / SZ_4K; i++) {
/*
- * Don't assume @sig is page-aligned although it is 4KB-aligned.
- * Always add the in-page offset to get the physical address.
+ * @sig is 4KB-aligned, but that does not imply PAGE_SIZE
+ * alignment when PAGE_SIZE != SZ_4K. Always include the
+ * in-page offset.
*/
params->sigstruct_pa[i] = (vmalloc_to_pfn(ptr) << PAGE_SHIFT) +
((unsigned long)ptr & ~PAGE_MASK);
@@ -136,6 +145,10 @@ static struct seamldr_params *alloc_seamldr_params(const void *module, unsigned
* Note this structure differs from the reference above: the two variable-length
* fields "@sigstruct" and "@module" are represented as a single "@data" field
* here and split programmatically using the offset_of_module value.
+ *
+ * Note @offset_of_module is relative to the start of struct tdx_blob, not
+ * @data, and @length is the total length of the blob, not the length of
+ * @data.
*/
struct tdx_blob {
u16 version;
@@ -196,7 +209,7 @@ enum module_update_state {
static struct {
enum module_update_state state;
int thread_ack;
- int failed;
+ bool failed;
/*
* Protect update_data. Raw spinlock as it will be acquired from
* interrupt-disabled contexts.
@@ -234,7 +247,6 @@ static void print_update_failure_message(void)
static int do_seamldr_install_module(void *seamldr_params)
{
enum module_update_state newstate, curstate = MODULE_UPDATE_START;
- struct tdx_module_args args = {};
int cpu = smp_processor_id();
bool primary;
int ret = 0;
@@ -254,8 +266,7 @@ static int do_seamldr_install_module(void *seamldr_params)
ret = tdx_module_shutdown();
break;
case MODULE_UPDATE_CPU_INSTALL:
- args.rcx = __pa(seamldr_params);
- ret = seamldr_call(P_SEAMLDR_INSTALL, &args);
+ ret = seamldr_install(seamldr_params);
break;
case MODULE_UPDATE_CPU_INIT:
ret = tdx_cpu_enable();
@@ -269,8 +280,7 @@ static int do_seamldr_install_module(void *seamldr_params)
}
if (ret) {
- scoped_guard(raw_spinlock, &update_data.lock)
- update_data.failed++;
+ WRITE_ONCE(update_data.failed, true);
if (curstate > MODULE_UPDATE_SHUTDOWN)
print_update_failure_message();
} else {
@@ -314,7 +324,7 @@ int seamldr_install_module(const u8 *data, u32 size)
if (IS_ERR(params))
return PTR_ERR(params);
- update_data.failed = 0;
+ update_data.failed = false;
set_target_state(MODULE_UPDATE_START + 1);
ret = stop_machine(do_seamldr_install_module, params, cpu_online_mask);
if (ret)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index b76b8c393425..3f4221098b78 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -1194,7 +1194,7 @@ int tdx_module_shutdown(void)
*/
args.rcx = tdx_sysinfo.handoff.module_hv;
- if (tdx_supports_update_compatibility(&tdx_sysinfo))
+ if (tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_UPDATE_COMPAT)
args.rcx |= TDX_SYS_SHUTDOWN_AVOID_COMPAT_SENSITIVE;
ret = seamcall(TDH_SYS_SHUTDOWN, &args);
@@ -1214,11 +1214,12 @@ int tdx_module_shutdown(void)
sysinit_ret = 0;
/*
- * By reaching here CPUHP is disabled and all present CPUs
- * are online. It's safe to just loop all online CPUs and
- * reset the per-cpu flag.
+ * Since the TDX module is shut down and gone, mark all CPUs
+ * (including offlined ones) as uninitialied. This is called in
+ * stop_machine() (where CPU hotplug is disabled), preventing
+ * races with other tdx_lp_initialized accesses.
*/
- for_each_online_cpu(cpu)
+ for_each_possible_cpu(cpu)
per_cpu(tdx_lp_initialized, cpu) = false;
return 0;
}
diff --git a/arch/x86/virt/vmx/tdx/tdx_global_metadata.c b/arch/x86/virt/vmx/tdx/tdx_global_metadata.c
index d6a4fa8deb5e..1b6f9b80b197 100644
--- a/arch/x86/virt/vmx/tdx/tdx_global_metadata.c
+++ b/arch/x86/virt/vmx/tdx/tdx_global_metadata.c
@@ -102,13 +102,15 @@ static int get_tdx_sys_info_td_conf(struct tdx_sys_info_td_conf *sysinfo_td_conf
static int get_tdx_sys_info_handoff(struct tdx_sys_info_handoff *sysinfo_handoff)
{
- int ret = 0;
+ int ret;
u64 val;
- if (!ret && !(ret = read_sys_metadata_field(0x8900000100000000, &val)))
- sysinfo_handoff->module_hv = val;
+ ret = read_sys_metadata_field(0x8900000100000000, &val);
+ if (ret)
+ return ret;
- return ret;
+ sysinfo_handoff->module_hv = val;
+ return 0;
}
static int get_tdx_sys_info(struct tdx_sys_info *sysinfo)
diff --git a/drivers/virt/coco/tdx-host/tdx-host.c b/drivers/virt/coco/tdx-host/tdx-host.c
index 8cf3cc99024a..f236119c2748 100644
--- a/drivers/virt/coco/tdx-host/tdx-host.c
+++ b/drivers/virt/coco/tdx-host/tdx-host.c
@@ -21,6 +21,12 @@ static const struct x86_cpu_id tdx_host_ids[] = {
};
MODULE_DEVICE_TABLE(x86cpu, tdx_host_ids);
+/*
+ * TDX module and P-SEAMLDR version convention: "major.minor.update"
+ * (e.g., "1.5.08") with zero-padded two-digit update field.
+ */
+#define TDX_VERSION_FMT "%u.%u.%02u"
+
static ssize_t version_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
@@ -32,9 +38,9 @@ static ssize_t version_show(struct device *dev, struct device_attribute *attr,
ver = &tdx_sysinfo->version;
- return sysfs_emit(buf, "%u.%u.%02u\n", ver->major_version,
- ver->minor_version,
- ver->update_version);
+ return sysfs_emit(buf, TDX_VERSION_FMT"\n", ver->major_version,
+ ver->minor_version,
+ ver->update_version);
}
static DEVICE_ATTR_RO(version);
@@ -42,7 +48,10 @@ static struct attribute *tdx_host_attrs[] = {
&dev_attr_version.attr,
NULL,
};
-ATTRIBUTE_GROUPS(tdx_host);
+
+static const struct attribute_group tdx_host_group = {
+ .attrs = tdx_host_attrs,
+};
static ssize_t seamldr_version_show(struct device *dev, struct device_attribute *attr,
char *buf)
@@ -54,9 +63,9 @@ static ssize_t seamldr_version_show(struct device *dev, struct device_attribute
if (ret)
return ret;
- return sysfs_emit(buf, "%u.%u.%02u\n", info.major_version,
- info.minor_version,
- info.update_version);
+ return sysfs_emit(buf, TDX_VERSION_FMT"\n", info.major_version,
+ info.minor_version,
+ info.update_version);
}
static ssize_t num_remaining_updates_show(struct device *dev,
@@ -90,9 +99,41 @@ static struct attribute *seamldr_attrs[] = {
NULL,
};
+static bool can_expose_seamldr(void)
+{
+ const struct tdx_sys_info *sysinfo = tdx_get_sysinfo();
+
+ if (!sysinfo)
+ return false;
+
+ /*
+ * Calling P-SEAMLDR on CPUs with the seamret_invd_vmcs bug clears
+ * the current VMCS, which breaks KVM. Verify the erratum is not
+ * present before exposing P-SEAMLDR features.
+ */
+ if (boot_cpu_has_bug(X86_BUG_SEAMRET_INVD_VMCS))
+ return false;
+
+ return tdx_supports_runtime_update(sysinfo);
+}
+
+static bool seamldr_group_visible(struct kobject *kobj)
+{
+ return can_expose_seamldr();
+}
+
+DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE(seamldr);
+
static const struct attribute_group seamldr_group = {
.name = "seamldr",
.attrs = seamldr_attrs,
+ .is_visible = SYSFS_GROUP_VISIBLE(seamldr),
+};
+
+static const struct attribute_group *tdx_host_groups[] = {
+ &tdx_host_group,
+ &seamldr_group,
+ NULL,
};
static enum fw_upload_err tdx_fw_prepare(struct fw_upload *fwl,
@@ -122,8 +163,6 @@ static enum fw_upload_err tdx_fw_write(struct fw_upload *fwl, const u8 *data,
return FW_UPLOAD_ERR_BUSY;
case -EIO:
return FW_UPLOAD_ERR_HW_ERROR;
- case -ENOSPC:
- return FW_UPLOAD_ERR_WEAROUT;
case -ENOMEM:
return FW_UPLOAD_ERR_RW_ERROR;
default:
@@ -164,22 +203,9 @@ static void seamldr_deinit(void *tdx_fwl)
static int seamldr_init(struct device *dev)
{
- const struct tdx_sys_info *tdx_sysinfo = tdx_get_sysinfo();
struct fw_upload *tdx_fwl;
- int ret;
-
- if (WARN_ON_ONCE(!tdx_sysinfo))
- return -EIO;
- if (!tdx_supports_runtime_update(tdx_sysinfo))
- return 0;
-
- /*
- * Calling P-SEAMLDR on CPUs with the seamret_invd_vmcs bug clears
- * the current VMCS, which breaks KVM. Verify the erratum is not
- * present before exposing P-SEAMLDR features.
- */
- if (boot_cpu_has_bug(X86_BUG_SEAMRET_INVD_VMCS))
+ if (!can_expose_seamldr())
return 0;
tdx_fwl = firmware_upload_register(THIS_MODULE, dev, "tdx_module",
@@ -187,11 +213,7 @@ static int seamldr_init(struct device *dev)
if (IS_ERR(tdx_fwl))
return PTR_ERR(tdx_fwl);
- ret = devm_add_action_or_reset(dev, seamldr_deinit, tdx_fwl);
- if (ret)
- return ret;
-
- return devm_device_add_group(dev, &seamldr_group);
+ return devm_add_action_or_reset(dev, seamldr_deinit, tdx_fwl);
}
static int tdx_host_probe(struct faux_device *fdev)
^ permalink raw reply related
* [PATCH v6 22/22] x86/virt/seamldr: Log TDX module update failures
From: Chao Gao @ 2026-03-26 8:44 UTC (permalink / raw)
To: linux-kernel, linux-coco, kvm
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Chao Gao, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260326084448.29947-1-chao.gao@intel.com>
Currently, there is no way to restore a TDX module from shutdown state to
running state. This means if errors occur after a successful module
shutdown, they are unrecoverable since the old module is gone but the new
module isn't installed. All subsequent SEAMCALLs to the TDX module will
fail, so TDs will be killed due to SEAMCALL failures.
Log a message to clarify that SEAMCALL errors are expected in this
scenario. This ensures that after update failures, the first message in
dmesg explains the situation rather than showing confusing call traces from
various code paths.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Acked-by: Kai Huang <kai.huang@intel.com>
---
v4:
- Use pr_warn_once() instead of reinventing it [Yilun]
v3:
- Rephrase the changelog to eliminate the confusing uses of 'i.e.' and 'e.g.'
[Dave/Yilun]
---
arch/x86/virt/vmx/tdx/seamldr.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
index 0ab2413e3754..276330179783 100644
--- a/arch/x86/virt/vmx/tdx/seamldr.c
+++ b/arch/x86/virt/vmx/tdx/seamldr.c
@@ -235,6 +235,11 @@ static void ack_state(void)
set_target_state(update_data.state + 1);
}
+static void print_update_failure_message(void)
+{
+ pr_err_once("update failed, SEAMCALLs will report failure until TDs killed\n");
+}
+
/*
* See multi_cpu_stop() from where this multi-cpu state-machine was
* adopted, and the rationale for touch_nmi_watchdog().
@@ -274,10 +279,13 @@ static int do_seamldr_install_module(void *seamldr_params)
break;
}
- if (ret)
+ if (ret) {
WRITE_ONCE(update_data.failed, true);
- else
+ if (curstate > MODULE_UPDATE_SHUTDOWN)
+ print_update_failure_message();
+ } else {
ack_state();
+ }
} else {
touch_nmi_watchdog();
rcu_momentary_eqs();
--
2.47.3
^ permalink raw reply related
* [PATCH v6 21/22] x86/virt/tdx: Document TDX module update
From: Chao Gao @ 2026-03-26 8:44 UTC (permalink / raw)
To: linux-kernel, linux-doc, linux-coco, kvm
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Chao Gao, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin,
Jonathan Corbet, Shuah Khan
In-Reply-To: <20260326084448.29947-1-chao.gao@intel.com>
Document TDX module update as a subsection of "TDX Host Kernel Support" to
provide background information and cover key points that developers and
users may need to know, for example:
- update is done in stop_machine() context
- update instructions and results
- update policy and tooling
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
v5:
- use "update" when refer to the update feature/concept [Kai]
---
Documentation/arch/x86/tdx.rst | 36 ++++++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)
diff --git a/Documentation/arch/x86/tdx.rst b/Documentation/arch/x86/tdx.rst
index 61670e7df2f7..d4e257542d4c 100644
--- a/Documentation/arch/x86/tdx.rst
+++ b/Documentation/arch/x86/tdx.rst
@@ -99,6 +99,42 @@ initialize::
[..] virt/tdx: module initialization failed ...
+TDX module Runtime Update
+-------------------------
+
+The TDX architecture includes a persistent SEAM loader (P-SEAMLDR) that
+runs in SEAM mode separately from the TDX module. The kernel can
+communicate with P-SEAMLDR to perform runtime updates of the TDX module.
+
+During update, the TDX module becomes unresponsive to other TDX operations.
+To prevent components using TDX (such as KVM) from experiencing unexpected
+errors during updates, updates are performed in stop_machine() context.
+
+TDX module update has complex compatibility requirements; the new module
+must be compatible with the current CPU, P-SEAMLDR, and running TDX module.
+Rather than implementing complex module selection and policy enforcement
+logic in the kernel, userspace is responsible for auditing and selecting
+appropriate updates.
+
+Updates use the standard firmware upload interface. See
+Documentation/driver-api/firmware/fw_upload.rst for detailed instructions
+
+Successful updates are logged in dmesg:
+ [..] virt/tdx: version 1.5.20 -> 1.5.24
+
+If updates failed, running TDs may be killed and further TDX operations may
+be not possible until reboot. For detailed error information, see
+Documentation/ABI/testing/sysfs-devices-faux-tdx-host.
+
+Given the risk of losing existing TDs, userspace should verify that the
+update is compatible with the current system and properly validated before
+applying it.
+
+A reference userspace tool that implements necessary checks is available
+at:
+
+ https://github.com/intel/tdx-module-binaries
+
TDX Interaction to Other Kernel Components
------------------------------------------
--
2.47.3
^ permalink raw reply related
* [PATCH v6 18/22] coco/tdx-host: Don't expose P-SEAMLDR features on CPUs with erratum
From: Chao Gao @ 2026-03-26 8:44 UTC (permalink / raw)
To: linux-kernel, kvm, linux-coco
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Chao Gao, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260326084448.29947-1-chao.gao@intel.com>
Some TDX-capable CPUs have an erratum, as documented in Intel® Trust
Domain CPU Architectural Extensions (May 2021 edition) Chapter 2.3:
SEAMRET from the P-SEAMLDR clears the current VMCS structure pointed
to by the current-VMCS pointer. A VMM that invokes the P-SEAMLDR using
SEAMCALL must reload the current-VMCS, if required, using the VMPTRLD
instruction.
Clearing the current VMCS behind KVM's back will break KVM.
This erratum is not present when IA32_VMX_BASIC[60] is set. Add a CPU
bug bit for this erratum and refuse to expose P-SEAMLDR features (e.g.,
TDX module updates) on affected CPUs.
== Alternatives ==
Two workarounds were considered but both were rejected:
1. Save/restore the current VMCS around P-SEAMLDR calls. This produces ugly
assembly code [1] and doesn't play well with #MCE or #NMI if they
need to use the current VMCS.
2. Move KVM's VMCS tracking logic to the TDX core code, which would break
the boundary between KVM and the TDX core code [2].
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Link: https://lore.kernel.org/kvm/fedb3192-e68c-423c-93b2-a4dc2f964148@intel.com/ # [1]
Link: https://lore.kernel.org/kvm/aYIXFmT-676oN6j0@google.com/ # [2]
---
arch/x86/include/asm/cpufeatures.h | 1 +
arch/x86/include/asm/vmx.h | 1 +
arch/x86/virt/vmx/tdx/tdx.c | 11 +++++++++++
drivers/virt/coco/tdx-host/tdx-host.c | 8 ++++++++
4 files changed, 21 insertions(+)
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index dbe104df339b..377d009b7e2e 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -572,4 +572,5 @@
#define X86_BUG_ITS_NATIVE_ONLY X86_BUG( 1*32+ 8) /* "its_native_only" CPU is affected by ITS, VMX is not affected */
#define X86_BUG_TSA X86_BUG( 1*32+ 9) /* "tsa" CPU is affected by Transient Scheduler Attacks */
#define X86_BUG_VMSCAPE X86_BUG( 1*32+10) /* "vmscape" CPU is affected by VMSCAPE attacks from guests */
+#define X86_BUG_SEAMRET_INVD_VMCS X86_BUG( 1*32+11) /* "seamret_invd_vmcs" SEAMRET from P-SEAMLDR clears the current VMCS */
#endif /* _ASM_X86_CPUFEATURES_H */
diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h
index b92ff87e3560..a5a5b373ec42 100644
--- a/arch/x86/include/asm/vmx.h
+++ b/arch/x86/include/asm/vmx.h
@@ -136,6 +136,7 @@
#define VMX_BASIC_INOUT BIT_ULL(54)
#define VMX_BASIC_TRUE_CTLS BIT_ULL(55)
#define VMX_BASIC_NO_HW_ERROR_CODE_CC BIT_ULL(56)
+#define VMX_BASIC_NO_SEAMRET_INVD_VMCS BIT_ULL(60)
static inline u32 vmx_basic_vmcs_revision_id(u64 vmx_basic)
{
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index ad5b83390e61..3f4221098b78 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -39,6 +39,7 @@
#include <asm/cpu_device_id.h>
#include <asm/processor.h>
#include <asm/mce.h>
+#include <asm/vmx.h>
#include "seamcall_internal.h"
#include "tdx.h"
@@ -1455,6 +1456,8 @@ static struct notifier_block tdx_memory_nb = {
static void __init check_tdx_erratum(void)
{
+ u64 basic_msr;
+
/*
* These CPUs have an erratum. A partial write from non-TD
* software (e.g. via MOVNTI variants or UC/WC mapping) to TDX
@@ -1466,6 +1469,14 @@ static void __init check_tdx_erratum(void)
case INTEL_EMERALDRAPIDS_X:
setup_force_cpu_bug(X86_BUG_TDX_PW_MCE);
}
+
+ /*
+ * Some TDX-capable CPUs have an erratum where the current VMCS is
+ * cleared after calling into P-SEAMLDR.
+ */
+ rdmsrq(MSR_IA32_VMX_BASIC, basic_msr);
+ if (!(basic_msr & VMX_BASIC_NO_SEAMRET_INVD_VMCS))
+ setup_force_cpu_bug(X86_BUG_SEAMRET_INVD_VMCS);
}
void __init tdx_init(void)
diff --git a/drivers/virt/coco/tdx-host/tdx-host.c b/drivers/virt/coco/tdx-host/tdx-host.c
index d4a552853021..f236119c2748 100644
--- a/drivers/virt/coco/tdx-host/tdx-host.c
+++ b/drivers/virt/coco/tdx-host/tdx-host.c
@@ -106,6 +106,14 @@ static bool can_expose_seamldr(void)
if (!sysinfo)
return false;
+ /*
+ * Calling P-SEAMLDR on CPUs with the seamret_invd_vmcs bug clears
+ * the current VMCS, which breaks KVM. Verify the erratum is not
+ * present before exposing P-SEAMLDR features.
+ */
+ if (boot_cpu_has_bug(X86_BUG_SEAMRET_INVD_VMCS))
+ return false;
+
return tdx_supports_runtime_update(sysinfo);
}
--
2.47.3
^ permalink raw reply related
* [PATCH v6 20/22] coco/tdx-host: Document TDX module update compatibility criteria
From: Chao Gao @ 2026-03-26 8:44 UTC (permalink / raw)
To: x86, linux-coco, kvm, linux-kernel
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Chao Gao
In-Reply-To: <20260326084448.29947-1-chao.gao@intel.com>
The TDX module update protocol facilitates compatible runtime updates.
Document the compatibility criteria and indicators of various update
failures, including violations of the compatibility criteria.
Note that runtime TDX module updates are an "update at your own risk"
operation; userspace must enforce all of the above compatibility
criteria.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
v6:
- improve the error scenario descriptions
v5:
- drop "dead documentation" about tdxctl
- add a note in the changelog clarifying that users update at their own risk
- revise the error code for update limit exhaustion—it changed after
dropping the related patch.
v4:
- Drop "compat_capable" kernel ABI [Dan]
- Document Linux compatibility expectations and results of violating
them [Dan]
---
.../ABI/testing/sysfs-devices-faux-tdx-host | 47 +++++++++++++++++++
1 file changed, 47 insertions(+)
diff --git a/Documentation/ABI/testing/sysfs-devices-faux-tdx-host b/Documentation/ABI/testing/sysfs-devices-faux-tdx-host
index f7221f2e5fec..e1a2f3b2ea65 100644
--- a/Documentation/ABI/testing/sysfs-devices-faux-tdx-host
+++ b/Documentation/ABI/testing/sysfs-devices-faux-tdx-host
@@ -26,3 +26,50 @@ Description: (RO) Report the number of remaining updates. TDX maintains a
See Intel® Trust Domain Extensions - SEAM Loader (SEAMLDR)
Interface Specification, Chapter "SEAMLDR_INFO" and Chapter
"SEAMLDR.INSTALL" for more information.
+
+What: /sys/devices/faux/tdx_host/firmware/tdx_module
+Contact: linux-coco@lists.linux.dev
+Description: (Directory) The tdx_module directory implements the fw_upload
+ sysfs ABI, see Documentation/ABI/testing/sysfs-class-firmware
+ for the general description of the attributes @data, @cancel,
+ @error, @loading, @remaining_size, and @status. This ABI
+ facilitates "Compatible TDX Module Updates". A compatible update
+ is one that meets the following criteria:
+
+ Does not interrupt or interfere with any current TDX
+ operation or TD VM.
+
+ Does not invalidate any previously consumed Module metadata
+ values outside of the TEE_TCB_SVN_2 field (updated Security
+ Version Number) in TD Quotes.
+
+ Does not require validation of new Module metadata fields. By
+ implication, new Module features and capabilities are only
+ available by installing the Module at reboot (BIOS or EFI
+ helper loaded).
+
+ See tdx_host/firmware/tdx_module/error for information on
+ compatibility check failures.
+
+What: /sys/devices/faux/tdx_host/firmware/tdx_module/error
+Contact: linux-coco@lists.linux.dev
+Description: (RO) See Documentation/ABI/testing/sysfs-class-firmware for
+ baseline expectations for this file. The <ERROR> part in the
+ <STATUS>:<ERROR> format can be:
+
+ "device-busy": Conflicting operations are in progress, e.g., TD
+ build or TD migration.
+
+ "read-write-error": Memory allocation failed.
+
+ "hw-error": Communication with P-SEAMLDR or TDX module failed
+ or update limit exhausted.
+
+ "firmware-invalid": The provided TDX module update is invalid,
+ or other unexpected errors occurred.
+
+ "hw-error" or "firmware-invalid" may be fatal, causing all TDs
+ and the TDX module to be lost and preventing further TDX
+ operations. This occurs when reading
+ /sys/devices/faux/tdx_host/version returns -ENXIO. For other
+ errors, TDs and the (previous) TDX module stay running.
--
2.47.3
^ permalink raw reply related
* [PATCH v6 19/22] x86/virt/tdx: Enable TDX module runtime updates
From: Chao Gao @ 2026-03-26 8:44 UTC (permalink / raw)
To: linux-kernel, linux-coco, kvm
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Chao Gao, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260326084448.29947-1-chao.gao@intel.com>
All pieces of TDX module runtime updates are in place. Enable it if it
is supported.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
arch/x86/include/asm/tdx.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index 8bf99e76d32f..6351d2c21513 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -36,6 +36,7 @@
#define TDX_UPDATE_COMPAT_SENSITIVE 0x8000051200000000ULL
/* Bit definitions of TDX_FEATURES0 metadata field */
+#define TDX_FEATURES0_TD_PRESERVING BIT_ULL(1)
#define TDX_FEATURES0_NO_RBP_MOD BIT_ULL(18)
#define TDX_FEATURES0_UPDATE_COMPAT BIT_ULL(47)
@@ -112,8 +113,7 @@ const struct tdx_sys_info *tdx_get_sysinfo(void);
static inline bool tdx_supports_runtime_update(const struct tdx_sys_info *sysinfo)
{
- /* To be enabled when kernel is ready. */
- return false;
+ return sysinfo->features.tdx_features0 & TDX_FEATURES0_TD_PRESERVING;
}
int tdx_guest_keyid_alloc(void);
--
2.47.3
^ permalink raw reply related
* [PATCH v6 17/22] x86/virt/tdx: Avoid updates during update-sensitive operations
From: Chao Gao @ 2026-03-26 8:44 UTC (permalink / raw)
To: linux-kernel, linux-coco, kvm
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Chao Gao, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260326084448.29947-1-chao.gao@intel.com>
A runtime TDX module update can conflict with TD lifecycle operations that
are update-sensitive.
Today, update-sensitive operations include:
- TD build: TD measurement is accumulated across multiple
TDH.MEM.PAGE.ADD, TDH.MR.EXTEND, and TDH.MR.FINALIZE calls.
- TD migration: intermediate crypto state is saved/restored across
interrupted/resumed TDH.EXPORT.STATE.* and TDH.IMPORT.STATE.* flows.
If an update races TD build, for example, TD measurement can become
incorrect and attestation can fail.
The TDX architecture exposes two approaches:
1) Avoid updates during update-sensitive operations.
2) Detect incompatibility after update and recover.
Post-update detection (option #2) is not a good fit: as discussed in [1],
future module behavior may expand update-sensitive operations in ways that
make KVM ABIs unstable and will break userspace.
"Do nothing" is also not preferred: while it keeps kernel code simple, it
lets the issue leak into the broader stack, where both detection and
recovery require significantly more effort.
So, use option #1. Specifically, request "avoid update-sensitive" behavior
during TDX module shutdown and map the resulting failure to -EBUSY so
userspace can distinguish an update race from other failures.
When the "avoid update-sensitive" feature isn't supported by the TDX
module, proceed with updates and let userspace update at their own risk.
Userspace can check if the feature is supported or not. The alternative of
blocking updates entirely is rejected [2] as it introduces permanent kernel
complexity to accommodate limitations in early TDX module releases that
userspace can handle.
Note: this implementation is based on a reference patch by Vishal [3].
Note2: moving "NO_RBP_MOD" is just to centralize bit definitions.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Link: https://lore.kernel.org/linux-coco/aQIbM5m09G0FYTzE@google.com/ # [1]
Link: https://lore.kernel.org/kvm/699fe97dc212f_2f4a100b@dwillia2-mobl4.notmuch/ # [2]
Link: https://lore.kernel.org/linux-coco/CAGtprH_oR44Vx9Z0cfxvq5-QbyLmy_+Gn3tWm3wzHPmC1nC0eg@mail.gmail.com/ # [3]
---
v6:
- Revise the changelog to clarify behavior when "avoid
update-sensitive" isn't supported.
- Drop unnecessary wrapper for feature capability check
---
arch/x86/include/asm/tdx.h | 11 +++++++++--
arch/x86/kvm/vmx/tdx_errno.h | 2 --
arch/x86/virt/vmx/tdx/tdx.c | 23 +++++++++++++++++++----
arch/x86/virt/vmx/tdx/tdx.h | 3 ---
4 files changed, 28 insertions(+), 11 deletions(-)
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index 1fb2a3f6b9e1..8bf99e76d32f 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -26,11 +26,18 @@
#define TDX_SEAMCALL_GP (TDX_SW_ERROR | X86_TRAP_GP)
#define TDX_SEAMCALL_UD (TDX_SW_ERROR | X86_TRAP_UD)
+#define TDX_SEAMCALL_STATUS_MASK 0xFFFFFFFF00000000ULL
+
/*
* TDX module SEAMCALL leaf function error codes
*/
-#define TDX_SUCCESS 0ULL
-#define TDX_RND_NO_ENTROPY 0x8000020300000000ULL
+#define TDX_SUCCESS 0ULL
+#define TDX_RND_NO_ENTROPY 0x8000020300000000ULL
+#define TDX_UPDATE_COMPAT_SENSITIVE 0x8000051200000000ULL
+
+/* Bit definitions of TDX_FEATURES0 metadata field */
+#define TDX_FEATURES0_NO_RBP_MOD BIT_ULL(18)
+#define TDX_FEATURES0_UPDATE_COMPAT BIT_ULL(47)
#ifndef __ASSEMBLER__
diff --git a/arch/x86/kvm/vmx/tdx_errno.h b/arch/x86/kvm/vmx/tdx_errno.h
index 6ff4672c4181..215c00d76a94 100644
--- a/arch/x86/kvm/vmx/tdx_errno.h
+++ b/arch/x86/kvm/vmx/tdx_errno.h
@@ -4,8 +4,6 @@
#ifndef __KVM_X86_TDX_ERRNO_H
#define __KVM_X86_TDX_ERRNO_H
-#define TDX_SEAMCALL_STATUS_MASK 0xFFFFFFFF00000000ULL
-
/*
* TDX SEAMCALL Status Codes (returned in RAX)
*/
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 69c97b73e243..ad5b83390e61 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -1176,10 +1176,13 @@ int tdx_enable(void)
}
EXPORT_SYMBOL_FOR_KVM(tdx_enable);
+#define TDX_SYS_SHUTDOWN_AVOID_COMPAT_SENSITIVE BIT(16)
+
int tdx_module_shutdown(void)
{
struct tdx_module_args args = {};
- int ret, cpu;
+ u64 ret;
+ int cpu;
/*
* Shut down the TDX module and prepare handoff data for the next
@@ -1189,9 +1192,21 @@ int tdx_module_shutdown(void)
* modules as new modules likely have higher handoff version.
*/
args.rcx = tdx_sysinfo.handoff.module_hv;
- ret = seamcall_prerr(TDH_SYS_SHUTDOWN, &args);
- if (ret)
- return ret;
+
+ if (tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_UPDATE_COMPAT)
+ args.rcx |= TDX_SYS_SHUTDOWN_AVOID_COMPAT_SENSITIVE;
+
+ ret = seamcall(TDH_SYS_SHUTDOWN, &args);
+
+ /*
+ * Return -EBUSY to signal that there is one or more ongoing flows
+ * which may not be compatible with an updated TDX module, so that
+ * userspace can retry on this error.
+ */
+ if ((ret & TDX_SEAMCALL_STATUS_MASK) == TDX_UPDATE_COMPAT_SENSITIVE)
+ return -EBUSY;
+ else if (ret)
+ return -EIO;
tdx_module_status = TDX_MODULE_UNINITIALIZED;
sysinit_done = false;
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index f8686247c660..2435f88c6994 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -88,9 +88,6 @@ struct tdmr_info {
DECLARE_FLEX_ARRAY(struct tdmr_reserved_area, reserved_areas);
} __packed __aligned(TDMR_INFO_ALIGNMENT);
-/* Bit definitions of TDX_FEATURES0 metadata field */
-#define TDX_FEATURES0_NO_RBP_MOD BIT(18)
-
/*
* Do not put any hardware-defined TDX structure representations below
* this comment!
--
2.47.3
^ permalink raw reply related
* [PATCH v6 16/22] x86/virt/tdx: Update tdx_sysinfo and check features post-update
From: Chao Gao @ 2026-03-26 8:44 UTC (permalink / raw)
To: linux-kernel, linux-coco, kvm
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Chao Gao, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260326084448.29947-1-chao.gao@intel.com>
tdx_sysinfo contains all metadata of the active TDX module, including
versions, supported features, and TDMR/TDCS/TDVPS information etc. These
values may change over updates. Blindly refreshing the entire tdx_sysinfo
could disrupt running software, as it may subtly rely on the previous state
unless proven otherwise.
Adopt a conservative approach, like microcode updates, by only refreshing
version information that does not affect functionality, while ignoring
all other changes. This is acceptable as new modules are required to
maintain backward compatibility.
Any updates to metadata beyond versions should be justified and reviewed on
a case-by-case basis.
Note that preallocating a tdx_sys_info buffer before updates is to avoid
having to handle -ENOMEM when updating tdx_sysinfo after a successful
update.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
v5:
- Drop the comment above tdx_module_post_update() [Kai]
v3:
- use 'old' instead of 'cur' as the local variable to represent the
sysinfo of the previous module [Binbin]
- combine if(ret) and WARN_ONCE(1, ...) to WARN_ONCE(ret, ...) [Binbin]
- Improve the print log messages after detecting new features from updates.
[Binbin]
v2:
- don't add a separate function for version and feature checks. Do them
directly in tdx_module_post_update()
- add a comment about preallocating a tdx_sys_info buffer in
seamldr_install_module().
---
arch/x86/virt/vmx/tdx/seamldr.c | 18 ++++++++++++++-
arch/x86/virt/vmx/tdx/tdx.c | 39 +++++++++++++++++++++++++++++++++
arch/x86/virt/vmx/tdx/tdx.h | 3 +++
3 files changed, 59 insertions(+), 1 deletion(-)
diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
index 6d3ea6d36f36..0ab2413e3754 100644
--- a/arch/x86/virt/vmx/tdx/seamldr.c
+++ b/arch/x86/virt/vmx/tdx/seamldr.c
@@ -299,6 +299,18 @@ DEFINE_FREE(free_seamldr_params, struct seamldr_params *,
*/
int seamldr_install_module(const u8 *data, u32 size)
{
+ int ret;
+
+ /*
+ * Preallocating a tdx_sys_info buffer before an update is to avoid
+ * having to handle -ENOMEM when updating tdx_sysinfo after a
+ * successful update.
+ */
+ struct tdx_sys_info *sysinfo __free(kfree) = kzalloc(sizeof(*sysinfo),
+ GFP_KERNEL);
+ if (!sysinfo)
+ return -ENOMEM;
+
struct seamldr_params *params __free(free_seamldr_params) =
init_seamldr_params(data, size);
if (IS_ERR(params))
@@ -306,6 +318,10 @@ int seamldr_install_module(const u8 *data, u32 size)
update_data.failed = false;
set_target_state(MODULE_UPDATE_START + 1);
- return stop_machine(do_seamldr_install_module, params, cpu_online_mask);
+ ret = stop_machine(do_seamldr_install_module, params, cpu_online_mask);
+ if (ret)
+ return ret;
+
+ return tdx_module_post_update(sysinfo);
}
EXPORT_SYMBOL_FOR_MODULES(seamldr_install_module, "tdx-host");
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 7592cba58c19..69c97b73e243 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -1224,6 +1224,45 @@ int tdx_module_run_update(void)
return 0;
}
+int tdx_module_post_update(struct tdx_sys_info *info)
+{
+ struct tdx_sys_info_version *old, *new;
+ int ret;
+
+ /* Shouldn't fail as the update has succeeded. */
+ ret = get_tdx_sys_info(info);
+ if (WARN_ONCE(ret, "version retrieval failed after update, replace the TDX module\n"))
+ return ret;
+
+ old = &tdx_sysinfo.version;
+ new = &info->version;
+ pr_info("version %u.%u.%02u -> %u.%u.%02u\n", old->major_version,
+ old->minor_version,
+ old->update_version,
+ new->major_version,
+ new->minor_version,
+ new->update_version);
+
+ /*
+ * Blindly refreshing the entire tdx_sysinfo could disrupt running
+ * software, as it may subtly rely on the previous state unless
+ * proven otherwise.
+ *
+ * Only refresh version information (including handoff version)
+ * that does not affect functionality, and ignore all other
+ * changes.
+ */
+ tdx_sysinfo.version = info->version;
+ tdx_sysinfo.handoff = info->handoff;
+
+ if (!memcmp(&tdx_sysinfo, info, sizeof(*info)))
+ return 0;
+
+ pr_info("TDX module features have changed after updates, but might not take effect.\n");
+ pr_info("Please consider updating your BIOS to install the TDX module.\n");
+ return 0;
+}
+
static bool is_pamt_page(unsigned long phys)
{
struct tdmr_info_list *tdmr_list = &tdx_tdmr_list;
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index c62874b87d7a..f8686247c660 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -4,6 +4,8 @@
#include <linux/bits.h>
+#include <asm/tdx_global_metadata.h>
+
/*
* This file contains both macros and data structures defined by the TDX
* architecture and Linux defined software data structures and functions.
@@ -122,5 +124,6 @@ struct tdmr_info_list {
int tdx_module_shutdown(void);
int tdx_module_run_update(void);
+int tdx_module_post_update(struct tdx_sys_info *info);
#endif
--
2.47.3
^ permalink raw reply related
* [PATCH v6 15/22] x86/virt/tdx: Restore TDX module state
From: Chao Gao @ 2026-03-26 8:44 UTC (permalink / raw)
To: linux-kernel, linux-coco, kvm
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Chao Gao, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260326084448.29947-1-chao.gao@intel.com>
TDX module state was packed as handoff data during module shutdown. After
per-CPU initialization, the new module can restore TDX module state from
handoff data to preserve running TDs.
Once the restoration is done, the TDX module update is complete, which
means the new module is ready to handle requests from the host and guests.
Implement the new TDH.SYS.UPDATE SEAMCALL to restore TDX module state
and invoke it on one CPU since it only needs to be called once.
For error handling, Intel® Trust Domain Extensions (Intel® TDX)
Module Base Architecture Specification, Chapter "Restore TDX Module
State after a TD-Preserving Update" states
If TDH.SYS.UPDATE returns an error, then the host VMM can continue
with the non-update sequence (TDH.SYS.CONFIG, TDH.SYS.KEY.CONFIG
etc.). In this case all existing TDs are lost. Alternatively, the host
VMM can request the P-SEAMLDR to update to another TDX module. If that
update is successful, existing TDs are preserved.
Given the complexity and uncertain value of above recovery paths, simply
propagate errors and log an error message on failures.
Also note that the location and the format of handoff data is defined by
the TDX module. The new module knows where to get handoff data and how
to parse it. The kernel doesn't need to provide its location, format etc.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
v6:
- make clear how errors are handled in commit message [Kiryl]
v5:
- Massage changelog [Kai]
v3:
- use seamcall_prerr() rather than raw seamcall() [Binbin]
- use pr_err() to print error message [Binbin]
---
arch/x86/virt/vmx/tdx/seamldr.c | 5 +++++
arch/x86/virt/vmx/tdx/tdx.c | 16 ++++++++++++++++
arch/x86/virt/vmx/tdx/tdx.h | 2 ++
3 files changed, 23 insertions(+)
diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
index 8b196ce45546..6d3ea6d36f36 100644
--- a/arch/x86/virt/vmx/tdx/seamldr.c
+++ b/arch/x86/virt/vmx/tdx/seamldr.c
@@ -202,6 +202,7 @@ enum module_update_state {
MODULE_UPDATE_SHUTDOWN,
MODULE_UPDATE_CPU_INSTALL,
MODULE_UPDATE_CPU_INIT,
+ MODULE_UPDATE_RUN_UPDATE,
MODULE_UPDATE_DONE,
};
@@ -265,6 +266,10 @@ static int do_seamldr_install_module(void *seamldr_params)
case MODULE_UPDATE_CPU_INIT:
ret = tdx_cpu_enable();
break;
+ case MODULE_UPDATE_RUN_UPDATE:
+ if (primary)
+ ret = tdx_module_run_update();
+ break;
default:
break;
}
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index a2a46c734d5e..7592cba58c19 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -1208,6 +1208,22 @@ int tdx_module_shutdown(void)
return 0;
}
+int tdx_module_run_update(void)
+{
+ struct tdx_module_args args = {};
+ int ret;
+
+ ret = seamcall_prerr(TDH_SYS_UPDATE, &args);
+ if (ret) {
+ pr_err("update failed (%d)\n", ret);
+ tdx_module_status = TDX_MODULE_ERROR;
+ return ret;
+ }
+
+ tdx_module_status = TDX_MODULE_INITIALIZED;
+ return 0;
+}
+
static bool is_pamt_page(unsigned long phys)
{
struct tdmr_info_list *tdmr_list = &tdx_tdmr_list;
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index 1c4da9540ae0..c62874b87d7a 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -47,6 +47,7 @@
#define TDH_VP_WR 43
#define TDH_SYS_CONFIG 45
#define TDH_SYS_SHUTDOWN 52
+#define TDH_SYS_UPDATE 53
/*
* SEAMCALL leaf:
@@ -120,5 +121,6 @@ struct tdmr_info_list {
};
int tdx_module_shutdown(void);
+int tdx_module_run_update(void);
#endif
--
2.47.3
^ permalink raw reply related
* [PATCH v6 14/22] x86/virt/seamldr: Do TDX per-CPU initialization after updates
From: Chao Gao @ 2026-03-26 8:44 UTC (permalink / raw)
To: linux-kernel, linux-coco, kvm
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Chao Gao, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260326084448.29947-1-chao.gao@intel.com>
After installing the new TDX module, each CPU needs to be initialized
again to make the CPU ready to run any other SEAMCALLs. So, call
tdx_cpu_enable() on all CPUs.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
arch/x86/virt/vmx/tdx/seamldr.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
index 0c282a7565a1..8b196ce45546 100644
--- a/arch/x86/virt/vmx/tdx/seamldr.c
+++ b/arch/x86/virt/vmx/tdx/seamldr.c
@@ -201,6 +201,7 @@ enum module_update_state {
MODULE_UPDATE_START,
MODULE_UPDATE_SHUTDOWN,
MODULE_UPDATE_CPU_INSTALL,
+ MODULE_UPDATE_CPU_INIT,
MODULE_UPDATE_DONE,
};
@@ -261,6 +262,9 @@ static int do_seamldr_install_module(void *seamldr_params)
case MODULE_UPDATE_CPU_INSTALL:
ret = seamldr_install(seamldr_params);
break;
+ case MODULE_UPDATE_CPU_INIT:
+ ret = tdx_cpu_enable();
+ break;
default:
break;
}
--
2.47.3
^ permalink raw reply related
* [PATCH v6 12/22] x86/virt/tdx: Reset software states during TDX module shutdown
From: Chao Gao @ 2026-03-26 8:44 UTC (permalink / raw)
To: linux-kernel, linux-coco, kvm
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Chao Gao, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260326084448.29947-1-chao.gao@intel.com>
The TDX module requires a one-time global initialization (TDH.SYS.INIT) and
per-CPU initialization (TDH.SYS.LP.INIT) before use. These initializations
are guarded by software flags to prevent repetition.
After TDX module updates, the new TDX module requires the same global and
per-CPU initializations, but the existing software flags prevent
re-initialization.
Reset all software flags guarding the initialization flows to allow the
global and per-CPU initializations to be triggered again after updates.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
---
v6:
- reset tdx_lp_initialized for offlined CPUs and update the comment
accordingly [Kai]
v5:
- add a comment to clarify why state access doesn't require holding a
lock. [Kai]
---
arch/x86/virt/vmx/tdx/tdx.c | 23 ++++++++++++++++++++---
1 file changed, 20 insertions(+), 3 deletions(-)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index f87fad429f4e..a2a46c734d5e 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -55,6 +55,8 @@ static struct tdmr_info_list tdx_tdmr_list;
static enum tdx_module_status_t tdx_module_status;
static DEFINE_MUTEX(tdx_module_lock);
+static bool sysinit_done;
+static int sysinit_ret;
/* All TDX-usable memory regions. Protected by mem_hotplug_lock. */
static LIST_HEAD(tdx_memlist);
@@ -70,8 +72,6 @@ static int try_init_module_global(void)
{
struct tdx_module_args args = {};
static DEFINE_RAW_SPINLOCK(sysinit_lock);
- static bool sysinit_done;
- static int sysinit_ret;
lockdep_assert_irqs_disabled();
@@ -1179,6 +1179,7 @@ EXPORT_SYMBOL_FOR_KVM(tdx_enable);
int tdx_module_shutdown(void)
{
struct tdx_module_args args = {};
+ int ret, cpu;
/*
* Shut down the TDX module and prepare handoff data for the next
@@ -1188,7 +1189,23 @@ int tdx_module_shutdown(void)
* modules as new modules likely have higher handoff version.
*/
args.rcx = tdx_sysinfo.handoff.module_hv;
- return seamcall_prerr(TDH_SYS_SHUTDOWN, &args);
+ ret = seamcall_prerr(TDH_SYS_SHUTDOWN, &args);
+ if (ret)
+ return ret;
+
+ tdx_module_status = TDX_MODULE_UNINITIALIZED;
+ sysinit_done = false;
+ sysinit_ret = 0;
+
+ /*
+ * Since the TDX module is shut down and gone, mark all CPUs
+ * (including offlined ones) as uninitialied. This is called in
+ * stop_machine() (where CPU hotplug is disabled), preventing
+ * races with other tdx_lp_initialized accesses.
+ */
+ for_each_possible_cpu(cpu)
+ per_cpu(tdx_lp_initialized, cpu) = false;
+ return 0;
}
static bool is_pamt_page(unsigned long phys)
--
2.47.3
^ permalink raw reply related
* [PATCH v6 11/22] x86/virt/seamldr: Shut down the current TDX module
From: Chao Gao @ 2026-03-26 8:44 UTC (permalink / raw)
To: linux-kernel, linux-coco, kvm
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Chao Gao, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260326084448.29947-1-chao.gao@intel.com>
The first step of TDX module updates is shutting down the current TDX
Module. This step also packs state information that needs to be
preserved across updates as handoff data, which will be consumed by the
updated module. The handoff data is stored internally in the SEAM range
and is hidden from the kernel.
To ensure a successful update, the new module must be able to consume
the handoff data generated by the old module. Since handoff data layout
may change between modules, the handoff data is versioned. Each module
has a native handoff version and provides backward support for several
older versions.
The complete handoff versioning protocol is complex as it supports both
module upgrades and downgrades. See details in Intel® Trust Domain
Extensions (Intel® TDX) Module Base Architecture Specification, Chapter
"Handoff Versioning".
Ideally, the kernel needs to retrieve the handoff versions supported by
the current module and the new module and select a version supported by
both. But, since this implementation chooses to only support module
upgrades, simply request the current module to generate handoff data
using its highest supported version, expecting that the new module will
likely support it.
Note that only one CPU needs to call the TDX module's shutdown API.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
v5:
- Massage changelog [Kai]
- Avoid "refers to the global copy while populating the tdx_sys_info
passed as a pointer" [Rick/Yilun]
v4:
- skip the whole handoff metadata if runtime updates are not supported
[Yilun]
v3:
- remove autogeneration stuff in the changelog
v2:
- add a comment about how handoff version is chosen.
- remove the first !ret in get_tdx_sys_info_handoff() as we edited the
auto-generated code anyway
- remove !! when determining whether a CPU is the primary one
- remove unnecessary if-break nesting in TDP_SHUTDOWN
---
arch/x86/include/asm/tdx_global_metadata.h | 5 +++++
arch/x86/virt/vmx/tdx/seamldr.c | 11 ++++++++++-
arch/x86/virt/vmx/tdx/tdx.c | 15 +++++++++++++++
arch/x86/virt/vmx/tdx/tdx.h | 3 +++
arch/x86/virt/vmx/tdx/tdx_global_metadata.c | 20 ++++++++++++++++++++
5 files changed, 53 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/tdx_global_metadata.h b/arch/x86/include/asm/tdx_global_metadata.h
index 40689c8dc67e..8a9ebd895e70 100644
--- a/arch/x86/include/asm/tdx_global_metadata.h
+++ b/arch/x86/include/asm/tdx_global_metadata.h
@@ -40,12 +40,17 @@ struct tdx_sys_info_td_conf {
u64 cpuid_config_values[128][2];
};
+struct tdx_sys_info_handoff {
+ u16 module_hv;
+};
+
struct tdx_sys_info {
struct tdx_sys_info_version version;
struct tdx_sys_info_features features;
struct tdx_sys_info_tdmr tdmr;
struct tdx_sys_info_td_ctrl td_ctrl;
struct tdx_sys_info_td_conf td_conf;
+ struct tdx_sys_info_handoff handoff;
};
#endif
diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
index 771671b7755b..a8fd29818378 100644
--- a/arch/x86/virt/vmx/tdx/seamldr.c
+++ b/arch/x86/virt/vmx/tdx/seamldr.c
@@ -15,6 +15,7 @@
#include <asm/seamldr.h>
#include "seamcall_internal.h"
+#include "tdx.h"
/* P-SEAMLDR SEAMCALL leaf function */
#define P_SEAMLDR_INFO 0x8000000000000000
@@ -190,6 +191,7 @@ static struct seamldr_params *init_seamldr_params(const u8 *data, u32 size)
*/
enum module_update_state {
MODULE_UPDATE_START,
+ MODULE_UPDATE_SHUTDOWN,
MODULE_UPDATE_DONE,
};
@@ -229,8 +231,12 @@ static void ack_state(void)
static int do_seamldr_install_module(void *seamldr_params)
{
enum module_update_state newstate, curstate = MODULE_UPDATE_START;
+ int cpu = smp_processor_id();
+ bool primary;
int ret = 0;
+ primary = cpumask_first(cpu_online_mask) == cpu;
+
do {
/* Chill out and re-read update_data. */
cpu_relax();
@@ -239,7 +245,10 @@ static int do_seamldr_install_module(void *seamldr_params)
if (newstate != curstate) {
curstate = newstate;
switch (curstate) {
- /* TODO: add the update steps. */
+ case MODULE_UPDATE_SHUTDOWN:
+ if (primary)
+ ret = tdx_module_shutdown();
+ break;
default:
break;
}
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 172f6d4133b5..f87fad429f4e 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -1176,6 +1176,21 @@ int tdx_enable(void)
}
EXPORT_SYMBOL_FOR_KVM(tdx_enable);
+int tdx_module_shutdown(void)
+{
+ struct tdx_module_args args = {};
+
+ /*
+ * Shut down the TDX module and prepare handoff data for the next
+ * TDX module. This SEAMCALL requires a handoff version. Use the
+ * module's handoff version, as it is the highest version the
+ * module can produce and is more likely to be supported by new
+ * modules as new modules likely have higher handoff version.
+ */
+ args.rcx = tdx_sysinfo.handoff.module_hv;
+ return seamcall_prerr(TDH_SYS_SHUTDOWN, &args);
+}
+
static bool is_pamt_page(unsigned long phys)
{
struct tdmr_info_list *tdmr_list = &tdx_tdmr_list;
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index 82bb82be8567..1c4da9540ae0 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -46,6 +46,7 @@
#define TDH_PHYMEM_PAGE_WBINVD 41
#define TDH_VP_WR 43
#define TDH_SYS_CONFIG 45
+#define TDH_SYS_SHUTDOWN 52
/*
* SEAMCALL leaf:
@@ -118,4 +119,6 @@ struct tdmr_info_list {
int max_tdmrs; /* How many 'tdmr_info's are allocated */
};
+int tdx_module_shutdown(void);
+
#endif
diff --git a/arch/x86/virt/vmx/tdx/tdx_global_metadata.c b/arch/x86/virt/vmx/tdx/tdx_global_metadata.c
index 4c9917a9c2c3..1b6f9b80b197 100644
--- a/arch/x86/virt/vmx/tdx/tdx_global_metadata.c
+++ b/arch/x86/virt/vmx/tdx/tdx_global_metadata.c
@@ -100,6 +100,19 @@ static int get_tdx_sys_info_td_conf(struct tdx_sys_info_td_conf *sysinfo_td_conf
return ret;
}
+static int get_tdx_sys_info_handoff(struct tdx_sys_info_handoff *sysinfo_handoff)
+{
+ int ret;
+ u64 val;
+
+ ret = read_sys_metadata_field(0x8900000100000000, &val);
+ if (ret)
+ return ret;
+
+ sysinfo_handoff->module_hv = val;
+ return 0;
+}
+
static int get_tdx_sys_info(struct tdx_sys_info *sysinfo)
{
int ret = 0;
@@ -116,5 +129,12 @@ static int get_tdx_sys_info(struct tdx_sys_info *sysinfo)
ret = ret ?: get_tdx_sys_info_td_ctrl(&sysinfo->td_ctrl);
ret = ret ?: get_tdx_sys_info_td_conf(&sysinfo->td_conf);
+ /*
+ * Don't treat a module that doesn't support update as a failure.
+ * Only read the metadata optionally.
+ */
+ if (tdx_supports_runtime_update(sysinfo))
+ ret = ret ?: get_tdx_sys_info_handoff(&sysinfo->handoff);
+
return ret;
}
--
2.47.3
^ permalink raw reply related
* [PATCH v6 13/22] x86/virt/seamldr: Install a new TDX module
From: Chao Gao @ 2026-03-26 8:44 UTC (permalink / raw)
To: linux-kernel, linux-coco, kvm
Cc: binbin.wu, dan.j.williams, dave.hansen, ira.weiny, kai.huang, kas,
nik.borisov, paulmck, pbonzini, reinette.chatre, rick.p.edgecombe,
sagis, seanjc, tony.lindgren, vannapurve, vishal.l.verma,
yilun.xu, xiaoyao.li, yan.y.zhao, Chao Gao, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260326084448.29947-1-chao.gao@intel.com>
Following the shutdown of the existing TDX module, the update process
continues with installing the new module. P-SEAMLDR provides the
SEAMLDR.INSTALL SEAMCALL to perform this installation, which must be
executed on all CPUs.
Implement SEAMLDR.INSTALL and execute it on every CPU.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
---
v6:
- wrap seamldr_call(P_SEAMLDR_INSTALL..) in a helper [Kiryl]
v5:
- drop "serially" from the changelog as it doesn't matter to
this patch
---
arch/x86/virt/vmx/tdx/seamldr.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
index a8fd29818378..0c282a7565a1 100644
--- a/arch/x86/virt/vmx/tdx/seamldr.c
+++ b/arch/x86/virt/vmx/tdx/seamldr.c
@@ -19,6 +19,7 @@
/* P-SEAMLDR SEAMCALL leaf function */
#define P_SEAMLDR_INFO 0x8000000000000000
+#define P_SEAMLDR_INSTALL 0x8000000000000001
#define SEAMLDR_MAX_NR_MODULE_4KB_PAGES 496
#define SEAMLDR_MAX_NR_SIG_4KB_PAGES 4
@@ -73,6 +74,13 @@ int seamldr_get_info(struct seamldr_info *seamldr_info)
}
EXPORT_SYMBOL_FOR_MODULES(seamldr_get_info, "tdx-host");
+static int seamldr_install(const struct seamldr_params *params)
+{
+ struct tdx_module_args args = { .rcx = __pa(params) };
+
+ return seamldr_call(P_SEAMLDR_INSTALL, &args);
+}
+
static void free_seamldr_params(struct seamldr_params *params)
{
free_page((unsigned long)params);
@@ -192,6 +200,7 @@ static struct seamldr_params *init_seamldr_params(const u8 *data, u32 size)
enum module_update_state {
MODULE_UPDATE_START,
MODULE_UPDATE_SHUTDOWN,
+ MODULE_UPDATE_CPU_INSTALL,
MODULE_UPDATE_DONE,
};
@@ -249,6 +258,9 @@ static int do_seamldr_install_module(void *seamldr_params)
if (primary)
ret = tdx_module_shutdown();
break;
+ case MODULE_UPDATE_CPU_INSTALL:
+ ret = seamldr_install(seamldr_params);
+ break;
default:
break;
}
--
2.47.3
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox