Preserving TDX module during kexec with live update

Preserving TDX module during kexec with live update

[Sagi Shahar]

Hi,

I started looking into preserving the TDX module across live updates
and I'm not sure what the best approach is. I tried looking into using
flb as David M suggested but I don't think this approach works for the
TDX module.

Right now the way that TDX supports kexec is by disabling the TDX
module on kexec and then reinitilizing it on the next boot [1]. This
approach clears the TDX module state so any TDX VM from the source
kernel won't be able to resume running on the destination kernel.

In order to support live update while keeping the VMs alive we need to
skip the TDH_SYS_DISABLE call and instead preserve pages used by the
TDX module such as the TDMR pages from the source kernel.

From what I understand, using flb will tie the TDX module save/restore
to the lifecycle of the VMs depending on it. This means the module
will be restored too late during kernel boot, after the VMs are
restored using the live update IOCTLs.

I'm thinking we should preserve the TDX module during the shutdown
stage of the source kernel. Perhaps we can preserve the pages used by
the TDX module as we allocate them on the source kernel and pass
pointers and additional metadata during KHO. Then on the destination
we need to pipe this information to where the TDX module gets
initialized during the KVM module initialization.

I haven't found a device that acts in a similar way so I'm not sure
how complicated this will be or if there's a better approach.

I would love to hear your thoughts and if you have any pointers on
where this code can be implemented.

Thanks,
Sagi

[1] https://lore.kernel.org/all/20260323-fuller_tdx_kexec_support-v2-0-87a36409e051@intel.com/

Re: Preserving TDX module during kexec with live update

[Pasha Tatashin]

On 07-15 18:28, Sagi Shahar wrote:
> Hi,
> 
> I started looking into preserving the TDX module across live updates
> and I'm not sure what the best approach is. I tried looking into using
> flb as David M suggested but I don't think this approach works for the
> TDX module.
> 
> Right now the way that TDX supports kexec is by disabling the TDX
> module on kexec and then reinitilizing it on the next boot [1]. This
> approach clears the TDX module state so any TDX VM from the source
> kernel won't be able to resume running on the destination kernel.
> 
> In order to support live update while keeping the VMs alive we need to
> skip the TDH_SYS_DISABLE call and instead preserve pages used by the
> TDX module such as the TDMR pages from the source kernel.

Agreed.

> 
> From what I understand, using flb will tie the TDX module save/restore
> to the lifecycle of the VMs depending on it. This means the module
> will be restored too late during kernel boot, after the VMs are
> restored using the live update IOCTLs.

That is not correct. FLBs specifically can be restored very early in 
boot. We do that for HugeTLB preservation, for example.

liveupdate_flb_get_incoming() does not require userspace to resume or 
restore VMs. You can call it super early in boot to see if there is FLB 
data present and, if so, change the way the module is initialized during 
boot.

> 
> I'm thinking we should preserve the TDX module during the shutdown
> stage of the source kernel. Perhaps we can preserve the pages used by
> the TDX module as we allocate them on the source kernel and pass
> pointers and additional metadata during KHO. Then on the destination
> we need to pipe this information to where the TDX module gets
> initialized during the KVM module initialization.

Please don't invent another state machine; instead, use the existing 
one.

Pasha

> 
> I haven't found a device that acts in a similar way so I'm not sure
> how complicated this will be or if there's a better approach.
> 
> I would love to hear your thoughts and if you have any pointers on
> where this code can be implemented.
> 
> Thanks,
> Sagi
> 
> [1] https://lore.kernel.org/all/20260323-fuller_tdx_kexec_support-v2-0-87a36409e051@intel.com/