Re: [PATCH v9 07/18] x86/virt/tdx: Do TDX module per-cpu initialization

Huang, Kai
2023-02-13T23:43:55Z

On Mon, 2023-02-13 at 14:28 -0800, Dave Hansen wrote:
> On 2/13/23 13:13, Huang, Kai wrote:
> > Perhaps I didn't explain clearly in the comment. Below is the updated one:
> >
> > /*
> > * The previous call of __tdx_enable() may only have
> > * initialized part of present cpus during module
> > * initialization, and new cpus may have become online
> > * since then w/o doing per-cpu initialization.
> > *
> > * For example, a new CPU can become online when KVM is
> > * unloaded, in which case tdx_cpu_enable() is not called since
> > * KVM's CPU online callback has been removed.
> > *
> > * To make sure all online cpus are TDX-runnable, always
> > * do per-cpu initialization for all online cpus here
> > * even the module has been initialized.
> > */
>
> This is voodoo.
>
> I want a TDX-specific hotplug CPU handler. Period. Please make that
> happen.
>

Yes 100% agreed.

> Put that code in this patch. That handler should:
>
> 1. Run after the KVM handler (if present)
> 2. See if VMX is on
> 3. If VMX is on:
> 3a. Run smp_func_module_lp_init(), else
> 3b. Mark the CPU as needing smp_func_module_lp_init()
>
> Then, in the 'case TDX_MODULE_INITIALIZED:', you call a function to
> iterate over the cpumask that was generated in 3b.
>
> That makes the handoff *EXPLICIT*. You know exactly which CPUs need
> what done to them. A CPU hotplug either explicitly involves doing the
> work to make TDX work on the CPU, or explicitly defers the work to a
> specific later time in a specific later piece of code.

In 3b. we don't need to "explicitly mark the CPU as needing
smp_func_module_lp_init()". We already have __cpu_tdx_mask to track whether
LP.INIT has been done on one cpu and we can use that to determine:

Any online cpu which isn't set in __cpu_tdx_mask needs to do LP.INIT in
tdx_enable().

And the function module_lp_init_online_cpus() already handles that, and it can
be called directly in tdx_enable() path (as shown in this patch).

I'll do above as you suggested, but just use __cpu_tdx_mask as explained above.

( My main concern is "Run after the KVM handler" seems a little bit hacky to me.
Logically, it's more reasonable to have the TDX callback _before_ KVM's but not
_after_. If any user (KVM) has done tdx_enable() successfully, the TDX code
should give the user a "TDX-runnable" cpu before user (KVM)'s own callback is
involved. Anyway as mentioned above, I'll do above as you suggested.)