Re: [PATCH v4 1/4] KVM: TDX: Synchronize user-return MSRs immediately after VP.ENTER

Edgecombe, Rick P
2025-10-21T18:54:43Z

On Tue, 2025-10-21 at 08:06 -0700, Sean Christopherson wrote:
> I think we should be synchronizing only after a successful VP.ENTER with a real
> > > TD exit, but today instead we synchronize after any attempt to VP.ENTER.
>
> Well this is all completely @#($*#.  Looking at the TDX-Module source, if the
> TDX-Module synthesizes an exit, e.g. because it suspects a zero-step attack, it
> will signal a "normal" exit but not "restore" VMM state.

Oh yea, good point. So there is no way to tell from the return code if the
clobbering happened.

>
> > If the MSR's do not get clobbered, does it matter whether or not they get
> > restored.
>
> It matters because KVM needs to know the actual value in hardware.  If KVM thinks
> an MSR is 'X', but it's actually 'Y', then KVM could fail to write the correct
> value into hardware when returning to userspace and/or when running a different
> vCPU.
>
> Taking a step back, the entire approach of updating the "cache" after the fact is
> ridiculous.  TDX entry/exit is anything but fast; avoiding _at most_ 4x WRMSRs at
> the start of the run loop is a very, very premature optimization.  Preemptively
> load hardware with the value that the TDX-Module _might_ set and call it good.
>
> I'll replace patches 1 and 4 with this, tagged for stable@.

Seems reasonable to me in concept, but there is a bug. It looks like some
important MSR isn't getting restored right and the host gets into a bad state.
The first signs start with triggering this:

asmlinkage __visible noinstr struct pt_regs *fixup_bad_iret(struct pt_regs
*bad_regs)
{
struct pt_regs tmp, *new_stack;

/*
* This is called from entry_64.S early in handling a fault
* caused by a bad iret to user mode. To handle the fault
* correctly, we want to move our stack frame to where it would
* be had we entered directly on the entry stack (rather than
* just below the IRET frame) and we want to pretend that the
* exception came from the IRET target.
*/
new_stack = (struct pt_regs *)__this_cpu_read(cpu_tss_rw.x86_tss.sp0) -
1;

/* Copy the IRET target to the temporary storage. */
__memcpy(&tmp.ip, (void *)bad_regs->sp, 5*8);

/* Copy the remainder of the stack from the current stack. */
__memcpy(&tmp, bad_regs, offsetof(struct pt_regs, ip));

/* Update the entry stack */
__memcpy(new_stack, &tmp, sizeof(tmp));

BUG_ON(!user_mode(new_stack)); <---------------HERE

Need to debug.