Re: TDG quote analysis

[Peter Fang <peter.fang@intel.com>]

On Fri, Sep 18, 2026 at 02:32:23PM -0700, Sean Christopherson wrote:
> On Fri, Sep 18, 2026, Rick P Edgecombe wrote:
> > On Fri, 2026-09-18 at 06:13 -0700, Sean Christopherson wrote:
> > >  But given that you say "This idea came up before actually", maybe chunking
> > > the quote operation isn't a big lift?
> > 
> > The idea that came up was to have a saved state are per-TD. But this only
> > eliminated concurrency issues in the SW based flow. In the future a HW based
> > flow would be S3M limited. Which is why I keep saying a redesigned solution
> > should be robust to concurrency limitations.
> 
> IMO, that's the complete wrong way to look at this.  HW-based is inherently
> serialized, it does NOT have concurrency, period.  SW-based is has no inherent
> concurrency restrictions beyond existing CPU contention.
> 
> Trying to find a perfect one-size-fits-all solution is impossible, because the
> problems with each are so very different.  The problem you want to solve for
> SW-based is how to support preemption.  The problem you want to solve for HW-based
                             ^ more discussion below

> is how to hide the fact that someone in the future might think following AMD's
> lead and putting a precious resource into a tiny microprosser on a slow bus is a
> fantastic idea.

As shocking as it sounds, there are actually cloud providers out there
that are wanting this, or at the very least wanting to see how it goes
in real deployment. I think memory side channel attacks are a real
concern for some of them. But yeah this is not great from a performance
perspective.

> 
> By creating these system-wide thread pools, TDX has effectively created a bizarre
> M:N scheduling problem, *and* introduced a completely avoidable noisy-neighbor
> problem.

Hmm... Then how about M:M? Then the problem can be reduced to 1:1. I
think it would take some TDX module investigative work to say for sure
what this would look like. But just throwing it out there as an idea?

> 
> Assuming my understanding is (finally) correct, and the SW-based flows do all the
> work on the CPU, then the only way making the SW-based flows asynchronous adds
> value is if the host is willing to set aside CPU cores for such chores.  And for
> the use cases where TDX makes sense, AFAIK no CSP *wants* to do that.  Not to
> mention the RFC doesn't even support that, because KVM doesn't resume the vCPU
> until the quote is ready.

There was also the fairness/starvation problem at play. In RFC if the
vCPU thread waited on the lock (say using down_timeout(), or even some
kind of fancier waitqueue) and gave it up to reenter TD to handle a
guest interrupt, it would have to retry the lock again later. And then
there would be preemption-retry livelock risks. Moving this lock into
the TDX module could still create the same problem, because basically
the guest would be re-calling the TDCALL after preemption instead. So I
think there would need to be some sort of additional smarts to handle
both preemption and fairness at the same time.

> 
> > But now I'm wondering what your specific interest is. It seems you have more
> > interest in non-KVM in-the-loop guest side latency than you did on host side.
> > Which is fine. But it makes me want to double check: You want an average and
> > worse case latency for the first DICE attestation "SW" flow?
> 
> I care about future me not getting pulled into a customer issue because a vCPU
> got waylaid by a system-wide mutex for multiple seconds.

I think quoting is typically a once-per-TD-boot type of thing. Or it is
not usually expected to happen very often in a TD. Can you share what
kind of performance expectations you have for this? E.g. is seeing a
high steal time during TD boot a non-starter?

Thanks,
Peter