Re: [PATCH 12/16] KVM: TDX: Inhibit APICv for TDX guest

[Sean Christopherson <seanjc@google.com>]

On Mon, Jan 06, 2025, Vishal Annapurve wrote:
> On Sun, Jan 5, 2025 at 5:46 PM Binbin Wu <binbin.wu@linux.intel.com> wrote:
> > On 1/4/2025 5:59 AM, Vishal Annapurve wrote:
> > > On Sun, Dec 8, 2024 at 5:11 PM Binbin Wu <binbin.wu@linux.intel.com> wrote:
> > >> diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> > >> index b0f525069ebd..b51d2416acfb 100644
> > >> --- a/arch/x86/kvm/vmx/tdx.c
> > >> +++ b/arch/x86/kvm/vmx/tdx.c
> > >> @@ -2143,6 +2143,8 @@ static int __tdx_td_init(struct kvm *kvm, struct td_params *td_params,
> > >>                  goto teardown;
> > >>          }
> > >>
> > >> +       kvm_set_apicv_inhibit(kvm, APICV_INHIBIT_REASON_TDX);
> > >> +
> > >>          return 0;
> > >>
> > >>          /*
> > >> @@ -2528,6 +2530,7 @@ static int tdx_td_vcpu_init(struct kvm_vcpu *vcpu, u64 vcpu_rcx)
> > >>                  return -EIO;
> > >>          }
> > >>
> > >> +       vcpu->arch.apic->apicv_active = false;
> > > With this setting, apic_timer_expired[1] will always cause timer
> > > interrupts to be pending without injecting them right away. Injecting
> > > it after VM exit [2] could cause unbounded delays to timer interrupt
> > > injection.
> >
> > When apic->apicv_active is false, it will fallback to increasing the
> > apic->lapic_timer.pending and request KVM_REQ_UNBLOCK.
> > If apic_timer_expired() is called from timer function, the target vCPU
> > will be kicked out immediately.
> > So there is no unbounded delay to timer interrupt injection.
> 
> Ack. Though, wouldn't it be faster to just post timer interrupts right
> away without causing vcpu exit?

Yes, but if and only if hrtimers are offloaded to dedicated "housekeeping" CPUs.
If the hrtimer is running on the same pCPU that the vCPU is running on, then the
expiration of the underlying hardware timer (in practice, the "real" APIC timer)
will generate a host IRQ and thus a VM-Exit.  I.e. the vCPU will already be kicked
into the host, and the virtual timer IRQ will be delivered prior to re-entering
the guest.

Note, kvm_use_posted_timer_interrupt() uses a heuristic of HLT/MWAIT interception
being disabled to detect that it's worth trying to post a timer interrupt, but off
the top of my head I'm pretty sure that's unnecessary and pointless.  The
"vcpu->mode == IN_GUEST_MODE" is super cheap, and I can't think of any harm in
posting the time interrupt if the target vCPU happens to be in guest mode even
if the host wasn't configured just so.

> Another scenario I was thinking of was hrtimer expiry during vcpu exit
> being handled in KVM/userspace, which will cause timer interrupt
> injection after the next exit [1] delaying timer interrupt to guest.

No, the IRQ won't be delayed.  Expiration from the hrtimer callback will set
KVM_REQ_UNBLOCK, which will prevent actually entering the guest (see the call
to kvm_request_pending() in kvm_vcpu_exit_request()).

> This scenario is not specific to TDX VMs though.
> 
> [1] https://git.kernel.org/pub/scm/virt/kvm/kvm.git/tree/arch/x86/kvm/x86.c?h=kvm-coco-queue#n11263