Kernel KVM virtualization development
 help / color / mirror / Atom feed
From: Sean Christopherson <seanjc@google.com>
To: Jaroslav Pulchart <jaroslav.pulchart@gooddata.com>
Cc: Thorsten Leemhuis <regressions@leemhuis.info>,
	Lei Chen <lei.chen@smartx.com>,
	igor@gooddata.com,  jan.cipa@gooddata.com, kvm@vger.kernel.org,
	linux-kernel@vger.kernel.org,  pbonzini@redhat.com,
	 Linux kernel regressions list <regressions@lists.linux.dev>,
	 Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: [PATCH v2] KVM: x86: Rate-limit global clock updates on vCPU load
Date: Thu, 7 May 2026 12:09:09 -0700	[thread overview]
Message-ID: <afzjVYupUl-nlaDK@google.com> (raw)
In-Reply-To: <CAK8fFZ7LEyyR2ChXCD7B8vjvDcyG1c4jtv3wYMV4GCiUkOi1Pw@mail.gmail.com>

On Thu, May 07, 2026, Jaroslav Pulchart wrote:
> > On Wed, May 06, 2026, Jaroslav Pulchart wrote:
> > > > On Wed, May 06, 2026, Thorsten Leemhuis wrote:
> > I think the only remaining question is why/how KVM's master clock is getting
> > disabled.  But that's more of a question for your deployment than it is a question
> > for upstream; it's possible there's a different KVM bug lurking, but it's more
> > likely that something in your setup is incompatible with using the master clock.
> >
> > Note, it's certainly not "wrong" for the master clock to be disabled, but it's
> > quite suprising, especially for Firecracker VMs.  It's worth investigating as
> > there might be an underlying issue that's very easy to address, and "fixing" it
> > should provide (very) small performance benefits.
> 
> I've dug into the "master clock question" and have an idea.
> 
> Our Firecracker hosts are themselves L1 KVM VMs (nested
> virtualisation) running on AMD EPYC 9454P and EPYC 9455 hardware. Even
> though the compute nodes use cpu_mode=host-passthrough in qemu kvm,
> the invtsc CPUID bit is filtered out by QEMU, which I hadn't realized.
> Without it the guest kernel marks the TSC unstable at boot:
>     tsc: Marking TSC unstable due to TSCs unsynchronized
> and falls back to kvm-clock as its clocksource.
> 
> I suppose that in turn prevents KVM from enabling the master clock for
> any L2 guests (the Firecracker microVMs), am I right?
> 
> I have resolved the issue by explicitly adding +invtsc to
> cpu_model_extra_flags in our OpenStack nova.conf. After this change
> the L1 VMs now correctly show constant_tsc and nonstop_tsc in
> /proc/cpuinfo and switch clocksource to tsc. I also confirmed the IPI
> storm disappears without the v2 patch when +invtsc is present, and
> returns when it is absent on a vanilla 7.0.3 kernel.
> 
> So could this be the answer to your question: "the master clock was
> disabled because QEMU silently drops invtsc even in host-passthrough
> mode"?

Yep, that'd do it.   Linux-as-a-guest will prefer kvmclock over TSC if the TSC
isnt constant and non-stop.  That in turn will prevent KVM (as the L1 hypervisor)
from using the master clock, since it sees the kernel clocksource as not being
(directly) based on TSC.

Thanks for the follow-up!

  reply	other threads:[~2026-05-07 19:09 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-03-21 14:32 [REGRESSION 6.19, BISECTED] KVM: x86: kvmclock rate-limit removal causes IPI storm and high guest steal time Jaroslav Pulchart
2026-03-23  2:27 ` Lei Chen
2026-04-01  6:43   ` Lei Chen
2026-04-01 21:16     ` Sean Christopherson
2026-04-07  7:00       ` [PATCH v1] KVM: x86: Rate-limit global clock updates on vCPU load Lei Chen
2026-04-07 18:02         ` Sean Christopherson
2026-04-09 13:03           ` Lei Chen
2026-04-09 13:36           ` Lei Chen
2026-04-09 14:22           ` [PATCH v2] " Lei Chen
2026-04-09 19:21             ` Sean Christopherson
2026-05-06  9:48               ` Thorsten Leemhuis
2026-05-06 12:55                 ` Sean Christopherson
2026-05-06 14:09                   ` Thorsten Leemhuis
2026-05-06 15:22                     ` Sean Christopherson
2026-05-06 15:58                       ` Jaroslav Pulchart
2026-05-06 20:31                         ` Sean Christopherson
2026-05-07  9:27                           ` Jaroslav Pulchart
2026-05-07 19:09                             ` Sean Christopherson [this message]
2026-05-06 20:10             ` Jaroslav Pulchart

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=afzjVYupUl-nlaDK@google.com \
    --to=seanjc@google.com \
    --cc=igor@gooddata.com \
    --cc=jan.cipa@gooddata.com \
    --cc=jaroslav.pulchart@gooddata.com \
    --cc=kvm@vger.kernel.org \
    --cc=lei.chen@smartx.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=pbonzini@redhat.com \
    --cc=regressions@leemhuis.info \
    --cc=regressions@lists.linux.dev \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox