public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: Sean Christopherson <seanjc@google.com>
To: Jaroslav Pulchart <jaroslav.pulchart@gooddata.com>
Cc: Thorsten Leemhuis <regressions@leemhuis.info>,
	Lei Chen <lei.chen@smartx.com>,
	igor@gooddata.com,  jan.cipa@gooddata.com, kvm@vger.kernel.org,
	linux-kernel@vger.kernel.org,  pbonzini@redhat.com,
	 Linux kernel regressions list <regressions@lists.linux.dev>,
	 Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: [PATCH v2] KVM: x86: Rate-limit global clock updates on vCPU load
Date: Wed, 6 May 2026 13:31:49 -0700	[thread overview]
Message-ID: <afulNSVKMzrx35i_@google.com> (raw)
In-Reply-To: <CAK8fFZ4j1tYc+eYH79RQnNSjr9J-SocFL6duMuBg+76NHhRUxQ@mail.gmail.com>

On Wed, May 06, 2026, Jaroslav Pulchart wrote:
> > On Wed, May 06, 2026, Thorsten Leemhuis wrote:
> > > On 5/6/26 14:55, Sean Christopherson wrote:
> > > > On Wed, May 06, 2026, Thorsten Leemhuis wrote:
> > > >> On 4/9/26 21:21, Sean Christopherson wrote:
> > > >>> On Thu, Apr 09, 2026, Lei Chen wrote:
> > > >>>> commit 446fcce2a52b ("Revert "x86: kvm: rate-limit global clock updates"")
> > > >>>> dropped the rate limiting for KVM_REQ_GLOBAL_CLOCK_UPDATE.
> > > >>>>
> > > >>>> As a result, kvm_arch_vcpu_load() can queue global clock update requests
> > > >>>> every time a vCPU is scheduled when the master clock is disabled or when
> > > >>>> the vCPU is loaded for the first time.
> > > >>>>
> > > >>>> Restore the throttling with a per-VM ratelimit state and gate
> > > >>>> KVM_REQ_GLOBAL_CLOCK_UPDATE through __ratelimit(), so frequent vCPU
> > > >>>> scheduling does not generate a steady stream of redundant clock update
> > > >>>> requests.
> > > >>>>
> > > >>>> Fixes: 446fcce2a52b ("Revert "x86: kvm: rate-limit global clock updates"")
> > > >>>> Signed-off-by: Lei Chen <lei.chen@smartx.com>
> > > >>>> Reported-by: Jaroslav Pulchart <jaroslav.pulchart@gooddata.com>
> > > >>>> Closes: https://lore.kernel.org/all/CAK8fFZ5gY8_Mw2A=iZVFNVKQNrXQzVsn-HTd+Me9K6ZfmdgA+Q@mail.gmail.com/
> > > >>
> > > >> Was this performance regression ever addressed?
> > > > Nope, not yet.
> > > >
> > > >> Looks like this fall through the cracks, but it's easy to miss something.
> > > >
> > > > It's in my list of patches to apply (probably for 7.2?).  I didn't want to squeeze
> > > > it into the initial 7.1 pull request for a variety of reasons.
> > >
> > > Hmmm. CCing Linus so he can speak up if he wants to about the following:
> > >
> > > Given that this is a fix for a performance regression[1] I'd say it's
> > > not as urgent as a "something stopped working" case -- so I guess it's
> > > something where the "[fix] "within a week", preferably before the next
> > > rc" approach Linus recently mentioned does not need to be applied strictly.
> > >
> > > But Jaroslav OTOH reported it more than 7 weeks ago already and back
> > > then called it something that "severely impacts KVM hosts running many
> > > Firecracker microVMs."[1];
> >
> > For a setup that is likely broken.  On modern hardware, the path in question
> > should never actually be hit.  I do want to resolve the bug since older hardware
> > and funky setups do rely on the old behavior, but it's not pants-on-fire urgent.
> >
> > More importantly, the original reporter(s) hasn't responded to any of our questions,
> > or to the proposed fix.  I'm not going to rush in a fix if I don't actually *know*
> > it's going to fix the original problem.
> 
> Hi Sean, Thorsten,
> 
> sorry for the missing response from my side, this thread unfortunately
> ended up in trash due to mail filters on my side and I completely
> missed it.

No worries, gmail's Spam filter is my nemesis :-)

> I currently don't have the full context loaded back in yet, but I'll re-read
> the thread and follow up properly once I do.

I think the only remaining question is why/how KVM's master clock is getting
disabled.  But that's more of a question for your deployment than it is a question
for upstream; it's possible there's a different KVM bug lurking, but it's more
likely that something in your setup is incompatible with using the master clock.

Note, it's certainly not "wrong" for the master clock to be disabled, but it's
quite suprising, especially for Firecracker VMs.  It's worth investigating as
there might be an underlying issue that's very easy to address, and "fixing" it
should provide (very) small performance benefits.

  reply	other threads:[~2026-05-06 20:31 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-03-21 14:32 [REGRESSION 6.19, BISECTED] KVM: x86: kvmclock rate-limit removal causes IPI storm and high guest steal time Jaroslav Pulchart
2026-03-23  2:27 ` Lei Chen
2026-04-01  6:43   ` Lei Chen
2026-04-01 21:16     ` Sean Christopherson
2026-04-07  7:00       ` [PATCH v1] KVM: x86: Rate-limit global clock updates on vCPU load Lei Chen
2026-04-07 18:02         ` Sean Christopherson
2026-04-09 13:03           ` Lei Chen
2026-04-09 13:36           ` Lei Chen
2026-04-09 14:22           ` [PATCH v2] " Lei Chen
2026-04-09 19:21             ` Sean Christopherson
2026-05-06  9:48               ` Thorsten Leemhuis
2026-05-06 12:55                 ` Sean Christopherson
2026-05-06 14:09                   ` Thorsten Leemhuis
2026-05-06 15:22                     ` Sean Christopherson
2026-05-06 15:58                       ` Jaroslav Pulchart
2026-05-06 20:31                         ` Sean Christopherson [this message]
2026-05-06 20:10             ` Jaroslav Pulchart

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=afulNSVKMzrx35i_@google.com \
    --to=seanjc@google.com \
    --cc=igor@gooddata.com \
    --cc=jan.cipa@gooddata.com \
    --cc=jaroslav.pulchart@gooddata.com \
    --cc=kvm@vger.kernel.org \
    --cc=lei.chen@smartx.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=pbonzini@redhat.com \
    --cc=regressions@leemhuis.info \
    --cc=regressions@lists.linux.dev \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox