[RFC PATCH 0/3] srcu: KVM: Add, export and use call_srcu_expedited()

[RFC PATCH 0/3] srcu: KVM: Add, export and use call_srcu_expedited()

[Sean Christopherson]

We've got a conundrum in KVM where we have multiple use cases that generally
want the same thing (eliminate waiting on guest configuration changes whenever
possible), but use KVM uAPIs in slightly different ways and effectively create
competing requirements.

The crux of the problem is that one use case wants KVM to free an object via
call_srcu() so that the task doesn't risk getting stalled waiting for a grace
period.  But for the other use case, using call_srcu() can trigger a
non-expedited grace period and cause a synchronize_srcu_expedited() in a
different ioctl (that must do a full sync, i.e. can't use call_srcu()) to stall
waiting for the non-expedited grace period.

Tagged RFC because while having the call_srcu() request do an expedited grace
period eliminates the unwanted synchronize_srcu_expedited() stalls, this feels
like a very crude fix.   That said, I'm definitely not opposed to this being a
final solution if it's the best option available.

Sean Christopherson (3):
  srcu: Declare exported symbols before including srcu{tiny,tree}.h
  srcu: Add and export call_srcu_expedited() to avoid transferring grace
    periods
  KVM: Expedite SRCU callbacks when freeing objects during I/O bus
    registration

 include/linux/srcu.h     | 10 +++++-----
 include/linux/srcutiny.h |  8 ++++++--
 include/linux/srcutree.h |  2 ++
 kernel/rcu/srcutree.c    |  7 +++++++
 virt/kvm/kvm_main.c      |  2 +-
 5 files changed, 21 insertions(+), 8 deletions(-)


base-commit: 5128b972fb2801ad9aca54d990a75611ab5283a9
-- 
2.53.0.473.g4a7958ca14-goog

[RFC PATCH 1/3] srcu: Declare exported symbols before including srcu{tiny,tree}.h

[Sean Christopherson]

Move the declarations of call_srcu(), cleanup_srcu_struct(), and
synchronize_srcu() above the inclusion of the implementation specific SRCU
header so that tiny SRCU can provide inline wrappers, e.g. for expedited
versions, without needing to re-declare synchronize_srcu() and call_srcu().

Opportunsitically use rcu_callback_t in the call_srcu() declaration instead
of an open coded equivalent (all implementations already use
rcu_callback_t).

No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 include/linux/srcu.h     | 10 +++++-----
 include/linux/srcutiny.h |  2 --
 2 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/include/linux/srcu.h b/include/linux/srcu.h
index bb44a0bd7696..1cbc37e3b59c 100644
--- a/include/linux/srcu.h
+++ b/include/linux/srcu.h
@@ -79,6 +79,11 @@ int init_srcu_struct_fast_updown(struct srcu_struct *ssp);
 						// instead of smp_mb().
 void __srcu_read_unlock(struct srcu_struct *ssp, int idx) __releases_shared(ssp);
 
+void call_srcu(struct srcu_struct *ssp, struct rcu_head *head,
+	       rcu_callback_t func);
+void cleanup_srcu_struct(struct srcu_struct *ssp);
+void synchronize_srcu(struct srcu_struct *ssp);
+
 #ifdef CONFIG_TINY_SRCU
 #include <linux/srcutiny.h>
 #elif defined(CONFIG_TREE_SRCU)
@@ -87,11 +92,6 @@ void __srcu_read_unlock(struct srcu_struct *ssp, int idx) __releases_shared(ssp)
 #error "Unknown SRCU implementation specified to kernel configuration"
 #endif
 
-void call_srcu(struct srcu_struct *ssp, struct rcu_head *head,
-		void (*func)(struct rcu_head *head));
-void cleanup_srcu_struct(struct srcu_struct *ssp);
-void synchronize_srcu(struct srcu_struct *ssp);
-
 #define SRCU_GET_STATE_COMPLETED 0x1
 
 /**
diff --git a/include/linux/srcutiny.h b/include/linux/srcutiny.h
index dec7cbe015aa..4976536e8b28 100644
--- a/include/linux/srcutiny.h
+++ b/include/linux/srcutiny.h
@@ -64,8 +64,6 @@ struct srcu_usage { };
 #define init_srcu_struct_fast_updown init_srcu_struct
 #endif // #ifndef CONFIG_DEBUG_LOCK_ALLOC
 
-void synchronize_srcu(struct srcu_struct *ssp);
-
 /*
  * Counts the new reader in the appropriate per-CPU element of the
  * srcu_struct.  Can be invoked from irq/bh handlers, but the matching
-- 
2.53.0.473.g4a7958ca14-goog

[RFC PATCH 2/3] srcu: Add and export call_srcu_expedited() to avoid transferring grace periods

[Sean Christopherson]

Add and export an expedited version of call_srcu() so that users of
synchronize_srcu_expedited() can avoid "transferring" non-expedited grace
periods.  In response to userspace changes to guest devices and memory,
KVM uses call_srcu() when freeing an object to avoid having to wait for
readers to go away, but then often emits a synchronize_srcu_expedited() in
a largely unrelated path shortly thereafter (on the same SRCU object).

Due to differences in how VMMs manage guest devices, and in the
architecture being emulated by userspace, some updates trigger call_srcu()
with concurrent readers (i.e. while the VM is active), while others occur
without readers, e.g. when configuring devices during a pre-boot setup.
For the later case (no concurrent readers), using the vanilla call_srcu()
is problematic, as it can kick off a normal grace period (totally fine for
freeing the object) and effectively transfer the non-expedited grace period
to the upcoming synchronize_srcu_expedited().

For micro-VM use cases with CONFIG_HZ=100 kernels, the resulting ~20ms
delay on the would-be-expedited sync can increase the boot time of the VM
by 15% or more.

Link: https://lore.kernel.org/all/a84ddba8-12da-489a-9dd1-ccdf7451a1ba@amazon.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 include/linux/srcutiny.h | 6 ++++++
 include/linux/srcutree.h | 2 ++
 kernel/rcu/srcutree.c    | 7 +++++++
 3 files changed, 15 insertions(+)

diff --git a/include/linux/srcutiny.h b/include/linux/srcutiny.h
index 4976536e8b28..e2fc8c138e6a 100644
--- a/include/linux/srcutiny.h
+++ b/include/linux/srcutiny.h
@@ -130,6 +130,12 @@ static inline void srcu_barrier(struct srcu_struct *ssp)
 	synchronize_srcu(ssp);
 }
 
+static inline void call_srcu_expedited(struct srcu_struct *ssp, struct rcu_head *rhp,
+				       rcu_callback_t func)
+{
+	call_srcu(ssp, rhp, func);
+}
+
 static inline void srcu_expedite_current(struct srcu_struct *ssp) { }
 #define srcu_check_read_flavor(ssp, read_flavor) do { } while (0)
 
diff --git a/include/linux/srcutree.h b/include/linux/srcutree.h
index 958cb7ef41cb..ed3cbbe7f5ce 100644
--- a/include/linux/srcutree.h
+++ b/include/linux/srcutree.h
@@ -234,6 +234,8 @@ struct srcu_struct {
 					__DEFINE_SRCU(name, SRCU_READ_FLAVOR_FAST_UPDOWN, static)
 
 int __srcu_read_lock(struct srcu_struct *ssp) __acquires_shared(ssp);
+void call_srcu_expedited(struct srcu_struct *ssp, struct rcu_head *head,
+			 rcu_callback_t func);
 void synchronize_srcu_expedited(struct srcu_struct *ssp);
 void srcu_barrier(struct srcu_struct *ssp);
 void srcu_expedite_current(struct srcu_struct *ssp);
diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c
index aef8e91ad33e..77076d2a1c57 100644
--- a/kernel/rcu/srcutree.c
+++ b/kernel/rcu/srcutree.c
@@ -1495,6 +1495,13 @@ void call_srcu(struct srcu_struct *ssp, struct rcu_head *rhp,
 }
 EXPORT_SYMBOL_GPL(call_srcu);
 
+void call_srcu_expedited(struct srcu_struct *ssp, struct rcu_head *rhp,
+			 rcu_callback_t func)
+{
+	__call_srcu(ssp, rhp, func, rcu_gp_is_normal());
+}
+EXPORT_SYMBOL_GPL(call_srcu_expedited);
+
 /*
  * Helper function for synchronize_srcu() and synchronize_srcu_expedited().
  */
-- 
2.53.0.473.g4a7958ca14-goog

[RFC PATCH 3/3] KVM: Expedite SRCU callbacks when freeing objects during I/O bus registration

[Sean Christopherson]

Use the recently introduced call_srcu_expedited() when freeing the old I/O
bug during device registration to avoid triggering a non-expedited grace
period.  Delaying the freeing of the object by a full grace period is a
complete non-issue, but the grace period also gets transferred to future
synchronizations, e.g. to the synchronize_srcu_expedited() invocation in
kvm_swap_active_memslots().

For micro-VM use cases, effectively transferring the non-expedited grace
period to memslot updates results in a meaningful delay in overall boot
time.  E.g. with a CONFIG_HZ=100 kernel, the sync triggers a ~20ms delay,
increasing boot times by 15% or more.

Fixes: 7d9a0273c459 ("KVM: Avoid synchronize_srcu() in kvm_io_bus_register_dev()")
Reported-by: Nikita Kalyazin <kalyazin@amazon.com>
Closes: https://lore.kernel.org/all/a84ddba8-12da-489a-9dd1-ccdf7451a1ba@amazon.com
Cc: Keir Fraser <keirf@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 virt/kvm/kvm_main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 9faf70ccae7a..ceaf08a03428 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -6021,7 +6021,7 @@ int kvm_io_bus_register_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
 	memcpy(new_bus->range + i + 1, bus->range + i,
 		(bus->dev_count - i) * sizeof(struct kvm_io_range));
 	rcu_assign_pointer(kvm->buses[bus_idx], new_bus);
-	call_srcu(&kvm->srcu, &bus->rcu, __free_bus);
+	call_srcu_expedited(&kvm->srcu, &bus->rcu, __free_bus);
 
 	return 0;
 }
-- 
2.53.0.473.g4a7958ca14-goog

Re: [RFC PATCH 0/3] srcu: KVM: Add, export and use call_srcu_expedited()

[Kunwu Chan]

On 3/10/26 03:30, Sean Christopherson wrote:
> We've got a conundrum in KVM where we have multiple use cases that generally
> want the same thing (eliminate waiting on guest configuration changes whenever
> possible), but use KVM uAPIs in slightly different ways and effectively create
> competing requirements.
> 
> The crux of the problem is that one use case wants KVM to free an object via
> call_srcu() so that the task doesn't risk getting stalled waiting for a grace
> period.  But for the other use case, using call_srcu() can trigger a
> non-expedited grace period and cause a synchronize_srcu_expedited() in a
> different ioctl (that must do a full sync, i.e. can't use call_srcu()) to stall
> waiting for the non-expedited grace period.
> 
> Tagged RFC because while having the call_srcu() request do an expedited grace
> period eliminates the unwanted synchronize_srcu_expedited() stalls, this feels
> like a very crude fix.   That said, I'm definitely not opposed to this being a
> final solution if it's the best option available.
> 
> Sean Christopherson (3):
>   srcu: Declare exported symbols before including srcu{tiny,tree}.h
>   srcu: Add and export call_srcu_expedited() to avoid transferring grace
>     periods

Hi,

Thanks for writing this up.

The scenario you describe looks plausible.

That said, the cover letter wording might be a bit stronger than
current SRCU behavior warrants. A later synchronize_srcu_expedited()
can attempt to expedite an in-flight grace period, but it cannot
avoid delay already incurred (for example, if the GP has already
gone to sleep).

More generally, before adding an exported call_srcu_expedited()
helper, should we consider improving existing in-flight promotion
or delay behavior, or otherwise making the "expedite current GP"
case more explicit without introducing a new callback-facing API?


Thanx, Kunwu

>   KVM: Expedite SRCU callbacks when freeing objects during I/O bus
>     registration
> 
>  include/linux/srcu.h     | 10 +++++-----
>  include/linux/srcutiny.h |  8 ++++++--
>  include/linux/srcutree.h |  2 ++
>  kernel/rcu/srcutree.c    |  7 +++++++
>  virt/kvm/kvm_main.c      |  2 +-
>  5 files changed, 21 insertions(+), 8 deletions(-)
> 
> 
> base-commit: 5128b972fb2801ad9aca54d990a75611ab5283a9

Re: [RFC PATCH 0/3] srcu: KVM: Add, export and use call_srcu_expedited()

[Sean Christopherson]

On Fri, Mar 13, 2026, Kunwu Chan wrote:
> On 3/10/26 03:30, Sean Christopherson wrote:
> > We've got a conundrum in KVM where we have multiple use cases that generally
> > want the same thing (eliminate waiting on guest configuration changes whenever
> > possible), but use KVM uAPIs in slightly different ways and effectively create
> > competing requirements.
> > 
> > The crux of the problem is that one use case wants KVM to free an object via
> > call_srcu() so that the task doesn't risk getting stalled waiting for a grace
> > period.  But for the other use case, using call_srcu() can trigger a
> > non-expedited grace period and cause a synchronize_srcu_expedited() in a
> > different ioctl (that must do a full sync, i.e. can't use call_srcu()) to stall
> > waiting for the non-expedited grace period.
> > 
> > Tagged RFC because while having the call_srcu() request do an expedited grace
> > period eliminates the unwanted synchronize_srcu_expedited() stalls, this feels
> > like a very crude fix.   That said, I'm definitely not opposed to this being a
> > final solution if it's the best option available.
> > 
> > Sean Christopherson (3):
> >   srcu: Declare exported symbols before including srcu{tiny,tree}.h
> >   srcu: Add and export call_srcu_expedited() to avoid transferring grace
> >     periods
> 
> Hi,
> 
> Thanks for writing this up.
> 
> The scenario you describe looks plausible.
> 
> That said, the cover letter wording might be a bit stronger than
> current SRCU behavior warrants. A later synchronize_srcu_expedited()
> can attempt to expedite an in-flight grace period, but it cannot
> avoid delay already incurred (for example, if the GP has already
> gone to sleep).
> 
> More generally, before adding an exported call_srcu_expedited()
> helper, should we consider improving existing in-flight promotion
> or delay behavior, or otherwise making the "expedite current GP"
> case more explicit without introducing a new callback-facing API?

I'm all for a general solution, but that's far, far beyond my SRCU knowledge level.
I was quite proud of myself for piecing together the incurred-delay. :-)

FYI, I'm going to be unavailable for ~2 weeks.  Nikita (Cc'd) can likely help test
potential fixes.

Thanks!