Re: [RFC PATCH 05/10] rcu: Enable RCU callbacks to benefit from expedited grace periods

[Frederic Weisbecker <frederic@kernel.org>]

On Fri, Apr 17, 2026 at 04:11:53PM -0700, Puranjay Mohan wrote:
> Currently, RCU callbacks only track normal grace period sequence
> numbers.  This means callbacks must wait for normal grace periods to
> complete even when expedited grace periods have already elapsed.
> 
> This commit uses the full rcu_gp_oldstate structure (which tracks both
> normal and expedited GP sequences) throughout the callback
> infrastructure.
> 
> The rcu_segcblist_advance() function now checks both normal and
> expedited GP completion via poll_state_synchronize_rcu_full(), becoming
> parameterless since it reads the GP state internally.
> rcu_segcblist_accelerate() stores the full GP state (both normal and
> expedited sequences) instead of just the normal sequence.
> 
> The rcu_accelerate_cbs() and rcu_accelerate_cbs_unlocked() functions use
> get_state_synchronize_rcu_full() to capture both GP sequences. The NOCB
> code uses poll_state_synchronize_rcu_full() for advance checks instead
> of comparing only the normal GP sequence.
> 
> srcu_segcblist_advance() become standalone implementations because
> compares SRCU sequences directly (it cannot use
> poll_state_synchronize_rcu_full(), which reads RCU-specific globals).
> srcu_segcblist_accelerate() sets rgos_exp to RCU_GET_STATE_NOT_TRACKED
> so poll_state_synchronize_rcu_full() only compares the rgosp->rgos_norm
> and ignores rgos_exp.
> 
> Reviewed-by: Paul E. McKenney <paulmck@kernel.org>
> Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
> ---
>  kernel/rcu/rcu_segcblist.c | 30 ++++++++++++++++++++++++------
>  kernel/rcu/rcu_segcblist.h |  2 +-
>  kernel/rcu/tree.c          |  9 +++------
>  kernel/rcu/tree_nocb.h     | 33 +++++++++++++++++++++++----------
>  4 files changed, 51 insertions(+), 23 deletions(-)
> 
> diff --git a/kernel/rcu/rcu_segcblist.c b/kernel/rcu/rcu_segcblist.c
> index 00e164db8b74..11174e2be3c2 100644
> --- a/kernel/rcu/rcu_segcblist.c
> +++ b/kernel/rcu/rcu_segcblist.c
> @@ -12,6 +12,7 @@
>  #include <linux/kernel.h>
>  #include <linux/types.h>
>  
> +#include "rcu.h"
>  #include "rcu_segcblist.h"
>  
>  /* Initialize simple callback list. */
> @@ -494,9 +495,9 @@ static void rcu_segcblist_advance_compact(struct rcu_segcblist *rsclp, int i)
>  
>  /*
>   * Advance the callbacks in the specified rcu_segcblist structure based
> - * on the current value passed in for the grace-period counter.
> + * on the current value of the grace-period counter.
>   */
> -void rcu_segcblist_advance(struct rcu_segcblist *rsclp, struct rcu_gp_oldstate *rgosp)
> +void rcu_segcblist_advance(struct rcu_segcblist *rsclp)
>  {
>  	int i;
>  
> @@ -509,7 +510,7 @@ void rcu_segcblist_advance(struct rcu_segcblist *rsclp, struct rcu_gp_oldstate *
>  	 * are ready to invoke, and put them into the RCU_DONE_TAIL segment.
>  	 */
>  	for (i = RCU_WAIT_TAIL; i < RCU_NEXT_TAIL; i++) {
> -		if (ULONG_CMP_LT(rgosp->rgos_norm, rsclp->gp_seq_full[i].rgos_norm))
> +		if (!poll_state_synchronize_rcu_full(&rsclp->gp_seq_full[i]))

Ok that should work but it's worth noting two subtle changes:

- The sequence number to be compared against the segment snapshot one is now
  always from the root while it often used to be the one from the leaf node.
  It shouldn't matter I think especially as now the source of the segment
  snapshot is always either rcu_seq_snap() on the state or get_state_synchronize_rcu_full()
  and thus rcu_seq_snap() on the state as well.

- But poll_state_synchronize_rcu_full() does two full memory barriers. Hmm
  probably the one after reading the root state is necessary because we may not
  be holding the root node lock. But is the first smp_mb() necessary here?
  
Thanks.  

-- 
Frederic Weisbecker
SUSE Labs