Re: [PATCH v9 3/7] srcu: Fix invoke_rcu_callbacks() segcb length adjustment

Neeraj Upadhyay
2020-11-04T13:38:17Z

On 11/3/2020 7:55 PM, Joel Fernandes (Google) wrote:
> With earlier patches, the negative counting of the unsegmented list
> cannot be used to adjust the segmented one. To fix this, sample the
> unsegmented length in advance, and use it after CB execution to adjust
> the segmented list's length.
>
> Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
> Suggested-by: Frederic Weisbecker <frederic@kernel.org>
> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> ---

Reviewed-by: Neeraj Upadhyay <neeraju@codeaurora.org>

> kernel/rcu/srcutree.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c
> index 0f23d20d485a..79b7081143a7 100644
> --- a/kernel/rcu/srcutree.c
> +++ b/kernel/rcu/srcutree.c
> @@ -1160,6 +1160,7 @@ static void srcu_advance_state(struct srcu_struct *ssp)
> */
> static void srcu_invoke_callbacks(struct work_struct *work)
> {
> + long len;
> bool more;
> struct rcu_cblist ready_cbs;
> struct rcu_head *rhp;
> @@ -1182,6 +1183,7 @@ static void srcu_invoke_callbacks(struct work_struct *work)
> /* We are on the job! Extract and invoke ready callbacks. */
> sdp->srcu_cblist_invoking = true;
> rcu_segcblist_extract_done_cbs(&sdp->srcu_cblist, &ready_cbs);
> + len = ready_cbs.len;
> spin_unlock_irq_rcu_node(sdp);
> rhp = rcu_cblist_dequeue(&ready_cbs);
> for (; rhp != NULL; rhp = rcu_cblist_dequeue(&ready_cbs)) {
> @@ -1190,13 +1192,14 @@ static void srcu_invoke_callbacks(struct work_struct *work)
> rhp->func(rhp);
> local_bh_enable();
> }
> + WARN_ON_ONCE(ready_cbs.len);
>
> /*
> * Update counts, accelerate new callbacks, and if needed,
> * schedule another round of callback invocation.
> */
> spin_lock_irq_rcu_node(sdp);
> - rcu_segcblist_insert_count(&sdp->srcu_cblist, &ready_cbs);
> + rcu_segcblist_add_len(&sdp->srcu_cblist, -len);
> (void)rcu_segcblist_accelerate(&sdp->srcu_cblist,
> rcu_seq_snap(&ssp->srcu_gp_seq));
> sdp->srcu_cblist_invoking = false;
>

--
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a
member of the Code Aurora Forum, hosted by The Linux Foundation