Re: [PATCH v3 3/4] rcu/nocb: Protect NOCB state via local_lock() under PREEMPT_RT

[Frederic Weisbecker <frederic@kernel.org>]

[-- Attachment #1: Type: text/plain, Size: 2366 bytes --]

On Thu, Aug 12, 2021 at 05:20:45PM -0700, Paul E. McKenney wrote:
> Another concern...
> 
> During the preemption of rcu_check_quiescent_state() someone might report
> a quiescent state on behalf of CPU x (perhaps due to its having recently
> been idle) and then the RCU grace-period kthread might start running on
> CPU x, where it might initialize a new grace period in rcu_gp_init().
> It can then invoke __note_gp_changes(), also on CPU x.
> 
> If preempted as shown above just after checking >core_needs_qs, the
> ->cpu_no_qs.b.norm field will be set by the grace-period kthread, which
> will cause the rcu_check_quiescent_state() function's subsequent check
> of ->cpu_no_qs.b.norm to take an early exit.  So OK here.
> 
> On the other hand, if preempted just after the rcu_check_quiescent_state()
> function's check of ->cpu_no_qs.b.norm, the later invocation of
> rcu_report_qs_rdp() should take an early exit due to ->gp_seq mismatch.
> So OK here.

It looks like this could already happen through call_rcu() interrupting
rcu_core(). call_rcu() can also call note_gp_changes().

> 
> However, this should be added to the commit log.  Might be a big commit
> log, but mass storage is cheap these days.  ;-)

Agreed, and a comment.

> 
> This needs a review of each and every manipulation of ->core_needs_qs
> and ->cpu_no_qs.b.norm.  For example, the preemptions will cause the
> scheduler to invoke RCU's context-switch hooks, which also mess with
> ->cpu_no_qs.b.norm.

I think those situations should be fine. In the worst case rcu_core() misses
an opportunity to propagate an rdp quiescent state, waiting for the next core
invocation. The most important is that we don't report a quiescent state that
applied to a previous gp_seq but rcu_report_qs_rdp() should take care about that.

Of course I can easily miss something. Btw I'm attaching a map of all the
interactions with rdp.cpu_no_qs if that can help anyone. I used that to stop
getting lost alone with my headlamp.

> I can get to that some time next week (or tomorrow,
> if things go better than expected), but it would be good for you (and
> others) to check as well.
> 
> Frederic should look this over, but I am taking a quick pass in the
> meantime.  Please see below.

I have yet to check again the nocb bits. They look fine from a quick glance
but I need to sit down a bit longer.

Thanks.

[-- Attachment #2: rdp.cpu_no_qs.norm.jpg --]
[-- Type: image/jpeg, Size: 472684 bytes --]