Re: need_heavy_qs flag for PREEMPT=y kernels

[Joel Fernandes <joel@joelfernandes.org>]

On Sun, Aug 11, 2019 at 02:13:18PM -0700, Paul E. McKenney wrote:
[snip]
> This leaves NO_HZ_FULL=y&&PREEMPT=y kernels.  In that case, RCU is
> more aggressive about using resched_cpu() on CPUs that have not yet
> reported a quiescent state for the current grace period.

Just wanted to ask something - how does resched_cpu() help for this case?

Consider a nohz_full CPU and a PREEMPT=y kernel. Say a single task is running
in kernel mode with scheduler tick off. As we discussed, we have no help from
cond_resched() (since its a PREEMPT=y kernel).  Because enough time has
passed (jtsq*3), we send the CPU a re-scheduling IPI.

This seems not that useful. Even if we enter the scheduler due to the
rescheduling flags set on that CPU, nothing will do the rcu_report_qs_rdp()
or rcu_report_qs_rnp() on those CPUs, which are needed to propagate the
quiescent state to the leaf node. Neither will anything to do a
rcu_momentary_dyntick_idle() for that CPU. Without this, the grace period
will still end up getting blocked.

Could you clarify which code paths on a nohz_full CPU running PREEMPT=y
kernel actually helps to end the grace period when we call resched_cpu() on
it?  Don't we need atleast do a rcu_momentary_dyntick_idle() from the
scheduler IPI handler or from resched_cpu() for the benefit of a nohz_full
CPU? Maybe I should do an experiment to see this all play out.

And I need to write down everything I learnt today before I go crazy... ;-)