Re: [PATCH RFC tip/core/rcu 0/5] Expedited grace periods encouraging normal ones

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Thu, Jul 02, 2015 at 08:35:57PM +0200, Ingo Molnar wrote:
> 
> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> 
> > > And it's not like it's that hard to stem the flow of algorithmic sloppiness at 
> > > the source, right?
> > 
> > OK, first let me make sure that I understand what you are asking for:
> > 
> > 1.	Completely eliminate synchronize_rcu_expedited() and
> > 	synchronize_sched_expedited(), replacing all uses with their
> > 	unexpedited counterparts.  (Note that synchronize_srcu_expedited()
> > 	does not wake up CPUs, courtesy of its read-side memory barriers.)
> > 	The fast-boot guys are probably going to complain, along with
> > 	the networking guys.
> > 
> > 2.	Keep synchronize_rcu_expedited() and synchronize_sched_expedited(),
> > 	but push back hard on any new uses and question any existing uses.
> > 
> > 3.	Revert 74b51ee152b6 ("ACPI / osl: speedup grace period in
> > 	acpi_os_map_cleanup").
> > 
> > 4.	Something else?
> 
> I'd love to have 1) but 2) would be a realistic second best option? ;-)

OK, how about the following checkpatch.pl patch?

And here are some other actions I have taken and will take to improve
the situation, both for OS jitter and for scalability:

o	Reduce OS jitter by switching from try_stop_cpus() to
	stop_one_cpu_nowait(), courtesy of Peter Zijlstra.

	I expect to send this in v4.3 or v4.4, depending on how
	testing and review goes.

o	Eliminate expedited-grace-period-induced OS jitter on idle CPUs.
	This went into v3.19.  Note that this also avoids IPIing
	nohz_full CPUs.

o	I believe that I can reduce OS jitter by a further factor of two
	(worst case) or factor of five (average case), but I am still
	thinking about exactly how to do this.	(This also has the
	benefit of shutting up a lockdep false positive.)

o	There is a global counter that synchronize_sched_expedited()
	uses to determine when all the CPUs have passed through a
	quiescent state.  This is a scalability bottleneck on modest
	systems under heavy load, so I will be switching this to
	instead use the combining tree.

o	Because both synchronize_sched_expedited() and
	synchronize_rcu_expedited() can potentially wake up each and
	every CPU, on sufficiently large systems, they are quite slow.
	If this scalability problem ever becomes real, I intend to use
	multiple kthreads to do the wakeups on large systems.

Seem reasonable?

							Thanx, Paul

------------------------------------------------------------------------

scripts: Make checkpatch.pl warn on expedited RCU grace periods

The synchronize_rcu_expedited() and synchronize_sched_expedited()
expedited-grace-period primitives induce OS jitter, which can degrade
real-time response.  This commit therefore adds a checkpatch.pl warning
on any patch adding them.

Note that this patch does not warn on synchronize_srcu_expedited()
because it does not induce OS jitter, courtesy of its otherwise
much-maligned read-side memory barriers.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Andy Whitcroft <apw@canonical.com>
Cc: Joe Perches <joe@perches.com>

diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 89b1df4e72ab..ddd82d743bba 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -4898,6 +4898,12 @@ sub process {
 				     "memory barrier without comment\n" . $herecurr);
 			}
 		}
+# check for expedited grace periods that interrupt CPUs.
+# note that synchronize_srcu_expedited() does -not- do this, so no complaints.
+		if ($line =~ /\b(synchronize_rcu_expedited|synchronize_sched_expedited)\(/) {
+			WARN("EXPEDITED_RCU_GRACE_PERIOD",
+			     "expedited RCU grace periods should be avoided\n" . $herecurr);
+		}
 # check of hardware specific defines
 		if ($line =~ m@^.\s*\#\s*if.*\b(__i386__|__powerpc64__|__sun__|__s390x__)\b@ && $realfile !~ m@include/asm-@) {
 			CHK("ARCH_DEFINES",