Re: [PATCH RFC tip/core/rcu 4/6] rcu: Clarify help text for RCU_BOOST_PRIO

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Thu, Apr 26, 2012 at 02:46:31PM +0200, Peter Zijlstra wrote:
> On Mon, 2012-04-23 at 09:42 -0700, Paul E. McKenney wrote:
> > +         This option specifies the real-time priority to which long-term
> > +         preempted RCU readers are to be boosted.  If you are working
> > +         with a real-time application that has one or more CPU-bound
> > +         threads running at a real-time priority level,
> 
> Then your application is broken ;-) the kernel is known to mis-behave
> under these circumstances since it doesn't get to run house-keeping
> tasks. RCU is just one of these and elevating it doesn't make it work.

As you say, CPU-bound RT tasks have a number of problems, and RCU is but
one of them.  That said, an RCU-induced memory-exhaustion system hang
is an extremely unfriendly diagnostic message, and use of RCU priority
boosting allows them a better debugging environment.

> >  you should set
> > +         RCU_BOOST_PRIO to a priority higher then the highest-priority
> > +         real-time CPU-bound thread.  The default RCU_BOOST_PRIO value
> > +         of 1 is appropriate in the common case, which is real-time
> > +         applications that do not have any CPU-bound threads.
> 
> Alternatively, 1 is the worst possible choice forcing people to consider
> the issue.

You say that as if forcing people to consider the issue was a
bad thing.  ;-)

> > +         Some real-time applications might not have a single real-time
> > +         thread that saturates a given CPU, but instead might have
> > +         multiple real-time threads that, taken together, fully utilize
> > +         that CPU.  In this case, you should set RCU_BOOST_PRIO to
> > +         a priority higher than the lowest-priority thread that is
> > +         conspiring to prevent the CPU from running any non-real-time
> > +         tasks.  For example, if one thread at priority 10 and another
> > +         thread at priority 5 are between themselves fully consuming
> > +         the CPU time on a given CPU, then RCU_BOOST_PRIO should be
> > +         set to priority 6 or higher. 
> 
> I'd call this misleading, who's to say that preempting the 5 would yield
> enough time to complete the RCU work?

Yep, hence the "or higher".

> This all gets us back to the fun question of RCU delayed bandwidth
> budgeting.. ideally every 'task' that does call_rcu() should donate some
> of its budget towards the thread running the callback.

There was an academic interested in that topic a few years ago, but
I don't believe anything came of it.  An interesting approach would
be to do EDF scheduling on the callbacks themselves, but having a
separate thread for each callback sounds like overkill.

> Anyway, I'd argue both the old and new description are bonkers.

Indeed, my goal was "less bonkers" rather than "not bonkers".  A
"not bonkers" description remains a long-term aspiration rather than
a short-term goal for the moment.  I can only hope that the timeframe
is shorter than it was for RCU back in the early 1990s.  ;-)

							Thanx, Paul