Re: [PATCH 1/3] rcu: Use static initializer for krc.lock

[Joel Fernandes <joel@joelfernandes.org>]

On Thu, Apr 16, 2020 at 01:25:30PM -0700, Paul E. McKenney wrote:
> On Thu, Apr 16, 2020 at 10:05:57PM +0200, Uladzislau Rezki wrote:
> > On Thu, Apr 16, 2020 at 12:53:27PM -0700, Paul E. McKenney wrote:
> > > On Thu, Apr 16, 2020 at 03:26:23PM -0400, Steven Rostedt wrote:
> > > > On Thu, 16 Apr 2020 14:59:34 -0400
> > > > Joel Fernandes <joel@joelfernandes.org> wrote:
> > > > 
> > > > > But, then will it be safe for kfree_rcu() callers from hard IRQ context to
> > > > > call this in PREEMPT_RT? That could would just break then as you cannot sleep
> > > > > in hard IRQ context even on PREEMPT_RT.
> > > > 
> > > > But where in PREEMPT_RT would it be called in hard IRQ context?
> > > 
> > > I believe that call_rcu() is invoked with raw spinlocks held, so we should
> > > allow kfree_rcu() to be invoked from similar contexts.  It obviously
> > > cannot allocate memory in such contexts, so perhaps the rule is that
> > > single-argument kfree_rcu() cannot be invoked within hard IRQ contexts
> > > or with raw spinlocks held.  In those contexts, you would instead need
> > > to invoke two-argument kfree_rcu(), which never needs to allocate memory.
> > > 
> > > Seem reasonable?

In current rcu/dev, we do GFP_NOWAIT and in all WIP/future patches, we would
be doing GFP_NOWAIT in all scenarios, so would RT really have a problem with
memory allocation?

Or, is the issue that -RT needs memory allocation to finish in bounded time?

Either way, I believe our last discussion led to not retrying hard and not
sleeping during allocation so -RT should be good I think. I believe
Sebastian's main motivation is to just make the code preemptible but I am not
fully sure.

> > Paul, just to make it more clear, even invoking two arguments fkree_rcu()
> > currently does an allocation. We maintain an array that contains pointers
> > for "bulk logic". 
> True, but that is an optimization rather than an absolute necessity.
> In addition, most two-argument kfree_rcu() callers will take another slot
> from the array that has already been allocated.  So one alternative is
> to do the allocation only if both interrupts and preemption are enabled.
> As long as most kfree_rcu() invocations can allocate, you get good
> performance and things work nicely in -rt.

As per my understanding, we can't detect context dynamically to do different
things without the calling code telling us. I also landed on some LWN
articles on the topic. I tried this last weekend: preemptible() would not
know whether the code is called in a sleepable context or not on
!CONFIG_PREEMPT so can't be used as a signal. For this to work, it seems
CONFIG_PREEMPT_COUNT also has to be enabled (which AIUI is possible to be
enabled only on debug kernels). Hopefully I didn't miss something about that.

thanks,

 - Joel