Re: Soft lockup in __slab_free (SLUB)

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Thu, Sep 29, 2016 at 01:13:29PM +1000, Nicholas Piggin wrote:
> On Wed, 28 Sep 2016 19:47:05 -0700
> "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:
> 
> > On Thu, Sep 29, 2016 at 12:30:07PM +1000, Nicholas Piggin wrote:
> > > On Wed, 28 Sep 2016 19:11:00 -0700
> > > "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:  
> > > > On Thu, Sep 29, 2016 at 10:40:24AM +0900, Joonsoo Kim wrote:  
> > > > > On Wed, Sep 28, 2016 at 10:15:01AM +0300, Nikolay Borisov wrote:    
> > > > > > 
> > > > > > I don't think it's an RCU problem per-se since ext4_i_callback is being
> > > > > > called from RCU due to the way inodes are being freed.    
> > > > > 
> > > > > That doesn't mean that RCU has no problem. IIUC, the fact is that RCU
> > > > > has no scheduling point in rcu_process_callbacks() and it would be
> > > > > problematic. It just depends on workload.    
> > > > 
> > > > You mean rcu_do_batch()?  It does limit the callbacks invoked per call
> > > > to rcu_do_batch() under normal conditions, see the "++count >= bl" check.
> > > > 
> > > > Now, if you dump a huge number of callbacks down call_rcu()'s throat,
> > > > it will stop being Mr. Nice Guy and will start executing the callbacks
> > > > as fast as it can for potentially quite some time.  But a huge number
> > > > will be in the millions.  Per CPU.  In which case I just might have a
> > > > few questions about exactly what you are trying to do.
> > > > 
> > > > Nevertheless, it is entirely possible that RCU's callback-invocation
> > > > throttling strategy needs improvement.  
> > > 
> > > Would it be useful to have a call_rcu variant that may sleep. Callers would
> > > use it preferentially if they can. Implementation might be exactly the same
> > > for now, but it would give you more flexibility with throttling strategies
> > > in future.  
> > 
> > You can specify callback-offloading at build and boot time, which will have
> > each CPU's callbacks being processed by a kernel thread:
> > 
> > CONFIG_RCU_NOCB_CPU
> > CONFIG_RCU_NOCB_CPU_{NONE,ZERO,ALL}
> > rcu_nocbs=
> > 
> > However, this still executes the individual callbacks with bh disabled.
> > If you want the actual callbacks themselves to be able to sleep, make
> > the callback hand off to a workqueue, wake up a kthread, or some such.
> > 
> > But yes, if enough people were just having the RCU callback immediately
> > invoke a workqueue, that could easily be special cased, just as
> > kfree_rcu() is now.
> > 
> > Or am I missing your point?
> 
> I just meant where the call_rcu() caller can sleep. RCU could block
> there to throttle production if necessary.

Good point!  Yes, if the problem is RCU getting flooded with callbacks,
this would be one possible resolution.  I am hoping that ftrace will
better identify the actual problem.  Naive of me, I know!  ;-)

							Thanx, Paul

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>