Re: rcu: frequent rcu lockups

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Fri, Mar 20, 2015 at 10:15:34AM -0400, Sasha Levin wrote:
> On 03/15/2015 08:02 PM, Paul E. McKenney wrote:
> > On Sun, Mar 15, 2015 at 07:32:32PM -0400, Sasha Levin wrote:
> >> > On 03/15/2015 04:43 PM, Paul E. McKenney wrote:
> >>> > > But I did find a bug that would result in the other warnings, and could
> >>> > > also result in too-short grace periods, which could in turn result in
> >>> > > arbitrarily arbitrary misbehavior.  The patch below, which is also on
> >>> > > its way into -next, should fix this.  Please let me know how it does
> >>> > > for you.
> >> > 
> >> > I've stopped seeing the warnings I've previously reported, but started
> >> > seeing a new one:
> >> > 
> >> > [  788.564596] WARNING: CPU: 12 PID: 9711 at kernel/rcu/tree.c:2201 rcu_report_qs_rnp+0x42e/0x5a0()
> >> > [  788.568123] Modules linked in:
> >> > [  788.568123] CPU: 12 PID: 9711 Comm: trinity-main Not tainted 4.0.0-rc3-next-20150313-sasha-00041-g83a3dc8-dirty #2078
> >> > [  788.568123]  ffff8803a1ba0000 00000000400df16a ffff880442807cc8 ffffffffb1ab01ca
> >> > [  788.568123]  0000000000000000 0000000000000000 ffff880442807d18 ffffffffa71e261a
> >> > [  788.568123]  dffffc0000000000 ffffffffa733d2ee ffff880442807d28 ffffffffb4724000
> >> > [  788.568123] Call Trace:
> >> > [  788.568123] <IRQ> dump_stack (lib/dump_stack.c:52)
> >> > [  788.568123] warn_slowpath_common (kernel/panic.c:447)
> >> > [  788.568123] ? rcu_report_qs_rnp (kernel/rcu/tree.c:2201 (discriminator 3))
> >> > [  788.568123] warn_slowpath_null (kernel/panic.c:481)
> >> > [  788.568123] rcu_report_qs_rnp (kernel/rcu/tree.c:2201 (discriminator 3))
> >> > [  788.568123] rcu_process_callbacks (kernel/rcu/tree.c:2302 kernel/rcu/tree.c:2338 kernel/rcu/tree.c:2824 kernel/rcu/tree.c:2857)
> >> > [  788.568123] __do_softirq (kernel/softirq.c:273 include/linux/jump_label.h:114 include/trace/events/irq.h:126 kernel/softirq.c:274)
> >> > [  788.568123] irq_exit (kernel/softirq.c:350 kernel/softirq.c:391)
> >> > [  788.568123] smp_apic_timer_interrupt (arch/x86/kernel/apic/apic.c:918)
> >> > [  788.568123] apic_timer_interrupt (arch/x86/kernel/entry_64.S:920)
> >> > [  788.568123] <EOI> ? mark_held_locks (kernel/locking/lockdep.c:2525)
> >> > [  788.568123] finish_task_switch (kernel/sched/core.c:2231)
> >> > [  788.568123] __schedule (kernel/sched/core.c:2337 kernel/sched/core.c:2795)
> >> > [  788.568123] schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> >> > [  788.568123] schedule_preempt_disabled (kernel/sched/core.c:2856)
> >> > [  788.568123] mutex_lock_nested (kernel/locking/mutex.c:585 kernel/locking/mutex.c:623)
> >> > [  788.568123] kernfs_iop_permission (fs/kernfs/inode.c:366)
> >> > [  788.568123] __inode_permission (fs/namei.c:374 fs/namei.c:408)
> >> > [  788.568123] inode_permission (fs/namei.c:460)
> >> > [  788.568123] link_path_walk (fs/namei.c:1520 fs/namei.c:1782)
> >> > [  788.568123] path_init (fs/namei.c:1947)
> >> > [  788.568123] path_lookupat (fs/namei.c:1989)
> >> > [  788.568123] filename_lookup (fs/namei.c:2025)
> >> > [  788.568123] user_path_at_empty (fs/namei.c:2189)
> >> > [  788.568123] user_path_at (fs/namei.c:2200)
> >> > [  788.568123] vfs_fstatat (fs/stat.c:106)
> >> > [  788.568123] SYSC_newfstatat (fs/stat.c:298)
> >> > [  788.568123] SyS_newfstatat (fs/stat.c:291)
> >> > [  788.568123] tracesys_phase2 (arch/x86/kernel/entry_64.S:347)
> > OK, I guess it would help to update the WARN_ON()s while I am at it.  :-/
> > 
> > Here is an updated patch that replaces the one resulting in the above
> > splat.
> 
> That did the trick for me, but I don't think it made it to -next?

Good to hear!  May I add your Tested-by?

And thank you for the reminder on -next, pushing it out now.

								Thanx, Paul