* next-20081103 - possible circular locking dependency detected while bring up eth1
[not found] <20081103183214.aac32543.sfr@canb.auug.org.au>
@ 2008-11-05 7:13 ` Kamalesh Babulal
2008-11-05 7:48 ` Peter Zijlstra
0 siblings, 1 reply; 3+ messages in thread
From: Kamalesh Babulal @ 2008-11-05 7:13 UTC (permalink / raw)
To: Stephen Rothwell; +Cc: linux-next, LKML, netdev, mel, peterz
While booting with the next-20081103 kernel on x86 box, circular locking
dependency is detected.
Bringing up interface eth1: [ 31.988230]
[ 31.988234] =======================================================
[ 31.989072] [ INFO: possible circular locking dependency detected ]
[ 31.989072] 2.6.28-rc3-next-20081103-autotest #1
[ 31.989072] -------------------------------------------------------
[ 31.989072] events/3/18 is trying to acquire lock:
[ 32.074777] ADDRCONF(NETDEV_UP): eth1: link is not ready
[ 31.989072] (rtnl_mutex){--..}, at: [<c05e8ef5>] rtnl_lock+0xf/0x11
[ 31.989072]
[ 31.989072] but task is already holding lock:
[ 31.989072] ((linkwatch_work).work){--..}, at: [<c04367bf>] run_workqueue+0x80/0x189
[ 31.989072]
[ 31.989072] which lock already depends on the new lock.
[ 31.989072]
[ 31.989072]
[ 31.989072] the existing dependency chain (in reverse order) is:
[ 31.989072]
[ 31.989072] -> #4 ((linkwatch_work).work){--..}:
[ 31.989072] [<c0445edb>] validate_chain+0x86e/0xb35
[ 31.989072] [<c0446822>] __lock_acquire+0x680/0x70e
[ 31.989072] [<c044690d>] lock_acquire+0x5d/0x7a
[ 31.989072] [<c04367f8>] run_workqueue+0xb9/0x189
[ 31.989072] [<c04371f3>] worker_thread+0xb4/0xbf
[ 31.989072] [<c04396a6>] kthread+0x3b/0x61
[ 31.989072] [<c040481b>] kernel_thread_helper+0x7/0x10
[ 31.989072] [<ffffffff>] 0xffffffff
[ 31.989072]
[ 31.989072] -> #3 (events){--..}:
[ 31.989072] [<c0445edb>] validate_chain+0x86e/0xb35
[ 31.989072] [<c0446822>] __lock_acquire+0x680/0x70e
[ 31.989072] [<c044690d>] lock_acquire+0x5d/0x7a
[ 31.989072] [<c0436e21>] flush_work+0x45/0xb6
[ 31.989072] [<c043729a>] schedule_on_each_cpu+0x9c/0xca
[ 31.989072] [<c0466c97>] lru_add_drain_all+0xd/0xf
[ 31.989072] [<c046f1b6>] __mlock_vma_pages_range+0x96/0x1e5
[ 31.989072] [<c046f43a>] mlock_fixup+0x135/0x199
[ 31.989072] [<c046f50b>] do_mlockall+0x6d/0x82
[ 31.989072] [<c046f839>] sys_mlockall+0x7b/0x9e
[ 31.989072] [<c040398d>] sysenter_do_call+0x12/0x31
[ 31.989072] [<ffffffff>] 0xffffffff
[ 31.989072]
[ 31.989072] -> #2 (&mm->mmap_sem){----}:
[ 31.989072] [<c0445edb>] validate_chain+0x86e/0xb35
[ 31.989072] [<c0446822>] __lock_acquire+0x680/0x70e
[ 31.989072] [<c044690d>] lock_acquire+0x5d/0x7a
[ 31.989072] [<c046e570>] might_fault+0x53/0x73
[ 31.989072] [<c0508bf5>] copy_to_user+0x28/0x3f
[ 31.989072] [<c048cd11>] filldir+0x88/0xc8
[ 31.989072] [<c04bee82>] sysfs_readdir+0x11d/0x156
[ 31.989072] [<c048cdb9>] vfs_readdir+0x68/0x94
[ 31.989072] [<c048d00b>] sys_getdents+0x5f/0xa0
[ 31.989072] [<c0403a66>] syscall_call+0x7/0xb
[ 31.989072] [<ffffffff>] 0xffffffff
[ 31.989072]
[ 31.989072] -> #1 (sysfs_mutex){--..}:
[ 31.989072] [<c0445edb>] validate_chain+0x86e/0xb35
[ 31.989072] [<c0446822>] __lock_acquire+0x680/0x70e
[ 31.989072] [<c044690d>] lock_acquire+0x5d/0x7a
[ 31.989072] [<c0651c04>] mutex_lock_nested+0xdf/0x251
[ 31.989072] [<c04bf039>] sysfs_addrm_start+0x23/0x90
[ 31.989072] [<c04bf4af>] create_dir+0x3a/0x72
[ 31.989072] [<c04bf514>] sysfs_create_dir+0x2d/0x41
[ 31.989072] [<c0503e05>] kobject_add_internal+0xe5/0x189
[ 31.989072] [<c0503f54>] kobject_add_varg+0x35/0x41
[ 31.989072] [<c05042f7>] kobject_add+0x49/0x4f
[ 31.989072] [<c05777e1>] device_add+0x76/0x4c8
[ 31.989072] [<c05eb70e>] netdev_register_kobject+0x64/0x69
[ 31.989072] [<c05e1581>] register_netdevice+0x1fe/0x274
[ 31.989072] [<c05e1629>] register_netdev+0x32/0x3f
[ 31.989072] [<c07fa965>] loopback_net_init+0x2e/0x5d
[ 31.989072] [<c05de93b>] register_pernet_operations+0x13/0x15
[ 31.989072] [<c05de9a4>] register_pernet_device+0x1f/0x4c
[ 31.989072] [<c07fa935>] loopback_init+0xd/0xf
[ 31.989072] [<c0401130>] _stext+0x48/0x10d
[ 31.989072] [<c07db530>] kernel_init+0xf1/0x142
[ 31.989072] [<c040481b>] kernel_thread_helper+0x7/0x10
[ 31.989072] [<ffffffff>] 0xffffffff
[ 31.989072]
[ 31.989072] -> #0 (rtnl_mutex){--..}:
[ 31.989072] [<c0445c10>] validate_chain+0x5a3/0xb35
[ 31.989072] [<c0446822>] __lock_acquire+0x680/0x70e
[ 31.989072] [<c044690d>] lock_acquire+0x5d/0x7a
[ 31.989072] [<c0651c04>] mutex_lock_nested+0xdf/0x251
[ 31.989072] [<c05e8ef5>] rtnl_lock+0xf/0x11
[ 31.989072] [<c05ea07a>] linkwatch_event+0x8/0x27
[ 31.989072] [<c04367fd>] run_workqueue+0xbe/0x189
[ 31.989072] [<c04371f3>] worker_thread+0xb4/0xbf
[ 31.989072] [<c04396a6>] kthread+0x3b/0x61
[ 31.989072] [<c040481b>] kernel_thread_helper+0x7/0x10
[ 31.989072] [<ffffffff>] 0xffffffff
[ 31.989072]
[ 31.989072] other info that might help us debug this:
[ 31.989072]
[ 31.989072] 2 locks held by events/3/18:
[ 31.989072] #0: (events){--..}, at: [<c04367bf>] run_workqueue+0x80/0x189
[ 31.989072] #1: ((linkwatch_work).work){--..}, at: [<c04367bf>] run_workqueue+0x80/0x189
[ 31.989072]
[ 31.989072] stack backtrace:
[ 31.989072] Pid: 18, comm: events/3 Not tainted 2.6.28-rc3-next-20081103-autotest #1
[ 31.989072] Call Trace:
[ 31.989072] [<c0445662>] print_circular_bug_tail+0xa4/0xaf
[ 31.989072] [<c0445c10>] validate_chain+0x5a3/0xb35
[ 31.989072] [<c0446822>] __lock_acquire+0x680/0x70e
[ 31.989072] [<c04367bf>] ? run_workqueue+0x80/0x189
[ 31.989072] [<c044690d>] lock_acquire+0x5d/0x7a
[ 31.989072] [<c05e8ef5>] ? rtnl_lock+0xf/0x11
[ 31.989072] [<c0651c04>] mutex_lock_nested+0xdf/0x251
[ 31.989072] [<c05e8ef5>] ? rtnl_lock+0xf/0x11
[ 31.989072] [<c05e8ef5>] ? rtnl_lock+0xf/0x11
[ 31.989072] [<c05e8ef5>] rtnl_lock+0xf/0x11
[ 31.989072] [<c05ea07a>] linkwatch_event+0x8/0x27
[ 31.989072] [<c04367fd>] run_workqueue+0xbe/0x189
[ 31.989072] [<c04367bf>] ? run_workqueue+0x80/0x189
[ 31.989072] [<c05ea072>] ? linkwatch_event+0x0/0x27
[ 31.989072] [<c043713f>] ? worker_thread+0x0/0xbf
[ 31.989072] [<c04371f3>] worker_thread+0xb4/0xbf
[ 31.989072] [<c0439765>] ? autoremove_wake_function+0x0/0x33
[ 31.989072] [<c04396a6>] kthread+0x3b/0x61
[ 33.690691] tg3: eth1: Link is up at 100 Mbps, full duplex.
[ 33.690696] tg3: eth1: Flow control is off for TX and off for RX.
[ 31.989072] [<c043966b>] ? kthread+0x0/0x61
[ 31.989072] [<c040481b>] kernel_thread_helper+0x7/0x10
[ 33.762336] ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
[ OK ]
0xc05e8ef5 is in rtnl_lock (net/core/rtnetlink.c:67).
62 static DEFINE_MUTEX(rtnl_mutex);
63
64 void rtnl_lock(void)
65 {
66 mutex_lock(&rtnl_mutex);
67 }
68
69 void __rtnl_unlock(void)
70 {
71 mutex_unlock(&rtnl_mutex);
--
Thanks & Regards,
Kamalesh Babulal,
Linux Technology Center,
IBM, ISTL.
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: next-20081103 - possible circular locking dependency detected while bring up eth1
2008-11-05 7:13 ` next-20081103 - possible circular locking dependency detected while bring up eth1 Kamalesh Babulal
@ 2008-11-05 7:48 ` Peter Zijlstra
2008-11-05 9:14 ` KOSAKI Motohiro
0 siblings, 1 reply; 3+ messages in thread
From: Peter Zijlstra @ 2008-11-05 7:48 UTC (permalink / raw)
To: Kamalesh Babulal
Cc: Stephen Rothwell, linux-next, LKML, netdev, mel, KOSAKI Motohiro
On Wed, 2008-11-05 at 12:43 +0530, Kamalesh Babulal wrote:
> While booting with the next-20081103 kernel on x86 box, circular locking
> dependency is detected.
>
> Bringing up interface eth1: [ 31.988230]
> [ 31.988234] =======================================================
> [ 31.989072] [ INFO: possible circular locking dependency detected ]
> [ 31.989072] 2.6.28-rc3-next-20081103-autotest #1
> [ 31.989072] -------------------------------------------------------
> [ 31.989072] events/3/18 is trying to acquire lock:
> [ 32.074777] ADDRCONF(NETDEV_UP): eth1: link is not ready
> [ 31.989072] (rtnl_mutex){--..}, at: [<c05e8ef5>] rtnl_lock+0xf/0x11
> [ 31.989072]
> [ 31.989072] but task is already holding lock:
> [ 31.989072] ((linkwatch_work).work){--..}, at: [<c04367bf>] run_workqueue+0x80/0x189
> [ 31.989072]
> [ 31.989072] which lock already depends on the new lock.
> [ 31.989072]
> [ 31.989072]
> [ 31.989072] the existing dependency chain (in reverse order) is:
> [ 31.989072]
> [ 31.989072] -> #4 ((linkwatch_work).work){--..}:
> [ 31.989072] [<c0445edb>] validate_chain+0x86e/0xb35
> [ 31.989072] [<c0446822>] __lock_acquire+0x680/0x70e
> [ 31.989072] [<c044690d>] lock_acquire+0x5d/0x7a
> [ 31.989072] [<c04367f8>] run_workqueue+0xb9/0x189
> [ 31.989072] [<c04371f3>] worker_thread+0xb4/0xbf
> [ 31.989072] [<c04396a6>] kthread+0x3b/0x61
> [ 31.989072] [<c040481b>] kernel_thread_helper+0x7/0x10
> [ 31.989072] [<ffffffff>] 0xffffffff
> [ 31.989072]
> [ 31.989072] -> #3 (events){--..}:
> [ 31.989072] [<c0445edb>] validate_chain+0x86e/0xb35
> [ 31.989072] [<c0446822>] __lock_acquire+0x680/0x70e
> [ 31.989072] [<c044690d>] lock_acquire+0x5d/0x7a
> [ 31.989072] [<c0436e21>] flush_work+0x45/0xb6
> [ 31.989072] [<c043729a>] schedule_on_each_cpu+0x9c/0xca
> [ 31.989072] [<c0466c97>] lru_add_drain_all+0xd/0xf
> [ 31.989072] [<c046f1b6>] __mlock_vma_pages_range+0x96/0x1e5
> [ 31.989072] [<c046f43a>] mlock_fixup+0x135/0x199
> [ 31.989072] [<c046f50b>] do_mlockall+0x6d/0x82
> [ 31.989072] [<c046f839>] sys_mlockall+0x7b/0x9e
> [ 31.989072] [<c040398d>] sysenter_do_call+0x12/0x31
> [ 31.989072] [<ffffffff>] 0xffffffff
> [ 31.989072]
> [ 31.989072] -> #2 (&mm->mmap_sem){----}:
> [ 31.989072] [<c0445edb>] validate_chain+0x86e/0xb35
> [ 31.989072] [<c0446822>] __lock_acquire+0x680/0x70e
> [ 31.989072] [<c044690d>] lock_acquire+0x5d/0x7a
> [ 31.989072] [<c046e570>] might_fault+0x53/0x73
> [ 31.989072] [<c0508bf5>] copy_to_user+0x28/0x3f
> [ 31.989072] [<c048cd11>] filldir+0x88/0xc8
> [ 31.989072] [<c04bee82>] sysfs_readdir+0x11d/0x156
> [ 31.989072] [<c048cdb9>] vfs_readdir+0x68/0x94
> [ 31.989072] [<c048d00b>] sys_getdents+0x5f/0xa0
> [ 31.989072] [<c0403a66>] syscall_call+0x7/0xb
> [ 31.989072] [<ffffffff>] 0xffffffff
> [ 31.989072]
> [ 31.989072] -> #1 (sysfs_mutex){--..}:
> [ 31.989072] [<c0445edb>] validate_chain+0x86e/0xb35
> [ 31.989072] [<c0446822>] __lock_acquire+0x680/0x70e
> [ 31.989072] [<c044690d>] lock_acquire+0x5d/0x7a
> [ 31.989072] [<c0651c04>] mutex_lock_nested+0xdf/0x251
> [ 31.989072] [<c04bf039>] sysfs_addrm_start+0x23/0x90
> [ 31.989072] [<c04bf4af>] create_dir+0x3a/0x72
> [ 31.989072] [<c04bf514>] sysfs_create_dir+0x2d/0x41
> [ 31.989072] [<c0503e05>] kobject_add_internal+0xe5/0x189
> [ 31.989072] [<c0503f54>] kobject_add_varg+0x35/0x41
> [ 31.989072] [<c05042f7>] kobject_add+0x49/0x4f
> [ 31.989072] [<c05777e1>] device_add+0x76/0x4c8
> [ 31.989072] [<c05eb70e>] netdev_register_kobject+0x64/0x69
> [ 31.989072] [<c05e1581>] register_netdevice+0x1fe/0x274
> [ 31.989072] [<c05e1629>] register_netdev+0x32/0x3f
> [ 31.989072] [<c07fa965>] loopback_net_init+0x2e/0x5d
> [ 31.989072] [<c05de93b>] register_pernet_operations+0x13/0x15
> [ 31.989072] [<c05de9a4>] register_pernet_device+0x1f/0x4c
> [ 31.989072] [<c07fa935>] loopback_init+0xd/0xf
> [ 31.989072] [<c0401130>] _stext+0x48/0x10d
> [ 31.989072] [<c07db530>] kernel_init+0xf1/0x142
> [ 31.989072] [<c040481b>] kernel_thread_helper+0x7/0x10
> [ 31.989072] [<ffffffff>] 0xffffffff
> [ 31.989072]
> [ 31.989072] -> #0 (rtnl_mutex){--..}:
> [ 31.989072] [<c0445c10>] validate_chain+0x5a3/0xb35
> [ 31.989072] [<c0446822>] __lock_acquire+0x680/0x70e
> [ 31.989072] [<c044690d>] lock_acquire+0x5d/0x7a
> [ 31.989072] [<c0651c04>] mutex_lock_nested+0xdf/0x251
> [ 31.989072] [<c05e8ef5>] rtnl_lock+0xf/0x11
> [ 31.989072] [<c05ea07a>] linkwatch_event+0x8/0x27
> [ 31.989072] [<c04367fd>] run_workqueue+0xbe/0x189
> [ 31.989072] [<c04371f3>] worker_thread+0xb4/0xbf
> [ 31.989072] [<c04396a6>] kthread+0x3b/0x61
> [ 31.989072] [<c040481b>] kernel_thread_helper+0x7/0x10
> [ 31.989072] [<ffffffff>] 0xffffffff
> [ 31.989072]
> [ 31.989072] other info that might help us debug this:
> [ 31.989072]
> [ 31.989072] 2 locks held by events/3/18:
> [ 31.989072] #0: (events){--..}, at: [<c04367bf>] run_workqueue+0x80/0x189
> [ 31.989072] #1: ((linkwatch_work).work){--..}, at: [<c04367bf>] run_workqueue+0x80/0x189
> [ 31.989072]
> [ 31.989072] stack backtrace:
> [ 31.989072] Pid: 18, comm: events/3 Not tainted 2.6.28-rc3-next-20081103-autotest #1
> [ 31.989072] Call Trace:
> [ 31.989072] [<c0445662>] print_circular_bug_tail+0xa4/0xaf
> [ 31.989072] [<c0445c10>] validate_chain+0x5a3/0xb35
> [ 31.989072] [<c0446822>] __lock_acquire+0x680/0x70e
> [ 31.989072] [<c04367bf>] ? run_workqueue+0x80/0x189
> [ 31.989072] [<c044690d>] lock_acquire+0x5d/0x7a
> [ 31.989072] [<c05e8ef5>] ? rtnl_lock+0xf/0x11
> [ 31.989072] [<c0651c04>] mutex_lock_nested+0xdf/0x251
> [ 31.989072] [<c05e8ef5>] ? rtnl_lock+0xf/0x11
> [ 31.989072] [<c05e8ef5>] ? rtnl_lock+0xf/0x11
> [ 31.989072] [<c05e8ef5>] rtnl_lock+0xf/0x11
> [ 31.989072] [<c05ea07a>] linkwatch_event+0x8/0x27
> [ 31.989072] [<c04367fd>] run_workqueue+0xbe/0x189
> [ 31.989072] [<c04367bf>] ? run_workqueue+0x80/0x189
> [ 31.989072] [<c05ea072>] ? linkwatch_event+0x0/0x27
> [ 31.989072] [<c043713f>] ? worker_thread+0x0/0xbf
> [ 31.989072] [<c04371f3>] worker_thread+0xb4/0xbf
> [ 31.989072] [<c0439765>] ? autoremove_wake_function+0x0/0x33
> [ 31.989072] [<c04396a6>] kthread+0x3b/0x61
> [ 33.690691] tg3: eth1: Link is up at 100 Mbps, full duplex.
> [ 33.690696] tg3: eth1: Flow control is off for TX and off for RX.
> [ 31.989072] [<c043966b>] ? kthread+0x0/0x61
> [ 31.989072] [<c040481b>] kernel_thread_helper+0x7/0x10
> [ 33.762336] ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
> [ OK ]
I think we have to go with Kosaki-san's vm workqueue...
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: next-20081103 - possible circular locking dependency detected while bring up eth1
2008-11-05 7:48 ` Peter Zijlstra
@ 2008-11-05 9:14 ` KOSAKI Motohiro
0 siblings, 0 replies; 3+ messages in thread
From: KOSAKI Motohiro @ 2008-11-05 9:14 UTC (permalink / raw)
To: Peter Zijlstra
Cc: kosaki.motohiro, Kamalesh Babulal, Stephen Rothwell, linux-next,
LKML, netdev, mel
> > [ 31.989072] 2 locks held by events/3/18:
> > [ 31.989072] #0: (events){--..}, at: [<c04367bf>] run_workqueue+0x80/0x189
> > [ 31.989072] #1: ((linkwatch_work).work){--..}, at: [<c04367bf>] run_workqueue+0x80/0x189
> > [ 31.989072]
> > [ 31.989072] stack backtrace:
> > [ 31.989072] Pid: 18, comm: events/3 Not tainted 2.6.28-rc3-next-20081103-autotest #1
> > [ 31.989072] Call Trace:
> > [ 31.989072] [<c0445662>] print_circular_bug_tail+0xa4/0xaf
> > [ 31.989072] [<c0445c10>] validate_chain+0x5a3/0xb35
> > [ 31.989072] [<c0446822>] __lock_acquire+0x680/0x70e
> > [ 31.989072] [<c04367bf>] ? run_workqueue+0x80/0x189
> > [ 31.989072] [<c044690d>] lock_acquire+0x5d/0x7a
> > [ 31.989072] [<c05e8ef5>] ? rtnl_lock+0xf/0x11
> > [ 31.989072] [<c0651c04>] mutex_lock_nested+0xdf/0x251
> > [ 31.989072] [<c05e8ef5>] ? rtnl_lock+0xf/0x11
> > [ 31.989072] [<c05e8ef5>] ? rtnl_lock+0xf/0x11
> > [ 31.989072] [<c05e8ef5>] rtnl_lock+0xf/0x11
> > [ 31.989072] [<c05ea07a>] linkwatch_event+0x8/0x27
> > [ 31.989072] [<c04367fd>] run_workqueue+0xbe/0x189
> > [ 31.989072] [<c04367bf>] ? run_workqueue+0x80/0x189
> > [ 31.989072] [<c05ea072>] ? linkwatch_event+0x0/0x27
> > [ 31.989072] [<c043713f>] ? worker_thread+0x0/0xbf
> > [ 31.989072] [<c04371f3>] worker_thread+0xb4/0xbf
> > [ 31.989072] [<c0439765>] ? autoremove_wake_function+0x0/0x33
> > [ 31.989072] [<c04396a6>] kthread+0x3b/0x61
> > [ 33.690691] tg3: eth1: Link is up at 100 Mbps, full duplex.
> > [ 33.690696] tg3: eth1: Flow control is off for TX and off for RX.
> > [ 31.989072] [<c043966b>] ? kthread+0x0/0x61
> > [ 31.989072] [<c040481b>] kernel_thread_helper+0x7/0x10
> > [ 33.762336] ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
> > [ OK ]
>
> I think we have to go with Kosaki-san's vm workqueue...
Very sorry for my lazyness.
akpm did nak vm workqueue patch.
So I prepare new patch now. (it is under testing now)
I expect I can post it tommorow.
Thanks.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2008-11-05 9:14 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20081103183214.aac32543.sfr@canb.auug.org.au>
2008-11-05 7:13 ` next-20081103 - possible circular locking dependency detected while bring up eth1 Kamalesh Babulal
2008-11-05 7:48 ` Peter Zijlstra
2008-11-05 9:14 ` KOSAKI Motohiro
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).