* starting mc triggers lockdep
@ 2006-07-07 16:13 Dave Jones
2006-07-07 18:48 ` Arjan van de Ven
0 siblings, 1 reply; 6+ messages in thread
From: Dave Jones @ 2006-07-07 16:13 UTC (permalink / raw)
To: Linux Kernel; +Cc: netdev
With 2.6.18rc1 + a selection of the lockdep tweaks found so far,
midnight commander makes the kernel unhappy.
Dave
=======================================================
[ INFO: possible circular locking dependency detected ]
-------------------------------------------------------
mc/4913 is trying to acquire lock:
(sk_lock-AF_INET){--..}, at: [<ffffffff8022800c>] tcp_sendmsg+0x1f/0xb1a
but task is already holding lock:
(&inode->i_mutex){--..}, at: [<ffffffff802692e0>] mutex_lock+0x2a/0x2e
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #2 (&inode->i_mutex){--..}:
[<ffffffff802ab6d5>] lock_acquire+0x4a/0x69
[<ffffffff80269102>] __mutex_lock_slowpath+0xeb/0x29f
[<ffffffff802692df>] mutex_lock+0x29/0x2e
[<ffffffff8030f4a0>] create_dir+0x2c/0x1e2
[<ffffffff8030fa5b>] sysfs_create_dir+0x59/0x78
[<ffffffff8034d2e2>] kobject_add+0x114/0x1d8
[<ffffffff803bb1e7>] class_device_add+0xb5/0x49d
[<ffffffff804300b1>] netdev_register_sysfs+0x98/0xa2
[<ffffffff80426f58>] register_netdevice+0x28c/0x376
[<ffffffff8042709c>] register_netdev+0x5a/0x69
[<ffffffff8098aa12>] loopback_init+0x4e/0x53
[<ffffffff8098a918>] net_olddevs_init+0xb/0xb7
[<ffffffff80270918>] init+0x177/0x348
[<ffffffff80263cdd>] child_rip+0x7/0x12
-> #1 (rtnl_mutex){--..}:
[<ffffffff802ab6d5>] lock_acquire+0x4a/0x69
[<ffffffff80269102>] __mutex_lock_slowpath+0xeb/0x29f
[<ffffffff802692df>] mutex_lock+0x29/0x2e
[<ffffffff8042e0a2>] rtnl_lock+0xf/0x12
[<ffffffff8045c7b8>] ip_mc_leave_group+0x1e/0xae
[<ffffffff804467f7>] do_ip_setsockopt+0x6ad/0x9b2
[<ffffffff80446baa>] ip_setsockopt+0x2a/0x84
[<ffffffff80454a55>] udp_setsockopt+0xd/0x1c
[<ffffffff8041f7ea>] sock_common_setsockopt+0xe/0x11
[<ffffffff8041e965>] sys_setsockopt+0x8e/0xb4
[<ffffffff80262f19>] tracesys+0xd0/0xdb
-> #0 (sk_lock-AF_INET){--..}:
[<ffffffff802ab6d5>] lock_acquire+0x4a/0x69
[<ffffffff802371ea>] lock_sock+0xd4/0xe7
[<ffffffff8022800b>] tcp_sendmsg+0x1e/0xb1a
[<ffffffff80248f4b>] inet_sendmsg+0x45/0x53
[<ffffffff80259d25>] sock_sendmsg+0x110/0x130
[<ffffffff8041f462>] kernel_sendmsg+0x3c/0x52
[<ffffffff885399e9>] xs_tcp_send_request+0x117/0x320 [sunrpc]
[<ffffffff885388d5>] xprt_transmit+0x105/0x21e [sunrpc]
[<ffffffff8853771e>] call_transmit+0x1f4/0x239 [sunrpc]
[<ffffffff8853c06e>] __rpc_execute+0x9b/0x1e6 [sunrpc]
[<ffffffff8853c1de>] rpc_execute+0x1a/0x1d [sunrpc]
[<ffffffff885364ad>] rpc_call_sync+0x87/0xb9 [sunrpc]
[<ffffffff885a2587>] nfs3_rpc_wrapper+0x2e/0x74 [nfs]
[<ffffffff885a2a14>] nfs3_proc_lookup+0xe0/0x163 [nfs]
[<ffffffff88594b10>] nfs_lookup+0xef/0x1d6 [nfs]
[<ffffffff8020d300>] do_lookup+0xd0/0x18c
[<ffffffff80209f27>] __link_path_walk+0xa29/0xf7d
[<ffffffff8020f076>] link_path_walk+0x69/0x101
[<ffffffff8020a22b>] __link_path_walk+0xd2d/0xf7d
[<ffffffff8020f076>] link_path_walk+0x69/0x101
[<ffffffff8020d096>] do_path_lookup+0x27b/0x2e7
[<ffffffff802258da>] __user_walk_fd+0x40/0x5c
[<ffffffff8022ae4a>] vfs_stat_fd+0x26/0x5d
[<ffffffff80225592>] sys_newstat+0x21/0x3c
[<ffffffff80262f19>] tracesys+0xd0/0xdb
other info that might help us debug this:
1 lock held by mc/4913:
#0: (&inode->i_mutex){--..}, at: [<ffffffff802692e0>] mutex_lock+0x2a/0x2e
stack backtrace:
Call Trace:
[<ffffffff80271910>] show_trace+0xaa/0x23d
[<ffffffff80271ab8>] dump_stack+0x15/0x17
[<ffffffff802a992f>] print_circular_bug_tail+0x6c/0x77
[<ffffffff802aaf34>] __lock_acquire+0x853/0xa54
[<ffffffff802ab6d6>] lock_acquire+0x4b/0x69
[<ffffffff802371eb>] lock_sock+0xd5/0xe7
[<ffffffff8022800c>] tcp_sendmsg+0x1f/0xb1a
[<ffffffff80248f4c>] inet_sendmsg+0x46/0x53
[<ffffffff80259d26>] sock_sendmsg+0x111/0x130
[<ffffffff8041f463>] kernel_sendmsg+0x3d/0x52
[<ffffffff885399ea>] :sunrpc:xs_tcp_send_request+0x118/0x320
[<ffffffff885388d6>] :sunrpc:xprt_transmit+0x106/0x21e
[<ffffffff8853771f>] :sunrpc:call_transmit+0x1f5/0x239
[<ffffffff8853c06f>] :sunrpc:__rpc_execute+0x9c/0x1e6
[<ffffffff8853c1df>] :sunrpc:rpc_execute+0x1b/0x1d
[<ffffffff885364ae>] :sunrpc:rpc_call_sync+0x88/0xb9
[<ffffffff885a2588>] :nfs:nfs3_rpc_wrapper+0x2f/0x74
[<ffffffff885a2a15>] :nfs:nfs3_proc_lookup+0xe1/0x163
[<ffffffff88594b11>] :nfs:nfs_lookup+0xf0/0x1d6
[<ffffffff8020d301>] do_lookup+0xd1/0x18c
[<ffffffff80209f28>] __link_path_walk+0xa2a/0xf7d
[<ffffffff8020f077>] link_path_walk+0x6a/0x101
[<ffffffff8020a22c>] __link_path_walk+0xd2e/0xf7d
[<ffffffff8020f077>] link_path_walk+0x6a/0x101
[<ffffffff8020d097>] do_path_lookup+0x27c/0x2e7
[<ffffffff802258db>] __user_walk_fd+0x41/0x5c
[<ffffffff8022ae4b>] vfs_stat_fd+0x27/0x5d
[<ffffffff80225593>] sys_newstat+0x22/0x3c
[<ffffffff80262f1a>] tracesys+0xd1/0xdb
--
http://www.codemonkey.org.uk
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: starting mc triggers lockdep
2006-07-07 16:13 starting mc triggers lockdep Dave Jones
@ 2006-07-07 18:48 ` Arjan van de Ven
2006-07-08 3:00 ` Herbert Xu
0 siblings, 1 reply; 6+ messages in thread
From: Arjan van de Ven @ 2006-07-07 18:48 UTC (permalink / raw)
To: Dave Jones; +Cc: mingo, akpm, Linux Kernel, netdev
On Fri, 2006-07-07 at 12:13 -0400, Dave Jones wrote:
> With 2.6.18rc1 + a selection of the lockdep tweaks found so far,
> midnight commander makes the kernel unhappy.
>
> Dave
>
>
> =======================================================
> [ INFO: possible circular locking dependency detected ]
> -------------------------------------------------------
> mc/4913 is trying to acquire lock:
> (sk_lock-AF_INET){--..}, at: [<ffffffff8022800c>] tcp_sendmsg+0x1f/0xb1a
>
> but task is already holding lock:
> (&inode->i_mutex){--..}, at: [<ffffffff802692e0>] mutex_lock+0x2a/0x2e
>
> which lock already depends on the new lock.
ok this looks like a fun three-way variant of the traditional AB-BA
deadlock... an ABCA deadlock :)
Lock A: rtnl_mutex
Lock B: i_mutex
Lock C: sk_lock for AF_INET
i_mutex is taken within rtln_mutex like this:
[<ffffffff8030f4a0>] create_dir+0x2c/0x1e2
[<ffffffff8030fa5b>] sysfs_create_dir+0x59/0x78
[<ffffffff8034d2e2>] kobject_add+0x114/0x1d8
[<ffffffff803bb1e7>] class_device_add+0xb5/0x49d
[<ffffffff804300b1>] netdev_register_sysfs+0x98/0xa2
[<ffffffff80426f58>] register_netdevice+0x28c/0x376
[<ffffffff8042709c>] register_netdev+0x5a/0x69
creating the AB dependency
now for the second part:
if you use the IP_DROP_MEMBERSHIP socket option, you end up going into
the function do_ip_setsockopt(), which first does a lock_sock(), and
then a call to ip_mc_leave_group(), which calls rtnl_lock()
So we have the CA dependency right here.
now for the third part, which involves the nfs client:
stat on an nfs file, which ends up taken the i_mutex of a directory in
the path (obvious), and then does
[<ffffffff8022800b>] tcp_sendmsg+0x1e/0xb1a
[<ffffffff80248f4b>] inet_sendmsg+0x45/0x53
[<ffffffff80259d25>] sock_sendmsg+0x110/0x130
[<ffffffff8041f462>] kernel_sendmsg+0x3c/0x52
[<ffffffff885399e9>] xs_tcp_send_request+0x117/0x320 [sunrpc]
[<ffffffff885388d5>] xprt_transmit+0x105/0x21e [sunrpc]
[<ffffffff8853771e>] call_transmit+0x1f4/0x239 [sunrpc]
[<ffffffff8853c06e>] __rpc_execute+0x9b/0x1e6 [sunrpc]
[<ffffffff8853c1de>] rpc_execute+0x1a/0x1d [sunrpc]
[<ffffffff885364ad>] rpc_call_sync+0x87/0xb9 [sunrpc]
[<ffffffff885a2587>] nfs3_rpc_wrapper+0x2e/0x74 [nfs]
[<ffffffff885a2a14>] nfs3_proc_lookup+0xe0/0x163 [nfs]
where tcp_sendmsg calls lock_sock. So this is the BC dependency.
In summary, we have an AB dependency, a BC dependency and a CA
dependency. This is reason for lockdep to conclude that there is a
circular dependency happening, which, looking at my analysis, seems to
be at least reasonably right...
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: starting mc triggers lockdep
2006-07-07 18:48 ` Arjan van de Ven
@ 2006-07-08 3:00 ` Herbert Xu
2006-07-08 9:53 ` Arjan van de Ven
0 siblings, 1 reply; 6+ messages in thread
From: Herbert Xu @ 2006-07-08 3:00 UTC (permalink / raw)
To: Arjan van de Ven; +Cc: davej, mingo, akpm, linux-kernel, netdev
Arjan van de Ven <arjan@infradead.org> wrote:
>
> i_mutex is taken within rtln_mutex like this:
> [<ffffffff8030f4a0>] create_dir+0x2c/0x1e2
> [<ffffffff8030fa5b>] sysfs_create_dir+0x59/0x78
> [<ffffffff8034d2e2>] kobject_add+0x114/0x1d8
> [<ffffffff803bb1e7>] class_device_add+0xb5/0x49d
> [<ffffffff804300b1>] netdev_register_sysfs+0x98/0xa2
> [<ffffffff80426f58>] register_netdevice+0x28c/0x376
> [<ffffffff8042709c>] register_netdev+0x5a/0x69
> creating the AB dependency
This is a sysfs inode.
> now for the third part, which involves the nfs client:
> stat on an nfs file, which ends up taken the i_mutex of a directory in
> the path (obvious), and then does
> [<ffffffff8022800b>] tcp_sendmsg+0x1e/0xb1a
> [<ffffffff80248f4b>] inet_sendmsg+0x45/0x53
> [<ffffffff80259d25>] sock_sendmsg+0x110/0x130
> [<ffffffff8041f462>] kernel_sendmsg+0x3c/0x52
> [<ffffffff885399e9>] xs_tcp_send_request+0x117/0x320 [sunrpc]
> [<ffffffff885388d5>] xprt_transmit+0x105/0x21e [sunrpc]
> [<ffffffff8853771e>] call_transmit+0x1f4/0x239 [sunrpc]
> [<ffffffff8853c06e>] __rpc_execute+0x9b/0x1e6 [sunrpc]
> [<ffffffff8853c1de>] rpc_execute+0x1a/0x1d [sunrpc]
> [<ffffffff885364ad>] rpc_call_sync+0x87/0xb9 [sunrpc]
> [<ffffffff885a2587>] nfs3_rpc_wrapper+0x2e/0x74 [nfs]
> [<ffffffff885a2a14>] nfs3_proc_lookup+0xe0/0x163 [nfs]
> where tcp_sendmsg calls lock_sock. So this is the BC dependency.
This is an nfs inode.
Did I miss something?
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: starting mc triggers lockdep
2006-07-08 3:00 ` Herbert Xu
@ 2006-07-08 9:53 ` Arjan van de Ven
2006-07-08 11:12 ` Herbert Xu
2006-07-08 13:46 ` Christoph Hellwig
0 siblings, 2 replies; 6+ messages in thread
From: Arjan van de Ven @ 2006-07-08 9:53 UTC (permalink / raw)
To: Herbert Xu; +Cc: davej, mingo, akpm, linux-kernel, netdev
On Sat, 2006-07-08 at 13:00 +1000, Herbert Xu wrote:
> Arjan van de Ven <arjan@infradead.org> wrote:
> >
> > i_mutex is taken within rtln_mutex like this:
> > [<ffffffff8030f4a0>] create_dir+0x2c/0x1e2
> > [<ffffffff8030fa5b>] sysfs_create_dir+0x59/0x78
> > [<ffffffff8034d2e2>] kobject_add+0x114/0x1d8
> > [<ffffffff803bb1e7>] class_device_add+0xb5/0x49d
> > [<ffffffff804300b1>] netdev_register_sysfs+0x98/0xa2
> > [<ffffffff80426f58>] register_netdevice+0x28c/0x376
> > [<ffffffff8042709c>] register_netdev+0x5a/0x69
> > creating the AB dependency
>
> This is a sysfs inode.
>
> > now for the third part, which involves the nfs client:
> > stat on an nfs file, which ends up taken the i_mutex of a directory in
> > the path (obvious), and then does
> > [<ffffffff8022800b>] tcp_sendmsg+0x1e/0xb1a
> > [<ffffffff80248f4b>] inet_sendmsg+0x45/0x53
> > [<ffffffff80259d25>] sock_sendmsg+0x110/0x130
> > [<ffffffff8041f462>] kernel_sendmsg+0x3c/0x52
> > [<ffffffff885399e9>] xs_tcp_send_request+0x117/0x320 [sunrpc]
> > [<ffffffff885388d5>] xprt_transmit+0x105/0x21e [sunrpc]
> > [<ffffffff8853771e>] call_transmit+0x1f4/0x239 [sunrpc]
> > [<ffffffff8853c06e>] __rpc_execute+0x9b/0x1e6 [sunrpc]
> > [<ffffffff8853c1de>] rpc_execute+0x1a/0x1d [sunrpc]
> > [<ffffffff885364ad>] rpc_call_sync+0x87/0xb9 [sunrpc]
> > [<ffffffff885a2587>] nfs3_rpc_wrapper+0x2e/0x74 [nfs]
> > [<ffffffff885a2a14>] nfs3_proc_lookup+0xe0/0x163 [nfs]
> > where tcp_sendmsg calls lock_sock. So this is the BC dependency.
>
> This is an nfs inode.
>
> Did I miss something?
is it not possible to nfs export /sys, and then mount it over loopback?
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: starting mc triggers lockdep
2006-07-08 9:53 ` Arjan van de Ven
@ 2006-07-08 11:12 ` Herbert Xu
2006-07-08 13:46 ` Christoph Hellwig
1 sibling, 0 replies; 6+ messages in thread
From: Herbert Xu @ 2006-07-08 11:12 UTC (permalink / raw)
To: Arjan van de Ven; +Cc: davej, mingo, akpm, linux-kernel, netdev
On Sat, Jul 08, 2006 at 11:53:20AM +0200, Arjan van de Ven wrote:
>
> > > now for the third part, which involves the nfs client:
> > > stat on an nfs file, which ends up taken the i_mutex of a directory in
> > > the path (obvious), and then does
> > > [<ffffffff8022800b>] tcp_sendmsg+0x1e/0xb1a
> > > [<ffffffff80248f4b>] inet_sendmsg+0x45/0x53
> > > [<ffffffff80259d25>] sock_sendmsg+0x110/0x130
> > > [<ffffffff8041f462>] kernel_sendmsg+0x3c/0x52
> > > [<ffffffff885399e9>] xs_tcp_send_request+0x117/0x320 [sunrpc]
> > > [<ffffffff885388d5>] xprt_transmit+0x105/0x21e [sunrpc]
> > > [<ffffffff8853771e>] call_transmit+0x1f4/0x239 [sunrpc]
> > > [<ffffffff8853c06e>] __rpc_execute+0x9b/0x1e6 [sunrpc]
> > > [<ffffffff8853c1de>] rpc_execute+0x1a/0x1d [sunrpc]
> > > [<ffffffff885364ad>] rpc_call_sync+0x87/0xb9 [sunrpc]
> > > [<ffffffff885a2587>] nfs3_rpc_wrapper+0x2e/0x74 [nfs]
> > > [<ffffffff885a2a14>] nfs3_proc_lookup+0xe0/0x163 [nfs]
> > > where tcp_sendmsg calls lock_sock. So this is the BC dependency.
> >
> > This is an nfs inode.
> >
> > Did I miss something?
>
> is it not possible to nfs export /sys, and then mount it over loopback?
Possibly, but not with the backtrace above. You'd need an nfs server
backtrace to get the real sysfs inode.
In any case, the sock lock from the other backtrace that you had
(udp setsockopt) cannot be held by the kernel nfs client or server
since the kernel nfs sockets are not visible to user space.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: starting mc triggers lockdep
2006-07-08 9:53 ` Arjan van de Ven
2006-07-08 11:12 ` Herbert Xu
@ 2006-07-08 13:46 ` Christoph Hellwig
1 sibling, 0 replies; 6+ messages in thread
From: Christoph Hellwig @ 2006-07-08 13:46 UTC (permalink / raw)
To: Arjan van de Ven; +Cc: Herbert Xu, davej, mingo, akpm, linux-kernel, netdev
On Sat, Jul 08, 2006 at 11:53:20AM +0200, Arjan van de Ven wrote:
> > Did I miss something?
>
> is it not possible to nfs export /sys, and then mount it over loopback?
No, it's not. A filesystem needs dedicated routines to support nfs
exporting and sysfs doesn't have those.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2006-07-08 13:46 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-07-07 16:13 starting mc triggers lockdep Dave Jones
2006-07-07 18:48 ` Arjan van de Ven
2006-07-08 3:00 ` Herbert Xu
2006-07-08 9:53 ` Arjan van de Ven
2006-07-08 11:12 ` Herbert Xu
2006-07-08 13:46 ` Christoph Hellwig
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).