* inconsistent lock state
@ 2010-03-07 10:21 Sergey Senozhatsky
0 siblings, 0 replies; 10+ messages in thread
From: Sergey Senozhatsky @ 2010-03-07 10:21 UTC (permalink / raw)
To: netdev; +Cc: linux-kernel, Ingo Molnar
[-- Attachment #1: Type: text/plain, Size: 5293 bytes --]
Hello,
Hardly reproducible.
/*
* 2.6.33. x86. ASUS f3jc
*/
[329645.010697] =================================
[329645.010699] [ INFO: inconsistent lock state ]
[329645.010703] 2.6.33-33-0-dbg #31
[329645.010705] ---------------------------------
[329645.010708] inconsistent {IN-SOFTIRQ-W} -> {SOFTIRQ-ON-W} usage.
[329645.010712] events/0/9 [HC0[0]:SC0[0]:HE1:SE1] takes:
[329645.010715] (&(&table->hash[i].lock)->rlock){+.?.-.}, at: [<c12c1892>] spin_lock+0x8/0xa
[329645.010729] {IN-SOFTIRQ-W} state was registered at:
[329645.010732] [<c105878f>] __lock_acquire+0x27e/0xb86
[329645.010739] [<c1059988>] lock_acquire+0xa1/0xb8
[329645.010744] [<c12f39b2>] _raw_spin_lock+0x28/0x58
[329645.010750] [<c12c1892>] spin_lock+0x8/0xa
[329645.010755] [<c12c2c8f>] T.958+0x3c/0x141
[329645.010759] [<c12c2f7a>] __udp4_lib_rcv+0x1e6/0x3cd
[329645.010764] [<c12c3173>] udp_rcv+0x12/0x14
[329645.010769] [<c12a559b>] ip_local_deliver_finish+0xc9/0x130
[329645.010774] [<c12a5663>] ip_local_deliver+0x61/0x66
[329645.010778] [<c12a51a5>] ip_rcv_finish+0x275/0x29d
[329645.010783] [<c12a539a>] ip_rcv+0x1cd/0x1ed
[329645.010786] [<c128858d>] netif_receive_skb+0x340/0x360
[329645.010791] [<fd1ab6f4>] rtl8169_rx_interrupt+0x2bf/0x37e [r8169]
[329645.010801] [<fd1adab8>] rtl8169_poll+0x29/0x15a [r8169]
[329645.010808] [<c1288b8c>] net_rx_action+0x95/0x1af
[329645.010812] [<c1037243>] __do_softirq+0xc6/0x187
[329645.010819] [<c103732f>] do_softirq+0x2b/0x43
[329645.010823] [<c10374bf>] irq_exit+0x38/0x75
[329645.010828] [<c1004116>] do_IRQ+0x88/0x9c
[329645.010833] [<c1002f35>] common_interrupt+0x35/0x3c
[329645.010837] [<c1275d0b>] cpuidle_idle_call+0x72/0xd3
[329645.010844] [<c1001c1b>] cpu_idle+0x92/0xbf
[329645.010848] [<c12e3f16>] rest_init+0x76/0x78
[329645.010853] [<c14c2868>] start_kernel+0x33c/0x341
[329645.010859] [<c14c2092>] i386_start_kernel+0x92/0x99
[329645.010864] irq event stamp: 157782307
[329645.010866] hardirqs last enabled at (157782307): [<c10b74b8>] kmem_cache_free+0x97/0xd6
[329645.010873] hardirqs last disabled at (157782306): [<c10b745a>] kmem_cache_free+0x39/0xd6
[329645.010878] softirqs last enabled at (157782304): [<c12928ef>] rcu_read_unlock_bh+0x1c/0x1e
[329645.010885] softirqs last disabled at (157782302): [<c129289b>] rcu_read_lock_bh+0x8/0x26
[329645.010892]
[329645.010893] other info that might help us debug this:
[329645.010896] 5 locks held by events/0/9:
[329645.010898] #0: (events){+.+.+.}, at: [<c1044f2f>] worker_thread+0x16a/0x27c
[329645.010908] #1: ((&(&tp->task)->work)){+.+...}, at: [<c1044f2f>] worker_thread+0x16a/0x27c
[329645.010915] #2: (rtnl_mutex){+.+.+.}, at: [<c12911dd>] rtnl_lock+0xf/0x11
[329645.010922] #3: (rcu_read_lock){.+.+..}, at: [<c1286a01>] rcu_read_lock+0x0/0x2b
[329645.010931] #4: (rcu_read_lock){.+.+..}, at: [<c12a4ed0>] rcu_read_lock+0x0/0x2b
[329645.010938]
[329645.010939] stack backtrace:
[329645.010942] Pid: 9, comm: events/0 Not tainted 2.6.33-33-0-dbg #31
[329645.010945] Call Trace:
[329645.010950] [<c12f18ec>] ? printk+0xf/0x11
[329645.010955] [<c1057800>] valid_state+0x12a/0x13d
[329645.010960] [<c1057904>] mark_lock+0xf1/0x1e2
[329645.010965] [<c1057ffa>] ? check_usage_backwards+0x0/0x6f
[329645.010970] [<c10587fd>] __lock_acquire+0x2ec/0xb86
[329645.010976] [<c100864f>] ? native_sched_clock+0x48/0x8d
[329645.010982] [<c104d24b>] ? sched_clock_local+0x17/0x11e
[329645.010987] [<c12c1892>] ? spin_lock+0x8/0xa
[329645.010992] [<c1059988>] lock_acquire+0xa1/0xb8
[329645.010997] [<c12c1892>] ? spin_lock+0x8/0xa
[329645.011002] [<c12f39b2>] _raw_spin_lock+0x28/0x58
[329645.011006] [<c12c1892>] ? spin_lock+0x8/0xa
[329645.011010] [<c12c1892>] spin_lock+0x8/0xa
[329645.011015] [<c12c2c8f>] T.958+0x3c/0x141
[329645.011020] [<c104d472>] ? sched_clock_cpu+0x120/0x128
[329645.011025] [<c100864f>] ? native_sched_clock+0x48/0x8d
[329645.011031] [<c1059088>] ? __lock_acquire+0xb77/0xb86
[329645.011037] [<c12a1345>] ? rcu_read_unlock+0x0/0x35
[329645.011042] [<c12a38c0>] ? ip_route_input+0x102/0xacb
[329645.011046] [<c1057831>] ? mark_lock+0x1e/0x1e2
[329645.011051] [<c12a4ed0>] ? rcu_read_lock+0x0/0x2b
[329645.011056] [<c12c2f7a>] __udp4_lib_rcv+0x1e6/0x3cd
[329645.011061] [<c12c3173>] udp_rcv+0x12/0x14
[329645.011065] [<c12a559b>] ip_local_deliver_finish+0xc9/0x130
[329645.011070] [<c12a5663>] ip_local_deliver+0x61/0x66
[329645.011074] [<c12a51a5>] ip_rcv_finish+0x275/0x29d
[329645.011078] [<c12a539a>] ip_rcv+0x1cd/0x1ed
[329645.011083] [<c128858d>] netif_receive_skb+0x340/0x360
[329645.011093] [<fd1ab6f4>] rtl8169_rx_interrupt+0x2bf/0x37e [r8169]
[329645.011100] [<fd1aba02>] rtl8169_reset_task+0x38/0xcd [r8169]
[329645.011105] [<c1044f71>] worker_thread+0x1ac/0x27c
[329645.011110] [<c1044f2f>] ? worker_thread+0x16a/0x27c
[329645.011116] [<fd1ab9ca>] ? rtl8169_reset_task+0x0/0xcd [r8169]
[329645.011123] [<c1048725>] ? autoremove_wake_function+0x0/0x2f
[329645.011128] [<c1044dc5>] ? worker_thread+0x0/0x27c
[329645.011132] [<c104838a>] kthread+0x6a/0x6f
[329645.011137] [<c1048320>] ? kthread+0x0/0x6f
[329645.011142] [<c1002f42>] kernel_thread_helper+0x6/0x1a
Sergey
[-- Attachment #2: Type: application/pgp-signature, Size: 316 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: inconsistent lock state
[not found] <20100307192305.GA598@elte.hu>
@ 2010-03-08 12:51 ` Oleg Nesterov
2010-03-15 21:01 ` Eric Dumazet
0 siblings, 1 reply; 10+ messages in thread
From: Oleg Nesterov @ 2010-03-08 12:51 UTC (permalink / raw)
To: Ingo Molnar, Sergey Senozhatsky, Francois Romieu
Cc: Peter Zijlstra, netdev, linux-kernel, David Miller
Sergey Senozhatsky wrote:
>
> Hello,
>
> Hardly reproducible.
> /*
> * 2.6.33. x86. ASUS f3jc
> */
>
> [329645.010697] =================================
> [329645.010699] [ INFO: inconsistent lock state ]
> [329645.010703] 2.6.33-33-0-dbg #31
> [329645.010705] ---------------------------------
> [329645.010708] inconsistent {IN-SOFTIRQ-W} -> {SOFTIRQ-ON-W} usage.
>
> ...
>
> [329645.011083] [<c128858d>] netif_receive_skb+0x340/0x360
> [329645.011093] [<fd1ab6f4>] rtl8169_rx_interrupt+0x2bf/0x37e [r8169]
> [329645.011100] [<fd1aba02>] rtl8169_reset_task+0x38/0xcd [r8169]
> [329645.011105] [<c1044f71>] worker_thread+0x1ac/0x27c
I don't understand this code, but at first glance drivers/net/r8169.c
is obviously buggy.
The work->func, rtl8169_reset_task(), calls rtl8169_rx_interrupt() ->
netif_receive_skb(), and the last one may only be called from softirq.
Oleg.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: inconsistent lock state
2010-03-08 12:51 ` Oleg Nesterov
@ 2010-03-15 21:01 ` Eric Dumazet
2010-03-15 21:09 ` David Miller
0 siblings, 1 reply; 10+ messages in thread
From: Eric Dumazet @ 2010-03-15 21:01 UTC (permalink / raw)
To: Oleg Nesterov
Cc: Ingo Molnar, Sergey Senozhatsky, Francois Romieu, Peter Zijlstra,
netdev, linux-kernel, David Miller
Le lundi 08 mars 2010 à 13:51 +0100, Oleg Nesterov a écrit :
> Sergey Senozhatsky wrote:
> >
> > Hello,
> >
> > Hardly reproducible.
> > /*
> > * 2.6.33. x86. ASUS f3jc
> > */
> >
> > [329645.010697] =================================
> > [329645.010699] [ INFO: inconsistent lock state ]
> > [329645.010703] 2.6.33-33-0-dbg #31
> > [329645.010705] ---------------------------------
> > [329645.010708] inconsistent {IN-SOFTIRQ-W} -> {SOFTIRQ-ON-W} usage.
> >
> > ...
> >
> > [329645.011083] [<c128858d>] netif_receive_skb+0x340/0x360
> > [329645.011093] [<fd1ab6f4>] rtl8169_rx_interrupt+0x2bf/0x37e [r8169]
> > [329645.011100] [<fd1aba02>] rtl8169_reset_task+0x38/0xcd [r8169]
> > [329645.011105] [<c1044f71>] worker_thread+0x1ac/0x27c
>
> I don't understand this code, but at first glance drivers/net/r8169.c
> is obviously buggy.
>
> The work->func, rtl8169_reset_task(), calls rtl8169_rx_interrupt() ->
> netif_receive_skb(), and the last one may only be called from softirq.
>
Yes, this is wrong. In this context (process context, not softirq), we
should use netif_rx() or just drop frames if we are in reset phase.
Also, this driver schedules a reset in case a fifo error is reported in
rtl8169_rx_interrupt()
if (status & RxFOVF) {
rtl8169_schedule_work(dev, rtl8169_reset_task);
dev->stats.rx_fifo_errors++;
}
This seems very strange too : In case of a RX spike, we reset card ?
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: inconsistent lock state
2010-03-15 21:01 ` Eric Dumazet
@ 2010-03-15 21:09 ` David Miller
0 siblings, 0 replies; 10+ messages in thread
From: David Miller @ 2010-03-15 21:09 UTC (permalink / raw)
To: eric.dumazet
Cc: oleg, mingo, sergey.senozhatsky, romieu, a.p.zijlstra, netdev,
linux-kernel
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Mon, 15 Mar 2010 22:01:05 +0100
> Also, this driver schedules a reset in case a fifo error is reported in
> rtl8169_rx_interrupt()
>
> if (status & RxFOVF) {
> rtl8169_schedule_work(dev, rtl8169_reset_task);
> dev->stats.rx_fifo_errors++;
> }
>
>
> This seems very strange too : In case of a RX spike, we reset card ?
It's possible that this has been found to hang the card.
If so, it should be documented because otherwise yes we
should not be doing this.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: inconsistent lock state
[not found] <20100615112434.GA3967@swordfish.minsk.epam.com>
@ 2010-06-18 20:30 ` Andrew Morton
2010-06-19 8:33 ` Sergey Senozhatsky
` (4 more replies)
0 siblings, 5 replies; 10+ messages in thread
From: Andrew Morton @ 2010-06-18 20:30 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: Alexander Viro, Peter Zijlstra, Sage Weil, linux-fsdevel,
linux-kernel, Dominik Brodowski, Maciej Rutecki, Eric Dumazet,
Paul E. McKenney, Lai Jiangshan, David S. Miller, netdev
This was also reported by Dominik and is being tracked at
https://bugzilla.kernel.org/show_bug.cgi?id=16230
On Tue, 15 Jun 2010 14:24:34 +0300
Sergey Senozhatsky <sergey.senozhatsky@gmail.com> wrote:
> Hello,
>
> kernel: [ 3272.351191]
> kernel: [ 3272.351194] =================================
> kernel: [ 3272.351199] [ INFO: inconsistent lock state ]
> kernel: [ 3272.351204] 2.6.35-rc3-dbg-00106-ga75e02b-dirty #15
> kernel: [ 3272.351206] ---------------------------------
> kernel: [ 3272.351210] inconsistent {IN-HARDIRQ-W} -> {HARDIRQ-ON-W} usage.
> kernel: [ 3272.351215] X/3827 [HC0[0]:SC0[0]:HE1:SE1] takes:
> kernel: [ 3272.351218] (&(&new->fa_lock)->rlock){?.-...}, at: [<c10aefb4>] kill_fasync+0x37/0x71
> kernel: [ 3272.351232] {IN-HARDIRQ-W} state was registered at:
> kernel: [ 3272.351235] [<c104e95c>] __lock_acquire+0x281/0xbe1
> kernel: [ 3272.351243] [<c104f652>] lock_acquire+0x59/0x70
> kernel: [ 3272.351248] [<c12c6c48>] _raw_spin_lock+0x25/0x34
> kernel: [ 3272.351255] [<c10aefb4>] kill_fasync+0x37/0x71
> kernel: [ 3272.351261] [<fd220c81>] evdev_event+0x135/0x190 [evdev]
> kernel: [ 3272.351275] [<c1232003>] input_pass_event+0x6f/0xae
> kernel: [ 3272.351283] [<c1232ef5>] input_handle_event+0x38d/0x396
> kernel: [ 3272.351288] [<c1232fbf>] input_event+0x4f/0x62
> kernel: [ 3272.351293] [<c12368e4>] input_sync+0xe/0x11
> kernel: [ 3272.351299] [<c1236d72>] atkbd_interrupt+0x48b/0x541
> kernel: [ 3272.351304] [<c122ecb2>] serio_interrupt+0x35/0x68
> kernel: [ 3272.351309] [<c122fbff>] i8042_interrupt+0x264/0x26e
> kernel: [ 3272.351314] [<c106bb02>] handle_IRQ_event+0x1d/0x98
> kernel: [ 3272.351321] [<c106d506>] handle_edge_irq+0xc0/0x107
> kernel: [ 3272.351326] [<c10045ca>] handle_irq+0x1a/0x20
> kernel: [ 3272.351332] [<c100435f>] do_IRQ+0x43/0x8d
> kernel: [ 3272.351337] [<c1002d75>] common_interrupt+0x35/0x3c
> kernel: [ 3272.351342] [<c124723d>] cpuidle_idle_call+0x6a/0xa0
> kernel: [ 3272.351349] [<c100170d>] cpu_idle+0x89/0xbe
> kernel: [ 3272.351354] [<c12b6d11>] rest_init+0xb5/0xba
> kernel: [ 3272.351361] [<c148a7bf>] start_kernel+0x33b/0x340
> kernel: [ 3272.351368] [<c148a0c9>] i386_start_kernel+0xc9/0xd0
> kernel: [ 3272.351374] irq event stamp: 54104917
> kernel: [ 3272.351377] hardirqs last enabled at (54104917): [<c12c70f2>] _raw_spin_unlock_irqrestore+0x36/0x5b
> kernel: [ 3272.351384] hardirqs last disabled at (54104916): [<c12c6ced>] _raw_spin_lock_irqsave+0x13/0x42
> kernel: [ 3272.351391] softirqs last enabled at (54104732): [<c1032cf2>] __do_softirq+0xfd/0x10c
> kernel: [ 3272.351398] softirqs last disabled at (54104703): [<c1032d30>] do_softirq+0x2f/0x47
> kernel: [ 3272.351404]
> kernel: [ 3272.351405] other info that might help us debug this:
> kernel: [ 3272.351409] 3 locks held by X/3827:
> kernel: [ 3272.351412] #0: (rcu_read_lock){.+.+..}, at: [<c124fdfa>] rcu_read_lock+0x0/0x26
> kernel: [ 3272.351423] #1: (rcu_read_lock){.+.+..}, at: [<c124d5d9>] rcu_read_lock+0x0/0x26
> kernel: [ 3272.351432] #2: (rcu_read_lock){.+.+..}, at: [<c10ae429>] rcu_read_lock+0x0/0x26
> kernel: [ 3272.351442]
> kernel: [ 3272.351443] stack backtrace:
> kernel: [ 3272.351448] Pid: 3827, comm: X Not tainted 2.6.35-rc3-dbg-00106-ga75e02b-dirty #15
> kernel: [ 3272.351451] Call Trace:
> kernel: [ 3272.351456] [<c12c4ff1>] ? printk+0xf/0x11
> kernel: [ 3272.351462] [<c104e51a>] valid_state+0x133/0x141
> kernel: [ 3272.351468] [<c104e5f7>] mark_lock+0xcf/0x1b3
> kernel: [ 3272.351473] [<c104e54e>] ? mark_lock+0x26/0x1b3
> kernel: [ 3272.351479] [<c104dfd2>] ? check_usage_backwards+0x0/0x68
> kernel: [ 3272.351484] [<c104e9d0>] __lock_acquire+0x2f5/0xbe1
> kernel: [ 3272.351489] [<c104ea44>] ? __lock_acquire+0x369/0xbe1
> kernel: [ 3272.351495] [<c104ea44>] ? __lock_acquire+0x369/0xbe1
> kernel: [ 3272.351502] [<c102ab40>] ? try_to_wake_up+0x2a8/0x2bb
> kernel: [ 3272.351508] [<c104f652>] lock_acquire+0x59/0x70
> kernel: [ 3272.351513] [<c10aefb4>] ? kill_fasync+0x37/0x71
> kernel: [ 3272.351519] [<c12c6c48>] _raw_spin_lock+0x25/0x34
> kernel: [ 3272.351524] [<c10aefb4>] ? kill_fasync+0x37/0x71
> kernel: [ 3272.351529] [<c10aefb4>] kill_fasync+0x37/0x71
> kernel: [ 3272.351534] [<c124d694>] sock_wake_async+0x77/0x83
> kernel: [ 3272.351540] [<c124fe4d>] sk_wake_async+0x2d/0x32
> kernel: [ 3272.351545] [<c1250004>] sock_def_readable+0x45/0x51
> kernel: [ 3272.351551] [<c12b0247>] unix_stream_sendmsg+0x1e2/0x269
> kernel: [ 3272.351557] [<c124fe6e>] ? rcu_read_unlock+0x1c/0x1e
> kernel: [ 3272.351562] [<c124cf1a>] __sock_sendmsg+0x51/0x5a
> kernel: [ 3272.351567] [<c124cff7>] sock_aio_write+0xd4/0xdd
> kernel: [ 3272.351575] [<c10a4d95>] do_sync_readv_writev+0x84/0xb7
> kernel: [ 3272.351582] [<c10a4288>] ? copy_from_user+0x8/0xa
> kernel: [ 3272.351587] [<c10a4e69>] ? rw_copy_check_uvector+0x55/0xc7
> kernel: [ 3272.351594] [<c1164082>] ? security_file_permission+0xf/0x11
> kernel: [ 3272.351599] [<c10a47e5>] ? rw_verify_area+0x90/0xac
> kernel: [ 3272.351605] [<c10a4f58>] do_readv_writev+0x7d/0xdf
> kernel: [ 3272.351610] [<c124cf23>] ? sock_aio_write+0x0/0xdd
> kernel: [ 3272.351615] [<c1164082>] ? security_file_permission+0xf/0x11
> kernel: [ 3272.351621] [<c10a47e5>] ? rw_verify_area+0x90/0xac
> kernel: [ 3272.351626] [<c10a4ff3>] vfs_writev+0x39/0x42
> kernel: [ 3272.351632] [<c10a5102>] sys_writev+0x3b/0x8c
> kernel: [ 3272.351637] [<c10027d3>] sysenter_do_call+0x12/0x32
>
This, I think?
From: Andrew Morton <akpm@linux-foundation.org>
Fix a lockdep-splat-causing regression introduced by
: commit 989a2979205dd34269382b357e6d4b4b6956b889
: Author: Eric Dumazet <eric.dumazet@gmail.com>
: AuthorDate: Wed Apr 14 09:55:35 2010 +0000
: Commit: David S. Miller <davem@davemloft.net>
: CommitDate: Wed Apr 21 16:19:29 2010 -0700
:
: fasync: RCU and fine grained locking
kill_fasync() can be called from both process and hard-irq context, so
fa_lock must be taken with IRQs disabled.
Addresses https://bugzilla.kernel.org/show_bug.cgi?id=16230
Reported-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Reported-by: Dominik Brodowski <linux@dominikbrodowski.net>
Cc: Maciej Rutecki <maciej.rutecki@gmail.com>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
Cc: "David S. Miller" <davem@davemloft.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
fs/fcntl.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff -puN fs/fcntl.c~fs-fcntlc-kill_fasync_rcu-fa_lock-must-be-irq-safe fs/fcntl.c
--- a/fs/fcntl.c~fs-fcntlc-kill_fasync_rcu-fa_lock-must-be-irq-safe
+++ a/fs/fcntl.c
@@ -733,12 +733,14 @@ static void kill_fasync_rcu(struct fasyn
{
while (fa) {
struct fown_struct *fown;
+ unsigned long flags;
+
if (fa->magic != FASYNC_MAGIC) {
printk(KERN_ERR "kill_fasync: bad magic number in "
"fasync_struct!\n");
return;
}
- spin_lock(&fa->fa_lock);
+ spin_lock_irqsave(&fa->fa_lock, flags);
if (fa->fa_file) {
fown = &fa->fa_file->f_owner;
/* Don't send SIGURG to processes which have not set a
@@ -747,7 +749,7 @@ static void kill_fasync_rcu(struct fasyn
if (!(sig == SIGURG && fown->signum == 0))
send_sigio(fown, fa->fa_fd, band);
}
- spin_unlock(&fa->fa_lock);
+ spin_unlock_irqrestore(&fa->fa_lock, flags);
fa = rcu_dereference(fa->fa_next);
}
}
_
afaict all other lockers of fa_lock are OK (but one never really knows
with spin_lock_irq()).
Guys, please review-and-ack and I'll get it merged up.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: inconsistent lock state
2010-06-18 20:30 ` inconsistent lock state Andrew Morton
@ 2010-06-19 8:33 ` Sergey Senozhatsky
2010-06-19 13:05 ` Dominik Brodowski
` (3 subsequent siblings)
4 siblings, 0 replies; 10+ messages in thread
From: Sergey Senozhatsky @ 2010-06-19 8:33 UTC (permalink / raw)
To: Andrew Morton
Cc: Sergey Senozhatsky, Alexander Viro, Peter Zijlstra, Sage Weil,
linux-fsdevel, linux-kernel, Dominik Brodowski, Maciej Rutecki,
Eric Dumazet, Paul E. McKenney, Lai Jiangshan, David S. Miller,
netdev
[-- Attachment #1: Type: text/plain, Size: 8190 bytes --]
Hello Andrew,
Thanks. I'll test.
Sergey
On (06/18/10 13:30), Andrew Morton wrote:
> This was also reported by Dominik and is being tracked at
> https://bugzilla.kernel.org/show_bug.cgi?id=16230
>
> On Tue, 15 Jun 2010 14:24:34 +0300
> Sergey Senozhatsky <sergey.senozhatsky@gmail.com> wrote:
>
> > Hello,
> >
> > kernel: [ 3272.351191]
> > kernel: [ 3272.351194] =================================
> > kernel: [ 3272.351199] [ INFO: inconsistent lock state ]
> > kernel: [ 3272.351204] 2.6.35-rc3-dbg-00106-ga75e02b-dirty #15
> > kernel: [ 3272.351206] ---------------------------------
> > kernel: [ 3272.351210] inconsistent {IN-HARDIRQ-W} -> {HARDIRQ-ON-W} usage.
> > kernel: [ 3272.351215] X/3827 [HC0[0]:SC0[0]:HE1:SE1] takes:
> > kernel: [ 3272.351218] (&(&new->fa_lock)->rlock){?.-...}, at: [<c10aefb4>] kill_fasync+0x37/0x71
> > kernel: [ 3272.351232] {IN-HARDIRQ-W} state was registered at:
> > kernel: [ 3272.351235] [<c104e95c>] __lock_acquire+0x281/0xbe1
> > kernel: [ 3272.351243] [<c104f652>] lock_acquire+0x59/0x70
> > kernel: [ 3272.351248] [<c12c6c48>] _raw_spin_lock+0x25/0x34
> > kernel: [ 3272.351255] [<c10aefb4>] kill_fasync+0x37/0x71
> > kernel: [ 3272.351261] [<fd220c81>] evdev_event+0x135/0x190 [evdev]
> > kernel: [ 3272.351275] [<c1232003>] input_pass_event+0x6f/0xae
> > kernel: [ 3272.351283] [<c1232ef5>] input_handle_event+0x38d/0x396
> > kernel: [ 3272.351288] [<c1232fbf>] input_event+0x4f/0x62
> > kernel: [ 3272.351293] [<c12368e4>] input_sync+0xe/0x11
> > kernel: [ 3272.351299] [<c1236d72>] atkbd_interrupt+0x48b/0x541
> > kernel: [ 3272.351304] [<c122ecb2>] serio_interrupt+0x35/0x68
> > kernel: [ 3272.351309] [<c122fbff>] i8042_interrupt+0x264/0x26e
> > kernel: [ 3272.351314] [<c106bb02>] handle_IRQ_event+0x1d/0x98
> > kernel: [ 3272.351321] [<c106d506>] handle_edge_irq+0xc0/0x107
> > kernel: [ 3272.351326] [<c10045ca>] handle_irq+0x1a/0x20
> > kernel: [ 3272.351332] [<c100435f>] do_IRQ+0x43/0x8d
> > kernel: [ 3272.351337] [<c1002d75>] common_interrupt+0x35/0x3c
> > kernel: [ 3272.351342] [<c124723d>] cpuidle_idle_call+0x6a/0xa0
> > kernel: [ 3272.351349] [<c100170d>] cpu_idle+0x89/0xbe
> > kernel: [ 3272.351354] [<c12b6d11>] rest_init+0xb5/0xba
> > kernel: [ 3272.351361] [<c148a7bf>] start_kernel+0x33b/0x340
> > kernel: [ 3272.351368] [<c148a0c9>] i386_start_kernel+0xc9/0xd0
> > kernel: [ 3272.351374] irq event stamp: 54104917
> > kernel: [ 3272.351377] hardirqs last enabled at (54104917): [<c12c70f2>] _raw_spin_unlock_irqrestore+0x36/0x5b
> > kernel: [ 3272.351384] hardirqs last disabled at (54104916): [<c12c6ced>] _raw_spin_lock_irqsave+0x13/0x42
> > kernel: [ 3272.351391] softirqs last enabled at (54104732): [<c1032cf2>] __do_softirq+0xfd/0x10c
> > kernel: [ 3272.351398] softirqs last disabled at (54104703): [<c1032d30>] do_softirq+0x2f/0x47
> > kernel: [ 3272.351404]
> > kernel: [ 3272.351405] other info that might help us debug this:
> > kernel: [ 3272.351409] 3 locks held by X/3827:
> > kernel: [ 3272.351412] #0: (rcu_read_lock){.+.+..}, at: [<c124fdfa>] rcu_read_lock+0x0/0x26
> > kernel: [ 3272.351423] #1: (rcu_read_lock){.+.+..}, at: [<c124d5d9>] rcu_read_lock+0x0/0x26
> > kernel: [ 3272.351432] #2: (rcu_read_lock){.+.+..}, at: [<c10ae429>] rcu_read_lock+0x0/0x26
> > kernel: [ 3272.351442]
> > kernel: [ 3272.351443] stack backtrace:
> > kernel: [ 3272.351448] Pid: 3827, comm: X Not tainted 2.6.35-rc3-dbg-00106-ga75e02b-dirty #15
> > kernel: [ 3272.351451] Call Trace:
> > kernel: [ 3272.351456] [<c12c4ff1>] ? printk+0xf/0x11
> > kernel: [ 3272.351462] [<c104e51a>] valid_state+0x133/0x141
> > kernel: [ 3272.351468] [<c104e5f7>] mark_lock+0xcf/0x1b3
> > kernel: [ 3272.351473] [<c104e54e>] ? mark_lock+0x26/0x1b3
> > kernel: [ 3272.351479] [<c104dfd2>] ? check_usage_backwards+0x0/0x68
> > kernel: [ 3272.351484] [<c104e9d0>] __lock_acquire+0x2f5/0xbe1
> > kernel: [ 3272.351489] [<c104ea44>] ? __lock_acquire+0x369/0xbe1
> > kernel: [ 3272.351495] [<c104ea44>] ? __lock_acquire+0x369/0xbe1
> > kernel: [ 3272.351502] [<c102ab40>] ? try_to_wake_up+0x2a8/0x2bb
> > kernel: [ 3272.351508] [<c104f652>] lock_acquire+0x59/0x70
> > kernel: [ 3272.351513] [<c10aefb4>] ? kill_fasync+0x37/0x71
> > kernel: [ 3272.351519] [<c12c6c48>] _raw_spin_lock+0x25/0x34
> > kernel: [ 3272.351524] [<c10aefb4>] ? kill_fasync+0x37/0x71
> > kernel: [ 3272.351529] [<c10aefb4>] kill_fasync+0x37/0x71
> > kernel: [ 3272.351534] [<c124d694>] sock_wake_async+0x77/0x83
> > kernel: [ 3272.351540] [<c124fe4d>] sk_wake_async+0x2d/0x32
> > kernel: [ 3272.351545] [<c1250004>] sock_def_readable+0x45/0x51
> > kernel: [ 3272.351551] [<c12b0247>] unix_stream_sendmsg+0x1e2/0x269
> > kernel: [ 3272.351557] [<c124fe6e>] ? rcu_read_unlock+0x1c/0x1e
> > kernel: [ 3272.351562] [<c124cf1a>] __sock_sendmsg+0x51/0x5a
> > kernel: [ 3272.351567] [<c124cff7>] sock_aio_write+0xd4/0xdd
> > kernel: [ 3272.351575] [<c10a4d95>] do_sync_readv_writev+0x84/0xb7
> > kernel: [ 3272.351582] [<c10a4288>] ? copy_from_user+0x8/0xa
> > kernel: [ 3272.351587] [<c10a4e69>] ? rw_copy_check_uvector+0x55/0xc7
> > kernel: [ 3272.351594] [<c1164082>] ? security_file_permission+0xf/0x11
> > kernel: [ 3272.351599] [<c10a47e5>] ? rw_verify_area+0x90/0xac
> > kernel: [ 3272.351605] [<c10a4f58>] do_readv_writev+0x7d/0xdf
> > kernel: [ 3272.351610] [<c124cf23>] ? sock_aio_write+0x0/0xdd
> > kernel: [ 3272.351615] [<c1164082>] ? security_file_permission+0xf/0x11
> > kernel: [ 3272.351621] [<c10a47e5>] ? rw_verify_area+0x90/0xac
> > kernel: [ 3272.351626] [<c10a4ff3>] vfs_writev+0x39/0x42
> > kernel: [ 3272.351632] [<c10a5102>] sys_writev+0x3b/0x8c
> > kernel: [ 3272.351637] [<c10027d3>] sysenter_do_call+0x12/0x32
> >
>
> This, I think?
>
>
> From: Andrew Morton <akpm@linux-foundation.org>
>
> Fix a lockdep-splat-causing regression introduced by
>
> : commit 989a2979205dd34269382b357e6d4b4b6956b889
> : Author: Eric Dumazet <eric.dumazet@gmail.com>
> : AuthorDate: Wed Apr 14 09:55:35 2010 +0000
> : Commit: David S. Miller <davem@davemloft.net>
> : CommitDate: Wed Apr 21 16:19:29 2010 -0700
> :
> : fasync: RCU and fine grained locking
>
> kill_fasync() can be called from both process and hard-irq context, so
> fa_lock must be taken with IRQs disabled.
>
> Addresses https://bugzilla.kernel.org/show_bug.cgi?id=16230
>
> Reported-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> Reported-by: Dominik Brodowski <linux@dominikbrodowski.net>
> Cc: Maciej Rutecki <maciej.rutecki@gmail.com>
> Cc: Eric Dumazet <eric.dumazet@gmail.com>
> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
> Cc: "David S. Miller" <davem@davemloft.net>
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> ---
>
> fs/fcntl.c | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff -puN fs/fcntl.c~fs-fcntlc-kill_fasync_rcu-fa_lock-must-be-irq-safe fs/fcntl.c
> --- a/fs/fcntl.c~fs-fcntlc-kill_fasync_rcu-fa_lock-must-be-irq-safe
> +++ a/fs/fcntl.c
> @@ -733,12 +733,14 @@ static void kill_fasync_rcu(struct fasyn
> {
> while (fa) {
> struct fown_struct *fown;
> + unsigned long flags;
> +
> if (fa->magic != FASYNC_MAGIC) {
> printk(KERN_ERR "kill_fasync: bad magic number in "
> "fasync_struct!\n");
> return;
> }
> - spin_lock(&fa->fa_lock);
> + spin_lock_irqsave(&fa->fa_lock, flags);
> if (fa->fa_file) {
> fown = &fa->fa_file->f_owner;
> /* Don't send SIGURG to processes which have not set a
> @@ -747,7 +749,7 @@ static void kill_fasync_rcu(struct fasyn
> if (!(sig == SIGURG && fown->signum == 0))
> send_sigio(fown, fa->fa_fd, band);
> }
> - spin_unlock(&fa->fa_lock);
> + spin_unlock_irqrestore(&fa->fa_lock, flags);
> fa = rcu_dereference(fa->fa_next);
> }
> }
> _
>
>
> afaict all other lockers of fa_lock are OK (but one never really knows
> with spin_lock_irq()).
>
> Guys, please review-and-ack and I'll get it merged up.
>
[-- Attachment #2: Type: application/pgp-signature, Size: 316 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: inconsistent lock state
2010-06-18 20:30 ` inconsistent lock state Andrew Morton
2010-06-19 8:33 ` Sergey Senozhatsky
@ 2010-06-19 13:05 ` Dominik Brodowski
2010-06-21 6:54 ` Peter Zijlstra
` (2 subsequent siblings)
4 siblings, 0 replies; 10+ messages in thread
From: Dominik Brodowski @ 2010-06-19 13:05 UTC (permalink / raw)
To: Andrew Morton
Cc: Sergey Senozhatsky, Alexander Viro, Peter Zijlstra, Sage Weil,
linux-fsdevel, linux-kernel, Maciej Rutecki, Eric Dumazet,
Paul E. McKenney, Lai Jiangshan, David S. Miller, netdev
Hey,
On Fri, Jun 18, 2010 at 01:30:04PM -0700, Andrew Morton wrote:
> From: Andrew Morton <akpm@linux-foundation.org>
>
> Fix a lockdep-splat-causing regression introduced by
>
> : commit 989a2979205dd34269382b357e6d4b4b6956b889
> : Author: Eric Dumazet <eric.dumazet@gmail.com>
> : AuthorDate: Wed Apr 14 09:55:35 2010 +0000
> : Commit: David S. Miller <davem@davemloft.net>
> : CommitDate: Wed Apr 21 16:19:29 2010 -0700
> :
> : fasync: RCU and fine grained locking
>
> kill_fasync() can be called from both process and hard-irq context, so
> fa_lock must be taken with IRQs disabled.
>
> Addresses https://bugzilla.kernel.org/show_bug.cgi?id=16230
Seems to work fine -- thanks!
Best,
Dominik
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: inconsistent lock state
2010-06-18 20:30 ` inconsistent lock state Andrew Morton
2010-06-19 8:33 ` Sergey Senozhatsky
2010-06-19 13:05 ` Dominik Brodowski
@ 2010-06-21 6:54 ` Peter Zijlstra
2010-06-22 8:59 ` Eric Dumazet
2010-06-22 17:17 ` Paul E. McKenney
4 siblings, 0 replies; 10+ messages in thread
From: Peter Zijlstra @ 2010-06-21 6:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Sergey Senozhatsky, Alexander Viro, Sage Weil, linux-fsdevel,
linux-kernel, Dominik Brodowski, Maciej Rutecki, Eric Dumazet,
Paul E.McKenney, Lai Jiangshan, David S.Miller, netdev
On Fri, 2010-06-18 at 13:30 -0700, Andrew Morton wrote:
> From: Andrew Morton <akpm@linux-foundation.org>
>
> Fix a lockdep-splat-causing regression introduced by
>
> : commit 989a2979205dd34269382b357e6d4b4b6956b889
> : Author: Eric Dumazet <eric.dumazet@gmail.com>
> : AuthorDate: Wed Apr 14 09:55:35 2010 +0000
> : Commit: David S. Miller <davem@davemloft.net>
> : CommitDate: Wed Apr 21 16:19:29 2010 -0700
> :
> : fasync: RCU and fine grained locking
>
> kill_fasync() can be called from both process and hard-irq context, so
> fa_lock must be taken with IRQs disabled.
>
> Addresses https://bugzilla.kernel.org/show_bug.cgi?id=16230
>
> Reported-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> Reported-by: Dominik Brodowski <linux@dominikbrodowski.net>
> Cc: Maciej Rutecki <maciej.rutecki@gmail.com>
> Cc: Eric Dumazet <eric.dumazet@gmail.com>
> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
> Cc: "David S. Miller" <davem@davemloft.net>
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: inconsistent lock state
2010-06-18 20:30 ` inconsistent lock state Andrew Morton
` (2 preceding siblings ...)
2010-06-21 6:54 ` Peter Zijlstra
@ 2010-06-22 8:59 ` Eric Dumazet
2010-06-22 17:17 ` Paul E. McKenney
4 siblings, 0 replies; 10+ messages in thread
From: Eric Dumazet @ 2010-06-22 8:59 UTC (permalink / raw)
To: Andrew Morton
Cc: Sergey Senozhatsky, Alexander Viro, Peter Zijlstra, Sage Weil,
linux-fsdevel, linux-kernel, Dominik Brodowski, Maciej Rutecki,
Paul E.McKenney, Lai Jiangshan, David S.Miller, netdev
Le vendredi 18 juin 2010 à 13:30 -0700, Andrew Morton a écrit :
> This was also reported by Dominik and is being tracked at
> https://bugzilla.kernel.org/show_bug.cgi?id=16230
>
> On Tue, 15 Jun 2010 14:24:34 +0300
> Sergey Senozhatsky <sergey.senozhatsky@gmail.com> wrote:
>
> > Hello,
> >
> > kernel: [ 3272.351191]
> > kernel: [ 3272.351194] =================================
> > kernel: [ 3272.351199] [ INFO: inconsistent lock state ]
> > kernel: [ 3272.351204] 2.6.35-rc3-dbg-00106-ga75e02b-dirty #15
> > kernel: [ 3272.351206] ---------------------------------
> > kernel: [ 3272.351210] inconsistent {IN-HARDIRQ-W} -> {HARDIRQ-ON-W} usage.
> > kernel: [ 3272.351215] X/3827 [HC0[0]:SC0[0]:HE1:SE1] takes:
> > kernel: [ 3272.351218] (&(&new->fa_lock)->rlock){?.-...}, at: [<c10aefb4>] kill_fasync+0x37/0x71
> > kernel: [ 3272.351232] {IN-HARDIRQ-W} state was registered at:
> > kernel: [ 3272.351235] [<c104e95c>] __lock_acquire+0x281/0xbe1
> > kernel: [ 3272.351243] [<c104f652>] lock_acquire+0x59/0x70
> > kernel: [ 3272.351248] [<c12c6c48>] _raw_spin_lock+0x25/0x34
> > kernel: [ 3272.351255] [<c10aefb4>] kill_fasync+0x37/0x71
> > kernel: [ 3272.351261] [<fd220c81>] evdev_event+0x135/0x190 [evdev]
> > kernel: [ 3272.351275] [<c1232003>] input_pass_event+0x6f/0xae
> > kernel: [ 3272.351283] [<c1232ef5>] input_handle_event+0x38d/0x396
> > kernel: [ 3272.351288] [<c1232fbf>] input_event+0x4f/0x62
> > kernel: [ 3272.351293] [<c12368e4>] input_sync+0xe/0x11
> > kernel: [ 3272.351299] [<c1236d72>] atkbd_interrupt+0x48b/0x541
> > kernel: [ 3272.351304] [<c122ecb2>] serio_interrupt+0x35/0x68
> > kernel: [ 3272.351309] [<c122fbff>] i8042_interrupt+0x264/0x26e
> > kernel: [ 3272.351314] [<c106bb02>] handle_IRQ_event+0x1d/0x98
> > kernel: [ 3272.351321] [<c106d506>] handle_edge_irq+0xc0/0x107
> > kernel: [ 3272.351326] [<c10045ca>] handle_irq+0x1a/0x20
> > kernel: [ 3272.351332] [<c100435f>] do_IRQ+0x43/0x8d
> > kernel: [ 3272.351337] [<c1002d75>] common_interrupt+0x35/0x3c
> > kernel: [ 3272.351342] [<c124723d>] cpuidle_idle_call+0x6a/0xa0
> > kernel: [ 3272.351349] [<c100170d>] cpu_idle+0x89/0xbe
> > kernel: [ 3272.351354] [<c12b6d11>] rest_init+0xb5/0xba
> > kernel: [ 3272.351361] [<c148a7bf>] start_kernel+0x33b/0x340
> > kernel: [ 3272.351368] [<c148a0c9>] i386_start_kernel+0xc9/0xd0
> > kernel: [ 3272.351374] irq event stamp: 54104917
> > kernel: [ 3272.351377] hardirqs last enabled at (54104917): [<c12c70f2>] _raw_spin_unlock_irqrestore+0x36/0x5b
> > kernel: [ 3272.351384] hardirqs last disabled at (54104916): [<c12c6ced>] _raw_spin_lock_irqsave+0x13/0x42
> > kernel: [ 3272.351391] softirqs last enabled at (54104732): [<c1032cf2>] __do_softirq+0xfd/0x10c
> > kernel: [ 3272.351398] softirqs last disabled at (54104703): [<c1032d30>] do_softirq+0x2f/0x47
> > kernel: [ 3272.351404]
> > kernel: [ 3272.351405] other info that might help us debug this:
> > kernel: [ 3272.351409] 3 locks held by X/3827:
> > kernel: [ 3272.351412] #0: (rcu_read_lock){.+.+..}, at: [<c124fdfa>] rcu_read_lock+0x0/0x26
> > kernel: [ 3272.351423] #1: (rcu_read_lock){.+.+..}, at: [<c124d5d9>] rcu_read_lock+0x0/0x26
> > kernel: [ 3272.351432] #2: (rcu_read_lock){.+.+..}, at: [<c10ae429>] rcu_read_lock+0x0/0x26
> > kernel: [ 3272.351442]
> > kernel: [ 3272.351443] stack backtrace:
> > kernel: [ 3272.351448] Pid: 3827, comm: X Not tainted 2.6.35-rc3-dbg-00106-ga75e02b-dirty #15
> > kernel: [ 3272.351451] Call Trace:
> > kernel: [ 3272.351456] [<c12c4ff1>] ? printk+0xf/0x11
> > kernel: [ 3272.351462] [<c104e51a>] valid_state+0x133/0x141
> > kernel: [ 3272.351468] [<c104e5f7>] mark_lock+0xcf/0x1b3
> > kernel: [ 3272.351473] [<c104e54e>] ? mark_lock+0x26/0x1b3
> > kernel: [ 3272.351479] [<c104dfd2>] ? check_usage_backwards+0x0/0x68
> > kernel: [ 3272.351484] [<c104e9d0>] __lock_acquire+0x2f5/0xbe1
> > kernel: [ 3272.351489] [<c104ea44>] ? __lock_acquire+0x369/0xbe1
> > kernel: [ 3272.351495] [<c104ea44>] ? __lock_acquire+0x369/0xbe1
> > kernel: [ 3272.351502] [<c102ab40>] ? try_to_wake_up+0x2a8/0x2bb
> > kernel: [ 3272.351508] [<c104f652>] lock_acquire+0x59/0x70
> > kernel: [ 3272.351513] [<c10aefb4>] ? kill_fasync+0x37/0x71
> > kernel: [ 3272.351519] [<c12c6c48>] _raw_spin_lock+0x25/0x34
> > kernel: [ 3272.351524] [<c10aefb4>] ? kill_fasync+0x37/0x71
> > kernel: [ 3272.351529] [<c10aefb4>] kill_fasync+0x37/0x71
> > kernel: [ 3272.351534] [<c124d694>] sock_wake_async+0x77/0x83
> > kernel: [ 3272.351540] [<c124fe4d>] sk_wake_async+0x2d/0x32
> > kernel: [ 3272.351545] [<c1250004>] sock_def_readable+0x45/0x51
> > kernel: [ 3272.351551] [<c12b0247>] unix_stream_sendmsg+0x1e2/0x269
> > kernel: [ 3272.351557] [<c124fe6e>] ? rcu_read_unlock+0x1c/0x1e
> > kernel: [ 3272.351562] [<c124cf1a>] __sock_sendmsg+0x51/0x5a
> > kernel: [ 3272.351567] [<c124cff7>] sock_aio_write+0xd4/0xdd
> > kernel: [ 3272.351575] [<c10a4d95>] do_sync_readv_writev+0x84/0xb7
> > kernel: [ 3272.351582] [<c10a4288>] ? copy_from_user+0x8/0xa
> > kernel: [ 3272.351587] [<c10a4e69>] ? rw_copy_check_uvector+0x55/0xc7
> > kernel: [ 3272.351594] [<c1164082>] ? security_file_permission+0xf/0x11
> > kernel: [ 3272.351599] [<c10a47e5>] ? rw_verify_area+0x90/0xac
> > kernel: [ 3272.351605] [<c10a4f58>] do_readv_writev+0x7d/0xdf
> > kernel: [ 3272.351610] [<c124cf23>] ? sock_aio_write+0x0/0xdd
> > kernel: [ 3272.351615] [<c1164082>] ? security_file_permission+0xf/0x11
> > kernel: [ 3272.351621] [<c10a47e5>] ? rw_verify_area+0x90/0xac
> > kernel: [ 3272.351626] [<c10a4ff3>] vfs_writev+0x39/0x42
> > kernel: [ 3272.351632] [<c10a5102>] sys_writev+0x3b/0x8c
> > kernel: [ 3272.351637] [<c10027d3>] sysenter_do_call+0x12/0x32
> >
>
> This, I think?
>
>
> From: Andrew Morton <akpm@linux-foundation.org>
>
> Fix a lockdep-splat-causing regression introduced by
>
> : commit 989a2979205dd34269382b357e6d4b4b6956b889
> : Author: Eric Dumazet <eric.dumazet@gmail.com>
> : AuthorDate: Wed Apr 14 09:55:35 2010 +0000
> : Commit: David S. Miller <davem@davemloft.net>
> : CommitDate: Wed Apr 21 16:19:29 2010 -0700
> :
> : fasync: RCU and fine grained locking
>
> kill_fasync() can be called from both process and hard-irq context, so
> fa_lock must be taken with IRQs disabled.
>
> Addresses https://bugzilla.kernel.org/show_bug.cgi?id=16230
>
> Reported-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> Reported-by: Dominik Brodowski <linux@dominikbrodowski.net>
> Cc: Maciej Rutecki <maciej.rutecki@gmail.com>
> Cc: Eric Dumazet <eric.dumazet@gmail.com>
> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
> Cc: "David S. Miller" <davem@davemloft.net>
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> ---
>
> fs/fcntl.c | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff -puN fs/fcntl.c~fs-fcntlc-kill_fasync_rcu-fa_lock-must-be-irq-safe fs/fcntl.c
> --- a/fs/fcntl.c~fs-fcntlc-kill_fasync_rcu-fa_lock-must-be-irq-safe
> +++ a/fs/fcntl.c
> @@ -733,12 +733,14 @@ static void kill_fasync_rcu(struct fasyn
> {
> while (fa) {
> struct fown_struct *fown;
> + unsigned long flags;
> +
> if (fa->magic != FASYNC_MAGIC) {
> printk(KERN_ERR "kill_fasync: bad magic number in "
> "fasync_struct!\n");
> return;
> }
> - spin_lock(&fa->fa_lock);
> + spin_lock_irqsave(&fa->fa_lock, flags);
> if (fa->fa_file) {
> fown = &fa->fa_file->f_owner;
> /* Don't send SIGURG to processes which have not set a
> @@ -747,7 +749,7 @@ static void kill_fasync_rcu(struct fasyn
> if (!(sig == SIGURG && fown->signum == 0))
> send_sigio(fown, fa->fa_fd, band);
> }
> - spin_unlock(&fa->fa_lock);
> + spin_unlock_irqrestore(&fa->fa_lock, flags);
> fa = rcu_dereference(fa->fa_next);
> }
> }
> _
>
>
> afaict all other lockers of fa_lock are OK (but one never really knows
> with spin_lock_irq()).
>
> Guys, please review-and-ack and I'll get it merged up.
>
Sorry for the delay, I was travelling...
Alternative solution would be to change fasync_remove_entry() and
fasync_add_entry() to use spin_lock_irq(&fasync_lock) and
spin_lock(&fa->fa_lock), but we would disable IRQ on possibly long scans
(as before my "fasync: RCU and fine grained locking" patch).
Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: inconsistent lock state
2010-06-18 20:30 ` inconsistent lock state Andrew Morton
` (3 preceding siblings ...)
2010-06-22 8:59 ` Eric Dumazet
@ 2010-06-22 17:17 ` Paul E. McKenney
4 siblings, 0 replies; 10+ messages in thread
From: Paul E. McKenney @ 2010-06-22 17:17 UTC (permalink / raw)
To: Andrew Morton
Cc: Sergey Senozhatsky, Alexander Viro, Peter Zijlstra, Sage Weil,
linux-fsdevel, linux-kernel, Dominik Brodowski, Maciej Rutecki,
Eric Dumazet, Lai Jiangshan, David S. Miller, netdev
On Fri, Jun 18, 2010 at 01:30:04PM -0700, Andrew Morton wrote:
>
> This was also reported by Dominik and is being tracked at
> https://bugzilla.kernel.org/show_bug.cgi?id=16230
>
> On Tue, 15 Jun 2010 14:24:34 +0300
> Sergey Senozhatsky <sergey.senozhatsky@gmail.com> wrote:
>
> > Hello,
> >
> > kernel: [ 3272.351191]
> > kernel: [ 3272.351194] =================================
> > kernel: [ 3272.351199] [ INFO: inconsistent lock state ]
> > kernel: [ 3272.351204] 2.6.35-rc3-dbg-00106-ga75e02b-dirty #15
> > kernel: [ 3272.351206] ---------------------------------
> > kernel: [ 3272.351210] inconsistent {IN-HARDIRQ-W} -> {HARDIRQ-ON-W} usage.
> > kernel: [ 3272.351215] X/3827 [HC0[0]:SC0[0]:HE1:SE1] takes:
> > kernel: [ 3272.351218] (&(&new->fa_lock)->rlock){?.-...}, at: [<c10aefb4>] kill_fasync+0x37/0x71
> > kernel: [ 3272.351232] {IN-HARDIRQ-W} state was registered at:
> > kernel: [ 3272.351235] [<c104e95c>] __lock_acquire+0x281/0xbe1
> > kernel: [ 3272.351243] [<c104f652>] lock_acquire+0x59/0x70
> > kernel: [ 3272.351248] [<c12c6c48>] _raw_spin_lock+0x25/0x34
> > kernel: [ 3272.351255] [<c10aefb4>] kill_fasync+0x37/0x71
> > kernel: [ 3272.351261] [<fd220c81>] evdev_event+0x135/0x190 [evdev]
> > kernel: [ 3272.351275] [<c1232003>] input_pass_event+0x6f/0xae
> > kernel: [ 3272.351283] [<c1232ef5>] input_handle_event+0x38d/0x396
> > kernel: [ 3272.351288] [<c1232fbf>] input_event+0x4f/0x62
> > kernel: [ 3272.351293] [<c12368e4>] input_sync+0xe/0x11
> > kernel: [ 3272.351299] [<c1236d72>] atkbd_interrupt+0x48b/0x541
> > kernel: [ 3272.351304] [<c122ecb2>] serio_interrupt+0x35/0x68
> > kernel: [ 3272.351309] [<c122fbff>] i8042_interrupt+0x264/0x26e
> > kernel: [ 3272.351314] [<c106bb02>] handle_IRQ_event+0x1d/0x98
> > kernel: [ 3272.351321] [<c106d506>] handle_edge_irq+0xc0/0x107
> > kernel: [ 3272.351326] [<c10045ca>] handle_irq+0x1a/0x20
> > kernel: [ 3272.351332] [<c100435f>] do_IRQ+0x43/0x8d
> > kernel: [ 3272.351337] [<c1002d75>] common_interrupt+0x35/0x3c
> > kernel: [ 3272.351342] [<c124723d>] cpuidle_idle_call+0x6a/0xa0
> > kernel: [ 3272.351349] [<c100170d>] cpu_idle+0x89/0xbe
> > kernel: [ 3272.351354] [<c12b6d11>] rest_init+0xb5/0xba
> > kernel: [ 3272.351361] [<c148a7bf>] start_kernel+0x33b/0x340
> > kernel: [ 3272.351368] [<c148a0c9>] i386_start_kernel+0xc9/0xd0
> > kernel: [ 3272.351374] irq event stamp: 54104917
> > kernel: [ 3272.351377] hardirqs last enabled at (54104917): [<c12c70f2>] _raw_spin_unlock_irqrestore+0x36/0x5b
> > kernel: [ 3272.351384] hardirqs last disabled at (54104916): [<c12c6ced>] _raw_spin_lock_irqsave+0x13/0x42
> > kernel: [ 3272.351391] softirqs last enabled at (54104732): [<c1032cf2>] __do_softirq+0xfd/0x10c
> > kernel: [ 3272.351398] softirqs last disabled at (54104703): [<c1032d30>] do_softirq+0x2f/0x47
> > kernel: [ 3272.351404]
> > kernel: [ 3272.351405] other info that might help us debug this:
> > kernel: [ 3272.351409] 3 locks held by X/3827:
> > kernel: [ 3272.351412] #0: (rcu_read_lock){.+.+..}, at: [<c124fdfa>] rcu_read_lock+0x0/0x26
> > kernel: [ 3272.351423] #1: (rcu_read_lock){.+.+..}, at: [<c124d5d9>] rcu_read_lock+0x0/0x26
> > kernel: [ 3272.351432] #2: (rcu_read_lock){.+.+..}, at: [<c10ae429>] rcu_read_lock+0x0/0x26
> > kernel: [ 3272.351442]
> > kernel: [ 3272.351443] stack backtrace:
> > kernel: [ 3272.351448] Pid: 3827, comm: X Not tainted 2.6.35-rc3-dbg-00106-ga75e02b-dirty #15
> > kernel: [ 3272.351451] Call Trace:
> > kernel: [ 3272.351456] [<c12c4ff1>] ? printk+0xf/0x11
> > kernel: [ 3272.351462] [<c104e51a>] valid_state+0x133/0x141
> > kernel: [ 3272.351468] [<c104e5f7>] mark_lock+0xcf/0x1b3
> > kernel: [ 3272.351473] [<c104e54e>] ? mark_lock+0x26/0x1b3
> > kernel: [ 3272.351479] [<c104dfd2>] ? check_usage_backwards+0x0/0x68
> > kernel: [ 3272.351484] [<c104e9d0>] __lock_acquire+0x2f5/0xbe1
> > kernel: [ 3272.351489] [<c104ea44>] ? __lock_acquire+0x369/0xbe1
> > kernel: [ 3272.351495] [<c104ea44>] ? __lock_acquire+0x369/0xbe1
> > kernel: [ 3272.351502] [<c102ab40>] ? try_to_wake_up+0x2a8/0x2bb
> > kernel: [ 3272.351508] [<c104f652>] lock_acquire+0x59/0x70
> > kernel: [ 3272.351513] [<c10aefb4>] ? kill_fasync+0x37/0x71
> > kernel: [ 3272.351519] [<c12c6c48>] _raw_spin_lock+0x25/0x34
> > kernel: [ 3272.351524] [<c10aefb4>] ? kill_fasync+0x37/0x71
> > kernel: [ 3272.351529] [<c10aefb4>] kill_fasync+0x37/0x71
> > kernel: [ 3272.351534] [<c124d694>] sock_wake_async+0x77/0x83
> > kernel: [ 3272.351540] [<c124fe4d>] sk_wake_async+0x2d/0x32
> > kernel: [ 3272.351545] [<c1250004>] sock_def_readable+0x45/0x51
> > kernel: [ 3272.351551] [<c12b0247>] unix_stream_sendmsg+0x1e2/0x269
> > kernel: [ 3272.351557] [<c124fe6e>] ? rcu_read_unlock+0x1c/0x1e
> > kernel: [ 3272.351562] [<c124cf1a>] __sock_sendmsg+0x51/0x5a
> > kernel: [ 3272.351567] [<c124cff7>] sock_aio_write+0xd4/0xdd
> > kernel: [ 3272.351575] [<c10a4d95>] do_sync_readv_writev+0x84/0xb7
> > kernel: [ 3272.351582] [<c10a4288>] ? copy_from_user+0x8/0xa
> > kernel: [ 3272.351587] [<c10a4e69>] ? rw_copy_check_uvector+0x55/0xc7
> > kernel: [ 3272.351594] [<c1164082>] ? security_file_permission+0xf/0x11
> > kernel: [ 3272.351599] [<c10a47e5>] ? rw_verify_area+0x90/0xac
> > kernel: [ 3272.351605] [<c10a4f58>] do_readv_writev+0x7d/0xdf
> > kernel: [ 3272.351610] [<c124cf23>] ? sock_aio_write+0x0/0xdd
> > kernel: [ 3272.351615] [<c1164082>] ? security_file_permission+0xf/0x11
> > kernel: [ 3272.351621] [<c10a47e5>] ? rw_verify_area+0x90/0xac
> > kernel: [ 3272.351626] [<c10a4ff3>] vfs_writev+0x39/0x42
> > kernel: [ 3272.351632] [<c10a5102>] sys_writev+0x3b/0x8c
> > kernel: [ 3272.351637] [<c10027d3>] sysenter_do_call+0x12/0x32
> >
>
> This, I think?
>
>
> From: Andrew Morton <akpm@linux-foundation.org>
>
> Fix a lockdep-splat-causing regression introduced by
>
> : commit 989a2979205dd34269382b357e6d4b4b6956b889
> : Author: Eric Dumazet <eric.dumazet@gmail.com>
> : AuthorDate: Wed Apr 14 09:55:35 2010 +0000
> : Commit: David S. Miller <davem@davemloft.net>
> : CommitDate: Wed Apr 21 16:19:29 2010 -0700
> :
> : fasync: RCU and fine grained locking
>
> kill_fasync() can be called from both process and hard-irq context, so
> fa_lock must be taken with IRQs disabled.
>
> Addresses https://bugzilla.kernel.org/show_bug.cgi?id=16230
>
> Reported-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> Reported-by: Dominik Brodowski <linux@dominikbrodowski.net>
> Cc: Maciej Rutecki <maciej.rutecki@gmail.com>
> Cc: Eric Dumazet <eric.dumazet@gmail.com>
> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
> Cc: "David S. Miller" <davem@davemloft.net>
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> ---
>
> fs/fcntl.c | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff -puN fs/fcntl.c~fs-fcntlc-kill_fasync_rcu-fa_lock-must-be-irq-safe fs/fcntl.c
> --- a/fs/fcntl.c~fs-fcntlc-kill_fasync_rcu-fa_lock-must-be-irq-safe
> +++ a/fs/fcntl.c
> @@ -733,12 +733,14 @@ static void kill_fasync_rcu(struct fasyn
> {
> while (fa) {
> struct fown_struct *fown;
> + unsigned long flags;
> +
> if (fa->magic != FASYNC_MAGIC) {
> printk(KERN_ERR "kill_fasync: bad magic number in "
> "fasync_struct!\n");
> return;
> }
> - spin_lock(&fa->fa_lock);
> + spin_lock_irqsave(&fa->fa_lock, flags);
> if (fa->fa_file) {
> fown = &fa->fa_file->f_owner;
> /* Don't send SIGURG to processes which have not set a
> @@ -747,7 +749,7 @@ static void kill_fasync_rcu(struct fasyn
> if (!(sig == SIGURG && fown->signum == 0))
> send_sigio(fown, fa->fa_fd, band);
> }
> - spin_unlock(&fa->fa_lock);
> + spin_unlock_irqrestore(&fa->fa_lock, flags);
> fa = rcu_dereference(fa->fa_next);
> }
> }
> _
>
>
> afaict all other lockers of fa_lock are OK (but one never really knows
> with spin_lock_irq()).
>
> Guys, please review-and-ack and I'll get it merged up.
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2010-06-22 17:17 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20100615112434.GA3967@swordfish.minsk.epam.com>
2010-06-18 20:30 ` inconsistent lock state Andrew Morton
2010-06-19 8:33 ` Sergey Senozhatsky
2010-06-19 13:05 ` Dominik Brodowski
2010-06-21 6:54 ` Peter Zijlstra
2010-06-22 8:59 ` Eric Dumazet
2010-06-22 17:17 ` Paul E. McKenney
[not found] <20100307192305.GA598@elte.hu>
2010-03-08 12:51 ` Oleg Nesterov
2010-03-15 21:01 ` Eric Dumazet
2010-03-15 21:09 ` David Miller
2010-03-07 10:21 Sergey Senozhatsky
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).