* shm broken on MIPS in current -git
@ 2011-08-01 12:51 Manuel Lauss
2011-08-01 13:11 ` Vasiliy Kulikov
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Manuel Lauss @ 2011-08-01 12:51 UTC (permalink / raw)
To: Vasiliy Kulikov, linux-kernel; +Cc: Linux-MIPS
Hello Vasiliy,
Commits 5774ed014f02120db9a6945a1ecebeb97c2acccb
(shm: handle separate PID namespaces case)
and 4c677e2eefdba9c5bfc4474e2e91b26ae8458a1d
(shm: optimize locking and ipc_namespace getting)
break on my MIPS systems. The following oops is
printed on boot, and as a result, more than 300 zombie kworker
kernel processes are resident. I don't see this oops on x86 or x64.
ra points to the down_write() in ipc/shm.c::exit_shm()
CPU 0 Unable to handle kernel paging request at virtual address
00000000, epc == 80527c64, ra == 8024afb8
Oops[#1]:
Cpu 0
$ 0 : 00000000 10003c00 00000000 10003c01
$ 4 : 8060d114 8fc45e60 8060d118 8fc3f520
$ 8 : 00000000 01312d00 0016e35f 00020000
$12 : 00000080 00000010 00000010 8fc3001c
$16 : 8fc3f520 00000000 00000000 00000000
$20 : 00000000 00000000 8fc45eb4 00000000
$24 : 00000000 8011f9b0
$28 : 8fc44000 8fc45e50 00000001 8024afb8
Hi : 00000000
Lo : 01312d00
epc : 80527c64 __down_write_nested+0x68/0xf0
Not tainted
ra : 8024afb8 exit_shm+0x30/0x64
Status: 10003c02 KERNEL EXL
Cause : 0080800c
BadVA : 00000000
PrId : 800c8000 (Au1300)
Process kworker/u:0 (pid: 9, threadinfo=8fc44000, task=8fc3f520, tls=00000000)
Stack : 14200972 d3054429 00000000 56b8e493 8060d118 00000000 8fc3f520 00000002
8fc2c000 8060d114 00000000 8024afb8 00000000 00000000 00000000 00000000
8060d0c0 00000000 8fc3f520 80128ae8 30480023 0b309f84 34ffeedb ef9e65d6
38019941 af430015 f6d9ebeb 00000000 00000000 00000000 8fc29ce0 8fc15300
00000000 00000000 00000000 00000000 00000000 00000000 00000000 80139578
...
Call Trace:
[<80527c64>] __down_write_nested+0x68/0xf0
[<8024afb8>] exit_shm+0x30/0x64
[<80128ae8>] do_exit+0x50c/0x664
[<80139578>] ____call_usermodehelper+0xfc/0x118
[<801061bc>] kernel_thread_helper+0x10/0x18
Code: ac850008 afa60010 afa20014 <ac450000> 40016000 30630001
3421001f 3821001f 00611825
Disabling lock debugging due to kernel taint
Fixing recursive fault but reboot is needed!
Thanks,
Manuel Lauss
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: shm broken on MIPS in current -git
2011-08-01 12:51 shm broken on MIPS in current -git Manuel Lauss
@ 2011-08-01 13:11 ` Vasiliy Kulikov
2011-08-01 13:22 ` Manuel Lauss
2011-08-01 13:23 ` Vasiliy Kulikov
2011-08-01 13:39 ` Vasiliy Kulikov
2 siblings, 1 reply; 6+ messages in thread
From: Vasiliy Kulikov @ 2011-08-01 13:11 UTC (permalink / raw)
To: Manuel Lauss; +Cc: linux-kernel, Linux-MIPS
Hi Manuel,
On Mon, Aug 01, 2011 at 14:51 +0200, Manuel Lauss wrote:
> Commits 5774ed014f02120db9a6945a1ecebeb97c2acccb
> (shm: handle separate PID namespaces case)
> and 4c677e2eefdba9c5bfc4474e2e91b26ae8458a1d
> (shm: optimize locking and ipc_namespace getting)
> break on my MIPS systems.
Can you please identify what commit breaks the system?
> The following oops is
> printed on boot, and as a result, more than 300 zombie kworker
> kernel processes are resident. I don't see this oops on x86 or x64.
Do you have the same configs for x86 and mips?
> ra points to the down_write() in ipc/shm.c::exit_shm()
>
> CPU 0 Unable to handle kernel paging request at virtual address
> 00000000, epc == 80527c64, ra == 8024afb8
> Oops[#1]:
> Cpu 0
> $ 0 : 00000000 10003c00 00000000 10003c01
> $ 4 : 8060d114 8fc45e60 8060d118 8fc3f520
> $ 8 : 00000000 01312d00 0016e35f 00020000
> $12 : 00000080 00000010 00000010 8fc3001c
> $16 : 8fc3f520 00000000 00000000 00000000
> $20 : 00000000 00000000 8fc45eb4 00000000
> $24 : 00000000 8011f9b0
> $28 : 8fc44000 8fc45e50 00000001 8024afb8
> Hi : 00000000
> Lo : 01312d00
> epc : 80527c64 __down_write_nested+0x68/0xf0
> Not tainted
> ra : 8024afb8 exit_shm+0x30/0x64
> Status: 10003c02 KERNEL EXL
> Cause : 0080800c
> BadVA : 00000000
> PrId : 800c8000 (Au1300)
> Process kworker/u:0 (pid: 9, threadinfo=8fc44000, task=8fc3f520, tls=00000000)
> Stack : 14200972 d3054429 00000000 56b8e493 8060d118 00000000 8fc3f520 00000002
> 8fc2c000 8060d114 00000000 8024afb8 00000000 00000000 00000000 00000000
> 8060d0c0 00000000 8fc3f520 80128ae8 30480023 0b309f84 34ffeedb ef9e65d6
> 38019941 af430015 f6d9ebeb 00000000 00000000 00000000 8fc29ce0 8fc15300
> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 80139578
> ...
> Call Trace:
> [<80527c64>] __down_write_nested+0x68/0xf0
> [<8024afb8>] exit_shm+0x30/0x64
> [<80128ae8>] do_exit+0x50c/0x664
> [<80139578>] ____call_usermodehelper+0xfc/0x118
> [<801061bc>] kernel_thread_helper+0x10/0x18
>
>
> Code: ac850008 afa60010 afa20014 <ac450000> 40016000 30630001
> 3421001f 3821001f 00611825
> Disabling lock debugging due to kernel taint
> Fixing recursive fault but reboot is needed!
>
>
> Thanks,
> Manuel Lauss
--
Vasiliy Kulikov
http://www.openwall.com - bringing security into open computing environments
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: shm broken on MIPS in current -git
2011-08-01 13:11 ` Vasiliy Kulikov
@ 2011-08-01 13:22 ` Manuel Lauss
0 siblings, 0 replies; 6+ messages in thread
From: Manuel Lauss @ 2011-08-01 13:22 UTC (permalink / raw)
To: Vasiliy Kulikov; +Cc: linux-kernel, Linux-MIPS
On Mon, Aug 1, 2011 at 3:11 PM, Vasiliy Kulikov <segoon@openwall.com> wrote:
> On Mon, Aug 01, 2011 at 14:51 +0200, Manuel Lauss wrote:
>> Commits 5774ed014f02120db9a6945a1ecebeb97c2acccb
>> (shm: handle separate PID namespaces case)
>> and 4c677e2eefdba9c5bfc4474e2e91b26ae8458a1d
>> (shm: optimize locking and ipc_namespace getting)
>> break on my MIPS systems.
>
> Can you please identify what commit breaks the system?
Reverting this one fixes the oops for good:
5774ed014f02120db9a6945a1ecebeb97c2acccb
(shm: handle separate PID namespaces case)
>> The following oops is
>> printed on boot, and as a result, more than 300 zombie kworker
>> kernel processes are resident. I don't see this oops on x86 or x64.
>
> Do you have the same configs for x86 and mips?
MIPS is UP, SLAB without preempt, x86 is UP/SMP, SLUB with preempt.
It's not SLAB/SLUB and not preempt, and both MIPS+x86 have all
namespace options enabled.
both configs are temporarily available here:
http://mlau.at/config-mips
http://mlau.at/config-x64
Thanks!
Manuel Lauss
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: shm broken on MIPS in current -git
2011-08-01 12:51 shm broken on MIPS in current -git Manuel Lauss
2011-08-01 13:11 ` Vasiliy Kulikov
@ 2011-08-01 13:23 ` Vasiliy Kulikov
2011-08-01 13:39 ` Vasiliy Kulikov
2 siblings, 0 replies; 6+ messages in thread
From: Vasiliy Kulikov @ 2011-08-01 13:23 UTC (permalink / raw)
To: Manuel Lauss; +Cc: linux-kernel, Linux-MIPS
On Mon, Aug 01, 2011 at 14:51 +0200, Manuel Lauss wrote:
> Commits 5774ed014f02120db9a6945a1ecebeb97c2acccb
> (shm: handle separate PID namespaces case)
> and 4c677e2eefdba9c5bfc4474e2e91b26ae8458a1d
> (shm: optimize locking and ipc_namespace getting)
> break on my MIPS systems. The following oops is
> printed on boot, and as a result, more than 300 zombie kworker
> kernel processes are resident. I don't see this oops on x86 or x64.
Also it might help if you send the whole config file to me (privately,
as it is too big for ML).
Thanks,
--
Vasiliy Kulikov
http://www.openwall.com - bringing security into open computing environments
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: shm broken on MIPS in current -git
2011-08-01 12:51 shm broken on MIPS in current -git Manuel Lauss
2011-08-01 13:11 ` Vasiliy Kulikov
2011-08-01 13:23 ` Vasiliy Kulikov
@ 2011-08-01 13:39 ` Vasiliy Kulikov
2011-08-01 14:57 ` Manuel Lauss
2 siblings, 1 reply; 6+ messages in thread
From: Vasiliy Kulikov @ 2011-08-01 13:39 UTC (permalink / raw)
To: Manuel Lauss; +Cc: linux-kernel, Linux-MIPS
Manuel,
On Mon, Aug 01, 2011 at 14:51 +0200, Manuel Lauss wrote:
> CPU 0 Unable to handle kernel paging request at virtual address
> 00000000, epc == 80527c64, ra == 8024afb8
> Oops[#1]:
[...]
> epc : 80527c64 __down_write_nested+0x68/0xf0
Could you post the asm code of __down_write_nested() of your image?
What pointer is NULL? Looks like MIPS handles namespaces in usermode
helpers other way.
> ra : 8024afb8 exit_shm+0x30/0x64
> Status: 10003c02 KERNEL EXL
> Cause : 0080800c
> BadVA : 00000000
> PrId : 800c8000 (Au1300)
> Process kworker/u:0 (pid: 9, threadinfo=8fc44000, task=8fc3f520, tls=00000000)
> Stack : 14200972 d3054429 00000000 56b8e493 8060d118 00000000 8fc3f520 00000002
> 8fc2c000 8060d114 00000000 8024afb8 00000000 00000000 00000000 00000000
> 8060d0c0 00000000 8fc3f520 80128ae8 30480023 0b309f84 34ffeedb ef9e65d6
> 38019941 af430015 f6d9ebeb 00000000 00000000 00000000 8fc29ce0 8fc15300
> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 80139578
> ...
> Call Trace:
> [<80527c64>] __down_write_nested+0x68/0xf0
> [<8024afb8>] exit_shm+0x30/0x64
> [<80128ae8>] do_exit+0x50c/0x664
> [<80139578>] ____call_usermodehelper+0xfc/0x118
> [<801061bc>] kernel_thread_helper+0x10/0x18
Thanks,
--
Vasiliy Kulikov
http://www.openwall.com - bringing security into open computing environments
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: shm broken on MIPS in current -git
2011-08-01 13:39 ` Vasiliy Kulikov
@ 2011-08-01 14:57 ` Manuel Lauss
0 siblings, 0 replies; 6+ messages in thread
From: Manuel Lauss @ 2011-08-01 14:57 UTC (permalink / raw)
To: Vasiliy Kulikov; +Cc: linux-kernel, Linux-MIPS
On Mon, Aug 1, 2011 at 3:39 PM, Vasiliy Kulikov <segoon@openwall.com> wrote:
> Manuel,
>
> On Mon, Aug 01, 2011 at 14:51 +0200, Manuel Lauss wrote:
>> CPU 0 Unable to handle kernel paging request at virtual address
>> 00000000, epc == 80527c64, ra == 8024afb8
>> Oops[#1]:
> [...]
>> epc : 80527c64 __down_write_nested+0x68/0xf0
>
> Could you post the asm code of __down_write_nested() of your image?
> What pointer is NULL? Looks like MIPS handles namespaces in usermode
> helpers other way.
I'm no MIPS expert, but this looks like a problem with the waiter_list
in struct rw_semaphore:
Dump of assembler code for function __down_write_nested:
0x80530ee4 <+0>: addiu sp,sp,-48
0x80530ee8 <+4>: sw ra,44(sp)
0x80530eec <+8>: sw s1,40(sp)
0x80530ef0 <+12>: sw s0,36(sp)
0x80530ef4 <+16>: mfc0 v1,c0_status
0x80530ef8 <+20>: ori at,v1,0x1f
0x80530efc <+24>: xori at,at,0x1f
0x80530f00 <+28>: mtc0 at,c0_status
0x80530f04 <+32>: lw v0,0(a0)
0x80530f08 <+36>: beqz v0,0x80530fa4 <__down_write_nested+192>
0x80530f0c <+40>: addiu a2,a0,4
tsk = current:
0x80530f10 <+44>: lw s0,0(gp)
set_task_state(tsk, UNINTERRUPTIBLE):
0x80530f14 <+48>: li v0,2
0x80530f18 <+52>: sw v0,0(s0)
waiter.tsk = tsk, ...
0x80530f1c <+56>: sw v0,28(sp)
0x80530f20 <+60>: sw s0,24(sp)
get_task_struct():
0x80530f24 <+64>: addiu v0,s0,8
0x80530f28 <+68>: ll a1,0(v0)
0x80530f2c <+72>: addiu a1,a1,1
0x80530f30 <+76>: sc a1,0(v0)
0x80530f34 <+80>: beqz a1,0x80530f28 <__down_write_nested+68>
list_add_tail:
0x80530f38 <+84>: addiu a1,sp,16 ; a1 = &waiter.list
0x80530f3c <+88>: lw v0,8(a0) ; v0 = &sem->wait_list
0x80530f40 <+92>: sw a1,8(a0)
0x80530f44 <+96>: sw a2,16(sp)
0x80530f48 <+100>: sw v0,20(sp)
0x80530f4c <+104>: sw a1,0(v0) <===== oops here, because v0 = 0
spin_unlock_irqrestore:
0x80530f50 <+108>: mfc0 at,c0_status
0x80530f54 <+112>: andi v1,v1,0x1
0x80530f58 <+116>: ori at,at,0x1f
0x80530f5c <+120>: xori at,at,0x1f
0x80530f60 <+124>: or v1,v1,at
0x80530f64 <+128>: mtc0 v1,c0_status
[...]
0x80530f68 <+132>: lw v0,24(sp)
0x80530f6c <+136>: beqz v0,0x80530f8c <__down_write_nested+168>
0x80530f70 <+140>: li s1,2
0x80530f74 <+144>: jal 0x8052ef94 <schedule>
0x80530f78 <+148>: nop
0x80530f7c <+152>: sw s1,0(s0)
0x80530f80 <+156>: lw v0,24(sp)
0x80530f84 <+160>: bnez v0,0x80530f74 <__down_write_nested+144>
0x80530f88 <+164>: nop
0x80530f8c <+168>: sw zero,0(s0)
0x80530f90 <+172>: lw ra,44(sp)
0x80530f94 <+176>: lw s1,40(sp)
0x80530f98 <+180>: lw s0,36(sp)
0x80530f9c <+184>: jr ra
0x80530fa0 <+188>: addiu sp,sp,48
0x80530fa4 <+192>: lw v0,4(a0)
0x80530fa8 <+196>: bne v0,a2,0x80530f10 <__down_write_nested+44>
0x80530fac <+200>: li a1,-1
0x80530fb0 <+204>: sw a1,0(a0)
0x80530fb4 <+208>: mfc0 at,c0_status
0x80530fb8 <+212>: andi v1,v1,0x1
0x80530fbc <+216>: ori at,at,0x1f
0x80530fc0 <+220>: xori at,at,0x1f
0x80530fc4 <+224>: or v1,v1,at
0x80530fc8 <+228>: mtc0 v1,c0_status
0x80530fcc <+232>: j 0x80530f94 <__down_write_nested+176>
0x80530fd0 <+236>: lw ra,44(sp)
Manuel Lauss
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-08-01 14:58 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-08-01 12:51 shm broken on MIPS in current -git Manuel Lauss
2011-08-01 13:11 ` Vasiliy Kulikov
2011-08-01 13:22 ` Manuel Lauss
2011-08-01 13:23 ` Vasiliy Kulikov
2011-08-01 13:39 ` Vasiliy Kulikov
2011-08-01 14:57 ` Manuel Lauss
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.