Dipankar Sarma wrote:

> Can you look at that each cpu is running (backtrace) using
> sysrq ? That may tell us what is holding up RCU. I will look
> at it myself later.

Okay, I'm not sure what you're looking for exactly, so I've included 
inline a number of sysrq-p dumps from different iterations right before 
the oom killer kicks in.  I've also attached a sysrq-t dump.

This is on a UP system running 2.6.14-rc2.

Pid: 913, comm:             rename14
EIP: 0060:[<c015b6ec>] CPU: 0
EIP is at link_path_walk+0x2c/0xe0
  EFLAGS: 00000202    Not tainted  (2.6.14-rc2-gc90eef84)
EAX: 00000007 EBX: dc35df28 ECX: 00000000 EDX: f7e7dbb8
ESI: dc35df68 EDI: dc35ded4 EBP: dc35dee0 DS: 007b ES: 007b
CR0: 8005003b CR2: b7db1000 CR3: 3470c000 CR4: 000006d0
  [<c0100f9d>] show_regs+0x14d/0x174
  [<c01f41b5>] sysrq_handle_showregs+0x15/0x20
  [<c01f43cf>] __handle_sysrq+0x7f/0x110
  [<c01f4493>] handle_sysrq+0x33/0x40
  [<c01f9dce>] receive_chars+0x22e/0x290
  [<c01fa141>] serial8250_interrupt+0xe1/0xf0
  [<c0130cfe>] handle_IRQ_event+0x3e/0x90
  [<c0130dab>] __do_IRQ+0x5b/0xc0
  [<c0104d4c>] do_IRQ+0x1c/0x30
  [<c0103586>] common_interrupt+0x1a/0x20
  [<c015b9f8>] path_lookup+0x78/0x120
  [<c015dd19>] sys_rename+0x99/0x240
  [<c0102b5f>] sysenter_past_esp+0x54/0x75

Pid: 919, comm:             rename14
EIP: 0060:[<c01c9e53>] CPU: 0
EIP is at _atomic_dec_and_lock+0x23/0x40
  EFLAGS: 00000246    Not tainted  (2.6.14-rc2-gc90eef84)
EAX: 00000000 EBX: f7fd66f0 ECX: f7fd66f0 EDX: 00000006
ESI: f7fd66f0 EDI: ce84ab2c EBP: f7fbff2c DS: 007b ES: 007b
CR0: 8005003b CR2: b7db1000 CR3: 06289000 CR4: 000006d0
  [<c0100f9d>] show_regs+0x14d/0x174
  [<c01f41b5>] sysrq_handle_showregs+0x15/0x20
  [<c01f43cf>] __handle_sysrq+0x7f/0x110
  [<c01f4493>] handle_sysrq+0x33/0x40
  [<c01f9dce>] receive_chars+0x22e/0x290
  [<c01fa141>] serial8250_interrupt+0xe1/0xf0
  [<c0130cfe>] handle_IRQ_event+0x3e/0x90
  [<c0130dab>] __do_IRQ+0x5b/0xc0
  [<c0104d4c>] do_IRQ+0x1c/0x30
  [<c0103586>] common_interrupt+0x1a/0x20
  [<c01641a6>] dput+0x56/0x1d0
  [<c014e098>] __fput+0x108/0x180
  [<c014df88>] fput+0x18/0x20
  [<c014c704>] filp_close+0x44/0x80
  [<c014c795>] sys_close+0x55/0x80
  [<c0102b5f>] sysenter_past_esp+0x54/0x75

Pid: 931, comm:             rename14
EIP: 0060:[<c01cebaf>] CPU: 0
EIP is at strncpy_from_user+0x3f/0x60
  EFLAGS: 00000216    Not tainted  (2.6.14-rc2-gc90eef84)
EAX: cd0d202f EBX: 0804aac9 ECX: 00000fff EDX: 00001000
ESI: 0804aaca EDI: dd477002 EBP: cd0d3edc DS: 007b ES: 007b
CR0: 8005003b CR2: b7e37000 CR3: 37eb7000 CR4: 000006d0
  [<c0100f9d>] show_regs+0x14d/0x174
  [<c01f41b5>] sysrq_handle_showregs+0x15/0x20
  [<c01f43cf>] __handle_sysrq+0x7f/0x110
  [<c01f4493>] handle_sysrq+0x33/0x40
  [<c01f9dce>] receive_chars+0x22e/0x290
  [<c01fa141>] serial8250_interrupt+0xe1/0xf0
  [<c0130cfe>] handle_IRQ_event+0x3e/0x90
  [<c0130dab>] __do_IRQ+0x5b/0xc0
  [<c0104d4c>] do_IRQ+0x1c/0x30
  [<c0103586>] common_interrupt+0x1a/0x20
  [<c015a235>] getname+0x75/0xc0
  [<c015dc9d>] sys_rename+0x1d/0x240
  [<c0102b5f>] sysenter_past_esp+0x54/0x75

Pid: 937, comm:             rename14
EIP: 0060:[<c015a386>] CPU: 0
EIP is at generic_permission+0x106/0x140
  EFLAGS: 00000246    Not tainted  (2.6.14-rc2-gc90eef84)
EAX: 00000000 EBX: 000041ff ECX: 00000000 EDX: f7fc10b0
ESI: ce84a994 EDI: 00000003 EBP: ca037e70 DS: 007b ES: 007b
CR0: 8005003b CR2: b7e37000 CR3: 0b40a000 CR4: 000006d0
  [<c0100f9d>] show_regs+0x14d/0x174
  [<c01f41b5>] sysrq_handle_showregs+0x15/0x20
  [<c01f43cf>] __handle_sysrq+0x7f/0x110
  [<c01f4493>] handle_sysrq+0x33/0x40
  [<c01f9dce>] receive_chars+0x22e/0x290
  [<c01fa141>] serial8250_interrupt+0xe1/0xf0
  [<c0130cfe>] handle_IRQ_event+0x3e/0x90
  [<c0130dab>] __do_IRQ+0x5b/0xc0
  [<c0104d4c>] do_IRQ+0x1c/0x30
  [<c0103586>] common_interrupt+0x1a/0x20
  [<c015a444>] permission+0x84/0xb0
  [<c015bdd0>] vfs_create+0x70/0xf0
  [<c015c140>] open_namei+0xe0/0x5f0
  [<c014c3a4>] filp_open+0x54/0xa0
  [<c014c5b9>] do_sys_open+0x59/0x100
  [<c014c67f>] sys_open+0x1f/0x30
  [<c014c6b1>] sys_creat+0x21/0x30
  [<c0102b5f>] sysenter_past_esp+0x54/0x75


Chris