another Ultra2 SMP crash

another Ultra2 SMP crash

[Chris Ricker]

This is with 2.6.3-rc2, compiled SMP

All I had on the serial console was

spin_lock(fffff8003fef5eb0) CPU#1 stuck at 0045c0fc, owner PC(0045c0fc):CPU(1)
spin_lock(fffff8003fef5eb0) CPU#1 stuck at 0045c0fc, owner PC(0045c0fc):CPU(1)
spin_lock(fffff80000011140) CPU#0 stuck at 0044a3a4, owner PC(0044a3a4):CPU(1)
spin_lock(fffff80000011140) CPU#0 stuck at 0044a3a4, owner PC(0044a3a4):CPU(1)
spin_lock(fffff8003fef5eb0) CPU#1 stuck at 0045c0fc, owner PC(0045c0fc):CPU(1)
spin_lock(fffff80000011140) CPU#0 stuck at 0044a3a4, owner PC(0044a3a4):CPU(1)

0045c0fc has nearby:

000000000045bde0 t send_signal
000000000045bf60 t specific_send_sig_info
000000000045c0e0 T force_sig_info
000000000045c200 T force_sig_specific
000000000045c300 T zap_other_threads
000000000045c3e0 T group_send_sig_info


0044a3a4 has nearby:

000000000044a080 t __set_cpus_allowed
000000000044a1c0 T wait_task_inactive
000000000044a2e0 T kick_process
000000000044a360 t try_to_wake_up
000000000044a6a0 T wake_up_process
000000000044a6c0 T wake_up_state
000000000044a6e0 T sched_fork

Please let me know if any other info would be useful

thanks,
chris

Re: another Ultra2 SMP crash

[Ben Collins]

On Wed, Feb 18, 2004 at 04:29:30PM -0500, Chris Ricker wrote:
> This is with 2.6.3-rc2, compiled SMP
> 
> All I had on the serial console was

Do you have preempt enabled? I've already noticed from other people that
it is causing problems (one showed as a race conditition with autoload
of modules).

-- 
Debian     - http://www.debian.org/
Linux 1394 - http://www.linux1394.org/
Subversion - http://subversion.tigris.org/
WatchGuard - http://www.watchguard.com/

Re: another Ultra2 SMP crash

[Chris Ricker]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 486 bytes --]

On Wed, 18 Feb 2004, Ben Collins wrote:

> On Wed, Feb 18, 2004 at 04:29:30PM -0500, Chris Ricker wrote:
> > This is with 2.6.3-rc2, compiled SMP
> > 
> > All I had on the serial console was
> 
> Do you have preempt enabled? I've already noticed from other people that
> it is causing problems (one showed as a race conditition with autoload
> of modules).

I don't have it enabled.

I've attached my complete config, in case there's anything else potentially 
suspicious

thanks,
chris

[-- Attachment #2: Type: APPLICATION/octet-stream, Size: 5998 bytes --]

Re: another Ultra2 SMP crash

[Chris Ricker]

On Wed, 18 Feb 2004, Chris Ricker wrote:

After I upgraded to 2.6.3-smp, I got another crash on a stuck spinlock:

spin_lock(00000000006481c8) CPU#1 stuck at 0046ae40, owner PC(0046ae40):CPU(1)
spin_lock(fffff8003e291370) CPU#0 stuck at 005c5390, owner PC(005848a8):CPU(1)
spin_lock(00000000006481c8) CPU#1 stuck at 0046ae40, owner PC(0046ae40):CPU(1)
spin_lock(fffff8003e291370) CPU#0 stuck at 005c5390, owner PC(005848a8):CPU(1)
spin_lock(00000000006481c8) CPU#1 stuck at 0046ae40, owner PC(0046ae40):CPU(1)
spin_lock(fffff8003e291370) CPU#0 stuck at 005c5390, owner PC(005848a8):CPU(1)

0046ae40 is near:

000000000046ad00 t m_next
000000000046ad40 t m_stop
000000000046ad60 t m_show
000000000046ae20 T search_module_extables
000000000046aec0 T module_text_address
000000000046af40 T struct_module
000000000046af60 T kallsyms_lookup

005c5390 is near:

00000000005c4a60 t tcp_v4_checksum_init
00000000005c4be0 T tcp_v4_do_rcv
00000000005c4d40 T tcp_v4_rcv
00000000005c5820 t __tcp_v4_rehash
00000000005c5860 t tcp_v4_reselect_saddr
00000000005c5ac0 T tcp_v4_rebuild_header
00000000005c5d00 t v4_addr2sockaddr

The 006481c8 spinlock is modlist_lock:

00000000006481a0 D cpucontrol
00000000006481c8 d modlist_lock
00000000006481d8 d module_mutex
0000000000648200 d modules

Re: another Ultra2 SMP crash

[Chris Ricker]

On Fri, 20 Feb 2004, Chris Ricker wrote:

> On Wed, 18 Feb 2004, Chris Ricker wrote:
> 
> After I upgraded to 2.6.3-smp, I got another crash on a stuck spinlock:
> 
> spin_lock(00000000006481c8) CPU#1 stuck at 0046ae40, owner PC(0046ae40):CPU(1)
> spin_lock(fffff8003e291370) CPU#0 stuck at 005c5390, owner PC(005848a8):CPU(1)
> spin_lock(00000000006481c8) CPU#1 stuck at 0046ae40, owner PC(0046ae40):CPU(1)
> spin_lock(fffff8003e291370) CPU#0 stuck at 005c5390, owner PC(005848a8):CPU(1)
> spin_lock(00000000006481c8) CPU#1 stuck at 0046ae40, owner PC(0046ae40):CPU(1)
> spin_lock(fffff8003e291370) CPU#0 stuck at 005c5390, owner PC(005848a8):CPU(1)
> 
> 0046ae40 is near:
> 
> 000000000046ad00 t m_next
> 000000000046ad40 t m_stop
> 000000000046ad60 t m_show
> 000000000046ae20 T search_module_extables
> 000000000046aec0 T module_text_address
> 000000000046af40 T struct_module
> 000000000046af60 T kallsyms_lookup
> 
> 005c5390 is near:
> 
> 00000000005c4a60 t tcp_v4_checksum_init
> 00000000005c4be0 T tcp_v4_do_rcv
> 00000000005c4d40 T tcp_v4_rcv
> 00000000005c5820 t __tcp_v4_rehash
> 00000000005c5860 t tcp_v4_reselect_saddr
> 00000000005c5ac0 T tcp_v4_rebuild_header
> 00000000005c5d00 t v4_addr2sockaddr
> 
> The 006481c8 spinlock is modlist_lock:
> 
> 00000000006481a0 D cpucontrol
> 00000000006481c8 d modlist_lock
> 00000000006481d8 d module_mutex
> 0000000000648200 d modules

Oops, hit send too soon.

I was doing an scp off of the system at the time it crashed, so the network 
would've been active. I can't think of anything happening on it which 
would've been modules-related, though.

thanks,
chris