All of lore.kernel.org
 help / color / mirror / Atom feed
From: Kamalesh Babulal <kamalesh@linux.vnet.ibm.com>
To: Ingo Molnar <mingo@elte.hu>
Cc: Dhaval Giani <dhaval@linux.vnet.ibm.com>,
	Jens Axboe <jens.axboe@oracle.com>,
	Srivatsa Vaddagiri <vatsa@linux.vnet.ibm.com>,
	LKML <linux-kernel@vger.kernel.org>,
	linuxppc-dev@ozlabs.org, Balbir Singh <balbir@linux.vnet.ibm.com>
Subject: Re: [BUG] 2.6.25-rc1-git1 softlockup while bootup on powerpc
Date: Thu, 14 Feb 2008 15:29:56 +0530	[thread overview]
Message-ID: <47B4111C.1010301@linux.vnet.ibm.com> (raw)
In-Reply-To: <20080212075812.GC11775@elte.hu>

Ingo Molnar wrote:
> * Kamalesh Babulal <kamalesh@linux.vnet.ibm.com> wrote:
> 
>> While booting with the 2.6.25-rc1-git1 kernel on the powerbox the 
>> softlockup is seen, with following trace.
> 
>> BUG: soft lockup - CPU#1 stuck for 61s! [insmod:377]
>> TASK = c00000077cb2f0e0[377] 'insmod' THREAD: c00000077cb28000 CPU: 1
>> NIP [c0000000001b172c] .radix_tree_gang_lookup+0xdc/0x1e4
>> LR [c0000000001a6f00] .call_for_each_cic+0x50/0x10c
>> Call Trace:
>> [c00000077cb2bb20] [c0000000001a6f60] .call_for_each_cic+0xb0/0x10c (unreliable)
>> [c00000077cb2bc60] [c00000000019ecd8] .exit_io_context+0xf0/0x110
>> [c00000077cb2bcf0] [c00000000006254c] .do_exit+0x820/0x850
>> [c00000077cb2bda0] [c000000000062648] .do_group_exit+0xcc/0xe8
>> [c00000077cb2be30] [c00000000000872c] syscall_exit+0x0/0x40
> 
> this call_for_each_cic/radix_tree_gang_lookup locked up, and all other 
> CPUs deadlocked in stopmachine, due to this one.
> 
> call_for_each_cic is in ./block/cfq-iosched.c uses RCU, but you've got 
> classic-RCU:
> 
>   CONFIG_CLASSIC_RCU=y
>   # CONFIG_PREEMPT_RCU is not set
> 
> so it's not related to the preempt-RCU changes either.
> 
> It is this part that locks up:
> 
>         do {
> ...
>                 nr = radix_tree_gang_lookup(&ioc->radix_root, (void **) cics,
>                                                 index, CIC_GANG_NR);
> ...
>         } while (nr == CIC_GANG_NR);
> ...
> 
> it seems the radix tree will yield new entries again and again. Either 
> it got corrupted, or some other CPU is filling it faster than we can 
> deplete it [unlikely i think].
> 
> 	Ingo

This softlockup is seen with the 2.6.25-rc1-git3 also. Let me know if you
need more details.

-- 
Thanks & Regards,
Kamalesh Babulal,
Linux Technology Center,
IBM, ISTL.

WARNING: multiple messages have this Message-ID (diff)
From: Kamalesh Babulal <kamalesh@linux.vnet.ibm.com>
To: Ingo Molnar <mingo@elte.hu>
Cc: LKML <linux-kernel@vger.kernel.org>,
	linuxppc-dev@ozlabs.org, Dhaval Giani <dhaval@linux.vnet.ibm.com>,
	Srivatsa Vaddagiri <vatsa@linux.vnet.ibm.com>,
	Andy Whitcroft <apw@shadowen.org>,
	Balbir Singh <balbir@linux.vnet.ibm.com>,
	Jens Axboe <jens.axboe@oracle.com>
Subject: Re: [BUG] 2.6.25-rc1-git1 softlockup while bootup on powerpc
Date: Thu, 14 Feb 2008 15:29:56 +0530	[thread overview]
Message-ID: <47B4111C.1010301@linux.vnet.ibm.com> (raw)
In-Reply-To: <20080212075812.GC11775@elte.hu>

Ingo Molnar wrote:
> * Kamalesh Babulal <kamalesh@linux.vnet.ibm.com> wrote:
> 
>> While booting with the 2.6.25-rc1-git1 kernel on the powerbox the 
>> softlockup is seen, with following trace.
> 
>> BUG: soft lockup - CPU#1 stuck for 61s! [insmod:377]
>> TASK = c00000077cb2f0e0[377] 'insmod' THREAD: c00000077cb28000 CPU: 1
>> NIP [c0000000001b172c] .radix_tree_gang_lookup+0xdc/0x1e4
>> LR [c0000000001a6f00] .call_for_each_cic+0x50/0x10c
>> Call Trace:
>> [c00000077cb2bb20] [c0000000001a6f60] .call_for_each_cic+0xb0/0x10c (unreliable)
>> [c00000077cb2bc60] [c00000000019ecd8] .exit_io_context+0xf0/0x110
>> [c00000077cb2bcf0] [c00000000006254c] .do_exit+0x820/0x850
>> [c00000077cb2bda0] [c000000000062648] .do_group_exit+0xcc/0xe8
>> [c00000077cb2be30] [c00000000000872c] syscall_exit+0x0/0x40
> 
> this call_for_each_cic/radix_tree_gang_lookup locked up, and all other 
> CPUs deadlocked in stopmachine, due to this one.
> 
> call_for_each_cic is in ./block/cfq-iosched.c uses RCU, but you've got 
> classic-RCU:
> 
>   CONFIG_CLASSIC_RCU=y
>   # CONFIG_PREEMPT_RCU is not set
> 
> so it's not related to the preempt-RCU changes either.
> 
> It is this part that locks up:
> 
>         do {
> ...
>                 nr = radix_tree_gang_lookup(&ioc->radix_root, (void **) cics,
>                                                 index, CIC_GANG_NR);
> ...
>         } while (nr == CIC_GANG_NR);
> ...
> 
> it seems the radix tree will yield new entries again and again. Either 
> it got corrupted, or some other CPU is filling it faster than we can 
> deplete it [unlikely i think].
> 
> 	Ingo

This softlockup is seen with the 2.6.25-rc1-git3 also. Let me know if you
need more details.

-- 
Thanks & Regards,
Kamalesh Babulal,
Linux Technology Center,
IBM, ISTL.

  reply	other threads:[~2008-02-14 10:00 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-02-12  7:09 [BUG] 2.6.25-rc1-git1 softlockup while bootup on powerpc Kamalesh Babulal
2008-02-12  7:58 ` Ingo Molnar
2008-02-12  7:58   ` Ingo Molnar
2008-02-14  9:59   ` Kamalesh Babulal [this message]
2008-02-14  9:59     ` Kamalesh Babulal

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=47B4111C.1010301@linux.vnet.ibm.com \
    --to=kamalesh@linux.vnet.ibm.com \
    --cc=balbir@linux.vnet.ibm.com \
    --cc=dhaval@linux.vnet.ibm.com \
    --cc=jens.axboe@oracle.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linuxppc-dev@ozlabs.org \
    --cc=mingo@elte.hu \
    --cc=vatsa@linux.vnet.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.