From: Ingo Molnar <mingo@elte.hu>
To: linux-tip-commits@vger.kernel.org, Nick Piggin <npiggin@suse.de>,
Peter Zijlstra <a.p.zijlstra@chello.nl>,
Pekka Enberg <penberg@cs.helsinki.fi>,
Matt Mackall <mpm@selenic.com>
Cc: linux-kernel@vger.kernel.org, npiggin@suse.de, hpa@zytor.com,
mingo@redhat.com, a.p.zijlstra@chello.nl, tglx@linutronix.de
Subject: SLOB lockup (was: Re: [tip:core/locking] lockdep: annotate reclaim context (__GFP_NOFS), fix SLOB)
Date: Sun, 15 Mar 2009 07:48:18 +0100 [thread overview]
Message-ID: <20090315064818.GA1193@elte.hu> (raw)
In-Reply-To: <tip-bf722c9d324864b4256edaa330751b77f2a19861@git.kernel.org>
* Ingo Molnar <mingo@elte.hu> wrote:
> Commit-ID: bf722c9d324864b4256edaa330751b77f2a19861
> Gitweb: http://git.kernel.org/tip/bf722c9d324864b4256edaa330751b77f2a19861
> Author: Ingo Molnar <mingo@elte.hu>
> AuthorDate: Sun, 15 Mar 2009 06:03:11 +0100
> Commit: Ingo Molnar <mingo@elte.hu>
> CommitDate: Sun, 15 Mar 2009 06:03:11 +0100
>
> lockdep: annotate reclaim context (__GFP_NOFS), fix SLOB
>
> Impact: build fix
>
> fix typo in mm/slob.c:
>
> mm/slob.c:469: error: ‘flags’ undeclared (first use in this function)
> mm/slob.c:469: error: (Each undeclared identifier is reported only once
> mm/slob.c:469: error: for each function it appears in.)
>
> Cc: Nick Piggin <npiggin@suse.de>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> LKML-Reference: <20090128135457.350751756@chello.nl>
> Signed-off-by: Ingo Molnar <mingo@elte.hu>
and with this fixed, and with SLOB now being tested in -tip, the
new lockdep assert attached below (followed by a real lockup)
pops up.
Seems like a genuine SLOB bug, probably present upstream as
well.
Ingo
--------------------------->
Remounting root filesystem in read-write mode: [ 137.292031] EXT3 FS on sda6, internal journal
[ OK ]
[ 137.324031]
[ 137.324031] =============================================
[ 137.324031] [ INFO: possible recursive locking detected ]
[ 137.324031] 2.6.29-rc8-tip #35362
[ 137.324031] ---------------------------------------------
[ 137.324031] rc.sysinit/5461 is trying to acquire lock:
[ 137.324031] (slob_lock){-.-.-.}, at: [<ffffffff8031d446>] slob_free+0x66/0x290
[ 137.324031]
[ 137.324031] but task is already holding lock:
[ 137.324031] (slob_lock){-.-.-.}, at: [<ffffffff8031d446>] slob_free+0x66/0x290
[ 137.324031]
[ 137.324031] other info that might help us debug this:
[ 137.324031] 3 locks held by rc.sysinit/5461:
[ 137.324031] #0: (&sb->s_type->i_mutex_key#6){+.+.+.}, at: [<ffffffff80320f0e>] do_truncate+0x5e/0x90
[ 137.324031] #1: (&sb->s_type->i_alloc_sem_key#3){+.+...}, at: [<ffffffff80338815>] notify_change+0x245/0x320
[ 137.324031] #2: (slob_lock){-.-.-.}, at: [<ffffffff8031d446>] slob_free+0x66/0x290
[ 137.324031]
[ 137.324031] stack backtrace:
[ 137.324031] Pid: 5461, comm: rc.sysinit Not tainted 2.6.29-rc8-tip #35362
[ 137.324031] Call Trace:
[ 137.324031] <IRQ> [<ffffffff802a24c0>] validate_chain+0xbd0/0x12c0
[ 137.324031] [<ffffffff802a1dc9>] ? validate_chain+0x4d9/0x12c0
[ 137.324031] [<ffffffff802a2f36>] __lock_acquire+0x386/0xb40
[ 137.324031] [<ffffffff802a378f>] lock_acquire+0x9f/0x140
[ 137.324031] [<ffffffff8031d446>] ? slob_free+0x66/0x290
[ 137.324031] [<ffffffff80f05553>] _spin_lock_irqsave+0x53/0x90
[ 137.324031] [<ffffffff8031d446>] ? slob_free+0x66/0x290
[ 137.324031] [<ffffffff8031d446>] slob_free+0x66/0x290
[ 137.324031] [<ffffffff8031d6a5>] __kmem_cache_free+0x35/0x40
[ 137.324031] [<ffffffff8031d6d4>] kmem_cache_free+0x24/0x70
[ 137.324031] [<ffffffff806b97ec>] free_object+0x6c/0xd0
[ 137.324031] [<ffffffff806b99e3>] __debug_check_no_obj_freed+0x193/0x1d0
[ 137.324031] [<ffffffff8029dcad>] ? trace_hardirqs_off+0xd/0x10
[ 137.324031] [<ffffffff806b9a37>] debug_check_no_obj_freed+0x17/0x20
[ 137.324031] [<ffffffff802f3619>] free_hot_cold_page+0x109/0x2b0
[ 137.324031] [<ffffffff802f3830>] free_hot_page+0x10/0x20
[ 137.324031] [<ffffffff802f3865>] __free_pages+0x25/0x40
[ 137.324031] [<ffffffff802f38cf>] free_pages+0x4f/0x60
[ 137.324031] [<ffffffff8031d530>] slob_free+0x150/0x290
[ 137.324031] [<ffffffff8031d6a5>] __kmem_cache_free+0x35/0x40
[ 137.324031] [<ffffffff8031d6d4>] kmem_cache_free+0x24/0x70
[ 137.324031] [<ffffffff80333765>] __d_free+0x45/0x70
[ 137.324031] [<ffffffff80333da5>] d_callback+0x15/0x20
[ 137.324031] [<ffffffff802cdd92>] rcu_process_callbacks+0x82/0xd0
[ 137.324031] [<ffffffff8027cfd2>] __do_softirq+0xa2/0x220
[ 137.324031] [<ffffffff8022f21c>] call_softirq+0x1c/0x30
[ 137.324031] [<ffffffff8023138a>] do_softirq+0x6a/0xb0
[ 137.324031] [<ffffffff8027ceb7>] irq_exit+0x97/0xa0
[ 137.324031] [<ffffffff80248104>] smp_apic_timer_interrupt+0x74/0xb0
[ 137.324031] [<ffffffff8022ec13>] apic_timer_interrupt+0x13/0x20
[ 137.324031] <EOI> [<ffffffff802a37a4>] ? lock_acquire+0xb4/0x140
[ 137.324031] [<ffffffff80338815>] ? notify_change+0x245/0x320
[ 137.324031] [<ffffffff80f039b8>] ? down_write+0x48/0x80
[ 137.324031] [<ffffffff80338815>] ? notify_change+0x245/0x320
[ 137.324031] [<ffffffff806610e9>] ? cap_inode_setattr+0x9/0x10
[ 137.324031] [<ffffffff80338815>] ? notify_change+0x245/0x320
[ 137.324031] [<ffffffff80320f1a>] ? do_truncate+0x6a/0x90
[ 137.324031] [<ffffffff8026bcbe>] ? sub_preempt_count+0xae/0xf0
[ 137.324031] [<ffffffff80661209>] ? cap_path_truncate+0x9/0x10
[ 137.324031] [<ffffffff8032dc34>] ? may_open+0x224/0x2b0
[ 137.324031] [<ffffffff8032e18c>] ? do_filp_open+0x17c/0x980
[ 137.324031] [<ffffffff8029dd2e>] ? put_lock_stats+0xe/0x30
[ 137.324031] [<ffffffff8026bcbe>] ? sub_preempt_count+0xae/0xf0
[ 137.324031] [<ffffffff80339236>] ? alloc_fd+0x116/0x140
[ 137.324031] [<ffffffff803203a3>] ? do_sys_open+0x63/0xf0
[ 137.324031] [<ffffffff80320470>] ? sys_open+0x20/0x30
[ 137.324031] [<ffffffff8022dfbb>] ? system_call_fastpath+0x16/0x1b
[ 137.324031] BUG: spinlock lockup on CPU#1, rcu_torture_rea/697, ffffffff813bf440
[ 137.324031] Pid: 697, comm: rcu_torture_rea Not tainted 2.6.29-rc8-tip #35362
[ 137.324031] Call Trace:
[ 137.324031] <IRQ> [<ffffffff806a8ed9>] ? delay_loop+0x9/0x40
[ 137.324031] [<ffffffff806b91be>] _raw_spin_lock+0x17e/0x190
[ 137.324031] [<ffffffff80f05573>] _spin_lock_irqsave+0x73/0x90
[ 137.324031] [<ffffffff8031d878>] ? slob_alloc+0x58/0x200
[ 137.324031] [<ffffffff8031d878>] slob_alloc+0x58/0x200
[ 137.324031] [<ffffffff8031dcde>] kmem_cache_alloc_node+0xbe/0xf0
[ 137.324031] [<ffffffff80d0f74e>] __alloc_skb+0x4e/0x150
[ 137.324031] [<ffffffff80d0a677>] sock_alloc_send_skb+0x1a7/0x200
[ 137.324031] [<ffffffff802cdc70>] ? __rcu_read_unlock+0x20/0xc0
[ 137.324031] [<ffffffff80d71a57>] ip_append_data+0x6f7/0xa60
[ 137.324031] [<ffffffff80d95d20>] ? icmp_glue_bits+0x0/0x70
[ 137.324031] [<ffffffff80d95c68>] icmp_push_reply+0x58/0x110
[ 137.324031] [<ffffffff80d95f0f>] icmp_reply+0x17f/0x1e0
[ 137.324031] [<ffffffff80e8f74a>] ? csum_partial+0xa/0x180
[ 137.324031] [<ffffffff80d9642d>] icmp_echo+0x5d/0x60
[ 137.324031] BUG: spinlock lockup on CPU#0, rc.sysinit/5461, ffffffff813bf440
[ 137.324031] Pid: 5461, comm: rc.sysinit Not tainted 2.6.29-rc8-tip #35362
[ 137.324031] Call Trace:
[ 137.324031] <IRQ> [<ffffffff806a8ed9>] ? delay_loop+0x9/0x40
[ 137.324031] [<ffffffff806b91be>] _raw_spin_lock+0x17e/0x190
[ 137.324031] [<ffffffff80f05573>] _spin_lock_irqsave+0x73/0x90
[ 137.324031] [<ffffffff8031d446>] ? slob_free+0x66/0x290
[ 137.324031] [<ffffffff8031d446>] slob_free+0x66/0x290
[ 137.324031] [<ffffffff8031d6a5>] __kmem_cache_free+0x35/0x40
[ 137.324031] [<ffffffff8031d6d4>] kmem_cache_free+0x24/0x70
[ 137.324031] [<ffffffff806b97ec>] free_object+0x6c/0xd0
[ 137.324031] [<ffffffff806b99e3>] __debug_check_no_obj_freed+0x193/0x1d0
[ 137.324031] [<ffffffff8029dcad>] ? trace_hardirqs_off+0xd/0x10
[ 137.324031] [<ffffffff806b9a37>] debug_check_no_obj_freed+0x17/0x20
[ 137.324031] [<ffffffff802f3619>] free_hot_cold_page+0x109/0x2b0
[ 137.324031] [<ffffffff802f3830>] free_hot_page+0x10/0x20
[ 137.324031] [<ffffffff802f3865>] __free_pages+0x25/0x40
[ 137.324031] [<ffffffff802f38cf>] free_pages+0x4f/0x60
[ 137.324031] [<ffffffff8031d530>] slob_free+0x150/0x290
[ 137.324031] [<ffffffff8031d6a5>] __kmem_cache_free+0x35/0x40
[ 137.324031] [<ffffffff8031d6d4>] kmem_cache_free+0x24/0x70
[ 137.324031] [<ffffffff80333765>] __d_free+0x45/0x70
[ 137.324031] [<ffffffff80333da5>] d_callback+0x15/0x20
[ 137.324031] [<ffffffff802cdd92>] rcu_process_callbacks+0x82/0xd0
[ 137.324031] [<ffffffff8027cfd2>] __do_softirq+0xa2/0x220
[ 137.324031] [<ffffffff8022f21c>] call_softirq+0x1c/0x30
[ 137.324031] [<ffffffff8023138a>] do_softirq+0x6a/0xb0
[ 137.324031] [<ffffffff8027ceb7>] irq_exit+0x97/0xa0
[ 137.324031] [<ffffffff80248104>] smp_apic_timer_interrupt+0x74/0xb0
[ 137.324031] [<ffffffff8022ec13>] apic_timer_interrupt+0x13/0x20
[ 137.324031] <EOI> [<ffffffff802a37a4>] ? lock_acquire+0xb4/0x140
[ 137.324031] [<ffffffff80338815>] ? notify_change+0x245/0x320
[ 137.324031] [<ffffffff80f039b8>] ? down_write+0x48/0x80
[ 137.324031] [<ffffffff80338815>] ? notify_change+0x245/0x320
[ 137.324031] [<ffffffff806610e9>] ? cap_inode_setattr+0x9/0x10
[ 137.324031] [<ffffffff80338815>] ? notify_change+0x245/0x320
[ 137.324031] [<ffffffff80320f1a>] ? do_truncate+0x6a/0x90
[ 137.324031] [<ffffffff8026bcbe>] ? sub_preempt_count+0xae/0xf0
[ 137.324031] [<ffffffff80661209>] ? cap_path_truncate+0x9/0x10
[ 137.324031] [<ffffffff8032dc34>] ? may_open+0x224/0x2b0
[ 137.324031] [<ffffffff8032e18c>] ? do_filp_open+0x17c/0x980
[ 137.324031] [<ffffffff8029dd2e>] ? put_lock_stats+0xe/0x30
[ 137.324031] [<ffffffff8026bcbe>] ? sub_preempt_count+0xae/0xf0
[ 137.324031] [<ffffffff80339236>] ? alloc_fd+0x116/0x140
[ 137.324031] [<ffffffff803203a3>] ? do_sys_open+0x63/0xf0
[ 137.324031] [<ffffffff80320470>] ? sys_open+0x20/0x30
[ 137.324031] [<ffffffff8022dfbb>] ? system_call_fastpath+0x16/0x1b
[ 137.324031] [<ffffffff80d11c9b>] ? __skb_checksum_complete_head+0x1b/0x70
[ 137.324031] [<ffffffff80d11d01>] ? __skb_checksum_complete+0x11/0x20
[ 137.324031] [<ffffffff80d960a2>] icmp_rcv+0x132/0x2e0
[ 137.324031] [<ffffffff80d6e526>] ip_local_deliver_finish+0x76/0x1e0
[ 137.324031] [<ffffffff80d6eb30>] ip_local_deliver+0x40/0xa0
[ 137.324031] [<ffffffff80d6e219>] ip_rcv_finish+0x129/0x3c0
[ 137.324031] [<ffffffff80d6ea0c>] ip_rcv+0x23c/0x320
[ 137.324031] [<ffffffff80d16c9c>] netif_receive_skb+0x2dc/0x540
[ 137.324031] [<ffffffff808f3efd>] nv_napi_poll+0x3cd/0x6c0
[ 137.324031] [<ffffffff80d19d5e>] net_rx_action+0x13e/0x210
[ 137.324031] [<ffffffff8024a071>] ? irq_complete_move+0x21/0x240
[ 137.324031] [<ffffffff8027cfd2>] __do_softirq+0xa2/0x220
[ 137.324031] [<ffffffff8022f21c>] call_softirq+0x1c/0x30
[ 137.324031] [<ffffffff8023138a>] do_softirq+0x6a/0xb0
[ 137.324031] [<ffffffff8027ceb7>] irq_exit+0x97/0xa0
[ 137.324031] [<ffffffff802306a5>] do_IRQ+0x95/0x110
[ 137.324031] [<ffffffff8022ea13>] ret_from_intr+0x0/0xf
[ 137.324031] <EOI> [<ffffffff80f05a26>] ? _spin_unlock_irq+0x36/0x60
[ 137.324031] [<ffffffff80f0186d>] ? thread_return+0x1e3/0x916
[ 137.324031] [<ffffffff8022eac0>] ? restore_args+0x0/0x30
[ 137.324031] [<ffffffff802cbff8>] ? rcu_torture_reader+0x188/0x2e0
[ 137.324031] [<ffffffff8029fc9d>] ? trace_hardirqs_on+0xd/0x10
[ 137.324031] [<ffffffff802cbff8>] ? rcu_torture_reader+0x188/0x2e0
[ 137.324031] [<ffffffff802cc9a0>] ? rcu_torture_timer+0x0/0x150
[ 137.324031] [<ffffffff80f059b7>] ? _spin_unlock_irqrestore+0x47/0x80
[ 137.324031] [<ffffffff802cbe70>] ? rcu_torture_reader+0x0/0x2e0
[ 137.324031] [<ffffffff8028e213>] ? kthread+0x53/0x80
[ 137.324031] [<ffffffff8022f11a>] ? child_rip+0xa/0x20
[ 137.324031] [<ffffffff8026bb68>] ? finish_task_switch+0x98/0x140
[ 137.324031] [<ffffffff80f05a2b>] ? _spin_unlock_irq+0x3b/0x60
[ 137.324031] [<ffffffff8022eac0>] ? restore_args+0x0/0x30
[ 137.324031] [<ffffffff8028e1c0>] ? kthread+0x0/0x80
[ 137.324031] [<ffffffff8022f110>] ? child_rip+0x0/0x20
next prev parent reply other threads:[~2009-03-15 6:49 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-01-28 13:53 [PATCH 00/21] lockdep: pending queue Peter Zijlstra
2009-01-28 13:53 ` [PATCH 01/21] lockdep: annotate reclaim context (__GFP_NOFS) Peter Zijlstra
[not found] ` <tip-bf722c9d324864b4256edaa330751b77f2a19861@git.kernel.org>
2009-03-15 6:48 ` Ingo Molnar [this message]
2009-03-15 6:51 ` SLOB lockup (was: Re: [tip:core/locking] lockdep: annotate reclaim context (__GFP_NOFS), fix SLOB) Ingo Molnar
2009-03-15 9:06 ` Nick Piggin
2009-03-15 9:47 ` Ingo Molnar
2009-03-15 10:04 ` Nick Piggin
2009-03-15 15:16 ` Thomas Gleixner
2009-03-16 9:49 ` Nick Piggin
2009-03-17 11:30 ` [tip:core/debugobjects] debugobjects: replace static objects when slab cache becomes available Thomas Gleixner
2009-03-17 11:30 ` [tip:core/debugobjects] debugobjects: delay free of internal objects Thomas Gleixner
2009-03-15 14:56 ` SLOB lockup (was: Re: [tip:core/locking] lockdep: annotate reclaim context (__GFP_NOFS), fix SLOB) Matt Mackall
2009-03-16 10:00 ` Nick Piggin
2009-03-16 14:52 ` Matt Mackall
2009-03-16 15:00 ` Nick Piggin
2009-03-23 8:42 ` Pekka Enberg
2009-03-15 18:56 ` Ingo Molnar
2009-01-28 13:53 ` [PATCH 02/21] lockdep: sanitize bit names Peter Zijlstra
2009-01-28 13:53 ` [PATCH 03/21] lockdep: sanitize reclaim " Peter Zijlstra
2009-01-28 13:53 ` [PATCH 04/21] lockdep: lockdep_states.h Peter Zijlstra
2009-01-28 13:53 ` [PATCH 05/21] lockdep: simplify mark_held_locks Peter Zijlstra
2009-01-28 13:53 ` [PATCH 06/21] lockdep: simplify mark_lock() Peter Zijlstra
2009-01-28 13:53 ` [PATCH 07/21] lockdep: move state bit definitions around Peter Zijlstra
2009-01-28 13:54 ` [PATCH 08/21] lockdep: generate the state bit definitions Peter Zijlstra
2009-01-28 13:54 ` [PATCH 09/21] lockdep: generate usage strings Peter Zijlstra
2009-01-28 13:54 ` [PATCH 10/21] lockdep: split up mark_lock_irq() Peter Zijlstra
2009-01-28 13:54 ` [PATCH 11/21] lockdep: simplify the mark_lock_irq() helpers Peter Zijlstra
2009-01-28 13:54 ` [PATCH 12/21] lockdep: further simplify " Peter Zijlstra
2009-01-28 13:54 ` [PATCH 13/21] lockdep: simplify mark_lock_irq() helpers #3 Peter Zijlstra
2009-01-28 13:54 ` [PATCH 14/21] lockdep: merge the _READ mark_lock_irq() helpers Peter Zijlstra
2009-01-28 13:54 ` [PATCH 15/21] lockdep: merge the !_READ " Peter Zijlstra
2009-01-28 13:54 ` [PATCH 16/21] lockdep: fully reduce mark_lock_irq() Peter Zijlstra
2009-01-28 13:54 ` [PATCH 17/21] lockdep: remove macro usage from mark_held_locks() Peter Zijlstra
2009-01-28 13:54 ` [PATCH 18/21] lockdep: add comments to mark_lock_irq() Peter Zijlstra
2009-01-28 13:54 ` [PATCH 19/21] lockdep: simplify get_user_chars() Peter Zijlstra
2009-01-28 13:54 ` [PATCH 20/21] lockdep: get_user_chars() redo Peter Zijlstra
2009-01-28 13:54 ` [PATCH 21/21] lockdep: simplify check_prev_add_irq() Peter Zijlstra
2009-01-29 13:54 ` [PATCH 22/21] lockdep: use stringify.h Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20090315064818.GA1193@elte.hu \
--to=mingo@elte.hu \
--cc=a.p.zijlstra@chello.nl \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=mpm@selenic.com \
--cc=npiggin@suse.de \
--cc=penberg@cs.helsinki.fi \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox