From: Ming Lei <tom.leiming@gmail.com>
To: ebiederm@xmission.com (Eric W. Biederman)
Cc: Greg KH <greg@kroah.com>,
Linus Torvalds <torvalds@linux-foundation.org>,
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
Borislav Petkov <petkovbb@googlemail.com>,
David Airlie <airlied@linux.ie>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Al Viro <viro@ZenIV.linux.org.uk>, Tejun Heo <tj@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
linux-pcmcia@lists.infradead.org,
Dominik Brodowski <linux@dominikbrodowski.net>
Subject: Re: [PATCH] sysfs: Add lockdep annotations for the sysfs active reference
Date: Mon, 18 Jan 2010 00:26:47 +0800 [thread overview]
Message-ID: <20100118002647.35035e77@tom-lei> (raw)
In-Reply-To: <m11vi8yyrb.fsf_-_@fess.ebiederm.org>
On Sat, 02 Jan 2010 13:37:12 -0800
ebiederm@xmission.com (Eric W. Biederman) wrote:
>
> Holding locks over device_del -> kobject_del -> sysfs_deactivate can
> cause deadlocks if those same locks are grabbed in sysfs show or store
> methods.
>
> The I model s_active count + completion as a sleeping read/write lock.
> I describe to lockdep sysfs_get_active as a read_trylock,
> sysfs_put_active as a read_unlock, and sysfs_deactivate as a
> write_lock and write_unlock pair. This seems to capture the essence
> for purposes of finding deadlocks, and in my testing gives finds real
> issues and ignores non-issues.
>
> This brings us back to holding locks over kobject_del is a problem
> that ideally we should find a way of addressing, but at least lockdep
> can tell us about the problems instead of requiring developers to
> debug rare strange system deadlocks, that happen when sysfs files are
> removed while being written to.
The model has hit a possible deadlock in pcmcia, and the lockdep warning
comes when I unplug my wlan card from pcmcia slot.
Looks like socket->skt_mutex is held in remove path, and it is also
grabbed in .stor method.
[ 9207.755883] pcmcia_socket pcmcia_socket0: pccard: card ejected from slot 0
[ 9207.786583]
[ 9207.786586] =======================================================
[ 9207.786595] [ INFO: possible circular locking dependency detected ]
[ 9207.786602] 2.6.33-rc4-wl #8
[ 9207.786607] -------------------------------------------------------
[ 9207.786614] pccardd/841 is trying to acquire lock:
[ 9207.786620] (s_active){++++.+}, at: [<ffffffff811637f1>] sysfs_addrm_finish+0x36/0x55
[ 9207.786643]
[ 9207.786645] but task is already holding lock:
[ 9207.786651] (&socket->skt_mutex){+.+.+.}, at: [<ffffffff812ed04c>] pccardd+0x15d/0x25f
[ 9207.786669]
[ 9207.786671] which lock already depends on the new lock.
[ 9207.786674]
[ 9207.786679]
[ 9207.786680] the existing dependency chain (in reverse order) is:
[ 9207.786687]
[ 9207.786688] -> #1 (&socket->skt_mutex){+.+.+.}:
[ 9207.786702] [<ffffffff810796c0>] __lock_acquire+0xb73/0xd2b
[ 9207.786716] [<ffffffff8107a36b>] lock_acquire+0xe1/0x105
[ 9207.786726] [<ffffffff813b5ac5>] __mutex_lock_common+0x59/0x49d
[ 9207.786741] [<ffffffff813b5fbe>] mutex_lock_nested+0x39/0x3e
[ 9207.786752] [<ffffffff812ef3ed>] pccard_store_resource+0x6b/0xc5
[ 9207.786763] [<ffffffff812a55da>] dev_attr_store+0x20/0x22
[ 9207.786775] [<ffffffff8116259a>] sysfs_write_file+0x108/0x144
[ 9207.786787] [<ffffffff8110cf48>] vfs_write+0xae/0x10b
[ 9207.786798] [<ffffffff8110d065>] sys_write+0x4a/0x6e
[ 9207.786808] [<ffffffff81009bc2>] system_call_fastpath+0x16/0x1b
[ 9207.786822]
[ 9207.786824] -> #0 (s_active){++++.+}:
[ 9207.786835] [<ffffffff8107956a>] __lock_acquire+0xa1d/0xd2b
[ 9207.786847] [<ffffffff8107a36b>] lock_acquire+0xe1/0x105
[ 9207.786857] [<ffffffff81163230>] sysfs_deactivate+0x8b/0xe0
[ 9207.786868] [<ffffffff811637f1>] sysfs_addrm_finish+0x36/0x55
[ 9207.786879] [<ffffffff81161c26>] sysfs_hash_and_remove+0x53/0x6a
[ 9207.786890] [<ffffffff811629fd>] sysfs_remove_file+0x15/0x17
[ 9207.786900] [<ffffffff812a650e>] device_remove_file+0x17/0x19
[ 9207.786911] [<ffffffff81213ef7>] pci_remove_sysfs_dev_files+0x6b/0x10c
[ 9207.786924] [<ffffffff8120dce1>] pci_stop_bus_device+0x55/0x83
[ 9207.786936] [<ffffffff8120dd99>] pci_remove_bus_device+0x1a/0xba
[ 9207.786947] [<ffffffff8120de5f>] pci_remove_behind_bridge+0x26/0x3f
[ 9207.786958] [<ffffffff812efb96>] cb_free+0x4a/0x4f
[ 9207.786969] [<ffffffff812ec682>] socket_shutdown+0x91/0xfd
[ 9207.786979] [<ffffffff812ec88a>] socket_remove+0x4e/0x57
[ 9207.786989] [<ffffffff812ed08b>] pccardd+0x19c/0x25f
[ 9207.787000] [<ffffffff810674dd>] kthread+0x7f/0x87
[ 9207.787011] [<ffffffff8100aa64>] kernel_thread_helper+0x4/0x10
[ 9207.787023]
[ 9207.787025] other info that might help us debug this:
[ 9207.787027]
[ 9207.787034] 1 lock held by pccardd/841:
[ 9207.787039] #0: (&socket->skt_mutex){+.+.+.}, at: [<ffffffff812ed04c>] pccardd+0x15d/0x25f
[ 9207.787058]
[ 9207.787060] stack backtrace:
[ 9207.787068] Pid: 841, comm: pccardd Not tainted 2.6.33-rc4-wl #8
[ 9207.787069] Call Trace:
[ 9207.787069] [<ffffffff8107871d>] print_circular_bug+0xa8/0xb6
[ 9207.787069] [<ffffffff8107956a>] __lock_acquire+0xa1d/0xd2b
[ 9207.787069] [<ffffffff811637f1>] ? sysfs_addrm_finish+0x36/0x55
[ 9207.787069] [<ffffffff8107a36b>] lock_acquire+0xe1/0x105
[ 9207.787069] [<ffffffff811637f1>] ? sysfs_addrm_finish+0x36/0x55
[ 9207.787069] [<ffffffff81163230>] sysfs_deactivate+0x8b/0xe0
[ 9207.787069] [<ffffffff811637f1>] ? sysfs_addrm_finish+0x36/0x55
[ 9207.787069] [<ffffffff8107760e>] ? trace_hardirqs_off+0xd/0xf
[ 9207.787069] [<ffffffff813b5897>] ? __mutex_unlock_slowpath+0x119/0x14e
[ 9207.787069] [<ffffffff811637f1>] sysfs_addrm_finish+0x36/0x55
[ 9207.787069] [<ffffffff81161c26>] sysfs_hash_and_remove+0x53/0x6a
[ 9207.787069] [<ffffffff811629fd>] sysfs_remove_file+0x15/0x17
[ 9207.787069] [<ffffffff812a650e>] device_remove_file+0x17/0x19
[ 9207.787069] [<ffffffff81213ef7>] pci_remove_sysfs_dev_files+0x6b/0x10c
[ 9207.787069] [<ffffffff8120dce1>] pci_stop_bus_device+0x55/0x83
[ 9207.787069] [<ffffffff8120dd99>] pci_remove_bus_device+0x1a/0xba
[ 9207.787069] [<ffffffff8120de5f>] pci_remove_behind_bridge+0x26/0x3f
[ 9207.787069] [<ffffffff812efb96>] cb_free+0x4a/0x4f
[ 9207.787069] [<ffffffff812ec682>] socket_shutdown+0x91/0xfd
[ 9207.787069] [<ffffffff812ec88a>] socket_remove+0x4e/0x57
[ 9207.787069] [<ffffffff812ed08b>] pccardd+0x19c/0x25f
[ 9207.787069] [<ffffffff812eceef>] ? pccardd+0x0/0x25f
[ 9207.787069] [<ffffffff810674dd>] kthread+0x7f/0x87
[ 9207.787069] [<ffffffff8100aa64>] kernel_thread_helper+0x4/0x10
[ 9207.787069] [<ffffffff813b8014>] ? restore_args+0x0/0x30
[ 9207.787069] [<ffffffff8106745e>] ? kthread+0x0/0x87
[ 9207.787069] [<ffffffff8100aa60>] ? kernel_thread_helper+0x0/0x10
[ 9207.788150] device: '0000:16:00.0': device_unregister
[ 9207.788166] PM: Removing info for pci:0000:16:00.0
[ 9207.788405] bus: 'pci': remove device 0000:16:00.0
next prev parent reply other threads:[~2010-01-17 16:27 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-12-24 22:00 Linux 2.6.33-rc2 - Merry Christmas Linus Torvalds
2009-12-25 10:27 ` -tip: origin tree boot crash Ingo Molnar
2009-12-25 19:49 ` Dmitry Torokhov
2009-12-26 20:19 ` Len Brown
2009-12-26 20:17 ` Len Brown
2009-12-27 4:20 ` Len Brown
2009-12-28 9:44 ` Ingo Molnar
2009-12-28 12:01 ` Ingo Molnar
2009-12-28 15:02 ` Paul Rolland
2009-12-28 16:15 ` Paul Rolland
2009-12-28 16:53 ` Paul Rolland
2009-12-28 20:17 ` Dmitry Torokhov
2009-12-30 6:14 ` Len Brown
2009-12-30 7:13 ` Paul Rolland
2009-12-30 6:19 ` [PATCH] wmi: check find_guid() return value to prevent oops Len Brown
2009-12-30 6:21 ` [PATCH] dell-wmi: sys_init_module: 'dell_wmi'->init suspiciously returned 21, it should follow 0/-E convention Len Brown
2009-12-25 13:10 ` Linux 2.6.33-rc2 - Blank screen for Intel KMS Miguel Calleja
2009-12-29 9:50 ` Miguel Calleja
2009-12-29 14:01 ` Rafael J. Wysocki
2009-12-25 20:00 ` Linux 2.6.33-rc2 - Merry Christmas Borislav Petkov
2009-12-25 21:50 ` Borislav Petkov
2009-12-26 6:00 ` Jesse Barnes
2009-12-26 8:02 ` Borislav Petkov
2009-12-26 9:36 ` EHCI resume sysfs duplicates (was: Re: Linux 2.6.33-rc2 - Merry Christmas ...) Borislav Petkov
2009-12-26 9:45 ` drm_vm.c:drm_mmap: possible circular locking dependency detected " Borislav Petkov
2009-12-28 0:40 ` KOSAKI Motohiro
2009-12-30 21:10 ` Linus Torvalds
2009-12-30 21:34 ` Eric W. Biederman
2009-12-30 22:03 ` Linus Torvalds
2009-12-31 8:40 ` Eric W. Biederman
2009-12-31 19:04 ` Linus Torvalds
2010-01-01 13:58 ` [PATCH] sysfs: Cache the last sysfs_dirent to improve readdir scalability Eric W. Biederman
2010-01-01 15:33 ` Borislav Petkov
2010-01-01 18:56 ` Linus Torvalds
2010-01-01 22:43 ` [PATCH] sysfs: Cache the last sysfs_dirent to improve readdir scalability v2 Eric W. Biederman
2010-01-01 23:10 ` Linus Torvalds
2010-01-02 5:59 ` Greg KH
2010-01-02 15:40 ` Borislav Petkov
2010-01-01 15:16 ` drm_vm.c:drm_mmap: possible circular locking dependency detected (was: Re: Linux 2.6.33-rc2 - Merry Christmas ...) Eric W. Biederman
2010-01-02 2:59 ` drm_vm.c:drm_mmap: possible circular locking dependency detected Tejun Heo
2010-01-02 21:37 ` [PATCH] sysfs: Add lockdep annotations for the sysfs active reference Eric W. Biederman
2010-01-03 0:02 ` Tejun Heo
2010-01-17 16:26 ` Ming Lei [this message]
2010-01-17 17:18 ` Eric W. Biederman
2010-01-17 18:03 ` Dominik Brodowski
2010-01-02 21:49 ` drm_vm.c:drm_mmap: possible circular locking dependency detected Eric W. Biederman
2010-01-03 0:32 ` Tejun Heo
2010-01-03 2:06 ` Eric W. Biederman
2010-01-03 5:01 ` Tejun Heo
2010-01-03 5:38 ` Eric W. Biederman
2010-01-03 6:05 ` Tejun Heo
2010-01-03 7:47 ` Dmitry Torokhov
2010-01-03 10:57 ` Eric W. Biederman
2010-01-03 11:14 ` Eric W. Biederman
2010-01-04 19:16 ` Dmitry Torokhov
2010-01-04 18:57 ` Dmitry Torokhov
2010-01-04 19:43 ` Eric W. Biederman
2010-01-04 21:12 ` Dmitry Torokhov
2010-01-04 23:09 ` Tejun Heo
2009-12-31 8:40 ` drm_vm.c:drm_mmap: possible circular locking dependency detected (was: Re: Linux 2.6.33-rc2 - Merry Christmas ...) Eric W. Biederman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20100118002647.35035e77@tom-lei \
--to=tom.leiming@gmail.com \
--cc=airlied@linux.ie \
--cc=ebiederm@xmission.com \
--cc=greg@kroah.com \
--cc=kosaki.motohiro@jp.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pcmcia@lists.infradead.org \
--cc=linux@dominikbrodowski.net \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=petkovbb@googlemail.com \
--cc=tj@kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=viro@ZenIV.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.