From: Ming Lei <tom.leiming@gmail.com>
To: ebiederm@xmission.com (Eric W. Biederman)
Cc: Greg KH <greg@kroah.com>,
Linus Torvalds <torvalds@linux-foundation.org>,
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
Borislav Petkov <petkovbb@googlemail.com>,
David Airlie <airlied@linux.ie>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Al Viro <viro@ZenIV.linux.org.uk>, Tejun Heo <tj@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
linux-pcmcia@lists.infradead.org,
Dominik Brodowski <linux@dominikbrodowski.net>
Subject: Re: [PATCH] sysfs: Add lockdep annotations for the sysfs active reference
Date: Mon, 18 Jan 2010 00:26:47 +0800 [thread overview]
Message-ID: <20100118002647.35035e77@tom-lei> (raw)
In-Reply-To: <m11vi8yyrb.fsf_-_@fess.ebiederm.org>
On Sat, 02 Jan 2010 13:37:12 -0800
ebiederm@xmission.com (Eric W. Biederman) wrote:
>
> Holding locks over device_del -> kobject_del -> sysfs_deactivate can
> cause deadlocks if those same locks are grabbed in sysfs show or store
> methods.
>
> The I model s_active count + completion as a sleeping read/write lock.
> I describe to lockdep sysfs_get_active as a read_trylock,
> sysfs_put_active as a read_unlock, and sysfs_deactivate as a
> write_lock and write_unlock pair. This seems to capture the essence
> for purposes of finding deadlocks, and in my testing gives finds real
> issues and ignores non-issues.
>
> This brings us back to holding locks over kobject_del is a problem
> that ideally we should find a way of addressing, but at least lockdep
> can tell us about the problems instead of requiring developers to
> debug rare strange system deadlocks, that happen when sysfs files are
> removed while being written to.
The model has hit a possible deadlock in pcmcia, and the lockdep warning
comes when I unplug my wlan card from pcmcia slot.
Looks like socket->skt_mutex is held in remove path, and it is also
grabbed in .stor method.
[ 9207.755883] pcmcia_socket pcmcia_socket0: pccard: card ejected from slot 0
[ 9207.786583]
[ 9207.786586] =======================================================
[ 9207.786595] [ INFO: possible circular locking dependency detected ]
[ 9207.786602] 2.6.33-rc4-wl #8
[ 9207.786607] -------------------------------------------------------
[ 9207.786614] pccardd/841 is trying to acquire lock:
[ 9207.786620] (s_active){++++.+}, at: [<ffffffff811637f1>] sysfs_addrm_finish+0x36/0x55
[ 9207.786643]
[ 9207.786645] but task is already holding lock:
[ 9207.786651] (&socket->skt_mutex){+.+.+.}, at: [<ffffffff812ed04c>] pccardd+0x15d/0x25f
[ 9207.786669]
[ 9207.786671] which lock already depends on the new lock.
[ 9207.786674]
[ 9207.786679]
[ 9207.786680] the existing dependency chain (in reverse order) is:
[ 9207.786687]
[ 9207.786688] -> #1 (&socket->skt_mutex){+.+.+.}:
[ 9207.786702] [<ffffffff810796c0>] __lock_acquire+0xb73/0xd2b
[ 9207.786716] [<ffffffff8107a36b>] lock_acquire+0xe1/0x105
[ 9207.786726] [<ffffffff813b5ac5>] __mutex_lock_common+0x59/0x49d
[ 9207.786741] [<ffffffff813b5fbe>] mutex_lock_nested+0x39/0x3e
[ 9207.786752] [<ffffffff812ef3ed>] pccard_store_resource+0x6b/0xc5
[ 9207.786763] [<ffffffff812a55da>] dev_attr_store+0x20/0x22
[ 9207.786775] [<ffffffff8116259a>] sysfs_write_file+0x108/0x144
[ 9207.786787] [<ffffffff8110cf48>] vfs_write+0xae/0x10b
[ 9207.786798] [<ffffffff8110d065>] sys_write+0x4a/0x6e
[ 9207.786808] [<ffffffff81009bc2>] system_call_fastpath+0x16/0x1b
[ 9207.786822]
[ 9207.786824] -> #0 (s_active){++++.+}:
[ 9207.786835] [<ffffffff8107956a>] __lock_acquire+0xa1d/0xd2b
[ 9207.786847] [<ffffffff8107a36b>] lock_acquire+0xe1/0x105
[ 9207.786857] [<ffffffff81163230>] sysfs_deactivate+0x8b/0xe0
[ 9207.786868] [<ffffffff811637f1>] sysfs_addrm_finish+0x36/0x55
[ 9207.786879] [<ffffffff81161c26>] sysfs_hash_and_remove+0x53/0x6a
[ 9207.786890] [<ffffffff811629fd>] sysfs_remove_file+0x15/0x17
[ 9207.786900] [<ffffffff812a650e>] device_remove_file+0x17/0x19
[ 9207.786911] [<ffffffff81213ef7>] pci_remove_sysfs_dev_files+0x6b/0x10c
[ 9207.786924] [<ffffffff8120dce1>] pci_stop_bus_device+0x55/0x83
[ 9207.786936] [<ffffffff8120dd99>] pci_remove_bus_device+0x1a/0xba
[ 9207.786947] [<ffffffff8120de5f>] pci_remove_behind_bridge+0x26/0x3f
[ 9207.786958] [<ffffffff812efb96>] cb_free+0x4a/0x4f
[ 9207.786969] [<ffffffff812ec682>] socket_shutdown+0x91/0xfd
[ 9207.786979] [<ffffffff812ec88a>] socket_remove+0x4e/0x57
[ 9207.786989] [<ffffffff812ed08b>] pccardd+0x19c/0x25f
[ 9207.787000] [<ffffffff810674dd>] kthread+0x7f/0x87
[ 9207.787011] [<ffffffff8100aa64>] kernel_thread_helper+0x4/0x10
[ 9207.787023]
[ 9207.787025] other info that might help us debug this:
[ 9207.787027]
[ 9207.787034] 1 lock held by pccardd/841:
[ 9207.787039] #0: (&socket->skt_mutex){+.+.+.}, at: [<ffffffff812ed04c>] pccardd+0x15d/0x25f
[ 9207.787058]
[ 9207.787060] stack backtrace:
[ 9207.787068] Pid: 841, comm: pccardd Not tainted 2.6.33-rc4-wl #8
[ 9207.787069] Call Trace:
[ 9207.787069] [<ffffffff8107871d>] print_circular_bug+0xa8/0xb6
[ 9207.787069] [<ffffffff8107956a>] __lock_acquire+0xa1d/0xd2b
[ 9207.787069] [<ffffffff811637f1>] ? sysfs_addrm_finish+0x36/0x55
[ 9207.787069] [<ffffffff8107a36b>] lock_acquire+0xe1/0x105
[ 9207.787069] [<ffffffff811637f1>] ? sysfs_addrm_finish+0x36/0x55
[ 9207.787069] [<ffffffff81163230>] sysfs_deactivate+0x8b/0xe0
[ 9207.787069] [<ffffffff811637f1>] ? sysfs_addrm_finish+0x36/0x55
[ 9207.787069] [<ffffffff8107760e>] ? trace_hardirqs_off+0xd/0xf
[ 9207.787069] [<ffffffff813b5897>] ? __mutex_unlock_slowpath+0x119/0x14e
[ 9207.787069] [<ffffffff811637f1>] sysfs_addrm_finish+0x36/0x55
[ 9207.787069] [<ffffffff81161c26>] sysfs_hash_and_remove+0x53/0x6a
[ 9207.787069] [<ffffffff811629fd>] sysfs_remove_file+0x15/0x17
[ 9207.787069] [<ffffffff812a650e>] device_remove_file+0x17/0x19
[ 9207.787069] [<ffffffff81213ef7>] pci_remove_sysfs_dev_files+0x6b/0x10c
[ 9207.787069] [<ffffffff8120dce1>] pci_stop_bus_device+0x55/0x83
[ 9207.787069] [<ffffffff8120dd99>] pci_remove_bus_device+0x1a/0xba
[ 9207.787069] [<ffffffff8120de5f>] pci_remove_behind_bridge+0x26/0x3f
[ 9207.787069] [<ffffffff812efb96>] cb_free+0x4a/0x4f
[ 9207.787069] [<ffffffff812ec682>] socket_shutdown+0x91/0xfd
[ 9207.787069] [<ffffffff812ec88a>] socket_remove+0x4e/0x57
[ 9207.787069] [<ffffffff812ed08b>] pccardd+0x19c/0x25f
[ 9207.787069] [<ffffffff812eceef>] ? pccardd+0x0/0x25f
[ 9207.787069] [<ffffffff810674dd>] kthread+0x7f/0x87
[ 9207.787069] [<ffffffff8100aa64>] kernel_thread_helper+0x4/0x10
[ 9207.787069] [<ffffffff813b8014>] ? restore_args+0x0/0x30
[ 9207.787069] [<ffffffff8106745e>] ? kthread+0x0/0x87
[ 9207.787069] [<ffffffff8100aa60>] ? kernel_thread_helper+0x0/0x10
[ 9207.788150] device: '0000:16:00.0': device_unregister
[ 9207.788166] PM: Removing info for pci:0000:16:00.0
[ 9207.788405] bus: 'pci': remove device 0000:16:00.0
next prev parent reply other threads:[~2010-01-17 16:27 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-12-24 22:00 Linux 2.6.33-rc2 - Merry Christmas Linus Torvalds
2009-12-25 10:27 ` -tip: origin tree boot crash Ingo Molnar
2009-12-25 19:49 ` Dmitry Torokhov
2009-12-26 20:19 ` Len Brown
2009-12-26 20:17 ` Len Brown
2009-12-27 4:20 ` Len Brown
2009-12-28 9:44 ` Ingo Molnar
2009-12-28 12:01 ` Ingo Molnar
2009-12-28 15:02 ` Paul Rolland
2009-12-28 16:15 ` Paul Rolland
2009-12-28 16:53 ` Paul Rolland
2009-12-28 20:17 ` Dmitry Torokhov
2009-12-30 6:14 ` Len Brown
2009-12-30 7:13 ` Paul Rolland
2009-12-30 6:19 ` [PATCH] wmi: check find_guid() return value to prevent oops Len Brown
2009-12-30 6:21 ` [PATCH] dell-wmi: sys_init_module: 'dell_wmi'->init suspiciously returned 21, it should follow 0/-E convention Len Brown
2009-12-25 13:10 ` Linux 2.6.33-rc2 - Blank screen for Intel KMS Miguel Calleja
2009-12-29 9:50 ` Miguel Calleja
2009-12-29 14:01 ` Rafael J. Wysocki
2009-12-25 20:00 ` Linux 2.6.33-rc2 - Merry Christmas Borislav Petkov
2009-12-25 21:50 ` Borislav Petkov
2009-12-26 6:00 ` Jesse Barnes
2009-12-26 8:02 ` Borislav Petkov
2009-12-26 9:36 ` EHCI resume sysfs duplicates (was: Re: Linux 2.6.33-rc2 - Merry Christmas ...) Borislav Petkov
2009-12-26 9:45 ` drm_vm.c:drm_mmap: possible circular locking dependency detected " Borislav Petkov
2009-12-28 0:40 ` KOSAKI Motohiro
2009-12-30 21:10 ` Linus Torvalds
2009-12-30 21:34 ` Eric W. Biederman
2009-12-30 22:03 ` Linus Torvalds
2009-12-31 8:40 ` Eric W. Biederman
2009-12-31 19:04 ` Linus Torvalds
2010-01-01 13:58 ` [PATCH] sysfs: Cache the last sysfs_dirent to improve readdir scalability Eric W. Biederman
2010-01-01 15:33 ` Borislav Petkov
2010-01-01 18:56 ` Linus Torvalds
2010-01-01 22:43 ` [PATCH] sysfs: Cache the last sysfs_dirent to improve readdir scalability v2 Eric W. Biederman
2010-01-01 23:10 ` Linus Torvalds
2010-01-02 5:59 ` Greg KH
2010-01-02 15:40 ` Borislav Petkov
2010-01-01 15:16 ` drm_vm.c:drm_mmap: possible circular locking dependency detected (was: Re: Linux 2.6.33-rc2 - Merry Christmas ...) Eric W. Biederman
2010-01-02 2:59 ` drm_vm.c:drm_mmap: possible circular locking dependency detected Tejun Heo
2010-01-02 21:37 ` [PATCH] sysfs: Add lockdep annotations for the sysfs active reference Eric W. Biederman
2010-01-03 0:02 ` Tejun Heo
2010-01-17 16:26 ` Ming Lei [this message]
2010-01-17 17:18 ` Eric W. Biederman
2010-01-17 18:03 ` Dominik Brodowski
2010-01-02 21:49 ` drm_vm.c:drm_mmap: possible circular locking dependency detected Eric W. Biederman
2010-01-03 0:32 ` Tejun Heo
2010-01-03 2:06 ` Eric W. Biederman
2010-01-03 5:01 ` Tejun Heo
2010-01-03 5:38 ` Eric W. Biederman
2010-01-03 6:05 ` Tejun Heo
2010-01-03 7:47 ` Dmitry Torokhov
2010-01-03 10:57 ` Eric W. Biederman
2010-01-03 11:14 ` Eric W. Biederman
2010-01-04 19:16 ` Dmitry Torokhov
2010-01-04 18:57 ` Dmitry Torokhov
2010-01-04 19:43 ` Eric W. Biederman
2010-01-04 21:12 ` Dmitry Torokhov
2010-01-04 23:09 ` Tejun Heo
2009-12-31 8:40 ` drm_vm.c:drm_mmap: possible circular locking dependency detected (was: Re: Linux 2.6.33-rc2 - Merry Christmas ...) Eric W. Biederman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20100118002647.35035e77@tom-lei \
--to=tom.leiming@gmail.com \
--cc=airlied@linux.ie \
--cc=ebiederm@xmission.com \
--cc=greg@kroah.com \
--cc=kosaki.motohiro@jp.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pcmcia@lists.infradead.org \
--cc=linux@dominikbrodowski.net \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=petkovbb@googlemail.com \
--cc=tj@kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=viro@ZenIV.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox