From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org, Douglas Anderson <dianders@chromium.org>,
Alan Stern <stern@rowland.harvard.edu>
Subject: [PATCH 4.13 08/47] USB: core: Avoid race of async_completed() w/ usbdev_release()
Date: Fri, 8 Sep 2017 15:18:40 +0200 [thread overview]
Message-ID: <20170908131823.915624944@linuxfoundation.org> (raw)
In-Reply-To: <20170908131823.546721606@linuxfoundation.org>
4.13-stable review patch. If anyone has any objections, please let me know.
------------------
From: Douglas Anderson <dianders@chromium.org>
commit ed62ca2f4f51c17841ea39d98c0c409cb53a3e10 upstream.
While running reboot tests w/ a specific set of USB devices (and
slub_debug enabled), I found that once every few hours my device would
be crashed with a stack that looked like this:
[ 14.012445] BUG: spinlock bad magic on CPU#0, modprobe/2091
[ 14.012460] lock: 0xffffffc0cb055978, .magic: ffffffc0, .owner: cryption contexts: %lu/%lu
[ 14.012460] /1025536097, .owner_cpu: 0
[ 14.012466] CPU: 0 PID: 2091 Comm: modprobe Not tainted 4.4.79 #352
[ 14.012468] Hardware name: Google Kevin (DT)
[ 14.012471] Call trace:
[ 14.012483] [<....>] dump_backtrace+0x0/0x160
[ 14.012487] [<....>] show_stack+0x20/0x28
[ 14.012494] [<....>] dump_stack+0xb4/0xf0
[ 14.012500] [<....>] spin_dump+0x8c/0x98
[ 14.012504] [<....>] spin_bug+0x30/0x3c
[ 14.012508] [<....>] do_raw_spin_lock+0x40/0x164
[ 14.012515] [<....>] _raw_spin_lock_irqsave+0x64/0x74
[ 14.012521] [<....>] __wake_up+0x2c/0x60
[ 14.012528] [<....>] async_completed+0x2d0/0x300
[ 14.012534] [<....>] __usb_hcd_giveback_urb+0xc4/0x138
[ 14.012538] [<....>] usb_hcd_giveback_urb+0x54/0xf0
[ 14.012544] [<....>] xhci_irq+0x1314/0x1348
[ 14.012548] [<....>] usb_hcd_irq+0x40/0x50
[ 14.012553] [<....>] handle_irq_event_percpu+0x1b4/0x3f0
[ 14.012556] [<....>] handle_irq_event+0x4c/0x7c
[ 14.012561] [<....>] handle_fasteoi_irq+0x158/0x1c8
[ 14.012564] [<....>] generic_handle_irq+0x30/0x44
[ 14.012568] [<....>] __handle_domain_irq+0x90/0xbc
[ 14.012572] [<....>] gic_handle_irq+0xcc/0x18c
Investigation using kgdb() found that the wait queue that was passed
into wake_up() had been freed (it was filled with slub_debug poison).
I analyzed and instrumented the code and reproduced. My current
belief is that this is happening:
1. async_completed() is called (from IRQ). Moves "as" onto the
completed list.
2. On another CPU, proc_reapurbnonblock_compat() calls
async_getcompleted(). Blocks on spinlock.
3. async_completed() releases the lock; keeps running; gets blocked
midway through wake_up().
4. proc_reapurbnonblock_compat() => async_getcompleted() gets the
lock; removes "as" from completed list and frees it.
5. usbdev_release() is called. Frees "ps".
6. async_completed() finally continues running wake_up(). ...but
wake_up() has a pointer to the freed "ps".
The instrumentation that led me to believe this was based on adding
some trace_printk() calls in a select few functions and then using
kdb's "ftdump" at crash time. The trace follows (NOTE: in the trace
below I cheated a little bit and added a udelay(1000) in
async_completed() after releasing the spinlock because I wanted it to
trigger quicker):
<...>-2104 0d.h2 13759034us!: async_completed at start: as=ffffffc0cc638200
mtpd-2055 3.... 13759356us : async_getcompleted before spin_lock_irqsave
mtpd-2055 3d..1 13759362us : async_getcompleted after list_del_init: as=ffffffc0cc638200
mtpd-2055 3.... 13759371us+: proc_reapurbnonblock_compat: free_async(ffffffc0cc638200)
mtpd-2055 3.... 13759422us+: async_getcompleted before spin_lock_irqsave
mtpd-2055 3.... 13759479us : usbdev_release at start: ps=ffffffc0cc042080
mtpd-2055 3.... 13759487us : async_getcompleted before spin_lock_irqsave
mtpd-2055 3.... 13759497us!: usbdev_release after kfree(ps): ps=ffffffc0cc042080
<...>-2104 0d.h2 13760294us : async_completed before wake_up(): as=ffffffc0cc638200
To fix this problem we can just move the wake_up() under the ps->lock.
There should be no issues there that I'm aware of.
Signed-off-by: Douglas Anderson <dianders@chromium.org>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/usb/core/devio.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--- a/drivers/usb/core/devio.c
+++ b/drivers/usb/core/devio.c
@@ -623,6 +623,8 @@ static void async_completed(struct urb *
if (as->status < 0 && as->bulk_addr && as->status != -ECONNRESET &&
as->status != -ENOENT)
cancel_bulk_urbs(ps, as->bulk_addr);
+
+ wake_up(&ps->wait);
spin_unlock(&ps->lock);
if (signr) {
@@ -630,8 +632,6 @@ static void async_completed(struct urb *
put_pid(pid);
put_cred(cred);
}
-
- wake_up(&ps->wait);
}
static void destroy_async(struct usb_dev_state *ps, struct list_head *list)
next prev parent reply other threads:[~2017-09-08 13:50 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-09-08 13:18 [PATCH 4.13 00/47] 4.13.1-stable review Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 01/47] usb: quirks: add delay init quirk for Corsair Strafe RGB keyboard Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 02/47] USB: serial: option: add support for D-Link DWM-157 C1 Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 03/47] usb: Add device quirk for Logitech HD Pro Webcam C920-C Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 04/47] usb:xhci:Fix regression when ATI chipsets detected Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 05/47] USB: musb: fix external abort on suspend Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 06/47] ANDROID: binder: add padding to binder_fd_array_object Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 07/47] ANDROID: binder: add hwbinder,vndbinder to BINDER_DEVICES Greg Kroah-Hartman
2017-09-08 13:18 ` Greg Kroah-Hartman [this message]
2017-09-08 13:18 ` [PATCH 4.13 09/47] staging/rts5208: fix incorrect shift to extract upper nybble Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 10/47] staging: ccree: save ciphertext for CTS IV Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 12/47] iio: adc: ti-ads1015: fix incorrect data rate setting update Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 13/47] iio: adc: ti-ads1015: fix scale information for ADS1115 Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 14/47] iio: adc: ti-ads1015: enable conversion when CONFIG_PM is not set Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 15/47] iio: adc: ti-ads1015: avoid getting stale result after runtime resume Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 16/47] iio: adc: ti-ads1015: dont return invalid value from buffer setup callbacks Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 17/47] iio: adc: ti-ads1015: add adequate wait time to get correct conversion Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 18/47] driver core: bus: Fix a potential double free Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 19/47] HID: wacom: Do not completely map WACOM_HID_WD_TOUCHRINGSTATUS usage Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 20/47] binder: free memory on error Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 23/47] thunderbolt: Fix reset response_type Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 24/47] fpga: altera-hps2fpga: fix multiple init of l3_remap_lock Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 25/47] intel_th: pci: Add Cannon Lake PCH-H support Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 26/47] intel_th: pci: Add Cannon Lake PCH-LP support Greg Kroah-Hartman
2017-09-08 13:18 ` [PATCH 4.13 27/47] ath10k: fix memory leak in rx ring buffer allocation Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 28/47] drm/vgem: Pin our pages for dmabuf exports Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 31/47] rtlwifi: rtl_pci_probe: Fix fail path of _rtl_pci_find_adapter Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 32/47] Bluetooth: Add support of 13d3:3494 RTL8723BE device Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 33/47] iwlwifi: pci: add new PCI ID for 7265D Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 35/47] mwifiex: correct channel stat buffer overflows Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 36/47] MCB: add support for SC31 to mcb-lpc Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 37/47] s390/mm: avoid empty zero pages for KVM guests to avoid postcopy hangs Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 38/47] drm/nouveau/pci/msi: disable MSI on big-endian platforms by default Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 39/47] drm/nouveau: Fix error handling in nv50_disp_atomic_commit Greg Kroah-Hartman
2017-09-08 13:19 ` Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 40/47] workqueue: Fix flag collision Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 41/47] ahci: dont use MSI for devices with the silly Intel NVMe remapping scheme Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 42/47] cs5536: add support for IDE controller variant Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 43/47] scsi: sg: protect against races between mmap() and SG_SET_RESERVED_SIZE Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 44/47] scsi: sg: recheck MMAP_IO request length with lock held Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 45/47] of/device: Prevent buffer overflow in of_device_modalias() Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 46/47] rtlwifi: Fix memory leak when firmware request fails Greg Kroah-Hartman
2017-09-08 13:19 ` [PATCH 4.13 47/47] rtlwifi: Fix fallback firmware loading Greg Kroah-Hartman
2017-09-08 16:58 ` [PATCH 4.13 00/47] 4.13.1-stable review Shuah Khan
2017-09-09 4:50 ` Greg Kroah-Hartman
2017-09-09 13:49 ` Guenter Roeck
2017-09-09 14:18 ` Greg Kroah-Hartman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170908131823.915624944@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=dianders@chromium.org \
--cc=linux-kernel@vger.kernel.org \
--cc=stable@vger.kernel.org \
--cc=stern@rowland.harvard.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.