uvcvideo: deadlock when async control drops the last PM reference from the status-event path

uvcvideo: deadlock when async control drops the last PM reference from the status-event path

[Jay Chooi]

Hello,

We are hitting a reproducible, unkillable deadlock in uvcvideo's
status-event / power-management path on v7.0.6 (Ubuntu 7.0.0-27-generic);
by code inspection the same paths are present in current mainline.

Hardware: 2x Intel RealSense D405 (8086:0b5b) and 1x D435 (8086:0b07) on
the same host. The trigger is ordinary control access (exposure /
auto-exposure, which these devices expose as asynchronous, auto-update
controls). With camera firmware 5.15.1.55 one unit wedged on most control
accesses and the other rarely; after updating both to firmware 5.17.3.10
the deadlock became deterministic on both — the newer firmware appears to
deliver the control-change status interrupt more reliably, hitting the
race window every time.

Analysis
--------

uvc_ctrl_set_handle() takes a PM reference (uvc_pm_get) for a pending
asynchronous control, released by uvc_ctrl_clear_handle() -> uvc_pm_put()
when the control-change status event arrives. That release runs inside
uvc_ctrl_status_event(), which is reached from two contexts where dropping
the LAST status user deadlocks:

1. From the async worker uvc_ctrl_status_event_work():
   uvc_pm_put -> uvc_status_put (takes status_lock) -> uvc_status_stop ->
   cancel_work_sync(&dev->async_ctrl.work) — cancelling the work item it
   is currently executing in. It waits for itself forever while holding
   status_lock.

2. From uvc_status_stop() itself, via the synchronous flush
   ("if (cancel_work_sync(&w->work)) uvc_ctrl_status_event(...)"):
   the same put chain re-acquires status_lock, which the caller
   (uvc_status_put/uvc_status_suspend) already holds. Self-deadlock on a
   non-recursive mutex.

Once either happens, every subsequent open/ioctl blocks in
uvc_status_get() on status_lock (unkillable D state), and
uvc_status_unregister() -> uvc_status_suspend() blocks the USB disconnect
path, which in turn wedges the hub workqueue (usb_hub_wq in D state) —
after that no device on the hub can enumerate and only a reboot recovers.
System shutdown also hangs indefinitely waiting on the D-state tasks.

Hung-task stacks (Ubuntu 7.0.0-27-generic)
------------------------------------------

Blocked ioctl side:
  uvc_v4l2_unlocked_ioctl+0x97/0x190 [uvcvideo]
  uvc_status_get+0x1d/0x90 [uvcvideo]

Blocked worker side:
  Workqueue: events uvc_ctrl_status_event_work [uvcvideo]
  uvc_ctrl_status_event_work+0x22/0x90 [uvcvideo]
  uvc_ctrl_status_event+0x77/0x180 [uvcvideo]
  uvc_pm_put+0x12/0x40 [uvcvideo]
  uvc_status_put+0x6d/0x80 [uvcvideo]
  uvc_status_stop.part.0+0x22/0xb0 [uvcvideo]

(Full traces available on request.)

Workaround we validated
-----------------------

A local module parameter that skips arming the status interrupt endpoint
in uvc_status_init() (no events -> no worker -> no deadlock) eliminates
the hang across repeated control-access stress on both firmware versions.
Expected side effect confirmed: pending async controls are then never
cleared (GET on them returns -EBUSY indefinitely), so this is a
workaround, not a fix.

Downstream report:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2160752

It looks like dropping the pending-async-control PM reference must not
synchronously stop the status URB from the event-delivery contexts —
e.g. defer the final uvc_status_stop() when current_work() ==
&dev->async_ctrl.work, and avoid re-entering uvc_status_put under
status_lock in the synchronous-flush path. Happy to test patches; the
setup reproduces the deadlock within seconds.

Reported-by: Jay Chooi <jay@robocurve.org>

Re: uvcvideo: deadlock when async control drops the last PM reference from the status-event path

[Ricardo Ribalda]

Hi Jay,

Thanks for the detailed report.

Can you check if this patch is part of your kernel?
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/media/usb/uvc?id=6d27f92c54ce28cfbd2a8a479a96d6f4a781b7d2

It landed in 7.1 but I cannot see it in
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/log/drivers/media/usb/uvc?h=linux-7.0.y

Regards!

On Thu, 16 Jul 2026 at 09:17, Jay Chooi <jay@robocurve.org> wrote:
>
> Hello,
>
> We are hitting a reproducible, unkillable deadlock in uvcvideo's
> status-event / power-management path on v7.0.6 (Ubuntu 7.0.0-27-generic);
> by code inspection the same paths are present in current mainline.
>
> Hardware: 2x Intel RealSense D405 (8086:0b5b) and 1x D435 (8086:0b07) on
> the same host. The trigger is ordinary control access (exposure /
> auto-exposure, which these devices expose as asynchronous, auto-update
> controls). With camera firmware 5.15.1.55 one unit wedged on most control
> accesses and the other rarely; after updating both to firmware 5.17.3.10
> the deadlock became deterministic on both — the newer firmware appears to
> deliver the control-change status interrupt more reliably, hitting the
> race window every time.
>
> Analysis
> --------
>
> uvc_ctrl_set_handle() takes a PM reference (uvc_pm_get) for a pending
> asynchronous control, released by uvc_ctrl_clear_handle() -> uvc_pm_put()
> when the control-change status event arrives. That release runs inside
> uvc_ctrl_status_event(), which is reached from two contexts where dropping
> the LAST status user deadlocks:
>
> 1. From the async worker uvc_ctrl_status_event_work():
>    uvc_pm_put -> uvc_status_put (takes status_lock) -> uvc_status_stop ->
>    cancel_work_sync(&dev->async_ctrl.work) — cancelling the work item it
>    is currently executing in. It waits for itself forever while holding
>    status_lock.
>
> 2. From uvc_status_stop() itself, via the synchronous flush
>    ("if (cancel_work_sync(&w->work)) uvc_ctrl_status_event(...)"):
>    the same put chain re-acquires status_lock, which the caller
>    (uvc_status_put/uvc_status_suspend) already holds. Self-deadlock on a
>    non-recursive mutex.
>
> Once either happens, every subsequent open/ioctl blocks in
> uvc_status_get() on status_lock (unkillable D state), and
> uvc_status_unregister() -> uvc_status_suspend() blocks the USB disconnect
> path, which in turn wedges the hub workqueue (usb_hub_wq in D state) —
> after that no device on the hub can enumerate and only a reboot recovers.
> System shutdown also hangs indefinitely waiting on the D-state tasks.
>
> Hung-task stacks (Ubuntu 7.0.0-27-generic)
> ------------------------------------------
>
> Blocked ioctl side:
>   uvc_v4l2_unlocked_ioctl+0x97/0x190 [uvcvideo]
>   uvc_status_get+0x1d/0x90 [uvcvideo]
>
> Blocked worker side:
>   Workqueue: events uvc_ctrl_status_event_work [uvcvideo]
>   uvc_ctrl_status_event_work+0x22/0x90 [uvcvideo]
>   uvc_ctrl_status_event+0x77/0x180 [uvcvideo]
>   uvc_pm_put+0x12/0x40 [uvcvideo]
>   uvc_status_put+0x6d/0x80 [uvcvideo]
>   uvc_status_stop.part.0+0x22/0xb0 [uvcvideo]
>
> (Full traces available on request.)
>
> Workaround we validated
> -----------------------
>
> A local module parameter that skips arming the status interrupt endpoint
> in uvc_status_init() (no events -> no worker -> no deadlock) eliminates
> the hang across repeated control-access stress on both firmware versions.
> Expected side effect confirmed: pending async controls are then never
> cleared (GET on them returns -EBUSY indefinitely), so this is a
> workaround, not a fix.
>
> Downstream report:
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2160752
>
> It looks like dropping the pending-async-control PM reference must not
> synchronously stop the status URB from the event-delivery contexts —
> e.g. defer the final uvc_status_stop() when current_work() ==
> &dev->async_ctrl.work, and avoid re-entering uvc_status_put under
> status_lock in the synchronous-flush path. Happy to test patches; the
> setup reproduces the deadlock within seconds.
>
> Reported-by: Jay Chooi <jay@robocurve.org>



-- 
Ricardo Ribalda

Re: uvcvideo: deadlock when async control drops the last PM reference from the status-event path

[Jay Chooi]

> Can you check if this patch is part of your kernel?

It is not. Ubuntu 7.0.0-27-generic (based on v7.0.6) has no uvcvideo
changes on top; disassembly of the shipped module shows no current_work()
check in uvc_status_stop() and no flush_work() in uvc_status_start().
The deadlock reproduces there on every async control sequence with our
D405s on camera firmware 5.17.3.10.

> It landed in 7.1 but I cannot see it in
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/log/drivers/media/usb/uvc?h=linux-7.0.y

Confirmed missing from linux-7.0.y. I backported 6d27f92c54ce alone onto
the 7.0.0-27 tree as an out-of-tree module and re-ran the same stress on
the same hardware: no hang across 15/15 apply/verify cycles, and async
control readback completes, where the unpatched module hung unkillably on
the first attempt. Please queue it for linux-7.0.y; it carries
Cc: stable and Fixes: a32d9c41bdb8, which is in 7.0. Also requested for
the Ubuntu kernel in
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2160752

Tested-by: Jay Chooi <jay@robocurve.org>

One thing I noticed while reading the code: the synchronous flush in
uvc_status_stop() ("if (cancel_work_sync(&w->work))
uvc_ctrl_status_event(...)") still runs the event handler under
status_lock. If that event clears a pending async control handle,
uvc_pm_put() -> uvc_status_put() re-acquires status_lock on the same
thread. All our hung-task traces were the worker-side path that
6d27f92c54ce fixes, so this may be unreachable in practice, but it looked
worth a second pair of eyes.

Best regards,
Jay

On Thu, Jul 16, 2026 at 12:27 AM Ricardo Ribalda <ribalda@chromium.org> wrote:
>
> Hi Jay,
>
> Thanks for the detailed report.
>
> Can you check if this patch is part of your kernel?
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/media/usb/uvc?id=6d27f92c54ce28cfbd2a8a479a96d6f4a781b7d2
>
> It landed in 7.1 but I cannot see it in
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/log/drivers/media/usb/uvc?h=linux-7.0.y
>
> Regards!
>
> On Thu, 16 Jul 2026 at 09:17, Jay Chooi <jay@robocurve.org> wrote:
> >
> > Hello,
> >
> > We are hitting a reproducible, unkillable deadlock in uvcvideo's
> > status-event / power-management path on v7.0.6 (Ubuntu 7.0.0-27-generic);
> > by code inspection the same paths are present in current mainline.
> >
> > Hardware: 2x Intel RealSense D405 (8086:0b5b) and 1x D435 (8086:0b07) on
> > the same host. The trigger is ordinary control access (exposure /
> > auto-exposure, which these devices expose as asynchronous, auto-update
> > controls). With camera firmware 5.15.1.55 one unit wedged on most control
> > accesses and the other rarely; after updating both to firmware 5.17.3.10
> > the deadlock became deterministic on both — the newer firmware appears to
> > deliver the control-change status interrupt more reliably, hitting the
> > race window every time.
> >
> > Analysis
> > --------
> >
> > uvc_ctrl_set_handle() takes a PM reference (uvc_pm_get) for a pending
> > asynchronous control, released by uvc_ctrl_clear_handle() -> uvc_pm_put()
> > when the control-change status event arrives. That release runs inside
> > uvc_ctrl_status_event(), which is reached from two contexts where dropping
> > the LAST status user deadlocks:
> >
> > 1. From the async worker uvc_ctrl_status_event_work():
> >    uvc_pm_put -> uvc_status_put (takes status_lock) -> uvc_status_stop ->
> >    cancel_work_sync(&dev->async_ctrl.work) — cancelling the work item it
> >    is currently executing in. It waits for itself forever while holding
> >    status_lock.
> >
> > 2. From uvc_status_stop() itself, via the synchronous flush
> >    ("if (cancel_work_sync(&w->work)) uvc_ctrl_status_event(...)"):
> >    the same put chain re-acquires status_lock, which the caller
> >    (uvc_status_put/uvc_status_suspend) already holds. Self-deadlock on a
> >    non-recursive mutex.
> >
> > Once either happens, every subsequent open/ioctl blocks in
> > uvc_status_get() on status_lock (unkillable D state), and
> > uvc_status_unregister() -> uvc_status_suspend() blocks the USB disconnect
> > path, which in turn wedges the hub workqueue (usb_hub_wq in D state) —
> > after that no device on the hub can enumerate and only a reboot recovers.
> > System shutdown also hangs indefinitely waiting on the D-state tasks.
> >
> > Hung-task stacks (Ubuntu 7.0.0-27-generic)
> > ------------------------------------------
> >
> > Blocked ioctl side:
> >   uvc_v4l2_unlocked_ioctl+0x97/0x190 [uvcvideo]
> >   uvc_status_get+0x1d/0x90 [uvcvideo]
> >
> > Blocked worker side:
> >   Workqueue: events uvc_ctrl_status_event_work [uvcvideo]
> >   uvc_ctrl_status_event_work+0x22/0x90 [uvcvideo]
> >   uvc_ctrl_status_event+0x77/0x180 [uvcvideo]
> >   uvc_pm_put+0x12/0x40 [uvcvideo]
> >   uvc_status_put+0x6d/0x80 [uvcvideo]
> >   uvc_status_stop.part.0+0x22/0xb0 [uvcvideo]
> >
> > (Full traces available on request.)
> >
> > Workaround we validated
> > -----------------------
> >
> > A local module parameter that skips arming the status interrupt endpoint
> > in uvc_status_init() (no events -> no worker -> no deadlock) eliminates
> > the hang across repeated control-access stress on both firmware versions.
> > Expected side effect confirmed: pending async controls are then never
> > cleared (GET on them returns -EBUSY indefinitely), so this is a
> > workaround, not a fix.
> >
> > Downstream report:
> > https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2160752
> >
> > It looks like dropping the pending-async-control PM reference must not
> > synchronously stop the status URB from the event-delivery contexts —
> > e.g. defer the final uvc_status_stop() when current_work() ==
> > &dev->async_ctrl.work, and avoid re-entering uvc_status_put under
> > status_lock in the synchronous-flush path. Happy to test patches; the
> > setup reproduces the deadlock within seconds.
> >
> > Reported-by: Jay Chooi <jay@robocurve.org>
>
>
>
> --
> Ricardo Ribalda

Re: uvcvideo: deadlock when async control drops the last PM reference from the status-event path

[Ricardo Ribalda]

Dear stable

Could you help landing  6d27f92c54ce28cfbd2a8a479a96d6f4a781b7d2 in
the stable branches?

Thanks!

On Fri, 17 Jul 2026 at 03:59, Jay Chooi <jay@robocurve.org> wrote:
>
> > Can you check if this patch is part of your kernel?
>
> It is not. Ubuntu 7.0.0-27-generic (based on v7.0.6) has no uvcvideo
> changes on top; disassembly of the shipped module shows no current_work()
> check in uvc_status_stop() and no flush_work() in uvc_status_start().
> The deadlock reproduces there on every async control sequence with our
> D405s on camera firmware 5.17.3.10.
>
> > It landed in 7.1 but I cannot see it in
> > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/log/drivers/media/usb/uvc?h=linux-7.0.y
>
> Confirmed missing from linux-7.0.y. I backported 6d27f92c54ce alone onto
> the 7.0.0-27 tree as an out-of-tree module and re-ran the same stress on
> the same hardware: no hang across 15/15 apply/verify cycles, and async
> control readback completes, where the unpatched module hung unkillably on
> the first attempt. Please queue it for linux-7.0.y; it carries
> Cc: stable and Fixes: a32d9c41bdb8, which is in 7.0. Also requested for
> the Ubuntu kernel in
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2160752
>
> Tested-by: Jay Chooi <jay@robocurve.org>
>
> One thing I noticed while reading the code: the synchronous flush in
> uvc_status_stop() ("if (cancel_work_sync(&w->work))
> uvc_ctrl_status_event(...)") still runs the event handler under
> status_lock. If that event clears a pending async control handle,
> uvc_pm_put() -> uvc_status_put() re-acquires status_lock on the same
> thread. All our hung-task traces were the worker-side path that
> 6d27f92c54ce fixes, so this may be unreachable in practice, but it looked
> worth a second pair of eyes.
>
> Best regards,
> Jay
>
> On Thu, Jul 16, 2026 at 12:27 AM Ricardo Ribalda <ribalda@chromium.org> wrote:
> >
> > Hi Jay,
> >
> > Thanks for the detailed report.
> >
> > Can you check if this patch is part of your kernel?
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/media/usb/uvc?id=6d27f92c54ce28cfbd2a8a479a96d6f4a781b7d2
> >
> > It landed in 7.1 but I cannot see it in
> > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/log/drivers/media/usb/uvc?h=linux-7.0.y
> >
> > Regards!
> >
> > On Thu, 16 Jul 2026 at 09:17, Jay Chooi <jay@robocurve.org> wrote:
> > >
> > > Hello,
> > >
> > > We are hitting a reproducible, unkillable deadlock in uvcvideo's
> > > status-event / power-management path on v7.0.6 (Ubuntu 7.0.0-27-generic);
> > > by code inspection the same paths are present in current mainline.
> > >
> > > Hardware: 2x Intel RealSense D405 (8086:0b5b) and 1x D435 (8086:0b07) on
> > > the same host. The trigger is ordinary control access (exposure /
> > > auto-exposure, which these devices expose as asynchronous, auto-update
> > > controls). With camera firmware 5.15.1.55 one unit wedged on most control
> > > accesses and the other rarely; after updating both to firmware 5.17.3.10
> > > the deadlock became deterministic on both — the newer firmware appears to
> > > deliver the control-change status interrupt more reliably, hitting the
> > > race window every time.
> > >
> > > Analysis
> > > --------
> > >
> > > uvc_ctrl_set_handle() takes a PM reference (uvc_pm_get) for a pending
> > > asynchronous control, released by uvc_ctrl_clear_handle() -> uvc_pm_put()
> > > when the control-change status event arrives. That release runs inside
> > > uvc_ctrl_status_event(), which is reached from two contexts where dropping
> > > the LAST status user deadlocks:
> > >
> > > 1. From the async worker uvc_ctrl_status_event_work():
> > >    uvc_pm_put -> uvc_status_put (takes status_lock) -> uvc_status_stop ->
> > >    cancel_work_sync(&dev->async_ctrl.work) — cancelling the work item it
> > >    is currently executing in. It waits for itself forever while holding
> > >    status_lock.
> > >
> > > 2. From uvc_status_stop() itself, via the synchronous flush
> > >    ("if (cancel_work_sync(&w->work)) uvc_ctrl_status_event(...)"):
> > >    the same put chain re-acquires status_lock, which the caller
> > >    (uvc_status_put/uvc_status_suspend) already holds. Self-deadlock on a
> > >    non-recursive mutex.
> > >
> > > Once either happens, every subsequent open/ioctl blocks in
> > > uvc_status_get() on status_lock (unkillable D state), and
> > > uvc_status_unregister() -> uvc_status_suspend() blocks the USB disconnect
> > > path, which in turn wedges the hub workqueue (usb_hub_wq in D state) —
> > > after that no device on the hub can enumerate and only a reboot recovers.
> > > System shutdown also hangs indefinitely waiting on the D-state tasks.
> > >
> > > Hung-task stacks (Ubuntu 7.0.0-27-generic)
> > > ------------------------------------------
> > >
> > > Blocked ioctl side:
> > >   uvc_v4l2_unlocked_ioctl+0x97/0x190 [uvcvideo]
> > >   uvc_status_get+0x1d/0x90 [uvcvideo]
> > >
> > > Blocked worker side:
> > >   Workqueue: events uvc_ctrl_status_event_work [uvcvideo]
> > >   uvc_ctrl_status_event_work+0x22/0x90 [uvcvideo]
> > >   uvc_ctrl_status_event+0x77/0x180 [uvcvideo]
> > >   uvc_pm_put+0x12/0x40 [uvcvideo]
> > >   uvc_status_put+0x6d/0x80 [uvcvideo]
> > >   uvc_status_stop.part.0+0x22/0xb0 [uvcvideo]
> > >
> > > (Full traces available on request.)
> > >
> > > Workaround we validated
> > > -----------------------
> > >
> > > A local module parameter that skips arming the status interrupt endpoint
> > > in uvc_status_init() (no events -> no worker -> no deadlock) eliminates
> > > the hang across repeated control-access stress on both firmware versions.
> > > Expected side effect confirmed: pending async controls are then never
> > > cleared (GET on them returns -EBUSY indefinitely), so this is a
> > > workaround, not a fix.
> > >
> > > Downstream report:
> > > https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2160752
> > >
> > > It looks like dropping the pending-async-control PM reference must not
> > > synchronously stop the status URB from the event-delivery contexts —
> > > e.g. defer the final uvc_status_stop() when current_work() ==
> > > &dev->async_ctrl.work, and avoid re-entering uvc_status_put under
> > > status_lock in the synchronous-flush path. Happy to test patches; the
> > > setup reproduces the deadlock within seconds.
> > >
> > > Reported-by: Jay Chooi <jay@robocurve.org>
> >
> >
> >
> > --
> > Ricardo Ribalda



-- 
Ricardo Ribalda

Re: uvcvideo: deadlock when async control drops the last PM reference from the status-event path

[Greg KH]

On Fri, Jul 17, 2026 at 08:54:06AM +0200, Ricardo Ribalda wrote:
> Dear stable
> 
> Could you help landing  6d27f92c54ce28cfbd2a8a479a96d6f4a781b7d2 in
> the stable branches?

It's in the queue, still about 500 patches deep to get through.  That's
the "joy" of getting bug fixes into -rc1, and not earlier releases, they
get delayed due to everyone doing the same thing :(

thanks,

greg k-h

Re: uvcvideo: deadlock when async control drops the last PM reference from the status-event path

[Ricardo Ribalda]

Hi Greg

No worries, thanks for the update.

Is there a public page where we can see the status of patches
(queued/rejected) so we do not have to bother you?

Thanks!

On Fri, 17 Jul 2026 at 09:05, Greg KH <greg@kroah.com> wrote:
>
> On Fri, Jul 17, 2026 at 08:54:06AM +0200, Ricardo Ribalda wrote:
> > Dear stable
> >
> > Could you help landing  6d27f92c54ce28cfbd2a8a479a96d6f4a781b7d2 in
> > the stable branches?
>
> It's in the queue, still about 500 patches deep to get through.  That's
> the "joy" of getting bug fixes into -rc1, and not earlier releases, they
> get delayed due to everyone doing the same thing :(
>
> thanks,
>
> greg k-h



-- 
Ricardo Ribalda

Re: uvcvideo: deadlock when async control drops the last PM reference from the status-event path

[Greg KH]

On Fri, Jul 17, 2026 at 09:07:12AM +0200, Ricardo Ribalda wrote:
> Hi Greg
> 
> No worries, thanks for the update.
> 
> Is there a public page where we can see the status of patches
> (queued/rejected) so we do not have to bother you?

Nope, not really, that would take more effort to set up and maintain
that it would be to just keep the mboxes locally as I currently do :)

And normally we do stay on top, this summer it's been a bit harder due
to massive conference travel combined with massive amounts of patches
being tagged for this -rc1 window.

thanks,

greg k-h