Re: mhi resume failure on reboot with 6.13-rc2

[Johan Hovold <johan@kernel.org>]

On Wed, Dec 18, 2024 at 07:39:10PM +0530, Manivannan Sadhasivam wrote:
> On Wed, Dec 18, 2024 at 02:55:02PM +0100, Johan Hovold wrote:
> > On Wed, Dec 18, 2024 at 06:00:19PM +0530, Manivannan Sadhasivam wrote:
> > > On Wed, Dec 18, 2024 at 01:02:39PM +0100, Johan Hovold wrote:
> > > > On Wed, Dec 18, 2024 at 05:08:30PM +0530, Manivannan Sadhasivam wrote:
> > > > > On Wed, Dec 18, 2024 at 09:40:45AM +0100, Johan Hovold wrote:
> > > > 
> > > > > > I've tracked down the hang to a deadlock on the parent device lock.
> > > > > > 
> > > > > > Driver core takes the parent device lock before calling shutdown(), and
> > > > > > then mhi_pci_shutdown() waits indefinitely for the recovery thread to
> > > > > > finish.
> > > > 
> > > > > Thanks for tracking the deadlock. I think we should use pci_try_reset_function()
> > > > > instead of pci_reset_function() in mhi_pci_recovery_work().
> > > > > 
> > > > > If the pci_dev_lock() is already taken, it will return with -EAGAIN and we do
> > > > > not need to worry in that case since the host is going to be powered off anyway
> > > > > (and so the device).
> > > > 
> > > > That may work. But note that I've now also seen this deadlock during
> > > > suspend (i.e. when the device is not going away). The
> > > > pci_try_reset_function() should avoid the deadlock here too, but we'll
> > > > end up in funny state.
> > > 
> > > Hopefully, recovery_work() started by mhi_pci_runtime_resume() would be able to
> > > reset the device.
> > 
> > But that's not going to happen as that reset is what is currently
> > causing the deadlock and which would simply be skipped if you switch to
> > pci_try_reset_function().
> > 
> 
> mhi_pci_runtime_resume() will queue the recovery_work() and return. So I was
> hoping that by the time pci_try_reset_function() is called, the lock would be
> available.

We can't rely on luck with timings, and this is the very reason for the
deadlock I'm currently seeing (i.e. the recovery thread is still running
when another thread grabs the lock and waits for the recovery thread to
finish).

Perhaps the recovery work should be done synchronously in the resume
handler to avoid such issues.

Johan