All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v3 00/11] scsi: mpt3sas: Fix delayed not-responding block deadlock and related bugs
@ 2026-11-18 23:09 Jeff Layton
  2026-11-18 23:09 ` [PATCH v3 01/11] scsi: mpt3sas: Don't leave requeued firmware events pending at teardown Jeff Layton
                   ` (11 more replies)
  0 siblings, 12 replies; 17+ messages in thread
From: Jeff Layton @ 2026-11-18 23:09 UTC (permalink / raw)
  To: Sathya Prakash, Sreekanth Reddy, Suganath Prabu Subramani,
	Ranjan Kumar, James E.J. Bottomley, Martin K. Petersen,
	Nagalakshmi Nandigama, James Bottomley
  Cc: Keith Busch, Sreekanth Reddy, MPT-FusionLinux.pdl, linux-scsi,
	linux-kernel, Jeff Layton

I posted v2 some time ago. Since then, I did a lot of adversarial review
with AI agents. The review found a large number of bugs. Many of these
bugs were preexisting.

This series fixes a deadlock that occurs regularly in Meta's fleet. The
fix unblocks sdevs after a timeout, and this breaks the deadlock.

I don't do much SCSI driver work, but this problem came to me. I
relied heavily on AI agents for this series.

If this series is acceptable, I would like to see it in linux-next
soon.

Thanks,
Jeff

Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
Changes in v3:
- New patches 1-8 and 10 fix pre-existing bugs in the code that this
  series changes:
  - 1-2: fix UAFs and refcounting in the cleanup of the fw event queue.
    Do not leave requeued events pending at teardown.
  - 3: requeue topology events only for busy devices. Do not requeue
    them for devices that the driver cannot add.
  - 4-7: fix these TUR polling bugs:
    - a pcie_device leak
    - a lost unblock retry
    - polling that continues after the scan is cancelled
    - a retry count that is not per device
  - 8: cache the missing delays that the IOC actually uses
  - 10: do not use freed hostdata while the driver polls devices
- Patch 9: also publish sdev->hostdata under ioc->block_lock in
  scsih_sdev_init()
- Patch 11:
  - Use the PCIe missing delay (from PCIe IO Unit Page 1) for NVMe
    devices.
  - Arm dnr_unblock_work for the earliest deadline.
  - Skip the timed unblock during host reset, but not during SCSI EH.
  - Clear the deadline of an sdev under ioc->block_lock before the fw
    event worker polls the sdev.
  - Do not put a timed block on offline sdevs.
  - When a hard reset cancels polling, give new deadlines to the sdevs
    that stay blocked.
  - Give deadlines to the sdevs that a broadcast primitive pass blocks.
    Prevent dnr_unblock_work from unblocking them while the pass runs.
  - A repeat DNR restarts the deadline, at most
    MPT3SAS_DNR_BLOCK_MAX_EXTENDS times.
  - At the next worker unblock, do a TUR check on expired sdevs.
  - Keep the blocks that the driver makes during polling.
  - Use system_dfl_wq instead of system_wq.
- Link to v2: https://lore.kernel.org/r/20261102-sas-abba-v2-0-b3b9ba874b5a@kernel.org

Changes in v2:
- New patch 1: use a new ioc->block_lock to serialize the
  interrupt-time sdev block with the hostdata teardown in
  scsih_sdev_destroy(). This fixes a pre-existing UAF.
- Do the timed unblock under ioc->block_lock. This prevents a race with
  an interrupt-time block.
- Clear block_deadline before the fw event worker polls a returning
  device with TURs. Then dnr_unblock_work cannot unblock the device
  during the poll.
- Link to v1: https://lore.kernel.org/r/20261030-sas-abba-v1-1-b8cdd714a37c@kernel.org

---
Jeff Layton (11):
      scsi: mpt3sas: Don't leave requeued firmware events pending at teardown
      scsi: mpt3sas: Hold event references while cleaning up the event queue
      scsi: mpt3sas: Requeue topology events only for busy devices
      scsi: mpt3sas: Fix pcie_device leak in _scsih_ublock_io_device_wait()
      scsi: mpt3sas: Restore unblock retry in _scsih_ublock_io_device_wait()
      scsi: mpt3sas: Stop polling devices when the unresponding scan is cancelled
      scsi: mpt3sas: Reset the TUR retry count for each device
      scsi: mpt3sas: Cache the missing delays the IOC actually uses
      scsi: mpt3sas: Serialize interrupt-time sdev block with hostdata teardown
      scsi: mpt3sas: Don't use freed hostdata while polling devices
      scsi: mpt3sas: Time out delayed not-responding device blocks

 drivers/scsi/mpt3sas/mpt3sas_base.c  |  62 ++-
 drivers/scsi/mpt3sas/mpt3sas_base.h  |  25 +
 drivers/scsi/mpt3sas/mpt3sas_scsih.c | 869 +++++++++++++++++++++++++++++------
 3 files changed, 810 insertions(+), 146 deletions(-)
---
base-commit: e8017d29067fd1310a65c3494ce1a1b492adbac6
change-id: 20261030-sas-abba-eacfcc45fe36

Best regards,
-- 
Jeff Layton <jlayton@kernel.org>


^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2026-11-18 23:33 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18 23:09 [PATCH v3 00/11] scsi: mpt3sas: Fix delayed not-responding block deadlock and related bugs Jeff Layton
2026-11-18 23:09 ` [PATCH v3 01/11] scsi: mpt3sas: Don't leave requeued firmware events pending at teardown Jeff Layton
2026-11-18 23:33   ` sashiko-bot
2026-11-18 23:09 ` [PATCH v3 02/11] scsi: mpt3sas: Hold event references while cleaning up the event queue Jeff Layton
2026-11-18 23:10 ` [PATCH v3 03/11] scsi: mpt3sas: Requeue topology events only for busy devices Jeff Layton
2026-11-18 23:10 ` [PATCH v3 04/11] scsi: mpt3sas: Fix pcie_device leak in _scsih_ublock_io_device_wait() Jeff Layton
2026-11-18 23:10 ` [PATCH v3 05/11] scsi: mpt3sas: Restore unblock retry " Jeff Layton
2026-11-18 23:10 ` [PATCH v3 06/11] scsi: mpt3sas: Stop polling devices when the unresponding scan is cancelled Jeff Layton
2026-11-18 23:27   ` sashiko-bot
2026-11-18 23:10 ` [PATCH v3 07/11] scsi: mpt3sas: Reset the TUR retry count for each device Jeff Layton
2026-11-18 23:10 ` [PATCH v3 08/11] scsi: mpt3sas: Cache the missing delays the IOC actually uses Jeff Layton
2026-11-18 23:10 ` [PATCH v3 09/11] scsi: mpt3sas: Serialize interrupt-time sdev block with hostdata teardown Jeff Layton
2026-11-18 23:10 ` [PATCH v3 10/11] scsi: mpt3sas: Don't use freed hostdata while polling devices Jeff Layton
2026-11-18 23:23   ` sashiko-bot
2026-11-18 23:10 ` [PATCH v3 11/11] scsi: mpt3sas: Time out delayed not-responding device blocks Jeff Layton
2026-11-18 23:33   ` sashiko-bot
2026-11-18 23:13 ` [PATCH v3 00/11] scsi: mpt3sas: Fix delayed not-responding block deadlock and related bugs Jeff Layton

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.