From: Tomas Henzl <thenzl@redhat.com>
To: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>,
linux-scsi@vger.kernel.org,
James Bottomley <James.Bottomley@HansenPartnership.com>,
"Nicholas A. Bellinger" <nab@linux-iscsi.org>,
Brian King <brking@linux.vnet.ibm.com>,
Ian Munsie <imunsie@au1.ibm.com>,
Daniel Axtens <dja@ozlabs.au.ibm.com>,
Andrew Donnellan <andrew.donnellan@au1.ibm.com>,
David Laight <David.Laight@ACULAB.COM>
Cc: Michael Neuling <mikey@neuling.org>,
"Manoj N. Kumar" <manoj@linux.vnet.ibm.com>,
linuxppc-dev@lists.ozlabs.org
Subject: Re: [PATCH v6 32/37] cxlflash: Fix to avoid potential deadlock on EEH
Date: Fri, 23 Oct 2015 16:01:13 +0200 [thread overview]
Message-ID: <562A3DA9.4070904@redhat.com> (raw)
In-Reply-To: <1445458552-61150-1-git-send-email-mrochs@linux.vnet.ibm.com>
On 21.10.2015 22:15, Matthew R. Ochs wrote:
> Ioctl threads that use scsi_execute() can run for an excessive amount
> of time due to the fact that they have lengthy timeouts and retry logic
> built in. Under normal operation this is not an issue. However, once EEH
> enters the picture, a long execution time coupled with the possibility
> that a timeout can trigger entry to the driver via registered reset
> callbacks becomes a liability.
>
> In particular, a deadlock can occur when an EEH event is encountered
> while in running in scsi_execute(). As part of the recovery, the EEH
> handler drains all currently running ioctls, waiting until they have
> completed before proceeding with a reset. As the scsi_execute()'s are
> situated on the ioctl path, the EEH handler will wait until they (and
> the remainder of the ioctl handler they're associated with) have
> completed. Normally this would not be much of an issue aside from the
> longer recovery period. Unfortunately, the scsi_execute() triggers a
> reset when it times out. The reset handler will see that the device is
> already being reset and wait until that reset completed. This creates
> a condition where the EEH handler becomes stuck, infinitely waiting for
> the ioctl thread to complete.
>
> To avoid this behavior, temporarily unmark the scsi_execute() threads
> as an ioctl thread by releasing the ioctl read semaphore. This allows
> the EEH handler to proceed with a recovery while the thread is still
> running. Once the scsi_execute() returns, the ioctl read semaphore is
> reacquired and the adapter state is rechecked in case it changed while
> inside of scsi_execute(). The state check will wait if the adapter is
> still being recovered or returns a failure if the recovery failed. In
> the event that the adapter reset failed, the failure is simply returned
> as the ioctl would be unable to continue.
>
> Reported-by: Brian King <brking@linux.vnet.ibm.com>
> Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
> Signed-off-by: Manoj N. Kumar <manoj@linux.vnet.ibm.com>
> Reviewed-by: Brian King <brking@linux.vnet.ibm.com>
> Reviewed-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Tomas Henzl <thenzl@redhat.com>
Tomas
next prev parent reply other threads:[~2015-10-23 14:01 UTC|newest]
Thread overview: 70+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-21 20:08 [PATCH v6 00/37] cxlflash: Miscellaneous bug fixes and corrections Matthew R. Ochs
2015-10-21 20:10 ` [PATCH v6 01/37] cxlflash: Fix to avoid invalid port_sel value Matthew R. Ochs
2015-10-21 20:11 ` [PATCH v6 02/37] cxlflash: Replace magic numbers with literals Matthew R. Ochs
2015-10-21 20:11 ` [PATCH v6 03/37] cxlflash: Fix read capacity timeout Matthew R. Ochs
2015-10-21 20:11 ` [PATCH v6 04/37] cxlflash: Fix potential oops following LUN removal Matthew R. Ochs
2015-10-21 20:11 ` [PATCH v6 05/37] cxlflash: Fix data corruption when vLUN used over multiple cards Matthew R. Ochs
2015-10-21 20:11 ` [PATCH v6 06/37] cxlflash: Fix to avoid sizeof(bool) Matthew R. Ochs
2015-10-21 20:11 ` [PATCH v6 07/37] cxlflash: Fix context encode mask width Matthew R. Ochs
2015-10-21 20:11 ` [PATCH v6 08/37] cxlflash: Fix to avoid CXL services during EEH Matthew R. Ochs
2015-10-21 20:12 ` [PATCH v6 09/37] cxlflash: Correct naming of limbo state and waitq Matthew R. Ochs
2015-10-21 20:12 ` [PATCH v6 10/37] cxlflash: Make functions static Matthew R. Ochs
2015-10-21 20:12 ` [PATCH v6 11/37] cxlflash: Refine host/device attributes Matthew R. Ochs
2015-10-23 13:33 ` Tomas Henzl
2015-10-21 20:13 ` [PATCH v6 12/37] cxlflash: Fix to avoid spamming the kernel log Matthew R. Ochs
2015-10-23 13:33 ` Tomas Henzl
2015-10-21 20:13 ` [PATCH v6 13/37] cxlflash: Fix to avoid stall while waiting on TMF Matthew R. Ochs
2015-10-23 13:36 ` Tomas Henzl
2015-10-21 20:13 ` [PATCH v6 14/37] cxlflash: Fix location of setting resid Matthew R. Ochs
2015-10-23 13:37 ` Tomas Henzl
2015-10-21 20:13 ` [PATCH v6 15/37] cxlflash: Fix host link up event handling Matthew R. Ochs
2015-10-23 13:38 ` Tomas Henzl
2015-10-21 20:13 ` [PATCH v6 16/37] cxlflash: Fix async interrupt bypass logic Matthew R. Ochs
2015-10-23 3:40 ` Andrew Donnellan
2015-10-23 13:39 ` Tomas Henzl
2015-10-21 20:13 ` [PATCH v6 17/37] cxlflash: Remove dual port online dependency Matthew R. Ochs
2015-10-23 13:39 ` Tomas Henzl
2015-10-21 20:14 ` [PATCH v6 18/37] cxlflash: Fix AFU version access/storage and add check Matthew R. Ochs
2015-10-23 13:40 ` Tomas Henzl
2015-10-21 20:14 ` [PATCH v6 19/37] cxlflash: Correct usage of scsi_host_put() Matthew R. Ochs
2015-10-23 13:41 ` Tomas Henzl
2015-10-21 20:14 ` [PATCH v6 20/37] cxlflash: Fix to prevent workq from accessing freed memory Matthew R. Ochs
2015-10-23 13:41 ` Tomas Henzl
2015-10-21 20:14 ` [PATCH v6 21/37] cxlflash: Correct behavior in device reset handler following EEH Matthew R. Ochs
2015-10-23 13:42 ` Tomas Henzl
2015-10-21 20:14 ` [PATCH v6 22/37] cxlflash: Remove unnecessary scsi_block_requests Matthew R. Ochs
2015-10-23 13:42 ` Tomas Henzl
2015-10-21 20:14 ` [PATCH v6 23/37] cxlflash: Fix function prolog parameters and return codes Matthew R. Ochs
2015-10-23 13:45 ` Tomas Henzl
2015-10-21 20:14 ` [PATCH v6 24/37] cxlflash: Fix MMIO and endianness errors Matthew R. Ochs
2015-10-23 13:53 ` Tomas Henzl
2015-10-21 20:14 ` [PATCH v6 25/37] cxlflash: Fix to prevent EEH recovery failure Matthew R. Ochs
2015-10-23 13:54 ` Tomas Henzl
2015-10-21 20:15 ` [PATCH v6 26/37] cxlflash: Correct spelling, grammar, and alignment mistakes Matthew R. Ochs
2015-10-23 13:54 ` Tomas Henzl
2015-10-21 20:15 ` [PATCH v6 27/37] cxlflash: Fix to prevent stale AFU RRQ Matthew R. Ochs
2015-10-23 13:55 ` Tomas Henzl
2015-10-21 20:15 ` [PATCH v6 28/37] MAINTAINERS: Add cxlflash driver Matthew R. Ochs
2015-10-21 20:15 ` [PATCH v6 29/37] cxlflash: Fix to double the delay each time Matthew R. Ochs
2015-10-23 13:57 ` Tomas Henzl
2015-10-21 20:15 ` [PATCH v6 30/37] cxlflash: Fix to avoid corrupting adapter fops Matthew R. Ochs
2015-10-23 14:00 ` Tomas Henzl
2015-10-21 20:15 ` [PATCH v6 31/37] cxlflash: Correct trace string Matthew R. Ochs
2015-10-23 14:00 ` Tomas Henzl
2015-10-21 20:15 ` [PATCH v6 32/37] cxlflash: Fix to avoid potential deadlock on EEH Matthew R. Ochs
2015-10-23 14:01 ` Tomas Henzl [this message]
2015-10-21 20:16 ` [PATCH v6 33/37] cxlflash: Fix to avoid leaving dangling interrupt resources Matthew R. Ochs
2015-10-23 14:01 ` Tomas Henzl
2015-10-21 20:16 ` [PATCH v6 34/37] cxlflash: Fix to escalate to LINK_RESET on login timeout Matthew R. Ochs
2015-10-23 14:01 ` Tomas Henzl
2015-10-21 20:16 ` [PATCH v6 35/37] cxlflash: Fix to avoid corrupting port selection mask Matthew R. Ochs
2015-10-22 17:17 ` Manoj Kumar
2015-10-23 3:52 ` Andrew Donnellan
2015-10-21 20:16 ` [PATCH v6 36/37] cxlflash: Fix to avoid lock instrumentation rejection Matthew R. Ochs
2015-10-22 17:34 ` Manoj Kumar
2015-10-23 3:22 ` Andrew Donnellan
2015-10-21 20:16 ` [PATCH v6 37/37] cxlflash: Fix to avoid bypassing context cleanup Matthew R. Ochs
2015-10-22 2:01 ` Andrew Donnellan
2015-10-22 18:05 ` Manoj Kumar
2015-10-27 23:30 ` [PATCH v6 00/37] cxlflash: Miscellaneous bug fixes and corrections Matthew R. Ochs
2015-10-27 23:30 ` Matthew R. Ochs
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=562A3DA9.4070904@redhat.com \
--to=thenzl@redhat.com \
--cc=David.Laight@ACULAB.COM \
--cc=James.Bottomley@HansenPartnership.com \
--cc=andrew.donnellan@au1.ibm.com \
--cc=brking@linux.vnet.ibm.com \
--cc=dja@ozlabs.au.ibm.com \
--cc=imunsie@au1.ibm.com \
--cc=linux-scsi@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=manoj@linux.vnet.ibm.com \
--cc=mikey@neuling.org \
--cc=mrochs@linux.vnet.ibm.com \
--cc=nab@linux-iscsi.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.