From: james.smart@broadcom.com (James Smart)
Subject: [PATCH 2/2] nvme: flush scan_work when resetting controller
Date: Fri, 21 Jun 2019 10:23:09 -0700 [thread overview]
Message-ID: <3ad2ba8d-980b-e154-a181-b58c5f6d5fdb@broadcom.com> (raw)
In-Reply-To: <5ec67ad6-a61a-b28f-9676-864a5f04bbad@suse.de>
>> Your check can't help wrt. the deadlock, for example:
>>
>> 1) in scan work context:
>>
>> - blk_mq_freeze_queue() is being started after passing the controller
>> state check
>>
>> 2) timeout & reset is triggered in another context at the exact same time:
>>
>> - all in-flight IOs won't be freed until disable controller & reset is done.
>>
>> - now flush_work() in reset context can't move on, because
>> blk_mq_freeze_queue() in scan context can't make progress.
>>
> There might be a difference between RDMA and FC implementations; for FC
> we terminate all outstanding I/Os from the HW side, so each I/O will be
> returned with an aborted status.
> Which for all tests I (and NetApp :-) did was enough to get
> 'blk_mq_freeze_queue()' unstuck and the flush_work to complete.
> We _did_ observed, however, that the state checks are absolutely
> critical to this, otherwise we indeed ended up with a stuck flush_work().
RDMA and TCP does the same thing at what is basically the same point -
as far as terminating all outstanding io. The difference is how they
terminate.? RDMA and TCP use nvme_cancel_request() rather than a call
that induces work on the link. nvme_cancel_request() is near immediate.
FC will take longer for the i/o's to clear - that's the difference.
-- james
next prev parent reply other threads:[~2019-06-21 17:23 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-18 10:10 [PATCH 0/2] nvme: flush rescan worker before resetting Hannes Reinecke
2019-06-18 10:10 ` [PATCH 1/2] nvme: Do not remove namespaces during reset Hannes Reinecke
2019-06-18 17:30 ` Sagi Grimberg
2019-06-20 1:22 ` Ming Lei
2019-06-18 10:10 ` [PATCH 2/2] nvme: flush scan_work when resetting controller Hannes Reinecke
2019-06-18 17:41 ` Sagi Grimberg
2019-06-19 6:22 ` Hannes Reinecke
2019-06-19 16:56 ` Sagi Grimberg
2019-06-19 18:45 ` Hannes Reinecke
2019-06-19 20:04 ` Sagi Grimberg
2019-06-21 16:26 ` Sagi Grimberg
2019-06-24 5:48 ` Hannes Reinecke
2019-06-24 6:13 ` Hannes Reinecke
2019-06-24 18:08 ` Sagi Grimberg
2019-06-24 18:51 ` James Smart
2019-06-25 6:07 ` Hannes Reinecke
2019-06-25 21:50 ` Sagi Grimberg
2019-06-26 5:34 ` Hannes Reinecke
2019-06-26 20:22 ` Sagi Grimberg
2019-07-02 5:38 ` Sagi Grimberg
2019-07-02 13:29 ` Hannes Reinecke
2019-06-20 1:36 ` Ming Lei
2019-06-21 6:14 ` Hannes Reinecke
2019-06-21 6:58 ` Ming Lei
2019-06-21 7:59 ` Hannes Reinecke
2019-06-21 17:23 ` James Smart
2019-06-21 17:23 ` James Smart [this message]
2019-06-24 3:29 ` Ming Lei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3ad2ba8d-980b-e154-a181-b58c5f6d5fdb@broadcom.com \
--to=james.smart@broadcom.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.