* [PATCH v9 1/9] Fix race between starved list processing and device removal
2013-02-15 12:02 [PATCH v9 0/9] More device removal fixes Bart Van Assche
@ 2013-02-15 12:04 ` Bart Van Assche
2013-02-15 12:05 ` [PATCH v9 2/9] Remove get_device() / put_device() pair from scsi_request_fn() Bart Van Assche
` (7 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Bart Van Assche @ 2013-02-15 12:04 UTC (permalink / raw)
To: linux-scsi
Cc: James Bottomley, Mike Christie, Tejun Heo, Chanho Min,
David Milburn
scsi_run_queue() examines all SCSI devices that are present on
the starved list. Since scsi_run_queue() unlocks the SCSI host
lock before running a queue a SCSI device can get removed after
it has been removed from the starved list and before its queue
is run. Protect against that race condition by increasing the
sdev refcount before running the sdev queue. Also, remove a
SCSI device from the starved list before __scsi_remove_device()
decreases the sdev refcount such that the get_device() call
added in scsi_run_queue() is guaranteed to succeed.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Reported-and-tested-by: Chanho Min <chanho.min@lge.com>
Reference: http://lkml.org/lkml/2012/8/2/96
Acked-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Mike Christie <michaelc@cs.wisc.edu>
Cc: Hannes Reinecke <hare@suse.de>
Cc: <stable@vger.kernel.org>
---
drivers/scsi/scsi_lib.c | 16 +++++++++++-----
drivers/scsi/scsi_sysfs.c | 14 +++++++++++++-
2 files changed, 24 insertions(+), 6 deletions(-)
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index f1bf5af..5c67339 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -452,11 +452,17 @@ static void scsi_run_queue(struct request_queue *q)
continue;
}
- spin_unlock(shost->host_lock);
- spin_lock(sdev->request_queue->queue_lock);
- __blk_run_queue(sdev->request_queue);
- spin_unlock(sdev->request_queue->queue_lock);
- spin_lock(shost->host_lock);
+ /*
+ * Obtain a reference before unlocking the host_lock such that
+ * the sdev can't disappear before blk_run_queue() is invoked.
+ */
+ get_device(&sdev->sdev_gendev);
+ spin_unlock_irqrestore(shost->host_lock, flags);
+
+ blk_run_queue(sdev->request_queue);
+ put_device(&sdev->sdev_gendev);
+
+ spin_lock_irqsave(shost->host_lock, flags);
}
/* put any unprocessed entries back */
list_splice(&starved_list, &shost->starved_list);
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 931a7d9..34f1b39 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -345,7 +345,6 @@ static void scsi_device_dev_release_usercontext(struct work_struct *work)
starget->reap_ref++;
list_del(&sdev->siblings);
list_del(&sdev->same_target_siblings);
- list_del(&sdev->starved_entry);
spin_unlock_irqrestore(sdev->host->host_lock, flags);
cancel_work_sync(&sdev->event_work);
@@ -953,6 +952,8 @@ int scsi_sysfs_add_sdev(struct scsi_device *sdev)
void __scsi_remove_device(struct scsi_device *sdev)
{
struct device *dev = &sdev->sdev_gendev;
+ struct Scsi_Host *shost = sdev->host;
+ unsigned long flags;
if (sdev->is_visible) {
if (scsi_device_set_state(sdev, SDEV_CANCEL) != 0)
@@ -974,6 +975,17 @@ void __scsi_remove_device(struct scsi_device *sdev)
blk_cleanup_queue(sdev->request_queue);
cancel_work_sync(&sdev->requeue_work);
+ /*
+ * Remove a SCSI device from the starved list after blk_cleanup_queue()
+ * finished such that scsi_request_fn() can't add it back to that list.
+ * Also remove an sdev from the starved list before invoking
+ * put_device() such that get_device() is guaranteed to succeed for an
+ * sdev present on the starved list.
+ */
+ spin_lock_irqsave(shost->host_lock, flags);
+ list_del(&sdev->starved_entry);
+ spin_unlock_irqrestore(shost->host_lock, flags);
+
if (sdev->host->hostt->slave_destroy)
sdev->host->hostt->slave_destroy(sdev);
transport_destroy_device(dev);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v9 2/9] Remove get_device() / put_device() pair from scsi_request_fn()
2013-02-15 12:02 [PATCH v9 0/9] More device removal fixes Bart Van Assche
2013-02-15 12:04 ` [PATCH v9 1/9] Fix race between starved list processing and device removal Bart Van Assche
@ 2013-02-15 12:05 ` Bart Van Assche
2013-02-15 12:06 ` [PATCH v9 3/9] Avoid calling __scsi_remove_device() twice Bart Van Assche
` (6 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Bart Van Assche @ 2013-02-15 12:05 UTC (permalink / raw)
To: linux-scsi
Cc: James Bottomley, Mike Christie, Tejun Heo, Chanho Min,
David Milburn
Now that all scsi_request_fn() callers hold a reference on the
SCSI device that function is invoked for and since
blk_cleanup_queue() waits until scsi_request_fn() has finished
it is safe to remove the get_device() / put_device() pair from
scsi_request_fn().
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Acked-by: Tejun Heo <tj@kernel.org>
Cc: James Bottomley <JBottomley@Parallels.com>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Mike Christie <michaelc@cs.wisc.edu>
---
drivers/scsi/scsi_lib.c | 14 +++-----------
1 file changed, 3 insertions(+), 11 deletions(-)
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 5c67339..db4836bc 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -1529,16 +1529,14 @@ static void scsi_softirq_done(struct request *rq)
* Lock status: IO request lock assumed to be held when called.
*/
static void scsi_request_fn(struct request_queue *q)
+ __releases(q->queue_lock)
+ __acquires(q->queue_lock)
{
struct scsi_device *sdev = q->queuedata;
struct Scsi_Host *shost;
struct scsi_cmnd *cmd;
struct request *req;
- if(!get_device(&sdev->sdev_gendev))
- /* We must be tearing the block queue down already */
- return;
-
/*
* To start with, we keep looping until the queue is empty, or until
* the host is no longer able to accept any more requests.
@@ -1627,7 +1625,7 @@ static void scsi_request_fn(struct request_queue *q)
goto out_delay;
}
- goto out;
+ return;
not_ready:
spin_unlock_irq(shost->host_lock);
@@ -1646,12 +1644,6 @@ static void scsi_request_fn(struct request_queue *q)
out_delay:
if (sdev->device_busy == 0)
blk_delay_queue(q, SCSI_QUEUE_DELAY);
-out:
- /* must be careful here...if we trigger the ->remove() function
- * we cannot be holding the q lock */
- spin_unlock_irq(q->queue_lock);
- put_device(&sdev->sdev_gendev);
- spin_lock_irq(q->queue_lock);
}
u64 scsi_calculate_bounce_limit(struct Scsi_Host *shost)
--
1.7.10.4
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v9 3/9] Avoid calling __scsi_remove_device() twice
2013-02-15 12:02 [PATCH v9 0/9] More device removal fixes Bart Van Assche
2013-02-15 12:04 ` [PATCH v9 1/9] Fix race between starved list processing and device removal Bart Van Assche
2013-02-15 12:05 ` [PATCH v9 2/9] Remove get_device() / put_device() pair from scsi_request_fn() Bart Van Assche
@ 2013-02-15 12:06 ` Bart Van Assche
2013-02-15 12:06 ` [PATCH v9 4/9] Disallow changing the device state via sysfs into "deleted" Bart Van Assche
` (5 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Bart Van Assche @ 2013-02-15 12:06 UTC (permalink / raw)
To: linux-scsi
Cc: James Bottomley, Mike Christie, Tejun Heo, Chanho Min,
David Milburn
SCSI devices are added to the shost->__devices list from inside
scsi_alloc_sdev(). If something goes wrong during LUN scanning,
e.g. a transport layer failure occurs, then __scsi_remove_device()
can get invoked by the LUN scanning code for a SCSI device in
state SDEV_CREATED_BLOCK or SDEV_BLOCKED. If this happens this
happens before a SCSI device is added to sysfs (is_visible == 0).
Make sure that if this happens these devices are transitioned
into state SDEV_DEL. This avoids that __scsi_remove_device()
gets invoked a second time by scsi_forget_host().
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Cc: James Bottomley <JBottomley@Parallels.com>
Cc: Mike Christie <michaelc@cs.wisc.edu>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Tejun Heo <tj@kernel.org>
---
drivers/scsi/scsi_lib.c | 2 ++
drivers/scsi/scsi_sysfs.c | 5 ++---
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index db4836bc..555d2a8 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -2171,6 +2171,8 @@ scsi_device_set_state(struct scsi_device *sdev, enum scsi_device_state state)
case SDEV_OFFLINE:
case SDEV_TRANSPORT_OFFLINE:
case SDEV_CANCEL:
+ case SDEV_BLOCK:
+ case SDEV_CREATED_BLOCK:
break;
default:
goto illegal;
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 34f1b39..c46751b 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -956,8 +956,7 @@ void __scsi_remove_device(struct scsi_device *sdev)
unsigned long flags;
if (sdev->is_visible) {
- if (scsi_device_set_state(sdev, SDEV_CANCEL) != 0)
- return;
+ WARN_ON_ONCE(scsi_device_set_state(sdev, SDEV_CANCEL) != 0);
bsg_unregister_queue(sdev->request_queue);
device_unregister(&sdev->sdev_dev);
@@ -971,7 +970,7 @@ void __scsi_remove_device(struct scsi_device *sdev)
* scsi_run_queue() invocations have finished before tearing down the
* device.
*/
- scsi_device_set_state(sdev, SDEV_DEL);
+ WARN_ON_ONCE(scsi_device_set_state(sdev, SDEV_DEL) != 0);
blk_cleanup_queue(sdev->request_queue);
cancel_work_sync(&sdev->requeue_work);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v9 4/9] Disallow changing the device state via sysfs into "deleted"
2013-02-15 12:02 [PATCH v9 0/9] More device removal fixes Bart Van Assche
` (2 preceding siblings ...)
2013-02-15 12:06 ` [PATCH v9 3/9] Avoid calling __scsi_remove_device() twice Bart Van Assche
@ 2013-02-15 12:06 ` Bart Van Assche
2013-02-15 12:07 ` [PATCH v9 5/9] Avoid saving/restoring interrupt state inside scsi_remove_host() Bart Van Assche
` (4 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Bart Van Assche @ 2013-02-15 12:06 UTC (permalink / raw)
To: linux-scsi
Cc: James Bottomley, Mike Christie, Tejun Heo, Chanho Min,
David Milburn
Changing the state of a SCSI device via sysfs into "cancel" or
"deleted" prevents removal of these devices by scsi_remove_host().
Hence do not allow this. Also, introduce the symbolic name
INVALID_SDEV_STATE, representing a value different from any valid
SCSI device state. Update scsi_device_set_state() such that gcc
does not issue a warning about an enumeration value not being
handled inside a switch statement.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Cc: Tejun Heo <tj@kernel.org>
Cc: James Bottomley <JBottomley@Parallels.com>
Cc: Mike Christie <michaelc@cs.wisc.edu>
Cc: Hannes Reinecke <hare@suse.de>
Cc: David Milburn <dmilburn@redhat.com>
---
drivers/scsi/scsi_lib.c | 2 ++
drivers/scsi/scsi_sysfs.c | 5 +++--
include/scsi/scsi_device.h | 6 +++++-
3 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 555d2a8..d2a2a2f 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -2179,6 +2179,8 @@ scsi_device_set_state(struct scsi_device *sdev, enum scsi_device_state state)
}
break;
+ case INVALID_SDEV_STATE:
+ goto illegal;
}
sdev->sdev_state = state;
return 0;
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index c46751b..47c1e0e 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -594,7 +594,7 @@ store_state_field(struct device *dev, struct device_attribute *attr,
{
int i;
struct scsi_device *sdev = to_scsi_device(dev);
- enum scsi_device_state state = 0;
+ enum scsi_device_state state = INVALID_SDEV_STATE;
for (i = 0; i < ARRAY_SIZE(sdev_states); i++) {
const int len = strlen(sdev_states[i].name);
@@ -604,7 +604,8 @@ store_state_field(struct device *dev, struct device_attribute *attr,
break;
}
}
- if (!state)
+ if (state == INVALID_SDEV_STATE || state == SDEV_CANCEL ||
+ state == SDEV_DEL)
return -EINVAL;
if (scsi_device_set_state(sdev, state))
diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h
index e65c62e..08a2878 100644
--- a/include/scsi/scsi_device.h
+++ b/include/scsi/scsi_device.h
@@ -29,7 +29,11 @@ struct scsi_mode_data {
* scsi_lib:scsi_device_set_state().
*/
enum scsi_device_state {
- SDEV_CREATED = 1, /* device created but not added to sysfs
+ INVALID_SDEV_STATE, /* Not a valid SCSI device state but a
+ * symbolic name that can be used wherever
+ * a value is needed that is different from
+ * any valid SCSI device state. */
+ SDEV_CREATED, /* device created but not added to sysfs
* Only internal commands allowed (for inq) */
SDEV_RUNNING, /* device properly configured
* All commands allowed */
--
1.7.10.4
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v9 5/9] Avoid saving/restoring interrupt state inside scsi_remove_host()
2013-02-15 12:02 [PATCH v9 0/9] More device removal fixes Bart Van Assche
` (3 preceding siblings ...)
2013-02-15 12:06 ` [PATCH v9 4/9] Disallow changing the device state via sysfs into "deleted" Bart Van Assche
@ 2013-02-15 12:07 ` Bart Van Assche
2013-02-15 12:08 ` [PATCH v9 6/9] Make scsi_remove_host() wait until error handling finished Bart Van Assche
` (3 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Bart Van Assche @ 2013-02-15 12:07 UTC (permalink / raw)
To: linux-scsi
Cc: James Bottomley, Mike Christie, Tejun Heo, Chanho Min,
David Milburn
Since it is not allowed to invoke scsi_remove_host() with interrupts
disabled, avoid saving and restoring the interrupt state inside
scsi_remove_host(). This patch does not change the functionality of
the function scsi_remove_host().
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Acked-by: Tejun Heo <tj@kernel.org>
Acked-by: Hannes Reinecke <hare@suse.de>
Cc: Mike Christie <michaelc@cs.wisc.edu>
Cc: James Bottomley <JBottomley@Parallels.com>
---
drivers/scsi/hosts.c | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/drivers/scsi/hosts.c b/drivers/scsi/hosts.c
index 593085a..6ae16cd 100644
--- a/drivers/scsi/hosts.c
+++ b/drivers/scsi/hosts.c
@@ -156,27 +156,25 @@ EXPORT_SYMBOL(scsi_host_set_state);
**/
void scsi_remove_host(struct Scsi_Host *shost)
{
- unsigned long flags;
-
mutex_lock(&shost->scan_mutex);
- spin_lock_irqsave(shost->host_lock, flags);
+ spin_lock_irq(shost->host_lock);
if (scsi_host_set_state(shost, SHOST_CANCEL))
if (scsi_host_set_state(shost, SHOST_CANCEL_RECOVERY)) {
- spin_unlock_irqrestore(shost->host_lock, flags);
+ spin_unlock_irq(shost->host_lock);
mutex_unlock(&shost->scan_mutex);
return;
}
- spin_unlock_irqrestore(shost->host_lock, flags);
+ spin_unlock_irq(shost->host_lock);
scsi_autopm_get_host(shost);
scsi_forget_host(shost);
mutex_unlock(&shost->scan_mutex);
scsi_proc_host_rm(shost);
- spin_lock_irqsave(shost->host_lock, flags);
+ spin_lock_irq(shost->host_lock);
if (scsi_host_set_state(shost, SHOST_DEL))
BUG_ON(scsi_host_set_state(shost, SHOST_DEL_RECOVERY));
- spin_unlock_irqrestore(shost->host_lock, flags);
+ spin_unlock_irq(shost->host_lock);
transport_unregister_device(&shost->shost_gendev);
device_unregister(&shost->shost_dev);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v9 6/9] Make scsi_remove_host() wait until error handling finished
2013-02-15 12:02 [PATCH v9 0/9] More device removal fixes Bart Van Assche
` (4 preceding siblings ...)
2013-02-15 12:07 ` [PATCH v9 5/9] Avoid saving/restoring interrupt state inside scsi_remove_host() Bart Van Assche
@ 2013-02-15 12:08 ` Bart Van Assche
2013-02-15 12:08 ` [PATCH v9 7/9] Avoid that scsi_device_set_state() triggers a race Bart Van Assche
` (2 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Bart Van Assche @ 2013-02-15 12:08 UTC (permalink / raw)
To: linux-scsi
Cc: James Bottomley, Mike Christie, Tejun Heo, Chanho Min,
David Milburn
A SCSI LLD may start cleaning up host resources as soon as
scsi_remove_host() returns. These host resources may be needed by
the LLD in an implementation of one of the eh_* functions. So if
one of the eh_* functions is in progress when scsi_remove_host()
is invoked, wait until the eh_* function has finished. Also, do
not invoke any of the eh_* functions after scsi_remove_host() has
started. Remove Scsi_Host.tmf_in_progress because it is now
superfluous.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Mike Christie <michaelc@cs.wisc.edu>
Cc: Tejun Heo <tj@kernel.org>
---
drivers/scsi/hosts.c | 6 ++++
drivers/scsi/scsi_error.c | 86 ++++++++++++++++++++++++++++++++++++++-------
include/scsi/scsi_host.h | 6 ++--
3 files changed, 81 insertions(+), 17 deletions(-)
diff --git a/drivers/scsi/hosts.c b/drivers/scsi/hosts.c
index 6ae16cd..ea80ca4 100644
--- a/drivers/scsi/hosts.c
+++ b/drivers/scsi/hosts.c
@@ -176,6 +176,12 @@ void scsi_remove_host(struct Scsi_Host *shost)
BUG_ON(scsi_host_set_state(shost, SHOST_DEL_RECOVERY));
spin_unlock_irq(shost->host_lock);
+ /*
+ * Wait until the error handler has finished invoking LLD callbacks
+ * before allowing the LLD to proceed.
+ */
+ wait_event(shost->host_wait, shost->eh_active == 0);
+
transport_unregister_device(&shost->shost_gendev);
device_unregister(&shost->shost_dev);
device_del(&shost->shost_gendev);
diff --git a/drivers/scsi/scsi_error.c b/drivers/scsi/scsi_error.c
index c1b05a8..b739afe 100644
--- a/drivers/scsi/scsi_error.c
+++ b/drivers/scsi/scsi_error.c
@@ -536,8 +536,53 @@ static void scsi_eh_done(struct scsi_cmnd *scmd)
}
/**
+ * scsi_begin_eh - start host-related error handling
+ *
+ * Must be called before invoking an LLD callback function to avoid that
+ * scsi_remove_host() returns while one of these callback functions is in
+ * progress.
+ *
+ * Returns 0 if invoking an eh_* function is allowed and a negative value if
+ * not. If this function returns 0 then scsi_end_eh() must be called
+ * eventually.
+ */
+static int scsi_begin_eh(struct Scsi_Host *host)
+{
+ int res;
+
+ spin_lock_irq(host->host_lock);
+ switch (host->shost_state) {
+ case SHOST_DEL:
+ case SHOST_DEL_RECOVERY:
+ res = -ENODEV;
+ break;
+ default:
+ WARN_ON_ONCE(host->eh_active < 0);
+ host->eh_active++;
+ res = 0;
+ break;
+ }
+ spin_unlock_irq(host->host_lock);
+
+ return res;
+}
+
+/**
+ * scsi_end_eh - finish host-related error handling
+ */
+static void scsi_end_eh(struct Scsi_Host *host)
+{
+ spin_lock_irq(host->host_lock);
+ host->eh_active--;
+ WARN_ON_ONCE(host->eh_active < 0);
+ if (host->eh_active == 0)
+ wake_up(&host->host_wait);
+ spin_unlock_irq(host->host_lock);
+}
+
+/**
* scsi_try_host_reset - ask host adapter to reset itself
- * @scmd: SCSI cmd to send hsot reset.
+ * @scmd: SCSI cmd to send host reset.
*/
static int scsi_try_host_reset(struct scsi_cmnd *scmd)
{
@@ -552,6 +597,9 @@ static int scsi_try_host_reset(struct scsi_cmnd *scmd)
if (!hostt->eh_host_reset_handler)
return FAILED;
+ if (scsi_begin_eh(host))
+ return FAST_IO_FAIL;
+
rtn = hostt->eh_host_reset_handler(scmd);
if (rtn == SUCCESS) {
@@ -561,6 +609,7 @@ static int scsi_try_host_reset(struct scsi_cmnd *scmd)
scsi_report_bus_reset(host, scmd_channel(scmd));
spin_unlock_irqrestore(host->host_lock, flags);
}
+ scsi_end_eh(host);
return rtn;
}
@@ -582,6 +631,9 @@ static int scsi_try_bus_reset(struct scsi_cmnd *scmd)
if (!hostt->eh_bus_reset_handler)
return FAILED;
+ if (scsi_begin_eh(host))
+ return FAST_IO_FAIL;
+
rtn = hostt->eh_bus_reset_handler(scmd);
if (rtn == SUCCESS) {
@@ -591,6 +643,7 @@ static int scsi_try_bus_reset(struct scsi_cmnd *scmd)
scsi_report_bus_reset(host, scmd_channel(scmd));
spin_unlock_irqrestore(host->host_lock, flags);
}
+ scsi_end_eh(host);
return rtn;
}
@@ -621,6 +674,9 @@ static int scsi_try_target_reset(struct scsi_cmnd *scmd)
if (!hostt->eh_target_reset_handler)
return FAILED;
+ if (scsi_begin_eh(host))
+ return FAST_IO_FAIL;
+
rtn = hostt->eh_target_reset_handler(scmd);
if (rtn == SUCCESS) {
spin_lock_irqsave(host->host_lock, flags);
@@ -628,6 +684,7 @@ static int scsi_try_target_reset(struct scsi_cmnd *scmd)
__scsi_report_device_reset);
spin_unlock_irqrestore(host->host_lock, flags);
}
+ scsi_end_eh(host);
return rtn;
}
@@ -645,14 +702,20 @@ static int scsi_try_target_reset(struct scsi_cmnd *scmd)
static int scsi_try_bus_device_reset(struct scsi_cmnd *scmd)
{
int rtn;
- struct scsi_host_template *hostt = scmd->device->host->hostt;
+ struct Scsi_Host *host = scmd->device->host;
+ struct scsi_host_template *hostt = host->hostt;
if (!hostt->eh_device_reset_handler)
return FAILED;
+ if (scsi_begin_eh(host))
+ return FAST_IO_FAIL;
+
rtn = hostt->eh_device_reset_handler(scmd);
if (rtn == SUCCESS)
__scsi_report_device_reset(scmd->device, NULL);
+ scsi_end_eh(host);
+
return rtn;
}
@@ -795,6 +858,9 @@ static int scsi_send_eh_cmnd(struct scsi_cmnd *scmd, unsigned char *cmnd,
struct scsi_eh_save ses;
int rtn;
+ if (scsi_begin_eh(shost))
+ return FAILED;
+
scsi_eh_prep_cmnd(scmd, &ses, cmnd, cmnd_size, sense_bytes);
shost->eh_action = &done;
@@ -850,6 +916,8 @@ static int scsi_send_eh_cmnd(struct scsi_cmnd *scmd, unsigned char *cmnd,
rtn = sdrv->eh_action(scmd, cmnd, cmnd_size, rtn);
}
+ scsi_end_eh(shost);
+
return rtn;
}
@@ -1877,6 +1945,9 @@ int scsi_error_handler(void *data)
}
__set_current_state(TASK_RUNNING);
+ WARN_ONCE(shost->eh_active, "scsi_eh_%d: eh_active = %d\n",
+ shost->host_no, shost->eh_active);
+
SCSI_LOG_ERROR_RECOVERY(1,
printk("Error handler scsi_eh_%d exiting\n", shost->host_no));
shost->ehandler = NULL;
@@ -1973,7 +2044,6 @@ scsi_reset_provider(struct scsi_device *dev, int flag)
struct scsi_cmnd *scmd;
struct Scsi_Host *shost = dev->host;
struct request req;
- unsigned long flags;
int rtn;
if (scsi_autopm_get_host(shost) < 0)
@@ -1992,10 +2062,6 @@ scsi_reset_provider(struct scsi_device *dev, int flag)
scmd->sc_data_direction = DMA_BIDIRECTIONAL;
- spin_lock_irqsave(shost->host_lock, flags);
- shost->tmf_in_progress = 1;
- spin_unlock_irqrestore(shost->host_lock, flags);
-
switch (flag) {
case SCSI_TRY_RESET_DEVICE:
rtn = scsi_try_bus_device_reset(scmd);
@@ -2019,10 +2085,6 @@ scsi_reset_provider(struct scsi_device *dev, int flag)
rtn = FAILED;
}
- spin_lock_irqsave(shost->host_lock, flags);
- shost->tmf_in_progress = 0;
- spin_unlock_irqrestore(shost->host_lock, flags);
-
/*
* be sure to wake up anyone who was sleeping or had their queue
* suspended while we performed the TMF.
@@ -2031,8 +2093,6 @@ scsi_reset_provider(struct scsi_device *dev, int flag)
printk("%s: waking up host to restart after TMF\n",
__func__));
- wake_up(&shost->host_wait);
-
scsi_run_host_queues(shost);
scsi_next_command(scmd);
diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
index 4908480..bf78c08 100644
--- a/include/scsi/scsi_host.h
+++ b/include/scsi/scsi_host.h
@@ -576,6 +576,7 @@ struct Scsi_Host {
struct task_struct * ehandler; /* Error recovery thread. */
struct completion * eh_action; /* Wait for specific actions on the
host. */
+ int eh_active;
wait_queue_head_t host_wait;
struct scsi_host_template *hostt;
struct scsi_transport_template *transportt;
@@ -663,9 +664,6 @@ struct Scsi_Host {
*/
unsigned ordered_tag:1;
- /* Task mgmt function in progress */
- unsigned tmf_in_progress:1;
-
/* Asynchronous scan in progress */
unsigned async_scan:1;
@@ -769,7 +767,7 @@ static inline int scsi_host_in_recovery(struct Scsi_Host *shost)
return shost->shost_state == SHOST_RECOVERY ||
shost->shost_state == SHOST_CANCEL_RECOVERY ||
shost->shost_state == SHOST_DEL_RECOVERY ||
- shost->tmf_in_progress;
+ shost->eh_active;
}
extern int scsi_queue_work(struct Scsi_Host *, struct work_struct *);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v9 7/9] Avoid that scsi_device_set_state() triggers a race
2013-02-15 12:02 [PATCH v9 0/9] More device removal fixes Bart Van Assche
` (5 preceding siblings ...)
2013-02-15 12:08 ` [PATCH v9 6/9] Make scsi_remove_host() wait until error handling finished Bart Van Assche
@ 2013-02-15 12:08 ` Bart Van Assche
2013-02-15 12:09 ` [PATCH v9 8/9] Save and restore host_scribble during error handling Bart Van Assche
2013-02-15 12:10 ` [PATCH v9 9/9] Avoid reenabling I/O after the transport became offline Bart Van Assche
8 siblings, 0 replies; 11+ messages in thread
From: Bart Van Assche @ 2013-02-15 12:08 UTC (permalink / raw)
Cc: linux-scsi, James Bottomley, Mike Christie, Tejun Heo, Chanho Min,
David Milburn
Make concurrent invocations of scsi_device_set_state() safe.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Acked-by: Hannes Reinecke <hare@suse.de>
Cc: James Bottomley <JBottomley@Parallels.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Mike Christie <michaelc@cs.wisc.edu>
---
drivers/scsi/scsi_error.c | 4 ++++
drivers/scsi/scsi_lib.c | 43 ++++++++++++++++++++++++++++++++++---------
drivers/scsi/scsi_scan.c | 15 ++++++++-------
drivers/scsi/scsi_sysfs.c | 18 ++++++++++++++----
4 files changed, 60 insertions(+), 20 deletions(-)
diff --git a/drivers/scsi/scsi_error.c b/drivers/scsi/scsi_error.c
index b739afe..a1b7eff 100644
--- a/drivers/scsi/scsi_error.c
+++ b/drivers/scsi/scsi_error.c
@@ -1431,7 +1431,11 @@ static void scsi_eh_offline_sdevs(struct list_head *work_q,
list_for_each_entry_safe(scmd, next, work_q, eh_entry) {
sdev_printk(KERN_INFO, scmd->device, "Device offlined - "
"not ready after error recovery\n");
+
+ spin_lock_irq(scmd->device->host->host_lock);
scsi_device_set_state(scmd->device, SDEV_OFFLINE);
+ spin_unlock_irq(scmd->device->host->host_lock);
+
if (scmd->eh_eflags & SCSI_EH_CANCEL_CMD) {
/*
* FIXME: Handle lost cmds.
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index d2a2a2f..41c0eb1 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -2074,7 +2074,9 @@ EXPORT_SYMBOL(scsi_test_unit_ready);
* @state: state to change to.
*
* Returns zero if unsuccessful or an error if the requested
- * transition is illegal.
+ * transition is illegal. It is the responsibility of the caller to make
+ * sure that a call of this function does not race with other code that
+ * accesses the device state, e.g. by holding the host lock.
*/
int
scsi_device_set_state(struct scsi_device *sdev, enum scsi_device_state state)
@@ -2355,7 +2357,13 @@ EXPORT_SYMBOL_GPL(sdev_evt_send_simple);
int
scsi_device_quiesce(struct scsi_device *sdev)
{
- int err = scsi_device_set_state(sdev, SDEV_QUIESCE);
+ struct Scsi_Host *host = sdev->host;
+ int err;
+
+ spin_lock_irq(host->host_lock);
+ err = scsi_device_set_state(sdev, SDEV_QUIESCE);
+ spin_unlock_irq(host->host_lock);
+
if (err)
return err;
@@ -2379,13 +2387,21 @@ EXPORT_SYMBOL(scsi_device_quiesce);
*/
void scsi_device_resume(struct scsi_device *sdev)
{
+ struct Scsi_Host *host = sdev->host;
+ int err;
+
/* check if the device state was mutated prior to resume, and if
* so assume the state is being managed elsewhere (for example
* device deleted during suspend)
*/
- if (sdev->sdev_state != SDEV_QUIESCE ||
- scsi_device_set_state(sdev, SDEV_RUNNING))
+ spin_lock_irq(host->host_lock);
+ err = sdev->sdev_state == SDEV_QUIESCE ?
+ scsi_device_set_state(sdev, SDEV_RUNNING) : -EINVAL;
+ spin_unlock_irq(host->host_lock);
+
+ if (err)
return;
+
scsi_run_queue(sdev->request_queue);
}
EXPORT_SYMBOL(scsi_device_resume);
@@ -2435,17 +2451,19 @@ EXPORT_SYMBOL(scsi_target_resume);
int
scsi_internal_device_block(struct scsi_device *sdev)
{
+ struct Scsi_Host *host = sdev->host;
struct request_queue *q = sdev->request_queue;
unsigned long flags;
int err = 0;
+ spin_lock_irqsave(host->host_lock, flags);
err = scsi_device_set_state(sdev, SDEV_BLOCK);
- if (err) {
+ if (err)
err = scsi_device_set_state(sdev, SDEV_CREATED_BLOCK);
+ spin_unlock_irqrestore(host->host_lock, flags);
- if (err)
- return err;
- }
+ if (err)
+ return err;
/*
* The device has transitioned to SDEV_BLOCK. Stop the
@@ -2480,13 +2498,16 @@ int
scsi_internal_device_unblock(struct scsi_device *sdev,
enum scsi_device_state new_state)
{
+ struct Scsi_Host *host = sdev->host;
struct request_queue *q = sdev->request_queue;
unsigned long flags;
+ int ret = 0;
/*
* Try to transition the scsi device to SDEV_RUNNING or one of the
* offlined states and goose the device queue if successful.
*/
+ spin_lock_irqsave(host->host_lock, flags);
if ((sdev->sdev_state == SDEV_BLOCK) ||
(sdev->sdev_state == SDEV_TRANSPORT_OFFLINE))
sdev->sdev_state = new_state;
@@ -2498,7 +2519,11 @@ scsi_internal_device_unblock(struct scsi_device *sdev,
sdev->sdev_state = SDEV_CREATED;
} else if (sdev->sdev_state != SDEV_CANCEL &&
sdev->sdev_state != SDEV_OFFLINE)
- return -EINVAL;
+ ret = -EINVAL;
+ spin_unlock_irqrestore(host->host_lock, flags);
+
+ if (ret)
+ return ret;
spin_lock_irqsave(q->queue_lock, flags);
blk_start_queue(q);
diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index 3e58b22..5041aa8 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -898,18 +898,19 @@ static int scsi_add_lun(struct scsi_device *sdev, unsigned char *inq_result,
if (*bflags & BLIST_USE_10_BYTE_MS)
sdev->use_10_for_ms = 1;
+ spin_lock_irq(sdev->host->host_lock);
/* set the device running here so that slave configure
* may do I/O */
ret = scsi_device_set_state(sdev, SDEV_RUNNING);
- if (ret) {
+ if (ret)
ret = scsi_device_set_state(sdev, SDEV_BLOCK);
+ spin_unlock_irq(sdev->host->host_lock);
- if (ret) {
- sdev_printk(KERN_ERR, sdev,
- "in wrong state %s to complete scan\n",
- scsi_device_state_name(sdev->sdev_state));
- return SCSI_SCAN_NO_RESPONSE;
- }
+ if (ret) {
+ sdev_printk(KERN_ERR, sdev,
+ "in wrong state %s to complete scan\n",
+ scsi_device_state_name(sdev->sdev_state));
+ return SCSI_SCAN_NO_RESPONSE;
}
if (*bflags & BLIST_MS_192_BYTES_FOR_3F)
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 47c1e0e..1463931 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -592,7 +592,7 @@ static ssize_t
store_state_field(struct device *dev, struct device_attribute *attr,
const char *buf, size_t count)
{
- int i;
+ int i, ret;
struct scsi_device *sdev = to_scsi_device(dev);
enum scsi_device_state state = INVALID_SDEV_STATE;
@@ -608,9 +608,11 @@ store_state_field(struct device *dev, struct device_attribute *attr,
state == SDEV_DEL)
return -EINVAL;
- if (scsi_device_set_state(sdev, state))
- return -EINVAL;
- return count;
+ spin_lock_irq(sdev->host->host_lock);
+ ret = scsi_device_set_state(sdev, state);
+ spin_unlock_irq(sdev->host->host_lock);
+
+ return ret < 0 ? ret : count;
}
static ssize_t
@@ -870,7 +872,10 @@ int scsi_sysfs_add_sdev(struct scsi_device *sdev)
struct request_queue *rq = sdev->request_queue;
struct scsi_target *starget = sdev->sdev_target;
+ spin_lock_irq(sdev->host->host_lock);
error = scsi_device_set_state(sdev, SDEV_RUNNING);
+ spin_unlock_irq(sdev->host->host_lock);
+
if (error)
return error;
@@ -957,7 +962,9 @@ void __scsi_remove_device(struct scsi_device *sdev)
unsigned long flags;
if (sdev->is_visible) {
+ spin_lock_irqsave(shost->host_lock, flags);
WARN_ON_ONCE(scsi_device_set_state(sdev, SDEV_CANCEL) != 0);
+ spin_unlock_irqrestore(shost->host_lock, flags);
bsg_unregister_queue(sdev->request_queue);
device_unregister(&sdev->sdev_dev);
@@ -971,7 +978,10 @@ void __scsi_remove_device(struct scsi_device *sdev)
* scsi_run_queue() invocations have finished before tearing down the
* device.
*/
+ spin_lock_irqsave(shost->host_lock, flags);
WARN_ON_ONCE(scsi_device_set_state(sdev, SDEV_DEL) != 0);
+ spin_unlock_irqrestore(shost->host_lock, flags);
+
blk_cleanup_queue(sdev->request_queue);
cancel_work_sync(&sdev->requeue_work);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v9 8/9] Save and restore host_scribble during error handling
2013-02-15 12:02 [PATCH v9 0/9] More device removal fixes Bart Van Assche
` (6 preceding siblings ...)
2013-02-15 12:08 ` [PATCH v9 7/9] Avoid that scsi_device_set_state() triggers a race Bart Van Assche
@ 2013-02-15 12:09 ` Bart Van Assche
2013-02-15 12:10 ` [PATCH v9 9/9] Avoid reenabling I/O after the transport became offline Bart Van Assche
8 siblings, 0 replies; 11+ messages in thread
From: Bart Van Assche @ 2013-02-15 12:09 UTC (permalink / raw)
To: linux-scsi
Cc: James Bottomley, Mike Christie, Tejun Heo, Chanho Min,
David Milburn
A SCSI LLD may overwrite host_scribble in its queuecommand()
implementation. Several drivers need that field to process
requests and aborts correctly. Hence this field must be saved
by scsi_eh_prep_cmnd() and must be restored by
scsi_eh_restore_cmnd().
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Cc: James Bottomley <JBottomley@Parallels.com>
Cc: Mike Christie <michaelc@cs.wisc.edu>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Tejun Heo <tj@kernel.org>
---
drivers/scsi/scsi_error.c | 2 ++
include/scsi/scsi_eh.h | 1 +
2 files changed, 3 insertions(+)
diff --git a/drivers/scsi/scsi_error.c b/drivers/scsi/scsi_error.c
index a1b7eff..33dcce3 100644
--- a/drivers/scsi/scsi_error.c
+++ b/drivers/scsi/scsi_error.c
@@ -770,6 +770,7 @@ void scsi_eh_prep_cmnd(struct scsi_cmnd *scmd, struct scsi_eh_save *ses,
ses->result = scmd->result;
ses->underflow = scmd->underflow;
ses->prot_op = scmd->prot_op;
+ ses->host_scribble = scmd->host_scribble;
scmd->prot_op = SCSI_PROT_NORMAL;
scmd->cmnd = ses->eh_cmnd;
@@ -831,6 +832,7 @@ void scsi_eh_restore_cmnd(struct scsi_cmnd* scmd, struct scsi_eh_save *ses)
scmd->result = ses->result;
scmd->underflow = ses->underflow;
scmd->prot_op = ses->prot_op;
+ scmd->host_scribble = ses->host_scribble;
}
EXPORT_SYMBOL(scsi_eh_restore_cmnd);
diff --git a/include/scsi/scsi_eh.h b/include/scsi/scsi_eh.h
index 06a8790..7bdb02f 100644
--- a/include/scsi/scsi_eh.h
+++ b/include/scsi/scsi_eh.h
@@ -83,6 +83,7 @@ struct scsi_eh_save {
/* new command support */
unsigned char eh_cmnd[BLK_MAX_CDB];
struct scatterlist sense_sgl;
+ unsigned char *host_scribble;
};
extern void scsi_eh_prep_cmnd(struct scsi_cmnd *scmd,
--
1.7.10.4
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v9 9/9] Avoid reenabling I/O after the transport became offline
2013-02-15 12:02 [PATCH v9 0/9] More device removal fixes Bart Van Assche
` (7 preceding siblings ...)
2013-02-15 12:09 ` [PATCH v9 8/9] Save and restore host_scribble during error handling Bart Van Assche
@ 2013-02-15 12:10 ` Bart Van Assche
2013-02-15 21:56 ` Mike Christie
8 siblings, 1 reply; 11+ messages in thread
From: Bart Van Assche @ 2013-02-15 12:10 UTC (permalink / raw)
To: linux-scsi
Cc: James Bottomley, Mike Christie, Tejun Heo, Chanho Min,
David Milburn
Disallow the SDEV_TRANSPORT_OFFLINE to SDEV_CANCEL transition such
that no I/O is sent to devices for which the transport is offline.
Notes:
- Functions like sd_shutdown() use scsi_execute_req() and hence
set the REQ_PREEMPT flag. Such requests are passed to the LLD
queuecommand callback in the SDEV_CANCEL state.
- This patch does not affect Fibre Channel LLD drivers since these
drivers invoke fc_remote_port_chkready() before submitting a SCSI
request to the HBA. That prevents a timeout to occur in state
SDEV_CANCEL if the transport is offline.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Cc: James Bottomley <JBottomley@Parallels.com>
Cc: Mike Christie <michaelc@cs.wisc.edu>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Tejun Heo <tj@kernel.org>
---
drivers/scsi/scsi_lib.c | 1 -
drivers/scsi/scsi_sysfs.c | 3 ++-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 41c0eb1..fbf7826 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -2158,7 +2158,6 @@ scsi_device_set_state(struct scsi_device *sdev, enum scsi_device_state state)
case SDEV_RUNNING:
case SDEV_QUIESCE:
case SDEV_OFFLINE:
- case SDEV_TRANSPORT_OFFLINE:
case SDEV_BLOCK:
break;
default:
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 1463931..d0ca3e9 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -963,7 +963,8 @@ void __scsi_remove_device(struct scsi_device *sdev)
if (sdev->is_visible) {
spin_lock_irqsave(shost->host_lock, flags);
- WARN_ON_ONCE(scsi_device_set_state(sdev, SDEV_CANCEL) != 0);
+ WARN_ON_ONCE(scsi_device_set_state(sdev, SDEV_CANCEL) != 0
+ && sdev->sdev_state != SDEV_TRANSPORT_OFFLINE);
spin_unlock_irqrestore(shost->host_lock, flags);
bsg_unregister_queue(sdev->request_queue);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH v9 9/9] Avoid reenabling I/O after the transport became offline
2013-02-15 12:10 ` [PATCH v9 9/9] Avoid reenabling I/O after the transport became offline Bart Van Assche
@ 2013-02-15 21:56 ` Mike Christie
0 siblings, 0 replies; 11+ messages in thread
From: Mike Christie @ 2013-02-15 21:56 UTC (permalink / raw)
To: Bart Van Assche
Cc: linux-scsi, James Bottomley, Tejun Heo, Chanho Min, David Milburn
On 02/15/2013 06:10 AM, Bart Van Assche wrote:
> Disallow the SDEV_TRANSPORT_OFFLINE to SDEV_CANCEL transition such
> that no I/O is sent to devices for which the transport is offline.
> Notes:
> - Functions like sd_shutdown() use scsi_execute_req() and hence
> set the REQ_PREEMPT flag. Such requests are passed to the LLD
> queuecommand callback in the SDEV_CANCEL state.
> - This patch does not affect Fibre Channel LLD drivers since these
> drivers invoke fc_remote_port_chkready() before submitting a SCSI
> request to the HBA. That prevents a timeout to occur in state
> SDEV_CANCEL if the transport is offline.
>
Thanks for fixing this. Looks right to me.
Reviewed-by: Mike Christie <michaelc@cs.wisc.edu>
^ permalink raw reply [flat|nested] 11+ messages in thread