public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: Hannes Reinecke <hare@suse.de>
To: Jens Axboe <jens.axboe@oracle.com>
Cc: scameron@beardog.cca.cpqcorp.net, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] cciss: Ignore stale commands after reboot
Date: Thu, 02 Jul 2009 10:44:53 +0200	[thread overview]
Message-ID: <4A4C7385.4030609@suse.de> (raw)
In-Reply-To: <20090702082804.GP23611@kernel.dk>

Jens Axboe wrote:
> On Thu, Jul 02 2009, Hannes Reinecke wrote:
>> When doing an unexpected shutdown like kexec the cciss
>> firmware might still have some commands in flight, which
>> it is trying to complete.
>> The driver is doing it's best on resetting the HBA,
>> but sadly there's a firmware issue causing the firmware
>> _not_ to abort or drop old commands.
>> So the firmware will send us commands which we haven't
>> accounted for, causing the driver to panic.
>>
>> With this patch we're just ignoring these commands as
>> there is nothing we could be doing with them anyway.
>>
>> Signed-off-by: Hannes Reinecke <hare@suse.de>
>> ---
>>  drivers/block/cciss.c     |   14 ++++++++++++--
>>  drivers/block/cciss_cmd.h |    1 +
>>  2 files changed, 13 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/block/cciss.c b/drivers/block/cciss.c
>> index c7a527c..8dd4c0d 100644
>> --- a/drivers/block/cciss.c
>> +++ b/drivers/block/cciss.c
>> @@ -226,7 +226,16 @@ static inline void addQ(struct hlist_head *list, CommandList_struct *c)
>>  
>>  static inline void removeQ(CommandList_struct *c)
>>  {
>> -	if (WARN_ON(hlist_unhashed(&c->list)))
>> +	/*
>> +	 * After kexec/dump some commands might still
>> +	 * be in flight, which the firmware will try
>> +	 * to complete. Resetting the firmware doesn't work
>> +	 * with old fw revisions, so we have to mark
>> +	 * them off as 'stale' to prevent the driver from
>> +	 * falling over.
>> +	 */
>> +	if (unlikely(hlist_unhashed(&c->list))) {
>> +		c->cmd_type = CMD_MSG_STALE;
>>  		return;
>>
>>       hlist_del_init(&c->list);
> 
> Ehm, that looks rather dangerous. What's the level of testing this patch
> received?
> 
Where is the danger here?

With the original code we would be issuing a warning
and return.
But then we hit this codepath:

 	while (!hlist_empty(&h->cmpQ)) {
 		c = hlist_entry(h->cmpQ.first, CommandList_struct, list);
 		removeQ(c);
		c->err_info->CommandStatus = CMD_HARDWARE_ERR;

and the driver goes boom as c->err_info is not initialized.

This frequently happens if you're trying to do a kdump
while the system is doing I/O.
If you object to the removed WARN() I can easily put this
in, but without the fix there is a good chance that
kdump fails on cciss machines.

And note we can't do anything with the stale commands anyway,
as the context having sent the commands originally is long gone.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare@suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Markus Rex, HRB 16746 (AG Nürnberg)

  reply	other threads:[~2009-07-02  8:45 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-07-02  8:23 [PATCH] cciss: Ignore stale commands after reboot Hannes Reinecke
2009-07-02  8:28 ` Jens Axboe
2009-07-02  8:44   ` Hannes Reinecke [this message]
2009-07-02  9:18     ` Jens Axboe
2009-07-02  9:36       ` Hannes Reinecke
2009-07-02 10:26         ` Jens Axboe
2009-07-02 10:28           ` Hannes Reinecke
2009-07-06 20:33 ` Alan D. Brunelle
2009-07-07  7:34   ` Hannes Reinecke
  -- strict thread matches above, loose matches on Subject: below --
2009-07-02  9:36 Hannes Reinecke
2009-07-02 19:00 ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4A4C7385.4030609@suse.de \
    --to=hare@suse.de \
    --cc=jens.axboe@oracle.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=scameron@beardog.cca.cpqcorp.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox