From: Mike Snitzer <snitzer@redhat.com>
To: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Hannes Reinecke <hare@suse.de>,
"Jun'ichi Nomura" <j-nomura@ce.jp.nec.com>,
James Bottomley <James.Bottomley@HansenPartnership.com>,
Steffen Maier <maier@linux.vnet.ibm.com>,
"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
Jens Axboe <axboe@kernel.dk>,
Linux Kernel <linux-kernel@vger.kernel.org>,
Alan Stern <stern@rowland.harvard.edu>,
Thadeu Lima de Souza Cascardo <cascardo@linux.vnet.ibm.com>,
"Taraka R. Bodireddy" <tarak.reddy@in.ibm.com>,
"Seshagiri N. Ippili" <seshagiri.ippili@in.ibm.com>,
"Manvanthara B. Puttashankar" <mputtash@in.ibm.com>,
Jeff Moyer <jmoyer@redhat.com>, Shaohua Li <shaohua.li@intel.com>,
gmuelas@de.ibm.com, dm-devel@redhat.com
Subject: Re: [GIT PULL] Queue free fix (was Re: [PATCH] block: Free queue resources at blk_release_queue())
Date: Tue, 29 Nov 2011 15:18:03 -0500 [thread overview]
Message-ID: <20111129201803.GB6827@redhat.com> (raw)
In-Reply-To: <20111129120047.GA2456@osiris.boeblingen.de.ibm.com>
On Tue, Nov 29 2011 at 7:00am -0500,
Heiko Carstens <heiko.carstens@de.ibm.com> wrote:
> > > > Hmm. Just to be on the safe side, could you try this one:
> > > >
> > > > diff --git a/drivers/md/dm-mpath.c b/drivers/md/dm-mpath.c
> > > > index 5e0090e..e6fad46 100644
> > > > --- a/drivers/md/dm-mpath.c
> > > > +++ b/drivers/md/dm-mpath.c
> > > > @@ -920,8 +920,10 @@ static int multipath_map(struct dm_target *ti,
> > > > struct reque
> > > > st *clone,
> > > > map_context->ptr = mpio;
> > > > clone->cmd_flags |= REQ_FAILFAST_TRANSPORT;
> > > > r = map_io(m, clone, mpio, 0);
> > > > - if (r < 0 || r == DM_MAPIO_REQUEUE)
> > > > + if (r < 0 || r == DM_MAPIO_REQUEUE) {
> > > > mempool_free(mpio, m->mpio_pool);
> > > > + map_context->ptr = NULL;
> > > > + }
> > > >
> > > > return r;
> > > > }
> > >
> > > With your patch we haven't been able to reproduce the kernel crash until now.
> > > Now we "only" run into I/O stalls, which before your patch we also did. But
> > > repeatedly rebooting and retrying and ignoring the I/O stalls always lead to
> > > a crash.
> > > Gonzalo will run a couple of extra rounds so we can have a feeling if at least
> > > one of the bugs could be fixed with your patch ;)
> >
> > Hi,
> >
> > Any update after further testing with Hannes' patch?
>
> Sorry for the late update, our internal IBM IMAP servers have been down
> for nearly a week :/
>
> So, we were unable to reproduce the original bug with the patch applied
> during various runs.
OK, so it seems to be a benenficial change (and obviously correct to
me). Hannes, care to formally post your fix to dm-devel so we can get
it in 3.2-rc?
> However, we ran into this one instead, which is yet another use-after-free bug
> (I need to double check, but I'm quite sure that a freed struct scsi_cmnd
> caused this).
OK, yeah something is causing poisoned (POISON_FREE) memory to be used.
> [ 4906.683654] Unable to handle kernel pointer dereference at virtual kernel address 6b6b6b6b6b6b6000
...
> Gonzalo also tried 2.6.38.8 as suggested and ran into this one:
>
> [ 292.877936] ------------[ cut here ]------------
> [ 292.877939] Kernel BUG at 6b6b6b6b6b6b6b6d [verbose debug info unavailable]
Again, more poison.
Seems this test is causing us to fall on our face no matter what.
Likely, best to leave this 2.6.38 blk_unplug crash to one side and
continue focusing on latest upstream.
Mike
next prev parent reply other threads:[~2011-11-29 20:18 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <201109221318.p8MDIXPP020015@pentland.suse.de>
2011-09-28 0:47 ` [PATCH] block: Free queue resources at blk_release_queue() Jens Axboe
2011-09-28 0:55 ` Linus Torvalds
2011-09-28 1:15 ` Jens Axboe
2011-09-28 1:59 ` Linus Torvalds
2011-09-28 2:02 ` Jens Axboe
2011-09-28 4:10 ` James Bottomley
2011-09-28 14:08 ` Jens Axboe
2011-09-28 14:11 ` James Bottomley
2011-09-28 14:14 ` [GIT PULL] Queue free fix (was Re: [PATCH] block: Free queue resources at blk_release_queue()) Jens Axboe
2011-09-28 15:22 ` Linus Torvalds
2011-09-28 15:43 ` James Bottomley
2011-09-28 17:48 ` Vivek Goyal
2011-09-28 17:53 ` Christoph Hellwig
2011-09-28 18:09 ` Vivek Goyal
2011-09-28 18:16 ` Christoph Hellwig
2011-09-28 19:05 ` Eric Seppanen
2011-09-28 19:14 ` Christoph Hellwig
2011-11-30 10:18 ` Jens Axboe
2011-11-30 10:26 ` Christoph Hellwig
2011-09-28 22:34 ` Vivek Goyal
2011-09-28 17:59 ` James Bottomley
2011-10-13 13:09 ` Steffen Maier
2011-10-14 16:03 ` James Bottomley
2011-10-17 8:46 ` Jun'ichi Nomura
2011-10-17 14:06 ` James Bottomley
2011-10-18 13:31 ` Jun'ichi Nomura
2011-10-18 15:45 ` Heiko Carstens
2011-10-18 16:29 ` James Bottomley
2011-10-31 10:05 ` Heiko Carstens
2011-10-31 10:42 ` James Bottomley
2011-10-31 11:46 ` Jun'ichi Nomura
2011-10-31 13:00 ` Heiko Carstens
2011-11-02 12:37 ` Jun'ichi Nomura
2011-11-02 12:44 ` Hannes Reinecke
2011-11-02 13:47 ` Heiko Carstens
2011-11-04 4:07 ` Jun'ichi Nomura
2011-11-04 9:12 ` Heiko Carstens
2011-11-03 18:25 ` Mike Snitzer
2011-11-04 9:19 ` Heiko Carstens
2011-11-04 13:30 ` Mike Snitzer
2011-11-04 13:37 ` Hannes Reinecke
2011-11-07 11:31 ` Jun'ichi Nomura
2011-11-07 13:42 ` Mike Snitzer
2011-11-07 12:23 ` Heiko Carstens
2011-11-07 11:30 ` Jun'ichi Nomura
2011-11-07 15:36 ` Mike Snitzer
2011-11-07 16:43 ` Heiko Carstens
2011-11-07 17:10 ` Mike Snitzer
2011-11-07 21:44 ` Mike Snitzer
2011-11-09 9:37 ` Hannes Reinecke
2011-11-10 16:10 ` Heiko Carstens
2011-11-17 16:29 ` Mike Snitzer
2011-11-29 12:00 ` Heiko Carstens
2011-11-29 20:18 ` Mike Snitzer [this message]
2011-11-30 7:25 ` Hannes Reinecke
2011-12-12 12:39 ` Heiko Carstens
2011-12-13 16:50 ` Mike Snitzer
2011-10-31 13:21 ` Mike Snitzer
2011-10-31 13:40 ` Heiko Carstens
2011-10-31 14:01 ` Mike Snitzer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20111129201803.GB6827@redhat.com \
--to=snitzer@redhat.com \
--cc=James.Bottomley@HansenPartnership.com \
--cc=axboe@kernel.dk \
--cc=cascardo@linux.vnet.ibm.com \
--cc=dm-devel@redhat.com \
--cc=gmuelas@de.ibm.com \
--cc=hare@suse.de \
--cc=heiko.carstens@de.ibm.com \
--cc=j-nomura@ce.jp.nec.com \
--cc=jmoyer@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=maier@linux.vnet.ibm.com \
--cc=mputtash@in.ibm.com \
--cc=seshagiri.ippili@in.ibm.com \
--cc=shaohua.li@intel.com \
--cc=stern@rowland.harvard.edu \
--cc=tarak.reddy@in.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox