From: Jens Axboe <axboe@kernel.dk>
To: Parav Pandit <parav@mellanox.com>
Cc: "axboe@fb.com" <axboe@fb.com>,
"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
Sagi Grimberg <sagi@grimberg.me>, Christoph Hellwig <hch@lst.de>,
Max Gurtovoy <maxg@mellanox.com>
Subject: Re: connect cmd error for nvme-rdma with eventual kernel crash
Date: Tue, 28 Feb 2017 18:26:32 -0700 [thread overview]
Message-ID: <A520FEF3-9BB7-4265-ADBA-29ADE118B971@kernel.dk> (raw)
In-Reply-To: <VI1PR0502MB3008851A3D20ECF93FC70FCCD1290@VI1PR0502MB3008.eurprd05.prod.outlook.com>
> On Feb 28, 2017, at 5:57 PM, Parav Pandit <parav@mellanox.com> wrote:
>=20
> Hi Jens,
>=20
> With your commit 2af8cbe30531eca73c8f3ba277f155fc0020b01a in linux-block g=
it tree,
> There are two requests tables. Static and dynamic of same size.
> However function blk_mq_tag_to_rq() always try to get the tag from the dyn=
amic table which doesn't seem to be always initialized.
>=20
> I am running nvme-rdma initiator and it fails to find the request for the g=
iven tag when command completes.
> Command triggers error recovery with "tag not found" error.
> Eventually kernel is crashing in blk_mq_queue_tag_busy_iter() with NULL po=
inter. Seems to be additional bug in error recovery.
>=20
> To debug, I added initializing dynamic tags as well.
>=20
> blk_mq_alloc_rqs() {
> tags->static_rqs[i] =3D rq;
> + tags->rqs[i] =3D rq;
>=20
> This appears to resolve the issue. But that's not the fix.
> It appears to me that nvme stack is broken in certain conditions with rece=
nt static and dynamic rq tables change.
Can you try my for-linus branch?
next prev parent reply other threads:[~2017-03-01 1:26 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-01 0:57 connect cmd error for nvme-rdma with eventual kernel crash Parav Pandit
2017-03-01 1:26 ` Jens Axboe [this message]
2017-03-01 4:55 ` Parav Pandit
2017-03-01 5:50 ` Omar Sandoval
2017-03-01 15:06 ` Parav Pandit
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=A520FEF3-9BB7-4265-ADBA-29ADE118B971@kernel.dk \
--to=axboe@kernel.dk \
--cc=axboe@fb.com \
--cc=hch@lst.de \
--cc=linux-block@vger.kernel.org \
--cc=maxg@mellanox.com \
--cc=parav@mellanox.com \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox