From: hch@lst.de (Christoph Hellwig)
Subject: [PATCH RFC] nvme-rdma: support devices with queue size < 32
Date: Thu, 23 Mar 2017 15:00:42 +0100 [thread overview]
Message-ID: <20170323140042.GA30536@lst.de> (raw)
In-Reply-To: <1315914765.312051621.1490259849534.JavaMail.zimbra@kalray.eu>
On Thu, Mar 23, 2017@10:04:09AM +0100, Marta Rybczynska wrote:
> In the case of small NVMe-oF queue size (<32) we may enter
> a deadlock caused by the fact that the IB completions aren't sent
> waiting for 32 and the send queue will fill up.
>
> The error is seen as (using mlx5):
> [ 2048.693355] mlx5_0:mlx5_ib_post_send:3765:(pid 7273):
> [ 2048.693360] nvme nvme1: nvme_rdma_post_send failed with error code -12
>
> The patch doesn't change the behaviour for remote devices with
> larger queues.
Thanks, this looks useful. But wouldn't it be better to do something
like queue_size divided by 2 or 4 to get a better refill latency?
WARNING: multiple messages have this Message-ID (diff)
From: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
To: Marta Rybczynska <mrybczyn-FNhOzJFKnXGHXe+LvDLADg@public.gmane.org>
Cc: linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
keith.busch-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
axboe-b10kYP2dOMg@public.gmane.org,
hch-jcswGhMUV9g@public.gmane.org,
sagi-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org,
Samuel Jones
<samuel.jones-FNhOzJFKnXGHXe+LvDLADg@public.gmane.org>
Subject: Re: [PATCH RFC] nvme-rdma: support devices with queue size < 32
Date: Thu, 23 Mar 2017 15:00:42 +0100 [thread overview]
Message-ID: <20170323140042.GA30536@lst.de> (raw)
In-Reply-To: <1315914765.312051621.1490259849534.JavaMail.zimbra-FNhOzJFKnXGHXe+LvDLADg@public.gmane.org>
On Thu, Mar 23, 2017 at 10:04:09AM +0100, Marta Rybczynska wrote:
> In the case of small NVMe-oF queue size (<32) we may enter
> a deadlock caused by the fact that the IB completions aren't sent
> waiting for 32 and the send queue will fill up.
>
> The error is seen as (using mlx5):
> [ 2048.693355] mlx5_0:mlx5_ib_post_send:3765:(pid 7273):
> [ 2048.693360] nvme nvme1: nvme_rdma_post_send failed with error code -12
>
> The patch doesn't change the behaviour for remote devices with
> larger queues.
Thanks, this looks useful. But wouldn't it be better to do something
like queue_size divided by 2 or 4 to get a better refill latency?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2017-03-23 14:00 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-23 9:04 [PATCH RFC] nvme-rdma: support devices with queue size < 32 Marta Rybczynska
2017-03-23 9:04 ` Marta Rybczynska
2017-03-23 9:24 ` Marta Rybczynska
2017-03-23 14:00 ` Christoph Hellwig [this message]
2017-03-23 14:00 ` Christoph Hellwig
2017-03-23 14:36 ` Marta Rybczynska
2017-03-23 14:36 ` Marta Rybczynska
2017-03-28 11:09 ` Sagi Grimberg
2017-03-28 11:09 ` Sagi Grimberg
2017-03-28 11:20 ` Marta Rybczynska
2017-03-28 11:20 ` Marta Rybczynska
2017-03-28 11:30 ` Sagi Grimberg
2017-03-28 11:30 ` Sagi Grimberg
2017-03-29 9:36 ` Marta Rybczynska
2017-03-29 9:36 ` Marta Rybczynska
2017-03-29 13:29 ` Jason Gunthorpe
2017-03-29 13:29 ` Jason Gunthorpe
2017-03-29 15:47 ` Sagi Grimberg
2017-03-29 15:47 ` Sagi Grimberg
2017-03-29 16:27 ` Jason Gunthorpe
2017-03-29 16:27 ` Jason Gunthorpe
2017-03-29 16:39 ` Sagi Grimberg
2017-03-29 16:39 ` Sagi Grimberg
2017-03-29 16:44 ` Doug Ledford
2017-03-29 16:44 ` Doug Ledford
2017-03-29 16:47 ` Doug Ledford
2017-03-29 16:47 ` Doug Ledford
2017-03-29 16:59 ` Sagi Grimberg
2017-03-29 16:59 ` Sagi Grimberg
2017-03-29 22:19 ` Jason Gunthorpe
2017-03-29 22:19 ` Jason Gunthorpe
2017-03-30 14:23 ` Marta Rybczynska
2017-03-30 14:23 ` Marta Rybczynska
2017-04-06 12:29 ` Marta Rybczynska
2017-04-06 12:29 ` Marta Rybczynska
2017-04-06 13:02 ` Leon Romanovsky
2017-04-06 13:02 ` Leon Romanovsky
2017-04-07 13:31 ` Marta Rybczynska
2017-04-07 13:31 ` Marta Rybczynska
2017-04-09 12:31 ` Sagi Grimberg
2017-04-09 12:31 ` Sagi Grimberg
2017-04-10 11:29 ` Marta Rybczynska
2017-04-10 11:29 ` Marta Rybczynska
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170323140042.GA30536@lst.de \
--to=hch@lst.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.