From: Sagi Grimberg <sagi-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
To: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: Re: [PATCH v1 03/14] svcrdma: Eliminate RPCRDMA_SQ_DEPTH_MULT
Date: Tue, 21 Mar 2017 19:58:22 +0200 [thread overview]
Message-ID: <46eb6195-a542-b35c-4902-a2bebb38feba@grimberg.me> (raw)
In-Reply-To: <20170316155250.4482.49638.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
> The Send Queue depth is temporarily reduced to 1 SQE per credit. The
> new rdma_rw API does an internal computation, during QP creation, to
> increase the depth of the Send Queue to handle RDMA Read and Write
> operations.
>
> This change has to come before the NFSD code paths are updated to
> use the rdma_rw API. Without this patch, rdma_rw_init_qp() increases
> the size of the SQ too much, resulting in memory allocation failures
> during QP creation.
I agree this needs to happen, but turns out you don't have any
guarantees of the maximum size of the sq depending on your max_sge
parameter. I'd recommend having a fall-back shrinked size sq allocation
impllemented like srpt does.
We don't have it in nvmet-rdma nor iser, but its a good thing to have...
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2017-03-21 17:58 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-16 15:52 [PATCH v1 00/14] Server-side NFS/RDMA changes for v4.12 Chuck Lever
[not found] ` <20170316154132.4482.56769.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-16 15:52 ` [PATCH v1 01/14] svcrdma: Move send_wr to svc_rdma_op_ctxt Chuck Lever
[not found] ` <20170316155234.4482.94225.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-21 17:49 ` Sagi Grimberg
2017-03-16 15:52 ` [PATCH v1 02/14] svcrdma: Add svc_rdma_map_reply_hdr() Chuck Lever
[not found] ` <20170316155242.4482.64809.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-21 17:54 ` Sagi Grimberg
[not found] ` <f5000e25-6ca1-fc24-35c0-6089cf50923c-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-03-21 18:40 ` Chuck Lever
[not found] ` <A18F9D5E-09BA-4268-9AA6-3E5866101F76-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-03-22 13:07 ` Sagi Grimberg
2017-03-16 15:52 ` [PATCH v1 03/14] svcrdma: Eliminate RPCRDMA_SQ_DEPTH_MULT Chuck Lever
[not found] ` <20170316155250.4482.49638.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-21 17:58 ` Sagi Grimberg [this message]
[not found] ` <46eb6195-a542-b35c-4902-a2bebb38feba-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-03-21 18:44 ` Chuck Lever
[not found] ` <391F0D90-2A46-4B2F-BCF0-B3BE7D48A3EF-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-03-22 13:09 ` Sagi Grimberg
[not found] ` <ec82feb4-d6b9-7fb4-5b11-b8007e313845-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-03-22 13:36 ` Chuck Lever
[not found] ` <C9D8A91C-DE08-4A41-A07D-1F4C42DD9B97-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-03-22 19:06 ` Sagi Grimberg
[not found] ` <68e3eda0-90f4-5bca-28be-b2cf494ed172-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-03-22 19:30 ` Chuck Lever
2017-03-16 15:52 ` [PATCH v1 04/14] svcrdma: Add helper to save pages under I/O Chuck Lever
[not found] ` <20170316155258.4482.69182.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-21 18:01 ` Sagi Grimberg
2017-03-16 15:53 ` [PATCH v1 05/14] svcrdma: Introduce local rdma_rw API helpers Chuck Lever
[not found] ` <20170316155306.4482.68041.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-22 14:17 ` Sagi Grimberg
[not found] ` <cfa49433-ab26-d2f0-27d4-2a96ff0adaba-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-03-22 15:41 ` Chuck Lever
[not found] ` <1CAD2542-A121-47ED-A47C-624E188EB54F-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-03-24 22:19 ` Chuck Lever
2017-03-16 15:53 ` [PATCH v1 06/14] svcrdma: Use rdma_rw API in RPC reply path Chuck Lever
2017-03-16 15:53 ` [PATCH v1 07/14] svcrdma: Clean up RDMA_ERROR path Chuck Lever
[not found] ` <20170316155323.4482.8051.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-22 14:18 ` Sagi Grimberg
2017-03-16 15:53 ` [PATCH v1 08/14] svcrdma: Report Write/Reply chunk overruns Chuck Lever
[not found] ` <20170316155331.4482.7734.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-22 14:20 ` Sagi Grimberg
2017-03-16 15:53 ` [PATCH v1 09/14] svcrdma: Clean up RPC-over-RDMA backchannel reply processing Chuck Lever
2017-03-16 15:53 ` [PATCH v1 10/14] svcrdma: Reduce size of sge array in struct svc_rdma_op_ctxt Chuck Lever
[not found] ` <20170316155347.4482.74652.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-22 14:21 ` Sagi Grimberg
2017-03-16 15:53 ` [PATCH v1 11/14] svcrdma: Remove old RDMA Write completion handlers Chuck Lever
[not found] ` <20170316155355.4482.35026.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-22 14:22 ` Sagi Grimberg
2017-03-16 15:54 ` [PATCH v1 12/14] svcrdma: Remove the req_map cache Chuck Lever
[not found] ` <20170316155403.4482.2040.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-22 14:22 ` Sagi Grimberg
2017-03-16 15:54 ` [PATCH v1 13/14] svcrdma: Clean out old XDR encoders Chuck Lever
[not found] ` <20170316155411.4482.37224.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-22 14:23 ` Sagi Grimberg
2017-03-16 15:54 ` [PATCH v1 14/14] svcrdma: Clean up svc_rdma_post_recv() error handling Chuck Lever
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=46eb6195-a542-b35c-4902-a2bebb38feba@grimberg.me \
--to=sagi-nqwnxtmzq1alnmji0ikvqw@public.gmane.org \
--cc=chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org \
--cc=linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).