From: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
To: jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org,
sagi-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: [PATCH RFC 0/5] xprtrdma Send completion batching
Date: Tue, 05 Sep 2017 13:00:10 -0400 [thread overview]
Message-ID: <20170905164347.11106.27140.stgit@manet.1015granger.net> (raw)
Hi Jason, Sagi-
As we discussed a few weeks ago, this patch series implements the
following:
- Send SGEs are now managed via lock-less, wait-free circular queues
- Send SGEs referring to page cache pages are DMA unmapped during
Send completion
- Send completions are batched to reduce interrupts, but still
provide a periodic heartbeat signal for SQ housekeeping
- The circular queue prevents Send Queue overflow
The purpose of this change is to address the issue Sagi reported
where the HCA continues to retry a delayed Send request _after_ RPC
completion, resulting in a DMA error.
Also available as the first five commits in this topic branch:
http://git.linux-nfs.org/?p=cel/cel-2.6.git;a=shortlog;h=refs/heads/nfs-rdma-for-4.15
---
Chuck Lever (5):
xprtrdma: Clean up SGE accounting in rpcrdma_prepare_msg_sges()
xprtrdma: Change return value of rpcrdma_prepare_send_sges()
xprtrdma: Add data structure to manage RDMA Send arguments
xprtrdma: Manage RDMA Send arguments via lock-free circular queue
xprtrdma: Remove atomic send completion counting
net/sunrpc/xprtrdma/backchannel.c | 6 +
net/sunrpc/xprtrdma/frwr_ops.c | 8 -
net/sunrpc/xprtrdma/rpc_rdma.c | 104 +++++++++++-------
net/sunrpc/xprtrdma/transport.c | 6 +
net/sunrpc/xprtrdma/verbs.c | 210 +++++++++++++++++++++++++++++++++++--
net/sunrpc/xprtrdma/xprt_rdma.h | 66 +++++++-----
6 files changed, 308 insertions(+), 92 deletions(-)
--
Chuck Lever
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next reply other threads:[~2017-09-05 17:00 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-09-05 17:00 Chuck Lever [this message]
[not found] ` <20170905164347.11106.27140.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-09-05 17:00 ` [PATCH RFC 1/5] xprtrdma: Clean up SGE accounting in rpcrdma_prepare_msg_sges() Chuck Lever
2017-09-05 17:00 ` [PATCH RFC 2/5] xprtrdma: Change return value of rpcrdma_prepare_send_sges() Chuck Lever
2017-09-05 17:00 ` [PATCH RFC 3/5] xprtrdma: Add data structure to manage RDMA Send arguments Chuck Lever
2017-09-05 17:00 ` [PATCH RFC 4/5] xprtrdma: Manage RDMA Send arguments via lock-free circular queue Chuck Lever
[not found] ` <20170905170046.11106.41992.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-09-05 21:50 ` Chuck Lever
2017-09-05 17:00 ` [PATCH RFC 5/5] xprtrdma: Remove atomic send completion counting Chuck Lever
2017-09-05 20:06 ` [PATCH RFC 0/5] xprtrdma Send completion batching Jason Gunthorpe
[not found] ` <20170905200608.GA4055-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2017-09-05 21:22 ` Chuck Lever
[not found] ` <0D960F54-0BC7-4A5A-8001-0391673892DF-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-09-05 22:03 ` Jason Gunthorpe
[not found] ` <20170905220325.GA10835-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2017-09-06 14:17 ` Chuck Lever
2017-09-06 1:28 ` Tom Talpey
2017-09-06 11:54 ` Sagi Grimberg
[not found] ` <1230f9d9-07c1-6d00-b197-f408712fb5c1-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-09-06 14:15 ` Chuck Lever
[not found] ` <890CC58C-7F8F-4B7E-8620-21F07007D3AA-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-09-06 14:29 ` Sagi Grimberg
[not found] ` <6dcdcc25-2613-cdb5-1db2-6c944f05242b-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-09-06 15:11 ` Chuck Lever
[not found] ` <4E2E5580-69A5-4C3B-9FCA-E61AE2042E6B-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-09-06 15:23 ` Sagi Grimberg
[not found] ` <9059315f-1985-042e-a59f-26a66fbece3e-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-09-06 18:33 ` Chuck Lever
[not found] ` <5B2F42B8-2CBD-43F4-BBAD-71EDD4F871FB-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-09-06 19:39 ` Jason Gunthorpe
[not found] ` <20170906193946.GC18461-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2017-09-06 20:02 ` Chuck Lever
[not found] ` <4D7ECF64-0F68-4137-B72A-9E942147C8D4-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-09-06 20:09 ` Jason Gunthorpe
[not found] ` <20170906200957.GB22567-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2017-09-06 21:00 ` Chuck Lever
[not found] ` <C5144079-7F9C-4783-AF4D-A73411420D5E-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-09-06 21:11 ` Jason Gunthorpe
2017-09-07 13:17 ` Tom Talpey
[not found] ` <f2cb453d-a59f-de64-7a42-09e075562895-CLs1Zie5N5HQT0dZR+AlfA@public.gmane.org>
2017-09-07 15:08 ` Jason Gunthorpe
[not found] ` <20170907150829.GA20644-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2017-09-07 16:15 ` Tom Talpey
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170905164347.11106.27140.stgit@manet.1015granger.net \
--to=chuck.lever-qhclzuegtsvqt0dzr+alfa@public.gmane.org \
--cc=jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org \
--cc=linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=sagi-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox