Re: [PATCH v4 00/21] NFS/RDMA client patches for 3.17

[Anna Schumaker <Anna.Schumaker-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org>]

On 07/23/2014 05:23 PM, Chuck Lever wrote:
> 
> 
>> On Jul 23, 2014, at 2:53 PM, Anna Schumaker <Anna.Schumaker-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org> wrote:
>>
>> I've applied these changes and pushed them out in my nfs-rdma branch.  I'm planning to send a pull request to Trond on Friday so these changes can spend some time in linux-next before the next merge window opens.
> 
> Thank you! Did you resolve your question about who will merge the ib_dec_rkey patch, or do you want me to pursue that?

I don't have an answer yet (fingers crossed that it's not a problem!).  I think adding ib_dec_rkey() would need to go through Roland Dreier if it doesn't go through me.

Anna

> 
>>
>> Anna
>>
>>> On 07/22/2014 10:50 AM, Chuck Lever wrote:
>>> The main purpose of this series is to address connection drop
>>> recovery issues by fixing FRMR re-use to make it less likely the
>>> client will deadlock due to a memory management operation error.
>>> Some clean-ups and other fixes are present as well.
>>>
>>> Anna, v4 of this series should be ready for merging.
>>>
>>> See topic branch nfs-rdma-for-3.17 in
>>>
>>> git://git.linux-nfs.org/projects/cel/cel-2.6.git
>>>
>>> I tested with NFSv3 and NFSv4 on all three supported memory
>>> registration modes. Used cthon04, iozone, and dbench with both
>>> Solaris and Linux NFS/RDMA servers. Used xfstests with Linux.
>>>
>>> v4:
>>>
>>> - Rebased on v3.16-rc6
>>>
>>> - Add Tested-by: from Shirley Ma and Devesh Sharma
>>>
>>>
>>> v3:
>>> Only two substantive changes:
>>>
>>> - Patch 08/21 now uses generic IB helpers for managing FRMR
>>> rkeys
>>>
>>> - Add Tested-by: from Steve Wise
>>>
>>>
>>> v2:
>>> Many patches from v1 have been written or replaced.
>>>
>>> The MW ref counting approach in v1 is abandoned. Instead, I've
>>> eliminated signaling FAST_REG_MR and LOCAL_INV, and added
>>> appropriate recovery mechanisms after a transport reconnect that
>>> should prevent rkey dis-synchrony entirely.
>>>
>>> A couple of optimizations have been added, including:
>>>
>>> - Allocating each MW separately rather than carving each out of a
>>> large piece of contiguous memory
>>>
>>> - Now that the receive CQ upcall handler dequeues a bundle of CQEs
>>> at once, fire off the reply handler tasklet just once per upcall
>>> to reduce context switches and how often hard IRQs are disabled
>>>
>>> ---
>>>
>>> Chuck Lever (21):
>>>      xprtrdma: Fix panic in rpcrdma_register_frmr_external()
>>>      xprtrdma: Protect ia->ri_id when unmapping/invalidating MRs
>>>      xprtrdma: Limit data payload size for ALLPHYSICAL
>>>      xprtrdma: Update rkeys after transport reconnect
>>>      xprtrdma: On disconnect, don't ignore pending CQEs
>>>      xprtrdma: Don't invalidate FRMRs if registration fails
>>>      xprtrdma: Unclutter struct rpcrdma_mr_seg
>>>      xprtrdma: Back off rkey when FAST_REG_MR fails
>>>      xprtrdma: Chain together all MWs in same buffer pool
>>>      xprtrdma: Properly handle exhaustion of the rb_mws list
>>>      xprtrdma: Reset FRMRs when FAST_REG_MR is flushed by a disconnect
>>>      xprtrdma: Reset FRMRs after a flushed LOCAL_INV Work Request
>>>      xprtrdma: Don't post a LOCAL_INV in rpcrdma_register_frmr_external()
>>>      xprtrdma: Disable completions for FAST_REG_MR Work Requests
>>>      xprtrdma: Disable completions for LOCAL_INV Work Requests
>>>      xprtrdma: Rename frmr_wr
>>>      xprtrdma: Allocate each struct rpcrdma_mw separately
>>>      xprtrdma: Schedule reply tasklet once per upcall
>>>      xprtrdma: Make rpcrdma_ep_disconnect() return void
>>>      xprtrdma: Remove RPCRDMA_PERSISTENT_REGISTRATION macro
>>>      xprtrdma: Handle additional connection events
>>>
>>>
>>> include/linux/sunrpc/xprtrdma.h |    2 
>>> include/rdma/ib_verbs.h         |   11 +
>>> net/sunrpc/xprtrdma/rpc_rdma.c  |   83 ++--
>>> net/sunrpc/xprtrdma/transport.c |   17 +
>>> net/sunrpc/xprtrdma/verbs.c     |  744 ++++++++++++++++++++++++++-------------
>>> net/sunrpc/xprtrdma/xprt_rdma.h |   61 +++
>>> 6 files changed, 615 insertions(+), 303 deletions(-)
>>>
>>> --
>>> Chuck Lever
>>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html