From mboxrd@z Thu Jan 1 00:00:00 1970 From: Anna Schumaker Subject: Re: [PATCH v4 08/21] xprtrdma: Back off rkey when FAST_REG_MR fails Date: Thu, 24 Jul 2014 09:24:57 -0400 Message-ID: <53D10929.6010307@Netapp.com> References: <20140722144459.6010.99389.stgit@manet.1015granger.net> <20140722145208.6010.76678.stgit@manet.1015granger.net> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20140722145208.6010.76678.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org> Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Roland Dreier Cc: Chuck Lever , linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-Id: linux-rdma@vger.kernel.org Hi Roland, I'm collecting changes for NFS over RDMA, and I wasn't sure if the change to include/rdma/ib_verbs.h in this patch (adding ib_dec_rkey()) was something that could be merged through the NFS tree. What do you think? Thanks! Anna On 07/22/2014 10:52 AM, Chuck Lever wrote: > If posting a FAST_REG_MR Work Reqeust fails, revert the rkey update > to avoid subsequent IB_WC_MW_BIND_ERR completions. > > Suggested-by: Steve Wise > Signed-off-by: Chuck Lever > Tested-by: Shirley Ma > Tested-by: Devesh Sharma > --- > include/rdma/ib_verbs.h | 11 +++++++++++ > net/sunrpc/xprtrdma/verbs.c | 12 +++++------- > 2 files changed, 16 insertions(+), 7 deletions(-) > > diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h > index 7ccef34..c5c7ec6 100644 > --- a/include/rdma/ib_verbs.h > +++ b/include/rdma/ib_verbs.h > @@ -2479,6 +2479,17 @@ static inline u32 ib_inc_rkey(u32 rkey) > } > > /** > + * ib_dec_rkey - decrements the key portion of the given rkey. Can be used > + * when recovering from a local immediate error. > + * @rkey - the rkey to decrement. > + */ > +static inline u32 ib_dec_rkey(u32 rkey) > +{ > + const u32 mask = 0x000000ff; > + return ((rkey - 1) & mask) | (rkey & ~mask); > +} > + > +/** > * ib_alloc_mw - Allocates a memory window. > * @pd: The protection domain associated with the memory window. > * @type: The type of the memory window (1 or 2). > diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c > index 8bb7945..394b13f 100644 > --- a/net/sunrpc/xprtrdma/verbs.c > +++ b/net/sunrpc/xprtrdma/verbs.c > @@ -1497,13 +1497,12 @@ rpcrdma_register_frmr_external(struct rpcrdma_mr_seg *seg, > struct rpcrdma_frmr *frmr = &mw->r.frmr; > struct ib_mr *mr = frmr->fr_mr; > struct ib_send_wr invalidate_wr, frmr_wr, *bad_wr, *post_wr; > - > - u8 key; > int len, pageoff; > int i, rc; > int seg_len; > u64 pa; > int page_no; > + u32 rkey; > > pageoff = offset_in_page(seg1->mr_offset); > seg1->mr_offset -= pageoff; /* start of page */ > @@ -1558,14 +1557,11 @@ rpcrdma_register_frmr_external(struct rpcrdma_mr_seg *seg, > rc = -EIO; > goto out_err; > } > - > - /* Bump the key */ > - key = (u8)(mr->rkey & 0x000000FF); > - ib_update_fast_reg_key(mr, ++key); > - > frmr_wr.wr.fast_reg.access_flags = (writing ? > IB_ACCESS_REMOTE_WRITE | IB_ACCESS_LOCAL_WRITE : > IB_ACCESS_REMOTE_READ); > + rkey = ib_inc_rkey(mr->rkey); > + ib_update_fast_reg_key(mr, rkey); > frmr_wr.wr.fast_reg.rkey = mr->rkey; > DECR_CQCOUNT(&r_xprt->rx_ep); > > @@ -1574,6 +1570,8 @@ rpcrdma_register_frmr_external(struct rpcrdma_mr_seg *seg, > if (rc) { > dprintk("RPC: %s: failed ib_post_send for register," > " status %i\n", __func__, rc); > + rkey = ib_dec_rkey(mr->rkey); > + ib_update_fast_reg_key(mr, rkey); > goto out_err; > } else { > seg1->mr_rkey = mr->rkey; > -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html