From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Steve Wise" Subject: RE: NFSD generic R/W API (sendto path) performance results Date: Thu, 17 Nov 2016 14:03:41 -0600 Message-ID: <01f001d2410d$b16c97f0$1445c7d0$@opengridcomputing.com> References: <9170C872-DEE1-4D96-B9D8-E9D2B3F91915@oracle.com> <024601d23f7f$cef62500$6ce26f00$@opengridcomputing.com> <20161117124602.GA25821@lst.de> <84B43CFF-EBF7-4758-8751-8C97102C5BCF@oracle.com> <676323E9-2F30-4DB0-AEF8-CDE38E8A0715@oracle.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <676323E9-2F30-4DB0-AEF8-CDE38E8A0715-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> Content-Language: en-us Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: 'Chuck Lever' , 'Christoph Hellwig' Cc: 'Sagi Grimberg' , 'List Linux RDMA Mailing' List-Id: linux-rdma@vger.kernel.org > > > On Nov 17, 2016, at 10:04 AM, Chuck Lever wrote: > > > >> On Nov 17, 2016, at 7:46 AM, Christoph Hellwig wrote: > >> Also did you try to always register for > max_sge > >> calls? The code can already register all segments with the > >> rdma_rw_force_mr module option, so it would only need a small tweak for > >> that behavior. > > > > For various reasons I decided the design should build one WR chain for > > each RDMA segment provided by the client. Good clients expose just > > one RDMA segment for the whole NFS READ payload. > > > > Does force_mr make the generic API use FRWR with RDMA Write? I had > > assumed it changed only the behavior with RDMA Read. I'll try that > > too, if RDMA Write can easily be made to use FRWR. > > Unfortunately, some RPC replies are formed from two or three > discontiguous buffers. The gap test in ib_sg_to_pages returns > a smaller number than sg_nents in this case, and rdma_rw_init_ctx > fails. > > Thus with my current prototype I'm not able to test with FRWR. > > I could fix this in my prototype, but it would be nicer for me if > rdma_rw_init_ctx handled this case the same for FRWR as it does > for physical addressing, which doesn't seem to have any problem > with a discontiguous SGL. Just to make sure I'm understanding you, for rdma-rw to handle this, it would have to use multiple REG_MR registrations, one for each contiguous area in the scatter list. Right? -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html