Re: [PATCH RFC 0/9] Exploring biovec support in (R)DMA API

[Chuck Lever <chuck.lever@oracle.com>]

On Thu, Oct 19, 2023 at 04:53:43PM +0100, Matthew Wilcox wrote:
> On Thu, Oct 19, 2023 at 11:25:31AM -0400, Chuck Lever wrote:
> > The SunRPC stack manages pages (and eventually, folios) via an
> > array of struct biovec items within struct xdr_buf. We have not
> > fully committed to replacing the struct page array in xdr_buf
> > because, although the socket API supports biovec arrays, the RDMA
> > stack uses struct scatterlist rather than struct biovec.
> > 
> > This (incomplete) series explores what it might look like if the
> > RDMA core API could support struct biovec array arguments. The
> > series compiles on x86, but I haven't tested it further. I'm posting
> > early in hopes of starting further discussion.
> 
> Good call, because I think patch 2/9 is a complete non-starter.
> 
> The fundamental problem with scatterlist is that it is both input
> and output for the mapping operation.  You're replicating this mistake
> in a different data structure.

Fwiw, I'm not at all wedded to the "copy-and-paste SGL" approach.


> My vision for the future is that we have phyr as our input structure.
> That looks something like:
> 
> struct phyr {
> 	phys_addr_t start;
> 	size_t len;
> };
> 
> On 32-bit, that's 8 or 12 bytes; on 64-bit it's 16 bytes.  This is
> better than biovec because biovec is sometimes larger than that, and
> it allows specifying IO to memory that does not have a struct page.

Passing a folio rather than a page is indeed one of the benefits we
would like to gain for SunRPC.


> Our output structure can continue being called the scatterlist, but
> it needs to go on a diet and look more like:
> 
> struct scatterlist {
> 	dma_addr_t dma_address;
> 	size_t dma_length;
> };
> 
> Getting to this point is going to be a huge amount of work, and I need
> to finish folios first.  Or somebody else can work on it ;-)

I would like to see forward progress, as SunRPC has some skin in
this game. I'm happy to contribute code or review.

If there is some consensus on your proposed approach, I can start
with that.

-- 
Chuck Lever