Re: cifs RDMA restrictions

[Tom Talpey <tom@talpey.com>]

On 4/1/2025 12:55 PM, David Howells wrote:
> Hi Tom,
> 
> I'm looking at cleaning up the "struct page" instances in smbdirect.c as much
> as possible.  Can you tell me what are the restrictions on the size of a
> segment attached to an SGE?  For instance, by:

I started typing a long answer but I ended up not sure of the question.

A single sge is just a hunk of memory that is registered with the RDMA
provider. The routine you're quoting is generating the dma_addr that
the provider (adapter) will use to access the data so it's passing what
the ib_dma stuff wants.

The sge *list* is more interesting. The first sge can have a non-zero
offset but anything that follows it can't. Such a "hole" would require
a subsequent registration, yielding handle(s) that might need multiple
RDMA operations. But it's the caller of the routine you quoted that
figured this out.

I'm going to be testing at SambaXP next week, by the way. If you have
some code, let me know.

Tom.


> 	static bool smb_set_sge(struct smb_extract_to_rdma *rdma,
> 				struct page *lowest_page, size_t off, size_t len)
> 	{
> 		struct ib_sge *sge = &rdma->sge[rdma->nr_sge];
> 		u64 addr;
> 
> 		addr = ib_dma_map_page(rdma->device, lowest_page,
> 				       off, len, rdma->direction);
> 		if (ib_dma_mapping_error(rdma->device, addr))
> 			return false;
> 
> 		sge->addr   = addr;
> 		sge->length = len;
> 		sge->lkey   = rdma->local_dma_lkey;
> 		rdma->nr_sge++;
> 		return true;
> 	}
> 
> Thanks,
> David
> 
>