Re: linux rdma 3.14 merge plans

Or Gerlitz
2014-01-30T10:38:54Z

On 30/01/2014 12:07, Bart Van Assche wrote:
> On 01/30/14 09:19, Or Gerlitz wrote:
>> Thanks for narrowing this down, I see your point, however the solution I
>> propose if to remove this copy altogether... for those rare cases where
>> fast-registration can't be done -- in SRP I think the code goes to
>> indirect mode under which there no copy (right?). As for iSER, we will
>> add to our TODO enhancing the code to avoid using bounce-buffer, few
>> designs might be possible, one way would be to create set of
>> FMRS/Fast-reg element which are capable to map chunks which are <
>> PAGE_SIZE, e.g in the order of single block, and hence can map what ever
>> arbitrary set of blocks provided by the upper layer.
> I'm not sure how important this issue is on x86 systems since the page
> size on these systems is 4 KB and since many applications use an I/O
> size >= 4 KB. So if the I/O buffers are aligned on page boundaries the
> buffer memory can be registered as a single memory region without having
> to copy any data. However, on PPC systems, which have a page size of 64
> KB, if e.g. a database issues a readv() or writev() call or if an I/O
> scheduler coalesces several small I/O requests into a single I/O request
> the data buffer of these I/O requests is discontiguous. I think it is
> important that data copying can be avoided for such I/O requests.

We know that... and hence iser uses fast registration in chucks of 4KB
even when the page size is 64KB