Re: [PATCH v1 00/24] New fast registration API

[Sagi Grimberg <sagig@dev.mellanox.co.il>]

On 10/7/2015 6:46 PM, Bart Van Assche wrote:
> On 10/06/2015 11:42 PM, Sagi Grimberg wrote:
>> On 10/6/2015 9:49 PM, Bart Van Assche wrote:
>>> On 10/06/2015 01:37 AM, Sagi Grimberg wrote:
>>>> I see now the error you are referring to.
>>>>
>>>> The issue is that the device requires the MR page array to have
>>>> an alignment (0x40 for mlx4 and 0x400 for mlx5). When I modified the
>>>> page array allocation to be non-coherent I didn't take care of
>>>> alignment.
>>>>
>>>> Taking care of this alignment may result in a higher order allocation
>>>> as we'd need to add (alignment - 1) to the allocation size.
>>>>
>>>> e.g. a 512 pages on mlx4 will become:
>>>> 512 * 8 + 0x40 - 1 = 4159
>>>>
>>>> I'm leaning towards this approach. Any preference?
>>>>
>>>> I think this patch should take care of mlx4:
>>>> [ ... ]
>>>
>>> Hello Sagi,
>>>
>>> Thanks for the patch. But since the patch included in the previous
>>> e-mail mapped a memory range that could be outside the bounds of the
>>> allocated memory I have been testing the patch below:
>>
>> Thanks! I correct the patches.
>>
>> Can I take it as your Tested-by on srp?
>
> Sure :-) But please keep in mind that I currently only have access to
> ConnectX-3 HCA's for testing RDMA software and not to any other RDMA HCA
> model.

Thanks Bart. For what its worth, I've tested srp (and iser + nfs) on
both CX3 and CX4 with your config file.

Cheers,
Sagi.