Re: Socket Direct Protocol: help (2)

Steve Wise
2010-04-14T15:08:15Z

Amir Vadai wrote:
> You are right - I missed it.
>
> Andrea, Please open a bug at bugzilla (https://bugs.openfabrics.org) -
> so that you will be notified as soon as I will fix SDP not use FMR if
> not supported.
>
> As to fastreg_mrs support - I don't know this mechanism. Do you mean FRWR?
>
>

ib_alloc_fast_reg_mr(), ib_alloc_fast_reg_page_list() and friends, plus
the IB_WR_FAST_REG_MR work request.

> Thanks,
> Amir
>
>
> On 04/14/2010 05:54 PM, Steve Wise wrote:
>
>> Hey Amir,
>>
>> I don't think this helps because sdp_add_device() will not add rdma
>> devices that fail to create fmr pools.
>>
>> So I guess you could key off of fmr pool failures and set
>> sdp_zcopy_thresh to 0 and allow the device to be used?
>>
>> But what we really need is sdp support for fastreg_mrs as an
>> alternative to fmrs.
>>
>>
>> Steve.
>>
>>
>> Amir Vadai wrote:
>>
>>
>>> Hi,
>>>
>>> FMR are being used only in a special mode called ZCopy.
>>>
>>> You could disable this mode by setting the module paramter
>>> sdp_zcopy_thresh to 0, or by issuing:
>>> # echo 0 > /sys/module/ib_sdp/parameters/sdp_zcopy_thresh
>>>
>>> This means that you won't get the benefits of Zero-copy.
>>>
>>> - Amir
>>>
>>> On 04/14/2010 11:51 AM, Andrea Gozzelino wrote:
>>>
>>>
>>>
>>>> On Apr 13, 2010 10:22 PM, "Tung, Chien Tin" <chien.tin.tung-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>>>> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>>>> Chien, does the NE020 support FMRs? I looked at the nes ofed-1.5
>>>>>>>> code
>>>>>>>> and it appears to do nothing in the map_phys_fmr functions.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> We never implemented map_phys_fmr. Is it relevant to the # of SGEs?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> No, but SDP uses FMRs. I don't think it will run without FMR support.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> Good to know. Thanks.
>>>>>
>>>>> Chien
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> Hi Steve and Chien,
>>>>
>>>> I understand that NE020 cards have problem with SDP connected with
>>>> map_phy_fmr (FMR stands for Fast Memory Region).
>>>> Is it possible to solve/fix this point?
>>>> If yes, have you an idea about the time that is necessary to code
>>>> development/build?
>>>> If no, can you suggest me a card that supports SDP protocol?
>>>>
>>>> I work on NE020 cards from February 2010 for an INFN experimental
>>>> proposal, called REDIGO (Read out at 10 Gbits/s), about the data
>>>> acquisition and movement systems. The covergence of storage protocols
>>>> around 10 Gigabits/s Ethernet protocols shows that one way could be the
>>>> Remote Direct Memory Access (RDMA). The goals are the investigations of
>>>> latency time, the throughput, the buffer size schemes and finally the
>>>> global event building bandwidth.
>>>>
>>>> Do you know if NE020 cards have problems with librdma (RDMA procedures,
>>>> in general) and / or with MPI versions?
>>>>
>>>> Thank you very much,
>>>> Andrea
>>>>
>>>> Andrea Gozzelino
>>>>
>>>> INFN - Laboratori Nazionali di Legnaro (LNL)
>>>> Viale dell'Universita' 2
>>>> I-35020 - Legnaro (PD)- ITALIA
>>>> Tel: +39 049 8068346
>>>> Fax: +39 049 641925
>>>> Mail: andrea.gozzelino-PK20h7lG/Rc1GQ1Ptb7lUw@public.gmane.org
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>>>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>
>>>>
>>>>
>>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>> Please read the FAQ at http://www.tux.org/lkml/
>>>
>>>
>>>