Re: Socket Direct Protocol: help (2)

[Amir Vadai <amirv@mellanox.co.il>]

ok - actually I used it in an early version of SDP before changing to FMR...

- Amir

On 04/14/2010 06:08 PM, Steve Wise wrote:
> Amir Vadai wrote:
>   
>> You are right - I missed it.
>>
>> Andrea, Please open a bug at bugzilla (https://bugs.openfabrics.org) -
>> so that you will be notified as soon as I will fix SDP not use FMR if
>> not supported.
>>
>> As to fastreg_mrs support - I don't know this mechanism. Do you mean FRWR?
>>
>>   
>>     
> ib_alloc_fast_reg_mr(), ib_alloc_fast_reg_page_list() and friends, plus 
> the IB_WR_FAST_REG_MR work request.
>
>
>   
>> Thanks,
>> Amir
>>
>>
>> On 04/14/2010 05:54 PM, Steve Wise wrote:
>>   
>>     
>>> Hey Amir,
>>>
>>> I don't think this helps because sdp_add_device() will not add rdma 
>>> devices that fail to create fmr pools. 
>>>
>>> So I guess you could key off of fmr pool failures and set 
>>> sdp_zcopy_thresh to 0 and allow the device to be used?
>>>
>>> But what we really need is sdp support for fastreg_mrs  as an 
>>> alternative to fmrs.
>>>
>>>
>>> Steve.
>>>
>>>
>>> Amir Vadai wrote:
>>>   
>>>     
>>>       
>>>> Hi,
>>>>
>>>> FMR are being used only in a special mode called ZCopy.
>>>>
>>>> You could disable this mode by setting the module paramter
>>>> sdp_zcopy_thresh to 0, or by issuing:
>>>> # echo 0 > /sys/module/ib_sdp/parameters/sdp_zcopy_thresh
>>>>
>>>> This means that you won't get the benefits of Zero-copy.
>>>>
>>>> - Amir
>>>>
>>>> On 04/14/2010 11:51 AM, Andrea Gozzelino wrote:
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> On Apr 13, 2010 10:22 PM, "Tung, Chien Tin" <chien.tin.tung@intel.com>
>>>>> wrote:
>>>>>
>>>>>   
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>>>>>> Chien, does the NE020 support FMRs? I looked at the nes ofed-1.5
>>>>>>>>> code
>>>>>>>>> and it appears to do nothing in the map_phys_fmr functions.
>>>>>>>>>
>>>>>>>>>           
>>>>>>>>>             
>>>>>>>>>               
>>>>>>>>>                 
>>>>>>>>>                   
>>>>>>>> We never implemented map_phys_fmr. Is it relevant to the # of SGEs?
>>>>>>>>
>>>>>>>>         
>>>>>>>>           
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>>> No, but SDP uses FMRs. I don't think it will run without FMR support.
>>>>>>>
>>>>>>>       
>>>>>>>         
>>>>>>>           
>>>>>>>             
>>>>>>>               
>>>>>> Good to know.  Thanks.
>>>>>>
>>>>>> Chien
>>>>>>
>>>>>>
>>>>>>     
>>>>>>       
>>>>>>         
>>>>>>           
>>>>>>             
>>>>> Hi Steve and Chien,
>>>>>
>>>>> I understand that NE020 cards have problem with SDP connected with
>>>>> map_phy_fmr (FMR stands for Fast Memory Region).
>>>>> Is it possible to solve/fix this point?
>>>>> If yes, have you an idea about the time that is necessary to code
>>>>> development/build?
>>>>> If no, can you suggest me a card that supports SDP protocol?
>>>>>
>>>>> I work on NE020 cards from February 2010 for an INFN experimental
>>>>> proposal, called REDIGO (Read out at 10 Gbits/s), about the data
>>>>> acquisition and movement systems. The covergence of storage protocols
>>>>> around 10 Gigabits/s Ethernet protocols shows that one way could be the
>>>>> Remote Direct Memory Access (RDMA). The goals are the investigations of
>>>>> latency time, the throughput, the buffer size schemes and finally the
>>>>> global event building bandwidth.
>>>>>
>>>>> Do you know if NE020 cards have problems with librdma (RDMA procedures,
>>>>> in general) and / or with MPI versions?
>>>>>
>>>>> Thank you very much,
>>>>> Andrea
>>>>>
>>>>> Andrea Gozzelino
>>>>>
>>>>> INFN - Laboratori Nazionali di Legnaro	(LNL)
>>>>> Viale dell'Universita' 2
>>>>> I-35020 - Legnaro (PD)- ITALIA
>>>>> Tel: +39 049 8068346
>>>>> Fax: +39 049 641925
>>>>> Mail: andrea.gozzelino@lnl.infn.it			
>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>   
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>> Please read the FAQ at  http://www.tux.org/lkml/
>>>>   
>>>>     
>>>>       
>>>>         
>>>   
>>>     
>>>       
>
>