Re: Setting up siw devices

[Marcin Mielniczuk <marcin@golem.network>]

On 22.08.2019 19:21, Leon Romanovsky wrote:
> On Thu, Aug 22, 2019 at 07:05:12PM +0200, Marcin Mielniczuk wrote:
>> Thanks a lot, this did the trick. I think this is worth documenting
>> somewhere that this step is needed.
>> I'll make a PR, would README.md in the rdma-core repo be a good place?
> I'm not so sure, but it is better to have in some place instead of not having at all.
I think it's the first place one would look for some information. I'll
make a PR today or tomorrow.
>> Does <NAME> have any significance? I did:
>>
>>      sudo rdma link add siw0 type siw netdev enpXsYYfZ
>>
>> but the resulting device is called iwpXsYYfZ. I couldn't find a trace of
>> `siw0` anywhere.
> I would say that it is a bug in kernel part of SIW, because kernel rename
> (the thing which change your siw0 to be iw* name) is looking for absence
> of mentioning PCI inside of /sys/class/infiniband/siw0/*
> https://github.com/linux-rdma/rdma-core/blob/master/kernel-boot/rdma_rename.c#L378
I don't have /sys/class/infiniband/siw0 on my system, only
/sys/class/infiniband/iwpXsYYfZ.
iwp probably comes from iWARP.

Regards,
Marcin
> That rdma-core line works for RXE and SIW should be similar.
>
> Thanks
>
>> On 22.08.2019 17:52, Leon Romanovsky wrote:
>>> On Thu, Aug 22, 2019 at 09:13:25PM +0530, Krishnamraju Eraparaju wrote:
>>>> On Thursday, August 08/22/19, 2019 at 17:08:49 +0200, Marcin Mielniczuk wrote:
>>>>> Hi,
>>>>>
>>>>> I'm trying to test the recently merged siw module.
>>>>> I'm running kernel 5.3-rc5 (taken from the Ubuntu mainline-kernel
>>>>> repository [1]) on Ubuntu 18.04 (bionic).
>>>>> I also manually installed rdma-core 25.0 from tarball, using the
>>>>> included Debian packaging. I installed all the packages but ibacm.
>>>>>
>>>>> After booting the new kernel I manually loaded the kernel module by
>>>>>
>>>>>      modprobe siw
>>>>>      modprobe rdma_ucm
>>>>>
>>>>> Then ibv_devinfo shows: "No IB devices found".
>>>>> dmesg only shows:
>>>>>      [   29.856751] SoftiWARP attached
>>>>>
>>>>> According to this tutorial, [2] it should be enough to just load the siw
>>>>> module. (unlike RXE, where one needs to use rxe_cfg to set up the
>>>>> interface)
>>>>> Is this a bug in siw or just a configuration issue on my side?
>>>> Have you done "rdma link"?
>>>>
>>>> rdma link add <NAME> type siw netdev <NETDEV>
>>>>
>>>> http://man7.org/linux/man-pages/man8/rdma-link.8.html
>>> BTW, the same goes for RXE and rxe_cfg is discouraged.
>>>
>>> Thanks
>>>
>>>>> Thanks,
>>>>> Marcin
>>>>>
>>>>> [1] https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.3-rc5/
>>>>> [2] https://budevg.github.io/posts/tutorials/2017/04/29/rdma-101-1.html
>>>>>
>>