Re: rdma_rxe usage problem

[Yanjun Zhu <yanjun.zhu@linux.dev>]

If I remember correctly, can you use this command to make tests?

server:

ibv_rc_pingpong -d rxe0 -g 1

Client:

ibv_rc_pingpong -d rxe0 -g 1 server_ip_addr

Zhu Yanjun

在 2022/1/20 1:53, Alexander Kalentyev 写道:
> With rping everything was fiine, but I had to use a real Ip address.
>   >rping -s -C 10 -v
> server ping data: rdma-ping-0:
> ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqr
> server ping data: rdma-ping-1:
> BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs
> server ping data: rdma-ping-2:
> CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrst
> server ping data: rdma-ping-3:
> DEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstu
> server ping data: rdma-ping-4:
> EFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuv
> server ping data: rdma-ping-5:
> FGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvw
> server ping data: rdma-ping-6:
> GHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwx
> server ping data: rdma-ping-7:
> HIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxy
> server ping data: rdma-ping-8:
> IJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz
> server ping data: rdma-ping-9:
> JKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzA
> server DISCONNECT EVENT...
> wait for RDMA_READ_ADV state 10
>
>> rping -c -a 192.168.0.176 -C 10 -v
> ping data: rdma-ping-0: ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqr
> ping data: rdma-ping-1: BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs
> ping data: rdma-ping-2: CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrst
> ping data: rdma-ping-3: DEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstu
> ping data: rdma-ping-4: EFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuv
> ping data: rdma-ping-5: FGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvw
> ping data: rdma-ping-6: GHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwx
> ping data: rdma-ping-7: HIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxy
> ping data: rdma-ping-8: IJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz
> ping data: rdma-ping-9: JKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzA
> client DISCONNECT EVENT...
>
> Anyway the ibv_rc_pingpong shows an error:
>
>> ibv_rc_pingpong -d rxe0 -g 0
>    local address:  LID 0x0000, QPN 0x000015, PSN 0x015dd8, GID
> fe80::4a51:c5ff:fef6:e159
> Failed to modify QP to RTR
> Couldn't connect to remote QP
>
>> ibv_rc_pingpong -d rxe0 -g 0 192.168.0.176
>    local address:  LID 0x0000, QPN 0x000016, PSN 0x007fa7, GID
> fe80::4a51:c5ff:fef6:e159
> client read/write: No space left on device
> Couldn't read/write remote address
>
> ср, 19 янв. 2022 г. в 15:12, Yanjun Zhu <yanjun.zhu@linux.dev>:
>> 在 2022/1/19 19:42, Alexander Kalentyev 写道:
>>> I am trying to install and use soft RoCE for development purposes
>>> (right now on a localhost).
>>> I installed rdma-core on a MANJARO system from AUR.
>>> Then I did:
>>>
>>> sudo modprobe rdma_rxe
>>> sudo rdma link add rxe0 type rxe netdev wlp60s0
>>>
>>> then ibstat shows:
>>> CA 'rxe0'
>>>           CA type:
>>>           Number of ports: 1
>>>           Firmware version:
>>>           Hardware version:
>>>           Node GUID: 0x4a51c5fffef6e159
>>>           System image GUID: 0x4a51c5fffef6e159
>>>           Port 1:
>>>                   State: Active
>>>                   Physical state: LinkUp
>>>                   Rate: 2.5
>>>                   Base lid: 0
>>>                   LMC: 0
>>>                   SM lid: 0
>>>                   Capability mask: 0x00010000
>>>                   Port GUID: 0x4a51c5fffef6e159
>>>                   Link layer: Ethernet
>> Can rping work after you configured this test environment?
>>
>> Zhu Yanjun
>>
>>> But launching any example I always get an error by call of: ibv_modify_qp
>>> with an attempt to modify QP state to RTR (for example by launching
>>> .ibv_rc_pingpong)
>>> The type of the error is EINVAL.
>>> I believe I miss something very obvious.