Re: [RFC PATCH] RDMA/rxe: Export imm_data to WC when the related WR with imm_data finished on SQ

[Xiao Yang <yangx.jy@cn.fujitsu.com>]

On 2021/1/28 20:54, Leon Romanovsky wrote:
> On Thu, Jan 28, 2021 at 02:29:43PM +0800, Xiao Yang wrote:
>> On 2021/1/27 20:04, Leon Romanovsky wrote:
>>> On Wed, Jan 27, 2021 at 04:24:31PM +0800, Xiao Yang wrote:
>>>> Even if we enable sq_sig_all or IBV_SEND_SIGNALED, current rxe
>>>> module cannot set imm_data in WC when the related WR with imm_data
>>>> finished on SQ.
>>>>
>>>> Signed-off-by: Xiao Yang<yangx.jy@cn.fujitsu.com>
>>>> ---
>>>>
>>>> Current rxe module and other rdma modules(e.g. mlx5) only set
>>>> imm_data in WC when the related WR with imm_data finished on RQ.
>>>> I am not sure if it is a expected behavior.
>>> This is IBTA behavior.
>>>
>>> 5.2.11 IMMEDIATE DATA EXTENDED TRANSPORT HEADER (ImmDt) - 4 BYTES
>>> "Immediate Data (ImmDt) contains data that is placed in the receive
>>>    Completion Queue Element (CQE). The ImmDt is only allowed in SEND or
>>>    RDMA WRITE packets with Immediate Data."
>>>
>>> If I understand the spec, you shouldn't set imm_data in SQ.
>> Hi Leon,
>>
>> About the behavior, I have another question:
>> For send operation with imm_data, we can verify if the delivered imm_data is
>> correct by CQE on RQ.
>> For rdma write operation with imm_data, how to verify if the delivered
>> imm_data is correct? :-)
> Probably that I didn't understand the question, but the RDMA WRITE is
> marked with special opcode in the BTH that indicates imm_data.
Hi Leon,

The quesion is about how to get the imm_data in applications(programs in 
user space)
1) If client program does send operation with imm_data, server program 
can get the delivered imm_data by calling ibv_poll_cq(&wc)
2) If client program does rdma write operation with imm_data, server 
program cannot get the delivered imm_data by calling ibv_poll_cq(&wc).
     In this case, how does server program get the delivered imm_data?

Best Regards,
Xiao Yang
> Thanks
>
>> Best Regards,
>> Xiao Yang
>>> Thanks
>>>
>>>
>>> .
>>>
>>
>>
>
> .
>