Re: [PATCH for-next 3/5] RDMA/hns: Optimize the wr opcode conversion from ib to hns

[Leon Romanovsky <leon@kernel.org>]

On Thu, Mar 05, 2020 at 11:22:18AM +0000, liweihang wrote:
> On 2020/3/5 14:18, Leon Romanovsky wrote:
> > On Mon, Mar 02, 2020 at 08:11:31PM +0800, Weihang Li wrote:
> >> From: Xi Wang <wangxi11@huawei.com>
> >>
> >> Simplify the wr opcode conversion from ib to hns by using a map table
> >> instead of the switch-case statement.
> >>
> >> Signed-off-by: Xi Wang <wangxi11@huawei.com>
> >> Signed-off-by: Weihang Li <liweihang@huawei.com>
> >> ---
> >>  drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 70 ++++++++++++++++++------------
> >>  1 file changed, 43 insertions(+), 27 deletions(-)
> >>
> >> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> >> index c8c345f..ea61ccb 100644
> >> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> >> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> >> @@ -56,6 +56,47 @@ static void set_data_seg_v2(struct hns_roce_v2_wqe_data_seg *dseg,
> >>  	dseg->len  = cpu_to_le32(sg->length);
> >>  }
> >>
> >> +/*
> >> + * mapped-value = 1 + real-value
> >> + * The hns wr opcode real value is start from 0, In order to distinguish between
> >> + * initialized and uninitialized map values, we plus 1 to the actual value when
> >> + * defining the mapping, so that the validity can be identified by checking the
> >> + * mapped value is greater than 0.
> >> + */
> >> +#define HR_OPC_MAP(ib_key, hr_key) \
> >> +		[IB_WR_ ## ib_key] = 1 + HNS_ROCE_V2_WQE_OP_ ## hr_key
> >> +
> >> +static const u32 hns_roce_op_code[] = {
> >> +	HR_OPC_MAP(RDMA_WRITE,			RDMA_WRITE),
> >> +	HR_OPC_MAP(RDMA_WRITE_WITH_IMM,		RDMA_WRITE_WITH_IMM),
> >> +	HR_OPC_MAP(SEND,			SEND),
> >> +	HR_OPC_MAP(SEND_WITH_IMM,		SEND_WITH_IMM),
> >> +	HR_OPC_MAP(RDMA_READ,			RDMA_READ),
> >> +	HR_OPC_MAP(ATOMIC_CMP_AND_SWP,		ATOM_CMP_AND_SWAP),
> >> +	HR_OPC_MAP(ATOMIC_FETCH_AND_ADD,	ATOM_FETCH_AND_ADD),
> >> +	HR_OPC_MAP(SEND_WITH_INV,		SEND_WITH_INV),
> >> +	HR_OPC_MAP(LOCAL_INV,			LOCAL_INV),
> >> +	HR_OPC_MAP(MASKED_ATOMIC_CMP_AND_SWP,	ATOM_MSK_CMP_AND_SWAP),
> >> +	HR_OPC_MAP(MASKED_ATOMIC_FETCH_AND_ADD,	ATOM_MSK_FETCH_AND_ADD),
> >> +	HR_OPC_MAP(REG_MR,			FAST_REG_PMR),
> >> +	[IB_WR_RESERVED1] = 0,
> >
> > hns_roce_op_code[] is declared as static, everything is initialized to
> > 0, there is no need to set 0 again.
>
> OK, thank you.
>
> >
> >> +};
> >> +
> >> +static inline u32 to_hr_opcode(u32 ib_opcode)
> >
> > No inline functions in *.c, please.
>
> Hi Leon,
>
> Thanks for your comments.
>
> But I'm confused about when we should use static inline and when we should
> use macros if a function is only used in a *.c. A few days ago, Jason
> suggested me to use static inline functions, you can check the link below:
>
> https://patchwork.kernel.org/patch/11372851/
>
> Are there any rules about that in kernel or in our rdma subsystem? Should
> I use a macro, just remove the keyword "inline" from this definition or
> move this definition to .h?

Just drop "inline" word from the declaration.
https://elixir.bootlin.com/linux/latest/source/Documentation/process/coding-style.rst#L882

>
> >
> >> +{
> >> +	u32 hr_opcode = 0;
> >> +
> >> +	if (ib_opcode < IB_WR_RESERVED1)
> >
> > if (ib_opcode > ARRAY_SIZE(hns_roce_op_code) - 1)
> > 	return HNS_ROCE_V2_WQE_OP_MASK;
> >
> > return hns_roce_op_code[ib_opcode];
> >
>
> The index of ib_key in hns_roce_op_code[] is not continuous, so there
> are some invalid ib_wr_opcode for hns between the valid index.
>
> For hardware of HIP08, HNS_ROCE_V2_WQE_OP_MASK means invalid opcode but
> not zero. So we have to check if the ib_wr_opcode has a mapping value in
> hns_roce_op_code[], and if the mapping result is zero, we have to return
> HNS_ROCE_V2_WQE_OP_MASK. Is it ok like this?

I didn't mean that you will use my code as is, what about this?

if (ib_opcode > ARRAY_SIZE(hns_roce_op_code) - 1)
 	return HNS_ROCE_V2_WQE_OP_MASK;

return hns_roce_op_code[ib_opcode] ?: HNS_ROCE_V2_WQE_OP_MASK;

Thanks