Re: [PATCH v2 for-next 1/2] RDMA/hns: Add support for CQ stash

[liweihang <liweihang@huawei.com>]

On 2020/11/19 4:08, Jason Gunthorpe wrote:
> On Wed, Nov 18, 2020 at 10:49:14AM +0000, liweihang wrote:
>> On 2020/11/17 16:50, Leon Romanovsky wrote:
>>> On Tue, Nov 17, 2020 at 08:35:55AM +0000, liweihang wrote:
>>>> On 2020/11/17 15:21, Leon Romanovsky wrote:
>>>>> On Tue, Nov 17, 2020 at 06:37:58AM +0000, liweihang wrote:
>>>>>> On 2020/11/16 21:47, Leon Romanovsky wrote:
>>>>>>> On Mon, Nov 16, 2020 at 07:58:38PM +0800, Weihang Li wrote:
>>>>>>>> From: Lang Cheng <chenglang@huawei.com>
>>>>>>>>
>>>>>>>> Stash is a mechanism that uses the core information carried by the ARM AXI
>>>>>>>> bus to access the L3 cache. It can be used to improve the performance by
>>>>>>>> increasing the hit ratio of L3 cache. CQs need to enable stash by default.
>>>>>>>>
>>>>>>>> Signed-off-by: Lang Cheng <chenglang@huawei.com>
>>>>>>>> Signed-off-by: Weihang Li <liweihang@huawei.com>
>>>>>>>>  drivers/infiniband/hw/hns/hns_roce_common.h | 12 +++++++++
>>>>>>>>  drivers/infiniband/hw/hns/hns_roce_device.h |  1 +
>>>>>>>>  drivers/infiniband/hw/hns/hns_roce_hw_v2.c  |  3 +++
>>>>>>>>  drivers/infiniband/hw/hns/hns_roce_hw_v2.h  | 39 +++++++++++++++++------------
>>>>>>>>  4 files changed, 39 insertions(+), 16 deletions(-)
>>>>>>>>
>>>>>>>> diff --git a/drivers/infiniband/hw/hns/hns_roce_common.h b/drivers/infiniband/hw/hns/hns_roce_common.h
>>>>>>>> index f5669ff..8d96c4e 100644
>>>>>>>> +++ b/drivers/infiniband/hw/hns/hns_roce_common.h
>>>>>>>> @@ -53,6 +53,18 @@
>>>>>>>>  #define roce_set_bit(origin, shift, val) \
>>>>>>>>  	roce_set_field((origin), (1ul << (shift)), (shift), (val))
>>>>>>>>
>>>>>>>> +#define FIELD_LOC(field_h, field_l) field_h, field_l
>>>>>>>> +
>>>>>>>> +#define _hr_reg_set(arr, field_h, field_l)                                     \
>>>>>>>> +	do {                                                                   \
>>>>>>>> +		BUILD_BUG_ON(((field_h) / 32) != ((field_l) / 32));            \
>>>>>>>> +		BUILD_BUG_ON((field_h) / 32 >= ARRAY_SIZE(arr));               \
>>>>>>>> +		(arr)[(field_h) / 32] |=                                       \
>>>>>>>> +			cpu_to_le32(GENMASK((field_h) % 32, (field_l) % 32));  \
>>>>>>>> +	} while (0)
>>>>>>>> +
>>>>>>>> +#define hr_reg_set(arr, field) _hr_reg_set(arr, field)
>>>>>>>
>>>>>>> I afraid that it is too much.
>>>>>>
>>>>>> Hi Leon,
>>>>>>
>>>>>> Thanks for the comments.
>>>>>>
>>>>>>> 1. FIELD_LOC() macro to hide two fields.
>>>>>>
>>>>>> Jason has suggested us to simplify the function of setting/getting bit/field in
>>>>>> hns driver like IBA_SET and IBA_GET.
>>>>>>
>>>>>> https://patchwork.kernel.org/project/linux-rdma/patch/1589982799-28728-3-git-send-email-liweihang@huawei.com/
>>>>>>
>>>>>> So we try to make it easier and clearer to define a bitfield for developers.
>>>>>
>>>>> Jason asked to use genmask and FIELD_PREP, but you invented something else.
>>>>>
>>>>> Thanks
>>>>>
>>>>
>>>> We use them in another interface 'hr_reg_write(arr, field, val)' which hasn't been
>>>> used in this series.
>>>>
>>>> Does it make any unacceptable mistake? We would appreciate any suggestions :)
>>>
>>> The invention of FIELD_LOC() and hr_reg_set equal to __hr_reg_set are unacceptable.
>>> Pass directly your field_h and field_l to hr_reg_set().
>>>
>>> Thanks
>>>
>>
>> Hi Leon,
>>
>> We let hr_reg_set equal() to __hr_reg_set() because if not, there will be a compile error:
>>
>> .../hns_roce_hw_v2.c:4566:41: error: macro "_hr_reg_set" requires 3 arguments, but only 2 given
>> _hr_reg_set(cq_context->raw, CQC_STASH);
> 
> Yes, it is very un-intuitive but the rules for CPP require the extra
> macro pass to generate the correct expansion. Otherwise cpp will try
> to pass the value with commas in as a single argument, what we need
> here is to expand the commads first and have them break up into macro
> arguments as it goes down the macro chain.
> 

Thanks for your explanation :)

>> Let's compare the following implementations:
>>
>> 	#define _hr_reg_set(arr, field_h, field_l) \
>> 		(arr)[(field_h) / 32] |= \
>> 			cpu_to_le32(GENMASK((field_h) % 32, (field_l) % 32)) + \
>> 			BUILD_BUG_ON_ZERO(((field_h) / 32) != ((field_l) / 32)) + \
>> 			BUILD_BUG_ON_ZERO((field_h) / 32 >= ARRAY_SIZE(arr))
>>
>> 1)
>> 	#define hr_reg_set(arr, field) _hr_reg_set(arr, field)
>>
>> 	#define QPCEX_PASID_EN FIELD_LOC(111, 95)
>> 	hr_reg_set(context->ext, QPCEX_PASID_EN);
> 
> It is also weird that something called set can only write all ones to
> the field.
> 
> It feels saner to have a set that accepts a value and if the all ones
> case is something very common then use a macro to compute it from the
> field name.
> 
> Jason
> 

OK, We will achieve hr_reg_enable(arr, field) for setting a single bit
and hr_reg_write(arr, field, val) for filling a value into a field.

Thanks
Weihang