Re: [PATCH for-next v2 18/18] RDMA/rxe: Add parameters to control task type

[Jason Gunthorpe <jgg@nvidia.com>]

On Tue, Oct 25, 2022 at 09:31:25AM -0500, Bob Pearson wrote:
> On 10/25/22 06:18, Jason Gunthorpe wrote:
> > On Tue, Oct 25, 2022 at 09:35:01AM +0000, Daisuke Matsuda (Fujitsu) wrote:
> >> On Sat, Oct 22, 2022 5:01 AM Bob Pearson:
> >>>
> >>> Add modparams to control the task types for req, comp, and resp
> >>> tasks.
> >>>
> >>> It is expected that the work queue version will take the place of
> >>> the tasklet version once this patch series is accepted and moved
> >>> upstream. However, for now it is convenient to keep the option of
> >>> easily switching between the versions to help benchmarking and
> >>> debugging.
> >>>
> >>> Signed-off-by: Ian Ziemba <ian.ziemba@hpe.com>
> >>> Signed-off-by: Bob Pearson <rpearsonhpe@gmail.com>
> >>> ---
> >>>  drivers/infiniband/sw/rxe/rxe_qp.c   | 6 +++---
> >>>  drivers/infiniband/sw/rxe/rxe_task.c | 8 ++++++++
> >>>  drivers/infiniband/sw/rxe/rxe_task.h | 4 ++++
> >>>  3 files changed, 15 insertions(+), 3 deletions(-)
> >>
> >> <...>
> >>
> >>> diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c
> >>> index 9b8c9d28ee46..4568c4a05e85 100644
> >>> --- a/drivers/infiniband/sw/rxe/rxe_task.c
> >>> +++ b/drivers/infiniband/sw/rxe/rxe_task.c
> >>> @@ -6,6 +6,14 @@
> >>>
> >>>  #include "rxe.h"
> >>>
> >>> +int rxe_req_task_type = RXE_TASK_TYPE_TASKLET;
> >>> +int rxe_comp_task_type = RXE_TASK_TYPE_TASKLET;
> >>> +int rxe_resp_task_type = RXE_TASK_TYPE_TASKLET;
> >>
> >> As the tasklet version is to be eliminated in near future, why
> >> don't we make the workqueue version default now?
> > 
> > Why don't we just get rid of the tasklet version right away?
> > 
> > Jason
> 
> What time zone are you in? I thought you were out west.
> 
> There are several users out there who use rxe as a dev tool for other subsystems.
> I don't want to make a big change that they can't control. By letting them decide
> if and when to move that is avoided. I could back out the modparam and just let
> people recompile but I'd end up putting it back in for our use because it is easier.

As long as it is functionally the same I'm not worried about
this. Small performance variations are not going to effect dev work.

> No matter what we decide to do here, there is a question I have about how to
> surface tuning parameters to users. Not everyone can just recompile to make changes.
> What is the preferred way to do this?

I don't know that we have much in this area pre-existing. Most of the
devices do not seem to have tuning?

sysfs and rdma netlink are the usual answers, but I don't know about
exposing rxe specific tunables in netlink..

Jason