Re: [PATCH rdma-next v1 2/2] RDMA/mlx5: Allow modifying Relaxed Ordering via fast registration

[Christoph Hellwig <hch@infradead.org>]

On Wed, May 26, 2021 at 04:49:06PM -0300, Jason Gunthorpe wrote:
> Nothing does a FRWR with IB_ACCESS_DISABLE_RELAXED_ORDERING set
> 
> So why not leave the relaxed ordering bits masked in the UMR for FWRW
> so that the UMR doesn't change them at all and fail/panic if the
> caller requests IB_ACCESS_DISABLE_RELAXED_ORDERING ?

Yeah.  In fact we should check for that in the core, or by going even
further than my previous proposal and split IB_ACCESS_* even more fine
grained.

AFAICS we have the following uses cases:

 1) qp_access_flags as a bitmask of possible operations on the queue pair
    The way I understood the queue pairs this should really be just bits
    for remote read, remote write and atomics, but a few places also
    mess with memory windows and local write, which seems to be some
    sort of iWarp cludge
 2) IB_UVERBS_ACCESS_*.  These just get checked using ib_check_mr_access
    and then passed into ->reg_user_mr, ->rereg_user_mr and
    ->reg_user_mr_dmabuf
 3) in-kernel FRWR uses IB_ACCESS_*, but all users seem to hardcode it
    to IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_READ |
    IB_ACCESS_REMOTE_WRITE anyway

In other words:  I wonder if we should just kill off the current from of
IB_ACCESS_* entirely, as it is a weird mess used in totally different
ways in different code paths.