Re: [PATCH rdma-next v1 4/4] RDMA/mlx5: Sync device with CPU pages upon ODP MR registration

[Jason Gunthorpe <jgg@nvidia.com>]

On Mon, Sep 21, 2020 at 08:04:43PM +0300, Yishai Hadas wrote:
> On 9/21/2020 5:25 PM, Jason Gunthorpe wrote:
> > On Thu, Sep 17, 2020 at 02:21:52PM +0300, Leon Romanovsky wrote:
> > 
> > > diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
> > > index dea65e511a3e..234a5d25a072 100644
> > > +++ b/drivers/infiniband/hw/mlx5/mr.c
> > > @@ -1431,7 +1431,7 @@ struct ib_mr *mlx5_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
> > >   	mr->umem = umem;
> > >   	set_mr_fields(dev, mr, npages, length, access_flags);
> > > 
> > > -	if (xlt_with_umr) {
> > > +	if (xlt_with_umr && !(access_flags & IB_ACCESS_ON_DEMAND)) {
> > >   		/*
> > >   		 * If the MR was created with reg_create then it will be
> > >   		 * configured properly but left disabled. It is safe to go ahead
> > > @@ -1439,9 +1439,6 @@ struct ib_mr *mlx5_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
> > >   		 */
> > >   		int update_xlt_flags = MLX5_IB_UPD_XLT_ENABLE;
> > > 
> > > -		if (access_flags & IB_ACCESS_ON_DEMAND)
> > > -			update_xlt_flags |= MLX5_IB_UPD_XLT_ZAP;
> > > -
> > >   		err = mlx5_ib_update_xlt(mr, 0, ncont, page_shift,
> > >   					 update_xlt_flags);
> > >   		if (err) {
> > > @@ -1467,6 +1464,12 @@ struct ib_mr *mlx5_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
> > >   			dereg_mr(dev, mr);
> > >   			return ERR_PTR(err);
> > >   		}
> > > +
> > > +		err = mlx5_ib_init_odp_mr(mr, start, length, xlt_with_umr);
> > No reason to pass start/length, that is already in the mr
> > 
> > Jason
> 
> The start / iova is set on 'ib_mr' in the uverbs layer post returning from
> reg_user_mr for all drivers [1], so the function just got both.
> Makes sense ?

The information is stored in the umem_odp

Jason