Re: [PATCH 09/16] vdpa/mlx5: Allow creation/deletion of any given mr struct

From: Dragos Tatulea via Virtualization <virtualization@lists.linux-foundation.org>
To: "jasowang@redhat.com" <jasowang@redhat.com>
Cc: "xuanzhuo@linux.alibaba.com" <xuanzhuo@linux.alibaba.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"mst@redhat.com" <mst@redhat.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"virtualization@lists.linux-foundation.org"
	<virtualization@lists.linux-foundation.org>,
	"eperezma@redhat.com" <eperezma@redhat.com>,
	Saeed Mahameed <saeedm@nvidia.com>
Subject: Re: [PATCH 09/16] vdpa/mlx5: Allow creation/deletion of any given mr struct
Date: Sun, 8 Oct 2023 12:04:32 +0000	[thread overview]
Message-ID: <4f9759182636f7bb3cb15bc8b6ec6afbe0d6053d.camel@nvidia.com> (raw)
In-Reply-To: <CACGkMEvZESDuOiX_oOvMUh0YqCWYqvmD3Ve9YEJW3+RXXyvGew@mail.gmail.com>

On Sun, 2023-10-08 at 12:25 +0800, Jason Wang wrote:
> On Tue, Sep 26, 2023 at 3:21 PM Dragos Tatulea <dtatulea@nvidia.com> wrote:
> > 
> > On Tue, 2023-09-26 at 12:44 +0800, Jason Wang wrote:
> > > On Tue, Sep 12, 2023 at 9:02 PM Dragos Tatulea <dtatulea@nvidia.com>
> > > wrote:
> > > > 
> > > > This patch adapts the mr creation/deletion code to be able to work with
> > > > any given mr struct pointer. All the APIs are adapted to take an extra
> > > > parameter for the mr.
> > > > 
> > > > mlx5_vdpa_create/delete_mr doesn't need a ASID parameter anymore. The
> > > > check is done in the caller instead (mlx5_set_map).
> > > > 
> > > > This change is needed for a followup patch which will introduce an
> > > > additional mr for the vq descriptor data.
> > > > 
> > > > Signed-off-by: Dragos Tatulea <dtatulea@nvidia.com>
> > > > ---
> > > 
> > > Thinking of this decoupling I think I have a question.
> > > 
> > > We advertise 2 address spaces and 2 groups. So we actually don't know
> > > for example which address spaces will be used by dvq.
> > > 
> > > And actually we allow the user space to do something like
> > > 
> > > set_group_asid(dvq_group, 0)
> > > set_map(0)
> > > set_group_asid(dvq_group, 1)
> > > set_map(1)
> > > 
> > > I wonder if the decoupling like this patch can work and why.
> > > 
> > This scenario could indeed work. Especially if you look at the 13'th patch
> > [0]
> > where hw support is added. Are you wondering if this should work at all or
> > if it
> > should be blocked?
> 
> It would be great if it can work with the following patches. But at
> least for this patch, it seems not:
> 
> For example, what happens if we switch back to group 0 for dvq?
> 
> set_group_asid(dvq_group, 0)
> set_map(0)
> set_group_asid(dvq_group, 1)
> set_map(1)
> // here we destroy the mr created for asid 0
> set_group_asid(dvq_group, 0)
> 
If by destroy you mean .reset, I think it works: During .reset the mapping in
ASID 0 is reset back to the DMA/pysical map (mlx5_vdpa_create_dma_mr). Am I
missing something?

> Btw, if this is a new issue, I haven't checked whether or not it
> exists before this series (if yes, we can fix on top).

> > 
> > > It looks to me the most easy way is to let each AS be backed by an MR.
> > > Then we don't even need to care about the dvq, cvq.
> > That's what this patch series dowes.
> 
> Good to know this, I will review the series.
> 
I was planning to spin a v3 with Eugenio's suggestions. Should I wait for your
feedback before doing that?

Thanks,
Dragos
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization