Re: [PATCH V1] vfio: add attach_group_by_node to control behavior of attaching group to domain

[Jason Gunthorpe <jgg@nvidia.com>]

On Fri, Nov 17, 2023 at 03:27:46PM -0700, Alex Williamson wrote:
> > diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
> > index eacd6ec..6a5641e 100644
> > --- a/drivers/vfio/vfio_iommu_type1.c
> > +++ b/drivers/vfio/vfio_iommu_type1.c
> > @@ -59,6 +59,11 @@
> >  module_param_named(dma_entry_limit, dma_entry_limit, uint, 0644);
> >  MODULE_PARM_DESC(dma_entry_limit,
> >  		 "Maximum number of user DMA mappings per container (65535).");
> > +static uint attach_group_by_node;
> > +module_param_named(attach_group_by_node,
> > +		attach_group_by_node, uint, 0644);
> > +MODULE_PARM_DESC(attach_group_by_node,
> > +		 "Attach group to domain when it's in same node");

Definately no to any kind of module option..

> I question whether we need this solution at all though.  AIUI the
> initial domain is allocated in proximity to the initial group.  The
> problem comes when the user asks to add an additional group into the
> same container.  Another valid solution would be that the user
> recognizes that these groups are not within the same locality and
> creates a separate container for this group. 

qemu using iommufd should de-duplicate the IOAS though, so we would be
back to this discussion.

Regardless, I agree this should be handled by userspace. The kernel
driver should associate the NUMA locality of the iommu_domain to the
first iommu instance that it was allocated against.

The explicit HWPTs in iommufd already will allow userspace to choose
the appropriate locality. We many need to expose a bit more
information in iommufd ioctls showing the instance to device
association.

Jason