Re: [PATCH v5 vfio 3/7] vfio/pds: register with the pds_core PF

[Alex Williamson <alex.williamson@redhat.com>]

On Fri, 24 Mar 2023 15:36:57 -0700
Brett Creeley <bcreeley@amd.com> wrote:

> On 3/24/2023 3:02 PM, Alex Williamson wrote:
> > Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
> > 
> > 
> > On Wed, 22 Mar 2023 13:34:38 -0700
> > Brett Creeley <brett.creeley@amd.com> wrote:
> >   
> >> The pds_core driver will supply adminq services, so find the PF
> >> and register with the DSC services.
> >>
> >> Use the following commands to enable a VF:
> >> echo 1 > /sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfs
> >>
> >> Signed-off-by: Brett Creeley <brett.creeley@amd.com>
> >> Signed-off-by: Shannon Nelson <shannon.nelson@amd.com>
> >> ---
> >>   drivers/vfio/pci/pds/Makefile   |  1 +
> >>   drivers/vfio/pci/pds/cmds.c     | 67 +++++++++++++++++++++++++++++++++
> >>   drivers/vfio/pci/pds/cmds.h     | 12 ++++++
> >>   drivers/vfio/pci/pds/pci_drv.c  | 16 +++++++-
> >>   drivers/vfio/pci/pds/pci_drv.h  |  9 +++++
> >>   drivers/vfio/pci/pds/vfio_dev.c |  5 +++
> >>   drivers/vfio/pci/pds/vfio_dev.h |  2 +
> >>   include/linux/pds/pds_lm.h      | 12 ++++++
> >>   8 files changed, 123 insertions(+), 1 deletion(-)
> >>   create mode 100644 drivers/vfio/pci/pds/cmds.c
> >>   create mode 100644 drivers/vfio/pci/pds/cmds.h
> >>   create mode 100644 drivers/vfio/pci/pds/pci_drv.h
> >>   create mode 100644 include/linux/pds/pds_lm.h
> >>
> >> diff --git a/drivers/vfio/pci/pds/Makefile b/drivers/vfio/pci/pds/Makefile
> >> index e1a55ae0f079..87581111fa17 100644
> >> --- a/drivers/vfio/pci/pds/Makefile
> >> +++ b/drivers/vfio/pci/pds/Makefile
> >> @@ -4,5 +4,6 @@
> >>   obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.o
> >>
> >>   pds_vfio-y := \
> >> +     cmds.o          \
> >>        pci_drv.o       \
> >>        vfio_dev.o
> >> diff --git a/drivers/vfio/pci/pds/cmds.c b/drivers/vfio/pci/pds/cmds.c
> >> new file mode 100644
> >> index 000000000000..26e383ec4544
> >> --- /dev/null
> >> +++ b/drivers/vfio/pci/pds/cmds.c
> >> @@ -0,0 +1,67 @@
> >> +// SPDX-License-Identifier: GPL-2.0
> >> +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */
> >> +
> >> +#include <linux/io.h>
> >> +#include <linux/types.h>
> >> +
> >> +#include <linux/pds/pds_common.h>
> >> +#include <linux/pds/pds_core_if.h>
> >> +#include <linux/pds/pds_adminq.h>
> >> +#include <linux/pds/pds_lm.h>
> >> +
> >> +#include "vfio_dev.h"
> >> +#include "cmds.h"
> >> +
> >> +int
> >> +pds_vfio_register_client_cmd(struct pds_vfio_pci_device *pds_vfio)
> >> +{
> >> +     union pds_core_adminq_comp comp = { 0 };
> >> +     union pds_core_adminq_cmd cmd = { 0 };
> >> +     struct device *dev;
> >> +     int err, id;
> >> +     u16 ci;
> >> +
> >> +     id = PCI_DEVID(pds_vfio->pdev->bus->number,
> >> +                    pci_iov_virtfn_devfn(pds_vfio->pdev, pds_vfio->vf_id));
> >> +
> >> +     dev = &pds_vfio->pdev->dev;
> >> +     cmd.client_reg.opcode = PDS_AQ_CMD_CLIENT_REG;
> >> +     snprintf(cmd.client_reg.devname, sizeof(cmd.client_reg.devname),
> >> +              "%s.%d", PDS_LM_DEV_NAME, id);  
> > 
> > Does this devname need to be unique, and if so should it factor in
> > pci_domain_nr()?  The array seems to be wide enough to easily hold the
> > VF dev_name() but I haven't followed if there are additional
> > constraints.  Thanks,
> > 
> > Alex
> >   
> >> +  
> 
> Hey Alex,
> 
> We used the PCI_DEVID (id) as the suffix, which should be good enough to 
> provide uniqueness for each devname.

But PCI_DEVID is not unique unless this device is guaranteed never to
run on a system with multiple PCI segments, including cases where these
PFs and VFs might be assigned to multi-segment VMs.  Thanks,

Alex