Re: [PATCH v5 vfio 3/7] vfio/pds: register with the pds_core PF

[Brett Creeley <bcreeley@amd.com>]

On 3/24/2023 3:47 PM, Alex Williamson wrote:
> Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
> 
> 
> On Fri, 24 Mar 2023 15:36:57 -0700
> Brett Creeley <bcreeley@amd.com> wrote:
> 
>> On 3/24/2023 3:02 PM, Alex Williamson wrote:
>>> Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
>>>
>>>
>>> On Wed, 22 Mar 2023 13:34:38 -0700
>>> Brett Creeley <brett.creeley@amd.com> wrote:
>>>
>>>> The pds_core driver will supply adminq services, so find the PF
>>>> and register with the DSC services.
>>>>
>>>> Use the following commands to enable a VF:
>>>> echo 1 > /sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfs
>>>>
>>>> Signed-off-by: Brett Creeley <brett.creeley@amd.com>
>>>> Signed-off-by: Shannon Nelson <shannon.nelson@amd.com>
>>>> ---
>>>>    drivers/vfio/pci/pds/Makefile   |  1 +
>>>>    drivers/vfio/pci/pds/cmds.c     | 67 +++++++++++++++++++++++++++++++++
>>>>    drivers/vfio/pci/pds/cmds.h     | 12 ++++++
>>>>    drivers/vfio/pci/pds/pci_drv.c  | 16 +++++++-
>>>>    drivers/vfio/pci/pds/pci_drv.h  |  9 +++++
>>>>    drivers/vfio/pci/pds/vfio_dev.c |  5 +++
>>>>    drivers/vfio/pci/pds/vfio_dev.h |  2 +
>>>>    include/linux/pds/pds_lm.h      | 12 ++++++
>>>>    8 files changed, 123 insertions(+), 1 deletion(-)
>>>>    create mode 100644 drivers/vfio/pci/pds/cmds.c
>>>>    create mode 100644 drivers/vfio/pci/pds/cmds.h
>>>>    create mode 100644 drivers/vfio/pci/pds/pci_drv.h
>>>>    create mode 100644 include/linux/pds/pds_lm.h
>>>>
>>>> diff --git a/drivers/vfio/pci/pds/Makefile b/drivers/vfio/pci/pds/Makefile
>>>> index e1a55ae0f079..87581111fa17 100644
>>>> --- a/drivers/vfio/pci/pds/Makefile
>>>> +++ b/drivers/vfio/pci/pds/Makefile
>>>> @@ -4,5 +4,6 @@
>>>>    obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.o
>>>>
>>>>    pds_vfio-y := \
>>>> +     cmds.o          \
>>>>         pci_drv.o       \
>>>>         vfio_dev.o
>>>> diff --git a/drivers/vfio/pci/pds/cmds.c b/drivers/vfio/pci/pds/cmds.c
>>>> new file mode 100644
>>>> index 000000000000..26e383ec4544
>>>> --- /dev/null
>>>> +++ b/drivers/vfio/pci/pds/cmds.c
>>>> @@ -0,0 +1,67 @@
>>>> +// SPDX-License-Identifier: GPL-2.0
>>>> +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */
>>>> +
>>>> +#include <linux/io.h>
>>>> +#include <linux/types.h>
>>>> +
>>>> +#include <linux/pds/pds_common.h>
>>>> +#include <linux/pds/pds_core_if.h>
>>>> +#include <linux/pds/pds_adminq.h>
>>>> +#include <linux/pds/pds_lm.h>
>>>> +
>>>> +#include "vfio_dev.h"
>>>> +#include "cmds.h"
>>>> +
>>>> +int
>>>> +pds_vfio_register_client_cmd(struct pds_vfio_pci_device *pds_vfio)
>>>> +{
>>>> +     union pds_core_adminq_comp comp = { 0 };
>>>> +     union pds_core_adminq_cmd cmd = { 0 };
>>>> +     struct device *dev;
>>>> +     int err, id;
>>>> +     u16 ci;
>>>> +
>>>> +     id = PCI_DEVID(pds_vfio->pdev->bus->number,
>>>> +                    pci_iov_virtfn_devfn(pds_vfio->pdev, pds_vfio->vf_id));
>>>> +
>>>> +     dev = &pds_vfio->pdev->dev;
>>>> +     cmd.client_reg.opcode = PDS_AQ_CMD_CLIENT_REG;
>>>> +     snprintf(cmd.client_reg.devname, sizeof(cmd.client_reg.devname),
>>>> +              "%s.%d", PDS_LM_DEV_NAME, id);
>>>
>>> Does this devname need to be unique, and if so should it factor in
>>> pci_domain_nr()?  The array seems to be wide enough to easily hold the
>>> VF dev_name() but I haven't followed if there are additional
>>> constraints.  Thanks,
>>>
>>> Alex
>>>
>>>> +
>>
>> Hey Alex,
>>
>> We used the PCI_DEVID (id) as the suffix, which should be good enough to
>> provide uniqueness for each devname.
> 
> But PCI_DEVID is not unique unless this device is guaranteed never to
> run on a system with multiple PCI segments, including cases where these
> PFs and VFs might be assigned to multi-segment VMs.  Thanks,
> 
> Alex

Okay, that makes sense. Thanks for the feedback. We will rework the 
devname and repost.

Thanks,

Brett