From: Max Gurtovoy <mgurtovoy@nvidia.com>
To: Brett Creeley <brett.creeley@amd.com>,
kvm@vger.kernel.org, netdev@vger.kernel.org,
alex.williamson@redhat.com, cohuck@redhat.com, jgg@nvidia.com,
yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com,
kevin.tian@intel.com
Cc: shannon.nelson@amd.com, drivers@pensando.io, Oren Duer <oren@nvidia.com>
Subject: Re: [RFC PATCH vfio 0/7] pds vfio driver
Date: Sun, 11 Dec 2022 14:54:26 +0200 [thread overview]
Message-ID: <1352e2f7-4822-4e49-cf3b-d8a9d537a172@nvidia.com> (raw)
In-Reply-To: <20221207010705.35128-1-brett.creeley@amd.com>
On 12/7/2022 3:06 AM, Brett Creeley wrote:
> This is a first draft patchset for a new vendor specific VFIO driver for
> use with the AMD/Pensando Distributed Services Card (DSC). This driver
> (pds_vfio) is a client of the newly introduced pds_core driver.
>
> Reference to the pds_core patchset:
> https://lore.kernel.org/netdev/20221207004443.33779-1-shannon.nelson@amd.com/
>
> AMD/Pensando already supports a NVMe VF device (1dd8:1006) in the
> Distributed Services Card (DSC). This patchset adds the new pds_vfio
> driver in order to support NVMe VF live migration.
>
> This driver will use the pds_core device and auxiliary_bus as the VFIO
> control path to the DSC. The pds_core device creates auxiliary_bus devices
> for each live migratable VF. The devices are named by their feature plus
> the VF PCI BDF so the auxiliary_bus driver implemented by pds_vfio can find
> its related VF PCI driver instance. Once this auxiliary bus connection
> is configured, the pds_vfio driver can send admin queue commands to the
> device and receive events from pds_core.
>
> An ASCII diagram of a VFIO instance looks something like this and can
> be used with the VFIO subsystem to provide devices VFIO and live
> migration support.
>
> .------. .--------------------------.
> | QEMU |--| VM .-------------. |
> '......' | | nvme driver | |
> | | .-------------. |
> | | | SR-IOV VF | |
> | | '-------------' |
> | '---------------||---------'
> .--------------. ||
> |/dev/<vfio_fd>| ||
> '--------------' ||
> Host Userspace | ||
> =================================================== ||
> Host Kernel | ||
> | ||
> pds_core.LM.2305 <--+ .--------. ||
> | | |vfio-pci| ||
> | | '--------' ||
> | | | ||
> .------------. .-------------. ||
> | pds_core | | pds_vfio | ||
> '------------' '-------------' ||
> || || ||
> 09:00.0 09:00.1 ||
> == PCI ==================================================||=====
> || || ||
> .----------. .----------. ||
> ,-----| PF |---------| VF |-------------------,
> | '----------' '----------' | nvme |
> | DSC | data/control |
> | | path |
> -----------------------------------------------------------
Hi Brett,
what is the class code of the pds_core device ?
I see that pds_vfio class_code is PCI_CLASS_STORAGE_EXPRESS.
>
>
> The pds_vfio driver is targeted to reside in drivers/vfio/pci/pds.
> It makes use of and introduces new files in the common include/linux/pds
> include directory.
>
> Brett Creeley (7):
> pds_vfio: Initial support for pds_vfio VFIO driver
> pds_vfio: Add support to register as PDS client
> pds_vfio: Add VFIO live migration support
> vfio: Commonize combine_ranges for use in other VFIO drivers
> pds_vfio: Add support for dirty page tracking
> pds_vfio: Add support for firmware recovery
> pds_vfio: Add documentation files
>
> .../ethernet/pensando/pds_vfio.rst | 88 +++
> drivers/vfio/pci/Kconfig | 2 +
> drivers/vfio/pci/mlx5/cmd.c | 48 +-
> drivers/vfio/pci/pds/Kconfig | 10 +
> drivers/vfio/pci/pds/Makefile | 12 +
> drivers/vfio/pci/pds/aux_drv.c | 216 +++++++
> drivers/vfio/pci/pds/aux_drv.h | 30 +
> drivers/vfio/pci/pds/cmds.c | 486 ++++++++++++++++
> drivers/vfio/pci/pds/cmds.h | 44 ++
> drivers/vfio/pci/pds/dirty.c | 541 ++++++++++++++++++
> drivers/vfio/pci/pds/dirty.h | 49 ++
> drivers/vfio/pci/pds/lm.c | 484 ++++++++++++++++
> drivers/vfio/pci/pds/lm.h | 53 ++
> drivers/vfio/pci/pds/pci_drv.c | 134 +++++
> drivers/vfio/pci/pds/pci_drv.h | 9 +
> drivers/vfio/pci/pds/vfio_dev.c | 238 ++++++++
> drivers/vfio/pci/pds/vfio_dev.h | 42 ++
> drivers/vfio/vfio_main.c | 48 ++
> include/linux/pds/pds_core_if.h | 1 +
> include/linux/pds/pds_lm.h | 356 ++++++++++++
> include/linux/vfio.h | 3 +
> 21 files changed, 2847 insertions(+), 47 deletions(-)
> create mode 100644 Documentation/networking/device_drivers/ethernet/pensando/pds_vfio.rst
> create mode 100644 drivers/vfio/pci/pds/Kconfig
> create mode 100644 drivers/vfio/pci/pds/Makefile
> create mode 100644 drivers/vfio/pci/pds/aux_drv.c
> create mode 100644 drivers/vfio/pci/pds/aux_drv.h
> create mode 100644 drivers/vfio/pci/pds/cmds.c
> create mode 100644 drivers/vfio/pci/pds/cmds.h
> create mode 100644 drivers/vfio/pci/pds/dirty.c
> create mode 100644 drivers/vfio/pci/pds/dirty.h
> create mode 100644 drivers/vfio/pci/pds/lm.c
> create mode 100644 drivers/vfio/pci/pds/lm.h
> create mode 100644 drivers/vfio/pci/pds/pci_drv.c
> create mode 100644 drivers/vfio/pci/pds/pci_drv.h
> create mode 100644 drivers/vfio/pci/pds/vfio_dev.c
> create mode 100644 drivers/vfio/pci/pds/vfio_dev.h
> create mode 100644 include/linux/pds/pds_lm.h
>
> --
> 2.17.1
>
next prev parent reply other threads:[~2022-12-11 12:54 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-07 1:06 [RFC PATCH vfio 0/7] pds vfio driver Brett Creeley
2022-12-07 1:06 ` [RFC PATCH vfio 1/7] vfio/pds: Initial support for pds_vfio VFIO driver Brett Creeley
2022-12-07 1:07 ` [RFC PATCH vfio 2/7] vfio/pds: Add support to register as PDS client Brett Creeley
2022-12-07 1:07 ` [RFC PATCH vfio 3/7] vfio/pds: Add VFIO live migration support Brett Creeley
2022-12-07 17:09 ` Jason Gunthorpe
2022-12-07 21:32 ` Brett Creeley
2022-12-07 23:29 ` Jason Gunthorpe
2022-12-07 23:34 ` Brett Creeley
2022-12-07 1:07 ` [RFC PATCH vfio 4/7] vfio: Commonize combine_ranges for use in other VFIO drivers Brett Creeley
2022-12-07 1:07 ` [RFC PATCH vfio 5/7] vfio/pds: Add support for dirty page tracking Brett Creeley
2022-12-07 1:07 ` [RFC PATCH vfio 6/7] vfio/pds: Add support for firmware recovery Brett Creeley
2022-12-07 1:07 ` [RFC PATCH vfio 7/7] vfio/pds: Add Kconfig and documentation Brett Creeley
2022-12-07 7:43 ` [RFC PATCH vfio 0/7] pds vfio driver Christoph Hellwig
2022-12-11 12:54 ` Max Gurtovoy [this message]
2022-12-12 1:16 ` Brett Creeley
2022-12-12 17:46 ` Brett Creeley
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1352e2f7-4822-4e49-cf3b-d8a9d537a172@nvidia.com \
--to=mgurtovoy@nvidia.com \
--cc=alex.williamson@redhat.com \
--cc=brett.creeley@amd.com \
--cc=cohuck@redhat.com \
--cc=drivers@pensando.io \
--cc=jgg@nvidia.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=oren@nvidia.com \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=shannon.nelson@amd.com \
--cc=yishaih@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox