From: Brett Creeley <brett.creeley@amd.com>
To: <kvm@vger.kernel.org>, <netdev@vger.kernel.org>,
<alex.williamson@redhat.com>, <cohuck@redhat.com>,
<jgg@nvidia.com>, <yishaih@nvidia.com>,
<shameerali.kolothum.thodi@huawei.com>, <kevin.tian@intel.com>
Cc: <shannon.nelson@amd.com>, <drivers@pensando.io>, <brett.creeley@amd.com>
Subject: [PATCH v3 vfio 0/7] pds vfio driver
Date: Sun, 19 Feb 2023 00:39:01 -0800 [thread overview]
Message-ID: <20230219083908.40013-1-brett.creeley@amd.com> (raw)
This is a draft patchset for a new vendor specific VFIO driver
(pds_vfio) for use with the AMD/Pensando Distributed Services Card
(DSC). This driver is device type agnostic and live migration is
supported as long as the underlying SR-IOV VF supports live migration
on the DSC. This driver is a client of the newly introduced pds_core
driver, which the latest version can be referenced at:
https://lore.kernel.org/netdev/20230217225558.19837-1-shannon.nelson@amd.com/
This driver will use the pds_core device and auxiliary_bus as the VFIO
control path to the DSC. The pds_core device creates auxiliary_bus
devices for each live migratable VF. The devices are named by their
feature plus the VF PCI BDF so the auxiliary_bus driver implemented by
pds_vfio can find its related VF PCI driver instance. Once this
auxiliary bus connection is configured, the pds_vfio driver can send
admin queue commands to the device and receive events from pds_core.
An ASCII diagram of a VFIO instance looks something like this and can
be used with the VFIO subsystem to provide devices VFIO and live
migration support.
.------. .--------------------------.
| QEMU |--| VM .-------------. |
'......' | | PCI driver | |
| | .-------------. |
| | | SR-IOV VF | |
| | '-------------' |
| '---------------||---------'
.--------------. ||
|/dev/<vfio_fd>| ||
'--------------' ||
Host Userspace | ||
=================================================== ||
Host Kernel | ||
| ||
pds_core.LM.2305 <--+ .--------. ||
| | |vfio-pci| ||
| | '--------' ||
| | | ||
.------------. .-------------. ||
| pds_core | | pds_vfio | ||
'------------' '-------------' ||
|| || ||
09:00.0 09:00.1 ||
== PCI ==================================================||=====
|| || ||
.----------. .----------. ||
,-----| PF |---------| VF |-------------------,
| '----------' '----------' | VF |
| DSC | data/control |
| | path |
-----------------------------------------------------------
The pds_vfio driver is targeted to reside in drivers/vfio/pci/pds.
It makes use of and introduces new files in the common include/linux/pds
include directory.
Changes:
v3:
- Update copyright year to 2023 and use "Advanced Micro Devices, Inc."
for the company name
- Clarify the fact that AMD/Pensando's VFIO solution is device type
agnostic, which aligns with other current VFIO solutions
- Add line in drivers/vfio/pci/Makefile to build pds_vfio
- Move documentation to amd sub-directory
- Remove some dead code due to the pds_core implementation of
listening to BIND/UNBIND events
- Move a dev_dbg() to a previous patch in the series
- Add implementation for vfio_migration_ops.migration_get_data_size to
return the maximum possible device state size
RFC to v2:
https://lore.kernel.org/all/20221214232136.64220-1-brett.creeley@amd.com/
- Implement state transitions for VFIO_MIGRATION_P2P flag
- Improve auxiliary driver probe by returning EPROBE_DEFER
when the PCI driver is not set up correctly
- Add pointer to docs in
Documentation/networking/device_drivers/ethernet/index.rst
RFC:
https://lore.kernel.org/all/20221207010705.35128-1-brett.creeley@amd.com/
Brett Creeley (7):
vfio/pds: Initial support for pds_vfio VFIO driver
vfio/pds: Add support to register as PDS client
vfio/pds: Add VFIO live migration support
vfio: Commonize combine_ranges for use in other VFIO drivers
vfio/pds: Add support for dirty page tracking
vfio/pds: Add support for firmware recovery
vfio/pds: Add Kconfig and documentation
.../device_drivers/ethernet/amd/pds_vfio.rst | 88 +++
.../device_drivers/ethernet/index.rst | 1 +
MAINTAINERS | 7 +
drivers/vfio/pci/Kconfig | 2 +
drivers/vfio/pci/Makefile | 2 +
drivers/vfio/pci/mlx5/cmd.c | 48 +-
drivers/vfio/pci/pds/Kconfig | 19 +
drivers/vfio/pci/pds/Makefile | 12 +
drivers/vfio/pci/pds/aux_drv.c | 210 +++++++
drivers/vfio/pci/pds/aux_drv.h | 28 +
drivers/vfio/pci/pds/cmds.c | 485 ++++++++++++++++
drivers/vfio/pci/pds/cmds.h | 44 ++
drivers/vfio/pci/pds/dirty.c | 541 ++++++++++++++++++
drivers/vfio/pci/pds/dirty.h | 48 ++
drivers/vfio/pci/pds/lm.c | 491 ++++++++++++++++
drivers/vfio/pci/pds/lm.h | 53 ++
drivers/vfio/pci/pds/pci_drv.c | 126 ++++
drivers/vfio/pci/pds/pci_drv.h | 14 +
drivers/vfio/pci/pds/vfio_dev.c | 239 ++++++++
drivers/vfio/pci/pds/vfio_dev.h | 42 ++
drivers/vfio/vfio_main.c | 48 ++
include/linux/pds/pds_lm.h | 391 +++++++++++++
include/linux/vfio.h | 3 +
23 files changed, 2895 insertions(+), 47 deletions(-)
create mode 100644 Documentation/networking/device_drivers/ethernet/amd/pds_vfio.rst
create mode 100644 drivers/vfio/pci/pds/Kconfig
create mode 100644 drivers/vfio/pci/pds/Makefile
create mode 100644 drivers/vfio/pci/pds/aux_drv.c
create mode 100644 drivers/vfio/pci/pds/aux_drv.h
create mode 100644 drivers/vfio/pci/pds/cmds.c
create mode 100644 drivers/vfio/pci/pds/cmds.h
create mode 100644 drivers/vfio/pci/pds/dirty.c
create mode 100644 drivers/vfio/pci/pds/dirty.h
create mode 100644 drivers/vfio/pci/pds/lm.c
create mode 100644 drivers/vfio/pci/pds/lm.h
create mode 100644 drivers/vfio/pci/pds/pci_drv.c
create mode 100644 drivers/vfio/pci/pds/pci_drv.h
create mode 100644 drivers/vfio/pci/pds/vfio_dev.c
create mode 100644 drivers/vfio/pci/pds/vfio_dev.h
create mode 100644 include/linux/pds/pds_lm.h
--
2.17.1
next reply other threads:[~2023-02-19 8:39 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-02-19 8:39 Brett Creeley [this message]
2023-02-19 8:39 ` [PATCH v3 1/7] vfio/pds: Initial support for pds_vfio VFIO driver Brett Creeley
2023-02-19 8:39 ` [PATCH v3 2/7] vfio/pds: Add support to register as PDS client Brett Creeley
2023-02-19 8:39 ` [PATCH v3 3/7] vfio/pds: Add VFIO live migration support Brett Creeley
2023-02-19 8:39 ` [PATCH v3 4/7] vfio: Commonize combine_ranges for use in other VFIO drivers Brett Creeley
2023-02-19 8:39 ` [PATCH v3 5/7] vfio/pds: Add support for dirty page tracking Brett Creeley
2023-02-19 8:39 ` [PATCH v3 6/7] vfio/pds: Add support for firmware recovery Brett Creeley
2023-02-19 8:39 ` [PATCH v3 7/7] vfio/pds: Add Kconfig and documentation Brett Creeley
2023-02-20 6:29 ` [PATCH v3 vfio 0/7] pds vfio driver Christoph Hellwig
2023-02-21 0:45 ` Brett Creeley
2023-02-21 1:11 ` Jason Gunthorpe
2023-02-23 7:01 ` Brett Creeley
2023-02-23 13:26 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230219083908.40013-1-brett.creeley@amd.com \
--to=brett.creeley@amd.com \
--cc=alex.williamson@redhat.com \
--cc=cohuck@redhat.com \
--cc=drivers@pensando.io \
--cc=jgg@nvidia.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=shannon.nelson@amd.com \
--cc=yishaih@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.