* [PATCH] docs/devel: Add VFIO iommufd backend documentation
@ 2023-11-17 9:35 Zhenzhong Duan
2023-11-17 12:58 ` Cédric Le Goater
0 siblings, 1 reply; 4+ messages in thread
From: Zhenzhong Duan @ 2023-11-17 9:35 UTC (permalink / raw)
To: qemu-devel
Cc: alex.williamson, clg, jgg, nicolinc, joao.m.martins, eric.auger,
peterx, jasowang, kevin.tian, yi.l.liu, yi.y.sun, chao.p.peng,
Zhenzhong Duan
Suggested-by: Cédric Le Goater <clg@redhat.com>
Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
---
MAINTAINERS | 1 +
docs/devel/index-internals.rst | 1 +
docs/devel/vfio-iommufd.rst | 115 +++++++++++++++++++++++++++++++++
3 files changed, 117 insertions(+)
create mode 100644 docs/devel/vfio-iommufd.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index d86ba56a49..07990456ed 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2175,6 +2175,7 @@ F: backends/iommufd.c
F: include/sysemu/iommufd.h
F: include/qemu/chardev_open.h
F: util/chardev_open.c
+F: docs/devel/vfio-iommufd.rst
vhost
M: Michael S. Tsirkin <mst@redhat.com>
diff --git a/docs/devel/index-internals.rst b/docs/devel/index-internals.rst
index 6f81df92bc..3def4a138b 100644
--- a/docs/devel/index-internals.rst
+++ b/docs/devel/index-internals.rst
@@ -18,5 +18,6 @@ Details about QEMU's various subsystems including how to add features to them.
s390-dasd-ipl
tracing
vfio-migration
+ vfio-iommufd
writing-monitor-commands
virtio-backends
diff --git a/docs/devel/vfio-iommufd.rst b/docs/devel/vfio-iommufd.rst
new file mode 100644
index 0000000000..59804a7f26
--- /dev/null
+++ b/docs/devel/vfio-iommufd.rst
@@ -0,0 +1,115 @@
+===============================
+IOMMUFD BACKEND usage with VFIO
+===============================
+
+(Same meaning for backend/container/BE)
+
+With the introduction of iommufd, the Linux kernel provides a generic
+interface for user space drivers to propagate their DMA mappings to kernel
+for assigned devices. While the legacy kernel interface is group-centric,
+the new iommufd interface is device-centric, relying on device fd and iommufd.
+
+To support both interfaces in the QEMU VFIO device, introduce a base container
+to abstract the common part of VFIO legacy and iommufd container. So that the
+generic VFIO code can use either container.
+
+The base container implements generic functions such as memory_listener and
+address space management whereas the derived container implements callbacks
+specific to either legacy or iommufd. Each container has its own way to setup
+secure context and dma management interface. The below diagram shows how it
+looks like with both containers.
+
+ VFIO AddressSpace/Memory
+ +-------+ +----------+ +-----+ +-----+
+ | pci | | platform | | ap | | ccw |
+ +---+---+ +----+-----+ +--+--+ +--+--+ +----------------------+
+ | | | | | AddressSpace |
+ | | | | +------------+---------+
+ +---V-----------V-----------V--------V----+ /
+ | VFIOAddressSpace | <------------+
+ | | | MemoryListener
+ | VFIOContainerBase list |
+ +-------+----------------------------+----+
+ | |
+ | |
+ +-------V------+ +--------V----------+
+ | iommufd | | vfio legacy |
+ | container | | container |
+ +-------+------+ +--------+----------+
+ | |
+ | /dev/iommu | /dev/vfio/vfio
+ | /dev/vfio/devices/vfioX | /dev/vfio/$group_id
+Userspace | |
+============+============================+===========================
+Kernel | device fd |
+ +---------------+ | group/container fd
+ | (BIND_IOMMUFD | | (SET_CONTAINER/SET_IOMMU)
+ | ATTACH_IOAS) | | device fd
+ | | |
+ | +-------V------------V-----------------+
+ iommufd | | vfio |
+(map/unmap | +---------+--------------------+-------+
+ioas_copy) | | | map/unmap
+ | | |
+ +------V------+ +-----V------+ +------V--------+
+ | iommfd core | | device | | vfio iommu |
+ +-------------+ +------------+ +---------------+
+
+[Secure Context setup]
+- iommufd BE: uses device fd and iommufd to setup secure context
+ (bind_iommufd, attach_ioas)
+- vfio legacy BE: uses group fd and container fd to setup secure context
+ (set_container, set_iommu)
+
+[Device access]
+- iommufd BE: device fd is opened through /dev/vfio/devices/vfioX
+- vfio legacy BE: device fd is retrieved from group fd ioctl
+
+[DMA Mapping flow]
+1. VFIOAddressSpace receives MemoryRegion add/del via MemoryListener
+2. VFIO populates DMA map/unmap via the container BEs
+ *) iommufd BE: uses iommufd
+ *) vfio legacy BE: uses container fd
+
+
+Example configuration
+=====================
+
+Step 1: configure the host device
+---------------------------------
+
+It's exactly same as the VFIO device with legacy VFIO container.
+
+Step 2: configure QEMU
+----------------------
+
+Interactions with the /dev/iommu are abstracted by a new
+iommufd object (compiled in with the CONFIG_IOMMUFD option).
+
+Any QEMU device (e.g. VFIO device) wishing to use /dev/iommu must be
+linked with an iommufd object. It gets a new optional property named
+iommufd which allows to pass an iommufd object. Take vfio-pci device
+for example:
+
+ -object iommufd,id=iommufd0
+ -device vfio-pci,host=0000:02:00.0,iommufd=iommufd0
+
+Note the /dev/iommu and VFIO cdev can be externally opened by a
+management layer. In such a case the fd is passed, the fd supports
+a string naming the fd or a number, for example:
+
+ -object iommufd,id=iommufd0,fd=22
+ -device vfio-pci,iommufd=iommufd0,fd=23
+
+If the fd property is not passed, the fd is opened by QEMU.
+
+If no iommufd property is passed to the vfio-pci device, iommufd is
+not used and the user gets the behavior based on the legacy VFIO
+container:
+
+ -device vfio-pci,host=0000:02:00.0
+
+Supported platform
+==================
+
+Supports X86, ARM and S390X currently.
--
2.34.1
^ permalink raw reply related [flat|nested] 4+ messages in thread* Re: [PATCH] docs/devel: Add VFIO iommufd backend documentation
2023-11-17 9:35 [PATCH] docs/devel: Add VFIO iommufd backend documentation Zhenzhong Duan
@ 2023-11-17 12:58 ` Cédric Le Goater
2023-11-17 14:28 ` Cédric Le Goater
0 siblings, 1 reply; 4+ messages in thread
From: Cédric Le Goater @ 2023-11-17 12:58 UTC (permalink / raw)
To: Zhenzhong Duan, qemu-devel
Cc: alex.williamson, jgg, nicolinc, joao.m.martins, eric.auger,
peterx, jasowang, kevin.tian, yi.l.liu, yi.y.sun, chao.p.peng
On 11/17/23 10:35, Zhenzhong Duan wrote:
> Suggested-by: Cédric Le Goater <clg@redhat.com>
> Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
The content looks good but it lacks formatting. Please try to generate
the docs.
Thanks,
C.
> ---
> MAINTAINERS | 1 +
> docs/devel/index-internals.rst | 1 +
> docs/devel/vfio-iommufd.rst | 115 +++++++++++++++++++++++++++++++++
> 3 files changed, 117 insertions(+)
> create mode 100644 docs/devel/vfio-iommufd.rst
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index d86ba56a49..07990456ed 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2175,6 +2175,7 @@ F: backends/iommufd.c
> F: include/sysemu/iommufd.h
> F: include/qemu/chardev_open.h
> F: util/chardev_open.c
> +F: docs/devel/vfio-iommufd.rst
>
> vhost
> M: Michael S. Tsirkin <mst@redhat.com>
> diff --git a/docs/devel/index-internals.rst b/docs/devel/index-internals.rst
> index 6f81df92bc..3def4a138b 100644
> --- a/docs/devel/index-internals.rst
> +++ b/docs/devel/index-internals.rst
> @@ -18,5 +18,6 @@ Details about QEMU's various subsystems including how to add features to them.
> s390-dasd-ipl
> tracing
> vfio-migration
> + vfio-iommufd
> writing-monitor-commands
> virtio-backends
> diff --git a/docs/devel/vfio-iommufd.rst b/docs/devel/vfio-iommufd.rst
> new file mode 100644
> index 0000000000..59804a7f26
> --- /dev/null
> +++ b/docs/devel/vfio-iommufd.rst
> @@ -0,0 +1,115 @@
> +===============================
> +IOMMUFD BACKEND usage with VFIO
> +===============================
> +
> +(Same meaning for backend/container/BE)
> +
> +With the introduction of iommufd, the Linux kernel provides a generic
> +interface for user space drivers to propagate their DMA mappings to kernel
> +for assigned devices. While the legacy kernel interface is group-centric,
> +the new iommufd interface is device-centric, relying on device fd and iommufd.
> +
> +To support both interfaces in the QEMU VFIO device, introduce a base container
> +to abstract the common part of VFIO legacy and iommufd container. So that the
> +generic VFIO code can use either container.
> +
> +The base container implements generic functions such as memory_listener and
> +address space management whereas the derived container implements callbacks
> +specific to either legacy or iommufd. Each container has its own way to setup
> +secure context and dma management interface. The below diagram shows how it
> +looks like with both containers.
> +
> + VFIO AddressSpace/Memory
> + +-------+ +----------+ +-----+ +-----+
> + | pci | | platform | | ap | | ccw |
> + +---+---+ +----+-----+ +--+--+ +--+--+ +----------------------+
> + | | | | | AddressSpace |
> + | | | | +------------+---------+
> + +---V-----------V-----------V--------V----+ /
> + | VFIOAddressSpace | <------------+
> + | | | MemoryListener
> + | VFIOContainerBase list |
> + +-------+----------------------------+----+
> + | |
> + | |
> + +-------V------+ +--------V----------+
> + | iommufd | | vfio legacy |
> + | container | | container |
> + +-------+------+ +--------+----------+
> + | |
> + | /dev/iommu | /dev/vfio/vfio
> + | /dev/vfio/devices/vfioX | /dev/vfio/$group_id
> +Userspace | |
> +============+============================+===========================
> +Kernel | device fd |
> + +---------------+ | group/container fd
> + | (BIND_IOMMUFD | | (SET_CONTAINER/SET_IOMMU)
> + | ATTACH_IOAS) | | device fd
> + | | |
> + | +-------V------------V-----------------+
> + iommufd | | vfio |
> +(map/unmap | +---------+--------------------+-------+
> +ioas_copy) | | | map/unmap
> + | | |
> + +------V------+ +-----V------+ +------V--------+
> + | iommfd core | | device | | vfio iommu |
> + +-------------+ +------------+ +---------------+
> +
> +[Secure Context setup]
> +- iommufd BE: uses device fd and iommufd to setup secure context
> + (bind_iommufd, attach_ioas)
> +- vfio legacy BE: uses group fd and container fd to setup secure context
> + (set_container, set_iommu)
> +
> +[Device access]
> +- iommufd BE: device fd is opened through /dev/vfio/devices/vfioX
> +- vfio legacy BE: device fd is retrieved from group fd ioctl
> +
> +[DMA Mapping flow]
> +1. VFIOAddressSpace receives MemoryRegion add/del via MemoryListener
> +2. VFIO populates DMA map/unmap via the container BEs
> + *) iommufd BE: uses iommufd
> + *) vfio legacy BE: uses container fd
> +
> +
> +Example configuration
> +=====================
> +
> +Step 1: configure the host device
> +---------------------------------
> +
> +It's exactly same as the VFIO device with legacy VFIO container.
> +
> +Step 2: configure QEMU
> +----------------------
> +
> +Interactions with the /dev/iommu are abstracted by a new
> +iommufd object (compiled in with the CONFIG_IOMMUFD option).
> +
> +Any QEMU device (e.g. VFIO device) wishing to use /dev/iommu must be
> +linked with an iommufd object. It gets a new optional property named
> +iommufd which allows to pass an iommufd object. Take vfio-pci device
> +for example:
> +
> + -object iommufd,id=iommufd0
> + -device vfio-pci,host=0000:02:00.0,iommufd=iommufd0
> +
> +Note the /dev/iommu and VFIO cdev can be externally opened by a
> +management layer. In such a case the fd is passed, the fd supports
> +a string naming the fd or a number, for example:
> +
> + -object iommufd,id=iommufd0,fd=22
> + -device vfio-pci,iommufd=iommufd0,fd=23
> +
> +If the fd property is not passed, the fd is opened by QEMU.
> +
> +If no iommufd property is passed to the vfio-pci device, iommufd is
> +not used and the user gets the behavior based on the legacy VFIO
> +container:
> +
> + -device vfio-pci,host=0000:02:00.0
> +
> +Supported platform
> +==================
> +
> +Supports X86, ARM and S390X currently.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] docs/devel: Add VFIO iommufd backend documentation
2023-11-17 12:58 ` Cédric Le Goater
@ 2023-11-17 14:28 ` Cédric Le Goater
2023-11-20 3:40 ` Duan, Zhenzhong
0 siblings, 1 reply; 4+ messages in thread
From: Cédric Le Goater @ 2023-11-17 14:28 UTC (permalink / raw)
To: Zhenzhong Duan, qemu-devel
Cc: alex.williamson, jgg, nicolinc, joao.m.martins, eric.auger,
peterx, jasowang, kevin.tian, yi.l.liu, yi.y.sun, chao.p.peng
On 11/17/23 13:58, Cédric Le Goater wrote:
> On 11/17/23 10:35, Zhenzhong Duan wrote:
>> Suggested-by: Cédric Le Goater <clg@redhat.com>
>> Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
>
> The content looks good but it lacks formatting. Please try to generate
> the docs.
Please check my vfio-8.2 branch.
Thanks,
C.
^ permalink raw reply [flat|nested] 4+ messages in thread
* RE: [PATCH] docs/devel: Add VFIO iommufd backend documentation
2023-11-17 14:28 ` Cédric Le Goater
@ 2023-11-20 3:40 ` Duan, Zhenzhong
0 siblings, 0 replies; 4+ messages in thread
From: Duan, Zhenzhong @ 2023-11-20 3:40 UTC (permalink / raw)
To: Cédric Le Goater, qemu-devel@nongnu.org
Cc: alex.williamson@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com,
joao.m.martins@oracle.com, eric.auger@redhat.com,
peterx@redhat.com, jasowang@redhat.com, Tian, Kevin, Liu, Yi L,
Sun, Yi Y, Peng, Chao P
>-----Original Message-----
>From: Cédric Le Goater <clg@redhat.com>
>Sent: Friday, November 17, 2023 10:28 PM
>To: Duan, Zhenzhong <zhenzhong.duan@intel.com>; qemu-devel@nongnu.org
>Cc: alex.williamson@redhat.com; jgg@nvidia.com; nicolinc@nvidia.com;
>joao.m.martins@oracle.com; eric.auger@redhat.com; peterx@redhat.com;
>jasowang@redhat.com; Tian, Kevin <kevin.tian@intel.com>; Liu, Yi L
><yi.l.liu@intel.com>; Sun, Yi Y <yi.y.sun@intel.com>; Peng, Chao P
><chao.p.peng@intel.com>
>Subject: Re: [PATCH] docs/devel: Add VFIO iommufd backend documentation
>
>On 11/17/23 13:58, Cédric Le Goater wrote:
>> On 11/17/23 10:35, Zhenzhong Duan wrote:
>>> Suggested-by: Cédric Le Goater <clg@redhat.com>
>>> Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
>>
>> The content looks good but it lacks formatting. Please try to generate
>> the docs.
>
>Please check my vfio-8.2 branch.
Got it, thanks for help.
BRs.
Zhenzhong
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2023-11-20 3:41 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-11-17 9:35 [PATCH] docs/devel: Add VFIO iommufd backend documentation Zhenzhong Duan
2023-11-17 12:58 ` Cédric Le Goater
2023-11-17 14:28 ` Cédric Le Goater
2023-11-20 3:40 ` Duan, Zhenzhong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).