[RFC PATCH 0/4] vfio/pci: Atomic Ops completer support

[RFC PATCH 0/4] vfio/pci: Atomic Ops completer support

[Alex Williamson]

This RFC proposes automatically enabling PCIe Atomic Ops completer
support on PCIe root ports within the VM given the restrictions that
a) vfio-pci provides a capability on the vfio device indicating enabled
support for various operations, and b) requiring a configuration of
installing the vfio-pci as a cold-plug device immediately downstream of
the root port, thus avoiding a secondary issue for enabling Atomic Ops
routing.  This idea was outlined in reply[1] to another proposal
suggesting manual device options to pcie-root-port to do the same.

This support depends on this[2] kernel patch enabling the host Atomic Op
reporting.

There are however some concerns with an automatic approach.  First is
the issue of hotplug.  If we've marked Atomic Ops completion as
supported by the root port and subsequently unplug the endpoint that was
the basis for that enablement, do we let that capability remain?  Should
the device exit function clear that capability?  Is it even legal to
clear capabilities on the root port at runtime?  I'm not sure it's safe
to assume that other devices on the same host would have similar
capabilities, but regardless, it's not safe to assume the VM is even
running on the same host for a subsequent hot-add.

We could potentially require the vfio-pci-nohotplug variant, but this
then becomes a usability barrier that not only do we require management
tools to pick the correct device flavor in QEMU, but we lose support for
things like unplugging a device for migration and re-plugging a
replacement on the target system.

Potentially the best solution might be to clear these capability bits in
the QEMU device exit function.  I don't know if the spec specifically
disallows this, but I suspect we could get away with it from a guest
perspective.

In a similar vein, the second concern is that PCIe slots are composable
in QEMU, ie. a user could create a multi-function device where the
Atomic Ops support between devices is in conflict.  The simple solution
would be to virtualize DEVCAP2 on the device to only report Atomic Ops
capabilities when supported by the host routing (which would also
simplify the hot-plug situation), but it does not appear to be common
practice for Linux in-kernel drivers to look at this register on the
endpoint.

Maybe the solution is that we only enable Atomic Ops on the root port
for a device iff the device is at slot address zero and the
multifunction bit is clear.  Additionally the device exit function
should remove the capabilities it has enabled.

I'll work on a v2 with these changes but discussion is welcome whether
such a solution would be acceptable.  Thanks,

Alex

[1]https://lore.kernel.org/all/20230518161309.369a5d6c.alex.williamson@redhat.com/
[2]https://lore.kernel.org/all/20230519214748.402003-1-alex.williamson@redhat.com/

Alex Williamson (4):
  linux-headers: Update for vfio capability reporting AtomicOps
  vfio: Implement a common device info helper
  pcie: Add a PCIe capability version helper
  vfio/pci: Enable AtomicOps completers on root ports

 hw/pci/pcie.c                 |  7 ++++
 hw/s390x/s390-pci-vfio.c      | 37 +++-----------------
 hw/vfio/common.c              | 46 ++++++++++++++++++------
 hw/vfio/pci.c                 | 66 +++++++++++++++++++++++++++++++++++
 include/hw/pci/pcie.h         |  1 +
 include/hw/vfio/vfio-common.h |  1 +
 linux-headers/linux/vfio.h    | 14 ++++++++
 7 files changed, 129 insertions(+), 43 deletions(-)

-- 
2.39.2

[RFC PATCH 1/4] linux-headers: Update for vfio capability reporting AtomicOps

[Alex Williamson]

This is a partial linux-headers update for illustrative and testing
purposes only, NOT FOR COMMIT.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 linux-headers/linux/vfio.h | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
index 4a534edbdcba..443a8851e156 100644
--- a/linux-headers/linux/vfio.h
+++ b/linux-headers/linux/vfio.h
@@ -240,6 +240,20 @@ struct vfio_device_info {
 #define VFIO_DEVICE_INFO_CAP_ZPCI_UTIL		3
 #define VFIO_DEVICE_INFO_CAP_ZPCI_PFIP		4
 
+/*
+ * The following VFIO_DEVICE_INFO capability reports support for PCIe AtomicOp
+ * completion to the root bus with supported widths provided via flags.
+ */
+#define VFIO_DEVICE_INFO_CAP_PCI_ATOMIC_COMP	5
+struct vfio_device_info_cap_pci_atomic_comp {
+	struct vfio_info_cap_header header;
+	__u32 flags;
+#define VFIO_PCI_ATOMIC_COMP32	(1 << 0)
+#define VFIO_PCI_ATOMIC_COMP64	(1 << 1)
+#define VFIO_PCI_ATOMIC_COMP128	(1 << 2)
+	__u32 reserved;
+};
+
 /**
  * VFIO_DEVICE_GET_REGION_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 8,
  *				       struct vfio_region_info)
-- 
2.39.2

Re: [RFC PATCH 1/4] linux-headers: Update for vfio capability reporting AtomicOps

[Cédric Le Goater]

On 5/19/23 23:57, Alex Williamson wrote:
> This is a partial linux-headers update for illustrative and testing
> purposes only, NOT FOR COMMIT.
> 
> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
> ---
I am preparing a vfio-next tree including these changes plus a linux-headers
update. I am just waiting for the 6.5-rc1 tag to be pushed.

Thanks,

C.

>   linux-headers/linux/vfio.h | 14 ++++++++++++++
>   1 file changed, 14 insertions(+)
> 
> diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
> index 4a534edbdcba..443a8851e156 100644
> --- a/linux-headers/linux/vfio.h
> +++ b/linux-headers/linux/vfio.h
> @@ -240,6 +240,20 @@ struct vfio_device_info {
>   #define VFIO_DEVICE_INFO_CAP_ZPCI_UTIL		3
>   #define VFIO_DEVICE_INFO_CAP_ZPCI_PFIP		4
>   
> +/*
> + * The following VFIO_DEVICE_INFO capability reports support for PCIe AtomicOp
> + * completion to the root bus with supported widths provided via flags.
> + */
> +#define VFIO_DEVICE_INFO_CAP_PCI_ATOMIC_COMP	5
> +struct vfio_device_info_cap_pci_atomic_comp {
> +	struct vfio_info_cap_header header;
> +	__u32 flags;
> +#define VFIO_PCI_ATOMIC_COMP32	(1 << 0)
> +#define VFIO_PCI_ATOMIC_COMP64	(1 << 1)
> +#define VFIO_PCI_ATOMIC_COMP128	(1 << 2)
> +	__u32 reserved;
> +};
> +
>   /**
>    * VFIO_DEVICE_GET_REGION_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 8,
>    *				       struct vfio_region_info)

[RFC PATCH 2/4] vfio: Implement a common device info helper

[Alex Williamson]

A common helper implementing the realloc algorithm for handling
capabilities.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/s390x/s390-pci-vfio.c      | 37 ++++------------------------
 hw/vfio/common.c              | 46 ++++++++++++++++++++++++++---------
 include/hw/vfio/vfio-common.h |  1 +
 3 files changed, 41 insertions(+), 43 deletions(-)

diff --git a/hw/s390x/s390-pci-vfio.c b/hw/s390x/s390-pci-vfio.c
index f51190d4662f..66e8ec3bdef9 100644
--- a/hw/s390x/s390-pci-vfio.c
+++ b/hw/s390x/s390-pci-vfio.c
@@ -289,38 +289,11 @@ static void s390_pci_read_pfip(S390PCIBusDevice *pbdev,
     memcpy(pbdev->zpci_fn.pfip, cap->pfip, CLP_PFIP_NR_SEGMENTS);
 }
 
-static struct vfio_device_info *get_device_info(S390PCIBusDevice *pbdev,
-                                                uint32_t argsz)
+static struct vfio_device_info *get_device_info(S390PCIBusDevice *pbdev);
 {
-    struct vfio_device_info *info = g_malloc0(argsz);
-    VFIOPCIDevice *vfio_pci;
-    int fd;
+    VFIOPCIDevice *vfio_pci = container_of(pbdev->pdev, VFIOPCIDevice, pdev);
 
-    vfio_pci = container_of(pbdev->pdev, VFIOPCIDevice, pdev);
-    fd = vfio_pci->vbasedev.fd;
-
-    /*
-     * If the specified argsz is not large enough to contain all capabilities
-     * it will be updated upon return from the ioctl.  Retry until we have
-     * a big enough buffer to hold the entire capability chain.  On error,
-     * just exit and rely on CLP defaults.
-     */
-retry:
-    info->argsz = argsz;
-
-    if (ioctl(fd, VFIO_DEVICE_GET_INFO, info)) {
-        trace_s390_pci_clp_dev_info(vfio_pci->vbasedev.name);
-        g_free(info);
-        return NULL;
-    }
-
-    if (info->argsz > argsz) {
-        argsz = info->argsz;
-        info = g_realloc(info, argsz);
-        goto retry;
-    }
-
-    return info;
+    return vfio_get_device_info(vfio_pci->vbasedev.fd);
 }
 
 /*
@@ -335,7 +308,7 @@ bool s390_pci_get_host_fh(S390PCIBusDevice *pbdev, uint32_t *fh)
 
     assert(fh);
 
-    info = get_device_info(pbdev, sizeof(*info));
+    info = get_device_info(pbdev);
     if (!info) {
         return false;
     }
@@ -356,7 +329,7 @@ void s390_pci_get_clp_info(S390PCIBusDevice *pbdev)
 {
     g_autofree struct vfio_device_info *info = NULL;
 
-    info = get_device_info(pbdev, sizeof(*info));
+    info = get_device_info(pbdev);
     if (!info) {
         return;
     }
diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index 78358ede2764..ed142296e9fe 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -2843,11 +2843,35 @@ void vfio_put_group(VFIOGroup *group)
     }
 }
 
+struct vfio_device_info *vfio_get_device_info(int fd)
+{
+    struct vfio_device_info *info;
+    uint32_t argsz = sizeof(*info);
+
+    info = g_malloc0(argsz);
+
+retry:
+    info->argsz = argsz;
+
+    if (ioctl(fd, VFIO_DEVICE_GET_INFO, info)) {
+        g_free(info);
+        return NULL;
+    }
+
+    if (info->argsz > argsz) {
+        argsz = info->argsz;
+        info = g_realloc(info, argsz);
+        goto retry;
+    }
+
+    return info;
+}
+
 int vfio_get_device(VFIOGroup *group, const char *name,
                     VFIODevice *vbasedev, Error **errp)
 {
-    struct vfio_device_info dev_info = { .argsz = sizeof(dev_info) };
-    int ret, fd;
+    g_autofree struct vfio_device_info *info = NULL;
+    int fd;
 
     fd = ioctl(group->fd, VFIO_GROUP_GET_DEVICE_FD, name);
     if (fd < 0) {
@@ -2859,11 +2883,11 @@ int vfio_get_device(VFIOGroup *group, const char *name,
         return fd;
     }
 
-    ret = ioctl(fd, VFIO_DEVICE_GET_INFO, &dev_info);
-    if (ret) {
+    info = vfio_get_device_info(fd);
+    if (!info) {
         error_setg_errno(errp, errno, "error getting device info");
         close(fd);
-        return ret;
+        return -1;
     }
 
     /*
@@ -2891,14 +2915,14 @@ int vfio_get_device(VFIOGroup *group, const char *name,
     vbasedev->group = group;
     QLIST_INSERT_HEAD(&group->device_list, vbasedev, next);
 
-    vbasedev->num_irqs = dev_info.num_irqs;
-    vbasedev->num_regions = dev_info.num_regions;
-    vbasedev->flags = dev_info.flags;
+    vbasedev->num_irqs = info->num_irqs;
+    vbasedev->num_regions = info->num_regions;
+    vbasedev->flags = info->flags;
+
+    trace_vfio_get_device(name, info->flags, info->num_regions, info->num_irqs);
 
-    trace_vfio_get_device(name, dev_info.flags, dev_info.num_regions,
-                          dev_info.num_irqs);
+    vbasedev->reset_works = !!(info->flags & VFIO_DEVICE_FLAGS_RESET);
 
-    vbasedev->reset_works = !!(dev_info.flags & VFIO_DEVICE_FLAGS_RESET);
     return 0;
 }
 
diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
index eed244f25f34..a8dcda592c08 100644
--- a/include/hw/vfio/vfio-common.h
+++ b/include/hw/vfio/vfio-common.h
@@ -212,6 +212,7 @@ void vfio_region_finalize(VFIORegion *region);
 void vfio_reset_handler(void *opaque);
 VFIOGroup *vfio_get_group(int groupid, AddressSpace *as, Error **errp);
 void vfio_put_group(VFIOGroup *group);
+struct vfio_device_info *vfio_get_device_info(int fd);
 int vfio_get_device(VFIOGroup *group, const char *name,
                     VFIODevice *vbasedev, Error **errp);
 
-- 
2.39.2

Re: [RFC PATCH 2/4] vfio: Implement a common device info helper

[Philippe Mathieu-Daudé]

On 19/5/23 23:57, Alex Williamson wrote:
> A common helper implementing the realloc algorithm for handling
> capabilities.
> 
> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
> ---
>   hw/s390x/s390-pci-vfio.c      | 37 ++++------------------------
>   hw/vfio/common.c              | 46 ++++++++++++++++++++++++++---------
>   include/hw/vfio/vfio-common.h |  1 +
>   3 files changed, 41 insertions(+), 43 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Re: [RFC PATCH 2/4] vfio: Implement a common device info helper

[Cédric Le Goater]

On 5/19/23 23:57, Alex Williamson wrote:
> A common helper implementing the realloc algorithm for handling
> capabilities.
> 
> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>

Reviewed-by: Cédric Le Goater <clg@redhat.com>

This is a good independent clean up that could be merged now.

Thanks,

C.

> ---
>   hw/s390x/s390-pci-vfio.c      | 37 ++++------------------------
>   hw/vfio/common.c              | 46 ++++++++++++++++++++++++++---------
>   include/hw/vfio/vfio-common.h |  1 +
>   3 files changed, 41 insertions(+), 43 deletions(-)
> 
> diff --git a/hw/s390x/s390-pci-vfio.c b/hw/s390x/s390-pci-vfio.c
> index f51190d4662f..66e8ec3bdef9 100644
> --- a/hw/s390x/s390-pci-vfio.c
> +++ b/hw/s390x/s390-pci-vfio.c
> @@ -289,38 +289,11 @@ static void s390_pci_read_pfip(S390PCIBusDevice *pbdev,
>       memcpy(pbdev->zpci_fn.pfip, cap->pfip, CLP_PFIP_NR_SEGMENTS);
>   }
>   
> -static struct vfio_device_info *get_device_info(S390PCIBusDevice *pbdev,
> -                                                uint32_t argsz)
> +static struct vfio_device_info *get_device_info(S390PCIBusDevice *pbdev);
>   {
> -    struct vfio_device_info *info = g_malloc0(argsz);
> -    VFIOPCIDevice *vfio_pci;
> -    int fd;
> +    VFIOPCIDevice *vfio_pci = container_of(pbdev->pdev, VFIOPCIDevice, pdev);
>   
> -    vfio_pci = container_of(pbdev->pdev, VFIOPCIDevice, pdev);
> -    fd = vfio_pci->vbasedev.fd;
> -
> -    /*
> -     * If the specified argsz is not large enough to contain all capabilities
> -     * it will be updated upon return from the ioctl.  Retry until we have
> -     * a big enough buffer to hold the entire capability chain.  On error,
> -     * just exit and rely on CLP defaults.
> -     */
> -retry:
> -    info->argsz = argsz;
> -
> -    if (ioctl(fd, VFIO_DEVICE_GET_INFO, info)) {
> -        trace_s390_pci_clp_dev_info(vfio_pci->vbasedev.name);
> -        g_free(info);
> -        return NULL;
> -    }
> -
> -    if (info->argsz > argsz) {
> -        argsz = info->argsz;
> -        info = g_realloc(info, argsz);
> -        goto retry;
> -    }
> -
> -    return info;
> +    return vfio_get_device_info(vfio_pci->vbasedev.fd);
>   }
>   
>   /*
> @@ -335,7 +308,7 @@ bool s390_pci_get_host_fh(S390PCIBusDevice *pbdev, uint32_t *fh)
>   
>       assert(fh);
>   
> -    info = get_device_info(pbdev, sizeof(*info));
> +    info = get_device_info(pbdev);
>       if (!info) {
>           return false;
>       }
> @@ -356,7 +329,7 @@ void s390_pci_get_clp_info(S390PCIBusDevice *pbdev)
>   {
>       g_autofree struct vfio_device_info *info = NULL;
>   
> -    info = get_device_info(pbdev, sizeof(*info));
> +    info = get_device_info(pbdev);
>       if (!info) {
>           return;
>       }
> diff --git a/hw/vfio/common.c b/hw/vfio/common.c
> index 78358ede2764..ed142296e9fe 100644
> --- a/hw/vfio/common.c
> +++ b/hw/vfio/common.c
> @@ -2843,11 +2843,35 @@ void vfio_put_group(VFIOGroup *group)
>       }
>   }
>   
> +struct vfio_device_info *vfio_get_device_info(int fd)
> +{
> +    struct vfio_device_info *info;
> +    uint32_t argsz = sizeof(*info);
> +
> +    info = g_malloc0(argsz);
> +
> +retry:
> +    info->argsz = argsz;
> +
> +    if (ioctl(fd, VFIO_DEVICE_GET_INFO, info)) {
> +        g_free(info);
> +        return NULL;
> +    }
> +
> +    if (info->argsz > argsz) {
> +        argsz = info->argsz;
> +        info = g_realloc(info, argsz);
> +        goto retry;
> +    }
> +
> +    return info;
> +}
> +
>   int vfio_get_device(VFIOGroup *group, const char *name,
>                       VFIODevice *vbasedev, Error **errp)
>   {
> -    struct vfio_device_info dev_info = { .argsz = sizeof(dev_info) };
> -    int ret, fd;
> +    g_autofree struct vfio_device_info *info = NULL;
> +    int fd;
>   
>       fd = ioctl(group->fd, VFIO_GROUP_GET_DEVICE_FD, name);
>       if (fd < 0) {
> @@ -2859,11 +2883,11 @@ int vfio_get_device(VFIOGroup *group, const char *name,
>           return fd;
>       }
>   
> -    ret = ioctl(fd, VFIO_DEVICE_GET_INFO, &dev_info);
> -    if (ret) {
> +    info = vfio_get_device_info(fd);
> +    if (!info) {
>           error_setg_errno(errp, errno, "error getting device info");
>           close(fd);
> -        return ret;
> +        return -1;
>       }
>   
>       /*
> @@ -2891,14 +2915,14 @@ int vfio_get_device(VFIOGroup *group, const char *name,
>       vbasedev->group = group;
>       QLIST_INSERT_HEAD(&group->device_list, vbasedev, next);
>   
> -    vbasedev->num_irqs = dev_info.num_irqs;
> -    vbasedev->num_regions = dev_info.num_regions;
> -    vbasedev->flags = dev_info.flags;
> +    vbasedev->num_irqs = info->num_irqs;
> +    vbasedev->num_regions = info->num_regions;
> +    vbasedev->flags = info->flags;
> +
> +    trace_vfio_get_device(name, info->flags, info->num_regions, info->num_irqs);
>   
> -    trace_vfio_get_device(name, dev_info.flags, dev_info.num_regions,
> -                          dev_info.num_irqs);
> +    vbasedev->reset_works = !!(info->flags & VFIO_DEVICE_FLAGS_RESET);
>   
> -    vbasedev->reset_works = !!(dev_info.flags & VFIO_DEVICE_FLAGS_RESET);
>       return 0;
>   }
>   
> diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
> index eed244f25f34..a8dcda592c08 100644
> --- a/include/hw/vfio/vfio-common.h
> +++ b/include/hw/vfio/vfio-common.h
> @@ -212,6 +212,7 @@ void vfio_region_finalize(VFIORegion *region);
>   void vfio_reset_handler(void *opaque);
>   VFIOGroup *vfio_get_group(int groupid, AddressSpace *as, Error **errp);
>   void vfio_put_group(VFIOGroup *group);
> +struct vfio_device_info *vfio_get_device_info(int fd);
>   int vfio_get_device(VFIOGroup *group, const char *name,
>                       VFIODevice *vbasedev, Error **errp);
>   

[RFC PATCH 3/4] pcie: Add a PCIe capability version helper

[Alex Williamson]

Report the PCIe capability version for a device

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/pci/pcie.c         | 7 +++++++
 include/hw/pci/pcie.h | 1 +
 2 files changed, 8 insertions(+)

diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c
index b8c24cf45f7e..b7f107ed8dd4 100644
--- a/hw/pci/pcie.c
+++ b/hw/pci/pcie.c
@@ -274,6 +274,13 @@ uint8_t pcie_cap_get_type(const PCIDevice *dev)
             PCI_EXP_FLAGS_TYPE) >> PCI_EXP_FLAGS_TYPE_SHIFT;
 }
 
+uint8_t pcie_cap_get_version(const PCIDevice *dev)
+{
+    uint32_t pos = dev->exp.exp_cap;
+    assert(pos > 0);
+    return pci_get_word(dev->config + pos + PCI_EXP_FLAGS) & PCI_EXP_FLAGS_VERS;
+}
+
 /* MSI/MSI-X */
 /* pci express interrupt message number */
 /* 7.8.2 PCI Express Capabilities Register: Interrupt Message Number */
diff --git a/include/hw/pci/pcie.h b/include/hw/pci/pcie.h
index 3cc2b159570f..51ab57bc3c50 100644
--- a/include/hw/pci/pcie.h
+++ b/include/hw/pci/pcie.h
@@ -93,6 +93,7 @@ void pcie_cap_exit(PCIDevice *dev);
 int pcie_endpoint_cap_v1_init(PCIDevice *dev, uint8_t offset);
 void pcie_cap_v1_exit(PCIDevice *dev);
 uint8_t pcie_cap_get_type(const PCIDevice *dev);
+uint8_t pcie_cap_get_version(const PCIDevice *dev);
 void pcie_cap_flags_set_vector(PCIDevice *dev, uint8_t vector);
 uint8_t pcie_cap_flags_get_vector(PCIDevice *dev);
 
-- 
2.39.2

[RFC PATCH 4/4] vfio/pci: Enable AtomicOps completers on root ports

[Alex Williamson]

For cold-plug configurations where the vfio-pci device reports enabled
support for PCIe AtomicOps completers and we have a compatible VM
configuration, mirror the host AtomicOps completer support in the virtual
root port.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/pci.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index bf27a3990564..61d045785bf8 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -1826,6 +1826,70 @@ static void vfio_add_emulated_long(VFIOPCIDevice *vdev, int pos,
     vfio_set_long_bits(vdev->emulated_config_bits + pos, mask, mask);
 }
 
+/*
+ * For a cold-plugged device connected directly to a root port where
+ * vfio-pci reports AtomicOp completer support on the host, twiddle
+ * bits in the emulated root port to reflect this capability.
+ */
+static void vfio_pci_enable_rp_atomics(VFIOPCIDevice *vdev)
+{
+    struct vfio_device_info_cap_pci_atomic_comp *cap;
+    g_autofree struct vfio_device_info *info = NULL;
+    struct vfio_info_cap_header *hdr;
+    PCIDevice *parent;
+    uint32_t mask = 0;
+    PCIBus *bus;
+
+    /*
+     * XXX
+     *  - Should this also be restricted to TYPE_VFIO_PCI_NOHOTPLUG devices?
+     *  - Should we clear the capability on exit?
+     */
+    if (vdev->pdev.qdev.hotplugged) {
+        return;
+    }
+
+    info = vfio_get_device_info(vdev->vbasedev.fd);
+    if (!info) {
+        return;
+    }
+
+    hdr = vfio_get_device_info_cap(info, VFIO_DEVICE_INFO_CAP_PCI_ATOMIC_COMP);
+    if (!hdr) {
+        return;
+    }
+
+    cap = (void *)hdr;
+    if (cap->flags & VFIO_PCI_ATOMIC_COMP32) {
+        mask |= PCI_EXP_DEVCAP2_ATOMIC_COMP32;
+    }
+    if (cap->flags & VFIO_PCI_ATOMIC_COMP64) {
+        mask |= PCI_EXP_DEVCAP2_ATOMIC_COMP64;
+    }
+    if (cap->flags & VFIO_PCI_ATOMIC_COMP128) {
+        mask |= PCI_EXP_DEVCAP2_ATOMIC_COMP128;
+    }
+
+    if (!mask) {
+        return;
+    }
+
+    bus = pci_get_bus(&vdev->pdev);
+    parent = bus->parent_dev;
+
+    if (!parent || !pci_is_express(parent) || !parent->exp.exp_cap) {
+        return;
+    }
+
+    if (pcie_cap_get_type(parent) != PCI_EXP_TYPE_ROOT_PORT ||
+        pcie_cap_get_version(parent) != PCI_EXP_FLAGS_VER2) {
+        return;
+    }
+
+    pci_long_test_and_set_mask(parent->config + parent->exp.exp_cap +
+                               PCI_EXP_DEVCAP2, mask);
+}
+
 static int vfio_setup_pcie_cap(VFIOPCIDevice *vdev, int pos, uint8_t size,
                                Error **errp)
 {
@@ -1929,6 +1993,8 @@ static int vfio_setup_pcie_cap(VFIOPCIDevice *vdev, int pos, uint8_t size,
                            QEMU_PCI_EXP_LNKCAP_MLS(QEMU_PCI_EXP_LNK_2_5GT), ~0);
             vfio_add_emulated_word(vdev, pos + PCI_EXP_LNKCTL, 0, ~0);
         }
+
+        vfio_pci_enable_rp_atomics(vdev);
     }
 
     /*
-- 
2.39.2