[Qemu-devel] [PULL 0/4] vfio: Updates for QEMU 2.3-rc0

[Qemu-devel] [PULL 0/4] vfio: Updates for QEMU 2.3-rc0

[Alex Williamson]

The following changes since commit b8a173b25c887a606681fc35a46702c164d5b2d0:

  Merge remote-tracking branch 'remotes/ehabkost/tags/x86-pull-request' into staging (2015-03-02 14:25:48 +0000)

are available in the git repository at:


  git://github.com/awilliam/qemu-vfio.git tags/vfio-update-20150302.0

for you to fetch changes up to 47cbe50cc8d8e59129311bcdb827e1116e935bde:

  vfio-pci: Enable device request notification support (2015-03-02 11:38:55 -0700)

----------------------------------------------------------------
Updates for QEMU 2.3-rc0:
 - Error reporting and static cleanup (Alexey Kardashevskiy)
 - Runtime mmap disable for tracing (Samuel Pitoiset)
 - Support for host directed device request (Alex Williamson)

----------------------------------------------------------------
Alex Williamson (1):
      vfio-pci: Enable device request notification support

Alexey Kardashevskiy (2):
      vfio: Add ioctl number to error report
      vfio: Make type1 listener symbols static

Samuel Pitoiset (1):
      vfio: allow to disable MMAP per device with -x-mmap=off option

 hw/vfio/common.c              |  10 ++---
 hw/vfio/pci.c                 | 101 ++++++++++++++++++++++++++++++++++++++++++
 include/hw/vfio/vfio-common.h |   4 +-
 linux-headers/linux/vfio.h    |   1 +
 4 files changed, 108 insertions(+), 8 deletions(-)

[Qemu-devel] [PULL 1/4] vfio: Add ioctl number to error report

[Alex Williamson]

From: Alexey Kardashevskiy <aik@ozlabs.ru>

This makes the error report more informative.

Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/common.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index c5d1551..c9df08d 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -932,8 +932,8 @@ static int vfio_container_do_ioctl(AddressSpace *as, int32_t groupid,
     if (group->container) {
         ret = ioctl(container->fd, req, param);
         if (ret < 0) {
-            error_report("vfio: failed to ioctl container: ret=%d, %s",
-                         ret, strerror(errno));
+            error_report("vfio: failed to ioctl %d to container: ret=%d, %s",
+                         _IOC_NR(req) - VFIO_BASE, ret, strerror(errno));
         }
     }
 

[Qemu-devel] [PULL 2/4] vfio: Make type1 listener symbols static

[Alex Williamson]

From: Alexey Kardashevskiy <aik@ozlabs.ru>

They are not used from anywhere but common.c which is where these are
defined so make them static.

Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/common.c              |    4 ++--
 include/hw/vfio/vfio-common.h |    2 --
 2 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index c9df08d..4ff8cab 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -475,12 +475,12 @@ static void vfio_listener_region_del(MemoryListener *listener,
     }
 }
 
-const MemoryListener vfio_memory_listener = {
+static const MemoryListener vfio_memory_listener = {
     .region_add = vfio_listener_region_add,
     .region_del = vfio_listener_region_del,
 };
 
-void vfio_listener_release(VFIOContainer *container)
+static void vfio_listener_release(VFIOContainer *container)
 {
     memory_listener_unregister(&container->iommu_data.type1.listener);
 }
diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
index 5f3679b..3d3892c 100644
--- a/include/hw/vfio/vfio-common.h
+++ b/include/hw/vfio/vfio-common.h
@@ -131,7 +131,6 @@ void vfio_region_write(void *opaque, hwaddr addr,
                            uint64_t data, unsigned size);
 uint64_t vfio_region_read(void *opaque,
                           hwaddr addr, unsigned size);
-void vfio_listener_release(VFIOContainer *container);
 int vfio_mmap_region(Object *vdev, VFIORegion *region,
                      MemoryRegion *mem, MemoryRegion *submem,
                      void **map, size_t size, off_t offset,
@@ -143,7 +142,6 @@ int vfio_get_device(VFIOGroup *group, const char *name,
                     VFIODevice *vbasedev);
 
 extern const MemoryRegionOps vfio_region_ops;
-extern const MemoryListener vfio_memory_listener;
 extern QLIST_HEAD(vfio_group_head, VFIOGroup) vfio_group_list;
 extern QLIST_HEAD(vfio_as_head, VFIOAddressSpace) vfio_address_spaces;
 

[Qemu-devel] [PULL 3/4] vfio: allow to disable MMAP per device with -x-mmap=off option

[Alex Williamson]

From: Samuel Pitoiset <samuel.pitoiset@gmail.com>

Disabling MMAP support uses the slower read/write accesses but allows to
trace all MMIO accesses, which is not good for performance, but very
useful for reverse engineering PCI drivers. This option allows to
disable MMAP per device without a compile-time change.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/common.c              |    2 +-
 hw/vfio/pci.c                 |    1 +
 include/hw/vfio/vfio-common.h |    2 +-
 3 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index 4ff8cab..9db7d8d 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -493,7 +493,7 @@ int vfio_mmap_region(Object *obj, VFIORegion *region,
     int ret = 0;
     VFIODevice *vbasedev = region->vbasedev;
 
-    if (VFIO_ALLOW_MMAP && size && region->flags &
+    if (vbasedev->allow_mmap && size && region->flags &
         VFIO_REGION_INFO_FLAG_MMAP) {
         int prot = 0;
 
diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 84e9d99..3c71de3 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -3456,6 +3456,7 @@ static Property vfio_pci_dev_properties[] = {
     DEFINE_PROP_BIT("x-vga", VFIOPCIDevice, features,
                     VFIO_FEATURE_ENABLE_VGA_BIT, false),
     DEFINE_PROP_INT32("bootindex", VFIOPCIDevice, bootindex, -1),
+    DEFINE_PROP_BOOL("x-mmap", VFIOPCIDevice, vbasedev.allow_mmap, true),
     /*
      * TODO - support passed fds... is this necessary?
      * DEFINE_PROP_STRING("vfiofd", VFIOPCIDevice, vfiofd_name),
diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
index 3d3892c..0d1fb80 100644
--- a/include/hw/vfio/vfio-common.h
+++ b/include/hw/vfio/vfio-common.h
@@ -36,7 +36,6 @@
 #endif
 
 /* Extra debugging, trap acceleration paths for more logging */
-#define VFIO_ALLOW_MMAP 1
 #define VFIO_ALLOW_KVM_INTX 1
 #define VFIO_ALLOW_KVM_MSI 1
 #define VFIO_ALLOW_KVM_MSIX 1
@@ -102,6 +101,7 @@ typedef struct VFIODevice {
     int type;
     bool reset_works;
     bool needs_reset;
+    bool allow_mmap;
     VFIODeviceOps *ops;
     unsigned int num_irqs;
     unsigned int num_regions;

[Qemu-devel] [PULL 4/4] vfio-pci: Enable device request notification support

[Alex Williamson]

Linux v4.0-rc1 vfio-pci introduced a new virtual interrupt to allow
the kernel to request a device from the user.  When signaled, QEMU
will by default attmempt to hot-unplug the device.  This is a one-
shot attempt with the expectation that the kernel will continue to
poll for the device if it is not returned.  Returning the device when
requested is the expected standard model of cooperative usage, but we
also add an option option to disable this feature.  Initially this
opt-out is set as an experimental option because we really should
honor kernel requests for the device.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/pci.c              |  100 ++++++++++++++++++++++++++++++++++++++++++++
 linux-headers/linux/vfio.h |    1 
 2 files changed, 101 insertions(+)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 3c71de3..6b80539 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -153,13 +153,17 @@ typedef struct VFIOPCIDevice {
     VFIOVGA vga; /* 0xa0000, 0x3b0, 0x3c0 */
     PCIHostDeviceAddress host;
     EventNotifier err_notifier;
+    EventNotifier req_notifier;
     uint32_t features;
 #define VFIO_FEATURE_ENABLE_VGA_BIT 0
 #define VFIO_FEATURE_ENABLE_VGA (1 << VFIO_FEATURE_ENABLE_VGA_BIT)
+#define VFIO_FEATURE_ENABLE_REQ_BIT 1
+#define VFIO_FEATURE_ENABLE_REQ (1 << VFIO_FEATURE_ENABLE_REQ_BIT)
     int32_t bootindex;
     uint8_t pm_cap;
     bool has_vga;
     bool pci_aer;
+    bool req_enabled;
     bool has_flr;
     bool has_pm_reset;
     bool rom_read_failed;
@@ -3088,6 +3092,7 @@ static int vfio_populate_device(VFIOPCIDevice *vdev)
 
         vdev->has_vga = true;
     }
+
     irq_info.index = VFIO_PCI_ERR_IRQ_INDEX;
 
     ret = ioctl(vdev->vbasedev.fd, VFIO_DEVICE_GET_IRQ_INFO, &irq_info);
@@ -3223,6 +3228,97 @@ static void vfio_unregister_err_notifier(VFIOPCIDevice *vdev)
     event_notifier_cleanup(&vdev->err_notifier);
 }
 
+static void vfio_req_notifier_handler(void *opaque)
+{
+    VFIOPCIDevice *vdev = opaque;
+
+    if (!event_notifier_test_and_clear(&vdev->req_notifier)) {
+        return;
+    }
+
+    qdev_unplug(&vdev->pdev.qdev, NULL);
+}
+
+static void vfio_register_req_notifier(VFIOPCIDevice *vdev)
+{
+    struct vfio_irq_info irq_info = { .argsz = sizeof(irq_info),
+                                      .index = VFIO_PCI_REQ_IRQ_INDEX };
+    int argsz;
+    struct vfio_irq_set *irq_set;
+    int32_t *pfd;
+
+    if (!(vdev->features & VFIO_FEATURE_ENABLE_REQ)) {
+        return;
+    }
+
+    if (ioctl(vdev->vbasedev.fd,
+              VFIO_DEVICE_GET_IRQ_INFO, &irq_info) < 0 || irq_info.count < 1) {
+        return;
+    }
+
+    if (event_notifier_init(&vdev->req_notifier, 0)) {
+        error_report("vfio: Unable to init event notifier for device request");
+        return;
+    }
+
+    argsz = sizeof(*irq_set) + sizeof(*pfd);
+
+    irq_set = g_malloc0(argsz);
+    irq_set->argsz = argsz;
+    irq_set->flags = VFIO_IRQ_SET_DATA_EVENTFD |
+                     VFIO_IRQ_SET_ACTION_TRIGGER;
+    irq_set->index = VFIO_PCI_REQ_IRQ_INDEX;
+    irq_set->start = 0;
+    irq_set->count = 1;
+    pfd = (int32_t *)&irq_set->data;
+
+    *pfd = event_notifier_get_fd(&vdev->req_notifier);
+    qemu_set_fd_handler(*pfd, vfio_req_notifier_handler, NULL, vdev);
+
+    if (ioctl(vdev->vbasedev.fd, VFIO_DEVICE_SET_IRQS, irq_set)) {
+        error_report("vfio: Failed to set up device request notification");
+        qemu_set_fd_handler(*pfd, NULL, NULL, vdev);
+        event_notifier_cleanup(&vdev->req_notifier);
+    } else {
+        vdev->req_enabled = true;
+    }
+
+    g_free(irq_set);
+}
+
+static void vfio_unregister_req_notifier(VFIOPCIDevice *vdev)
+{
+    int argsz;
+    struct vfio_irq_set *irq_set;
+    int32_t *pfd;
+
+    if (!vdev->req_enabled) {
+        return;
+    }
+
+    argsz = sizeof(*irq_set) + sizeof(*pfd);
+
+    irq_set = g_malloc0(argsz);
+    irq_set->argsz = argsz;
+    irq_set->flags = VFIO_IRQ_SET_DATA_EVENTFD |
+                     VFIO_IRQ_SET_ACTION_TRIGGER;
+    irq_set->index = VFIO_PCI_REQ_IRQ_INDEX;
+    irq_set->start = 0;
+    irq_set->count = 1;
+    pfd = (int32_t *)&irq_set->data;
+    *pfd = -1;
+
+    if (ioctl(vdev->vbasedev.fd, VFIO_DEVICE_SET_IRQS, irq_set)) {
+        error_report("vfio: Failed to de-assign device request fd: %m");
+    }
+    g_free(irq_set);
+    qemu_set_fd_handler(event_notifier_get_fd(&vdev->req_notifier),
+                        NULL, NULL, vdev);
+    event_notifier_cleanup(&vdev->req_notifier);
+
+    vdev->req_enabled = false;
+}
+
 static int vfio_initfn(PCIDevice *pdev)
 {
     VFIOPCIDevice *vdev = DO_UPCAST(VFIOPCIDevice, pdev, pdev);
@@ -3370,6 +3466,7 @@ static int vfio_initfn(PCIDevice *pdev)
     }
 
     vfio_register_err_notifier(vdev);
+    vfio_register_req_notifier(vdev);
 
     return 0;
 
@@ -3397,6 +3494,7 @@ static void vfio_exitfn(PCIDevice *pdev)
 {
     VFIOPCIDevice *vdev = DO_UPCAST(VFIOPCIDevice, pdev, pdev);
 
+    vfio_unregister_req_notifier(vdev);
     vfio_unregister_err_notifier(vdev);
     pci_device_set_intx_routing_notifier(&vdev->pdev, NULL);
     vfio_disable_interrupts(vdev);
@@ -3455,6 +3553,8 @@ static Property vfio_pci_dev_properties[] = {
                        intx.mmap_timeout, 1100),
     DEFINE_PROP_BIT("x-vga", VFIOPCIDevice, features,
                     VFIO_FEATURE_ENABLE_VGA_BIT, false),
+    DEFINE_PROP_BIT("x-req", VFIOPCIDevice, features,
+                    VFIO_FEATURE_ENABLE_REQ_BIT, true),
     DEFINE_PROP_INT32("bootindex", VFIOPCIDevice, bootindex, -1),
     DEFINE_PROP_BOOL("x-mmap", VFIOPCIDevice, vbasedev.allow_mmap, true),
     /*
diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
index 0f21aa6..95ba870 100644
--- a/linux-headers/linux/vfio.h
+++ b/linux-headers/linux/vfio.h
@@ -333,6 +333,7 @@ enum {
 	VFIO_PCI_MSI_IRQ_INDEX,
 	VFIO_PCI_MSIX_IRQ_INDEX,
 	VFIO_PCI_ERR_IRQ_INDEX,
+	VFIO_PCI_REQ_IRQ_INDEX,
 	VFIO_PCI_NUM_IRQS
 };
 

Re: [Qemu-devel] [PULL 0/4] vfio: Updates for QEMU 2.3-rc0

[Peter Maydell]

On 3 March 2015 at 04:03, Alex Williamson <alex.williamson@redhat.com> wrote:
> The following changes since commit b8a173b25c887a606681fc35a46702c164d5b2d0:
>
>   Merge remote-tracking branch 'remotes/ehabkost/tags/x86-pull-request' into staging (2015-03-02 14:25:48 +0000)
>
> are available in the git repository at:
>
>
>   git://github.com/awilliam/qemu-vfio.git tags/vfio-update-20150302.0
>
> for you to fetch changes up to 47cbe50cc8d8e59129311bcdb827e1116e935bde:
>
>   vfio-pci: Enable device request notification support (2015-03-02 11:38:55 -0700)
>
> ----------------------------------------------------------------
> Updates for QEMU 2.3-rc0:
>  - Error reporting and static cleanup (Alexey Kardashevskiy)
>  - Runtime mmap disable for tracing (Samuel Pitoiset)
>  - Support for host directed device request (Alex Williamson)
>
> ----------------------------------------------------------------

Applied, thanks.

-- PMM