From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60328) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XMh9f-0003NL-QG for qemu-devel@nongnu.org; Wed, 27 Aug 2014 13:37:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XMh9X-0002s8-Mm for qemu-devel@nongnu.org; Wed, 27 Aug 2014 13:37:07 -0400 Received: from e38.co.us.ibm.com ([32.97.110.159]:59822) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XMh9X-0002rx-Fg for qemu-devel@nongnu.org; Wed, 27 Aug 2014 13:36:59 -0400 Received: from /spool/local by e38.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 27 Aug 2014 11:36:58 -0600 From: Michael Roth Date: Wed, 27 Aug 2014 12:36:00 -0500 Message-Id: <1409160982-16389-4-git-send-email-mdroth@linux.vnet.ibm.com> In-Reply-To: <1409160982-16389-1-git-send-email-mdroth@linux.vnet.ibm.com> References: <1409160982-16389-1-git-send-email-mdroth@linux.vnet.ibm.com> Subject: [Qemu-devel] [PATCH 03/25] vfio: Fix MSI-X vector expansion List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: qemu-stable@nongnu.org From: Alex Williamson When new MSI-X vectors are enabled we need to disable MSI-X and re-enable it with the correct number of vectors. That means we need to reprogram the eventfd triggers for each vector. Prior to f4d45d47 vector->use tracked whether a vector was masked or unmasked and we could always pick the KVM path when available for unmasked vectors. Now vfio doesn't track mask state itself and vector->use and virq remains configured even for masked vectors. Therefore we need to ask the MSI-X code whether a vector is masked in order to select the correct signaling path. As noted in the comment, MSI relies on hardware to handle masking. Signed-off-by: Alex Williamson Cc: qemu-stable@nongnu.org # QEMU 2.1 (cherry picked from commit c048be5cc92ae201c339d46984476c4629275ed6) Signed-off-by: Michael Roth --- hw/misc/vfio.c | 38 +++++++++++++++++++++++++++++--------- 1 file changed, 29 insertions(+), 9 deletions(-) diff --git a/hw/misc/vfio.c b/hw/misc/vfio.c index 0b9eba0..e88b610 100644 --- a/hw/misc/vfio.c +++ b/hw/misc/vfio.c @@ -120,11 +120,20 @@ typedef struct VFIOINTx { } VFIOINTx; typedef struct VFIOMSIVector { - EventNotifier interrupt; /* eventfd triggered on interrupt */ - EventNotifier kvm_interrupt; /* eventfd triggered for KVM irqfd bypass */ + /* + * Two interrupt paths are configured per vector. The first, is only used + * for interrupts injected via QEMU. This is typically the non-accel path, + * but may also be used when we want QEMU to handle masking and pending + * bits. The KVM path bypasses QEMU and is therefore higher performance, + * but requires masking at the device. virq is used to track the MSI route + * through KVM, thus kvm_interrupt is only available when virq is set to a + * valid (>= 0) value. + */ + EventNotifier interrupt; + EventNotifier kvm_interrupt; struct VFIODevice *vdev; /* back pointer to device */ MSIMessage msg; /* cache the MSI message so we know when it changes */ - int virq; /* KVM irqchip route for QEMU bypass */ + int virq; bool use; } VFIOMSIVector; @@ -681,13 +690,24 @@ static int vfio_enable_vectors(VFIODevice *vdev, bool msix) fds = (int32_t *)&irq_set->data; for (i = 0; i < vdev->nr_vectors; i++) { - if (!vdev->msi_vectors[i].use) { - fds[i] = -1; - } else if (vdev->msi_vectors[i].virq >= 0) { - fds[i] = event_notifier_get_fd(&vdev->msi_vectors[i].kvm_interrupt); - } else { - fds[i] = event_notifier_get_fd(&vdev->msi_vectors[i].interrupt); + int fd = -1; + + /* + * MSI vs MSI-X - The guest has direct access to MSI mask and pending + * bits, therefore we always use the KVM signaling path when setup. + * MSI-X mask and pending bits are emulated, so we want to use the + * KVM signaling path only when configured and unmasked. + */ + if (vdev->msi_vectors[i].use) { + if (vdev->msi_vectors[i].virq < 0 || + (msix && msix_is_masked(&vdev->pdev, i))) { + fd = event_notifier_get_fd(&vdev->msi_vectors[i].interrupt); + } else { + fd = event_notifier_get_fd(&vdev->msi_vectors[i].kvm_interrupt); + } } + + fds[i] = fd; } ret = ioctl(vdev->fd, VFIO_DEVICE_SET_IRQS, irq_set); -- 1.9.1