Re: [BUG] pci-passthrough generates "xen:events: Failed to obtain physical IRQ" for some devices

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

[-- Attachment #1: Type: text/plain, Size: 977 bytes --]

On Wed, Feb 03, 2016 at 03:22:30PM +0100, Marek Marczykowski-Górecki wrote:
> On Mon, Feb 01, 2016 at 09:50:53AM -0500, Konrad Rzeszutek Wilk wrote:
> > > The second bullet looks at first pretty interesting from this PoV,
> > > see http://xenbits.xen.org/xsa/advisory-157.html for info on the XSA and
> > > the various patches. Konrad is on the CC already so hopefully he has some
> > > ideas.
> > 
> > Thanks. I will try to reproduce this with the upstream kernel first as
> > those patches are there.
> 
> According to one Qubes OS user report[1], the bug was introduced between
> version, which differs only by XSA-155 patches (including one for
> pciback), especially not XSA-157. 
> Maybe on some code path, some value is not copied back to pdev->sh_info->op?

I found two bugs (attached the draft not-compiled patches). Upstream
wise I seem to be tripping over another issue.

There is also some more work required in there to fix the MSI-x enable op.

[-- Attachment #2: 0001-xen-pciback-Check-PF-instead-of-VF-for-PCI_COMMAND_M.patch --]
[-- Type: text/plain, Size: 1737 bytes --]

>From 34c754a209747826336f6e1f0477a55d214fcc28 Mon Sep 17 00:00:00 2001
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Date: Wed, 3 Feb 2016 10:18:18 -0500
Subject: [PATCH 1/2] xen/pciback: Check PF instead of VF for
 PCI_COMMAND_MEMORY

c/s 408fb0e5aa7fda0059db282ff58c3b2a4278baa0
"xen/pciback: Don't allow MSI-X ops if PCI_COMMAND_MEMORY is not set."
would check the device for PCI_COMMAND_MEMORY which is great.
Except that VF devices are unique - for example they have no
legacy interrupts, and also any writes to PCI_COMMAND_MEMORY
are silently ignored (by the hardware).

CC: stable@vger.kernel.org
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 drivers/xen/xen-pciback/pciback_ops.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/xen/xen-pciback/pciback_ops.c b/drivers/xen/xen-pciback/pciback_ops.c
index 73dafdc..8c86a53 100644
--- a/drivers/xen/xen-pciback/pciback_ops.c
+++ b/drivers/xen/xen-pciback/pciback_ops.c
@@ -213,6 +213,7 @@ int xen_pcibk_enable_msix(struct xen_pcibk_device *pdev,
 	int i, result;
 	struct msix_entry *entries;
 	u16 cmd;
+	struct pci_dev *phys_dev;
 
 	if (unlikely(verbose_request))
 		printk(KERN_DEBUG DRV_NAME ": %s: enable MSI-X\n",
@@ -227,8 +228,10 @@ int xen_pcibk_enable_msix(struct xen_pcibk_device *pdev,
 	/*
 	 * PCI_COMMAND_MEMORY must be enabled, otherwise we may not be able
 	 * to access the BARs where the MSI-X entries reside.
+	 * But VF devices are unique in which the PF needs to be checked.
 	 */
-	pci_read_config_word(dev, PCI_COMMAND, &cmd);
+	phys_dev = pci_physfn(dev);
+	pci_read_config_word(phys_dev, PCI_COMMAND, &cmd);
 	if (dev->msi_enabled || !(cmd & PCI_COMMAND_MEMORY))
 		return -ENXIO;
 
-- 
2.1.0


[-- Attachment #3: 0002-xen-pciback-Save-the-number-of-MSI-X-entries-to-be-c.patch --]
[-- Type: text/plain, Size: 2010 bytes --]

>From 0d83766e257e3bfb5ecd7483e2263ef89a986f4a Mon Sep 17 00:00:00 2001
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Date: Wed, 3 Feb 2016 10:22:02 -0500
Subject: [PATCH 2/2] xen/pciback: Save the number of MSI-X entries to be
 copied later.

c/s  8135cf8b092723dbfcc611fe6fdcb3a36c9951c5
"xen/pciback: Save xen_pci_op commands before processing it"
would copyback the processed values - which was great.

However it missed the case that xen_pcibk_enable_msix - when
completing would overwrite op->value (which had the number
of MSI-X vectors requested) with the return value (which for
success was zero). Hence the copy-back routine (which would use
op->value) would copy exactly zero MSI-X vectors back.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 drivers/xen/xen-pciback/pciback_ops.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/xen/xen-pciback/pciback_ops.c b/drivers/xen/xen-pciback/pciback_ops.c
index 8c86a53..2fc5880 100644
--- a/drivers/xen/xen-pciback/pciback_ops.c
+++ b/drivers/xen/xen-pciback/pciback_ops.c
@@ -335,7 +335,9 @@ void xen_pcibk_do_op(struct work_struct *data)
 	struct xen_pcibk_dev_data *dev_data = NULL;
 	struct xen_pci_op *op = &pdev->op;
 	int test_intx = 0;
-
+#ifdef CONFIG_PCI_MSI
+	unsigned int nr = 0;
+#endif
 	*op = pdev->sh_info->op;
 	barrier();
 	dev = xen_pcibk_get_pci_dev(pdev, op->domain, op->bus, op->devfn);
@@ -363,6 +365,7 @@ void xen_pcibk_do_op(struct work_struct *data)
 			op->err = xen_pcibk_disable_msi(pdev, dev, op);
 			break;
 		case XEN_PCI_OP_enable_msix:
+			nr = op->value;
 			op->err = xen_pcibk_enable_msix(pdev, dev, op);
 			break;
 		case XEN_PCI_OP_disable_msix:
@@ -385,7 +388,7 @@ void xen_pcibk_do_op(struct work_struct *data)
 	if (op->cmd == XEN_PCI_OP_enable_msix && op->err == 0) {
 		unsigned int i;
 
-		for (i = 0; i < op->value; i++)
+		for (i = 0; i < nr; i++)
 			pdev->sh_info->op.msix_entries[i].vector =
 				op->msix_entries[i].vector;
 	}
-- 
2.1.0


[-- Attachment #4: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel