[Qemu-devel] [PATCH for-2.8] pci-assign: sync MSI/MSI-X cap and table with PCIDevice

[Qemu-devel] [PATCH for-2.8] pci-assign: sync MSI/MSI-X cap and table with PCIDevice

[Peter Xu]

Since commit e1d4fb2d ("kvm-irqchip: x86: add msi route notify fn"),
kvm_irqchip_add_msi_route() starts to use pci_get_msi_message() to fetch
MSI info. This requires that we setup MSI related fields in PCIDevice.
For most devices, that won't be a problem, as long as we are using
general interfaces like msi_init()/msix_init().

However, for pci-assign devices, MSI/MSI-X is treated differently - PCI
assign devices are maintaining its own MSI table and cap information in
AssignedDevice struct. however that's not synced up with PCIDevice's
fields. That will leads to pci_get_msi_message() failed to find correct
MSI capability, even with an NULL msix_table.

A quick fix is to sync up the two places: both the capability bits and
table address for MSI/MSI-X.

Reported-by: Changlimin <changlimin@h3c.com>
Cc: qemu-stable@nongnu.org
Fixes: e1d4fb2d ("kvm-irqchip: x86: add msi route notify fn")
Signed-off-by: Peter Xu <peterx@redhat.com>

---
Do we still support pci-assign?
---
 hw/i386/kvm/pci-assign.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/hw/i386/kvm/pci-assign.c b/hw/i386/kvm/pci-assign.c
index 8238fbc..df9748d 100644
--- a/hw/i386/kvm/pci-assign.c
+++ b/hw/i386/kvm/pci-assign.c
@@ -1251,6 +1251,7 @@ static int assigned_device_pci_cap_init(PCIDevice *pci_dev, Error **errp)
             error_propagate(errp, local_err);
             return -ENOTSUP;
         }
+        dev->dev.cap_present |= QEMU_PCI_CAP_MSI;
         dev->cap.available |= ASSIGNED_DEVICE_CAP_MSI;
         /* Only 32-bit/no-mask currently supported */
         ret = pci_add_capability2(pci_dev, PCI_CAP_ID_MSI, pos, 10,
@@ -1285,6 +1286,7 @@ static int assigned_device_pci_cap_init(PCIDevice *pci_dev, Error **errp)
             error_propagate(errp, local_err);
             return -ENOTSUP;
         }
+        dev->dev.cap_present |= QEMU_PCI_CAP_MSIX;
         dev->cap.available |= ASSIGNED_DEVICE_CAP_MSIX;
         ret = pci_add_capability2(pci_dev, PCI_CAP_ID_MSIX, pos, 12,
                                   &local_err);
@@ -1648,6 +1650,7 @@ static void assigned_dev_register_msix_mmio(AssignedDevice *dev, Error **errp)
         dev->msix_table = NULL;
         return;
     }
+    dev->dev.msix_table = dev->msix_table;
 
     assigned_dev_msix_reset(dev);
 
@@ -1665,6 +1668,7 @@ static void assigned_dev_unregister_msix_mmio(AssignedDevice *dev)
         error_report("error unmapping msix_table! %s", strerror(errno));
     }
     dev->msix_table = NULL;
+    dev->dev.msix_table = NULL;
 }
 
 static const VMStateDescription vmstate_assigned_device = {
-- 
2.7.4

Re: [Qemu-devel] [PATCH for-2.8] pci-assign: sync MSI/MSI-X cap and table with PCIDevice

[Peter Xu]

CC Limin.

Limin, could you please help test this patch as well to see whether it
can solve your issue? This one is slightly cleaner IMHO.

Thanks,

On Thu, Nov 24, 2016 at 06:25:13PM +0800, Peter Xu wrote:
> Since commit e1d4fb2d ("kvm-irqchip: x86: add msi route notify fn"),
> kvm_irqchip_add_msi_route() starts to use pci_get_msi_message() to fetch
> MSI info. This requires that we setup MSI related fields in PCIDevice.
> For most devices, that won't be a problem, as long as we are using
> general interfaces like msi_init()/msix_init().
> 
> However, for pci-assign devices, MSI/MSI-X is treated differently - PCI
> assign devices are maintaining its own MSI table and cap information in
> AssignedDevice struct. however that's not synced up with PCIDevice's
> fields. That will leads to pci_get_msi_message() failed to find correct
> MSI capability, even with an NULL msix_table.
> 
> A quick fix is to sync up the two places: both the capability bits and
> table address for MSI/MSI-X.
> 
> Reported-by: Changlimin <changlimin@h3c.com>
> Cc: qemu-stable@nongnu.org
> Fixes: e1d4fb2d ("kvm-irqchip: x86: add msi route notify fn")
> Signed-off-by: Peter Xu <peterx@redhat.com>
> 
> ---
> Do we still support pci-assign?
> ---
>  hw/i386/kvm/pci-assign.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/hw/i386/kvm/pci-assign.c b/hw/i386/kvm/pci-assign.c
> index 8238fbc..df9748d 100644
> --- a/hw/i386/kvm/pci-assign.c
> +++ b/hw/i386/kvm/pci-assign.c
> @@ -1251,6 +1251,7 @@ static int assigned_device_pci_cap_init(PCIDevice *pci_dev, Error **errp)
>              error_propagate(errp, local_err);
>              return -ENOTSUP;
>          }
> +        dev->dev.cap_present |= QEMU_PCI_CAP_MSI;
>          dev->cap.available |= ASSIGNED_DEVICE_CAP_MSI;
>          /* Only 32-bit/no-mask currently supported */
>          ret = pci_add_capability2(pci_dev, PCI_CAP_ID_MSI, pos, 10,
> @@ -1285,6 +1286,7 @@ static int assigned_device_pci_cap_init(PCIDevice *pci_dev, Error **errp)
>              error_propagate(errp, local_err);
>              return -ENOTSUP;
>          }
> +        dev->dev.cap_present |= QEMU_PCI_CAP_MSIX;
>          dev->cap.available |= ASSIGNED_DEVICE_CAP_MSIX;
>          ret = pci_add_capability2(pci_dev, PCI_CAP_ID_MSIX, pos, 12,
>                                    &local_err);
> @@ -1648,6 +1650,7 @@ static void assigned_dev_register_msix_mmio(AssignedDevice *dev, Error **errp)
>          dev->msix_table = NULL;
>          return;
>      }
> +    dev->dev.msix_table = dev->msix_table;
>  
>      assigned_dev_msix_reset(dev);
>  
> @@ -1665,6 +1668,7 @@ static void assigned_dev_unregister_msix_mmio(AssignedDevice *dev)
>          error_report("error unmapping msix_table! %s", strerror(errno));
>      }
>      dev->msix_table = NULL;
> +    dev->dev.msix_table = NULL;
>  }
>  
>  static const VMStateDescription vmstate_assigned_device = {
> -- 
> 2.7.4
> 

-- peterx

Re: [Qemu-devel] [PATCH for-2.8] pci-assign: sync MSI/MSI-X cap and table with PCIDevice

[Changlimin]

Hi Peter,
  I have tested the patch, it has solved the issue.
  I think change
  dev->dev.msix_table = dev->msix_table;
  to
  dev->dev.msix_table = (uint8_t *)dev->msix_table;
  can avoid a gcc warning.

Regards
Chang limin

-----Original Message-----
From: Peter Xu [mailto:peterx@redhat.com]
Sent: Thursday, November 24, 2016 8:33 PM
To: qemu-devel@nongnu.org; changlimin 00148 (CCPL)
Cc: jan.kiszka@siemens.com; alex.williamson@redhat.com; qemu-stable@nongnu.org; pbonzini@redhat.com
Subject: Re: [PATCH for-2.8] pci-assign: sync MSI/MSI-X cap and table with PCIDevice

CC Limin.

Limin, could you please help test this patch as well to see whether it can solve your issue? This one is slightly cleaner IMHO.

Thanks,

On Thu, Nov 24, 2016 at 06:25:13PM +0800, Peter Xu wrote:
> Since commit e1d4fb2d ("kvm-irqchip: x86: add msi route notify fn"),
> kvm_irqchip_add_msi_route() starts to use pci_get_msi_message() to
> fetch MSI info. This requires that we setup MSI related fields in PCIDevice.
> For most devices, that won't be a problem, as long as we are using
> general interfaces like msi_init()/msix_init().
>
> However, for pci-assign devices, MSI/MSI-X is treated differently -
> PCI assign devices are maintaining its own MSI table and cap
> information in AssignedDevice struct. however that's not synced up
> with PCIDevice's fields. That will leads to pci_get_msi_message()
> failed to find correct MSI capability, even with an NULL msix_table.
>
> A quick fix is to sync up the two places: both the capability bits and
> table address for MSI/MSI-X.
>
> Reported-by: Changlimin <changlimin@h3c.com>
> Cc: qemu-stable@nongnu.org
> Fixes: e1d4fb2d ("kvm-irqchip: x86: add msi route notify fn")
> Signed-off-by: Peter Xu <peterx@redhat.com>
>
> ---
> Do we still support pci-assign?
> ---
>  hw/i386/kvm/pci-assign.c | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/hw/i386/kvm/pci-assign.c b/hw/i386/kvm/pci-assign.c index
> 8238fbc..df9748d 100644
> --- a/hw/i386/kvm/pci-assign.c
> +++ b/hw/i386/kvm/pci-assign.c
> @@ -1251,6 +1251,7 @@ static int assigned_device_pci_cap_init(PCIDevice *pci_dev, Error **errp)
>              error_propagate(errp, local_err);
>              return -ENOTSUP;
>          }
> +        dev->dev.cap_present |= QEMU_PCI_CAP_MSI;
>          dev->cap.available |= ASSIGNED_DEVICE_CAP_MSI;
>          /* Only 32-bit/no-mask currently supported */
>          ret = pci_add_capability2(pci_dev, PCI_CAP_ID_MSI, pos, 10,
> @@ -1285,6 +1286,7 @@ static int assigned_device_pci_cap_init(PCIDevice *pci_dev, Error **errp)
>              error_propagate(errp, local_err);
>              return -ENOTSUP;
>          }
> +        dev->dev.cap_present |= QEMU_PCI_CAP_MSIX;
>          dev->cap.available |= ASSIGNED_DEVICE_CAP_MSIX;
>          ret = pci_add_capability2(pci_dev, PCI_CAP_ID_MSIX, pos, 12,
>                                    &local_err); @@ -1648,6 +1650,7 @@
> static void assigned_dev_register_msix_mmio(AssignedDevice *dev, Error **errp)
>          dev->msix_table = NULL;
>          return;
>      }
> +    dev->dev.msix_table = dev->msix_table;
>
>      assigned_dev_msix_reset(dev);
>
> @@ -1665,6 +1668,7 @@ static void assigned_dev_unregister_msix_mmio(AssignedDevice *dev)
>          error_report("error unmapping msix_table! %s", strerror(errno));
>      }
>      dev->msix_table = NULL;
> +    dev->dev.msix_table = NULL;
>  }
>
>  static const VMStateDescription vmstate_assigned_device = {
> --
> 2.7.4
>

-- peterx
-------------------------------------------------------------------------------------------------------------------------------------
本邮件及其附件含有杭州华三通信技术有限公司的保密信息，仅限于发送给上面地址中列出
的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、
或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本
邮件！
This e-mail and its attachments contain confidential information from H3C, which is
intended only for the person or entity whose address is listed above. Any use of the
information contained herein in any way (including, but not limited to, total or partial
disclosure, reproduction, or dissemination) by persons other than the intended
recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender
by phone or email immediately and delete it!

Re: [Qemu-devel] [PATCH for-2.8] pci-assign: sync MSI/MSI-X cap and table with PCIDevice

[Peter Xu]

On Fri, Nov 25, 2016 at 12:55:12AM +0000, Changlimin wrote:
> Hi Peter,
>   I have tested the patch, it has solved the issue.
>   I think change
>   dev->dev.msix_table = dev->msix_table;
>   to
>   dev->dev.msix_table = (uint8_t *)dev->msix_table;
>   can avoid a gcc warning.

Thanks for the comment, will fix and repost.

Then I'll add your Tested-by as well if you won't mind. :-)

-- peterx