[Qemu-devel] [PATCH v5] pci : Add pba_offset PCI quirk for Chelsio T5 devices

[Qemu-devel] [PATCH v5] pci : Add pba_offset PCI quirk for Chelsio T5 devices

[Gabriel Laupre]

Fix pba_offset initialization value for Chelsio T5 Virtual Function
device. The T5 hardware has a bug in it where it reports a Pending Interrupt
Bit Array Offset of 0x8000 for its SR-IOV Virtual Functions instead
of the 0x1000 that the hardware actually uses internally. As the hardware
doesn't return the correct pba_offset value, add a quirk to instead
return a hardcoded value of 0x1000 when a Chelsio T5 VF device is
detected.

This bug has been fixed in the Chelsio's next chip series T6 but there are
no plans to respin the T5 ASIC for this bug. It is just documented in the
T5 Errata and left it at that.

v5: Reduce the test to only control that the pba_offset doesn't extend beyond
  the specified BAR. The rare cases of potential other wrong offset variables
  extending beyond their specific BAR are left to the sanity check in the
  msix_init() function.

v4: Correct coding style and specify the comments.

v3: Test the correctness of MSIX data compare to the specified BAR and apply a
  quirk if it comes from a Chelsio T5 Virtual Function, otherwise raise a
  config error.

v2: Replace and PCI_DEVICE_ID_CHELSIO_T5_SERIES_VF macro definition with
  the Chelsio's T5 VF devices identifier schema of 0x58xx

Signed-off-by: Gabriel Laupre <glaupre@chelsio.com>
---
 hw/vfio/pci.c            | 27 +++++++++++++++++++++++++++
 include/hw/pci/pci_ids.h |  2 ++
 2 files changed, 29 insertions(+)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index e0e339a..3257eed 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -2252,6 +2252,33 @@ static int vfio_early_setup_msix(VFIOPCIDevice *vdev)
     vdev->msix->pba_offset = pba & ~PCI_MSIX_FLAGS_BIRMASK;
     vdev->msix->entries = (ctrl & PCI_MSIX_FLAGS_QSIZE) + 1;
 
+    /*
+     * Test the size of the pba_offset variable and catch if it extends outside
+     * of the specified BAR. If it is the case, we need to apply a hardware
+     * specific quirk if the device is known or we have a broken configuration.
+     */
+    if (vdev->msix->pba_offset >=
+        vdev->bars[vdev->msix->pba_bar].region.size) {
+
+        PCIDevice *pdev = &vdev->pdev;
+        uint16_t vendor = pci_get_word(pdev->config + PCI_VENDOR_ID);
+        uint16_t device = pci_get_word(pdev->config + PCI_DEVICE_ID);
+
+        /*
+         * Chelsio T5 Virtual Function devices are encoded as 0x58xx for T5
+         * adapters. The T5 hardware returns an incorrect value of 0x8000 for
+         * the VF PBA offset while the BAR itself is only 8k. The correct value
+         * is 0x1000, so we hard code that here.
+         */
+        if (vendor == PCI_VENDOR_ID_CHELSIO && (device & 0xff00) == 0x5800) {
+            vdev->msix->pba_offset = 0x1000;
+        } else {
+            error_report("vfio: Hardware reports invalid configuration, "
+                         "MSIX PBA outside of specified BAR");
+            return -EINVAL;
+        }
+    }
+
     trace_vfio_early_setup_msix(vdev->vbasedev.name, pos,
                                 vdev->msix->table_bar,
                                 vdev->msix->table_offset,
diff --git a/include/hw/pci/pci_ids.h b/include/hw/pci/pci_ids.h
index 49c062b..d98e6c9 100644
--- a/include/hw/pci/pci_ids.h
+++ b/include/hw/pci/pci_ids.h
@@ -114,6 +114,8 @@
 #define PCI_VENDOR_ID_ENSONIQ            0x1274
 #define PCI_DEVICE_ID_ENSONIQ_ES1370     0x5000
 
+#define PCI_VENDOR_ID_CHELSIO            0x1425
+
 #define PCI_VENDOR_ID_FREESCALE          0x1957
 #define PCI_DEVICE_ID_MPC8533E           0x0030
 
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH v5] pci : Add pba_offset PCI quirk for Chelsio T5 devices

[Bandan Das]

Gabriel Laupre <glaupre@chelsio.com> writes:

> Fix pba_offset initialization value for Chelsio T5 Virtual Function
> device. The T5 hardware has a bug in it where it reports a Pending Interrupt
> Bit Array Offset of 0x8000 for its SR-IOV Virtual Functions instead
> of the 0x1000 that the hardware actually uses internally. As the hardware
> doesn't return the correct pba_offset value, add a quirk to instead
> return a hardcoded value of 0x1000 when a Chelsio T5 VF device is
> detected.
>
> This bug has been fixed in the Chelsio's next chip series T6 but there are
> no plans to respin the T5 ASIC for this bug. It is just documented in the
> T5 Errata and left it at that.
>
> v5: Reduce the test to only control that the pba_offset doesn't extend beyond
>   the specified BAR. The rare cases of potential other wrong offset variables
>   extending beyond their specific BAR are left to the sanity check in the
>   msix_init() function.
>
> v4: Correct coding style and specify the comments.
>
> v3: Test the correctness of MSIX data compare to the specified BAR and apply a
>   quirk if it comes from a Chelsio T5 Virtual Function, otherwise raise a
>   config error.
>
> v2: Replace and PCI_DEVICE_ID_CHELSIO_T5_SERIES_VF macro definition with
>   the Chelsio's T5 VF devices identifier schema of 0x58xx
>
> Signed-off-by: Gabriel Laupre <glaupre@chelsio.com>
> ---
>  hw/vfio/pci.c            | 27 +++++++++++++++++++++++++++
>  include/hw/pci/pci_ids.h |  2 ++
>  2 files changed, 29 insertions(+)
>
> diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
> index e0e339a..3257eed 100644
> --- a/hw/vfio/pci.c
> +++ b/hw/vfio/pci.c
> @@ -2252,6 +2252,33 @@ static int vfio_early_setup_msix(VFIOPCIDevice *vdev)
>      vdev->msix->pba_offset = pba & ~PCI_MSIX_FLAGS_BIRMASK;
>      vdev->msix->entries = (ctrl & PCI_MSIX_FLAGS_QSIZE) + 1;
>  
> +    /*
> +     * Test the size of the pba_offset variable and catch if it extends outside
> +     * of the specified BAR. If it is the case, we need to apply a hardware
> +     * specific quirk if the device is known or we have a broken configuration.
> +     */
> +    if (vdev->msix->pba_offset >=
> +        vdev->bars[vdev->msix->pba_bar].region.size) {
> +
> +        PCIDevice *pdev = &vdev->pdev;
> +        uint16_t vendor = pci_get_word(pdev->config + PCI_VENDOR_ID);
> +        uint16_t device = pci_get_word(pdev->config + PCI_DEVICE_ID);
> +
> +        /*
> +         * Chelsio T5 Virtual Function devices are encoded as 0x58xx for T5
> +         * adapters. The T5 hardware returns an incorrect value of 0x8000 for
> +         * the VF PBA offset while the BAR itself is only 8k. The correct value
> +         * is 0x1000, so we hard code that here.
> +         */
> +        if (vendor == PCI_VENDOR_ID_CHELSIO && (device & 0xff00) == 0x5800) {
> +            vdev->msix->pba_offset = 0x1000;
> +        } else {
> +            error_report("vfio: Hardware reports invalid configuration, "
> +                         "MSIX PBA outside of specified BAR");
> +            return -EINVAL;
> +        }
> +    }
> +
>      trace_vfio_early_setup_msix(vdev->vbasedev.name, pos,
>                                  vdev->msix->table_bar,
>                                  vdev->msix->table_offset,
> diff --git a/include/hw/pci/pci_ids.h b/include/hw/pci/pci_ids.h
> index 49c062b..d98e6c9 100644
> --- a/include/hw/pci/pci_ids.h
> +++ b/include/hw/pci/pci_ids.h
> @@ -114,6 +114,8 @@
>  #define PCI_VENDOR_ID_ENSONIQ            0x1274
>  #define PCI_DEVICE_ID_ENSONIQ_ES1370     0x5000
>  
> +#define PCI_VENDOR_ID_CHELSIO            0x1425
> +
>  #define PCI_VENDOR_ID_FREESCALE          0x1957
>  #define PCI_DEVICE_ID_MPC8533E           0x0030

Reviewed-by: Bandan Das <bsd@redhat.com>

Re: [Qemu-devel] [PATCH v5] pci : Add pba_offset PCI quirk for Chelsio T5 devices

[Michael S. Tsirkin]

On Wed, Jul 01, 2015 at 12:05:45PM -0700, Gabriel Laupre wrote:
> Fix pba_offset initialization value for Chelsio T5 Virtual Function
> device. The T5 hardware has a bug in it where it reports a Pending Interrupt
> Bit Array Offset of 0x8000 for its SR-IOV Virtual Functions instead
> of the 0x1000 that the hardware actually uses internally. As the hardware
> doesn't return the correct pba_offset value, add a quirk to instead
> return a hardcoded value of 0x1000 when a Chelsio T5 VF device is
> detected.
> 
> This bug has been fixed in the Chelsio's next chip series T6 but there are
> no plans to respin the T5 ASIC for this bug. It is just documented in the
> T5 Errata and left it at that.
> 
> v5: Reduce the test to only control that the pba_offset doesn't extend beyond
>   the specified BAR. The rare cases of potential other wrong offset variables
>   extending beyond their specific BAR are left to the sanity check in the
>   msix_init() function.
> 
> v4: Correct coding style and specify the comments.
> 
> v3: Test the correctness of MSIX data compare to the specified BAR and apply a
>   quirk if it comes from a Chelsio T5 Virtual Function, otherwise raise a
>   config error.
> 
> v2: Replace and PCI_DEVICE_ID_CHELSIO_T5_SERIES_VF macro definition with
>   the Chelsio's T5 VF devices identifier schema of 0x58xx

Pls put changelog after --- so it's not stored in git.

> Signed-off-by: Gabriel Laupre <glaupre@chelsio.com>

Reviewed-by: Michael S. Tsirkin <mst@redhat.com>



> ---
>  hw/vfio/pci.c            | 27 +++++++++++++++++++++++++++
>  include/hw/pci/pci_ids.h |  2 ++
>  2 files changed, 29 insertions(+)
> 
> diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
> index e0e339a..3257eed 100644
> --- a/hw/vfio/pci.c
> +++ b/hw/vfio/pci.c
> @@ -2252,6 +2252,33 @@ static int vfio_early_setup_msix(VFIOPCIDevice *vdev)
>      vdev->msix->pba_offset = pba & ~PCI_MSIX_FLAGS_BIRMASK;
>      vdev->msix->entries = (ctrl & PCI_MSIX_FLAGS_QSIZE) + 1;
>  
> +    /*
> +     * Test the size of the pba_offset variable and catch if it extends outside
> +     * of the specified BAR. If it is the case, we need to apply a hardware
> +     * specific quirk if the device is known or we have a broken configuration.
> +     */
> +    if (vdev->msix->pba_offset >=
> +        vdev->bars[vdev->msix->pba_bar].region.size) {
> +
> +        PCIDevice *pdev = &vdev->pdev;
> +        uint16_t vendor = pci_get_word(pdev->config + PCI_VENDOR_ID);
> +        uint16_t device = pci_get_word(pdev->config + PCI_DEVICE_ID);
> +
> +        /*
> +         * Chelsio T5 Virtual Function devices are encoded as 0x58xx for T5
> +         * adapters. The T5 hardware returns an incorrect value of 0x8000 for
> +         * the VF PBA offset while the BAR itself is only 8k. The correct value
> +         * is 0x1000, so we hard code that here.
> +         */
> +        if (vendor == PCI_VENDOR_ID_CHELSIO && (device & 0xff00) == 0x5800) {
> +            vdev->msix->pba_offset = 0x1000;
> +        } else {
> +            error_report("vfio: Hardware reports invalid configuration, "
> +                         "MSIX PBA outside of specified BAR");
> +            return -EINVAL;
> +        }
> +    }
> +
>      trace_vfio_early_setup_msix(vdev->vbasedev.name, pos,
>                                  vdev->msix->table_bar,
>                                  vdev->msix->table_offset,
> diff --git a/include/hw/pci/pci_ids.h b/include/hw/pci/pci_ids.h
> index 49c062b..d98e6c9 100644
> --- a/include/hw/pci/pci_ids.h
> +++ b/include/hw/pci/pci_ids.h
> @@ -114,6 +114,8 @@
>  #define PCI_VENDOR_ID_ENSONIQ            0x1274
>  #define PCI_DEVICE_ID_ENSONIQ_ES1370     0x5000
>  
> +#define PCI_VENDOR_ID_CHELSIO            0x1425
> +
>  #define PCI_VENDOR_ID_FREESCALE          0x1957
>  #define PCI_DEVICE_ID_MPC8533E           0x0030
>  
> -- 
> 1.8.3.1