[RFC][PATCH 0/2] boot interrupts on Intel X58 and 55x0

[RFC][PATCH 0/2] boot interrupts on Intel X58 and 55x0

[Stefan Assmann]

This patchset is meant to disable boot interrupts on Intel X58 and 55x0
chipsets (Tylersburg). A lot of effort from Kei Tokunaga has gone into these
patches. Thanks a lot Kei!

The reason why this consists of 2 patches is that the PCI config space of the
configuration device to disable boot interrupts on these chipsets is not
always accessible by default. The first patch is to ensure that the device is
visible while the second patch applies the necessary changes to stop the
generation of boot interrupts. We're not really sure whether the final X58 and
55x0 chipsets have the configuration device visible or not, so patch #1 might
be superfluous but we've seen at least 2 machines where this is not the case.
That's one of the reasons why this patchset is marked as RFC. The other reason
is more serious namely the onboard NIC (8086:10c9) is malfunctioning on some
of our test system if the second patch is applied. It fails to acquire an IP
from DHCP and we're pretty clueless on this issue right now.
Help is greatly appreciated!


A quick summary of why boot interrupts are better off than on.

Boot interrupts will be generated by the chipset if the interrupt line of a
non-primary IO-APIC is masked and an IRQ arrives there. In that case a boot
interrupt will be forwarded to the PIC _and_ primary IO-APIC. We're not quite
sure why it arrives at the primary IO-APIC as well but it has been observed on
various chipsets.

As there will be no interrupt handler installed (for the boot interrupt) on
the primary IO-APIC the interrupt will be counted as spurious, which can
result in disabling the entire interrupt line by the kernel in case of too
many spurious interrupts. The problem only shows up if the primary IO-APIC
already has an interrupt handler installed on that line, otherwise that line
would be masked anyway and the boot interrupt silently ignored (which makes it
tricky to observe).

When does this become a problem?

Any device connected to a non-primary IO-APIC (that doesn't use MSIs) will
trigger the generation of boot interrupts if it's IO-APIC pin is masked. There
can be many reasons for that for example:
- The interrupt is shared and a buggy device driver (from another device)
  causes the interrupt to get disabled by the kernel.
- The RT kernel masks interrupt lines during handling (threaded IRQ-handling).
- Kei reported from issues in the case of kdump when the first kernel disables
  the IO-APICs before the second kernel starts booting.

It becomes a problem when too many interrupts are counted as spurious (the
boot interrupts cannot be handled because the kernel doesn't expect them) and
the kernel decides to better bring down the interrupt line.

[RFC][PATCH 1/2] show Intel QuickPath Interconnect Routing and Protocol Layer Registers in PCI config space

[Stefan Assmann]

On some machines with Intel X58 or Intel 55x0 chipset the Intel QuickPath
Interconnect Routing and Protocol Layer Registers or not exposed to PCI config
space. Access to these is necessary to disable boot interrupts in the QPI
Protocol Interrupt Control Register.
DEVHIDE1 provides a method to (un)hide the PCI config space of devices inside
the IOH and is altered to guarantee that PCI config space is accessible.
See Intel document #320838-003, sections 17.6.5.10 and 17.11.2.21.

Signed-off-by: Stefan Assmann <sassmann@redhat.com>
---
 drivers/pci/quirks.c    |   24 ++++++++++++++++++++++++
 include/linux/pci_ids.h |    1 +
 2 files changed, 25 insertions(+)

--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -1612,6 +1612,30 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_IN
 DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,   PCI_DEVICE_ID_INTEL_ESB_10, 	quirk_disable_intel_boot_interrupt);
 
 /*
+ * Show Intel QuickPath Interconnect Routing and Protocol Layer Registers
+ * in PCI config space (ensures access to QPIPINTRC).
+ * See Intel document #320838-003, sections 17.6.5.10 and 17.11.2.21.
+ */
+#define INTEL_X58_55x0_DEVHIDE1_REG	0xf2
+#define INTEL_X58_55x0_HIDE_DEV17_FUN0	(1<<12)
+#define INTEL_X58_55x0_HIDE_DEV17_FUN1	(1<<13)
+
+static void quirk_show_intel_qpi_conf_register(struct pci_dev *dev)
+{
+	u16 pci_config_word;
+
+	pci_read_config_word(dev, INTEL_X58_55x0_DEVHIDE1_REG, &pci_config_word);
+	pci_config_word &= ~(INTEL_X58_55x0_HIDE_DEV17_FUN0 |
+			     INTEL_X58_55x0_HIDE_DEV17_FUN1);
+	pci_write_config_word(dev, INTEL_X58_55x0_DEVHIDE1_REG, pci_config_word);
+
+	/* need to rescan the bus for changes to take effect */
+	pci_rescan_bus(dev->bus);
+}
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,	PCI_DEVICE_ID_INTEL_IOAT_TBG9,	quirk_show_intel_qpi_conf_register);
+DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,	PCI_DEVICE_ID_INTEL_IOAT_TBG9,	quirk_show_intel_qpi_conf_register);
+
+/*
  * disable boot interrupts on HT-1000
  */
 #define BC_HT1000_FEATURE_REG		0x64
--- a/include/linux/pci_ids.h
+++ b/include/linux/pci_ids.h
@@ -2508,6 +2508,7 @@
 #define PCI_DEVICE_ID_INTEL_IOAT_TBG5	0x342a
 #define PCI_DEVICE_ID_INTEL_IOAT_TBG6	0x342b
 #define PCI_DEVICE_ID_INTEL_IOAT_TBG7	0x342c
+#define PCI_DEVICE_ID_INTEL_IOAT_TBG9	0x342e
 #define PCI_DEVICE_ID_INTEL_IOAT_TBG0	0x3430
 #define PCI_DEVICE_ID_INTEL_IOAT_TBG1	0x3431
 #define PCI_DEVICE_ID_INTEL_IOAT_TBG2	0x3432

Re: [RFC][PATCH 1/2] show Intel QuickPath Interconnect Routing and Protocol Layer Registers in PCI config space

[Yinghai Lu]

On Fri, Sep 4, 2009 at 9:55 AM, Stefan Assmann<sassmann@redhat.com> wrote:
> On some machines with Intel X58 or Intel 55x0 chipset the Intel QuickPath
> Interconnect Routing and Protocol Layer Registers or not exposed to PCI config
> space. Access to these is necessary to disable boot interrupts in the QPI
> Protocol Interrupt Control Register.
> DEVHIDE1 provides a method to (un)hide the PCI config space of devices inside
> the IOH and is altered to guarantee that PCI config space is accessible.
> See Intel document #320838-003, sections 17.6.5.10 and 17.11.2.21.
>
> Signed-off-by: Stefan Assmann <sassmann@redhat.com>
> ---
>  drivers/pci/quirks.c    |   24 ++++++++++++++++++++++++
>  include/linux/pci_ids.h |    1 +
>  2 files changed, 25 insertions(+)
>
> --- a/drivers/pci/quirks.c
> +++ b/drivers/pci/quirks.c
> @@ -1612,6 +1612,30 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_IN
>  DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,   PCI_DEVICE_ID_INTEL_ESB_10,    quirk_disable_intel_boot_interrupt);
>
>  /*
> + * Show Intel QuickPath Interconnect Routing and Protocol Layer Registers
> + * in PCI config space (ensures access to QPIPINTRC).
> + * See Intel document #320838-003, sections 17.6.5.10 and 17.11.2.21.
> + */
> +#define INTEL_X58_55x0_DEVHIDE1_REG    0xf2
> +#define INTEL_X58_55x0_HIDE_DEV17_FUN0 (1<<12)
> +#define INTEL_X58_55x0_HIDE_DEV17_FUN1 (1<<13)
> +
> +static void quirk_show_intel_qpi_conf_register(struct pci_dev *dev)
> +{
> +       u16 pci_config_word;
> +
> +       pci_read_config_word(dev, INTEL_X58_55x0_DEVHIDE1_REG, &pci_config_word);
> +       pci_config_word &= ~(INTEL_X58_55x0_HIDE_DEV17_FUN0 |
> +                            INTEL_X58_55x0_HIDE_DEV17_FUN1);
> +       pci_write_config_word(dev, INTEL_X58_55x0_DEVHIDE1_REG, pci_config_word);
> +
> +       /* need to rescan the bus for changes to take effect */
> +       pci_rescan_bus(dev->bus);

better to check if those bits are set already...

can you move the quirk to EARLY to avoid rescan bus?

YH

> +}
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,   PCI_DEVICE_ID_INTEL_IOAT_TBG9,  quirk_show_intel_qpi_conf_register);
> +DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,  PCI_DEVICE_ID_INTEL_IOAT_TBG9,  quirk_show_intel_qpi_conf_register);
> +
> +/*
>  * disable boot interrupts on HT-1000
>  */
>  #define BC_HT1000_FEATURE_REG          0x64
> --- a/include/linux/pci_ids.h
> +++ b/include/linux/pci_ids.h
> @@ -2508,6 +2508,7 @@
>  #define PCI_DEVICE_ID_INTEL_IOAT_TBG5  0x342a
>  #define PCI_DEVICE_ID_INTEL_IOAT_TBG6  0x342b
>  #define PCI_DEVICE_ID_INTEL_IOAT_TBG7  0x342c
> +#define PCI_DEVICE_ID_INTEL_IOAT_TBG9  0x342e
>  #define PCI_DEVICE_ID_INTEL_IOAT_TBG0  0x3430
>  #define PCI_DEVICE_ID_INTEL_IOAT_TBG1  0x3431
>  #define PCI_DEVICE_ID_INTEL_IOAT_TBG2  0x3432
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC][PATCH 1/2] show Intel QuickPath Interconnect Routing and Protocol Layer Registers in PCI config space

[Stefan Assmann]

On 04.09.2009 19:07, Yinghai Lu wrote:
> On Fri, Sep 4, 2009 at 9:55 AM, Stefan Assmann<sassmann@redhat.com> wrote:
>> On some machines with Intel X58 or Intel 55x0 chipset the Intel QuickPath
>> Interconnect Routing and Protocol Layer Registers or not exposed to PCI config
>> space. Access to these is necessary to disable boot interrupts in the QPI
>> Protocol Interrupt Control Register.
>> DEVHIDE1 provides a method to (un)hide the PCI config space of devices inside
>> the IOH and is altered to guarantee that PCI config space is accessible.
>> See Intel document #320838-003, sections 17.6.5.10 and 17.11.2.21.
>>
>> Signed-off-by: Stefan Assmann <sassmann@redhat.com>
>> ---
>>  drivers/pci/quirks.c    |   24 ++++++++++++++++++++++++
>>  include/linux/pci_ids.h |    1 +
>>  2 files changed, 25 insertions(+)
>>
>> --- a/drivers/pci/quirks.c
>> +++ b/drivers/pci/quirks.c
>> @@ -1612,6 +1612,30 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_IN
>>  DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,   PCI_DEVICE_ID_INTEL_ESB_10,    quirk_disable_intel_boot_interrupt);
>>
>>  /*
>> + * Show Intel QuickPath Interconnect Routing and Protocol Layer Registers
>> + * in PCI config space (ensures access to QPIPINTRC).
>> + * See Intel document #320838-003, sections 17.6.5.10 and 17.11.2.21.
>> + */
>> +#define INTEL_X58_55x0_DEVHIDE1_REG    0xf2
>> +#define INTEL_X58_55x0_HIDE_DEV17_FUN0 (1<<12)
>> +#define INTEL_X58_55x0_HIDE_DEV17_FUN1 (1<<13)
>> +
>> +static void quirk_show_intel_qpi_conf_register(struct pci_dev *dev)
>> +{
>> +       u16 pci_config_word;
>> +
>> +       pci_read_config_word(dev, INTEL_X58_55x0_DEVHIDE1_REG, &pci_config_word);
>> +       pci_config_word &= ~(INTEL_X58_55x0_HIDE_DEV17_FUN0 |
>> +                            INTEL_X58_55x0_HIDE_DEV17_FUN1);
>> +       pci_write_config_word(dev, INTEL_X58_55x0_DEVHIDE1_REG, pci_config_word);
>> +
>> +       /* need to rescan the bus for changes to take effect */
>> +       pci_rescan_bus(dev->bus);

Hi Yinghai,

> better to check if those bits are set already...

If that's really an issue I'll change it, however it shouldn't cause any
troubles to set an already set bit. I'm not really sure if it's worth
the effort. It has the advantage of possibly avoiding pci_rescan_bus,
you're right about that.

> can you move the quirk to EARLY to avoid rescan bus?

Already tried that and it didn't work.

  Stefan
--
Stefan Assmann         | Red Hat GmbH
Software Engineer      | Otto-Hahn-Strasse 20, 85609 Dornach
                       | HR: Amtsgericht Muenchen HRB 153243
                       | GF: Brendan Lane, Charlie Peters,
sassmann at redhat.com |     Michael Cunningham, Charles Cachera

[RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

[Stefan Assmann]

Disable boot interrupts on Intel X58, 55x0 systems by setting the Disable PCI
INTx Routing to ICH bit (default is 0).

Disable PCI INTx Routing to ICH: When this bit is set, local INTx messages
received from the CB DMA/PCI Express ports of the IOH are not routed to legacy
ICH - they are either converted into MSI via the integrated I/OxAPIC (if the
I/OxAPIC mask bit is clear in the appropriate entries) or cause no further action
(when mask bit is set). When this bit is clear, local INTx messages received from
the CB DMA/PCI Express ports of the IOH are routed to legacy ICH, provided the
corresponding mask bit in the IOAPIC is set.
See Intel document #321328-001, section 19.10.2.27.

Signed-off-by: Stefan Assmann <sassmann@redhat.com>
---

 drivers/pci/quirks.c    |   24 ++++++++++++++++++++++++
 include/linux/pci_ids.h |    1 +
 2 files changed, 25 insertions(+)

--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -1635,6 +1635,30 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_IN
 DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,	PCI_DEVICE_ID_INTEL_IOAT_TBG9,	quirk_show_intel_qpi_conf_register);
 
 /*
+ * Disable boot interrupts on Intel X58, 55x0 (Tylersburg).
+ * See Intel document #321328-001, section 19.10.2.27.
+ * (Disable PCI INTx Routing to ICH)
+ */
+#define INTEL_X58_55x0_QPIPINTRC_OFFSET	0xe0
+#define INTEL_X58_55x0_QPIPINTRC_BIT	(1<<25)
+static void quirk_disable_intel_tylersburg_boot_interrupt(struct pci_dev *dev)
+{
+	u32 pci_config_dword;
+
+	if (noioapicquirk)
+		return;
+
+	pci_read_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET, &pci_config_dword);
+	pci_config_dword |= INTEL_X58_55x0_QPIPINTRC_BIT;
+	pci_write_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET, pci_config_dword);
+
+	printk(KERN_INFO "disabled boot interrupt on device 0x%04x:0x%04x\n",
+		dev->vendor, dev->device);
+}
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,	PCI_DEVICE_ID_INTEL_QPI_TBG15,	quirk_disable_intel_tylersburg_boot_interrupt);
+DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,	PCI_DEVICE_ID_INTEL_QPI_TBG15,	quirk_disable_intel_tylersburg_boot_interrupt);
+
+/*
  * disable boot interrupts on HT-1000
  */
 #define BC_HT1000_FEATURE_REG		0x64
--- a/include/linux/pci_ids.h
+++ b/include/linux/pci_ids.h
@@ -2504,6 +2504,7 @@
 #define PCI_DEVICE_ID_INTEL_ICH9_7	0x2916
 #define PCI_DEVICE_ID_INTEL_ICH9_8	0x2918
 #define PCI_DEVICE_ID_INTEL_82855PM_HB	0x3340
+#define PCI_DEVICE_ID_INTEL_QPI_TBG15	0x3428
 #define PCI_DEVICE_ID_INTEL_IOAT_TBG4	0x3429
 #define PCI_DEVICE_ID_INTEL_IOAT_TBG5	0x342a
 #define PCI_DEVICE_ID_INTEL_IOAT_TBG6	0x342b

Re: [RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

[Daniel Walker]

On Fri, 2009-09-04 at 12:55 -0400, Stefan Assmann wrote:
> + * Disable boot interrupts on Intel X58, 55x0 (Tylersburg).
> + * See Intel document #321328-001, section 19.10.2.27.
> + * (Disable PCI INTx Routing to ICH)
> + */
> +#define INTEL_X58_55x0_QPIPINTRC_OFFSET        0xe0
> +#define INTEL_X58_55x0_QPIPINTRC_BIT   (1<<25)
> +static void quirk_disable_intel_tylersburg_boot_interrupt(struct
> pci_dev *dev)
> +{
> +       u32 pci_config_dword;
> +
> +       if (noioapicquirk)
> +               return;
> +
> +       pci_read_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET,
> &pci_config_dword);
> +       pci_config_dword |= INTEL_X58_55x0_QPIPINTRC_BIT;
> +       pci_write_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET,
> pci_config_dword);
> +
> +       printk(KERN_INFO "disabled boot interrupt on device 0x%04x:0x%
> 04x\n",
> +               dev->vendor, dev->device);
> +}
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,   PCI_DEVICE_ID_INTEL_QPI_TBG15,  quirk_disable_intel_tylersburg_boot_interrupt);
> +DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,  PCI_DEVICE_ID_INTEL_QPI_TBG15,  quirk_disable_intel_tylersburg_boot_interrupt);


These lines are wildly long .. Could you reduce these down to a max of
80 characters..

Daniel

Re: [RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

[Stefan Assmann]

On 04.09.2009 19:06, Daniel Walker wrote:
> On Fri, 2009-09-04 at 12:55 -0400, Stefan Assmann wrote:
>> + * Disable boot interrupts on Intel X58, 55x0 (Tylersburg).
>> + * See Intel document #321328-001, section 19.10.2.27.
>> + * (Disable PCI INTx Routing to ICH)
>> + */
>> +#define INTEL_X58_55x0_QPIPINTRC_OFFSET        0xe0
>> +#define INTEL_X58_55x0_QPIPINTRC_BIT   (1<<25)
>> +static void quirk_disable_intel_tylersburg_boot_interrupt(struct
>> pci_dev *dev)
>> +{
>> +       u32 pci_config_dword;
>> +
>> +       if (noioapicquirk)
>> +               return;
>> +
>> +       pci_read_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET,
>> &pci_config_dword);
>> +       pci_config_dword |= INTEL_X58_55x0_QPIPINTRC_BIT;
>> +       pci_write_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET,
>> pci_config_dword);
>> +
>> +       printk(KERN_INFO "disabled boot interrupt on device 0x%04x:0x%
>> 04x\n",
>> +               dev->vendor, dev->device);
>> +}
>> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,   PCI_DEVICE_ID_INTEL_QPI_TBG15,  quirk_disable_intel_tylersburg_boot_interrupt);
>> +DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,  PCI_DEVICE_ID_INTEL_QPI_TBG15,  quirk_disable_intel_tylersburg_boot_interrupt);
> 
> 
> These lines are wildly long .. Could you reduce these down to a max of
> 80 characters..

Hi Daniel,

you're right about the lines being to long, however if you take a peek
at drivers/pci/quirks.c you'll see that all the quirks are done this
way. :)

  Stefan
--
Stefan Assmann         | Red Hat GmbH
Software Engineer      | Otto-Hahn-Strasse 20, 85609 Dornach
                       | HR: Amtsgericht Muenchen HRB 153243
                       | GF: Brendan Lane, Charlie Peters,
sassmann at redhat.com |     Michael Cunningham, Charles Cachera

Re: [RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

[Daniel Walker]

On Sat, 2009-09-05 at 11:07 +0200, Stefan Assmann wrote:
> On 04.09.2009 19:06, Daniel Walker wrote:
> > On Fri, 2009-09-04 at 12:55 -0400, Stefan Assmann wrote:
> >> + * Disable boot interrupts on Intel X58, 55x0 (Tylersburg).
> >> + * See Intel document #321328-001, section 19.10.2.27.
> >> + * (Disable PCI INTx Routing to ICH)
> >> + */
> >> +#define INTEL_X58_55x0_QPIPINTRC_OFFSET        0xe0
> >> +#define INTEL_X58_55x0_QPIPINTRC_BIT   (1<<25)
> >> +static void quirk_disable_intel_tylersburg_boot_interrupt(struct
> >> pci_dev *dev)
> >> +{
> >> +       u32 pci_config_dword;
> >> +
> >> +       if (noioapicquirk)
> >> +               return;
> >> +
> >> +       pci_read_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET,
> >> &pci_config_dword);
> >> +       pci_config_dword |= INTEL_X58_55x0_QPIPINTRC_BIT;
> >> +       pci_write_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET,
> >> pci_config_dword);
> >> +
> >> +       printk(KERN_INFO "disabled boot interrupt on device 0x%04x:0x%
> >> 04x\n",
> >> +               dev->vendor, dev->device);
> >> +}
> >> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,   PCI_DEVICE_ID_INTEL_QPI_TBG15,  quirk_disable_intel_tylersburg_boot_interrupt);
> >> +DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,  PCI_DEVICE_ID_INTEL_QPI_TBG15,  quirk_disable_intel_tylersburg_boot_interrupt);
> > 
> > 
> > These lines are wildly long .. Could you reduce these down to a max of
> > 80 characters..
> 
> Hi Daniel,
> 
> you're right about the lines being to long, however if you take a peek
> at drivers/pci/quirks.c you'll see that all the quirks are done this
> way. :)

I looked, the whole thing is a mess .. Really all these lines need to be
cleaned up.. It doesn't help to have you add more to it tho. Especially
the tab's or spaces between the commas, I can say I understand why that
is.. If there is a good reason why it's like that I'm all ears..

Daniel

Re: [RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

[Olaf Dabrunz]

On 05-Sep-09, Daniel Walker wrote:
> On Sat, 2009-09-05 at 11:07 +0200, Stefan Assmann wrote:
> > On 04.09.2009 19:06, Daniel Walker wrote:
> > > On Fri, 2009-09-04 at 12:55 -0400, Stefan Assmann wrote:
> > >> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,   PCI_DEVICE_ID_INTEL_QPI_TBG15,  quirk_disable_intel_tylersburg_boot_interrupt);
> > >> +DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,  PCI_DEVICE_ID_INTEL_QPI_TBG15,  quirk_disable_intel_tylersburg_boot_interrupt);
> > > 
> > > 
> > > These lines are wildly long .. Could you reduce these down to a max of
> > > 80 characters..
> > 
> > Hi Daniel,
> > 
> > you're right about the lines being to long, however if you take a peek
> > at drivers/pci/quirks.c you'll see that all the quirks are done this
> > way. :)
> 
> I looked, the whole thing is a mess .. Really all these lines need to be
> cleaned up.. It doesn't help to have you add more to it tho. Especially
> the tab's or spaces between the commas, I can say I understand why that
> is.. If there is a good reason why it's like that I'm all ears..

It makes adding/deleting/modifying the entries easier, with one-line
editor operations. And if all data for one ID-to-quirk relationship is
on one line, with tabs to make for nice columns, I can more easily see
which device is linked to which quirk. That is why I usually prefer one
line per entry. And the entries in quirks.c are long, but not too long
for that IMHO.

I agree that the 80 columns restriction is a good thing in almost all
cases, as it limits the amount of information that is put on a single
line, and it limits the number of indentation levels within a function.

But in this case I think we are better off with one line per entry, even
if the line becomes longer than 80 characters. The information on one
line is not that much (one macro with 3 parameters), only the
identifiers try to be readable and discernible and therefore become
long.

So this will only be a problem for terminals that are limited to 80
rows, and I do not think that anyone is that restricted nowadays.

Regards,

-- 
Olaf Dabrunz (Olaf.Dabrunz <at> gmx.net)

Re: [RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

[Daniel Walker]

On Sat, 2009-09-05 at 18:18 +0200, Olaf Dabrunz wrote:
> On 05-Sep-09, Daniel Walker wrote:
> > On Sat, 2009-09-05 at 11:07 +0200, Stefan Assmann wrote:
> > > On 04.09.2009 19:06, Daniel Walker wrote:
> > > > On Fri, 2009-09-04 at 12:55 -0400, Stefan Assmann wrote:
> > > >> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,   PCI_DEVICE_ID_INTEL_QPI_TBG15,  quirk_disable_intel_tylersburg_boot_interrupt);
> > > >> +DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,  PCI_DEVICE_ID_INTEL_QPI_TBG15,  quirk_disable_intel_tylersburg_boot_interrupt);
> > > > 
> > > > 
> > > > These lines are wildly long .. Could you reduce these down to a max of
> > > > 80 characters..
> > > 
> > > Hi Daniel,
> > > 
> > > you're right about the lines being to long, however if you take a peek
> > > at drivers/pci/quirks.c you'll see that all the quirks are done this
> > > way. :)
> > 
> > I looked, the whole thing is a mess .. Really all these lines need to be
> > cleaned up.. It doesn't help to have you add more to it tho. Especially
> > the tab's or spaces between the commas, I can say I understand why that
> > is.. If there is a good reason why it's like that I'm all ears..
> 
> It makes adding/deleting/modifying the entries easier, with one-line
> editor operations. And if all data for one ID-to-quirk relationship is
> on one line, with tabs to make for nice columns, I can more easily see
> which device is linked to which quirk. That is why I usually prefer one
> line per entry. And the entries in quirks.c are long, but not too long
> for that IMHO.

Firstly I would say these entries aren't consistent at all.. Some are
comma tab delimited, some are comma space delimited, and some are mixed
space plus tab delimited plus comma delimited.. Many are actually broken
down into less than 80 max length already..

I'm not sure I see how this makes "adding/deleting/modifying" easier ..
We have lots of other situations where lines are broken down less than
80, function arguments, macros, structures all sorts of stuff. 

Taking this block for instance,

DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
                        PCI_DEVICE_ID_TIGON3_5780,
                        quirk_msi_intx_disable_bug);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
                        PCI_DEVICE_ID_TIGON3_5780S,
                        quirk_msi_intx_disable_bug);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
                        PCI_DEVICE_ID_TIGON3_5714,
                        quirk_msi_intx_disable_bug);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
                        PCI_DEVICE_ID_TIGON3_5714S,
                        quirk_msi_intx_disable_bug);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
                        PCI_DEVICE_ID_TIGON3_5715,
                        quirk_msi_intx_disable_bug);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
                        PCI_DEVICE_ID_TIGON3_5715S,
                        quirk_msi_intx_disable_bug);

In this block there is a fairly clear pattern you can follow. One
problem with using tabs is that in order to get the perceive benefit
your suggesting you would be forced to have a terminal at a certain
size. Without that the single lines just end up line wrapped but in a
less consistent way. You also have the problem that people have to
deliberately add this formatting , sometimes one tab is enough,
sometimes you need two tabs, maybe someone uses spaces instead .. So you
end up with people potentially using all sorts of different methods and
maybe not getting it right (not to mention it's special formatting just
for this file)..

> So this will only be a problem for terminals that are limited to 80
> rows, and I do not think that anyone is that restricted nowadays.

It's more than that.. In order to un-wrap the lines in this file you
have to have a terminal at over 130 characters in width.

I think it would be better to create a script that parses this file and
produces consistent list in the format your describing, just for review
purposes.. That way it takes the human element out of it, and it still
lets you look at the formatting that your looking for.

Daniel

Re: [RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

[Henrique de Moraes Holschuh]

On Sat, 05 Sep 2009, Daniel Walker wrote:
> I'm not sure I see how this makes "adding/deleting/modifying" easier ..
> We have lots of other situations where lines are broken down less than
> 80, function arguments, macros, structures all sorts of stuff. 
> 
> Taking this block for instance,
> 
> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
>                         PCI_DEVICE_ID_TIGON3_5780,
>                         quirk_msi_intx_disable_bug);
> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
>                         PCI_DEVICE_ID_TIGON3_5780S,
>                         quirk_msi_intx_disable_bug);
> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
>                         PCI_DEVICE_ID_TIGON3_5714,
>                         quirk_msi_intx_disable_bug);
> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
>                         PCI_DEVICE_ID_TIGON3_5714S,
>                         quirk_msix_intx_disable_bug);
> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
>                         PCI_DEVICE_ID_TIGON3_5715,
>                         quirk_msi_intx_disable_bug);
> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
>                         PCI_DEVICE_ID_TIGON3_5715S,
>                         quirk_msi_intx_disable_bug);

You must have an alien brain of some sort if reading that is easier than the
tabular (single-line) format for you.

Please try finding the one entry in there which is different (I changed one
of them) with a quick glance.  Chances are you'll just skip over it and
think all of them do the same thing.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

Re: [RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

[Daniel Walker]

On Sun, 2009-09-06 at 22:37 -0300, Henrique de Moraes Holschuh wrote:
> On Sat, 05 Sep 2009, Daniel Walker wrote:
> > I'm not sure I see how this makes "adding/deleting/modifying" easier ..
> > We have lots of other situations where lines are broken down less than
> > 80, function arguments, macros, structures all sorts of stuff. 
> > 
> > Taking this block for instance,
> > 
> > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> >                         PCI_DEVICE_ID_TIGON3_5780,
> >                         quirk_msi_intx_disable_bug);
> > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> >                         PCI_DEVICE_ID_TIGON3_5780S,
> >                         quirk_msi_intx_disable_bug);
> > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> >                         PCI_DEVICE_ID_TIGON3_5714,
> >                         quirk_msi_intx_disable_bug);
> > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> >                         PCI_DEVICE_ID_TIGON3_5714S,
> >                         quirk_msx_intx_disable_bug);
> > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> >                         PCI_DEVICE_ID_TIGON3_5715,
> >                         quirk_msi_intx_disable_bug);
> > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> >                         PCI_DEVICE_ID_TIGON3_5715S,
> >                         quirk_msi_intx_disable_bug);
> 
> You must have an alien brain of some sort if reading that is easier than the
> tabular (single-line) format for you.

My terminal isn't at 130 ..

> Please try finding the one entry in there which is different (I changed one
> of them) with a quick glance.  Chances are you'll just skip over it and
> think all of them do the same thing.

msix , the letters didn't line up, the semicolons don't line up either..
Even with tabs all this information is so similar your bound to overlook
small errors like that. Even if you think all this matter, assume
someone has a term at 80 then every thing is wrapper. That would put you
in the same situation. 

Daniel

Daniel

Re: [RFC][PATCH 0/2] boot interrupts on Intel X58 and 55x0

[Henrique de Moraes Holschuh]

On Fri, 04 Sep 2009, Stefan Assmann wrote:
> is more serious namely the onboard NIC (8086:10c9) is malfunctioning on some
> of our test system if the second patch is applied. It fails to acquire an IP
> from DHCP and we're pretty clueless on this issue right now.
> Help is greatly appreciated!

Did you read Intel documet 320839-009, August/2009, already?  It is at:
http://www.intel.com/Assets/PDF/specupdate/320839.pdf

I see some stuff there that might be a problem, many of the "issues" are
related to interrupt handling.  Errata 40 (EOI to IOAPIC can be blocked)
even states that one should avoid the X58 IOH IOAPIC on stepping B2
completely, and use the IOAPIC in the ICH10R instead.

And the B2 stepping is the one in most motherboards (B3 is quite new).

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

Re: [RFC][PATCH 0/2] boot interrupts on Intel X58 and 55x0

[Stefan Assmann]

On 07.09.2009 03:33, Henrique de Moraes Holschuh wrote:
> On Fri, 04 Sep 2009, Stefan Assmann wrote:
>> is more serious namely the onboard NIC (8086:10c9) is malfunctioning on some
>> of our test system if the second patch is applied. It fails to acquire an IP
>> from DHCP and we're pretty clueless on this issue right now.
>> Help is greatly appreciated!
> 
> Did you read Intel documet 320839-009, August/2009, already?  It is at:
> http://www.intel.com/Assets/PDF/specupdate/320839.pdf
> 
> I see some stuff there that might be a problem, many of the "issues" are
> related to interrupt handling.  Errata 40 (EOI to IOAPIC can be blocked)
> even states that one should avoid the X58 IOH IOAPIC on stepping B2
> completely, and use the IOAPIC in the ICH10R instead.

Hi Henrique,

good catch, I missed the errata. After reading the details of errata 40
I assume that it's not directly related because the onboard NIC is
connected to the IOH IO-APIC in the first place. No interaction with
another IO-APIC should be involved. The second reason why it's most
likely unrelated is that it only happens when booted with pci=nomsi
(should have mentioned that earlier, sorry) and the errata states:
"The End of Interrupt (EOI) message targeted to the I/OxAPIC will be
blocked in the case that an INTx from a PCIe device and an MSI are
pending inside the Intel X58 Express Chipset."

I've also looked through the other errata and while some of them look
pretty wild I couldn't find anything that might be the root cause for
the observed problem. Same goes for the 55x0 August/2009 spec update.
http://www.intel.com/Assets/PDF/specupdate/321329.pdf

  Stefan
--
Stefan Assmann         | Red Hat GmbH
Software Engineer      | Otto-Hahn-Strasse 20, 85609 Dornach
                       | HR: Amtsgericht Muenchen HRB 153243
                       | GF: Brendan Lane, Charlie Peters,
sassmann at redhat.com |     Michael Cunningham, Charles Cachera