sata_nv does not function in kernel > 2.6.20.21... possible ACPI or PCI involvement?

sata_nv does not function in kernel > 2.6.20.21... possible ACPI or PCI involvement?

[Matthew Hall]

[-- Attachment #1: Type: text/plain, Size: 1512 bytes --]

Hello All,

Jeff Garzik forwarded me here to get some assistance resolving a 
functionality regression in my sata_nv controller driver in recent Linux 
kernels. I have attached the same data here that I ended up sending to 
Jeff + others on the linux-ide list already. I assume you will probably 
need other different data from Jeff so please let me know if any other 
omitted information is required for diagnostics and I will respond ASAP.

I am using the Supermicro H8DCE motherboard. Some (not all) of the SATA
channels quit working due to some kind of resource conflict when I
upgrade to any kernel above 2.6.20.xx series, in my case I am running
2.6.20.21 SMP x86_64 presently.

The boot error which appears in dmesg for the missing SATA channels on
2.6.23.12 is pasted below.

Thanks,
Matthew Hall

ACPI: PCI Interrupt Link [LT3D] enabled at IRQ 46
ACPI: PCI Interrupt 0000:80:07.0[A] -> Link [LT3D] -> GSI 46 (level,
low) -> IRQ 46
sata_nv 0000:80:07.0: Using ADMA mode
PCI: Unable to reserve mem region #6:1000@dfefe000 for device
0000:80:07.0
ACPI: PCI interrupt for device 0000:80:07.0 disabled
sata_nv: probe of 0000:80:07.0 failed with error -16
ACPI: PCI Interrupt Link [LT2E] enabled at IRQ 45
ACPI: PCI Interrupt 0000:80:08.0[A] -> Link [LT2E] -> GSI 45 (level,
low) -> IRQ 45
sata_nv 0000:80:08.0: Using ADMA mode
PCI: Unable to reserve mem region #6:1000@dfefd000 for device
0000:80:08.0
ACPI: PCI interrupt for device 0000:80:08.0 disabled
sata_nv: probe of 0000:80:08.0 failed with error -16

[-- Attachment #2: iomem-2.6.20.21.gz --]
[-- Type: application/octet-stream, Size: 575 bytes --]

[-- Attachment #3: iomem-2.6.23.12.gz --]
[-- Type: application/octet-stream, Size: 561 bytes --]

[-- Attachment #4: lspci-verbose-2.6.20.21.gz --]
[-- Type: application/octet-stream, Size: 1393 bytes --]

[-- Attachment #5: lspci-verbose-2.6.23.12.gz --]
[-- Type: application/octet-stream, Size: 1400 bytes --]

Re: sata_nv does not function in kernel > 2.6.20.21... possible ACPI or PCI involvement?

[Bjorn Helgaas]

On Saturday 12 January 2008 01:59:39 pm Matthew Hall wrote:
> Jeff Garzik forwarded me here to get some assistance resolving a 
> functionality regression in my sata_nv controller driver in recent Linux 
> kernels. I have attached the same data here that I ended up sending to 
> Jeff + others on the linux-ide list already. I assume you will probably 
> need other different data from Jeff so please let me know if any other 
> omitted information is required for diagnostics and I will respond ASAP.
> 
> I am using the Supermicro H8DCE motherboard. Some (not all) of the SATA
> channels quit working due to some kind of resource conflict when I
> upgrade to any kernel above 2.6.20.xx series, in my case I am running
> 2.6.20.21 SMP x86_64 presently.
> 
> The boot error which appears in dmesg for the missing SATA channels on
> 2.6.23.12 is pasted below.

This looks like the same problem reported here:

  https://bugzilla.redhat.com/show_bug.cgi?id=313491
  https://bugzilla.redhat.com/show_bug.cgi?id=280641

> PCI: Unable to reserve mem region #6:1000@dfefe000 for device 0000:80:07.0
> PCI: Unable to reserve mem region #6:1000@dfefd000 for device 0000:80:08.0

Your 2.6.23.12 /proc/iomem shows this:

  dfefd000-dfefdfff : 0000:80:08.0
    dfefd000-dfefd3ff : pnp 00:07
  dfefe000-dfefefff : 0000:80:07.0
    dfefe000-dfefe3ff : pnp 00:07

That means the dfefd000-dfefd3ff range is mentioned both as a
resource of an ACPI motherboard device (we call it "pnp 00:07"
above) and as a BAR of the SATA PCI device 0000:80:08.0.

The PNP "system" driver reserves that range to keep us from
putting another device on top of it, and the sata_nv reservation
fails because it extends past the end of the PNP reservation.

I think this is a bug in the BIOS description of the motherboard
device.  In 2.6.20, the PNP system driver reserved ioport resources
but ignored mmio resources.  In 2.6.23, the system driver reserves
both ioport and mmio resources, which I think is more correct, but
exposes this bug.

I posted the attached test patch to the redhat bugzilla above, but
nobody's tested it yet.  Can you try it?

Bjorn


Index: w/drivers/pnp/quirks.c
===================================================================
--- w.orig/drivers/pnp/quirks.c	2007-10-11 15:36:12.000000000 -0600
+++ w/drivers/pnp/quirks.c	2007-10-11 16:35:49.000000000 -0600
@@ -108,6 +108,47 @@
 		       "pnp: SB audio device quirk - increasing port range\n");
 }
 
+static int overlaps(resource_size_t start1, resource_size_t end1,
+		    resource_size_t start2, resource_size_t end2)
+{
+	if (start2 <= start1 && start1 < end2)
+		return 1;
+	if (start2 <= end1 && end1 < end2)
+		return 1;
+	return 0;
+}
+
+
+#include <linux/pci.h>
+
+static void quirk_supermicro_motherboard(struct pnp_dev *dev)
+{
+	struct pci_dev *pdev = NULL;
+	int i, j;
+
+	for_each_pci_dev(pdev) {
+		for (i = 0; i < DEVICE_COUNT_RESOURCE; i++) {
+			if (!(pci_resource_flags(pdev, i) & IORESOURCE_MEM) ||
+			    pci_resource_len(pdev, i) == 0)
+				continue;
+
+			for (j = 0; j < PNP_MAX_MEM; j++) {
+				if (!pnp_mem_valid(dev, j) ||
+				    pnp_mem_len(dev, j) == 0)
+					continue;
+
+				if (overlaps(pnp_mem_start(dev, j),
+					     pnp_mem_end(dev, j),
+					     pci_resource_start(pdev, i),
+					     pci_resource_end(pdev, i))) {
+					dev_warn(&dev->dev, "mem resource overlaps %s BAR %d, disabling\n", pci_name(pdev), i);
+					pnp_mem_flags(dev, j) = 0;
+				}
+			}
+		}
+	}
+}
+
 /*
  *  PnP Quirks
  *  Cards or devices that need some tweaking due to incomplete resource info
@@ -128,6 +169,8 @@
 	{"CTL0043", quirk_sb16audio_resources},
 	{"CTL0044", quirk_sb16audio_resources},
 	{"CTL0045", quirk_sb16audio_resources},
+	{"PNP0c01", quirk_supermicro_motherboard},
+	{"PNP0c02", quirk_supermicro_motherboard},
 	{""}
 };
 

Re: sata_nv does not function in kernel > 2.6.20.21... possible ACPI or PCI involvement?

[Matthew Hall]

On Mon, Jan 14, 2008 at 05:02:26PM -0700, Bjorn Helgaas wrote:
> I think this is a bug in the BIOS description of the motherboard
> device.  In 2.6.20, the PNP system driver reserved ioport resources
> but ignored mmio resources.  In 2.6.23, the system driver reserves
> both ioport and mmio resources, which I think is more correct, but
> exposes this bug.

Bjorn,

I complained to Supermicro and mentioned that the beta BIOS they 
provided to one of the customers in the Redhat Bugzilla threads did not 
solve the issue at all. I also included links to both Bugzilla threads 
as well as a link to the mail you sent back to me as snipped above which 
contained your excellent analysis of the problem.

> I posted the attached test patch to the redhat bugzilla above, but
> nobody's tested it yet.  Can you try it?

The patch applied, booted, ran, and solved the problem perfectly. Do you 
want any output or configuration files to verify anything? Let me know 
if there is anything I can do to help. Other than that if you think it 
is safe to apply this, I am in favor of adding it to the mainline or 
other appropriate tree.

> Bjorn

Thanks,
Matthew Hall

Re: sata_nv does not function in kernel > 2.6.20.21... possible ACPI or PCI involvement?

[Bjorn Helgaas]

On Monday 14 January 2008 09:38:04 pm Matthew Hall wrote:
> On Mon, Jan 14, 2008 at 05:02:26PM -0700, Bjorn Helgaas wrote:
> > I think this is a bug in the BIOS description of the motherboard
> > device.  In 2.6.20, the PNP system driver reserved ioport resources
> > but ignored mmio resources.  In 2.6.23, the system driver reserves
> > both ioport and mmio resources, which I think is more correct, but
> > exposes this bug.
> 
> Bjorn,
> 
> I complained to Supermicro and mentioned that the beta BIOS they 
> provided to one of the customers in the Redhat Bugzilla threads did not 
> solve the issue at all. I also included links to both Bugzilla threads 
> as well as a link to the mail you sent back to me as snipped above which 
> contained your excellent analysis of the problem.

My patch is based on the assumption that a PCI resource (like the
SATA BAR here) is completely discoverable and configurable through
the standard PCI mechanisms, and it therefore should not also be
described as a resource of an ACPI device.

If Supermicro went to the trouble of providing a beta BIOS, maybe
they're interested enough to shed some light on why their BIOS does
it this way.  It's possible that my assumption is wrong, and Linux
is just not doing the right thing.  If you have a Supermicro contact,
you might pass this by them.

> > I posted the attached test patch to the redhat bugzilla above, but
> > nobody's tested it yet.  Can you try it?
> 
> The patch applied, booted, ran, and solved the problem perfectly. Do you 
> want any output or configuration files to verify anything? Let me know 
> if there is anything I can do to help. Other than that if you think it 
> is safe to apply this, I am in favor of adding it to the mainline or 
> other appropriate tree.

I'll clean up the patch and post it.  With luck, it might show up in
the -mm tree soon and then in 2.6.25.

Bjorn

Re: sata_nv does not function in kernel > 2.6.20.21... possible ACPI or PCI involvement?

[Bjorn Helgaas]

On Monday 14 January 2008 05:02:26 pm Bjorn Helgaas wrote:
> This looks like the same problem reported here:
> 
>   https://bugzilla.redhat.com/show_bug.cgi?id=313491
>   https://bugzilla.redhat.com/show_bug.cgi?id=280641
> 
> > PCI: Unable to reserve mem region #6:1000@dfefe000 for device 0000:80:07.0
> > PCI: Unable to reserve mem region #6:1000@dfefd000 for device 0000:80:08.0
> 
> Your 2.6.23.12 /proc/iomem shows this:
> 
>   dfefd000-dfefdfff : 0000:80:08.0
>     dfefd000-dfefd3ff : pnp 00:07
>   dfefe000-dfefefff : 0000:80:07.0
>     dfefe000-dfefe3ff : pnp 00:07
> 
> That means the dfefd000-dfefd3ff range is mentioned both as a
> resource of an ACPI motherboard device (we call it "pnp 00:07"
> above) and as a BAR of the SATA PCI device 0000:80:08.0.

Hmm...  The PNP quirk below is appealingly simple, and Matthew
confirmed that it fixes the problem on his box.

But now I'm a little hesitant.  It relies on the fact that we've
discovered the SATA PCI device before running the quirks for the
PNP system device.  That happens to be the case now, but I don't
think we should rely on it.

The ACPI root bridge device and the PNP system device are both
discovered via the ACPI namespace.  We currently treat them
differently -- one is treated as an ACPI device, and the other
is treated as a PNP device via the PNPACPI backend.  But that
distinction seems artificial to me, and I can imagine that the
ACPI PCI root bridge driver could someday be integrated into PNP.

In that case, the PNP system device might be discovered and have
its quirks run before the PCI root bridge is discovered and scanned.

As much as I'd like to have a generic quirk that would work around
this defect and others like it, I think the safest thing would be
to do something specific to the Supermicro H8DCE BIOS, since all
the reports I've seen are on that box:

    https://bugzilla.redhat.com/show_bug.cgi?id=280641
    https://bugzilla.redhat.com/show_bug.cgi?id=313491
    http://lkml.org/lkml/2008/1/9/449

Matthew, could you collect the dmidecode output?

Bjorn


> Index: w/drivers/pnp/quirks.c
> ===================================================================
> --- w.orig/drivers/pnp/quirks.c	2007-10-11 15:36:12.000000000 -0600
> +++ w/drivers/pnp/quirks.c	2007-10-11 16:35:49.000000000 -0600
> @@ -108,6 +108,47 @@
>  		       "pnp: SB audio device quirk - increasing port range\n");
>  }
>  
> +static int overlaps(resource_size_t start1, resource_size_t end1,
> +		    resource_size_t start2, resource_size_t end2)
> +{
> +	if (start2 <= start1 && start1 < end2)
> +		return 1;
> +	if (start2 <= end1 && end1 < end2)
> +		return 1;
> +	return 0;
> +}
> +
> +
> +#include <linux/pci.h>
> +
> +static void quirk_supermicro_motherboard(struct pnp_dev *dev)
> +{
> +	struct pci_dev *pdev = NULL;
> +	int i, j;
> +
> +	for_each_pci_dev(pdev) {
> +		for (i = 0; i < DEVICE_COUNT_RESOURCE; i++) {
> +			if (!(pci_resource_flags(pdev, i) & IORESOURCE_MEM) ||
> +			    pci_resource_len(pdev, i) == 0)
> +				continue;
> +
> +			for (j = 0; j < PNP_MAX_MEM; j++) {
> +				if (!pnp_mem_valid(dev, j) ||
> +				    pnp_mem_len(dev, j) == 0)
> +					continue;
> +
> +				if (overlaps(pnp_mem_start(dev, j),
> +					     pnp_mem_end(dev, j),
> +					     pci_resource_start(pdev, i),
> +					     pci_resource_end(pdev, i))) {
> +					dev_warn(&dev->dev, "mem resource overlaps %s BAR %d, disabling\n", pci_name(pdev), i);
> +					pnp_mem_flags(dev, j) = 0;
> +				}
> +			}
> +		}
> +	}
> +}
> +
>  /*
>   *  PnP Quirks
>   *  Cards or devices that need some tweaking due to incomplete resource info
> @@ -128,6 +169,8 @@
>  	{"CTL0043", quirk_sb16audio_resources},
>  	{"CTL0044", quirk_sb16audio_resources},
>  	{"CTL0045", quirk_sb16audio_resources},
> +	{"PNP0c01", quirk_supermicro_motherboard},
> +	{"PNP0c02", quirk_supermicro_motherboard},
>  	{""}
>  };
>  
> 
> 
> 

Re: sata_nv does not function in kernel > 2.6.20.21... possible ACPI or PCI involvement?

[Bjorn Helgaas]

On Monday 14 January 2008 09:38:04 pm Matthew Hall wrote:
> On Mon, Jan 14, 2008 at 05:02:26PM -0700, Bjorn Helgaas wrote:
> > I think this is a bug in the BIOS description of the motherboard
> > device.  In 2.6.20, the PNP system driver reserved ioport resources
> > but ignored mmio resources.  In 2.6.23, the system driver reserves
> > both ioport and mmio resources, which I think is more correct, but
> > exposes this bug.
> ... 
> > I posted the attached test patch to the redhat bugzilla above, but
> > nobody's tested it yet.  Can you try it?
> 
> The patch applied, booted, ran, and solved the problem perfectly. Do you 
> want any output or configuration files to verify anything? Let me know 
> if there is anything I can do to help. Other than that if you think it 
> is safe to apply this, I am in favor of adding it to the mainline or 
> other appropriate tree.

Matthew,

Can you please try the revised patch below?  It's not generic like
my first one, but I think it's a little safer because it doesn't
rely on the discovery of PCI devices before the PNP quirk is run.

Thanks,
  Bjorn




PNP: disable Supermicro H8DCE motherboard resources that overlap SATA BARs

Some Supermicro BIOSes apparently describe a SATA PCI BAR as a motherboard
resource.  The PNP system driver claims motherboard resources, and this
prevents the sata_nv driver from requesting it later.

This patch disables the PNP0C01/PNP0C02 resources so they won't be claimed
by the PNP system driver, so they'll available for sata_nv.

References:
    https://bugzilla.redhat.com/show_bug.cgi?id=280641
    https://bugzilla.redhat.com/show_bug.cgi?id=313491
    http://lkml.org/lkml/2008/1/9/449
    http://thread.gmane.org/gmane.linux.acpi.devel/27312

Signed-off-by: Bjorn Helgaas <bjorn.helgaas@hp.com>

Index: work5/drivers/pnp/quirks.c
===================================================================
--- work5.orig/drivers/pnp/quirks.c	2008-01-04 14:34:43.000000000 -0700
+++ work5/drivers/pnp/quirks.c	2008-01-16 13:12:53.000000000 -0700
@@ -17,6 +17,7 @@
 #include <linux/slab.h>
 #include <linux/pnp.h>
 #include <linux/io.h>
+#include <linux/dmi.h>
 #include <linux/kallsyms.h>
 #include "base.h"
 
@@ -108,6 +109,46 @@
 		       "pnp: SB audio device quirk - increasing port range\n");
 }
 
+static void quirk_supermicro_h8dce_system(struct pnp_dev *dev)
+{
+	int i;
+	static struct dmi_system_id supermicro_h8dce[] = {
+		{
+			.ident = "Supermicro H8DCE",
+			.matches = {
+				DMI_MATCH(DMI_SYS_VENDOR, "Supermicro"),
+				DMI_MATCH(DMI_PRODUCT_NAME, "H8DCE"),
+			},
+		},
+		{ }
+	};
+
+	if (!dmi_check_system(supermicro_h8dce))
+		return;
+
+	/*
+	 * On the Supermicro H8DCE, there's a system device with resources
+	 * that overlap BAR 6 of the built-in SATA PCI adapter.  If the PNP
+	 * system device claims them, the sata_nv driver won't be able to.
+	 * More details at:
+	 *     https://bugzilla.redhat.com/show_bug.cgi?id=280641
+	 *     https://bugzilla.redhat.com/show_bug.cgi?id=313491
+	 *     http://lkml.org/lkml/2008/1/9/449
+	 *     http://thread.gmane.org/gmane.linux.acpi.devel/27312
+	 */
+	for (i = 0; i < PNP_MAX_MEM; i++) {
+		if (pnp_mem_valid(dev, i) && pnp_mem_len(dev, i) &&
+		    (pnp_mem_start(dev, i) & 0xdfef0000) == 0xdfef0000) {
+			dev_warn(&dev->dev, "disabling 0x%llx-0x%llx to prevent"
+				" conflict with sata_nv PCI device\n",
+				(unsigned long long) pnp_mem_start(dev, i),
+				(unsigned long long) (pnp_mem_start(dev, i) +
+					pnp_mem_len(dev, i) - 1));
+			pnp_mem_flags(dev, i) = 0;
+		}
+	}
+}
+
 /*
  *  PnP Quirks
  *  Cards or devices that need some tweaking due to incomplete resource info
@@ -128,6 +169,8 @@
 	{"CTL0043", quirk_sb16audio_resources},
 	{"CTL0044", quirk_sb16audio_resources},
 	{"CTL0045", quirk_sb16audio_resources},
+	{"PNP0c01", quirk_supermicro_h8dce_system},
+	{"PNP0c02", quirk_supermicro_h8dce_system},
 	{""}
 };
 

Re: sata_nv does not function in kernel > 2.6.20.21... possible ACPI or PCI involvement?

[Matthew Hall]

On Wed, Jan 16, 2008 at 02:07:37PM -0700, Bjorn Helgaas wrote:
> Matthew,

Bjorn,

> Can you please try the revised patch below?

It works on my machine! I just had to change the first hunk to get the 
includes to work because your file had a few more includes than mine 
since it was newer code.

> Thanks,
>   Bjorn

Best Regards,
Matthew Hall