Re: [PATCH v7 01/60] sparc/PCI: Add mem64 resource parsing for root bus

[Bjorn Helgaas <helgaas@kernel.org>]

On Thu, Oct 08, 2015 at 02:38:20PM -0700, Yinghai Lu wrote:
> Found "no compatible bridge window" warning in boot log from T5-8.
> 
> pci 0000:00:01.0: can't claim BAR 15 [mem 0x100000000-0x4afffffff pref]: no compatible bridge window
> 
> That resource is above 4G, but does not get offset correctly as
> root bus only report io and mem32.
> 
> pci_sun4v f02dbcfc: PCI host bridge to bus 0000:00
> pci_bus 0000:00: root bus resource [io  0x804000000000-0x80400fffffff] (bus address [0x0000-0xfffffff])
> pci_bus 0000:00: root bus resource [mem 0x800000000000-0x80007effffff] (bus address [0x00000000-0x7effffff])
> pci_bus 0000:00: root bus resource [bus 00-77]
> 
> Add mem64 handling in pci_common for sparc, so we can have 64bit resource
> registered for root bus at first.
> 
> After patch, will have:
> pci_sun4v f02dbcfc: PCI host bridge to bus 0000:00
> pci_bus 0000:00: root bus resource [io  0x804000000000-0x80400fffffff] (bus address [0x0000-0xfffffff])
> pci_bus 0000:00: root bus resource [mem 0x800000000000-0x80007effffff] (bus address [0x00000000-0x7effffff])
> pci_bus 0000:00: root bus resource [mem 0x800100000000-0x8007ffffffff] (bus address [0x100000000-0x7ffffffff])
> pci_bus 0000:00: root bus resource [bus 00-77]
> 
> -v2: mem64_space should use mem_space.start as offset.
> -v3: add IORESOURCE_MEM_64 flag
> -v4: set name for mem64_space, otherwise /proc/iomem will have <bad> for name
> 
> Fixes: commit d63e2e1f3df9 ("sparc/PCI: Clip bridge windows to fit in upstream windows")
> Link: http://lkml.kernel.org/r/CAE9FiQU1gJY1LYrxs+ma5LCTEEe4xmtjRG0aXJ9K_Tsu+m9Wuw@mail.gmail.com
> Reported-by: David Ahern <david.ahern@oracle.com>
> Tested-by: David Ahern <david.ahern@oracle.com>
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> 
>  arch/sparc/kernel/pci.c        |    7 ++++++-
>  arch/sparc/kernel/pci_common.c |   17 +++++++++++++++--
>  arch/sparc/kernel/pci_impl.h   |    1 +
>  3 files changed, 22 insertions(+), 3 deletions(-)
> ---
>  arch/sparc/kernel/pci.c        |  7 ++++++-
>  arch/sparc/kernel/pci_common.c | 17 +++++++++++++++--
>  arch/sparc/kernel/pci_impl.h   |  1 +
>  3 files changed, 22 insertions(+), 3 deletions(-)

Applied with Khalid's tested-by to pci/resource for v4.4 as follows.
I cleaned up the changelog, URL, and diffstat for you.

We still need some unification of pci_parse_of_flags() (powerpc and
sparc versions), of_bus_pci_get_flags() (two sparc versions and a
generic version), and pci_determine_mem_io_space() (sparc version).
All of these decode basically the same thing but in different ways.

Bjorn


commit 7ed9b592adee90a03e7212c097961909eda32f80
Author: Yinghai Lu <yinghai@kernel.org>
Date:   Tue Oct 20 11:36:53 2015 -0500

    sparc/PCI: Add mem64 resource parsing for root bus
    
    David reported that a T5-8 sparc system failed to boot with:
    
      pci_sun4v f02dbcfc: PCI host bridge to bus 0000:00
      pci_bus 0000:00: root bus resource [io  0x804000000000-0x80400fffffff] (bus address [0x0000-0xfffffff])
      pci_bus 0000:00: root bus resource [mem 0x800000000000-0x80007effffff] (bus address [0x00000000-0x7effffff])
      pci 0000:00:01.0: can't claim BAR 15 [mem 0x100000000-0x4afffffff pref]: no compatible bridge window
    
    Note that we don't know about a host bridge aperture that contains
    BAR 15.  OF does report a MEM64 aperture, but before this patch,
    pci_determine_mem_io_space() ignored it.
    
    Add support for host bridge apertures with 64-bit PCI addresses.  Also
    set IORESOURCE_MEM_64 for PCI device and bridge resources in PCI 64-bit
    memory space.
    
    Sparc doesn't actually print the device and bridge resources, but after
    this patch, we should have the equivalent of this:
    
      pci_sun4v f02dbcfc: PCI host bridge to bus 0000:00
      pci_bus 0000:00: root bus resource [io  0x804000000000-0x80400fffffff] (bus address [0x0000-0xfffffff])
      pci_bus 0000:00: root bus resource [mem 0x800000000000-0x80007effffff] (bus address [0x00000000-0x7effffff])
      pci_bus 0000:00: root bus resource [mem 0x800100000000-0x8007ffffffff] (bus address [0x100000000-0x7ffffffff])
      pci 0000:00:01.0:   bridge window [mem 0x800100000000-0x8004afffffff 64bit pref]
    
    [bhelgaas: changelog, URL to David's report]
    Fixes: d63e2e1f3df9 ("sparc/PCI: Clip bridge windows to fit in upstream windows")
    Link: http://lkml.kernel.org/r/5514391F.2030300@oracle.com
    Reported-by: David Ahern <david.ahern@oracle.com>
    Tested-by: David Ahern <david.ahern@oracle.com>
    Tested-by: Khalid Aziz <khalid.aziz@oracle.com>
    Signed-off-by: Yinghai Lu <yinghai@kernel.org>
    Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c
index b91d7f1..badf095 100644
--- a/arch/sparc/kernel/pci.c
+++ b/arch/sparc/kernel/pci.c
@@ -185,8 +185,10 @@ static unsigned long pci_parse_of_flags(u32 addr0)
 
 	if (addr0 & 0x02000000) {
 		flags = IORESOURCE_MEM | PCI_BASE_ADDRESS_SPACE_MEMORY;
-		flags |= (addr0 >> 22) & PCI_BASE_ADDRESS_MEM_TYPE_64;
 		flags |= (addr0 >> 28) & PCI_BASE_ADDRESS_MEM_TYPE_1M;
+		if (addr0 & 0x01000000)
+			flags |= IORESOURCE_MEM_64
+				 | PCI_BASE_ADDRESS_MEM_TYPE_64;
 		if (addr0 & 0x40000000)
 			flags |= IORESOURCE_PREFETCH
 				 | PCI_BASE_ADDRESS_MEM_PREFETCH;
@@ -655,6 +657,9 @@ struct pci_bus *pci_scan_one_pbm(struct pci_pbm_info *pbm,
 				pbm->io_space.start);
 	pci_add_resource_offset(&resources, &pbm->mem_space,
 				pbm->mem_space.start);
+	if (pbm->mem64_space.flags)
+		pci_add_resource_offset(&resources, &pbm->mem64_space,
+					pbm->mem_space.start);
 	pbm->busn.start = pbm->pci_first_busno;
 	pbm->busn.end	= pbm->pci_last_busno;
 	pbm->busn.flags	= IORESOURCE_BUS;
diff --git a/arch/sparc/kernel/pci_common.c b/arch/sparc/kernel/pci_common.c
index 944a065..33524c1 100644
--- a/arch/sparc/kernel/pci_common.c
+++ b/arch/sparc/kernel/pci_common.c
@@ -406,6 +406,7 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 	}
 
 	num_pbm_ranges = i / sizeof(*pbm_ranges);
+	memset(&pbm->mem64_space, 0, sizeof(struct resource));
 
 	for (i = 0; i < num_pbm_ranges; i++) {
 		const struct linux_prom_pci_ranges *pr = &pbm_ranges[i];
@@ -451,7 +452,12 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 			break;
 
 		case 3:
-			/* XXX 64-bit MEM handling XXX */
+			/* 64-bit MEM handling */
+			pbm->mem64_space.start = a;
+			pbm->mem64_space.end = a + size - 1UL;
+			pbm->mem64_space.flags = IORESOURCE_MEM;
+			saw_mem = 1;
+			break;
 
 		default:
 			break;
@@ -465,15 +471,22 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 		prom_halt();
 	}
 
-	printk("%s: PCI IO[%llx] MEM[%llx]\n",
+	printk("%s: PCI IO[%llx] MEM[%llx]",
 	       pbm->name,
 	       pbm->io_space.start,
 	       pbm->mem_space.start);
+	if (pbm->mem64_space.flags)
+		printk(" MEM64[%llx]",
+		       pbm->mem64_space.start);
+	printk("\n");
 
 	pbm->io_space.name = pbm->mem_space.name = pbm->name;
+	pbm->mem64_space.name = pbm->name;
 
 	request_resource(&ioport_resource, &pbm->io_space);
 	request_resource(&iomem_resource, &pbm->mem_space);
+	if (pbm->mem64_space.flags)
+		request_resource(&iomem_resource, &pbm->mem64_space);
 
 	pci_register_legacy_regions(&pbm->io_space,
 				    &pbm->mem_space);
diff --git a/arch/sparc/kernel/pci_impl.h b/arch/sparc/kernel/pci_impl.h
index 75803c7..37222ca 100644
--- a/arch/sparc/kernel/pci_impl.h
+++ b/arch/sparc/kernel/pci_impl.h
@@ -97,6 +97,7 @@ struct pci_pbm_info {
 	/* PBM I/O and Memory space resources. */
 	struct resource			io_space;
 	struct resource			mem_space;
+	struct resource			mem64_space;
 	struct resource			busn;
 
 	/* Base of PCI Config space, can be per-PBM or shared. */