Re: [PATCH 02/11] PCI: Try to allocate mem64 above 4G at first

Yinghai Lu
2012-05-25T23:10:25Z

[-- Attachment #1: Type: text/plain, Size: 549 bytes --]

On Fri, May 25, 2012 at 2:55 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> I think we should fix this with a separate patch that removes
> PCIBIOS_MAX_MEM_32 altogether, replacing this use with an explicit
> 0xffffffff (or some other "max 32-bit value" symbol).  I don't think
> there's anything arch-specific about this.
>
> So I'd like to see two patches here:
>  1) Avoid allocating 64-bit regions for 32-bit BARs
>  2) Try to allocate regions above 4GB for 64-bit BARs

Sure. please check updated two patches.

Thanks

Yinghai

[-- Attachment #2: 32_bit_bar_allocation.patch --]
[-- Type: application/octet-stream, Size: 2556 bytes --]

Subject: [PATCH] PCI: cap resource allocation for 32bit bar for all archititures

Should not limit to x86 platform, that is pci requirement.

Add PCI_MAX_RESOURCE_32 and use that instead.

Also kill PCIBIOS_MAX_MEM_32.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>

---
arch/x86/include/asm/pci.h | 1 -
drivers/pci/bus.c | 4 ++--
drivers/pci/pci.h | 3 +++
include/linux/pci.h | 4 ----
4 files changed, 5 insertions(+), 7 deletions(-)

Index: linux-2.6/drivers/pci/bus.c
===================================================================
--- linux-2.6.orig/drivers/pci/bus.c
+++ linux-2.6/drivers/pci/bus.c
@@ -121,13 +121,13 @@ pci_bus_alloc_resource(struct pci_bus *b
{
int i, ret = -ENOMEM;
struct resource *r;
- resource_size_t max = -1;
+ resource_size_t max = PCI_MAX_RESOURCE;

type_mask |= IORESOURCE_IO | IORESOURCE_MEM;

/* don't allocate too high if the pref mem doesn't support 64bit*/
if (!(res->flags & IORESOURCE_MEM_64))
- max = PCIBIOS_MAX_MEM_32;
+ max = PCI_MAX_RESOURCE_32;

pci_bus_for_each_resource(bus, r, i) {
if (!r)
Index: linux-2.6/drivers/pci/pci.h
===================================================================
--- linux-2.6.orig/drivers/pci/pci.h
+++ linux-2.6/drivers/pci/pci.h
@@ -208,6 +208,9 @@ enum pci_bar_type {
pci_bar_mem64, /* A 64-bit memory BAR */
};

+#define PCI_MAX_RESOURCE ((resource_size_t)~0)
+#define PCI_MAX_RESOURCE_32 ((resource_size_t)0xffffffff)
+
bool pci_bus_read_dev_vendor_id(struct pci_bus *bus, int devfn, u32 *pl,
int crs_timeout);
extern int pci_setup_device(struct pci_dev *dev);
Index: linux-2.6/arch/x86/include/asm/pci.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/pci.h
+++ linux-2.6/arch/x86/include/asm/pci.h
@@ -149,7 +149,6 @@ void default_restore_msi_irqs(struct pci

/* generic pci stuff */
#include <asm-generic/pci.h>
-#define PCIBIOS_MAX_MEM_32 0xffffffff

#ifdef CONFIG_NUMA
/* Returns the node based on pci bus */
Index: linux-2.6/include/linux/pci.h
===================================================================
--- linux-2.6.orig/include/linux/pci.h
+++ linux-2.6/include/linux/pci.h
@@ -1340,10 +1340,6 @@ static inline int pci_domain_nr(struct p

#include <asm/pci.h>

-#ifndef PCIBIOS_MAX_MEM_32
-#define PCIBIOS_MAX_MEM_32 (-1)
-#endif
-
/* these helpers provide future and backwards compatibility
* for accessing popular PCI BAR info */
#define pci_resource_start(dev, bar) ((dev)->resource[(bar)].start)

[-- Attachment #3: allocate_high_at_first_v4.patch --]
[-- Type: application/octet-stream, Size: 2140 bytes --]

Subject: [PATCH] PCI: Try to allocate mem64 above 4G at first

and will fall back to below 4g if it can not find any above 4g.

x86 32bit without X86_PAE support will have bottom set to 0, becuase
resource_size_t is 32bit.

Also for 32bit with resource_size_t 64bit kernel on machine with pae support
we are safe because iomem_resource is limited to 32bit according to
x86_phys_bits.

-v2: update bottom assigning to make it clear for non-pae support machine.
-v3: Bjorn's change:
use MAX_REOURCE instead of -1
use start/end instead of bottom/max
for all arch instead of just x86_64
-v4: updated after PCI_MAX_RESOURCE_32 change.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>

---
drivers/pci/bus.c | 29 ++++++++++++++++++++++-------
1 file changed, 22 insertions(+), 7 deletions(-)

Index: linux-2.6/drivers/pci/bus.c
===================================================================
--- linux-2.6.orig/drivers/pci/bus.c
+++ linux-2.6/drivers/pci/bus.c
@@ -121,14 +121,23 @@ pci_bus_alloc_resource(struct pci_bus *b
{
int i, ret = -ENOMEM;
struct resource *r;
- resource_size_t max = PCI_MAX_RESOURCE;
+ resource_size_t end = PCI_MAX_RESOURCE;
+ resource_size_t start = 0;

type_mask |= IORESOURCE_IO | IORESOURCE_MEM;

- /* don't allocate too high if the pref mem doesn't support 64bit*/
- if (!(res->flags & IORESOURCE_MEM_64))
- max = PCI_MAX_RESOURCE_32;
+ /*
+ * don't allocate too high if the pref mem doesn't support 64bit
+ * also if this is a 64-bit mem resource, try above 4GB first
+ */
+ if (res->flags & IORESOURCE_MEM) {
+ if (!(res->flags & IORESOURCE_MEM_64))
+ end = PCI_MAX_RESOURCE_32;
+ else
+ start = (resource_size_t)(1ULL<<32);
+ }

+again:
pci_bus_for_each_resource(bus, r, i) {
if (!r)
continue;
@@ -145,12 +154,18 @@ pci_bus_alloc_resource(struct pci_bus *b

/* Ok, try it out.. */
ret = allocate_resource(r, res, size,
- r->start ? : min,
- max, align,
+ max(start, r->start ? : min),
+ end, align,
alignf, alignf_data);
if (ret == 0)
- break;
+ return 0;
+ }
+
+ if (start != 0) {
+ start = 0;
+ goto again;
}
+
return ret;
}