[RFC] Runtime allocation of PCI resources

[RFC] Runtime allocation of PCI resources

[Matthew Garrett]

BIOS authors don't always program all the features of hardware. This is 
often the case for Intel IDE controllers, which are usually able to run 
in AHCI mode but are rarely configured to do so. Reprogramming them is 
easy enough other than the requirement for some MMIO space. If the BIOS 
hasn't allocated this, it's necessary for us to do so manually.

This patch simply attempts to find some free address space and allocate 
it. The function is designed to be called after the rest of the 
resources have been allocated in order to avoid colliding with other 
devices.

This seems to work fine on x86, but I'm happy to admit that I know 
little about the intricacies of how PCI is implemented elsewhere. Is 
this approach valid in the general case? I'm also working on the 
assumption that the firmware will program hotplug PCI hardware in a way 
that isn't going to end up colliding with this. Anyway, any 
thoughts/comments?

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index e48fcf0..1e188e8 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -126,7 +126,7 @@ static inline unsigned int pci_calc_resource_flags(unsigned int flags)
 /*
  * Find the extent of a PCI decode..
  */
-static u32 pci_size(u32 base, u32 maxbase, u32 mask)
+u32 pci_size(u32 base, u32 maxbase, u32 mask)
 {
 	u32 size = mask & maxbase;	/* Find the significant bits */
 	if (!size)
diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index 6dfd861..37d484c 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -25,6 +25,51 @@
 #include <linux/slab.h>
 #include "pci.h"
 
+int
+pci_allocate_resource(struct pci_dev *dev, int resno)
+{
+	int err;
+	u32 size, mask, reg;
+	resource_size_t start;
+	struct resource *res = &dev->resource[resno];
+	struct resource *bres = pci_find_parent_resource(dev, res);
+
+	if (!bres)
+		return -ENOMEM;
+	
+	if (res->flags & IORESOURCE_IO) {
+		mask = (u32)PCI_BASE_ADDRESS_IO_MASK;
+		start = PCIBIOS_MIN_IO;
+	} else {
+		mask = (u32)PCI_BASE_ADDRESS_MEM_MASK;
+		start = PCIBIOS_MIN_MEM;
+	}
+
+	reg = PCI_BASE_ADDRESS_0 + 4 * resno;
+
+	pci_write_config_dword(dev, reg, ~0);
+	pci_read_config_dword(dev, reg, &size);
+
+	if (!size || size == 0xffffffff)
+		return 0;
+
+	size = pci_size(0, size, mask);
+	
+	if (res->start != 0 && res->end != 0)
+		release_resource (res);
+
+	err = allocate_resource (bres, res, size+1, start, ~0U, 1024, NULL,
+				 NULL);
+
+	if (err)
+		return err;
+
+	pci_update_resource (dev, res, resno);
+
+	return 0;
+}
+
+EXPORT_SYMBOL(pci_allocate_resource);
 
 void
 pci_update_resource(struct pci_dev *dev, struct resource *res, int resno)
diff --git a/include/linux/pci.h b/include/linux/pci.h
index fbf3766..502c34e 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -549,6 +549,7 @@ void pci_intx(struct pci_dev *dev, int enable);
 void pci_msi_off(struct pci_dev *dev);
 int pci_set_dma_mask(struct pci_dev *dev, u64 mask);
 int pci_set_consistent_dma_mask(struct pci_dev *dev, u64 mask);
+int pci_allocate_resource(struct pci_dev *dev, int resno);
 void pci_update_resource(struct pci_dev *dev, struct resource *res, int resno);
 int __must_check pci_assign_resource(struct pci_dev *dev, int i);
 int __must_check pci_assign_resource_fixed(struct pci_dev *dev, int i);
@@ -608,6 +609,7 @@ void pci_remove_behind_bridge(struct pci_dev *);
 struct pci_driver *pci_dev_driver(const struct pci_dev *);
 const struct pci_device_id *pci_match_device(struct pci_driver *drv, struct pci_dev *dev);
 const struct pci_device_id *pci_match_id(const struct pci_device_id *ids, struct pci_dev *dev);
+u32 pci_size(u32 base, u32 maxbase, u32 mask);
 int pci_scan_bridge(struct pci_bus *bus, struct pci_dev * dev, int max, int pass);
 
 void pci_walk_bus(struct pci_bus *top, void (*cb)(struct pci_dev *, void *),

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [RFC] Runtime allocation of PCI resources

[Matthew Garrett]

On Mon, Jun 11, 2007 at 08:11:24PM +0200, Andi Kleen wrote:
> Matthew Garrett <mjg59@srcf.ucam.org> writes:
> 
> > BIOS authors don't always program all the features of hardware. This is 
> > often the case for Intel IDE controllers, which are usually able to run 
> > in AHCI mode but are rarely configured to do so. Reprogramming them is 
> > easy enough other than the requirement for some MMIO space. If the BIOS 
> > hasn't allocated this, it's necessary for us to do so manually.
> 
> Traditionally we've had bad experiences forcing hardware to do something
> the BIOS didn't consider. e.g. occasionally the BIOS has good reasons
> to not allow it; maybe it knows about some errata or other problem
> the kernel doesn't. 

AHCI is generally disabled because it makes Windows insanely difficult 
to install. I'm not aware of any hardware issues in this case.

> I tried something similar some time ago for the IOMMU aperture.
> It turned out that some systems put something into the e820 holes
> and didn't boot anymore if you put something else in there.
> Your resource allocation will just do that.
> 
> You might be lucky because your resources are typically
> small (IOMMU aperture was 64+MB) or you might not.

Hm. Yes, that would be a problem (sigh BIOS authors).
-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [RFC] Runtime allocation of PCI resources

[Yinghai Lu]

On 11 Jun 2007 20:11:24 +0200, Andi Kleen <andi@firstfloor.org> wrote:
> Matthew Garrett <mjg59@srcf.ucam.org> writes:
>
> > BIOS authors don't always program all the features of hardware. This is
> > often the case for Intel IDE controllers, which are usually able to run
> > in AHCI mode but are rarely configured to do so. Reprogramming them is
> > easy enough other than the requirement for some MMIO space. If the BIOS
> > hasn't allocated this, it's necessary for us to do so manually.
>
> Traditionally we've had bad experiences forcing hardware to do something
> the BIOS didn't consider. e.g. occasionally the BIOS has good reasons
> to not allow it; maybe it knows about some errata or other problem
> the kernel doesn't.
>
> If you do this then don't make it default at least, but only allow
> to force it use an option.

Esp for AMD system with several Hypertranport chain. try to allocate
pci resource in kernel  also need some region adjust in NB
configuration space.

So leave some work to ACPI DSDT AML code at least

YH

Re: [RFC] Runtime allocation of PCI resources

[Andi Kleen]

Matthew Garrett <mjg59@srcf.ucam.org> writes:

> BIOS authors don't always program all the features of hardware. This is 
> often the case for Intel IDE controllers, which are usually able to run 
> in AHCI mode but are rarely configured to do so. Reprogramming them is 
> easy enough other than the requirement for some MMIO space. If the BIOS 
> hasn't allocated this, it's necessary for us to do so manually.

Traditionally we've had bad experiences forcing hardware to do something
the BIOS didn't consider. e.g. occasionally the BIOS has good reasons
to not allow it; maybe it knows about some errata or other problem
the kernel doesn't. 

If you do this then don't make it default at least, but only allow
to force it use an option.
 
> This patch simply attempts to find some free address space and allocate 
> it. The function is designed to be called after the rest of the 
> resources have been allocated in order to avoid colliding with other 
> devices.
> 
> This seems to work fine on x86, but I'm happy to admit that I know 
> little about the intricacies of how PCI is implemented elsewhere. Is 
> this approach valid in the general case? I'm also working on the 
> assumption that the firmware will program hotplug PCI hardware in a way 
> that isn't going to end up colliding with this. Anyway, any 
> thoughts/comments?

I tried something similar some time ago for the IOMMU aperture.
It turned out that some systems put something into the e820 holes
and didn't boot anymore if you put something else in there.
Your resource allocation will just do that.

You might be lucky because your resources are typically
small (IOMMU aperture was 64+MB) or you might not.

These problems are unfortunately very hard to debug; typically
you have to bisect.

-Andi