Re: [PATCH] pci: pciehp update the slot bridge res to get big range for pcie devices

[ebiederm@xmission.com (Eric W. Biederman)]

Bjorn Helgaas <bjorn.helgaas@hp.com> writes:

> On Thu, 2009-10-29 at 08:13 -0700, Eric W. Biederman wrote:
>> Bjorn Helgaas <bjorn.helgaas@hp.com> writes:
>> 
>> > On Thu, 2009-10-29 at 01:16 -0700, Eric W. Biederman wrote:
>> >> Yinghai Lu <yinghai@kernel.org> writes:
>> >> >
>> >> > after closing look up the code, it looks it will not break your setup.
>> >> >
>> >> > 1. before the patches:
>> >> > a. when master card is inserted, all bridge in that card will get assigned with min_size
>> >> > b. when new cards is inserted to those slots in master card, will get assigned in the bridge size.
>> >> >
>> >> > 2. after the patches: v5
>> >> > a. booted up, all leaf bridge mmio get clearred.
>> >> > b. when master card is inserted, all bridge in that card will get assigned with min_size, and master bridge will be sum of them
>> >> > c. when new cards is inserted to those slots in master card, will get assigned in the bridge size.
>> >> >
>> >> > can you check those two patches in your setup to verify it?
>> >> 
>> >> I have a much simpler case I will break, as I tried something similar by accident.
>> >> 
>> >> AMD cpu MCP55 with one pcie port setup as hotplug.
>> >> The system only has 2GB of RAM.  So plenty of space for pcie devices.
>> >> 
>> >> If the firmware assigns nothing and linux at boot time assigns the pci mmio space:
>> >> Reads from the bar of the hotplugged device work
>> >> Writes to the bar of the hotplugged device, cause further writes to go to lala land.
>> >> 
>> >> So I had to have the firmware make the assignment, because only it knows the
>> >> details of the hidden AMD bar registers for each hypertransport chain etc.
>> >
>> > Do you mean you had to have firmware program a hot-added device, or just
>> > that firmware had to program the apertures of the root port that was
>> > present at boot, even though it had no devices below it?
>> 
>> Firmware had to program the apertures of the root port that was present
>> at boot, even though it had no devices below it.
>> 
>> > Firmware normally supplies ACPI _CRS information that tells us how it
>> > programmed the host bridge windows.  On x86, Linux normally ignores that
>> > and just assumes a range based on memory size.  If we paid attention to
>> > it (as with "pci=use_crs"), it's likely that we could do a better job of
>> > doing this setup.
>> >
>> > Or, of course, we could add a Linux driver that knows about "the hidden
>> > AMD bar registers."  But I think that should be a last resort, for when
>> > firmware supplied incorrect _CRS information.
>> 
>> In this case there was no ACPI, and even if there was correct _CRS information
>> it would have said only those addresses routed to bars/apertures on the
>> root bridge was routed to the MCP55.  So while it looked like we had gobs
>> of unallocated space we could use.  In practice we did not.
>
> I know this is a hypothetical case since you don't have ACPI, but I'm
> curious about this.
>
> I assume the magic AMD BARs only affect the host bridge, and that the
> downstream root ports look like standard PCI-to-PCI bridges.  If that's
> the case, and if we have correct descriptions of the host bridge
> apertures, Linux should theoretically be able to do as well as firmware.
>
> But you seem to be suggesting that even with a correct host bridge
> description, there's space that *looks* available but is not.  I don't
> understand how this can be.

What I meant was simply that not all of the non-memory space was
routed down the hypertransport chain to the mcp55.  If you have an
accurate description of that you should be fine.

Eric