Re: PCI bridge range sizing bug

[Jesse Barnes <jesse.barnes@intel.com>]

On Thursday, April 5, 2007 3:37 pm Adam Jackson wrote:
> So I'm attempting to do something fairly heinous (X server across
> five video cards), and I hit a fun bug in bridge range setup.  See
> attached lspci and dmesg, but the short of it is I've got two VGA
> chips on one card behind a bridge, which is itself behind a second
> PCI bridge, and the bridge ranges get set up so that I can't map the
> ROMs, which means I can't post them, and therefore can't use them
> period.
>
> The alignment restriction on the ROMs seems a bit extreme:
>
> % sudo setpci -s 7:2 ROM_ADDRESS=ffffffff
> % sudo setpci -s 7:2 ROM_ADDRESS
> f0000001
>
> (same for 7:1) so that might be part of the problem.

...
Allocating PCI resources starting at 88000000 (gap: 80000000:7ff00000)
...

That's ~2G of space, which should be plenty for your PCI resources I 
hope?  If you have a bunch of cards with large BARS though you might be 
running out.

...
PCI: Bridge: 0000:00:01.0
  IO window: 4000-4fff
  MEM window: a3500000-a35fffff (1M)
  PREFETCH window: 90000000-97ffffff
PCI: Bridge: 0000:00:03.0
  IO window: disabled.
  MEM window: a3400000-a34fffff (1M)
  PREFETCH window: 98000000-9fffffff
PCI: Bridge: 0000:00:1c.0
  IO window: disabled.
  MEM window: a3300000-a33fffff (1M)
  PREFETCH window: 80000000-8fffffff
PCI: Bridge: 0000:00:1c.4
  IO window: 3000-3fff
  MEM window: a3200000-a32fffff (1M)
  PREFETCH window: a3700000-a37fffff
PCI: Bridge: 0000:00:1c.5
  IO window: 2000-2fff
  MEM window: a3100000-a31fffff (1M)
  PREFETCH window: disabled.
PCI: Failed to allocate mem resource #6:10000000@b0000000 for 
0000:07:01.0
PCI: Failed to allocate mem resource #6:10000000@b0000000 for 
0000:07:02.0
...

Yep, looks like those two devices had a problem.  Supposedly they want 
to sit at 256M?  Given that we're only giving each bridge 1M of memory 
space that would definitely be a problem.

The total so far is only 5M of PCI space... so we're not making good use 
of the 2G we were given.

...
PCI: Bridge: 0000:06:00.0
  IO window: disabled.
  MEM window: a1000000-a2ffffff (32M)
  PREFETCH window: disabled.
PCI: Bridge: 0000:00:1e.0
  IO window: 1000-1fff
  MEM window: a1000000-a30fffff (~32M)
  PREFETCH window: a0000000-a0ffffff
...

And these bridges got more space somehow...  Greg who's in charge of our 
bridge resource allocation code?

Jesse