PReP and generic PCI resource assignment

PReP and generic PCI resource assignment

[Hollis]

I'm trying to boot _2_4_devel on a Thinkpad 850. The only problem comes when
the video controller's PCI resource 1 (mapped by the firmware to
0x0..0x00ffffff) is relocated to 0xc1000000..0xc1ffffff. [The symptom is that
all new VGA text is drawn backwards-endian, which I would not have expected
but is very visible.] That's way out of bounds for PReP IO memory, which in
bus addresses can only be 0x0..0x40000000 (in CPU addresses that's
0xc0000000..0xfeffffff or so).

How did it get moved that drastically? The problem comes from generic PCI
code, starting with pci_assign_resource (called from
pcibios_assign_resources). The actual bad line of code is in
kernel/resource.c find_resource():
	new->start = root->start;
where it starts the new resource at the bottom of the parent resource's
range.

The problem is that root in this case is the PCI memory resource of the host
bridge, which correctly starts at 0xc0000000... but NOT in bus terms. In bus
terms it starts at 0x0. (CPU physical 0xc0000000 = PCI bus 0x0 [IO mem].) So
the new resource is assigned 0xc1000000, which is written back to the BAR
with pcibios_update_resource... which is way wrong.

Now if resources could be assigned properly in the first place this code path
wouldn't be taken, but quite a few things fail (request_resource's and
pci_find_parent_resource's) and it's proving difficult to track down why
(maybe I've been staring at this too long).

Anyways, when I change "root->start" above to be 0, the symptom is fixed, but
that's clearly a hack. I don't see how anything PReP (system IO nor IO
memory) could escape generic resource assignment unscathed though.

Thoughts?

-Hollis

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PReP and generic PCI resource assignment

[ashish anand]

if I remember correctly there should be some a board specific variable like pci_dram_offset
it is zero for your case.
0xc1000000 is a virtual address ..while pci bar should be programmed with bus address .
your case bios is not apropiately updating the resource assignment with 0xc0000000 , the kernel_base.

Hollis wrote:
>
> I'm trying to boot _2_4_devel on a Thinkpad 850. The only problem comes when
> the video controller's PCI resource 1 (mapped by the firmware to
> 0x0..0x00ffffff) is relocated to 0xc1000000..0xc1ffffff. [The symptom is that
> all new VGA text is drawn backwards-endian, which I would not have expected
> but is very visible.] That's way out of bounds for PReP IO memory, which in
> bus addresses can only be 0x0..0x40000000 (in CPU addresses that's
> 0xc0000000..0xfeffffff or so).
>
> How did it get moved that drastically? The problem comes from generic PCI
> code, starting with pci_assign_resource (called from
> pcibios_assign_resources). The actual bad line of code is in
> kernel/resource.c find_resource():
>         new->start = root->start;
> where it starts the new resource at the bottom of the parent resource's
> range.
>
> The problem is that root in this case is the PCI memory resource of the host
> bridge, which correctly starts at 0xc0000000... but NOT in bus terms. In bus
> terms it starts at 0x0. (CPU physical 0xc0000000 = PCI bus 0x0 [IO mem].) So
> the new resource is assigned 0xc1000000, which is written back to the BAR
> with pcibios_update_resource... which is way wrong.
>
> Now if resources could be assigned properly in the first place this code path
> wouldn't be taken, but quite a few things fail (request_resource's and
> pci_find_parent_resource's) and it's proving difficult to track down why
> (maybe I've been staring at this too long).
>
> Anyways, when I change "root->start" above to be 0, the symptom is fixed, but
> that's clearly a hack. I don't see how anything PReP (system IO nor IO
> memory) could escape generic resource assignment unscathed though.
>
> Thoughts?
>
> -Hollis
>

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PReP and generic PCI resource assignment

[Benjamin Herrenschmidt]

>
>The problem is that root in this case is the PCI memory resource of the host
>bridge, which correctly starts at 0xc0000000... but NOT in bus terms. In bus
>terms it starts at 0x0. (CPU physical 0xc0000000 = PCI bus 0x0 [IO mem].) So
>the new resource is assigned 0xc1000000, which is written back to the BAR
>with pcibios_update_resource... which is way wrong.

The resource is in CPU space. The problem must be with
pcibios_update_resource, which is responsible for doing the proper
offset. If you look closely, it substracts hose->pci_mem_offset from the
resource before writing it to the BAR.
If your hose pci_mem_offset is wrong, then it can't work. It should be
0xc0000000 on PReP.

>Now if resources could be assigned properly in the first place this code path
>wouldn't be taken, but quite a few things fail (request_resource's and
>pci_find_parent_resource's) and it's proving difficult to track down why
>(maybe I've been staring at this too long).
>
>Anyways, when I change "root->start" above to be 0, the symptom is fixed, but
>that's clearly a hack. I don't see how anything PReP (system IO nor IO
>memory) could escape generic resource assignment unscathed though.

The fix is to have pci_mem_offset set properly when setting up the
pci_controller structure.

ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PReP and generic PCI resource assignment

[Benjamin Herrenschmidt]

>
>if I remember correctly there should be some a board specific variable
>like pci_dram_offset
>it is zero for your case.
>0xc1000000 is a virtual address ..while pci bar should be programmed with
>bus address .
>your case bios is not apropiately updating the resource assignment with
>0xc0000000 , the kernel_base.

pci_dram_offset is the offset of system memory as seen by PCI devices,
what Hollis needs to look at is the pci_controller's pci_mem_offset.

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PReP and generic PCI resource assignment

[Hollis]

On Wednesday 08 August 2001 05:27, Benjamin Herrenschmidt wrote:
> >The problem is that root in this case is the PCI memory resource of the
> > host bridge, which correctly starts at 0xc0000000... but NOT in bus
> > terms. In bus terms it starts at 0x0. (CPU physical 0xc0000000 = PCI bus
> > 0x0 [IO mem].) So the new resource is assigned 0xc1000000, which is
> > written back to the BAR with pcibios_update_resource... which is way
> > wrong.
>
> The resource is in CPU space. The problem must be with
> pcibios_update_resource, which is responsible for doing the proper
> offset. If you look closely, it substracts hose->pci_mem_offset from the
> resource before writing it to the BAR.
> If your hose pci_mem_offset is wrong, then it can't work. It should be
> 0xc0000000 on PReP.

Yes, it is. I misinterpretted my debug output... :/

At any rate, relocating PCI resource 1 on this controller from 0x0 to
0x01000000 causes my VGA console to go backwards endian. I don't know why
this would be the case... Re-moving it back to 0x0 fixes the symptom. Any
ideas on why this could happen? I think VGA is all IO, no memory at all?

I'm curious about PCIBIOS_MIN_MEM... Why does that exist, and why was it
given that value?

-Hollis

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PReP and generic PCI resource assignment

[Benjamin Herrenschmidt]

>At any rate, relocating PCI resource 1 on this controller from 0x0 to
>0x01000000 causes my VGA console to go backwards endian. I don't know why
>this would be the case... Re-moving it back to 0x0 fixes the symptom. Any
>ideas on why this could happen? I think VGA is all IO, no memory at all?

VGA is both IO and memory. It's possible that your VGA card is so broken
that it only use the low-order bits of addresses and use the high addresses
as flags, for example to access a "bug endian" aperture :) That would suck
as it would mean you actually have address aliasing going on on the bus,
possiby causing weird conflicts.

>I'm curious about PCIBIOS_MIN_MEM... Why does that exist, and why was it
>given that value?

Note sure, I don't really like the way it's used and hard coded....

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PReP and generic PCI resource assignment

[Matt Porter]

On Thu, Aug 09, 2001 at 12:18:33PM +0200, Benjamin Herrenschmidt wrote:
> >At any rate, relocating PCI resource 1 on this controller from 0x0 to
> >0x01000000 causes my VGA console to go backwards endian. I don't know why
> >this would be the case... Re-moving it back to 0x0 fixes the symptom. Any
> >ideas on why this could happen? I think VGA is all IO, no memory at all?
>
> VGA is both IO and memory. It's possible that your VGA card is so broken
> that it only use the low-order bits of addresses and use the high addresses
> as flags, for example to access a "bug endian" aperture :) That would suck
> as it would mean you actually have address aliasing going on on the bus,
> possiby causing weird conflicts.
>
> >I'm curious about PCIBIOS_MIN_MEM... Why does that exist, and why was it
> >given that value?
>
> Note sure, I don't really like the way it's used and hard coded....

That "generic" code has caused me a number of problem on many PCI
equipped embedded platforms (not just PPC ones).  The algorithm
used to pick PCI Mem space addresses ignores the fact that there are
specific areas in CPU address space captured to generate mem cycles.
>From my read of the code, it simply picks any old address that's
not claimed by another resource starting at PCIBIOS_MIN_MEM.  That's
braindead and probably just another x86ism that'll work on those
platforms.

This is why I always use the pci_auto.c code to properly place base
addresses so the PCI resource system won't try to use it's braindead
algorithm to move them around.

--
Matt Porter
MontaVista Software, Inc.
mporter@mvista.com

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PReP and generic PCI resource assignment

[Geert Uytterhoeven]

On Thu, 9 Aug 2001, Matt Porter wrote:
> On Thu, Aug 09, 2001 at 12:18:33PM +0200, Benjamin Herrenschmidt wrote:
> > >At any rate, relocating PCI resource 1 on this controller from 0x0 to
> > >0x01000000 causes my VGA console to go backwards endian. I don't know why
> > >this would be the case... Re-moving it back to 0x0 fixes the symptom. Any
> > >ideas on why this could happen? I think VGA is all IO, no memory at all?
> >
> > VGA is both IO and memory. It's possible that your VGA card is so broken
> > that it only use the low-order bits of addresses and use the high addresses
> > as flags, for example to access a "bug endian" aperture :) That would suck
> > as it would mean you actually have address aliasing going on on the bus,
> > possiby causing weird conflicts.
> >
> > >I'm curious about PCIBIOS_MIN_MEM... Why does that exist, and why was it
> > >given that value?
> >
> > Note sure, I don't really like the way it's used and hard coded....
>
> That "generic" code has caused me a number of problem on many PCI
> equipped embedded platforms (not just PPC ones).  The algorithm
> used to pick PCI Mem space addresses ignores the fact that there are
> specific areas in CPU address space captured to generate mem cycles.
> >From my read of the code, it simply picks any old address that's
> not claimed by another resource starting at PCIBIOS_MIN_MEM.  That's
> braindead and probably just another x86ism that'll work on those
> platforms.

You should initialize the parent pointers in the PCI bus structure with
pointers to resources containing the correct region.

I once hardcoded the parents for the LongTrail, but that code is gone. Now it
tries to set up the parents from information in the OF device, which works
more or less. There are (or were, read as in `were the last time I tried it, a
few months ago) still some problems, though. One of them is that the hardcoded
ISA motherboard resources (i8259 and legacy stuff) are requested before PCI
resources are done, and that the PCI bus parent resources were not requested
and thus don't appear in the full resource tree.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PReP and generic PCI resource assignment

[Benjamin Herrenschmidt]

>You should initialize the parent pointers in the PCI bus structure with
>pointers to resources containing the correct region.
>
>I once hardcoded the parents for the LongTrail, but that code is gone. Now it
>tries to set up the parents from information in the OF device, which works
>more or less. There are (or were, read as in `were the last time I tried
it, a
>few months ago) still some problems, though. One of them is that the
hardcoded
>ISA motherboard resources (i8259 and legacy stuff) are requested before PCI
>resources are done, and that the PCI bus parent resources were not requested
>and thus don't appear in the full resource tree.

Parent pointer should be set properly nowadays on bk _2_4_devel at least.

However, the generic PCI code still wants to add that PCIBIOS_MIN_MEM
value when assigning resources, I bet we should probably #define that
to 0 on PPC.

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PReP and generic PCI resource assignment

[Geert Uytterhoeven]

On Fri, 10 Aug 2001, Benjamin Herrenschmidt wrote:
> >You should initialize the parent pointers in the PCI bus structure with
> >pointers to resources containing the correct region.
> >
> >I once hardcoded the parents for the LongTrail, but that code is gone. Now it
> >tries to set up the parents from information in the OF device, which works
> >more or less. There are (or were, read as in `were the last time I tried
> it, a
> >few months ago) still some problems, though. One of them is that the
> hardcoded
> >ISA motherboard resources (i8259 and legacy stuff) are requested before PCI
> >resources are done, and that the PCI bus parent resources were not requested
> >and thus don't appear in the full resource tree.
>
> Parent pointer should be set properly nowadays on bk _2_4_devel at least.

Are the parent resources registered using request_resource() yet?

Furthermore legacy resources should be requested after that, so they are
requested from the correct region, and no overlaps will occur. I had some
(fortunately harmless, it still worked) conflicts, because the PCI resource
tree and the normal I/O resources where in two separate resource trees.

> However, the generic PCI code still wants to add that PCIBIOS_MIN_MEM
> value when assigning resources, I bet we should probably #define that
> to 0 on PPC.

Yes, that should be OK, iff all PIC buses have valid parent resources.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PReP and generic PCI resource assignment

[Hollis Blanchard]

On Fri, Aug 10, 2001 at 10:38:04AM +0200, Geert Uytterhoeven wrote:
>
> There are (or were, read as in `were the last time I
> tried it, a few months ago) still some problems, though. One of them is that
> the hardcoded ISA motherboard resources (i8259 and legacy stuff) are
> requested before PCI resources are done, and that the PCI bus parent
> resources were not requested and thus don't appear in the full resource
> tree.

There is still that problem with VGA console, which does a
request_resource(&ioport_resource,...) after the PCI host bridge has been
found, but well before the PCI code has reserved all of *its* resources. So
reserving 0-ffffffff IO fails because VGA already grabbed a tiny chunk at
0x3d0 or so.

Even regardless of order, VGA console is requesting directly from
ioport_resource, which probably should be changed (Paul suggested
request_region instead)...

I didn't see any similar problem with i8259, but I wasn't looking for it
either.

I get a ton of failed request_resources on the boxes I play with the most, but
luckily most things work ok. Except for this video controller, which doesn't
like being moved away from 0x0... :/

-Hollis

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PReP and generic PCI resource assignment

[Geert Uytterhoeven]

On Fri, 10 Aug 2001, Hollis Blanchard wrote:
> On Fri, Aug 10, 2001 at 10:38:04AM +0200, Geert Uytterhoeven wrote:
> > There are (or were, read as in `were the last time I
> > tried it, a few months ago) still some problems, though. One of them is that
> > the hardcoded ISA motherboard resources (i8259 and legacy stuff) are
> > requested before PCI resources are done, and that the PCI bus parent
> > resources were not requested and thus don't appear in the full resource
> > tree.
>
> There is still that problem with VGA console, which does a
> request_resource(&ioport_resource,...) after the PCI host bridge has been
> found, but well before the PCI code has reserved all of *its* resources. So
> reserving 0-ffffffff IO fails because VGA already grabbed a tiny chunk at
> 0x3d0 or so.
>
> Even regardless of order, VGA console is requesting directly from
> ioport_resource, which probably should be changed (Paul suggested
> request_region instead)...

Yes, using request_region() is the correct fix.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/