PCI resource initialisation

PCI resource initialisation

[David Monro]

Hi,

I need to make some changes to the way we do our PCI resource allocation
:-)

Currently the code goes like this (in pci.c:pcibios_init()):

        pci_scan_bus(0, &generic_pci_ops, NULL);
        if (ppc_md.pcibios_fixup)
                ppc_md.pcibios_fixup();
        pcibios_claim_resources(&pci_root_buses);

So we find all the devices, hack any we don't like, and then grab all
the resources. This means that we pretty much accept whatever the
firmware provides, which includes leaving any uninitialised resources
alone. This doesn't work properly on IBM 850 machines because the
firmware doesn't initialise any devices it doesn't recognise, and also
(for example) doesn't bother initialising the memory mapped resource of
the built-in ethernet (presumably because they expected people to use
the IO resource instead, which in fact we do).

The 2.4 kernel does provide a function,
pci_assign_unassigned_resources(), which will find pci resources which
haven't been claimed yet and do the appropriate thing with them.

As I see it we have a few choices:

do:
	pci_scan_bus(0, &generic_pci_ops, NULL);
        if (ppc_md.pcibios_fixup)
                ppc_md.pcibios_fixup();
        pcibios_claim_resources(&pci_root_buses);
	pci_assign_unassigned_resources();

which should allocate any left alone by the original code, and does
work, but still means we have to hack the original resources (eg on IBM
850s the SCSI hos adapters seem to always get allocated IO port
resources above 0x20000000, which we relocate in a not very safe manner
down to 0x01000000). It also assumes we don't need to do any more fixups
for the newly discovered resources.

or we could do:
	pci_scan_bus(0, &generic_pci_ops, NULL);
	pci_assign_unassigned_resources();
        if (ppc_md.pcibios_fixup)
                ppc_md.pcibios_fixup();

which results in the kernel completely reassigning all the PCI
resources. This should mean that the resources all get allocated in
areas that we would like to find them in, and means that the
pcibios_fixup code should be a little simpler. This is my preferred
solution, but I guess might break something.

Note that I can't really do this just for the PReP code; if I put the
pci_assign_unassigned_resources() call in the prep_pcibios_fixup()
function it will completely remap the bus and claim all the resources,
and then pcibios_claim_resources() will walk the tree trying to claim
each resource again and get a conflict on every resource!

Of course we could simply make pcibios_init() another arch-dependant
function - what do people think?

On a related, is it true to say that every driver which uses ioremap()
to map memory on cards into virtual space needs to have isa_mem_base
added to the base address before it will work? As far as I can tell
virtually no drivers do this at the moment; I note a rather
dirty-looking hack in chrp_pci.c where we hack the resource entries for
the matrox card so that the driver doesn't need rewriting. Or should we
simply do this for all memory resources on all cards?

Cheers,

	David

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PCI resource initialisation

[Gabriel Paubert]

On Sat, 26 Aug 2000, David Monro wrote:

>
> Hi,
>
> I need to make some changes to the way we do our PCI resource allocation
> :-)
>
> Currently the code goes like this (in pci.c:pcibios_init()):
>
>         pci_scan_bus(0, &generic_pci_ops, NULL);
>         if (ppc_md.pcibios_fixup)
>                 ppc_md.pcibios_fixup();
>         pcibios_claim_resources(&pci_root_buses);
>
> So we find all the devices, hack any we don't like, and then grab all
> the resources. This means that we pretty much accept whatever the
> firmware provides, which includes leaving any uninitialised resources
> alone. This doesn't work properly on IBM 850 machines because the
> firmware doesn't initialise any devices it doesn't recognise, and also
> (for example) doesn't bother initialising the memory mapped resource of
> the built-in ethernet (presumably because they expected people to use
> the IO resource instead, which in fact we do).

For PrePboot I took the decision to reassign all PCI resources before te
kernel even ahd a chance to run. It works quite well for me, but I had no
choice since I basically switch from a PreP to a CHRP mapping. Actually
given the variety of hardware we are trying to supprt, I suspect that
pushing a few things into the bootloaders (like reading the OF tree if
available) can only be better.

>
> The 2.4 kernel does provide a function,
> pci_assign_unassigned_resources(), which will find pci resources which
> haven't been claimed yet and do the appropriate thing with them.

Not always, unfortunately.

> As I see it we have a few choices:
>
> do:
> 	pci_scan_bus(0, &generic_pci_ops, NULL);
>         if (ppc_md.pcibios_fixup)
>                 ppc_md.pcibios_fixup();
>         pcibios_claim_resources(&pci_root_buses);
> 	pci_assign_unassigned_resources();
>
> which should allocate any left alone by the original code, and does
> work, but still means we have to hack the original resources (eg on IBM
> 850s the SCSI hos adapters seem to always get allocated IO port
> resources above 0x20000000, which we relocate in a not very safe manner

You mean completely unsafe, it has caused hangs in some cases.

> down to 0x01000000). It also assumes we don't need to do any more fixups
> for the newly discovered resources.

>
> or we could do:
> 	pci_scan_bus(0, &generic_pci_ops, NULL);
> 	pci_assign_unassigned_resources();
>         if (ppc_md.pcibios_fixup)
>                 ppc_md.pcibios_fixup();
>
> which results in the kernel completely reassigning all the PCI
> resources. This should mean that the resources all get allocated in
> areas that we would like to find them in, and means that the
> pcibios_fixup code should be a little simpler. This is my preferred
> solution, but I guess might break something.
>
> Note that I can't really do this just for the PReP code; if I put the
> pci_assign_unassigned_resources() call in the prep_pcibios_fixup()
> function it will completely remap the bus and claim all the resources,
> and then pcibios_claim_resources() will walk the tree trying to claim
> each resource again and get a conflict on every resource!
>
> Of course we could simply make pcibios_init() another arch-dependant
> function - what do people think?

I am fed up with all these arch dependant functions, to the point that
I am convinced that ppc_md should disappear.

> On a related, is it true to say that every driver which uses ioremap()
> to map memory on cards into virtual space needs to have isa_mem_base
> added to the base address before it will work? As far as I can tell
> virtually no drivers do this at the moment; I note a rather
> dirty-looking hack in chrp_pci.c where we hack the resource entries for
> the matrox card so that the driver doesn't need rewriting. Or should we
> simply do this for all memory resources on all cards?

Yes, you should do it. Actually I had something like this in my tree, but
it's now disabled since I switch the hardware to a CHRP map.

The resource code is still insufficient, there should be a way to have the
physical address, the bus address and in some cases the kernel virtual
address of a given resource. For I/O space, which is permanently mapped,
the resource should contain a kernel virtual address (the existence of
isa_io_base in itself is an aberration, have you counted how many lwz
instructions in the kernel access it ?). For memory resources, it should
contain a processor physical address which is the input to ioremap.

	Regards,
	Gabriel.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PCI resource initialisation

[Matt Porter]

On Sat, Aug 26, 2000 at 11:44:21PM +0200, Gabriel Paubert wrote:
>
> On Sat, 26 Aug 2000, David Monro wrote:
>
> >
> > Hi,
> >
> > I need to make some changes to the way we do our PCI resource allocation
> > :-)
> >
> > Currently the code goes like this (in pci.c:pcibios_init()):
> >
> >         pci_scan_bus(0, &generic_pci_ops, NULL);
> >         if (ppc_md.pcibios_fixup)
> >                 ppc_md.pcibios_fixup();
> >         pcibios_claim_resources(&pci_root_buses);
> >
> > So we find all the devices, hack any we don't like, and then grab all
> > the resources. This means that we pretty much accept whatever the
> > firmware provides, which includes leaving any uninitialised resources
> > alone. This doesn't work properly on IBM 850 machines because the
> > firmware doesn't initialise any devices it doesn't recognise, and also
> > (for example) doesn't bother initialising the memory mapped resource of
> > the built-in ethernet (presumably because they expected people to use
> > the IO resource instead, which in fact we do).
>
> For PrePboot I took the decision to reassign all PCI resources before te
> kernel even ahd a chance to run. It works quite well for me, but I had no
> choice since I basically switch from a PreP to a CHRP mapping. Actually
> given the variety of hardware we are trying to supprt, I suspect that
> pushing a few things into the bootloaders (like reading the OF tree if
> available) can only be better.

I wrote a PCI autoconfiguration library that executes during
pcibios_init() in the MontaVista 2.2 kernel to accomplish this.  It's
debatable as to where this should go but it was convenient to use
the kernel config space access methods in the library so that's one
advantage.

> >
> > Note that I can't really do this just for the PReP code; if I put the
> > pci_assign_unassigned_resources() call in the prep_pcibios_fixup()
> > function it will completely remap the bus and claim all the resources,
> > and then pcibios_claim_resources() will walk the tree trying to claim
> > each resource again and get a conflict on every resource!
> >
> > Of course we could simply make pcibios_init() another arch-dependant
> > function - what do people think?
>
> I am fed up with all these arch dependant functions, to the point that
> I am convinced that ppc_md should disappear.

I'd like to hear your plan for supporting the wide variety of PPC-based
hardware that depend on having ppc_md.  Is it preprocessor conditionals?

I made pcibios_init() a machdep in my 2.2 tree so I could properly setup
the PCI bus on a board by board basis.  There is no one size fits all
way to do this.

> > On a related, is it true to say that every driver which uses ioremap()
> > to map memory on cards into virtual space needs to have isa_mem_base
> > added to the base address before it will work? As far as I can tell
> > virtually no drivers do this at the moment; I note a rather
> > dirty-looking hack in chrp_pci.c where we hack the resource entries for
> > the matrox card so that the driver doesn't need rewriting. Or should we
> > simply do this for all memory resources on all cards?
>
> Yes, you should do it. Actually I had something like this in my tree, but
> it's now disabled since I switch the hardware to a CHRP map.

I've used the dirty hack too on one board I left in a PReP map.  The
real solution is to have a pci_to_cpu() call which should be required
for all architectures/drivers.  It actually goes further since we really
need to be able to specify a pci bus domain (due to multiple physical
PCI bus segments with their own address spaces).

> The resource code is still insufficient, there should be a way to have the
> physical address, the bus address and in some cases the kernel virtual
> address of a given resource. For I/O space, which is permanently mapped,
> the resource should contain a kernel virtual address (the existence of
> isa_io_base in itself is an aberration, have you counted how many lwz
> instructions in the kernel access it ?). For memory resources, it should
> contain a processor physical address which is the input to ioremap.

pci_to_cpu() would do this without changing the resource code, though
though maybe loading up the resource code with this information would
be a good arch independent way to provide the information to PCI drivers.
An interesting thought...

--
Matt Porter
mmporter@home.com
This is Linux Country. On a quiet night, you can hear Windows reboot.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PCI resource initialisation

[Geert Uytterhoeven]

On Sat, 26 Aug 2000, David Monro wrote:
> As I see it we have a few choices:
>
> do:
> 	pci_scan_bus(0, &generic_pci_ops, NULL);
>         if (ppc_md.pcibios_fixup)
>                 ppc_md.pcibios_fixup();
>         pcibios_claim_resources(&pci_root_buses);
> 	pci_assign_unassigned_resources();
>
> which should allocate any left alone by the original code, and does
> work, but still means we have to hack the original resources (eg on IBM
> 850s the SCSI hos adapters seem to always get allocated IO port
> resources above 0x20000000, which we relocate in a not very safe manner
> down to 0x01000000). It also assumes we don't need to do any more fixups
> for the newly discovered resources.
>
> or we could do:
> 	pci_scan_bus(0, &generic_pci_ops, NULL);
> 	pci_assign_unassigned_resources();
>         if (ppc_md.pcibios_fixup)
>                 ppc_md.pcibios_fixup();
>
> which results in the kernel completely reassigning all the PCI
> resources. This should mean that the resources all get allocated in
> areas that we would like to find them in, and means that the
> pcibios_fixup code should be a little simpler. This is my preferred
> solution, but I guess might break something.

You could also start from my patches for the CHRP LongTrail (which seems to
work fine on PowerMac as well, especially on 7200 with extra SCSI cards :-).
It assigns resources to all devices that don't have (valid) resources
assigned.

By making sure you fill in the parent pointer of each PCI device you can
control the region resources should be allocated from (see the LongTrail code
for an example). If this parent resource doesn't contain addresses above
0x20000000, the kernel will relocate the SCSI in the 850, just like you want.
That way you don't need an explicit fixup to reassign it.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PCI resource initialisation

[Michel Lanners]

On  28 Aug, this message from Geert Uytterhoeven echoed through cyberspace:
> You could also start from my patches for the CHRP LongTrail (which seems to
> work fine on PowerMac as well, especially on 7200 with extra SCSI cards :-).
> It assigns resources to all devices that don't have (valid) resources
> assigned.

I already wanted to point out your patches, Geert :-)

How about actually getting them _into_ the PPC kernel trees? Do you have
any plans in that direction?

Cheers

Michel

-------------------------------------------------------------------------
Michel Lanners                 |  " Read Philosophy.  Study Art.
23, Rue Paul Henkes            |    Ask Questions.  Make Mistakes.
L-1710 Luxembourg              |
email   mlan@cpu.lu            |
http://www.cpu.lu/~mlan        |                     Learn Always. "


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PCI resource initialisation

[Geert Uytterhoeven]

On Tue, 29 Aug 2000, Michel Lanners wrote:
> On  28 Aug, this message from Geert Uytterhoeven echoed through cyberspace:
> > You could also start from my patches for the CHRP LongTrail (which seems to
> > work fine on PowerMac as well, especially on 7200 with extra SCSI cards :-).
> > It assigns resources to all devices that don't have (valid) resources
> > assigned.
>
> I already wanted to point out your patches, Geert :-)
>
> How about actually getting them _into_ the PPC kernel trees? Do you have
> any plans in that direction?

I do have plans, but since I'm currently unable to do anything with the
bitkeeper tree, I still hope some other bk power will do it for me [hint!]...

Besides, it would also allow us to remove the workaround in atyfb for broken OF
in some PowerBooks, i.e. _revert_ the following patch which went into 2.3.51:

--- v2.3.50/linux/drivers/video/atyfb.c	Sat Feb 26 22:31:51 2000
+++ linux/drivers/video/atyfb.c	Fri Mar 10 09:43:04 2000
@@ -2689,15 +2689,6 @@
     fix->smem_start = info->frame_buffer_phys;
     fix->smem_len = (u32)info->total_vram;

-#ifdef __LITTLE_ENDIAN
-    /*
-     *  Last page of 8 MB little-endian aperture is MMIO
-     *  FIXME: we should use the auxiliary aperture instead so we can acces the
-     *  full 8 MB of video RAM on 8 MB boards
-     */
-    if (fix->smem_len > 0x800000-GUI_RESERVE)
-	fix->smem_len = 0x800000-GUI_RESERVE;
-#endif
     /*
      *  Reg Block 0 (CT-compatible block) is at ati_regbase_phys
      *  Reg Block 1 (multimedia extensions) is at ati_regbase_phys-0x400
@@ -3501,11 +3492,14 @@
     }
 #endif

-    if (info->bus_type == ISA)
-	if ((info->total_vram == 0x400000) || (info->total_vram == 0x800000)) {
-	    /* protect GUI-regs if complete Aperture is VRAM */
+    /*
+     *  Last page of 8 MB (4 MB on ISA) aperture is MMIO
+     *  FIXME: we should use the auxiliary aperture instead so we can acces the
+     *  full 8 MB of video RAM on 8 MB boards
+     */
+    if (info->total_vram == 0x800000 ||
+	(info->bus_type == ISA && info->total_vram == 0x400000))
 	    info->total_vram -= GUI_RESERVE;
-	}

     /* Clear the video memory */
     fb_memset((void *)info->frame_buffer, 0, info->total_vram);


Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/