Re: Adding 'bus_offset' to pci_controller

Re: Adding 'bus_offset' to pci_controller

[Matt Porter]

On Thu, Sep 27, 2001 at 10:59:32AM +1000, Paul Mackerras wrote:
>
> Mark A. Greer writes:
>
> > Wait, I can't b/c then *everyone* will have this subtraction happen and
> > everyone but me would be broken.
> > With a separate field, I don't mess up other  board ports.
>
> Well, how many other boards are there out there that use
> indirect_pci.c and have more than one host bridge?  Anyone?

The CPC710 ports, K2 and Adirondack.  Soon to be a large number of
boards with GT64620 since it seems to be the high-end system
controller of choice for embedded 7xx/74xx these days and
everybody wants Linux.

> > Another way to go would be to #ifdef your suggestion into indirect_pci.c
> > for my board only.
> > Which do you prefer?
>
> Not the ifdef. :)  Either make your own version of the indirect_pci
> routines or add the field.

New field seems better than dupping 98% of indirect_pci, IMHO.

--
Matt Porter
MontaVista Software, Inc.
mporter@mvista.com

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Adding 'bus_offset' to pci_controller

[Mark A. Greer]

"Mark A. Greer" wrote:

> Paul Mackerras wrote:
>
> > For now something like what you have suggested should be OK except
> > that I would note that you could use the host->first_busno field,
> > which already exists, instead of adding a new field.
>
> Oh man, DUH...
>
> Thanks Paul, I'll make the change & push it in.

Wait, I can't b/c then *everyone* will have this subtraction happen and
everyone but me would be broken.
With a separate field, I don't mess up other  board ports.

Another way to go would be to #ifdef your suggestion into indirect_pci.c
for my board only.
Which do you prefer?

Mark


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Adding 'bus_offset' to pci_controller

[Paul Mackerras]

Mark A. Greer writes:

> Wait, I can't b/c then *everyone* will have this subtraction happen and
> everyone but me would be broken.
> With a separate field, I don't mess up other  board ports.

Well, how many other boards are there out there that use
indirect_pci.c and have more than one host bridge?  Anyone?

> Another way to go would be to #ifdef your suggestion into indirect_pci.c
> for my board only.
> Which do you prefer?

Not the ifdef. :)  Either make your own version of the indirect_pci
routines or add the field.

Paul.

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Adding 'bus_offset' to pci_controller

[mod+linuxppc-embedded]

I'm just catching up with this thread and I'm
not sure which problem it is that we're trying
to solve here.  Isn't it true that the various
pci_ops routines are passed a pci_dev from which
they can extract pci_dev->pci_bus->number and
decide which bus is meant from that?  Or does
that constitute a namespace (or number-space)
collision?  Aha - yes it might - each "node" in
a PCI bus "tree" has its own bus number, right?
So pci_dev->pci_bus->number is only intended to
be meaningful within a given tree and each tree
is (should be) its own domain...


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Adding 'bus_offset' to pci_controller

[Dan Taylor]

The bus number refers to the segment of PCI between bridges (host
PCI-PCI, or other) or attached to a single bridge; multiple host
bridges are referenced although I have never tried it.  So, from
your host bridge, for example the embedded MPC107 in an '8240, there
are four Ethernet devices, an IDE interface, and a PCI-PCI bridge,
and then there are two Fibre Channel devices attached to the PCI-
PCI bridge.  There will be two pci_bus structures, one for the PCI
bus from the host bridge to the PCI-PCI bridge, which should be bus
number 0, and another for the PCI bus on the "other" side of the
PCI-PCI bridge, which should be number 1.  All the Ethernet devices'
AND the PCI-PCI bridge's pci_dev structures will point to the
HOST BRIDGE pci_bus structure; the Fibre Channel devices will point
to a pci_bus structure associated with the PCI-PCI bridge.

Even if you have non-standard bridges, this works; I have used an
Intel 21554 in configurations where a card is sometimes the system
controller (assigner of PCI addresses for all of the other devices)
and sometimes not.

The routines for reading/writing configuration space, assigning
addresses, AND converting PCI addresses <-> CPU addresses have to
encapsulate the "magic" knowledge of the hardware configuration
to allow drivers to be portable across different implementations.

If you have hardware with offsets, then encode that in the bus_to_virt
conversion routine.  Set up tables in
arch/ppc/kernel/<your_implementation>.c,
if you have to; even with multiple host bridges, there should be NO
duplicated PCI addresses on different PCI busses, even though the
spec' allows it.  We HAVE the source; we CONTROL the address assignments;
even if you have offsets (as the 21554 does), don't replicate addresses.
Use only part of the available address space on each bus and use some
of the upper bits as tags so bus_to_virt, etc. can uniquely identify the
bus from the address and provide the correct response.


mod+linuxppc-embedded@MissionCriticalLinux.com wrote:
>
> I'm just catching up with this thread and I'm
> not sure which problem it is that we're trying
> to solve here.  Isn't it true that the various
> pci_ops routines are passed a pci_dev from which
> they can extract pci_dev->pci_bus->number and
> decide which bus is meant from that?  Or does
> that constitute a namespace (or number-space)
> collision?  Aha - yes it might - each "node" in
> a PCI bus "tree" has its own bus number, right?
> So pci_dev->pci_bus->number is only intended to
> be meaningful within a given tree and each tree
> is (should be) its own domain...
>

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Adding 'bus_offset' to pci_controller

[Mark A. Greer]

Dan Taylor wrote:

> The bus number refers to the segment of PCI between bridges (host
> PCI-PCI, or other) or attached to a single bridge; multiple host
> bridges are referenced although I have never tried it.  So, from
> your host bridge, for example the embedded MPC107 in an '8240, there
> are four Ethernet devices, an IDE interface, and a PCI-PCI bridge,
> and then there are two Fibre Channel devices attached to the PCI-
> PCI bridge.  There will be two pci_bus structures, one for the PCI
> bus from the host bridge to the PCI-PCI bridge, which should be bus
> number 0, and another for the PCI bus on the "other" side of the
> PCI-PCI bridge, which should be number 1.  All the Ethernet devices'
> AND the PCI-PCI bridge's pci_dev structures will point to the
> HOST BRIDGE pci_bus structure; the Fibre Channel devices will point
> to a pci_bus structure associated with the PCI-PCI bridge.

All of this is correct but it doesn't address the actual issue.

Background:
When generating a config addr on a PCI bus, you encode the bus number of the
target bus.  If the encoded bus number  matches the pci bus immediately below
the bridge, a Type 0 cycle is generated on that PCI bus; if not the same, a
Type 1 cycle is generated and should be claimed by the appropriate bridge on
that pci bus.  So, if you want to talk to a device on the PCI bus directly
below the host bridge, you must have the same bus number encoded in the config
addr that the bridge thinks that bus is (make sense?).

Up until now, host bridges with 2 pci buses (that's 2 "hoses" in linuxppc
nomenclature) hanging off them have allowed the s/w to set the bus number of
the second hose.  This lets us set up the second hose to "think" its pci bus
is the same bus number as we use up in linuxppc-land so linuxppc pci bus
number matches the physical bus numbering of the hoses.

With the gt64260, BOTH hoses are considered PCI bus 0 so when generating the
config addr that gets passed to the bridge for the second hose, you have to
subtract out the number of buses on the first hose.  That's the problem being
solved here.

Now that I've said all that, I think I've just found a way to set the bus
number of the second hose so, if it works, all of this becomes moot.

Mark


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/