Musings on PCI busses

Musings on PCI busses

[Grant Likely]

Hey all,

I've been having some thoughts on PCI host controller probing and how
best to handle it.  Currently AFAICT, the SoC platform code explicitly
calls the PCI setup code.  In all of the existing SoCs this works fine
because there is SoC specific platform code which knows what PCI
busses to expect on the SoC.

I'm now looking at the case of Xilinx Virtex FPGA support.  Because it
is an FPGA and the SW view of the design is so flexible and fluid, I'm
trying to keep platform code unified for all Virtex platforms.  (ie.
two radically different board layouts can look identical from the
CPU's perspective, or the behaviour of one board can be completely
changed with a new bitstream).

Roderick has written a patch to support one of the Xilinx reference
designs that includes a PCI host controller.  Right now the patch adds
a separate platform file, but I'm not comfortable with the approach
(despite suggesting it to Roderick in the first place) because it
relegates PCI support to only work with the ml510 reference design.
ie. PCI will not be automatically supported for anyone who modifies
the reference design or adds the PCI host controller to their own
design.  So I've been thinking of alternatives.  Here's what I've come
up with.

1) Probe the host controller in an of_platform driver.  This has the
advantage of simplicity.  The probe routine will get automatically
called when the PCI host controller device tree node is registered
with the of_platform bus.  The bus parenthood also gets reflected in
the device model and sysfs.  The disadvantage is that it defers PCI
bus probing until after the of_platform bus is probed (maybe this is
okay; maybe this already happens anyway).

2) Probe the host controller in an subsys_initcall().  Advantage is
PCI can be probed earlier in the init path.  Disadvantages (minor) are
that it will always get called if the driver is enabled, and it needs
to manually search the device tree for PCI nodes.

I'm leaning towards making it an of_platform driver.  Doing so also
makes it available to other powerpc processors (not just virtex) in
the case where a Xilinx FPGA is welded up to a discrete SoC and a host
controller instance is put into the FPGA.  (one of those weird things
people do when they have an FPGA in their system).

Comments?  Opinions?

Cheers,
g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Musings on PCI busses

[Arnd Bergmann]

On Tuesday 19 May 2009, Grant Likely wrote:
> 1) Probe the host controller in an of_platform driver. =A0This has the
> advantage of simplicity. =A0The probe routine will get automatically
> called when the PCI host controller device tree node is registered
> with the of_platform bus. =A0The bus parenthood also gets reflected in
> the device model and sysfs. =A0The disadvantage is that it defers PCI
> bus probing until after the of_platform bus is probed (maybe this is
> okay; maybe this already happens anyway).

This is also what we do on 64-bit systems (well, Cell at this time).
There are no fundamental problems with this, just some slightly
unexpected interactions e.g. when you have use a PCI serial port
as your console before probing PCI.

Making CONFIG_PPC_OF_PLATFORM_PCI probably requires some more
consolidation between pci_32.c and pci_64.c, but I'd consider
this a good thing.

	Arnd <><

RE: Musings on PCI busses

[Stephen Neuendorffer]

> 1) Probe the host controller in an of_platform driver.  This has the
> advantage of simplicity.  The probe routine will get automatically
> called when the PCI host controller device tree node is registered
> with the of_platform bus.  The bus parenthood also gets reflected in
> the device model and sysfs.  The disadvantage is that it defers PCI
> bus probing until after the of_platform bus is probed (maybe this is
> okay; maybe this already happens anyway).
> =

> 2) Probe the host controller in an subsys_initcall().  Advantage is
> PCI can be probed earlier in the init path.  Disadvantages (minor) are
> that it will always get called if the driver is enabled, and it needs
> to manually search the device tree for PCI nodes.
> =

> I'm leaning towards making it an of_platform driver.  Doing so also
> makes it available to other powerpc processors (not just virtex) in
> the case where a Xilinx FPGA is welded up to a discrete SoC and a host
> controller instance is put into the FPGA.  (one of those weird things
> people do when they have an FPGA in their system).

I agree that something is called for...  The first might be slightly
simpler, since it would probably transparently deal with the presence
of more than one PLB->PCI bridge?

Steve


This email and any attachments are intended for the sole use of the named r=
ecipient(s) and contain(s) confidential information that may be proprietary=
, privileged or copyrighted under applicable law. If you are not the intend=
ed recipient, do not read, copy, or forward this email message or any attac=
hments. Delete this email message and any attachments immediately.

Re: Musings on PCI busses

[Grant Likely]

On Tue, May 19, 2009 at 10:25 AM, Stephen Neuendorffer
<stephen.neuendorffer@xilinx.com> wrote:
>
>> 1) Probe the host controller in an of_platform driver. =A0This has the
>> advantage of simplicity. =A0The probe routine will get automatically
>> called when the PCI host controller device tree node is registered
>> with the of_platform bus. =A0The bus parenthood also gets reflected in
>> the device model and sysfs. =A0The disadvantage is that it defers PCI
>> bus probing until after the of_platform bus is probed (maybe this is
>> okay; maybe this already happens anyway).
>>
>> 2) Probe the host controller in an subsys_initcall(). =A0Advantage is
>> PCI can be probed earlier in the init path. =A0Disadvantages (minor) are
>> that it will always get called if the driver is enabled, and it needs
>> to manually search the device tree for PCI nodes.
>>
>> I'm leaning towards making it an of_platform driver. =A0Doing so also
>> makes it available to other powerpc processors (not just virtex) in
>> the case where a Xilinx FPGA is welded up to a discrete SoC and a host
>> controller instance is put into the FPGA. =A0(one of those weird things
>> people do when they have an FPGA in their system).
>
> I agree that something is called for... =A0The first might be slightly
> simpler, since it would probably transparently deal with the presence
> of more than one PLB->PCI bridge?

Yes, I think so.

g.


--=20
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Musings on PCI busses

[David Miller]

From: Arnd Bergmann <arnd@arndb.de>
Date: Tue, 19 May 2009 18:12:02 +0200

> On Tuesday 19 May 2009, Grant Likely wrote:
>> 1) Probe the host controller in an of_platform driver. =A0This has t=
he
>> advantage of simplicity. =A0The probe routine will get automatically=

>> called when the PCI host controller device tree node is registered
>> with the of_platform bus. =A0The bus parenthood also gets reflected =
in
>> the device model and sysfs. =A0The disadvantage is that it defers PC=
I
>> bus probing until after the of_platform bus is probed (maybe this is=

>> okay; maybe this already happens anyway).
> =

> This is also what we do on 64-bit systems (well, Cell at this time).
> There are no fundamental problems with this, just some slightly
> unexpected interactions e.g. when you have use a PCI serial port
> as your console before probing PCI.

This is also what sparc64 does :-)

All of the individual sparc64 PCI controller types have a OF driver
and then there is a common layer of OF PCI helper code to do most
of the work.

Re: Musings on PCI busses

[Grant Likely]

On Tue, May 19, 2009 at 1:05 PM, David Miller <davem@davemloft.net> wrote:
> From: Arnd Bergmann <arnd@arndb.de>
> Date: Tue, 19 May 2009 18:12:02 +0200
>
>> On Tuesday 19 May 2009, Grant Likely wrote:
>>> 1) Probe the host controller in an of_platform driver. =A0This has the
>>> advantage of simplicity. =A0The probe routine will get automatically
>>> called when the PCI host controller device tree node is registered
>>> with the of_platform bus. =A0The bus parenthood also gets reflected in
>>> the device model and sysfs. =A0The disadvantage is that it defers PCI
>>> bus probing until after the of_platform bus is probed (maybe this is
>>> okay; maybe this already happens anyway).
>>
>> This is also what we do on 64-bit systems (well, Cell at this time).
>> There are no fundamental problems with this, just some slightly
>> unexpected interactions e.g. when you have use a PCI serial port
>> as your console before probing PCI.
>
> This is also what sparc64 does :-)
>
> All of the individual sparc64 PCI controller types have a OF driver
> and then there is a common layer of OF PCI helper code to do most
> of the work.

Cool, well I have my solution then.  :-)

g.


--=20
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Musings on PCI busses

[Benjamin Herrenschmidt]

On Tue, 2009-05-19 at 09:28 -0600, Grant Likely wrote:

> 1) Probe the host controller in an of_platform driver.  This has the
> advantage of simplicity.  The probe routine will get automatically
> called when the PCI host controller device tree node is registered
> with the of_platform bus.  The bus parenthood also gets reflected in
> the device model and sysfs.  The disadvantage is that it defers PCI
> bus probing until after the of_platform bus is probed (maybe this is
> okay; maybe this already happens anyway).

We are doing that on Cell as mentioned by Arnd. However, there are a
few issues with that approach. One is that it relies on PCI-hotplug
related bits that currently only exist in 64-bit land. Another is
that doing so tends to expose various hidden issues or subtle
dependencies to the PCI stuff being setup early and probed early.

For example, some of the OF parsing bits may fail to convert memory
addresses to IO addresses if the PCI host bridges have not been
discovered yet, potentially causing issues with matching of resources
between the early serial stuff and the generic serial driver (if you
use an IO driven UART, the PCI 8250 driver may not properly figure out
that what it's finding in its IO BARs is actually the same port as
what it gets from the platform code as the later will end up with
memory addresses rather than IO ones). That's one example.

At this stage, I much prefer if we stick to doing the PHB discovery
at setup_arch() time, especially since there's other WIP in the area
of PCI coming from Kumar and I don't want to mixup multiple source of
problems in that area at the same time.

We can revisit that later but we'll have to be very careful.

Cheers,
Ben.

RE: Musings on PCI busses

[Benjamin Herrenschmidt]

On Tue, 2009-05-19 at 09:25 -0700, Stephen Neuendorffer wrote:

> I agree that something is called for...  The first might be slightly
> simpler, since it would probably transparently deal with the presence
> of more than one PLB->PCI bridge?

The current code doesn't already ?

Cheers,
Ben.

Re: Musings on PCI busses

[Grant Likely]

On Tue, May 19, 2009 at 9:02 PM, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> On Tue, 2009-05-19 at 09:25 -0700, Stephen Neuendorffer wrote:
>
>> I agree that something is called for... =A0The first might be slightly
>> simpler, since it would probably transparently deal with the presence
>> of more than one PLB->PCI bridge?
>
> The current code doesn't already ?

Current code doesn't exist in mainline.  :-)

And, no, the patch Roderick wrote doesn't handle more than one PHB.
No reason it couldn't though.

g.

--=20
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Musings on PCI busses

[Benjamin Herrenschmidt]

On Tue, 2009-05-19 at 21:17 -0600, Grant Likely wrote:
> On Tue, May 19, 2009 at 9:02 PM, Benjamin Herrenschmidt
> <benh@kernel.crashing.org> wrote:
> > On Tue, 2009-05-19 at 09:25 -0700, Stephen Neuendorffer wrote:
> >
> >> I agree that something is called for...  The first might be slightly
> >> simpler, since it would probably transparently deal with the presence
> >> of more than one PLB->PCI bridge?
> >
> > The current code doesn't already ?
> 
> Current code doesn't exist in mainline.  :-)

Well, I was thinking about ppc4xx_pci :-)

> And, no, the patch Roderick wrote doesn't handle more than one PHB.
> No reason it couldn't though.

Right, it shouldn't be hard.

Cheers,
Ben.

Re: Musings on PCI busses

[Benjamin Herrenschmidt]

On Tue, 2009-05-19 at 12:05 -0700, David Miller wrote:
> 
> This is also what sparc64 does :-)
> 
> All of the individual sparc64 PCI controller types have a OF driver
> and then there is a common layer of OF PCI helper code to do most
> of the work.

I agree it's a good idea in the long run, but on ppc32, I would be a bit
careful due to the amount of historical crap & early board fixup we have
that may need working PCI config space accesses and make assumption
about when PCI is probed.

We could add the necessary bits for ppc32 to be able to "opt-in" the new
scheme on a per board basis I suppose.

Cheers,
Ben.

Re: Musings on PCI busses

[David Miller]

From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Date: Wed, 20 May 2009 13:01:30 +1000

> For example, some of the OF parsing bits may fail to convert memory
> addresses to IO addresses if the PCI host bridges have not been
> discovered yet, potentially causing issues with matching of resources
> between the early serial stuff and the generic serial driver (if you
> use an IO driven UART, the PCI 8250 driver may not properly figure out
> that what it's finding in its IO BARs is actually the same port as
> what it gets from the platform code as the later will end up with
> memory addresses rather than IO ones). That's one example.

FWIW, I never run into this issue on sparc64 exactly because I
fully resolve all resources when I populate the OF device tree
in the kernel.

Re: Musings on PCI busses

[Benjamin Herrenschmidt]

On Tue, 2009-05-19 at 22:51 -0700, David Miller wrote:
> From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Date: Wed, 20 May 2009 13:01:30 +1000
> 
> > For example, some of the OF parsing bits may fail to convert memory
> > addresses to IO addresses if the PCI host bridges have not been
> > discovered yet, potentially causing issues with matching of resources
> > between the early serial stuff and the generic serial driver (if you
> > use an IO driven UART, the PCI 8250 driver may not properly figure out
> > that what it's finding in its IO BARs is actually the same port as
> > what it gets from the platform code as the later will end up with
> > memory addresses rather than IO ones). That's one example.
> 
> FWIW, I never run into this issue on sparc64 exactly because I
> fully resolve all resources when I populate the OF device tree
> in the kernel.

What do you mean by fully resolve ? IE. The issue above is specific to
IO space, which you can resolve both as MMIO, or as "IO" which in linux
means going through the special mapping for IO which allows the use of
the inX/outX instructions...

The later can only be done after we have established the special mapping
in question for a given PCI bus IO space, at which point, we know what
IO "base" corresponds to the IO space of that bridge. So in the
meantime, all we have is an MMIO resource, which is "resolved" in the
sense that it does contain all the right bits to get the serial ports
(address translation etc...) but isn't an IORESOURCE_IO type resource
and thus doesn't contain a magic token we can pass to inX/outX.

One of the issues we may have here on powerpc is that our very early
probe of serial ports (so we get some debug output early) may get to
those resources while the serial driver later sees the actual
IORESOURCE_IO resources coming from the PCI probe, and is unable to
figure that they are the same things.

Currently, we do a pass of "conversion" over those early resolved
resources just after the PCI bridges have been discovered to check if
they happen to land into the IO space of a bridge and then turn them
into IORESOURCE_IO tokens.

If we delay the probing of PCI bridges, things can get fishy there.

Cheers,
Ben.

Re: Musings on PCI busses

[David Miller]

From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Date: Wed, 20 May 2009 16:51:07 +1000

> On Tue, 2009-05-19 at 22:51 -0700, David Miller wrote:
>> From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
>> Date: Wed, 20 May 2009 13:01:30 +1000
>> 
>> > For example, some of the OF parsing bits may fail to convert memory
>> > addresses to IO addresses if the PCI host bridges have not been
>> > discovered yet, potentially causing issues with matching of resources
>> > between the early serial stuff and the generic serial driver (if you
>> > use an IO driven UART, the PCI 8250 driver may not properly figure out
>> > that what it's finding in its IO BARs is actually the same port as
>> > what it gets from the platform code as the later will end up with
>> > memory addresses rather than IO ones). That's one example.
>> 
>> FWIW, I never run into this issue on sparc64 exactly because I
>> fully resolve all resources when I populate the OF device tree
>> in the kernel.
> 
> What do you mean by fully resolve ? IE. The issue above is specific to
> IO space, which you can resolve both as MMIO, or as "IO" which in linux
> means going through the special mapping for IO which allows the use of
> the inX/outX instructions...

I mean that all OF devices have fully resolved MMIO resources.  So
very early serial devices that sit in I/O space on sparc64 end
up being OF device drivers.  See for example, drivers/net/sunsu.c
which is simply a 8250 chip that sits behind Sun's I/O space bus
that sits on PCI and provides things normally found via ISA on
x86 machines.  Another example is drivers/serial/sunsab.c

> One of the issues we may have here on powerpc is that our very early
> probe of serial ports (so we get some debug output early) may get to
> those resources while the serial driver later sees the actual
> IORESOURCE_IO resources coming from the PCI probe, and is unable to
> figure that they are the same things.

I would make these device drivers OF device drivers.

Another option is to make use of the "address" property if present.
You can bypass all of the hassle of figuring out what the 'reg'
property resolves to by using that and if the device is the
console it is pretty reliable to expect OF to provide that "address"
property.

Of course this doesn't work well for non-console devices.

Re: Musings on PCI busses

[Grant Likely]

On Wed, May 20, 2009 at 1:24 PM, David Miller <davem@davemloft.net> wrote:
>> What do you mean by fully resolve ? IE. The issue above is specific to
>> IO space, which you can resolve both as MMIO, or as "IO" which in linux
>> means going through the special mapping for IO which allows the use of
>> the inX/outX instructions...
>
> I mean that all OF devices have fully resolved MMIO resources. =A0So
> very early serial devices that sit in I/O space on sparc64 end
> up being OF device drivers. =A0See for example, drivers/net/sunsu.c
> which is simply a 8250 chip that sits behind Sun's I/O space bus
> that sits on PCI and provides things normally found via ISA on
> x86 machines. =A0Another example is drivers/serial/sunsab.c

Unfortunately in the embedded powerpc case we don't actually have real
OF.  We've only got the flattened device tree which usually doesn't
itemize the devices behind the PCI bus.  Instead we rely on the kernel
probing for them.

g.


--=20
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Musings on PCI busses

[Benjamin Herrenschmidt]

> I mean that all OF devices have fully resolved MMIO resources.  So
> very early serial devices that sit in I/O space on sparc64 end
> up being OF device drivers.  See for example, drivers/net/sunsu.c
> which is simply a 8250 chip that sits behind Sun's I/O space bus
> that sits on PCI and provides things normally found via ISA on
> x86 machines.  Another example is drivers/serial/sunsab.c

Right. On Cell, we use the "normal" 8250 driver, via of_serial which
basically initializes it from the of device and associated resources.

It gets a bit messy because we -also- create a platform driver that can
be matched by 8250, in fact, we have a bit of generic powerpc code that
does that early at boot, initializes our low level very early udbg
thingy with it and creates the platform devices.

And then the serial port can -also- be detected by 8250_pci.c when it's
on a PCI based UART :-)

Now, generally thinks work fine even if you use all 3 drivers because
they have the same resources and 8250 has some smarts to figure that out
and not re-create ports. It somewhat fails in the case where we end up
with IO looking like MMIO on one side and looking like PCI IO on another
tho.

We -could- try to standardize on a single of those 3 methods of course.
The of_serial one has some appeal but I quite like the platform driver
approach at this stage because we have code to handle all sort of weirdo
special cases in there (more or less insane device-tree's, weird SoC's,
etc...) and still pump out something 8250_platform can grok, and that
code can run -very- early and provide a udbg backend for early debug and
xmon usage.

> I would make these device drivers OF device drivers.
> 
> Another option is to make use of the "address" property if present.
> You can bypass all of the hassle of figuring out what the 'reg'
> property resolves to by using that and if the device is the
> console it is pretty reliable to expect OF to provide that "address"
> property.
> 
> Of course this doesn't work well for non-console devices.

Also we have various broken firmwares with non usable "address"
properties to deal with.

Ben.

Re: Musings on PCI busses

[Benjamin Herrenschmidt]

On Wed, 2009-05-20 at 14:06 -0600, Grant Likely wrote:
> On Wed, May 20, 2009 at 1:24 PM, David Miller <davem@davemloft.net> wrote:
> >> What do you mean by fully resolve ? IE. The issue above is specific to
> >> IO space, which you can resolve both as MMIO, or as "IO" which in linux
> >> means going through the special mapping for IO which allows the use of
> >> the inX/outX instructions...
> >
> > I mean that all OF devices have fully resolved MMIO resources.  So
> > very early serial devices that sit in I/O space on sparc64 end
> > up being OF device drivers.  See for example, drivers/net/sunsu.c
> > which is simply a 8250 chip that sits behind Sun's I/O space bus
> > that sits on PCI and provides things normally found via ISA on
> > x86 machines.  Another example is drivers/serial/sunsab.c
> 
> Unfortunately in the embedded powerpc case we don't actually have real
> OF.  We've only got the flattened device tree which usually doesn't
> itemize the devices behind the PCI bus.  Instead we rely on the kernel
> probing for them.

We -can- itemize the PCI bus, it's up to you.

That leads to another interesting discussion...

On ppc64, we have code that can "create" the pci_bus & pci_dev hierarchy
from the OF tree (avoiding the config space probing entirely). This is
very efficient and has some advantages such as avoiding touching the
BARs for sizing (which mean avoiding to temporarily disable access to
devices behind them, which can be useful when it's your serial port or
system controller there) etc...

It also have the advantage of avoiding a double PCI probe pass when the
FW already did it and since most FW do ...

I'm tempted to extract that code and stick it in a
drivers/pci/probe_of.c or something like that for more generic usage. We
also have a per-bus hook that allows to switch to "real" config space
probing at any point of the hierarchy (for example, you can setup things
so that only on-board devices are in OF and probed that away and
anything on your slot gets probed by linux using config space, that sort
of thing).

Cheers,
Ben.
 

Re: Musings on PCI busses

[Grant Likely]

On Wed, May 20, 2009 at 3:59 PM, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> On Wed, 2009-05-20 at 14:06 -0600, Grant Likely wrote:
>> On Wed, May 20, 2009 at 1:24 PM, David Miller <davem@davemloft.net> wrot=
e:
>> >> What do you mean by fully resolve ? IE. The issue above is specific t=
o
>> >> IO space, which you can resolve both as MMIO, or as "IO" which in lin=
ux
>> >> means going through the special mapping for IO which allows the use o=
f
>> >> the inX/outX instructions...
>> >
>> > I mean that all OF devices have fully resolved MMIO resources. =A0So
>> > very early serial devices that sit in I/O space on sparc64 end
>> > up being OF device drivers. =A0See for example, drivers/net/sunsu.c
>> > which is simply a 8250 chip that sits behind Sun's I/O space bus
>> > that sits on PCI and provides things normally found via ISA on
>> > x86 machines. =A0Another example is drivers/serial/sunsab.c
>>
>> Unfortunately in the embedded powerpc case we don't actually have real
>> OF. =A0We've only got the flattened device tree which usually doesn't
>> itemize the devices behind the PCI bus. =A0Instead we rely on the kernel
>> probing for them.
>
> We -can- itemize the PCI bus, it's up to you.

Yes, we could.  But I don't think it makes any sense to for the FDT
case.  The firmware infrastructure is not in place to fill out the
device tree with the PCI bus layout for the off board devices.

>
> That leads to another interesting discussion...
>
> On ppc64, we have code that can "create" the pci_bus & pci_dev hierarchy
> from the OF tree (avoiding the config space probing entirely). This is
> very efficient and has some advantages such as avoiding touching the
> BARs for sizing (which mean avoiding to temporarily disable access to
> devices behind them, which can be useful when it's your serial port or
> system controller there) etc...
>
> It also have the advantage of avoiding a double PCI probe pass when the
> FW already did it and since most FW do ...
>
> I'm tempted to extract that code and stick it in a
> drivers/pci/probe_of.c or something like that for more generic usage. We
> also have a per-bus hook that allows to switch to "real" config space
> probing at any point of the hierarchy (for example, you can setup things
> so that only on-board devices are in OF and probed that away and
> anything on your slot gets probed by linux using config space, that sort
> of thing).

That would be interesting.

g.

--=20
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Musings on PCI busses

[David Miller]

From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Date: Thu, 21 May 2009 07:59:42 +1000

> On ppc64, we have code that can "create" the pci_bus & pci_dev hierarchy
> from the OF tree (avoiding the config space probing entirely). This is
> very efficient and has some advantages such as avoiding touching the
> BARs for sizing (which mean avoiding to temporarily disable access to
> devices behind them, which can be useful when it's your serial port or
> system controller there) etc...
> 
> It also have the advantage of avoiding a double PCI probe pass when the
> FW already did it and since most FW do ...
> 
> I'm tempted to extract that code and stick it in a
> drivers/pci/probe_of.c or something like that for more generic usage. We
> also have a per-bus hook that allows to switch to "real" config space
> probing at any point of the hierarchy (for example, you can setup things
> so that only on-board devices are in OF and probed that away and
> anything on your slot gets probed by linux using config space, that sort
> of thing).

This is exactly what sparc64 does as well, I took the powerpc code. :)

It also avoids a bunch of bugs that get unearthed as a result of
scanning the entire hierarchy with PCI config access "pokes".  Some
PCI controllers hang when certain PCI config space addresses are
accessed, meanwhile some hypervisor versions don't generate the bus
timeout exception properly on PCI config accesses to nonexisting
devices, the list is endless.

And all of that went away when I imported the ppc64 code to do
this using the OF device tree.

So if you put this in a common place, let's consolidate the differences
accumulated in the sparc64 code so I can use it too :-)

Re: Musings on PCI busses

[Benjamin Herrenschmidt]

On Wed, 2009-05-20 at 15:41 -0700, David Miller wrote:
> This is exactly what sparc64 does as well, I took the powerpc code. :)
> 
> It also avoids a bunch of bugs that get unearthed as a result of
> scanning the entire hierarchy with PCI config access "pokes".  Some
> PCI controllers hang when certain PCI config space addresses are
> accessed, meanwhile some hypervisor versions don't generate the bus
> timeout exception properly on PCI config accesses to nonexisting
> devices, the list is endless.
> 
> And all of that went away when I imported the ppc64 code to do
> this using the OF device tree.
> 
> So if you put this in a common place, let's consolidate the differences
> accumulated in the sparc64 code so I can use it too :-)

Allright, I'll give that a go asap, maybe tomorrow.

Cheers,
Ben.