DTs, Linux drivers & PCI Devices

DTs, Linux drivers & PCI Devices

[Saridakis, Dean (US SSA)]

I've been diving into Linux kernel drivers & device trees for a new project & am looking for some help. 

First, is it reasonable/possible to define a PCI (PCIe) device in a device tree? Seems like you'd have to reference a BAR#s to do this (& still have addresses mapped by enumeration).  A couple reasons I'm interested in doing this:
- The PCI device logic will be migrated to a system bus
- DT properties provide a nice way to pass in driver configuration data 

It's not clear to me how the linux kernel interprets/processes the device tree. I'm working w/ powerpc (fsl QorIQ). E.g., shouldn't everything from the DT show up in /sys/bus/platform/devices ? I noticed that not everything is there, e.g., only 2 of the 4 PCIe RCs. (Noticed this after trying to add my own simple dummy entry to a reserved reg space as an experiment).

Ultimately I'd like to use the DT to describe a set of logic cores on a subordinate bus. There is also a subordinate interrupt controller, which I think I can figure out how to describe, but am not sure how kernel glue works -- somehow I'd need to bind that controller to routing code I provide. 

Documentation I've been reading includes:
- ePAPR
- DT wiki
- kernel doc (driver-model, devicetree)
- Linux DDs v3 (no coverage of platform drivers)
Any other recommended docs?

Thanks,
Dean 

DTs, Linux drivers & PCI Devices

[Saridakis, Dean (US SSA)]

I've been diving into Linux kernel drivers & device trees for a new project & am looking for some help. 

First, is it reasonable/possible to define a PCI (PCIe) device in a device tree? Seems like you'd have to reference a BAR#s to do this (& still have addresses mapped by enumeration).  A couple reasons I'm interested in doing this:
- The PCI device logic will be migrated to a system bus
- DT properties provide a nice way to pass in driver configuration data 

It's not clear to me how the linux kernel interprets/processes the device tree. I'm working w/ powerpc (fsl QorIQ). E.g., shouldn't everything from the DT show up in /sys/bus/platform/devices ? I noticed that not everything is there, e.g., only 2 of the 4 PCIe RCs. (Noticed this after trying to add my own simple dummy entry to a reserved reg space as an experiment).

Ultimately I'd like to use the DT to describe a set of logic cores on a subordinate bus. There is also a subordinate interrupt controller, which I think I can figure out how to describe, but am not sure how kernel glue works -- somehow I'd need to bind that controller to routing code I provide. 

Documentation I've been reading includes:
- ePAPR
- DT wiki
- kernel doc (driver-model, devicetree)
- Linux DDs v3 (no coverage of platform drivers) Any other recommended docs?

Thanks,
Dean 

Re: DTs, Linux drivers & PCI Devices

[David Gibson]

[-- Attachment #1.1: Type: text/plain, Size: 2719 bytes --]

On Mon, Jan 28, 2013 at 02:39:54PM +0000, Saridakis, Dean   (US SSA) wrote:
> I've been diving into Linux kernel drivers & device trees for a new project & am looking for some help. 
> 
> First, is it reasonable/possible to define a PCI (PCIe) device in a
> device tree? Seems like you'd have to reference a BAR#s to do this
> (& still have addresses mapped by enumeration).  A couple reasons
> I'm interested in doing this:
> - The PCI device logic will be migrated to a system bus
> - DT properties provide a nice way to pass in driver configuration data 

It's certainly possible.  On traditional OF systems, PCI devices are
always shown in the device tree (and only OF, not the OS does the PCI
probing).  Only including the host bridges in flattened trees is
essentially a relaxation of the usual rules, because it's usually not
necessary and can be awkward for a flat tree system.

> It's not clear to me how the linux kernel interprets/processes the
> device tree. I'm working w/ powerpc (fsl QorIQ). E.g., shouldn't
> everything from the DT show up in /sys/bus/platform/devices ?

Not necessarily, no.  platform devices are sort of a last-resort place
for things that aren't on a well defined bus.  That usually includes
top level devices from an fdt, and often includes things further down
the fdt for platform specific buses.  But things like PCI devices, or
i2c devices or what have you will still appear under the appropriate
bus types.

> I
> noticed that not everything is there, e.g., only 2 of the 4 PCIe
> RCs. (Noticed this after trying to add my own simple dummy entry to
> a reserved reg space as an experiment).

Hrm.  Not showing all the RCs is a bit odd, though.

> Ultimately I'd like to use the DT to describe a set of logic cores
> on a subordinate bus. There is also a subordinate interrupt
> controller, which I think I can figure out how to describe, but am
> not sure how kernel glue works -- somehow I'd need to bind that
> controller to routing code I provide. 
> 
> Documentation I've been reading includes:
> - ePAPR
> - DT wiki
> - kernel doc (driver-model, devicetree)
> - Linux DDs v3 (no coverage of platform drivers) Any other recommended docs?

Although large parts of it won't be terribly relevant to you, you
should probably look at the original OF spec - IEEE 1275, and also the
PCI Bus binding document for IEEE 1275.

http://www.complang.tuwien.ac.at/forth/1275.ps.gz

http://www.openfirmware.org/ofwg/bindings/pci/pci2_1.pdf

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 192 bytes --]

_______________________________________________
devicetree-discuss mailing list
devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
https://lists.ozlabs.org/listinfo/devicetree-discuss

RE: DTs, Linux drivers & PCI Devices

[Saridakis, Dean (US SSA)]

> > I've been diving into Linux kernel drivers & device trees for a new project &
> am looking for some help.
> >
> > First, is it reasonable/possible to define a PCI (PCIe) device in a
> > device tree? Seems like you'd have to reference a BAR#s to do this (&
> > still have addresses mapped by enumeration).  A couple reasons I'm
> > interested in doing this:
> > - The PCI device logic will be migrated to a system bus
> > - DT properties provide a nice way to pass in driver configuration
> > data
> 
> It's certainly possible.  On traditional OF systems, PCI devices are always
> shown in the device tree (and only OF, not the OS does the PCI probing).
> Only including the host bridges in flattened trees is essentially a relaxation of
> the usual rules, because it's usually not necessary and can be awkward for a
> flat tree system.
> 

I'm learning as I go here, thanks for your patience:
- I'm using U-boot & just discovered it modifies the device tree
- From what I read in the spec's you referenced, seems like the assigned-addresses (assigned-address?) property would be the how the BAR window addresses would be passed to the OS.

So not sure if U-boot boardsetup can be convinced to create or fill in PCI device nodes in the DT. Looks like it's based on your libfdt (thank you!), but details are "left as an excercise for the reader." 

And if I could get U-boot to fill things in, it's not clear if/how linux kernel init would handle it. 
(Saw a reference to "PCI OF bindings" in booting-without-of.txt, but wasn't sure what that was referencing...)

If that were to all come together, I'd also like to define child nodes under a PCI device BAR node. (That'd be the most elegant approach.) Since "reg" is used to define the child register offsets within the BAR address space, but the BAR address is established w/ assigned-addresses, not sure how those dots would connect. 
I guess an example of how a traditional OF system would represent a PCI device would clear up some of my confusion - any pointers?

> > It's not clear to me how the linux kernel interprets/processes the
> > device tree. I'm working w/ powerpc (fsl QorIQ). E.g., shouldn't
> > everything from the DT show up in /sys/bus/platform/devices ?
> 
> Not necessarily, no.  platform devices are sort of a last-resort place for things
> that aren't on a well defined bus.  That usually includes top level devices from
> an fdt, and often includes things further down the fdt for platform specific
> buses.  But things like PCI devices, or i2c devices or what have you will still
> appear under the appropriate bus types.
> 
> > I
> > noticed that not everything is there, e.g., only 2 of the 4 PCIe RCs.
> > (Noticed this after trying to add my own simple dummy entry to a
> > reserved reg space as an experiment).
> 
> Hrm.  Not showing all the RCs is a bit odd, though.

Looks like U-boot boardsetup filtered these out.

> 
> > Ultimately I'd like to use the DT to describe a set of logic cores on
> > a subordinate bus. There is also a subordinate interrupt controller,
> > which I think I can figure out how to describe, but am not sure how
> > kernel glue works -- somehow I'd need to bind that controller to
> > routing code I provide.
> >
> > Documentation I've been reading includes:
> > - ePAPR
> > - DT wiki
> > - kernel doc (driver-model, devicetree)
> > - Linux DDs v3 (no coverage of platform drivers) Any other recommended
> docs?
> 
> Although large parts of it won't be terribly relevant to you, you should
> probably look at the original OF spec - IEEE 1275, and also the PCI Bus binding
> document for IEEE 1275.
> 
> http://www.complang.tuwien.ac.at/forth/1275.ps.gz
> 
> http://www.openfirmware.org/ofwg/bindings/pci/pci2_1.pdf
> 
> --
> David Gibson			| I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
> 				| _way_ _around_!
> http://www.ozlabs.org/~dgibson

Hope I didn't ramble too much -- getting tired...

Re: DTs, Linux drivers & PCI Devices

[David Gibson]

[-- Attachment #1.1: Type: text/plain, Size: 5049 bytes --]

On Wed, Jan 30, 2013 at 12:58:59AM +0000, Saridakis, Dean   (US SSA) wrote:
> > > I've been diving into Linux kernel drivers & device trees for a new project &
> > am looking for some help.
> > >
> > > First, is it reasonable/possible to define a PCI (PCIe) device in a
> > > device tree? Seems like you'd have to reference a BAR#s to do this (&
> > > still have addresses mapped by enumeration).  A couple reasons I'm
> > > interested in doing this:
> > > - The PCI device logic will be migrated to a system bus
> > > - DT properties provide a nice way to pass in driver configuration
> > > data
> > 
> > It's certainly possible.  On traditional OF systems, PCI devices are always
> > shown in the device tree (and only OF, not the OS does the PCI probing).
> > Only including the host bridges in flattened trees is essentially a relaxation of
> > the usual rules, because it's usually not necessary and can be awkward for a
> > flat tree system.
> > 
> 
> I'm learning as I go here, thanks for your patience:

> - I'm using U-boot & just discovered it modifies the device tree

> - From what I read in the spec's you referenced, seems like the
> assigned-addresses (assigned-address?) property would be the how the
> BAR window addresses would be passed to the OS.

That's right, IIRC.  That's assuming that the firmware does actually
assign BARs.  That would always be the case in traditional OF, but not
necessarily in the flat tree world.  In that case I think a firmware
that did not assign BARs could just leave assigned-addresses missing.

> So not sure if U-boot boardsetup can be convinced to create or fill
> in PCI device nodes in the DT. Looks like it's based on your libfdt
> (thank you!), but details are "left as an excercise for the reader."
> 
> And if I could get U-boot to fill things in, it's not clear if/how
> linux kernel init would handle it.

On ARM, I don't know.  On powerpc I know our PCI code can deal with
PCI device nodes being either present or absent, but I think the
platform code usually tells it which to expect.

> (Saw a reference to "PCI OF bindings" in booting-without-of.txt, but
> wasn't sure what that was referencing...)
> 
> If that were to all come together, I'd also like to define child
> nodes under a PCI device BAR node. (That'd be the most elegant
> approach.) Since "reg" is used to define the child register offsets
> within the BAR address space, but the BAR address is established w/
> assigned-addresses, not sure how those dots would connect. 

Child nodes under a BAR?  Um.. I really have no idea what you're
looking for here.


> I guess an example of how a traditional OF system would represent a
> PCI device would clear up some of my confusion - any pointers?

The device tree on any server class powerpc machine or a sparc machine
with a PCI bus should show you that.  Where to get access to such a
machine is less obvious, sorry.

> > > It's not clear to me how the linux kernel interprets/processes the
> > > device tree. I'm working w/ powerpc (fsl QorIQ). E.g., shouldn't
> > > everything from the DT show up in /sys/bus/platform/devices ?
> > 
> > Not necessarily, no.  platform devices are sort of a last-resort place for things
> > that aren't on a well defined bus.  That usually includes top level devices from
> > an fdt, and often includes things further down the fdt for platform specific
> > buses.  But things like PCI devices, or i2c devices or what have you will still
> > appear under the appropriate bus types.
> > 
> > > I
> > > noticed that not everything is there, e.g., only 2 of the 4 PCIe RCs.
> > > (Noticed this after trying to add my own simple dummy entry to a
> > > reserved reg space as an experiment).
> > 
> > Hrm.  Not showing all the RCs is a bit odd, though.
> 
> Looks like U-boot boardsetup filtered these out.
> 
> > 
> > > Ultimately I'd like to use the DT to describe a set of logic cores on
> > > a subordinate bus. There is also a subordinate interrupt controller,
> > > which I think I can figure out how to describe, but am not sure how
> > > kernel glue works -- somehow I'd need to bind that controller to
> > > routing code I provide.
> > >
> > > Documentation I've been reading includes:
> > > - ePAPR
> > > - DT wiki
> > > - kernel doc (driver-model, devicetree)
> > > - Linux DDs v3 (no coverage of platform drivers) Any other recommended
> > docs?
> > 
> > Although large parts of it won't be terribly relevant to you, you should
> > probably look at the original OF spec - IEEE 1275, and also the PCI Bus binding
> > document for IEEE 1275.
> > 
> > http://www.complang.tuwien.ac.at/forth/1275.ps.gz
> > 
> > http://www.openfirmware.org/ofwg/bindings/pci/pci2_1.pdf
> > 
> 
> Hope I didn't ramble too much -- getting tired...
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 192 bytes --]

_______________________________________________
devicetree-discuss mailing list
devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
https://lists.ozlabs.org/listinfo/devicetree-discuss