Re: Help needed to describe a custom bus in the device tree

[Laurent Pinchart <laurentp@cse-semaphore.com>]

[-- Attachment #1: Type: text/plain, Size: 8406 bytes --]

Hi Dave,

On Tuesday 11 March 2008 23:54, David Gibson wrote:
> On Tue, Mar 11, 2008 at 03:27:26PM +0100, Laurent Pinchart wrote:
> > Hi everybody,
> >
> > the migration process from ARCH=ppc to ARCH=powerpc is easier than I
> > thought in some parts, but a few devices are still giving me
> > headaches. This should hopefully be one of my last major requests
> > for help (I'm sure most of you will be happy to see traffic on this
> > list going down when I'll be done :-))
> >
> > I'm having trouble describing a custom bus named MS bus (completely
> > unrelated to a well-known software company) in the device tree. The
> > hardware is MPC8248-based and has the following hardware topology.
> >
> > MPC8248 <-- localbus --> FPGA <-- ms bus --> Custom peripherals
> >
> > The bus interrupt controller, serial access (SPI) controller and
> > status registers are accessed through memory-mapped registers in the
> > FPGA. Parallel access to the MS bus is handled transparently by the
> > FPGA which handles address mapping.
> >
> > The FPGA is mapped on the locabus at address 0xf4000000. Bus control
> > registers are at 0xf4002000 - 0xf4003000. The parallel bus memory
> > window on the localbus is located at 0xf5000000.
> >
> > My current dts draft describes that topology as follows (unrelated
> > devices on the local bus such as flash memory are removed for
> > clarity).
> >
> > localbus@f0010100 {
> > 	compatible = "fsl,pq2-localbus";
> > 	#address-cells = <2>;
> > 	#size-cells = <1>;
> > 	reg = <f0010100 40>;
> >
> > 	ranges = <0 0 40000000 01000000
> > 		  2 0 f2000000 00100000
> > 		  3 0 f3000000 00100000
> > 		  4 0 f4000000 00100000
> > 		  5 0 f5000000 00100000>;
> >
> > 	fpga@4,0 {
> > 		#address-cells = <1>;
> > 		#size-cells = <1>;
> > 		ranges = <4 0 0 00010000>;
> >
> > 		msbus-arbitrer@2000 {
> > 			compatible = "tbox,cp11-msbus-arbitrer";
> > 			reg = <2000 4>;
> > 		};
> >
> > 		msbus_pic: interrupt-controller@2100 {
> > 			compatible = "tbox,cp11-msbus-pic";
> > 			reg = <2100 8>;
> > 			interrupts = <17 2>;
> > 			interrupt-parent = <&cpm_pic>;
> > 			#interrupt-cells = <1>;
> > 			interrupt-controller;
> > 		};
> >
> > 		msbus-spi@2200 {
> > 			compatible = "tbox,cp11-msbus-spi";
> > 			reg = <2200 100>;
> > 			interrupts = <18 8>;
> > 			interrupt-parent = <&cpm_pic>;
> > 		};
> >
> > 		sdhc@5000 {
> > 			compatible = "tbox,sdhci";
> > 			reg = <5000 1000>;
> > 			interrupts = <16 8>;
> > 			interrupt-parent = <&cpm_pic>;
> > 		};
> > 	};
> >
> > 	msbus@5,0 {
> > 		compatible = "tbox,cp11-msbus";
> > 		#address-cells = <1>;
> > 		#size-cells = <1>;
> > 		#interrupt-cells = <1>;
> > 		reg = <5 0 0 00000400>;
> > 		interrupt-parent = <&msbus_pic>;
> > 	};
> > };
> >
> > The device tree reflects the physical topology but makes driver
> > access to the bus quite complex. An OF platform device driver
> > matching on compatible = "tbox,cp11-msbus" will not have the bus
> > FPGA registers described in its device node.
> >
> > Having a look at the various device trees included in the kernel
> > sources, it seems platforms with a PCI bus experience a similar
> > problem. To solve it the PCI bus node address and registers describe
> > the configuration registers, and the memory window to access PCI
> > devices is described by the ranges property.  Applying that to my
> > custom bus would lead to the following tree.
> >
> > localbus@f0010100 {
> > 	compatible = "fsl,pq2-localbus";
> > 	#address-cells = <2>;
> > 	#size-cells = <1>;
> > 	reg = <f0010100 40>;
> >
> > 	ranges = <0 0 40000000 01000000
> > 		  2 0 f2000000 00100000
> > 		  3 0 f3000000 00100000
> > 		  4 0 f4000000 00100000
> > 		  4 1 f4002000 00000100
> > 		  5 0 f5000000 00100000>;
> >
> > 	fpga@4,0 {
> > 		#address-cells = <1>;
> > 		#size-cells = <1>;
> > 		ranges = <4 0 0 00010000>;
> >
> > 		msbus_pic: interrupt-controller@2100 {
> > 			compatible = "tbox,cp11-msbus-pic";
> > 			reg = <2100 8>;
> > 			interrupts = <17 2>;
> > 			interrupt-parent = <&cpm_pic>;
> > 			#interrupt-cells = <1>;
> > 			interrupt-controller;
> > 		};
> >
> > 		msbus-spi@2200 {
> > 			compatible = "tbox,cp11-msbus-spi";
> > 			reg = <2200 100>;
> > 			interrupts = <18 8>;
> > 			interrupt-parent = <&cpm_pic>;
> > 		};
> >
> > 		sdhc@5000 {
> > 			compatible = "tbox,sdhci";
> > 			reg = <5000 1000>;
> > 			interrupts = <16 8>;
> > 			interrupt-parent = <&cpm_pic>;
> > 		};
> > 	};
> >
> > 	msbus@4,1 {
> > 		compatible = "tbox,cp11-msbus";
> > 		#address-cells = <1>;
> > 		#size-cells = <1>;
> > 		#interrupt-cells = <1>;
> > 		reg = <4 1 4>;
> > 		interrupt-parent = <&msbus_pic>;
> > 		ranges = <5 0 0 00000400>;
> > 	};
> > };
> >
> > Is this correct ? Is that the best way to describe my custom bus in
> > the device tree ?
>
> Your second example looks closer to right.  Certainly you should use
> 'reg' only for bus control registers, and 'ranges' for windows into
> the bus address space itself.

Ok.

> The device tree describes hardware from a functional point of view, so
> I don't know that it's relevant that all the bus control functions are
> implemented in an FPGA.  Each of the subnodes are more-or-less
> independent devices, so they could just have separate nodes.
>
> Or, if this seems more sensible, you could decide that they're
> sufficiently closely related to put them all as one node, with
> multiple register blocks listed in the 'reg' property.  That would
> probably get messy for your PIC at the very least though.

I suppose I could implement PIC support in the bus driver itself, but having 
separate nodes with separate OF devices and separate drivers seems cleaner to 
me (although it can make dependencies a bit more difficult to handle).

> > How would the relationships between the bus and
> > its PIC and SPI controller be handled in the drivers ?
>
> If the msbus driver needs to work with the associated PIC and SPI
> controllers, then you should put properties in the msbus node giving
> their phandles.

The PIC can live pretty much by itself, but the SPI controller is used to 
enumerate devices on the bus (and perform some other tasks at runtime). I 
plan to have the SPI device and driver be self-contained, and have the bus 
node reference the SPI device through its phandle. I will just have to make 
sure the bus driver is initialised after the SPI driver.

> > I also don't
> > understand how interrupt mappings are supposed to be handled. PCI
> > busses have two CPM interrupt lines, one for the PCI PIC and one for
> > the PCI bus, with the PCI bus having the CPM PIC as its interrupt
> > controller. My bus PIC uses a single interrupt line. Is there some
> > documentation explaining how PICs and interrupt mappings should be
> > described ?
>
> Are interrupts from devices on the msbus routed over the msbus, or are
> they routed independently to the mspic or the cpm PIC?

There is a single active low interrupt line on the msbus. When the mspic 
detects an interrupt condition, it will read the interrupt source registers 
from all devices on the bus and generate a host interrupt to the cpm PIC. The 
mspic driver then process the CPM interrupt in its demux handler, reads the 
interrupt sources from the mspic registers and dispatch the interrupts to the 
msbus device drivers.

If I understand things correctly, the mspic node should have an 'interrupts' 
attribute describing the cascaded interrupt line (mspic -> cpm PIC irq), and 
the msbus node should have an 'interrupts' attribute describing the interrupt 
line used to report bus-related events (hotplug events for instance). Is that 
right ?

To make things a bit more complex, the msbus interrupt and the bus-related 
events interrupt share the same CPM irq line. Can I use the same virq number 
in both nodes, or do I have to demux the interrupts in a separate driver ? If 
I have to demux the interrupts in a separate PIC driver, how do I know what 
virtual irq number will be assigned to each device so that I can reference 
them in the device tree ?

Thanks a lot for your help.

Best regards,

-- 
Laurent Pinchart
CSE Semaphore Belgium

Chaussée de Bruxelles, 732A
B-1410 Waterloo
Belgium

T +32 (2) 387 42 59
F +32 (2) 387 42 75

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]