Re: [PATCH 2/2] drivers/amba: probe via device tree

Re: [PATCH 2/2] drivers/amba: probe via device tree

[Grant Likely]

[cc'ing Rafael and Kevin because this ventures into clock stuff]

On Fri, May 20, 2011 at 10:08 AM, Stephen Neuendorffer
<stephen.neuendorffer@xilinx.com> wrote:
>
>
>> -----Original Message-----
>> From:
> devicetree-discuss-bounces+stephen.neuendorffer=xilinx.com@lists.ozlabs.
> org [mailto:devicetree-
>> discuss-bounces+stephen.neuendorffer=xilinx.com@lists.ozlabs.org] On
> Behalf Of Rob Herring
>> Sent: Friday, May 20, 2011 8:18 AM
>> To: Arnd Bergmann
>> Cc: devicetree-discuss@lists.ozlabs.org; Jeremy Kerr;
> linux-arm-kernel@lists.infradead.org
>> Subject: Re: [PATCH 2/2] drivers/amba: probe via device tree
>>
>> Arnd,
>>
>> On 05/20/2011 09:21 AM, Arnd Bergmann wrote:
>> > On Friday 20 May 2011 15:24:26 Rob Herring wrote:
>> >> Maybe we are looking this in the wrong way.
>> >>
>> >> AMBA is not really the bus, but certain types of devices on the
> bus.
>> >> Granted, it may actually be an AMBA bus vs. vendor bus (i.MX AIPS),
> but
>> >> that is really transparent to s/w. Separating AMBA devices in the
>> >> devicetree is really Linux requirements defining the devicetree
>> >> structure. It is certainly conceivable that an OS could make no
>> >> distinction. In my case, there is a mixture of regular platform
> devices
>> >> and AMBA(Primecell really) devices all interleaved on the same bus.
>> >
>> > I don't see how that would work. If the bus is AMBA, it should
>> > only have AMBA devices on it, otherwise how would they be connected?
>> >
>> The ARM definition of AMBA encompasses a lot of things. It is the
>> definition of the AXI, AHB and APB buses. It also has the definition
> of
>> the peripheral ID register definitions which primarily only ARM Ltd
>> peripherals implement. You can have those bus types yet not have any
>> peripherals with the ID registers. The Linux amba bus primarily deals
>> with just the peripheral ID for probe matching. There is also bus
> clock
>> handling, but that's not really unique to an AMBA bus. Arguably the
>> platform bus could have bus clock handling as well.
>
> I tried to bring up exactly this issue, but I don't think I got my point
> across
> effectively.  (probably because I started off with "why the hell does
> this exist???")
> (face palm)  The amba_bus driver really deals with a bunch of issues
> that
> are specific to a very small number of platforms and the style of cores
> from ARM.
>
>> > Whether software is supposed to know care about this is a different
>> > question. The device tree should generally reflect the block
>> > diagram of the hardware,
>>
>> Agreed.
>>
>> > and I would expect the AMBA devices be
>> > on a different level from the rest there.
>> >
>> But this part is not true.
>>
>> >> Based on this, I think of_platform_populate should always just
> match
>> >> against "simple-bus" and make the matches parameter define the
> device
>> >> creation hook rather than the bus type. Or you could have both
>> >> bus_matches and dev_matches parameters.
>> >
>> > I think it would be much better to only look at the parent bus for
>> > device to add, never at the device itself.
>> >
>> > If the bus is AMBA, add all devices as amba_device, if it's
> simple-bus,
>> > add all devices as platform_device.
>> >
>> That is how it is currently, but the reality is that I only have 1 bus
>> with both ARM Primecell peripherals and other peripherals which are
>> standard platform bus devices. i2c-designware is one example. It is on
>> the APB just like the pl011 uart. So do you propose I create a amba
>> driver for it? It has no peripheral ID registers, so that may not even
>> work.

We should clarify one point here.  There is no such thing as a
"standard platform device".  The platform_bus_type is a construct used
by Linux to model devices that cannot be probed any other way.
Typcially they are memory mapped onto a processor local bus without
any special behaviour, but they can also appear as sub devices of a
multi function device, or to describe something that isn't memory
mapped at all like gpio-leds.

In the case we're talking about the bus really is an AMBA bus, and all
the devices on it are in some sense real amba devices.  The problem is
that not all of the devices on the bus implement peripheral ID
registers or other mechanisms that good upstanding AMBA devices are
expected to have.  Plus, drivers already exist for some of these
devices in the form of platform_drivers.  We *could* enforce all
children of an AMBA bus to be driven by amba_drivers, and we could
implement it right now by adding an amba_driver registration along
side each platform_driver (the bulk of the code being shared of
course), but it is a matter of constructive laziness that we choose
not to.  We choose not to because it adds a big chunk of new code
without really buying us anything.  In fact, there are probably "good
upstanding" amba devices that do implement the peripheral ID
registers, but Linux drives them via platform_driver anyway.

OMAP (hi Kevin!) ran into a similar problem in figuring out how to
represent the internal busses on OMAP chips.  They've got a bunch of
additional "hwmod" data that describes how to handle power management
for system, but all of that infrastructure is largely transparent to
the driver.  As far as the driver is concerned, platform_device and
platform_driver is about the right abstraction.  The TI folks took a
bit of a different approach and instead of creating a different bus
type, they now attach additional data to the device at driver probe
time flagging the device as an omap device and setting it up for the
omap infrastructure to manage manipulating the clocks.  The advantage
being that clocks and power rails can be manipulated for plain-jane
platform_devices, but the expense is that the OMAP infrastructure
needs to jump through hoops to setup up the power management
callbacks.  (This work is still somewhat ongoing, and it remains to be
seen what the final result will ultimately look like).

In the DT context, the question then becomes what do device nodes in
the tree get registered as?  platform_device or amba_device?  Given
the above, it's not even clear that the presence of an
arm,amba-deviceid or an arm,amba-device compatible value is a clean
indication that a device should be registered as an amba_device.  The
options on the table are:

1) drop amba-bus entirely and use platform_device everywhere, similar
to what OMAP has done
2) strictly create amba_devices for nodes compatible with "arm,amba-device"
3) be intelligent about amba device creation; create an amba_device
only for devices we know are driven with amba_driver.

Option 1) requires migrating amba_drivers to platform_drivers, and it
also requires figuring out how to do the amba clock management on
platform_devices (not trivial, but probably a good thing overall
because we're rapidly approaching the point where we need clock
management on platform_devices anyway).
Option 2) is probably the wrong thing because it requires
arm,amba-device to *not* appear in nodes if Linux currently uses a
platform_device to drive the device.  This is bad because it leaks
Linux kernel implementation details into the device tree, which falls
down if at some future point a platform_driver is migrated to an
amba_driver, or visa-versa.  In that case all platforms still using
the old data would break on a new kernel; a situation we strive to
avoid.
Option 3) is possibly the best solution, or at least best near term
solution.  Looking at the kernel tree, there are only about 15
amba_drivers currently implemented.  It would be trivial to give
of_platform_populate a list of compatible values that should be
registered as amba_devices instead of platform_devices.  In fact, it
would be easy to extend the match table passed into
of_platform_populate() so that the caller can provide a handler
function for certain compatible values.  If the .data member is
populated, then it is a callback, which could be something like
of_amba_device_create() for the list of known amba devices.

ie:  This structure should be shared by all platforms, or overridden
*only if absolutely necessary*
struct of_device_id amba_platform_match[] __initdata = {
        { .compatible = "arm,amba-pl022", .data = of_amba_device_create },
        { .compatible = "arm,amba-pl030", .data = of_amba_device_create },
        /* ... */
        { .compatible = "arm,amba-pl031", .data = of_amba_device_create },
        { .compatible = "simple-bus", }, /* no callback;
platform_device with child devices */
        { }
};

Russell, it seems to me that the primary behaviour that amba_bus has
over platform_bus is the clock management, and secondarily
verification of the type of device by the device id.  Am I correct, or
am I missing something?

>
> Same here.  I don't know what the right solution for it is, but I find
> amba_bus
> solution to get in the way more than help.  I had to hack the etm/etb
> driver
> and the amba_bus to shreds to get it to work for me at all.
> I think most of what amba_bus does could be better handled by:
> 1) generic support in platform bus for clock enabling/power domains.
> These are system concerns that most drivers shouldn't really know/care
> about, and

Which, as mentioned above, is what OMAP has done.

> will likely vary between SOCs.
> 2) helper functions for drivers for primecell devices that does the
> peripheral id checking.

I think this lines up with my option 3 above.

g.

Re: [PATCH 2/2] drivers/amba: probe via device tree

[Russell King - ARM Linux]

On Sat, May 21, 2011 at 11:42:34AM -0600, Grant Likely wrote:
> Russell, it seems to me that the primary behaviour that amba_bus has
> over platform_bus is the clock management, and secondarily
> verification of the type of device by the device id.  Am I correct, or
> am I missing something?

It matches by vendor/device ID just like PCI does, and does the bus
clock management and power management in a really nice way, which I
doubt platform devices will ever do.

The way this discussion is going, I'm going to suggest that we also
convert PCI stuff to being platform devices too.  I don't see the
point of PCI existing for all the same reasons being given in this
thread.

Re: [PATCH 2/2] drivers/amba: probe via device tree

[Rafael J. Wysocki]

On Sunday, May 22, 2011, Russell King - ARM Linux wrote:
> On Sat, May 21, 2011 at 11:42:34AM -0600, Grant Likely wrote:
> > Russell, it seems to me that the primary behaviour that amba_bus has
> > over platform_bus is the clock management, and secondarily
> > verification of the type of device by the device id.  Am I correct, or
> > am I missing something?
> 
> It matches by vendor/device ID just like PCI does, and does the bus
> clock management and power management in a really nice way, which I
> doubt platform devices will ever do.

Then we should use the AMBA bus where possible.  Also, if there are
AMBA devices that currently pretend to be platform devices, they should be
converted back to AMBA.

> The way this discussion is going, I'm going to suggest that we also
> convert PCI stuff to being platform devices too.  I don't see the
> point of PCI existing for all the same reasons being given in this
> thread.

I agree.

Thanks,
Rafael

Re: [PATCH 2/2] drivers/amba: probe via device tree

[Rob Herring]

On 05/21/2011 06:47 PM, Russell King - ARM Linux wrote:
> On Sat, May 21, 2011 at 11:42:34AM -0600, Grant Likely wrote:
>> Russell, it seems to me that the primary behaviour that amba_bus has
>> over platform_bus is the clock management, and secondarily
>> verification of the type of device by the device id.  Am I correct, or
>> am I missing something?
>
> It matches by vendor/device ID just like PCI does, and does the bus
> clock management and power management in a really nice way, which I
> doubt platform devices will ever do.
>

Matching by ID is just one aspect of PCI. AMBA devices require defining 
the base address and irq just like platform devices. Having the ID is 
optional on AMBA buses. In PCI the bus and devices are probe-able. For 
AMBA, the bus is not probe-able, only the devices (or maybe not).

I believe OMAP is doing clock and power mgt at the bus level for 
platform devices.

Rob

Re: [PATCH 2/2] drivers/amba: probe via device tree

[Grant Likely]

On Sat, May 21, 2011 at 5:47 PM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Sat, May 21, 2011 at 11:42:34AM -0600, Grant Likely wrote:
>> Russell, it seems to me that the primary behaviour that amba_bus has
>> over platform_bus is the clock management, and secondarily
>> verification of the type of device by the device id.  Am I correct, or
>> am I missing something?
>
> It matches by vendor/device ID just like PCI does, and does the bus
> clock management and power management in a really nice way, which I
> doubt platform devices will ever do.
>
> The way this discussion is going, I'm going to suggest that we also
> convert PCI stuff to being platform devices too.  I don't see the
> point of PCI existing for all the same reasons being given in this
> thread.

I certainly don't see that as being the direction this discussion is going.

I see a serious question about how best to model AMBA primecell
devices in the device tree, and a similarly serious question about
whether to instantiate them as platform_devices or amba_devices.
Modelled behaviour in this case (clock/power management) is
particularly important, and you're right, platform_devices will never
implement that behaviour in the core code (this issue has already been
pushed back on; see discussions about omap_device).

g.

Re: [PATCH 2/2] drivers/amba: probe via device tree

[Arnd Bergmann]

On Saturday 21 May 2011 19:42:34 Grant Likely wrote:
> 1) drop amba-bus entirely and use platform_device everywhere, similar
> to what OMAP has done
> 2) strictly create amba_devices for nodes compatible with "arm,amba-device"
> 3) be intelligent about amba device creation; create an amba_device
> only for devices we know are driven with amba_driver.

Or maybe

4) Use amba_device for all devices on an amba bus (identified by
the compatible property of the bus), but mark the ones that do
not have primecell compatible registers so that the amba bus
does not try to look at them but instead takes the information
from the device tree.

Even though this might seem a bit silly when most devices are
not primecell ones, but it could be used as an incentive for
hardware designers (I can dream, right?) to provide them in future
designs. It would also make it easier to document the actual bus
hierarchy correctly.

	Arnd

Re: [PATCH 2/2] drivers/amba: probe via device tree

[Linus Walleij]

2011/5/22 Arnd Bergmann <arnd@arndb.de>:
> On Saturday 21 May 2011 19:42:34 Grant Likely wrote:
>> 1) drop amba-bus entirely and use platform_device everywhere, similar
>> to what OMAP has done
>> 2) strictly create amba_devices for nodes compatible with "arm,amba-device"
>> 3) be intelligent about amba device creation; create an amba_device
>> only for devices we know are driven with amba_driver.
>
> Or maybe
>
> 4) Use amba_device for all devices on an amba bus (identified by
> the compatible property of the bus), but mark the ones that do
> not have primecell compatible registers so that the amba bus
> does not try to look at them but instead takes the information
> from the device tree.

Currently the amba/primecell bus driver in drivers/amba/bus.c
will have the hardware registers override any data provided
from the board.

There are pending patches to instead have the board potentially
override the hardware, see:
http://www.arm.linux.org.uk/developer/patches/viewpatch.php?id=6829/1

The reason is that some of our hardware has newer revisions
of the PrimeCells and still the hardware registers have not
been updated properly. (Yes, the people involved have been
informed.)

So you also have a case where you may want to provide an ID
and have it override the ID present in the hardware.

Yours,
Linus Walleij

Re: [PATCH 2/2] drivers/amba: probe via device tree

[Kristoffer Glembo]

Hi,

Grant Likely wrote:
> In the case we're talking about the bus really is an AMBA bus, and all
> the devices on it are in some sense real amba devices.  The problem is
> that not all of the devices on the bus implement peripheral ID
> registers or other mechanisms that good upstanding AMBA devices are
> expected to have.  


Before we go hardware bashing of non primecell AMBA devices I would just
want to point out that the primecell stuff is not part of the AMBA specification.

So the "amba bus" should really have a way of marking which devices are primecell
devices instead of assuming this.

We (Aeroflex Gaisler) use an AMBA bus on our LEON SPARC chips but don't have any primecells 
devices. We went the of_platform_driver route after concluding that the AMBA bus in Linux is 
really not an AMBA bus at all. Our bus is fully probable though as all information about the 
present devices are memory mapped in a "plug and play" area.

On the other hand, there is not much point of an AMBA bus driver which does nothing more than
the platform driver. But AMBA is really a misnomer in this case.

/Kristoffer

Re: [PATCH 2/2] drivers/amba: probe via device tree

[Russell King - ARM Linux]

On Mon, May 23, 2011 at 11:37:04AM +0200, Kristoffer Glembo wrote:
> Grant Likely wrote:
> > In the case we're talking about the bus really is an AMBA bus, and all
> > the devices on it are in some sense real amba devices.  The problem is
> > that not all of the devices on the bus implement peripheral ID
> > registers or other mechanisms that good upstanding AMBA devices are
> > expected to have.  
> 
> Before we go hardware bashing of non primecell AMBA devices I would just
> want to point out that the primecell stuff is not part of the AMBA
> specification.

And before we go down that route, let me point out that the 'amba bus'
stuff in the kernel is there to support primecells, rather than all
devices which the AMBA specification covers.

The reason it's called 'amba' is because back in 2001 or so when the
first primecell drivers were created, there was little information
available as to what AMBA, AHB, or APB even covered.  All I had to go
on were the primecell documents themselves.  The higher level documents
were not available to me.

So, despite it being called 'amba', it really is just for primecells
and if we didn't have the exposure to userspace, I'd have renamed it to
'apb' or similar instead.

Re: [PATCH 2/2] drivers/amba: probe via device tree

[Grant Likely]

On Mon, May 23, 2011 at 3:58 AM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Mon, May 23, 2011 at 11:37:04AM +0200, Kristoffer Glembo wrote:
>> Grant Likely wrote:
>> > In the case we're talking about the bus really is an AMBA bus, and all
>> > the devices on it are in some sense real amba devices.  The problem is
>> > that not all of the devices on the bus implement peripheral ID
>> > registers or other mechanisms that good upstanding AMBA devices are
>> > expected to have.
>>
>> Before we go hardware bashing of non primecell AMBA devices I would just
>> want to point out that the primecell stuff is not part of the AMBA
>> specification.
>
> And before we go down that route, let me point out that the 'amba bus'
> stuff in the kernel is there to support primecells, rather than all
> devices which the AMBA specification covers.
>
> The reason it's called 'amba' is because back in 2001 or so when the
> first primecell drivers were created, there was little information
> available as to what AMBA, AHB, or APB even covered.  All I had to go
> on were the primecell documents themselves.  The higher level documents
> were not available to me.
>
> So, despite it being called 'amba', it really is just for primecells
> and if we didn't have the exposure to userspace, I'd have renamed it to
> 'apb' or similar instead.

Okay, that clarifies things a lot, and lends weight to the arguement
that it is perfectly normal and acceptable to have both amba_devices
and platform_devices on the same bus segment.  Are there any cases
where amba primecells are being driven by platform_drivers?  If so,
should those drivers have an amba_driver registration added?

g.

Re: [PATCH 2/2] drivers/amba: probe via device tree

[Rob Herring]

Grant,

On 05/23/2011 10:09 AM, Grant Likely wrote:
> On Mon, May 23, 2011 at 3:58 AM, Russell King - ARM Linux
> <linux@arm.linux.org.uk>  wrote:
>> On Mon, May 23, 2011 at 11:37:04AM +0200, Kristoffer Glembo wrote:
>>> Grant Likely wrote:
>>>> In the case we're talking about the bus really is an AMBA bus, and all
>>>> the devices on it are in some sense real amba devices.  The problem is
>>>> that not all of the devices on the bus implement peripheral ID
>>>> registers or other mechanisms that good upstanding AMBA devices are
>>>> expected to have.
>>>
>>> Before we go hardware bashing of non primecell AMBA devices I would just
>>> want to point out that the primecell stuff is not part of the AMBA
>>> specification.
>>
>> And before we go down that route, let me point out that the 'amba bus'
>> stuff in the kernel is there to support primecells, rather than all
>> devices which the AMBA specification covers.
>>
>> The reason it's called 'amba' is because back in 2001 or so when the
>> first primecell drivers were created, there was little information
>> available as to what AMBA, AHB, or APB even covered.  All I had to go
>> on were the primecell documents themselves.  The higher level documents
>> were not available to me.
>>
>> So, despite it being called 'amba', it really is just for primecells
>> and if we didn't have the exposure to userspace, I'd have renamed it to
>> 'apb' or similar instead.
>
> Okay, that clarifies things a lot, and lends weight to the arguement
> that it is perfectly normal and acceptable to have both amba_devices
> and platform_devices on the same bus segment.  Are there any cases
> where amba primecells are being driven by platform_drivers?  If so,
> should those drivers have an amba_driver registration added?

I would be surprised if there are any implemented as platform_drivers 
that are not duplicates of an amba driver. The STMP uart is actually a 
pl011 and it's platform driver was recently removed IIRC. So I think we 
can consider platform drivers something that should be fixed in this case.

Do you still think we should have a global match table of all devices or 
a generic "arm,primecell" compatible property would work. Several 
drivers like the pl022 have several h/w variations they support, so we 
would either need to list all those variations or have a generic name 
per device.

I think having "arm,amba-deviceid" is not needed. The current code does 
nothing but warn if it doesn't match the h/w value. The drivers already 
have a list of id's that they support and the amba bus only matches 
against the h/w id value. The only use I can see is overriding a broken 
h/w value. Certainly seems like it should be optional at least.

Rob

Re: [PATCH 2/2] drivers/amba: probe via device tree

[Shawn Guo]

On Tue, May 24, 2011 at 10:03:35AM -0500, Rob Herring wrote:
> Grant,
> 
> On 05/23/2011 10:09 AM, Grant Likely wrote:
> >On Mon, May 23, 2011 at 3:58 AM, Russell King - ARM Linux
> ><linux@arm.linux.org.uk>  wrote:
> >>On Mon, May 23, 2011 at 11:37:04AM +0200, Kristoffer Glembo wrote:
> >>>Grant Likely wrote:
> >>>>In the case we're talking about the bus really is an AMBA bus, and all
> >>>>the devices on it are in some sense real amba devices.  The problem is
> >>>>that not all of the devices on the bus implement peripheral ID
> >>>>registers or other mechanisms that good upstanding AMBA devices are
> >>>>expected to have.
> >>>
> >>>Before we go hardware bashing of non primecell AMBA devices I would just
> >>>want to point out that the primecell stuff is not part of the AMBA
> >>>specification.
> >>
> >>And before we go down that route, let me point out that the 'amba bus'
> >>stuff in the kernel is there to support primecells, rather than all
> >>devices which the AMBA specification covers.
> >>
> >>The reason it's called 'amba' is because back in 2001 or so when the
> >>first primecell drivers were created, there was little information
> >>available as to what AMBA, AHB, or APB even covered.  All I had to go
> >>on were the primecell documents themselves.  The higher level documents
> >>were not available to me.
> >>
> >>So, despite it being called 'amba', it really is just for primecells
> >>and if we didn't have the exposure to userspace, I'd have renamed it to
> >>'apb' or similar instead.
> >
> >Okay, that clarifies things a lot, and lends weight to the arguement
> >that it is perfectly normal and acceptable to have both amba_devices
> >and platform_devices on the same bus segment.  Are there any cases
> >where amba primecells are being driven by platform_drivers?  If so,
> >should those drivers have an amba_driver registration added?
> 
> I would be surprised if there are any implemented as
> platform_drivers that are not duplicates of an amba driver. The STMP
> uart is actually a pl011 and it's platform driver was recently

It (duart than auart) is a platform driver in Freesccale BSP, and was
turned into 'amba' one when being upstreamed.

> removed IIRC. So I think we can consider platform drivers something
> that should be fixed in this case.
> 

-- 
Regards,
Shawn

Re: [PATCH 2/2] drivers/amba: probe via device tree

[Linus Walleij]

2011/5/24 Rob Herring <robherring2@gmail.com>:

> I think having "arm,amba-deviceid" is not needed. The current code does
> nothing but warn if it doesn't match the h/w value. The drivers already have
> a list of id's that they support and the amba bus only matches against the
> h/w id value. The only use I can see is overriding a broken h/w value.

We have this usecase in the Ux500. See these patches:
http://www.arm.linux.org.uk/developer/patches/viewpatch.php?id=6829/1
http://www.arm.linux.org.uk/developer/patches/viewpatch.php?id=6830/1

Alas, it's not yet merged for the old boardfile world usecase, and
causing us problems to drive our hardware already.

Yours,
Linus Walleij