[ARM ATTEND] arch/arm SoC organization

[ARM ATTEND] arch/arm SoC organization

[Jason Cooper]

All,

I'd like to take the opportunity to discuss the layout/config of the arm
SoCs.  By the time the KS rolls around, we should have at least two or more
use-cases to present and discuss (mvebu, bcm).

Between devicetree and multiplatform the concept of board-specific code
is really disappearing in arm-land.  eg mach-kirkwood/ is almost fully
converted, within the next window or two, we'll be moving all of the DT
capable boards from kirkwood, orion5x, and dove over to mvebu.
mach-mvebu will then hold 5 different SoCs (it already has Armada 370
and Armada XP).

mach-bcm is in a similar situation (multi-SoC).  I'm sure there are
others (nvidia, ti, etc).

So, I'd like to propose we discuss some lessons learned and maybe arrive
at some best practices.  eg, should we just go with mach-$COMPANY/?  How
best to handle config symbols for efficient building?  Deprecation path
for legacy (unconverted) boards?

Also, to head off the demons, I am *not* proposing reorganizing
everything.  Merely to find some consensus and good ideas so that as
things merge together, we are mostly on the same page.

I'm sure we all have different ideas and I think it'd be good to hear
what everyone else is thinking.

thx,

Jason.

[Ksummit-2013-discuss] [ARM ATTEND] arch/arm SoC organization

[Tony Lindgren]

* Jason Cooper <jason@lakedaemon.net> [130731 07:25]:
> All,
> 
> I'd like to take the opportunity to discuss the layout/config of the arm
> SoCs.  By the time the KS rolls around, we should have at least two or more
> use-cases to present and discuss (mvebu, bcm).
> 
> Between devicetree and multiplatform the concept of board-specific code
> is really disappearing in arm-land.  eg mach-kirkwood/ is almost fully
> converted, within the next window or two, we'll be moving all of the DT
> capable boards from kirkwood, orion5x, and dove over to mvebu.
> mach-mvebu will then hold 5 different SoCs (it already has Armada 370
> and Armada XP).
> 
> mach-bcm is in a similar situation (multi-SoC).  I'm sure there are
> others (nvidia, ti, etc).

Well we've pretty much always kept the omaps we can build together in
the same directory, otherwise we would be up to an insane number of
mach-omap related directories by now.. I think getting rid of the
mach- and plat- directories and making everything into drivers is
the next phase after device tree conversion is done.

> So, I'd like to propose we discuss some lessons learned and maybe arrive
> at some best practices.  eg, should we just go with mach-$COMPANY/?  How
> best to handle config symbols for efficient building?  Deprecation path
> for legacy (unconverted) boards?

A lot of that problem goes away by initializing everything as late
as possible, and making things to live under drivers. Of course it
may make sense to combine things meanwhile, but renaming things causes
quite a bit of churn.
 
> Also, to head off the demons, I am *not* proposing reorganizing
> everything.  Merely to find some consensus and good ideas so that as
> things merge together, we are mostly on the same page.
> 
> I'm sure we all have different ideas and I think it'd be good to hear
> what everyone else is thinking.

Yes sharing experiences on this stuff would be good de-briefing and
could help others :)

Regards,

Tony

[Ksummit-2013-discuss] [ARM ATTEND] arch/arm SoC organization

[Christian Daudt]

On Fri, Aug 2, 2013 at 1:33 AM, Tony Lindgren <tony@atomide.com> wrote:
> * Jason Cooper <jason@lakedaemon.net> [130731 07:25]:
>> So, I'd like to propose we discuss some lessons learned and maybe arrive
>> at some best practices.  eg, should we just go with mach-$COMPANY/?  How
>> best to handle config symbols for efficient building?  Deprecation path
>> for legacy (unconverted) boards?
>
> A lot of that problem goes away by initializing everything as late
> as possible, and making things to live under drivers.
One category of items that we haven't found a good place for in this
new multiplatform world is where does dt-driven non-driver code reside
? e.g. we have a secure monitor access function that only kicks in if
the appropriate dt entry is available . It currently resides in
mach-bcm/bcm_kona_smc.c as it seems like the only location for it at
the moment, but that doesn't seem like the best place because (a)
mach-bcm might end up littered with one-of cases like this and (b)
anything in mach-bcm is not visible to arm64 SoCs, and some of those
in the future will need to share with their arm32 cousins.
 But putting in drivers (e.g. drivers/smc) seems like the wrong thing
to do also because this is not a driver.
 We have a couple of other smallish pieces of IP that just need a bit
of generic init code to keep them happy, which we were discussing
internally where to best land them. At present they are also headed to
mach-bcm.
 Ultimately the question is 'what is allowed to reside in mach-<misc>
?' And by extension: 'is there a good home for everything else ?''

 Thanks,
   csd

[Ksummit-2013-discuss] [ARM ATTEND] arch/arm SoC organization

[Olof Johansson]

On Fri, Aug 2, 2013 at 4:06 PM, Christian Daudt <csd_b@daudt.org> wrote:
> On Fri, Aug 2, 2013 at 1:33 AM, Tony Lindgren <tony@atomide.com> wrote:
>> * Jason Cooper <jason@lakedaemon.net> [130731 07:25]:
>>> So, I'd like to propose we discuss some lessons learned and maybe arrive
>>> at some best practices.  eg, should we just go with mach-$COMPANY/?  How
>>> best to handle config symbols for efficient building?  Deprecation path
>>> for legacy (unconverted) boards?
>>
>> A lot of that problem goes away by initializing everything as late
>> as possible, and making things to live under drivers.
> One category of items that we haven't found a good place for in this
> new multiplatform world is where does dt-driven non-driver code reside
> ? e.g. we have a secure monitor access function that only kicks in if
> the appropriate dt entry is available . It currently resides in
> mach-bcm/bcm_kona_smc.c as it seems like the only location for it at
> the moment, but that doesn't seem like the best place because (a)
> mach-bcm might end up littered with one-of cases like this and (b)
> anything in mach-bcm is not visible to arm64 SoCs, and some of those
> in the future will need to share with their arm32 cousins.

ARM is working on standardizing some of these interfaces through
things like PSCI. It should take care of _some_ of the SMC needs. I
know you're already shipping some of these platforms though and
firmware interfaces might be locked in to something that's not
PSCI-compatible. :(

>  But putting in drivers (e.g. drivers/smc) seems like the wrong thing
> to do also because this is not a driver.

I really don't think we need to migrate everything to drivers/*. At
some point the need of exorcising code out of arch/arm will reach a
plateu, and there's some code that just simply belongs in that sub
directory.

The sharing with arm64 is the obvious sticky point, since the base
directories aren't the same. Luckily not too many vendors have shipped
arm64 platforms, and nobody is shipping in volume yet, so there's a
little time to sort out some of this.

>  We have a couple of other smallish pieces of IP that just need a bit
> of generic init code to keep them happy, which we were discussing
> internally where to best land them. At present they are also headed to
> mach-bcm.

I think the 3.12 merge window will see the introduction of
drivers/platform/arm. It must not become a dumping ground for
board-file equivalent code though, so I am a little scared to
introduce it. Mostly it should contain whatever glue code isn't
possible or practical to describe with device tree (or ACPI) today,
and in particular for established consumer devices, i.e. for actual
product needs. Compare to what's in drivers/platform/x86 to get a feel
for what we're talking about.

>  Ultimately the question is 'what is allowed to reside in mach-<misc>
> ?' And by extension: 'is there a good home for everything else ?''

Right, and see above about some of my opinions on the matter. arm64
does complicate things quite a bit since it's no longer easy to share
code between the architectures.

Having a few concise examples of what code you're looking for a home
for could be useful for the discussion though. Would you mind
volunteering a few? :)


-Olof

[Ksummit-2013-discuss] [ARM ATTEND] arch/arm SoC organization

[Christian Daudt]

On Fri, Aug 2, 2013 at 10:24 PM, Olof Johansson <olof@lixom.net> wrote:
> On Fri, Aug 2, 2013 at 4:06 PM, Christian Daudt <csd_b@daudt.org> wrote:
>> On Fri, Aug 2, 2013 at 1:33 AM, Tony Lindgren <tony@atomide.com> wrote:
>>> * Jason Cooper <jason@lakedaemon.net> [130731 07:25]:
>>>> So, I'd like to propose we discuss some lessons learned and maybe arrive
>>>> at some best practices.  eg, should we just go with mach-$COMPANY/?  How
>>>> best to handle config symbols for efficient building?  Deprecation path
>>>> for legacy (unconverted) boards?
>>>
>>> A lot of that problem goes away by initializing everything as late
>>> as possible, and making things to live under drivers.
>> One category of items that we haven't found a good place for in this
>> new multiplatform world is where does dt-driven non-driver code reside
>> ? e.g. we have a secure monitor access function that only kicks in if
>> the appropriate dt entry is available . It currently resides in
>> mach-bcm/bcm_kona_smc.c as it seems like the only location for it at
>> the moment, but that doesn't seem like the best place because (a)
>> mach-bcm might end up littered with one-of cases like this and (b)
>> anything in mach-bcm is not visible to arm64 SoCs, and some of those
>> in the future will need to share with their arm32 cousins.
>
> ARM is working on standardizing some of these interfaces through
> things like PSCI. It should take care of _some_ of the SMC needs. I
> know you're already shipping some of these platforms though and
> firmware interfaces might be locked in to something that's not
> PSCI-compatible. :(
>
Agreed, but that is in the future. Most ARMv7 platforms will likely
never see PSCI and they still need to talk to their secure half. Even
when PSCI code gets into mainline, where will it sit ?

>>  But putting in drivers (e.g. drivers/smc) seems like the wrong thing
>> to do also because this is not a driver.
>
> I really don't think we need to migrate everything to drivers/*. At
> some point the need of exorcising code out of arch/arm will reach a
> plateu, and there's some code that just simply belongs in that sub
> directory.
>
Ok, that's why I'm trying to determine. How low do we want to go with
the thinning of mach-* dirs.

> The sharing with arm64 is the obvious sticky point, since the base
> directories aren't the same. Luckily not too many vendors have shipped
> arm64 platforms, and nobody is shipping in volume yet, so there's a
> little time to sort out some of this.
>
>>  We have a couple of other smallish pieces of IP that just need a bit
>> of generic init code to keep them happy, which we were discussing
>> internally where to best land them. At present they are also headed to
>> mach-bcm.
>
> I think the 3.12 merge window will see the introduction of
> drivers/platform/arm. It must not become a dumping ground for
> board-file equivalent code though, so I am a little scared to
> introduce it. Mostly it should contain whatever glue code isn't
> possible or practical to describe with device tree (or ACPI) today,
> and in particular for established consumer devices, i.e. for actual
> product needs. Compare to what's in drivers/platform/x86 to get a feel
> for what we're talking about.
>
So putting things in drivers/platform solves the a32/a64 sharing
problem at least. So it's a positive step.

>>  Ultimately the question is 'what is allowed to reside in mach-<misc>
>> ?' And by extension: 'is there a good home for everything else ?''
>
> Right, and see above about some of my opinions on the matter. arm64
> does complicate things quite a bit since it's no longer easy to share
> code between the architectures.
>
> Having a few concise examples of what code you're looking for a home
> for could be useful for the discussion though. Would you mind
> volunteering a few? :)
Other than the already upstreamed SMC code, this again came up in the
context of a dma mux block that we have. The code needs to do static
configuration of the block @ bringup and then there's nothing else for
it to do. We're trying to figure out where the best place for it is.
Options being:
 a) write a couple of functions in arch/arm/mach-bcm/<filename>.c
which based on dt config do the necessary setup
 b) write those same functions with a bit of driver wrapper to make it
pretend to be a driver (i.e. add a probe/remove) and plunk it into
drivers/dma/<filename>.c
 c) ?


 Thanks,
   csd

[Ksummit-2013-discuss] [ARM ATTEND] arch/arm SoC organization

[Tony Lindgren]

* Christian Daudt <csd_b@daudt.org> [130802 16:13]:
> On Fri, Aug 2, 2013 at 1:33 AM, Tony Lindgren <tony@atomide.com> wrote:
> > * Jason Cooper <jason@lakedaemon.net> [130731 07:25]:
> >> So, I'd like to propose we discuss some lessons learned and maybe arrive
> >> at some best practices.  eg, should we just go with mach-$COMPANY/?  How
> >> best to handle config symbols for efficient building?  Deprecation path
> >> for legacy (unconverted) boards?
> >
> > A lot of that problem goes away by initializing everything as late
> > as possible, and making things to live under drivers.
> One category of items that we haven't found a good place for in this
> new multiplatform world is where does dt-driven non-driver code reside
> ? e.g. we have a secure monitor access function that only kicks in if
> the appropriate dt entry is available . It currently resides in
> mach-bcm/bcm_kona_smc.c as it seems like the only location for it at
> the moment, but that doesn't seem like the best place because (a)
> mach-bcm might end up littered with one-of cases like this and (b)
> anything in mach-bcm is not visible to arm64 SoCs, and some of those
> in the future will need to share with their arm32 cousins.
>  But putting in drivers (e.g. drivers/smc) seems like the wrong thing
> to do also because this is not a driver.
>  We have a couple of other smallish pieces of IP that just need a bit
> of generic init code to keep them happy, which we were discussing
> internally where to best land them. At present they are also headed to
> mach-bcm.
>  Ultimately the question is 'what is allowed to reside in mach-<misc>
> ?' And by extension: 'is there a good home for everything else ?''

Well I guess the question is how early do you need it?

We only need the following things early on that might be SoC specific:

1. Timers for clockevents

2. Interrupt controller

If you need the SoC specific code for the two items above, then
you probably want to set it up in the SoC specific init_early.

Everything else should be possible to do as device drivers with
initramfs. If the code has an interrupt handler, chances are it
can be a driver :)

Regards,

Tony

[Ksummit-2013-discuss] [ARM ATTEND] arch/arm SoC organization

[Christian Daudt]

On Sun, Aug 4, 2013 at 11:21 PM, Tony Lindgren <tony@atomide.com> wrote:
> * Christian Daudt <csd_b@daudt.org> [130802 16:13]:
>> On Fri, Aug 2, 2013 at 1:33 AM, Tony Lindgren <tony@atomide.com> wrote:
>> > * Jason Cooper <jason@lakedaemon.net> [130731 07:25]:
>> >> So, I'd like to propose we discuss some lessons learned and maybe arrive
>> >> at some best practices.  eg, should we just go with mach-$COMPANY/?  How
>> >> best to handle config symbols for efficient building?  Deprecation path
>> >> for legacy (unconverted) boards?
>> >
>> > A lot of that problem goes away by initializing everything as late
>> > as possible, and making things to live under drivers.
>> One category of items that we haven't found a good place for in this
>> new multiplatform world is where does dt-driven non-driver code reside
>> ? e.g. we have a secure monitor access function that only kicks in if
>> the appropriate dt entry is available . It currently resides in
>> mach-bcm/bcm_kona_smc.c as it seems like the only location for it at
>> the moment, but that doesn't seem like the best place because (a)
>> mach-bcm might end up littered with one-of cases like this and (b)
>> anything in mach-bcm is not visible to arm64 SoCs, and some of those
>> in the future will need to share with their arm32 cousins.
>>  But putting in drivers (e.g. drivers/smc) seems like the wrong thing
>> to do also because this is not a driver.
>>  We have a couple of other smallish pieces of IP that just need a bit
>> of generic init code to keep them happy, which we were discussing
>> internally where to best land them. At present they are also headed to
>> mach-bcm.
>>  Ultimately the question is 'what is allowed to reside in mach-<misc>
>> ?' And by extension: 'is there a good home for everything else ?''
>
> Well I guess the question is how early do you need it?
>
> We only need the following things early on that might be SoC specific:
>
> 1. Timers for clockevents
>
> 2. Interrupt controller
>
> If you need the SoC specific code for the two items above, then
> you probably want to set it up in the SoC specific init_early.
>
> Everything else should be possible to do as device drivers with
> initramfs. If the code has an interrupt handler, chances are it
> can be a driver :)
>
I agree that most anything can be made into a driver. I'm asking if
that's what we want to do for all arm code that doesn't need to be
done at the earliest stages. If there's code that doesn't need
interrupts, doesn't register for fileops, doesn't register for any
framework, etc... Do we still want to make a driver out of it ?

 thanks,
   csd

[Ksummit-2013-discuss] [ARM ATTEND] arch/arm SoC organization

[Tony Lindgren]

* Christian Daudt <csd_b@daudt.org> [130806 16:14]:
> On Sun, Aug 4, 2013 at 11:21 PM, Tony Lindgren <tony@atomide.com> wrote:
> > * Christian Daudt <csd_b@daudt.org> [130802 16:13]:
> >> On Fri, Aug 2, 2013 at 1:33 AM, Tony Lindgren <tony@atomide.com> wrote:
> >> > * Jason Cooper <jason@lakedaemon.net> [130731 07:25]:
> >> >> So, I'd like to propose we discuss some lessons learned and maybe arrive
> >> >> at some best practices.  eg, should we just go with mach-$COMPANY/?  How
> >> >> best to handle config symbols for efficient building?  Deprecation path
> >> >> for legacy (unconverted) boards?
> >> >
> >> > A lot of that problem goes away by initializing everything as late
> >> > as possible, and making things to live under drivers.
> >> One category of items that we haven't found a good place for in this
> >> new multiplatform world is where does dt-driven non-driver code reside
> >> ? e.g. we have a secure monitor access function that only kicks in if
> >> the appropriate dt entry is available . It currently resides in
> >> mach-bcm/bcm_kona_smc.c as it seems like the only location for it at
> >> the moment, but that doesn't seem like the best place because (a)
> >> mach-bcm might end up littered with one-of cases like this and (b)
> >> anything in mach-bcm is not visible to arm64 SoCs, and some of those
> >> in the future will need to share with their arm32 cousins.
> >>  But putting in drivers (e.g. drivers/smc) seems like the wrong thing
> >> to do also because this is not a driver.
> >>  We have a couple of other smallish pieces of IP that just need a bit
> >> of generic init code to keep them happy, which we were discussing
> >> internally where to best land them. At present they are also headed to
> >> mach-bcm.
> >>  Ultimately the question is 'what is allowed to reside in mach-<misc>
> >> ?' And by extension: 'is there a good home for everything else ?''
> >
> > Well I guess the question is how early do you need it?
> >
> > We only need the following things early on that might be SoC specific:
> >
> > 1. Timers for clockevents
> >
> > 2. Interrupt controller
> >
> > If you need the SoC specific code for the two items above, then
> > you probably want to set it up in the SoC specific init_early.
> >
> > Everything else should be possible to do as device drivers with
> > initramfs. If the code has an interrupt handler, chances are it
> > can be a driver :)
> >
> I agree that most anything can be made into a driver. I'm asking if
> that's what we want to do for all arm code that doesn't need to be
> done at the earliest stages. If there's code that doesn't need
> interrupts, doesn't register for fileops, doesn't register for any
> framework, etc... Do we still want to make a driver out of it ?

If we can, then yes I think that's the way to go. Making things into
loadable modules already automatically prevents quite a bit of
potential for nasty hacks and spaghetti code as the interfaces are
standard.

Also, then the related pieces can easily be shared across multiple
SoCs or archs, and we may even have proper maintainers for those
drivers eventually :)

Regards,

Tony