Status of arch/arm in linux-next

Status of arch/arm in linux-next

[Russell King - ARM Linux]

This morning, I looked at linux-next to find out how arch/arm is doing
for the next merge window.

$ git diff -C --cumulative v2.6.39-rc1... arch
  19.7% arch/arm/mach-imx/
  19.2% arch/arm/mach-mx3/
   3.4% arch/arm/mach-mxc91231/
  18.1% arch/arm/mach-ux500/
  74.1% arch/arm/
   3.2% arch/m68k/
   4.0% arch/mips/lantiq/
   6.9% arch/mips/
   3.1% arch/x86/kvm/
   7.6% arch/x86/
 100.0% arch/
$ git diff -C --cumulative v2.6.39-rc1... arch/arm
   4.7% arch/arm/mach-imx/
   5.9% arch/arm/mach-msm/
   6.3% arch/arm/mach-mx3/
   8.8% arch/arm/mach-mxc91231/
   7.6% arch/arm/mach-ux500/include/mach/
  46.1% arch/arm/mach-ux500/
   3.6% arch/arm/mach-zynq/
   5.9% arch/arm/plat-mxc/include/mach/
   7.3% arch/arm/plat-mxc/
 100.0% arch/arm/
$ git diff -C --shortstat v2.6.39-rc1... arch/arm
 450 files changed, 11973 insertions(+), 5736 deletions(-)

Note that with --cumulative, the %age given includes all child directories
(so the 74% for arch/arm includes everything below it).

So, almost 75% of changes in arch are down to arch/arm.  For arch/arm,
there's 6k new lines introduced - so there's a significant amount of new
work there.

Folk may want to consider that we're three weeks from Linus' rant, and
there's little sign of the situation changing.  It looks to me from
these statistics that it's business as usual.

Please take a moment to consider how Linus will react to this at the
next merge window.

Status of arch/arm in linux-next

[Tony Lindgren]

* Russell King - ARM Linux <linux@arm.linux.org.uk> [110414 12:42]:
> This morning, I looked at linux-next to find out how arch/arm is doing
> for the next merge window.
> 
> $ git diff -C --cumulative v2.6.39-rc1... arch
>   19.7% arch/arm/mach-imx/
>   19.2% arch/arm/mach-mx3/
>    3.4% arch/arm/mach-mxc91231/
>   18.1% arch/arm/mach-ux500/
>   74.1% arch/arm/
>    3.2% arch/m68k/
>    4.0% arch/mips/lantiq/
>    6.9% arch/mips/
>    3.1% arch/x86/kvm/
>    7.6% arch/x86/
>  100.0% arch/
> $ git diff -C --cumulative v2.6.39-rc1... arch/arm
>    4.7% arch/arm/mach-imx/
>    5.9% arch/arm/mach-msm/
>    6.3% arch/arm/mach-mx3/
>    8.8% arch/arm/mach-mxc91231/
>    7.6% arch/arm/mach-ux500/include/mach/
>   46.1% arch/arm/mach-ux500/
>    3.6% arch/arm/mach-zynq/
>    5.9% arch/arm/plat-mxc/include/mach/
>    7.3% arch/arm/plat-mxc/
>  100.0% arch/arm/
> $ git diff -C --shortstat v2.6.39-rc1... arch/arm
>  450 files changed, 11973 insertions(+), 5736 deletions(-)
> 
> Note that with --cumulative, the %age given includes all child directories
> (so the 74% for arch/arm includes everything below it).
> 
> So, almost 75% of changes in arch are down to arch/arm.  For arch/arm,
> there's 6k new lines introduced - so there's a significant amount of new
> work there.
> 
> Folk may want to consider that we're three weeks from Linus' rant, and
> there's little sign of the situation changing.  It looks to me from
> these statistics that it's business as usual.
> 
> Please take a moment to consider how Linus will react to this at the
> next merge window.

It's going to take some time before the consolidated code starts
being available.. And until we have something available, we all
should aim for negative diffstat.

Tony

Status of arch/arm in linux-next

[Russell King - ARM Linux]

On Thu, Apr 14, 2011 at 02:08:54PM +0300, Tony Lindgren wrote:
> * Russell King - ARM Linux <linux@arm.linux.org.uk> [110414 12:42]:
> > This morning, I looked at linux-next to find out how arch/arm is doing
> > for the next merge window.
> > 
> > $ git diff -C --cumulative v2.6.39-rc1... arch
> >   19.7% arch/arm/mach-imx/
> >   19.2% arch/arm/mach-mx3/
> >    3.4% arch/arm/mach-mxc91231/
> >   18.1% arch/arm/mach-ux500/
> >   74.1% arch/arm/
> >    3.2% arch/m68k/
> >    4.0% arch/mips/lantiq/
> >    6.9% arch/mips/
> >    3.1% arch/x86/kvm/
> >    7.6% arch/x86/
> >  100.0% arch/
> > $ git diff -C --cumulative v2.6.39-rc1... arch/arm
> >    4.7% arch/arm/mach-imx/
> >    5.9% arch/arm/mach-msm/
> >    6.3% arch/arm/mach-mx3/
> >    8.8% arch/arm/mach-mxc91231/
> >    7.6% arch/arm/mach-ux500/include/mach/
> >   46.1% arch/arm/mach-ux500/
> >    3.6% arch/arm/mach-zynq/
> >    5.9% arch/arm/plat-mxc/include/mach/
> >    7.3% arch/arm/plat-mxc/
> >  100.0% arch/arm/
> > $ git diff -C --shortstat v2.6.39-rc1... arch/arm
> >  450 files changed, 11973 insertions(+), 5736 deletions(-)
> > 
> > Note that with --cumulative, the %age given includes all child directories
> > (so the 74% for arch/arm includes everything below it).
> > 
> > So, almost 75% of changes in arch are down to arch/arm.  For arch/arm,
> > there's 6k new lines introduced - so there's a significant amount of new
> > work there.
> > 
> > Folk may want to consider that we're three weeks from Linus' rant, and
> > there's little sign of the situation changing.  It looks to me from
> > these statistics that it's business as usual.
> > 
> > Please take a moment to consider how Linus will react to this at the
> > next merge window.
> 
> It's going to take some time before the consolidated code starts
> being available.. And until we have something available, we all
> should aim for negative diffstat.

I think we've already lost any hope of a negative diffstat - with 6k new
lines, we will need a heck of a lot of consolidation to counter that.

The only thing which I've seen any work being put into is some of the
GPIO code - which will be lost in the noise in much the same way that
my ARM platform consolidation and deletion of two machine classes in
the last merge window was.  That resulted in the deletion of around 10k
lines from the kernel as a whole, or 4.5k lines from arch/arm.

Some people are believing that Linus' complaint is about the core ARM
code rather than the platform specific code which is the problem.

We need significant amounts of effort to reverse this - and the more new
stuff that appears in peoples trees (and therefore linux-next) the bigger
problem we have to convince Linus that we're taking him seriously.

So, I think we've already lost any hope of making Linus happy for the
next merge window.  I wish I'm wrong on that, but I don't think so.

Status of arch/arm in linux-next

[Tony Lindgren]

* Russell King - ARM Linux <linux@arm.linux.org.uk> [110414 14:59]:
> On Thu, Apr 14, 2011 at 02:08:54PM +0300, Tony Lindgren wrote:
> > 
> > It's going to take some time before the consolidated code starts
> > being available.. And until we have something available, we all
> > should aim for negative diffstat.
> 
> I think we've already lost any hope of a negative diffstat - with 6k new
> lines, we will need a heck of a lot of consolidation to counter that.
> 
> The only thing which I've seen any work being put into is some of the
> GPIO code - which will be lost in the noise in much the same way that
> my ARM platform consolidation and deletion of two machine classes in
> the last merge window was.  That resulted in the deletion of around 10k
> lines from the kernel as a whole, or 4.5k lines from arch/arm.
> 
> Some people are believing that Linus' complaint is about the core ARM
> code rather than the platform specific code which is the problem.

I think that 6k lines of new code should wait in the platform specific
trees and not get merged until we have some real progress on the
consolidation of code.
 
> We need significant amounts of effort to reverse this - and the more new
> stuff that appears in peoples trees (and therefore linux-next) the bigger
> problem we have to convince Linus that we're taking him seriously.
> 
> So, I think we've already lost any hope of making Linus happy for the
> next merge window.  I wish I'm wrong on that, but I don't think so.

We should make it clear that we're not merging new code except for code
consolidation. Looks like some people have missed it despite the
massive flaming.

Regards,

Tony

Status of arch/arm in linux-next

[Mark Brown]

On Thu, Apr 14, 2011 at 03:31:27PM +0300, Tony Lindgren wrote:

> I think that 6k lines of new code should wait in the platform specific
> trees and not get merged until we have some real progress on the
> consolidation of code.

> We should make it clear that we're not merging new code except for code
> consolidation. Looks like some people have missed it despite the
> massive flaming.

Personally I had formed the impression that this was an understandable
reaction on the part of Russell that he was applying to his own trees
rather than a general policy that was being applied over all of arch/arm.

I think we need to give people a tractable route to addressing the
consolidation issues with the code they're trying to contribute before
we start rejecting code.  If we can identify something useful that
people can do that's in some way related to what they're trying to
accomplish that's constructive and likely to inspire contributions to
the consolidation efforts but if we're rejecting the code without
any route to improving it that's much less so.  

Status of arch/arm in linux-next

[Tony Lindgren]

* Mark Brown <broonie@opensource.wolfsonmicro.com> [110414 17:17]:
> On Thu, Apr 14, 2011 at 03:31:27PM +0300, Tony Lindgren wrote:
> 
> > I think that 6k lines of new code should wait in the platform specific
> > trees and not get merged until we have some real progress on the
> > consolidation of code.
> 
> > We should make it clear that we're not merging new code except for code
> > consolidation. Looks like some people have missed it despite the
> > massive flaming.
> 
> Personally I had formed the impression that this was an understandable
> reaction on the part of Russell that he was applying to his own trees
> rather than a general policy that was being applied over all of arch/arm.
> 
> I think we need to give people a tractable route to addressing the
> consolidation issues with the code they're trying to contribute before
> we start rejecting code.  If we can identify something useful that
> people can do that's in some way related to what they're trying to
> accomplish that's constructive and likely to inspire contributions to
> the consolidation efforts but if we're rejecting the code without
> any route to improving it that's much less so.  

I don't have anything against adding new code as long as it makes sense
from consolidation point of view.

But obviously just adding thousands of lines of new platform specific
code the old way does not help the situation at all.

Regards,

Tony

Status of arch/arm in linux-next

[Russell King - ARM Linux]

On Thu, Apr 14, 2011 at 03:20:26PM +0100, Mark Brown wrote:
> On Thu, Apr 14, 2011 at 03:31:27PM +0300, Tony Lindgren wrote:
> 
> > I think that 6k lines of new code should wait in the platform specific
> > trees and not get merged until we have some real progress on the
> > consolidation of code.
> 
> > We should make it clear that we're not merging new code except for code
> > consolidation. Looks like some people have missed it despite the
> > massive flaming.
> 
> Personally I had formed the impression that this was an understandable
> reaction on the part of Russell that he was applying to his own trees
> rather than a general policy that was being applied over all of arch/arm.

Given that the non-platform stuff isn't the problem, such a policy has no
effect if its restricted to the core code.

Status of arch/arm in linux-next

[Mark Brown]

On Thu, Apr 14, 2011 at 03:31:27PM +0100, Russell King - ARM Linux wrote:
> On Thu, Apr 14, 2011 at 03:20:26PM +0100, Mark Brown wrote:

> > Personally I had formed the impression that this was an understandable
> > reaction on the part of Russell that he was applying to his own trees
> > rather than a general policy that was being applied over all of arch/arm.

> Given that the non-platform stuff isn't the problem, such a policy has no
> effect if its restricted to the core code.

Oh, of course - I didn't read it as being about the code at all.  I'd
registered it as being about you prioritising your time and effort to
focus on this issue rather than anything else.

Status of arch/arm in linux-next

[Grant Likely]

On Thu, Apr 14, 2011 at 6:31 AM, Tony Lindgren <tony@atomide.com> wrote:
> * Russell King - ARM Linux <linux@arm.linux.org.uk> [110414 14:59]:
>> On Thu, Apr 14, 2011 at 02:08:54PM +0300, Tony Lindgren wrote:
>> >
>> > It's going to take some time before the consolidated code starts
>> > being available.. And until we have something available, we all
>> > should aim for negative diffstat.
>>
>> I think we've already lost any hope of a negative diffstat - with 6k new
>> lines, we will need a heck of a lot of consolidation to counter that.
>>
>> The only thing which I've seen any work being put into is some of the
>> GPIO code - which will be lost in the noise in much the same way that
>> my ARM platform consolidation and deletion of two machine classes in
>> the last merge window was. ?That resulted in the deletion of around 10k
>> lines from the kernel as a whole, or 4.5k lines from arch/arm.
>>
>> Some people are believing that Linus' complaint is about the core ARM
>> code rather than the platform specific code which is the problem.
>
> I think that 6k lines of new code should wait in the platform specific
> trees and not get merged until we have some real progress on the
> consolidation of code.
>
>> We need significant amounts of effort to reverse this - and the more new
>> stuff that appears in peoples trees (and therefore linux-next) the bigger
>> problem we have to convince Linus that we're taking him seriously.
>>
>> So, I think we've already lost any hope of making Linus happy for the
>> next merge window. ?I wish I'm wrong on that, but I don't think so.

Linus isn't stupid either though.  He knows lots of effort is going
into fixing things but that it takes time to get it sorted out.  I
expect he'll provide some grace as we get our house in order, so to
speak.

>
> We should make it clear that we're not merging new code except for code
> consolidation. Looks like some people have missed it despite the
> massive flaming.

I hate to disagree, but I fear that stopping simply isn't an option.
We've finally got many of the vendors to care about upstreaming their
code (which in large part created this new problem).  Turning around
and saying no new code is accepted until an unspecified set of changes
and consolidations are made in mainline both risks that relationship
and it doesn't to anything to stem the tide of new platforms that need
to be supported in some way.

At the very least, if we say 'no' to new code, then at the same time
we must give developers specific options on what they can do right now
to make the code acceptable.  If we cannot provide constructive
alternatives, then it is better to go ahead and accept the code
(assuming the code quality is up-to-snuff, and it follows current
patterns).  I've heard from non-core developers who are more than
willing to make changes to their code to make it ready for upstream,
but are frustrated about simply not knowing how to change their code
to meet the new consolidation requirements.  For example, John Linn
has posted support for the new Xilinx ARM FPGA platform, has been
through several rounds of review, but now he fears that it will all be
rejected and he'll have to start over in 3 months.

On Thu, Apr 14, 2011 at 8:20 AM, Mark Brown
<broonie@opensource.wolfsonmicro.com> wrote:
> I think we need to give people a tractable route to addressing the
> consolidation issues with the code they're trying to contribute before
> we start rejecting code.  If we can identify something useful that
> people can do that's in some way related to what they're trying to
> accomplish that's constructive and likely to inspire contributions to
> the consolidation efforts but if we're rejecting the code without
> any route to improving it that's much less so.

+1

g.

Status of arch/arm in linux-next

[Russell King - ARM Linux]

On Fri, Apr 15, 2011 at 09:12:39AM -0600, Grant Likely wrote:
> For example, John Linn
> has posted support for the new Xilinx ARM FPGA platform, has been
> through several rounds of review, but now he fears that it will all be
> rejected and he'll have to start over in 3 months.

Well, what about my fear that we won't be able to get anything more
past Linus if we submit another 60k lines of code during the next
merge window?

Status of arch/arm in linux-next

[Grant Likely]

On Fri, Apr 15, 2011 at 9:56 AM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Fri, Apr 15, 2011 at 09:12:39AM -0600, Grant Likely wrote:
>> For example, John Linn
>> has posted support for the new Xilinx ARM FPGA platform, has been
>> through several rounds of review, but now he fears that it will all be
>> rejected and he'll have to start over in 3 months.
>
> Well, what about my fear that we won't be able to get anything more
> past Linus if we submit another 60k lines of code during the next
> merge window?

Worst he can say is no.  He /knows/ we're trying to fix the problem.
He also knows that development doesn't stop, even in the midst of
consolidation work that is going to take a lot of time.

To mitigate the risk, how about a
not-ideal-but-should-be-merged-anyway branch that is sent to Linus
after all the core code and consolidation changes have been merged?
We should even describe to Linus why this branch exists and give him
the option to say no if he wants to.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Status of arch/arm in linux-next

[Russell King - ARM Linux]

On Fri, Apr 15, 2011 at 10:10:04AM -0600, Grant Likely wrote:
> On Fri, Apr 15, 2011 at 9:56 AM, Russell King - ARM Linux
> <linux@arm.linux.org.uk> wrote:
> > On Fri, Apr 15, 2011 at 09:12:39AM -0600, Grant Likely wrote:
> >> For example, John Linn
> >> has posted support for the new Xilinx ARM FPGA platform, has been
> >> through several rounds of review, but now he fears that it will all be
> >> rejected and he'll have to start over in 3 months.
> >
> > Well, what about my fear that we won't be able to get anything more
> > past Linus if we submit another 60k lines of code during the next
> > merge window?
> 
> Worst he can say is no.  He /knows/ we're trying to fix the problem.
> He also knows that development doesn't stop, even in the midst of
> consolidation work that is going to take a lot of time.

I find that I can't agree with your point of view on this - I asked
Linus for his view on the message which started this thread.

Linus' big concern is the platform stuff, and he is close to refusing
to pull various git trees if he thinks people haven't taken his concern
on board.

As a result of that, I'm going to modify my position to be: bug fixes,
regression fixes, consolidation, and core code features.

Towards the end of the cycle, we may be able to consider some platforms,
but _only_ if they make use of the consolidated features and therefore
have _minimal_ additional code.

Status of arch/arm in linux-next

[Mark Brown]

On Sat, Apr 16, 2011 at 09:28:02AM +0100, Russell King - ARM Linux wrote:
> On Fri, Apr 15, 2011 at 10:10:04AM -0600, Grant Likely wrote:

> > Worst he can say is no.  He /knows/ we're trying to fix the problem.
> > He also knows that development doesn't stop, even in the midst of
> > consolidation work that is going to take a lot of time.

> I find that I can't agree with your point of view on this - I asked
> Linus for his view on the message which started this thread.

It'd be interesting to read what he wrote there.

> Linus' big concern is the platform stuff, and he is close to refusing
> to pull various git trees if he thinks people haven't taken his concern
> on board.

Right, and the discussion here is essentially about if that can be done
without also blocking development.  But ignoring that for the moment...

> Towards the end of the cycle, we may be able to consider some platforms,
> but _only_ if they make use of the consolidated features and therefore
> have _minimal_ additional code.

...this is the negative side of the message - what we're not willing to
accept.  What's the positive side of the message, what can people do to
help?  What is the level of consolidation work that's needed before we
can develop again, and what's needed to make progress there?

For example, with support for new machines are we saying that for
example we're going to refuse to accept anything that isn't device tree
based?  If so then what needs doing?

Status of arch/arm in linux-next

[Tony Lindgren]

* Mark Brown <broonie@opensource.wolfsonmicro.com> [110416 19:54]:
> On Sat, Apr 16, 2011 at 09:28:02AM +0100, Russell King - ARM Linux wrote:
> 
> > Towards the end of the cycle, we may be able to consider some platforms,
> > but _only_ if they make use of the consolidated features and therefore
> > have _minimal_ additional code.
> 
> ...this is the negative side of the message - what we're not willing to
> accept.  What's the positive side of the message, what can people do to
> help?  What is the level of consolidation work that's needed before we
> can develop again, and what's needed to make progress there?

I gues a large chunk of the consolidation work will happen only after
we have some new frameworks in place.

But meanwhile there is still tons of work left to do in coalescing
code just within the various ARM architectures. 

I think we _should_ accept new platforms if they're sane as we
don't have any alternative available.

But with the existing platforms, I think that the policy for the
next merge window should  be that more code disappears than gets
added.
 
> For example, with support for new machines are we saying that for
> example we're going to refuse to accept anything that isn't device tree
> based?  If so then what needs doing?

Well we can't require that until the device tree code is merged.
And for older platforms, we need the device tree append support.

It seems that there is still at least one problem with the device
tree append support, but once that's sorted out we should
probably merge that code.

Adding a new machine should be a minimal amount of code already.
So with existing platforms that amount of code can be "exchanged"
for some platform code consolidation patches :)

Regards,

Tony

Status of arch/arm in linux-next

[Mark Brown]

On Mon, Apr 18, 2011 at 11:10:52AM +0300, Tony Lindgren wrote:
> * Mark Brown <broonie@opensource.wolfsonmicro.com> [110416 19:54]:

> > ...this is the negative side of the message - what we're not willing to
> > accept.  What's the positive side of the message, what can people do to
> > help?  What is the level of consolidation work that's needed before we
> > can develop again, and what's needed to make progress there?

> I gues a large chunk of the consolidation work will happen only after
> we have some new frameworks in place.

> But meanwhile there is still tons of work left to do in coalescing
> code just within the various ARM architectures. 

> I think we _should_ accept new platforms if they're sane as we
> don't have any alternative available.

> But with the existing platforms, I think that the policy for the
> next merge window should  be that more code disappears than gets
> added.

This is all fairly sensible and reasonable but unfortunately it's not
what's actually being said - what's actually being said is a flat
refusal to accept any new code at all, even when we have no idea what a
viable alternative might be and no matter what other progress is made,
and no clear route to opening up that possibility.

I do think that a flat lines of code criterion isn't terribly helpful as
it isn't *really* what we're trying to optimise and will needlessly
peanalise newer architectures which have good reasons for active
development (like having been merged recently and only having stub
support initially).  There's a big difference between an established
architecture and a recent one here.

> > For example, with support for new machines are we saying that for
> > example we're going to refuse to accept anything that isn't device tree
> > based?  If so then what needs doing?

> Well we can't require that until the device tree code is merged.
> And for older platforms, we need the device tree append support.

> It seems that there is still at least one problem with the device
> tree append support, but once that's sorted out we should
> probably merge that code.

I think we need the append support for all platforms - the idea of
having the description of the CPU in each board device tree just doesn't
seem sensible to me.

> Adding a new machine should be a minimal amount of code already.
> So with existing platforms that amount of code can be "exchanged"
> for some platform code consolidation patches :)

You can easily be pushing at something in four digits by the time you
map out a large board, it's certainly not a trivial amount of code to go
trying to save especially when that's not really directly relevant to
improving the reuse for board drivers and you get into diminishing
returns fairly rapidly.

This does also come back to the whole thing about pointing at relevant
work that people can do - we're not telling people the code they're
submitting is problematic and they need to address things with it, we're
saying that we're not even willing to look at the code or talk about
things that would make it OK.  That's a really negative response that's
essentially impossible to work with.

Re: Status of arch/arm in linux-next

[Tony Lindgren]

* Mark Brown <broonie@opensource.wolfsonmicro.com> [110418 16:54]:
> On Mon, Apr 18, 2011 at 11:10:52AM +0300, Tony Lindgren wrote:
> > * Mark Brown <broonie@opensource.wolfsonmicro.com> [110416 19:54]:
> 
> > > ...this is the negative side of the message - what we're not willing to
> > > accept.  What's the positive side of the message, what can people do to
> > > help?  What is the level of consolidation work that's needed before we
> > > can develop again, and what's needed to make progress there?
> 
> > I gues a large chunk of the consolidation work will happen only after
> > we have some new frameworks in place.
> 
> > But meanwhile there is still tons of work left to do in coalescing
> > code just within the various ARM architectures. 
> 
> > I think we _should_ accept new platforms if they're sane as we
> > don't have any alternative available.
> 
> > But with the existing platforms, I think that the policy for the
> > next merge window should  be that more code disappears than gets
> > added.
> 
> This is all fairly sensible and reasonable but unfortunately it's not
> what's actually being said - what's actually being said is a flat
> refusal to accept any new code at all, even when we have no idea what a
> viable alternative might be and no matter what other progress is made,
> and no clear route to opening up that possibility.

Yeah you're right, it's all a bit unclear :) But already new common
code is popping up for various things like common SRAM support, genirq
etc. So it's a moving target for adding new platforms until that settles
down a bit.

> I do think that a flat lines of code criterion isn't terribly helpful as
> it isn't *really* what we're trying to optimise and will needlessly
> peanalise newer architectures which have good reasons for active
> development (like having been merged recently and only having stub
> support initially).  There's a big difference between an established
> architecture and a recent one here.

Sure. But for an existing platform it can tell something indirectly.
 
> > > For example, with support for new machines are we saying that for
> > > example we're going to refuse to accept anything that isn't device tree
> > > based?  If so then what needs doing?
> 
> > Well we can't require that until the device tree code is merged.
> > And for older platforms, we need the device tree append support.
> 
> > It seems that there is still at least one problem with the device
> > tree append support, but once that's sorted out we should
> > probably merge that code.
> 
> I think we need the append support for all platforms - the idea of
> having the description of the CPU in each board device tree just doesn't
> seem sensible to me.

I think the CPU or SoC can be just included into the board description
file. Or do you have something else in mind for that?

> > Adding a new machine should be a minimal amount of code already.
> > So with existing platforms that amount of code can be "exchanged"
> > for some platform code consolidation patches :)
> 
> You can easily be pushing at something in four digits by the time you
> map out a large board, it's certainly not a trivial amount of code to go
> trying to save especially when that's not really directly relevant to
> improving the reuse for board drivers and you get into diminishing
> returns fairly rapidly.

I guess I'd rather stick to only minimal board additions for now.
At least for me merging anything larger means that later on I may
have deal with sorting it out which is not nice..

BTW, this issue can be already avoided for most part by creating
generic platform init code, like what we have for gpmc-*.c for
any devices connected to the GPMC bus on omaps. And that's something
that can be done already for various platforms.
 
> This does also come back to the whole thing about pointing at relevant
> work that people can do - we're not telling people the code they're
> submitting is problematic and they need to address things with it, we're
> saying that we're not even willing to look at the code or talk about
> things that would make it OK.  That's a really negative response that's
> essentially impossible to work with.

I don't think that's the intention.. But I agree with you, we
need to coordinate things on the mailing lists so everybody knows
what can be done.

Maybe let's try to come up with some checklist on what people
can already help with? How about:

- Is there already generic code posted for review that could
  be used insted?

- Can the platform specific code and defconfigs be combined
  within the platform?

- Is the platform specific data separate from code so that
  the data can be eventually be passed from bootloader using
  device tree?

- Can the new code be made generic?

- Can the new code be made into a loadable module under
  drivers directory?

Regards,

Tony

Status of arch/arm in linux-next

[Tony Lindgren]

* Mark Brown <broonie@opensource.wolfsonmicro.com> [110418 16:54]:
> On Mon, Apr 18, 2011 at 11:10:52AM +0300, Tony Lindgren wrote:
> > * Mark Brown <broonie@opensource.wolfsonmicro.com> [110416 19:54]:
> 
> > > ...this is the negative side of the message - what we're not willing to
> > > accept.  What's the positive side of the message, what can people do to
> > > help?  What is the level of consolidation work that's needed before we
> > > can develop again, and what's needed to make progress there?
> 
> > I gues a large chunk of the consolidation work will happen only after
> > we have some new frameworks in place.
> 
> > But meanwhile there is still tons of work left to do in coalescing
> > code just within the various ARM architectures. 
> 
> > I think we _should_ accept new platforms if they're sane as we
> > don't have any alternative available.
> 
> > But with the existing platforms, I think that the policy for the
> > next merge window should  be that more code disappears than gets
> > added.
> 
> This is all fairly sensible and reasonable but unfortunately it's not
> what's actually being said - what's actually being said is a flat
> refusal to accept any new code at all, even when we have no idea what a
> viable alternative might be and no matter what other progress is made,
> and no clear route to opening up that possibility.

Yeah you're right, it's all a bit unclear :) But already new common
code is popping up for various things like common SRAM support, genirq
etc. So it's a moving target for adding new platforms until that settles
down a bit.

> I do think that a flat lines of code criterion isn't terribly helpful as
> it isn't *really* what we're trying to optimise and will needlessly
> peanalise newer architectures which have good reasons for active
> development (like having been merged recently and only having stub
> support initially).  There's a big difference between an established
> architecture and a recent one here.

Sure. But for an existing platform it can tell something indirectly.
 
> > > For example, with support for new machines are we saying that for
> > > example we're going to refuse to accept anything that isn't device tree
> > > based?  If so then what needs doing?
> 
> > Well we can't require that until the device tree code is merged.
> > And for older platforms, we need the device tree append support.
> 
> > It seems that there is still at least one problem with the device
> > tree append support, but once that's sorted out we should
> > probably merge that code.
> 
> I think we need the append support for all platforms - the idea of
> having the description of the CPU in each board device tree just doesn't
> seem sensible to me.

I think the CPU or SoC can be just included into the board description
file. Or do you have something else in mind for that?

> > Adding a new machine should be a minimal amount of code already.
> > So with existing platforms that amount of code can be "exchanged"
> > for some platform code consolidation patches :)
> 
> You can easily be pushing at something in four digits by the time you
> map out a large board, it's certainly not a trivial amount of code to go
> trying to save especially when that's not really directly relevant to
> improving the reuse for board drivers and you get into diminishing
> returns fairly rapidly.

I guess I'd rather stick to only minimal board additions for now.
At least for me merging anything larger means that later on I may
have deal with sorting it out which is not nice..

BTW, this issue can be already avoided for most part by creating
generic platform init code, like what we have for gpmc-*.c for
any devices connected to the GPMC bus on omaps. And that's something
that can be done already for various platforms.
 
> This does also come back to the whole thing about pointing at relevant
> work that people can do - we're not telling people the code they're
> submitting is problematic and they need to address things with it, we're
> saying that we're not even willing to look at the code or talk about
> things that would make it OK.  That's a really negative response that's
> essentially impossible to work with.

I don't think that's the intention.. But I agree with you, we
need to coordinate things on the mailing lists so everybody knows
what can be done.

Maybe let's try to come up with some checklist on what people
can already help with? How about:

- Is there already generic code posted for review that could
  be used insted?

- Can the platform specific code and defconfigs be combined
  within the platform?

- Is the platform specific data separate from code so that
  the data can be eventually be passed from bootloader using
  device tree?

- Can the new code be made generic?

- Can the new code be made into a loadable module under
  drivers directory?

Regards,

Tony

Re: Status of arch/arm in linux-next

[Mark Brown]

On Mon, Apr 18, 2011 at 05:41:14PM +0300, Tony Lindgren wrote:
> * Mark Brown <broonie@opensource.wolfsonmicro.com> [110418 16:54]:

> > I do think that a flat lines of code criterion isn't terribly helpful as
> > it isn't *really* what we're trying to optimise and will needlessly
> > peanalise newer architectures which have good reasons for active

> Sure. But for an existing platform it can tell something indirectly.

Right, but my point is that it's being treated as gospel not an
indicator.

> > I think we need the append support for all platforms - the idea of
> > having the description of the CPU in each board device tree just doesn't
> > seem sensible to me.

> I think the CPU or SoC can be just included into the board description
> file. Or do you have something else in mind for that?

There's the device tree bits that represent the internals of the CPU
(there was a push to use device tree there too) - that needs to be
merged with the off-chip definitions from the board.

> > You can easily be pushing at something in four digits by the time you
> > map out a large board, it's certainly not a trivial amount of code to go
> > trying to save especially when that's not really directly relevant to
> > improving the reuse for board drivers and you get into diminishing
> > returns fairly rapidly.

> I guess I'd rather stick to only minimal board additions for now.
> At least for me merging anything larger means that later on I may
> have deal with sorting it out which is not nice..

Like I say right now we're just flat out refusing to accept boards at
all so it's all rather moot.

> BTW, this issue can be already avoided for most part by creating
> generic platform init code, like what we have for gpmc-*.c for
> any devices connected to the GPMC bus on omaps. And that's something
> that can be done already for various platforms.

That doesn't really achieve a huge amount for platforms where it really
is just providing resources for the device rather than doing any bus
configuration like gpmc does - on some platforms you just spec the
memory regions and IRQ ranges and you're done.  TBH for those systems it
doesn't seem like a valuable use of time to implement this when device
tree is (probably) just round the corner as for these systems it's only
factoring out data, not actual code.

> > This does also come back to the whole thing about pointing at relevant
> > work that people can do - we're not telling people the code they're
> > submitting is problematic and they need to address things with it, we're
> > saying that we're not even willing to look at the code or talk about
> > things that would make it OK.  That's a really negative response that's
> > essentially impossible to work with.

> I don't think that's the intention.. But I agree with you, we
> need to coordinate things on the mailing lists so everybody knows
> what can be done.

And also so that when people can see what they're aiming for.

> Maybe let's try to come up with some checklist on what people
> can already help with? How about:

> - Is there already generic code posted for review that could
>   be used insted?

> - Can the platform specific code and defconfigs be combined
>   within the platform?

> - Is the platform specific data separate from code so that
>   the data can be eventually be passed from bootloader using
>   device tree?

> - Can the new code be made generic?

> - Can the new code be made into a loadable module under
>   drivers directory?

That looks pretty sensible to me - I'd probably merge the "can it be
generic" with the first point but other than that it looks OK and mostly
also covers drivers as well.

Status of arch/arm in linux-next

[Mark Brown]

On Mon, Apr 18, 2011 at 05:41:14PM +0300, Tony Lindgren wrote:
> * Mark Brown <broonie@opensource.wolfsonmicro.com> [110418 16:54]:

> > I do think that a flat lines of code criterion isn't terribly helpful as
> > it isn't *really* what we're trying to optimise and will needlessly
> > peanalise newer architectures which have good reasons for active

> Sure. But for an existing platform it can tell something indirectly.

Right, but my point is that it's being treated as gospel not an
indicator.

> > I think we need the append support for all platforms - the idea of
> > having the description of the CPU in each board device tree just doesn't
> > seem sensible to me.

> I think the CPU or SoC can be just included into the board description
> file. Or do you have something else in mind for that?

There's the device tree bits that represent the internals of the CPU
(there was a push to use device tree there too) - that needs to be
merged with the off-chip definitions from the board.

> > You can easily be pushing at something in four digits by the time you
> > map out a large board, it's certainly not a trivial amount of code to go
> > trying to save especially when that's not really directly relevant to
> > improving the reuse for board drivers and you get into diminishing
> > returns fairly rapidly.

> I guess I'd rather stick to only minimal board additions for now.
> At least for me merging anything larger means that later on I may
> have deal with sorting it out which is not nice..

Like I say right now we're just flat out refusing to accept boards at
all so it's all rather moot.

> BTW, this issue can be already avoided for most part by creating
> generic platform init code, like what we have for gpmc-*.c for
> any devices connected to the GPMC bus on omaps. And that's something
> that can be done already for various platforms.

That doesn't really achieve a huge amount for platforms where it really
is just providing resources for the device rather than doing any bus
configuration like gpmc does - on some platforms you just spec the
memory regions and IRQ ranges and you're done.  TBH for those systems it
doesn't seem like a valuable use of time to implement this when device
tree is (probably) just round the corner as for these systems it's only
factoring out data, not actual code.

> > This does also come back to the whole thing about pointing at relevant
> > work that people can do - we're not telling people the code they're
> > submitting is problematic and they need to address things with it, we're
> > saying that we're not even willing to look at the code or talk about
> > things that would make it OK.  That's a really negative response that's
> > essentially impossible to work with.

> I don't think that's the intention.. But I agree with you, we
> need to coordinate things on the mailing lists so everybody knows
> what can be done.

And also so that when people can see what they're aiming for.

> Maybe let's try to come up with some checklist on what people
> can already help with? How about:

> - Is there already generic code posted for review that could
>   be used insted?

> - Can the platform specific code and defconfigs be combined
>   within the platform?

> - Is the platform specific data separate from code so that
>   the data can be eventually be passed from bootloader using
>   device tree?

> - Can the new code be made generic?

> - Can the new code be made into a loadable module under
>   drivers directory?

That looks pretty sensible to me - I'd probably merge the "can it be
generic" with the first point but other than that it looks OK and mostly
also covers drivers as well.

Status of arch/arm in linux-next

[Russell King - ARM Linux]

On Mon, Apr 18, 2011 at 02:57:04PM +0100, Mark Brown wrote:
> This does also come back to the whole thing about pointing at relevant
> work that people can do - we're not telling people the code they're
> submitting is problematic and they need to address things with it, we're
> saying that we're not even willing to look at the code or talk about
> things that would make it OK.  That's a really negative response that's
> essentially impossible to work with.

Linus has replied in this thread with his view, which is not much
different from the view which I've been stating all along.

I'll let you read Linus' message and see whether you can translate it
into a positive response for platform maintainers.

I had given up discussing it, as people haven't really been listening.
So, this will probably be my last message in this thread and on this
subject, and I've said what I'm going to do, and that's exactly what
I'm going to do.

I'm very sorry for people with new platforms outstanding like John Linn
who are on the sharp end of this (who have platform code ready to go)
but I see no solution at the moment.  It really is the case that either
I can say no to it, or I can put it in my tree and Linus will say no to
it.  So putting it in my tree *doesn't* help.

Will we ever be able to put John's code in the kernel?  Honestly, I have
no idea.  What I do know is that unless we start doing something to solve
the problem we have today with the quantity of code under arch/arm _and_
the constant churn of that code, we will _never_ be able to add new
platform support in any shape or form to the kernel.

If this means that people like Xilinx decide to drop their Linux kernel
effort, or decide not to bother submitting to mainline, then that's
unfortunate, but is not something I have any control over given the
situation we find ourselves in.

Status of arch/arm in linux-next

[Mark Brown]

On Mon, Apr 18, 2011 at 06:18:57PM +0100, Russell King - ARM Linux wrote:

> Linus has replied in this thread with his view, which is not much
> different from the view which I've been stating all along.

Yeah, I saw that.  Quite frankly it's astonishing - I must apologise, I
had thought you were most likely misinterpreting what he was saying.

> Will we ever be able to put John's code in the kernel?  Honestly, I have
> no idea.  What I do know is that unless we start doing something to solve
> the problem we have today with the quantity of code under arch/arm _and_
> the constant churn of that code, we will _never_ be able to add new
> platform support in any shape or form to the kernel.

Given where we're at right now I'm guessing we're going to see ARM
development halted until at least the merge window after next which is
5-6 months or so.  We're not talking about trivial bits of
infrastructure here and obviously any substantial reworks here are going
to involve churn anyway.

To make matters worse unless people just give up the longer we keep the
tree shut down the larger the merge will be when it does reopen.

Status of arch/arm in linux-next

[Thomas Gleixner]

On Mon, 18 Apr 2011, Mark Brown wrote:
> On Mon, Apr 18, 2011 at 06:18:57PM +0100, Russell King - ARM Linux wrote:
> 
> > Linus has replied in this thread with his view, which is not much
> > different from the view which I've been stating all along.
> 
> Yeah, I saw that.  Quite frankly it's astonishing - I must apologise, I
> had thought you were most likely misinterpreting what he was saying.

Sigh.
 
> > Will we ever be able to put John's code in the kernel?  Honestly, I have
> > no idea.  What I do know is that unless we start doing something to solve
> > the problem we have today with the quantity of code under arch/arm _and_
> > the constant churn of that code, we will _never_ be able to add new
> > platform support in any shape or form to the kernel.
> 
> Given where we're at right now I'm guessing we're going to see ARM
> development halted until at least the merge window after next which is
> 5-6 months or so.  We're not talking about trivial bits of
> infrastructure here and obviously any substantial reworks here are going
> to involve churn anyway.
> 
> To make matters worse unless people just give up the longer we keep the
> tree shut down the larger the merge will be when it does reopen.

I tend to disagree. It's not rocket science to cleanup stuff just
along the way. I did the (not yet perfect) conversion of about 20 irq
chips on saturday just to see how far I get with that abstract
chip. And it simply got rid of >1200 LOC with some more potential
reduction already detected.

So when we come up with such generic abstractions then it's not
a too big burden for a maintainer to care about his 1 or 2 irq chips
and some other small cleanups here and there.

Linus does not expect that ARM code shrinks 80% in the next cycle, but
he wants to see that people take him seriously and actually cut down
code in a diffstat visible way.

Thanks,

	tglx

Status of arch/arm in linux-next

[Mark Brown]

On Mon, Apr 18, 2011 at 11:40:15PM +0200, Thomas Gleixner wrote:
> On Mon, 18 Apr 2011, Mark Brown wrote:

> > Yeah, I saw that.  Quite frankly it's astonishing - I must apologise, I
> > had thought you were most likely misinterpreting what he was saying.

> Sigh.

I find it hard to read it as saying anything other than that new code is
unwelcome and any substantial diffstat needs to be negative.  This is
pretty much what Russell has been saying (if a bit less hard line).

> > To make matters worse unless people just give up the longer we keep the
> > tree shut down the larger the merge will be when it does reopen.

> I tend to disagree. It's not rocket science to cleanup stuff just
> along the way. I did the (not yet perfect) conversion of about 20 irq
> chips on saturday just to see how far I get with that abstract
> chip. And it simply got rid of >1200 LOC with some more potential
> reduction already detected.

That's not terribly helpful to people trying to contribute anything
other than refactoring of existing code - the message that's been given
is that anything which adds new support (platforms, machines, whatever)
is not even worth looking at.  For people working on code that's already
in mainline they can refactor and that's fine for them.  If that's not
the case then people are just being told the code is not welcome until
some non-specific amount of cleanup work has been done to existing code.

This is not really constructive from the contributor side as it doesn't
offer any direct or clear route to addressing the problem that's
blocking their code, there's no improvement that can be made to code
that would make it worth considering.  What we're telling people to do
is work on random improvements to more or less tangentially related
code.  This doesn't seem entirely reasonable and is going to be
especially offputting for new contributors (like the people trying to
submit new platforms, many of them will be new to mainline work) as it's
a pretty big jump to start working on less familiar code when you're
still trying to find your feet and worried about stepping on people's
toes or breaking things, not to mention justifying your time to
management.

> So when we come up with such generic abstractions then it's not
> a too big burden for a maintainer to care about his 1 or 2 irq chips
> and some other small cleanups here and there.

Of course, it's obvious that as we add abstractions we'll be able to
make improvements.  Saying to people that they should use existing
abstractions (recently added or otherwise) or even that they should
factor out bits of their code so that others can use them is exactly the
approach I'd hope we'd be taking here.  That would be clear, relevant
review which gives people something direct they can work towards.

> Linus does not expect that ARM code shrinks 80% in the next cycle, but
> he wants to see that people take him seriously and actually cut down
> code in a diffstat visible way.

The issue for me is that we're doing that not by raising the bar on new
contributions and demonstrating that there's work going into factoring
stuff out but rather by shutting down entirely to pretty much anything
other than refactoring so that we force a negative diffstat.

It'd be much better if we could do something like what Grant suggested
earlier in the thread, splitting out the refactoring work (which you'd
hope will have a nice looking diffstat) from the run of the mill stuff
so that the cleanup is highlighted, especially when we combine this
approach with the more stringent review that pretty much everyone is
bought in to.  That seems more sustainable for the long term, and much
more helpful for ongoing development.

Status of arch/arm in linux-next

[Mark Brown]

On Thu, Apr 14, 2011 at 01:02:09PM +0100, Russell King - ARM Linux wrote:

> Some people are believing that Linus' complaint is about the core ARM
> code rather than the platform specific code which is the problem.

I don't think I've seen anyone saying that.  If you're referring to my
comments last night that was purely about the strict no new code policy
you've adopted in the trees you manage which to my mind is mostly
orthogonal to the actual problems in the code.

> So, I think we've already lost any hope of making Linus happy for the
> next merge window.  I wish I'm wrong on that, but I don't think so.

Given the scale of the problem it doesn't seem like there's ever been
any prospect of making Linus happy in a single merge window.

Status of arch/arm in linux-next

[Nico Erfurth]

On 14.04.11 14:02, Russell King - ARM Linux wrote:

>> It's going to take some time before the consolidated code starts
>> being available.. And until we have something available, we all
>> should aim for negative diffstat.
> 
> I think we've already lost any hope of a negative diffstat - with 6k new
> lines, we will need a heck of a lot of consolidation to counter that.

I've just skipped through the board-files in mach-at91, there is a lot
of potential for consolidation. But the big problem could be to find
people who can test it afterwards. It would also result in a bunch of
large commits. For an example, I've merged board-usb-a9260.c and
board-usb-a9263.c into board-usb-a926x.c

Result:
 arch/arm/mach-at91/board-usb-a9260.c |  236 --------------------------
 arch/arm/mach-at91/board-usb-a9263.c |  252 ---------------------------
 arch/arm/mach-at91/board-usb-a926x.c |  310
++++++++++++++++++++++++++++++++++
 3 files changed, 310 insertions(+), 488 deletions(-)

In the end, the total LOC is a lot smaller, but the changeset itself
"looks" big.

There are a lot more examples in the arch/arm/mach-* namespaces. Like
mach-kirkwood where {sheevaplug,guruplug,dockstart}-setup.c could be
easily merged together.

Nico

Status of arch/arm in linux-next

[Nicolas Ferre]

Le 15/04/2011 04:59, Nico Erfurth :
> On 14.04.11 14:02, Russell King - ARM Linux wrote:
> 
>>> It's going to take some time before the consolidated code starts
>>> being available.. And until we have something available, we all
>>> should aim for negative diffstat.
>>
>> I think we've already lost any hope of a negative diffstat - with 6k new
>> lines, we will need a heck of a lot of consolidation to counter that.
> 
> I've just skipped through the board-files in mach-at91, there is a lot
> of potential for consolidation. But the big problem could be to find
> people who can test it afterwards.

My opinion is that you can do the improvement and propose it to the
community. I guess that the maintainer of the board will take your
improvements and tests them.

> It would also result in a bunch of
> large commits. For an example, I've merged board-usb-a9260.c and
> board-usb-a9263.c into board-usb-a926x.c
> 
> Result:
>  arch/arm/mach-at91/board-usb-a9260.c |  236 --------------------------
>  arch/arm/mach-at91/board-usb-a9263.c |  252 ---------------------------
>  arch/arm/mach-at91/board-usb-a926x.c |  310
> ++++++++++++++++++++++++++++++++++
>  3 files changed, 310 insertions(+), 488 deletions(-)

Nice work!

So, anyway, even if the board maintainer is not testing, our at91
maintainer's view of it is that the move has to be done so we will
certainly merge it...

> In the end, the total LOC is a lot smaller, but the changeset itself
> "looks" big.

If it is consolidation work, it is ok.

Bye,
-- 
Nicolas Ferre

Status of arch/arm in linux-next

[Nico Erfurth]

On 15.04.11 10:21, Nicolas Ferre wrote:

>>> I think we've already lost any hope of a negative diffstat - with 6k new
>>> lines, we will need a heck of a lot of consolidation to counter that.
>>
>> I've just skipped through the board-files in mach-at91, there is a lot
>> of potential for consolidation. But the big problem could be to find
>> people who can test it afterwards.
> 
> My opinion is that you can do the improvement and propose it to the
> community. I guess that the maintainer of the board will take your
> improvements and tests them.

We'll see, most of the board-files in mach-at91 tree are pretty old and
for hardware which is not built anymore. ;)

>> It would also result in a bunch of
>> large commits. For an example, I've merged board-usb-a9260.c and
>> board-usb-a9263.c into board-usb-a926x.c
>>
>> Result:
>>  arch/arm/mach-at91/board-usb-a9260.c |  236 --------------------------
>>  arch/arm/mach-at91/board-usb-a9263.c |  252 ---------------------------
>>  arch/arm/mach-at91/board-usb-a926x.c |  310
>> ++++++++++++++++++++++++++++++++++
>>  3 files changed, 310 insertions(+), 488 deletions(-)
> 
> Nice work!
> 
> So, anyway, even if the board maintainer is not testing, our at91
> maintainer's view of it is that the move has to be done so we will
> certainly merge it...
> 
>> In the end, the total LOC is a lot smaller, but the changeset itself
>> "looks" big.
> 
> If it is consolidation work, it is ok.

Ok, I'll clean up the patch and send it in 2-3 days.

Nico

Status of arch/arm in linux-next

[Linus Walleij]

2011/4/14 Russell King - ARM Linux <linux@arm.linux.org.uk>:

> This morning, I looked at linux-next to find out how arch/arm is doing
> for the next merge window.
>
> $ git diff -C --cumulative v2.6.39-rc1... arch/arm
> (...)
> ? 7.6% arch/arm/mach-ux500/include/mach/
> ?46.1% arch/arm/mach-ux500/
> (...)
> Please take a moment to consider how Linus will react to this at the
> next merge window.

This instance of Linus feels guilty for that...

Since ~50% of it is ux500 that I usually merge through your tree,
I suspect you're simply not going to pull it so then half of the problem
is gone already :-D

Anyway, the bulk of that is a PRCMU driver, similar to the stuff
in arch/arm/mach-omap2/prcm*. So let's think about what we can
do about it.

Since this is a one-off kind of thing, a singleton driver that controls
power, reset and some GPIO on the chip. I contemplate moving the
stuff to either:

drivers/misc/ux500/*
include/linux/misc/ux500/*

or:

drivers/platform/arm/ux500/*
include/linux/platform/arm/ux500/*

Are any of these generally speaking good ideas?

Either place outside arch/arm/* is fine with me, creating something like
drivers/prcmu/* would be a bit thick since the hardware basically does
not look like anything else.

The basic problem it's reflecting is that ARM does not have something
like ACPI, that's basically what the driver is doing, and since every
vendor does their own HW thingy it's not like it's easily consolidated.

In the meantime I'm working on migrating GPIO drivers from mach-u300
and plat-nomadik into drivers/gpio so I will hopefully provide some negative
stats.

Yours,
Linus Walleij

Status of arch/arm in linux-next

[Tony Lindgren]

* Linus Walleij <linus.walleij@linaro.org> [110415 04:14]:
> 2011/4/14 Russell King - ARM Linux <linux@arm.linux.org.uk>:
> 
> > This morning, I looked at linux-next to find out how arch/arm is doing
> > for the next merge window.
> >
> > $ git diff -C --cumulative v2.6.39-rc1... arch/arm
> > (...)
> > ? 7.6% arch/arm/mach-ux500/include/mach/
> > ?46.1% arch/arm/mach-ux500/
> > (...)
> > Please take a moment to consider how Linus will react to this at the
> > next merge window.
> 
> This instance of Linus feels guilty for that...
> 
> Since ~50% of it is ux500 that I usually merge through your tree,
> I suspect you're simply not going to pull it so then half of the problem
> is gone already :-D
> 
> Anyway, the bulk of that is a PRCMU driver, similar to the stuff
> in arch/arm/mach-omap2/prcm*. So let's think about what we can
> do about it.

Yeah OK.
 
> Since this is a one-off kind of thing, a singleton driver that controls
> power, reset and some GPIO on the chip. I contemplate moving the
> stuff to either:
> 
> drivers/misc/ux500/*
> include/linux/misc/ux500/*
> 
> or:
> 
> drivers/platform/arm/ux500/*
> include/linux/platform/arm/ux500/*
> 
> Are any of these generally speaking good ideas?

Or maybe drivers/arm?

Anyways, whatever can be done as loadable modules should be done
that way. That makes the life for distros much easier ;)
 
> Either place outside arch/arm/* is fine with me, creating something like
> drivers/prcmu/* would be a bit thick since the hardware basically does
> not look like anything else.
> 
> The basic problem it's reflecting is that ARM does not have something
> like ACPI, that's basically what the driver is doing, and since every
> vendor does their own HW thingy it's not like it's easily consolidated.

Yeah and that's going to take time.
 
> In the meantime I'm working on migrating GPIO drivers from mach-u300
> and plat-nomadik into drivers/gpio so I will hopefully provide some negative
> stats.

We too can move the omap gpio code there too..

Regards,

Tony

Status of arch/arm in linux-next

[Arnd Bergmann]

On Friday 15 April 2011, Tony Lindgren wrote:
> > drivers/platform/arm/ux500/*
> > include/linux/platform/arm/ux500/*
> > 
> > Are any of these generally speaking good ideas?
> 
> Or maybe drivers/arm?
> 
> Anyways, whatever can be done as loadable modules should be done
> that way. That makes the life for distros much easier ;)

drivers/arm would just become the next pile of crap if we start that,
like drivers/misc is starting to look now.

There are more things that I think should become subsystems
for themselves, with a proper maintainer, rather than bits
that simply get copied across all platforms.

Besides gpio and regulators (which is already in drivers/gpio),
it would probably be good to have drivers/irq, drivers/iommu
and others.

For the prcmu, even after staring at the code for half an hour,
I still have no idea what it actually is. I just see function
names like prcmu_config_hotdog and prcmu_request_ape_opp_100_voltage
that tell me that it's both extremely generic and extremely
detailed, and that it deals with simians and food.

If I'm not mistaken, it's some sort of systems management, right?
My feeling is that grouping the bits together in prcmu files
is somewhat suboptimal, and it could be better to spread the
prcmu functions into multiple places, e.g. an IRQ driver,
an I2C driver, etc, and exporting just the common interfaces
from a file that handles the prcmu.

> > Either place outside arch/arm/* is fine with me, creating something like
> > drivers/prcmu/* would be a bit thick since the hardware basically does
> > not look like anything else.
> > 
> > The basic problem it's reflecting is that ARM does not have something
> > like ACPI, that's basically what the driver is doing, and since every
> > vendor does their own HW thingy it's not like it's easily consolidated.
> 
> Yeah and that's going to take time.

ACPI does a lot of things (unfortunately), and I did not get the impression
that prcmu does the one part we really need, which is complete enumeration
of all devices. If it did that, it should become a bus_type and replace
the hardcoded sets of platform devices.

> > In the meantime I'm working on migrating GPIO drivers from mach-u300
> > and plat-nomadik into drivers/gpio so I will hopefully provide some negative
> > stats.
> 
> We too can move the omap gpio code there too..

That sounds like a good idea. Same for the regulator drivers that are
currently in arch/arm instead of drivers/regulator.

	Arnd

Status of arch/arm in linux-next

[Mark Brown]

On Tue, Apr 19, 2011 at 04:16:01PM +0200, Arnd Bergmann wrote:

> That sounds like a good idea. Same for the regulator drivers that are
> currently in arch/arm instead of drivers/regulator.

We don't have any regulator drivers in arch/arm I think (unless the ST
one got merged, but that's actually a dual regulator/system management
driver rather than a proper regulator - when I reviewed it I didn't flag
the location due to the second interface on the side).

Status of arch/arm in linux-next

[Arnd Bergmann]

On Tuesday 19 April 2011, Mark Brown wrote:
> On Tue, Apr 19, 2011 at 04:16:01PM +0200, Arnd Bergmann wrote:
> 
> > That sounds like a good idea. Same for the regulator drivers that are
> > currently in arch/arm instead of drivers/regulator.
> 
> We don't have any regulator drivers in arch/arm I think (unless the ST
> one got merged, but that's actually a dual regulator/system management
> driver rather than a proper regulator - when I reviewed it I didn't flag
> the location due to the second interface on the side).

I was thinking of arch/arm/mach-ux500/regulator-db8500.c in linux-next.

I can only see regulator contents in there, what do you mean by
system management?

	Arnd

Status of arch/arm in linux-next

[Mark Brown]

On Tue, Apr 19, 2011 at 04:55:12PM +0200, Arnd Bergmann wrote:

> I was thinking of arch/arm/mach-ux500/regulator-db8500.c in linux-next.

> I can only see regulator contents in there, what do you mean by
> system management?

All the fiddling around with the PRCMU and the power state faff (which
on closer look now seems to be only in this file, hrm).

Status of arch/arm in linux-next

[Linus Walleij]

2011/4/19 Arnd Bergmann <arnd@arndb.de>:
> On Tuesday 19 April 2011, Mark Brown wrote:
>> On Tue, Apr 19, 2011 at 04:16:01PM +0200, Arnd Bergmann wrote:
>>
>> > That sounds like a good idea. Same for the regulator drivers that are
>> > currently in arch/arm instead of drivers/regulator.
>>
>> We don't have any regulator drivers in arch/arm I think (unless the ST
>> one got merged, but that's actually a dual regulator/system management
>> driver rather than a proper regulator - when I reviewed it I didn't flag
>> the location due to the second interface on the side).
>
> I was thinking of arch/arm/mach-ux500/regulator-db8500.c in linux-next.
>
> I can only see regulator contents in there, what do you mean by
> system management?

I will surely put that into drivers/regulator, Mark already requested that
as part of his review.

The problem is that it is dependent on the PRCMU driver which
provides the communication mechanism to actually control these
regulators.

The PRCMU is the Power Reset and Control Management Unit,
it is a register pages where you send commands to a firmware
running on its own CPU on the other side, partly using mailboxes.
The firmware handles things like voltage and power domains
(modeled as regulators), frequency changes (using CPUfreq),
idle states (CPUidle and sleep, idling), as well as resetting
particular memory blocks AND an I2C channel to the AB8500
chip (driven from drivers/ab8500-core.c indeed) and some
GPIO configuration.

We are indeed trying to spread the stuff in there out across
the proper subsystems.

Thinking of it, is it OK to put chip CPUfreq drivers into
drivers/cpufreq/* instead of into the arch/arm/* platform
code as everyone does right now? We could probably
fix that and bring down the diffstat considerably.

Yours,
Linus Walleij

Re: Status of arch/arm in linux-next

[Arnd Bergmann]

On Tuesday 19 April 2011, Linus Walleij wrote:
> I will surely put that into drivers/regulator, Mark already requested that
> as part of his review.
> 
> The problem is that it is dependent on the PRCMU driver which
> provides the communication mechanism to actually control these
> regulators.
> 
> The PRCMU is the Power Reset and Control Management Unit,
> it is a register pages where you send commands to a firmware
> running on its own CPU on the other side, partly using mailboxes.
> The firmware handles things like voltage and power domains
> (modeled as regulators), frequency changes (using CPUfreq),
> idle states (CPUidle and sleep, idling), as well as resetting
> particular memory blocks AND an I2C channel to the AB8500
> chip (driven from drivers/ab8500-core.c indeed) and some
> GPIO configuration.

Ok, thanks for the explanation.

> Thinking of it, is it OK to put chip CPUfreq drivers into
> drivers/cpufreq/* instead of into the arch/arm/* platform
> code as everyone does right now? We could probably
> fix that and bring down the diffstat considerably.

That's something to discuss with Dave Jones and other people
interested in cpufre. Right now, all cpufreq drivers, including
those for other architectures are in arch/.

I think it would be good to have the out of the individual
platforms, in particular in order to get better reviews of
new cpufreq drivers by people that are interested in them.

	Arnd

Status of arch/arm in linux-next

[Arnd Bergmann]

On Tuesday 19 April 2011, Linus Walleij wrote:
> I will surely put that into drivers/regulator, Mark already requested that
> as part of his review.
> 
> The problem is that it is dependent on the PRCMU driver which
> provides the communication mechanism to actually control these
> regulators.
> 
> The PRCMU is the Power Reset and Control Management Unit,
> it is a register pages where you send commands to a firmware
> running on its own CPU on the other side, partly using mailboxes.
> The firmware handles things like voltage and power domains
> (modeled as regulators), frequency changes (using CPUfreq),
> idle states (CPUidle and sleep, idling), as well as resetting
> particular memory blocks AND an I2C channel to the AB8500
> chip (driven from drivers/ab8500-core.c indeed) and some
> GPIO configuration.

Ok, thanks for the explanation.

> Thinking of it, is it OK to put chip CPUfreq drivers into
> drivers/cpufreq/* instead of into the arch/arm/* platform
> code as everyone does right now? We could probably
> fix that and bring down the diffstat considerably.

That's something to discuss with Dave Jones and other people
interested in cpufre. Right now, all cpufreq drivers, including
those for other architectures are in arch/.

I think it would be good to have the out of the individual
platforms, in particular in order to get better reviews of
new cpufreq drivers by people that are interested in them.

	Arnd

Re: Status of arch/arm in linux-next

[Mark Brown]

On Tue, Apr 19, 2011 at 06:01:19PM +0200, Arnd Bergmann wrote:

> I think it would be good to have the out of the individual
> platforms, in particular in order to get better reviews of
> new cpufreq drivers by people that are interested in them.

FWIW when I pushed the S3C64x0 cpufreq driver into mainline I posted the
driver to the cpufreq list and maintainers but never got any response
from them.

Status of arch/arm in linux-next

[Mark Brown]

On Tue, Apr 19, 2011 at 06:01:19PM +0200, Arnd Bergmann wrote:

> I think it would be good to have the out of the individual
> platforms, in particular in order to get better reviews of
> new cpufreq drivers by people that are interested in them.

FWIW when I pushed the S3C64x0 cpufreq driver into mainline I posted the
driver to the cpufreq list and maintainers but never got any response
from them.

Re: Status of arch/arm in linux-next

[Dave Jones]

On Tue, Apr 19, 2011 at 05:05:46PM +0100, Mark Brown wrote:
 > On Tue, Apr 19, 2011 at 06:01:19PM +0200, Arnd Bergmann wrote:
 > 
 > > I think it would be good to have the out of the individual
 > > platforms, in particular in order to get better reviews of
 > > new cpufreq drivers by people that are interested in them.
 > 
 > FWIW when I pushed the S3C64x0 cpufreq driver into mainline I posted the
 > driver to the cpufreq list and maintainers but never got any response
 > from them.

Like I said in another mail, the platform-specific nature of cpufreq drivers
means that the only people who can really review them are people familiar
with the architecture. (modulo the cpufreq api bits, but it's usually
either no-brainer stuff, or bugs that aren't immediately obvious from review,
like some of the locking mistakes we've historically seen)

This is why I don't believe that moving this code from arch/ to drivers/
will change anything.

if there's commonality between some of the ARM arch drivers, why can't
there be a arch/arm/cpufreq/ dir for the shared code, and do everything there ?

Right now "move it out of the arm tree into the cpufreq tree" just seems
like a cop-out to hide the problem.

	Dave

Status of arch/arm in linux-next

[Dave Jones]

On Tue, Apr 19, 2011 at 05:05:46PM +0100, Mark Brown wrote:
 > On Tue, Apr 19, 2011 at 06:01:19PM +0200, Arnd Bergmann wrote:
 > 
 > > I think it would be good to have the out of the individual
 > > platforms, in particular in order to get better reviews of
 > > new cpufreq drivers by people that are interested in them.
 > 
 > FWIW when I pushed the S3C64x0 cpufreq driver into mainline I posted the
 > driver to the cpufreq list and maintainers but never got any response
 > from them.

Like I said in another mail, the platform-specific nature of cpufreq drivers
means that the only people who can really review them are people familiar
with the architecture. (modulo the cpufreq api bits, but it's usually
either no-brainer stuff, or bugs that aren't immediately obvious from review,
like some of the locking mistakes we've historically seen)

This is why I don't believe that moving this code from arch/ to drivers/
will change anything.

if there's commonality between some of the ARM arch drivers, why can't
there be a arch/arm/cpufreq/ dir for the shared code, and do everything there ?

Right now "move it out of the arm tree into the cpufreq tree" just seems
like a cop-out to hide the problem.

	Dave

Re: Status of arch/arm in linux-next

[Nicolas Pitre]

On Thu, 21 Apr 2011, Dave Jones wrote:

> On Tue, Apr 19, 2011 at 05:05:46PM +0100, Mark Brown wrote:
>  > On Tue, Apr 19, 2011 at 06:01:19PM +0200, Arnd Bergmann wrote:
>  > 
>  > > I think it would be good to have the out of the individual
>  > > platforms, in particular in order to get better reviews of
>  > > new cpufreq drivers by people that are interested in them.
>  > 
>  > FWIW when I pushed the S3C64x0 cpufreq driver into mainline I posted the
>  > driver to the cpufreq list and maintainers but never got any response
>  > from them.
> 
> Like I said in another mail, the platform-specific nature of cpufreq drivers
> means that the only people who can really review them are people familiar
> with the architecture. (modulo the cpufreq api bits, but it's usually
> either no-brainer stuff, or bugs that aren't immediately obvious from review,
> like some of the locking mistakes we've historically seen)

So what? Lots of drivers in drivers/ are used and even usable only on 
one architecture already.  Moving the cpufreq drivers from arch/* into 
drivers/cpufreq/ won't change the fact that only the people familiar 
with the target hardware will be able to properly review the details.  
This is no different for most kind of drivers.

> This is why I don't believe that moving this code from arch/ to drivers/
> will change anything.

That at least would have the property of gathering drivers together 
according to their _purpose_, irrespective of their implementation 
details.  That's the case for all the other class of drivers already. 
Why would cpufreq drivers be different?

> if there's commonality between some of the ARM arch drivers, why can't
> there be a arch/arm/cpufreq/ dir for the shared code, and do everything there ?

Because usually there isn't.  "ARM" is just a CPU architecture, not a 
system architecture.  Everything around the core is different from one 
vendor to the next.  And when commonality exists it is much easier to 
deal with if it is close together.

According to your argument, we would have to create arch/arm/net, 
arch/arm/alsa, arch/arm/video, etc, which no one would agree with.  Yet 
almost all those ARM related drivers are highly platform-specific.

The criteria for sorting out driver location is normally the driver's 
purpose, not the platform where it is used. Why would cpufreq have to be 
different?

> Right now "move it out of the arm tree into the cpufreq tree" just seems
> like a cop-out to hide the problem.

No, there is nothing to hide.  If there is a code duplication problem, 
it will be even more visible if that code is moved to a common place, 
and therefore any needed consolidation would happen more naturally.  I 
doubt anyone would try to copy a driver just to perform slight 
modifications on it if it is to land in the same directory as the 
original.


Nicolas

Status of arch/arm in linux-next

[Nicolas Pitre]

On Thu, 21 Apr 2011, Dave Jones wrote:

> On Tue, Apr 19, 2011 at 05:05:46PM +0100, Mark Brown wrote:
>  > On Tue, Apr 19, 2011 at 06:01:19PM +0200, Arnd Bergmann wrote:
>  > 
>  > > I think it would be good to have the out of the individual
>  > > platforms, in particular in order to get better reviews of
>  > > new cpufreq drivers by people that are interested in them.
>  > 
>  > FWIW when I pushed the S3C64x0 cpufreq driver into mainline I posted the
>  > driver to the cpufreq list and maintainers but never got any response
>  > from them.
> 
> Like I said in another mail, the platform-specific nature of cpufreq drivers
> means that the only people who can really review them are people familiar
> with the architecture. (modulo the cpufreq api bits, but it's usually
> either no-brainer stuff, or bugs that aren't immediately obvious from review,
> like some of the locking mistakes we've historically seen)

So what? Lots of drivers in drivers/ are used and even usable only on 
one architecture already.  Moving the cpufreq drivers from arch/* into 
drivers/cpufreq/ won't change the fact that only the people familiar 
with the target hardware will be able to properly review the details.  
This is no different for most kind of drivers.

> This is why I don't believe that moving this code from arch/ to drivers/
> will change anything.

That at least would have the property of gathering drivers together 
according to their _purpose_, irrespective of their implementation 
details.  That's the case for all the other class of drivers already. 
Why would cpufreq drivers be different?

> if there's commonality between some of the ARM arch drivers, why can't
> there be a arch/arm/cpufreq/ dir for the shared code, and do everything there ?

Because usually there isn't.  "ARM" is just a CPU architecture, not a 
system architecture.  Everything around the core is different from one 
vendor to the next.  And when commonality exists it is much easier to 
deal with if it is close together.

According to your argument, we would have to create arch/arm/net, 
arch/arm/alsa, arch/arm/video, etc, which no one would agree with.  Yet 
almost all those ARM related drivers are highly platform-specific.

The criteria for sorting out driver location is normally the driver's 
purpose, not the platform where it is used. Why would cpufreq have to be 
different?

> Right now "move it out of the arm tree into the cpufreq tree" just seems
> like a cop-out to hide the problem.

No, there is nothing to hide.  If there is a code duplication problem, 
it will be even more visible if that code is moved to a common place, 
and therefore any needed consolidation would happen more naturally.  I 
doubt anyone would try to copy a driver just to perform slight 
modifications on it if it is to land in the same directory as the 
original.


Nicolas

Re: Status of arch/arm in linux-next

[Tony Lindgren]

* Nicolas Pitre <nicolas.pitre@linaro.org> [110421 13:59]:
> On Thu, 21 Apr 2011, Dave Jones wrote:
> 
> > On Tue, Apr 19, 2011 at 05:05:46PM +0100, Mark Brown wrote:
> 
> > This is why I don't believe that moving this code from arch/ to drivers/
> > will change anything.
> 
> That at least would have the property of gathering drivers together 
> according to their _purpose_, irrespective of their implementation 
> details.  That's the case for all the other class of drivers already. 
> Why would cpufreq drivers be different?

And drivers do have well defined standards, so that automatically prevents
people sneaking in spaghetti calls to platform specific code ;)

Tony

Status of arch/arm in linux-next

[Tony Lindgren]

* Nicolas Pitre <nicolas.pitre@linaro.org> [110421 13:59]:
> On Thu, 21 Apr 2011, Dave Jones wrote:
> 
> > On Tue, Apr 19, 2011 at 05:05:46PM +0100, Mark Brown wrote:
> 
> > This is why I don't believe that moving this code from arch/ to drivers/
> > will change anything.
> 
> That at least would have the property of gathering drivers together 
> according to their _purpose_, irrespective of their implementation 
> details.  That's the case for all the other class of drivers already. 
> Why would cpufreq drivers be different?

And drivers do have well defined standards, so that automatically prevents
people sneaking in spaghetti calls to platform specific code ;)

Tony

Re: Status of arch/arm in linux-next

[Arnd Bergmann]

On Thursday 21 April 2011, Nicolas Pitre wrote:
> > if there's commonality between some of the ARM arch drivers, why can't
> > there be a arch/arm/cpufreq/ dir for the shared code, and do everything there ?
> 
> Because usually there isn't.  "ARM" is just a CPU architecture, not a 
> system architecture.  Everything around the core is different from one 
> vendor to the next.  And when commonality exists it is much easier to 
> deal with if it is close together.

Exactly. To make matters worse, we are starting to see a number of vendors
that use multiple CPU architectures with the same I/O devices (e.g. Renesas,
Freescale, Xilinx, TI, ...). Not sure if any of these use the same cpufreq
register on more than one architecture, but it's quite likely to happen
at some point.

	Arnd

Status of arch/arm in linux-next

[Arnd Bergmann]

On Thursday 21 April 2011, Nicolas Pitre wrote:
> > if there's commonality between some of the ARM arch drivers, why can't
> > there be a arch/arm/cpufreq/ dir for the shared code, and do everything there ?
> 
> Because usually there isn't.  "ARM" is just a CPU architecture, not a 
> system architecture.  Everything around the core is different from one 
> vendor to the next.  And when commonality exists it is much easier to 
> deal with if it is close together.

Exactly. To make matters worse, we are starting to see a number of vendors
that use multiple CPU architectures with the same I/O devices (e.g. Renesas,
Freescale, Xilinx, TI, ...). Not sure if any of these use the same cpufreq
register on more than one architecture, but it's quite likely to happen
at some point.

	Arnd

Re: Status of arch/arm in linux-next

[Rafael J. Wysocki]

On Tuesday, April 26, 2011, Arnd Bergmann wrote:
> On Thursday 21 April 2011, Nicolas Pitre wrote:
> > > if there's commonality between some of the ARM arch drivers, why can't
> > > there be a arch/arm/cpufreq/ dir for the shared code, and do everything there ?
> > 
> > Because usually there isn't.  "ARM" is just a CPU architecture, not a 
> > system architecture.  Everything around the core is different from one 
> > vendor to the next.  And when commonality exists it is much easier to 
> > deal with if it is close together.
> 
> Exactly. To make matters worse, we are starting to see a number of vendors
> that use multiple CPU architectures with the same I/O devices (e.g. Renesas,
> Freescale, Xilinx, TI, ...). Not sure if any of these use the same cpufreq
> register on more than one architecture, but it's quite likely to happen
> at some point.

Indeed.  So in my opinion it makes sense to move code into the drivers
directory, at least the code that's going to be used by multiple platforms
(that need not be a complete driver).

We're doing the same thing with the runtime PM code at the moment.

Thanks,
Rafael

Status of arch/arm in linux-next

[Rafael J. Wysocki]

On Tuesday, April 26, 2011, Arnd Bergmann wrote:
> On Thursday 21 April 2011, Nicolas Pitre wrote:
> > > if there's commonality between some of the ARM arch drivers, why can't
> > > there be a arch/arm/cpufreq/ dir for the shared code, and do everything there ?
> > 
> > Because usually there isn't.  "ARM" is just a CPU architecture, not a 
> > system architecture.  Everything around the core is different from one 
> > vendor to the next.  And when commonality exists it is much easier to 
> > deal with if it is close together.
> 
> Exactly. To make matters worse, we are starting to see a number of vendors
> that use multiple CPU architectures with the same I/O devices (e.g. Renesas,
> Freescale, Xilinx, TI, ...). Not sure if any of these use the same cpufreq
> register on more than one architecture, but it's quite likely to happen
> at some point.

Indeed.  So in my opinion it makes sense to move code into the drivers
directory, at least the code that's going to be used by multiple platforms
(that need not be a complete driver).

We're doing the same thing with the runtime PM code at the moment.

Thanks,
Rafael

Re: Status of arch/arm in linux-next

[Dave Jones]

On Tue, Apr 26, 2011 at 07:04:45PM +0200, Rafael J. Wysocki wrote:
 > On Tuesday, April 26, 2011, Arnd Bergmann wrote:
 > > On Thursday 21 April 2011, Nicolas Pitre wrote:
 > > > > if there's commonality between some of the ARM arch drivers, why can't
 > > > > there be a arch/arm/cpufreq/ dir for the shared code, and do everything there ?
 > > > 
 > > > Because usually there isn't.  "ARM" is just a CPU architecture, not a 
 > > > system architecture.  Everything around the core is different from one 
 > > > vendor to the next.  And when commonality exists it is much easier to 
 > > > deal with if it is close together.
 > > 
 > > Exactly. To make matters worse, we are starting to see a number of vendors
 > > that use multiple CPU architectures with the same I/O devices (e.g. Renesas,
 > > Freescale, Xilinx, TI, ...). Not sure if any of these use the same cpufreq
 > > register on more than one architecture, but it's quite likely to happen
 > > at some point.
 > 
 > Indeed.  So in my opinion it makes sense to move code into the drivers
 > directory, at least the code that's going to be used by multiple platforms
 > (that need not be a complete driver).

Ok, so my opinion on this has changed a little over the weekend.
I don't totally hate it now, but I'm still not a huge fan.
That said, I won't stand in the way if this is what everyone agrees is
the way forward.

in cpufreq.next I moved the x86 drivers over.  Someone look it over ?
If that looks like what you all had in mind, start sending me the patches
for other arches, and I'll get them queued up for .40

	Dave

Status of arch/arm in linux-next

[Dave Jones]

On Tue, Apr 26, 2011 at 07:04:45PM +0200, Rafael J. Wysocki wrote:
 > On Tuesday, April 26, 2011, Arnd Bergmann wrote:
 > > On Thursday 21 April 2011, Nicolas Pitre wrote:
 > > > > if there's commonality between some of the ARM arch drivers, why can't
 > > > > there be a arch/arm/cpufreq/ dir for the shared code, and do everything there ?
 > > > 
 > > > Because usually there isn't.  "ARM" is just a CPU architecture, not a 
 > > > system architecture.  Everything around the core is different from one 
 > > > vendor to the next.  And when commonality exists it is much easier to 
 > > > deal with if it is close together.
 > > 
 > > Exactly. To make matters worse, we are starting to see a number of vendors
 > > that use multiple CPU architectures with the same I/O devices (e.g. Renesas,
 > > Freescale, Xilinx, TI, ...). Not sure if any of these use the same cpufreq
 > > register on more than one architecture, but it's quite likely to happen
 > > at some point.
 > 
 > Indeed.  So in my opinion it makes sense to move code into the drivers
 > directory, at least the code that's going to be used by multiple platforms
 > (that need not be a complete driver).

Ok, so my opinion on this has changed a little over the weekend.
I don't totally hate it now, but I'm still not a huge fan.
That said, I won't stand in the way if this is what everyone agrees is
the way forward.

in cpufreq.next I moved the x86 drivers over.  Someone look it over ?
If that looks like what you all had in mind, start sending me the patches
for other arches, and I'll get them queued up for .40

	Dave

Re: Status of arch/arm in linux-next

[Dave Jones]

On Tue, Apr 26, 2011 at 02:15:08PM -0400, Dave Jones wrote:

 >  > Indeed.  So in my opinion it makes sense to move code into the drivers
 >  > directory, at least the code that's going to be used by multiple platforms
 >  > (that need not be a complete driver).
 > 
 > Ok, so my opinion on this has changed a little over the weekend.
 > I don't totally hate it now, but I'm still not a huge fan.
 > That said, I won't stand in the way if this is what everyone agrees is
 > the way forward.
 > 
 > in cpufreq.next I moved the x86 drivers over.  Someone look it over ?
 > If that looks like what you all had in mind, start sending me the patches
 > for other arches, and I'll get them queued up for .40

FYI, this is now on the 'move-drivers' branch in cpufreq.git
Unless there's a good reason not to, I'm going to start pushing this to Linus
next merge window.

	Dave

Status of arch/arm in linux-next

[Dave Jones]

On Tue, Apr 26, 2011 at 02:15:08PM -0400, Dave Jones wrote:

 >  > Indeed.  So in my opinion it makes sense to move code into the drivers
 >  > directory, at least the code that's going to be used by multiple platforms
 >  > (that need not be a complete driver).
 > 
 > Ok, so my opinion on this has changed a little over the weekend.
 > I don't totally hate it now, but I'm still not a huge fan.
 > That said, I won't stand in the way if this is what everyone agrees is
 > the way forward.
 > 
 > in cpufreq.next I moved the x86 drivers over.  Someone look it over ?
 > If that looks like what you all had in mind, start sending me the patches
 > for other arches, and I'll get them queued up for .40

FYI, this is now on the 'move-drivers' branch in cpufreq.git
Unless there's a good reason not to, I'm going to start pushing this to Linus
next merge window.

	Dave

Re: Status of arch/arm in linux-next

[Nicolas Pitre]

On Fri, 29 Apr 2011, Dave Jones wrote:

> On Tue, Apr 26, 2011 at 02:15:08PM -0400, Dave Jones wrote:
> 
>  >  > Indeed.  So in my opinion it makes sense to move code into the drivers
>  >  > directory, at least the code that's going to be used by multiple platforms
>  >  > (that need not be a complete driver).
>  > 
>  > Ok, so my opinion on this has changed a little over the weekend.
>  > I don't totally hate it now, but I'm still not a huge fan.
>  > That said, I won't stand in the way if this is what everyone agrees is
>  > the way forward.
>  > 
>  > in cpufreq.next I moved the x86 drivers over.  Someone look it over ?
>  > If that looks like what you all had in mind, start sending me the patches
>  > for other arches, and I'll get them queued up for .40
> 
> FYI, this is now on the 'move-drivers' branch in cpufreq.git
> Unless there's a good reason not to, I'm going to start pushing this to Linus
> next merge window.

Thanks.  Additional drivers should follow suit.


Nicolas

Status of arch/arm in linux-next

[Nicolas Pitre]

On Fri, 29 Apr 2011, Dave Jones wrote:

> On Tue, Apr 26, 2011 at 02:15:08PM -0400, Dave Jones wrote:
> 
>  >  > Indeed.  So in my opinion it makes sense to move code into the drivers
>  >  > directory, at least the code that's going to be used by multiple platforms
>  >  > (that need not be a complete driver).
>  > 
>  > Ok, so my opinion on this has changed a little over the weekend.
>  > I don't totally hate it now, but I'm still not a huge fan.
>  > That said, I won't stand in the way if this is what everyone agrees is
>  > the way forward.
>  > 
>  > in cpufreq.next I moved the x86 drivers over.  Someone look it over ?
>  > If that looks like what you all had in mind, start sending me the patches
>  > for other arches, and I'll get them queued up for .40
> 
> FYI, this is now on the 'move-drivers' branch in cpufreq.git
> Unless there's a good reason not to, I'm going to start pushing this to Linus
> next merge window.

Thanks.  Additional drivers should follow suit.


Nicolas

[BUG?] Moving drivers to drivers/cpufreq/ causes all to be loaded

[Jonathan Nieder]

(-cc: linux-arm-kernel, +cc: linux-kernel, Mattia)
Hi,

Dave Jones wrote:

> in cpufreq.next I moved the x86 drivers over.  Someone look it over ?

Some people[1] have been seeing regressions after this change (when
moving from 2.6.39 to 3.0, presumably from commit bb0a56ecc4ba,
"[CPUFREQ] Move x86 drivers to drivers/cpufreq/").  The first symptom
was messages at boot:

| Loading cpufreq kernel modules...FATAL: Error inserting powernow_k7 (/lib/modules/3.0.0-1-486/kernel/drivers/cpufreq/powernow-k7.ko): No such device
| FATAL: Error inserting speedstep_ich (/lib/modules/3.0.0-1-486/kernel/drivers/cpufreq/speedstep-ich.ko): No such device
| FATAL: Error inserting acpi_cpufreq (/lib/modules/3.0.0-1-486/kernel/drivers/cpufreq/acpi-cpufreq.ko): Device or resource busy
[etc]

The second symptom was the wrong cpufreq driver being loaded
(p4-clockmod instead of acpi-cpufreq).  The cause seems to be some
code in init scripts that originated in powernowd 0.97-2ubuntu1 (2007)
or some time before that:

|         #get list of available modules (governors and helpers)
|         LOC="/lib/modules/$(uname -r)/kernel/drivers/cpufreq"
|         if [ -d $LOC ]; then
|           MODAVAIL=$( ( find $LOC -type f -name "*.o" -printf "basename %f .o\n"; \
|               find $LOC -type f -name "*.ko" -printf "basename %f .ko\n" ) | /bin/sh)
|         else
|           MODAVAIL=""
|         fi
| 
|         #echo "Loading cpufreq modules:"
|         for mod in $MODAVAIL; do
|         #        echo "     $mod"
|                 echo $LIST| grep -q -w "$mod" || modprobe $mod >/dev/null || /bin/true
|         done

This takes all kernel modules in drivers/cpufreq, blindly assumes
they must be governors or helpers, and loads them.  Nowadays it is
in the loadcpufreq script in cpufrequtils; so in cpufrequtils 007-2
(03 Aug 2011), the pattern changed to drivers/cpufreq/cpufreq_*.ko
which just matches the governors and helpers and everybody's happy.

Except:

 (1) This is still incredibly fragile.  What *should* cpufrequtils
     be doing to get the drivers it needs?

 (2) Using the 3.0 or later kernel with old userspace gives bad
     results (e.g., 30% increase in power consumption for one
     reporter).  That's a regression.  Bad kernel, no biscuit.

Ideas?

[1] http://bugs.debian.org/635348

Re: [BUG?] Moving drivers to drivers/cpufreq/ causes all to be loaded

[Jonathan Nieder]

Jonathan Nieder wrote:

>  (1) This is still incredibly fragile.  What *should* cpufrequtils
>      be doing to get the modules it needs?
>
>  (2) Using the 3.0 or later kernel with old userspace gives bad
>      results (e.g., 30% increase in power consumption for one
>      reporter).  That's a regression.

The "30% increase" part was an unrelated bug (i915.i915_enable_rc6=1
brings power consumption back to normal), for those who were
wondering. :)

Old userspace automatically loading the wrong cpufreq drivers still
does not seem great to me, though I don't have any great ideas about
how to prevent that (a separate drivers/cpufreq-drivers/ directory
does not sound too appealing).  I guess I'd be most interested in how
to fix (1) first.

Thanks,
Jonathan

Re: [BUG?] Moving drivers to drivers/cpufreq/ causes all to be loaded

[Dave Jones]

On Sat, Aug 13, 2011 at 02:02:46PM -0500, Jonathan Nieder wrote:
 > Jonathan Nieder wrote:
 > 
 > >  (1) This is still incredibly fragile.  What *should* cpufrequtils
 > >      be doing to get the modules it needs?
 > >
 > >  (2) Using the 3.0 or later kernel with old userspace gives bad
 > >      results (e.g., 30% increase in power consumption for one
 > >      reporter).  That's a regression.
 > 
 > The "30% increase" part was an unrelated bug (i915.i915_enable_rc6=1
 > brings power consumption back to normal), for those who were
 > wondering. :)
 > 
 > Old userspace automatically loading the wrong cpufreq drivers still
 > does not seem great to me, though I don't have any great ideas about
 > how to prevent that (a separate drivers/cpufreq-drivers/ directory
 > does not sound too appealing).  I guess I'd be most interested in how
 > to fix (1) first.

If we have to move stuff again, we could do drivers/cpufreq/x86/ etc..
Even if we do that though, you really want to fix that userspace, because
you're right that "load everything and see what sticks" is fragile,
and pure luck that it ever did the right thing.

What we used to do in Fedora grew more and more complex over time.
Here's the last incarnation: http://fpaste.org/uvDb/
As you can see in the start() function, it's pretty hairy, and even
that doesn't cover every possible case, which is why it allows a user
override in the $DRIVER variable. Messy.

Basic thinking is
- If AMD64, load powernow-k8
- if that fails to load : try acpi-cpufreq
- For everything else : acpi-cpufreq
- If acpi-cpufreq fails : p4-clockmod

What we're moving towards for Fedora 16 is to change all the drivers
to be linked in rather than modular, and rely on link order to do the right thing.
It sounds like a step backwards in some ways, towards the 'see what sticks'
approach, but the difference here is the link-order specifying the correct
order for initialisation. You get none of that with modules.

In an ideal world, we'd auto-load the right module on a hotplug event
from a cpu, but we're not there yet. I believe Kay is working on that.

Over time, 'the right driver' seems to be converging on acpi-cpufreq.
There's some patches pending to even move some of the powernow-k8 use-cases
to use acpi-cpufreq instead. But modern intel should be using that,
(where modern = almost everything since p4, except those that lack P-states)


And then there's the 'which governor' mess.
I'd really like that to eventually converge so that 'ondemand' is always the
right answer.  For this to happen, there needs to be no difference between
an idle machine running powersave, and ondemand or (harder) a busy machine
running performance.
(conservative with some work could be just a runtime mode for ondemand).

	Dave

Re: [BUG?] Moving drivers to drivers/cpufreq/ causes all to be loaded

[Mattia Dongili]

On Sat, Aug 13, 2011 at 05:11:42PM -0400, Dave Jones wrote:
> On Sat, Aug 13, 2011 at 02:02:46PM -0500, Jonathan Nieder wrote:
>  > Jonathan Nieder wrote:
>  > 
>  > >  (1) This is still incredibly fragile.  What *should* cpufrequtils
>  > >      be doing to get the modules it needs?
>  > >
>  > >  (2) Using the 3.0 or later kernel with old userspace gives bad
>  > >      results (e.g., 30% increase in power consumption for one
>  > >      reporter).  That's a regression.
>  > 
>  > The "30% increase" part was an unrelated bug (i915.i915_enable_rc6=1
>  > brings power consumption back to normal), for those who were
>  > wondering. :)
>  > 
>  > Old userspace automatically loading the wrong cpufreq drivers still
>  > does not seem great to me, though I don't have any great ideas about
>  > how to prevent that (a separate drivers/cpufreq-drivers/ directory
>  > does not sound too appealing).  I guess I'd be most interested in how
>  > to fix (1) first.
> 
> If we have to move stuff again, we could do drivers/cpufreq/x86/ etc..
> Even if we do that though, you really want to fix that userspace, because
> you're right that "load everything and see what sticks" is fragile,
> and pure luck that it ever did the right thing.

not sure why this bug landed here finally, it was clearly an
overlook in the Debian startup script and it's only specific to Debian.

The "load everything" part was not for cpu drivers but for governors
and helpers that used to sit into drivers/cpufreq alone.
The cpu driver loading part is fairly complex (or yes, messy as you say)
and not too dissimilar than the one from fedora.

-- 
mattia
:wq!

Re: [BUG?] Moving drivers to drivers/cpufreq/ causes all to be loaded

[Mattia Dongili]

On Sat, Aug 13, 2011 at 05:11:42PM -0400, Dave Jones wrote:
> On Sat, Aug 13, 2011 at 02:02:46PM -0500, Jonathan Nieder wrote:
>  > Jonathan Nieder wrote:
>  > 
>  > >  (1) This is still incredibly fragile.  What *should* cpufrequtils
>  > >      be doing to get the modules it needs?
>  > >
>  > >  (2) Using the 3.0 or later kernel with old userspace gives bad
>  > >      results (e.g., 30% increase in power consumption for one
>  > >      reporter).  That's a regression.
>  > 
>  > The "30% increase" part was an unrelated bug (i915.i915_enable_rc6=1
>  > brings power consumption back to normal), for those who were
>  > wondering. :)
>  > 
>  > Old userspace automatically loading the wrong cpufreq drivers still
>  > does not seem great to me, though I don't have any great ideas about
>  > how to prevent that (a separate drivers/cpufreq-drivers/ directory
>  > does not sound too appealing).  I guess I'd be most interested in how
>  > to fix (1) first.
> 
> If we have to move stuff again, we could do drivers/cpufreq/x86/ etc..
> Even if we do that though, you really want to fix that userspace, because
> you're right that "load everything and see what sticks" is fragile,
> and pure luck that it ever did the right thing.

not sure why this bug landed here finally, it was clearly an
overlook in the Debian startup script and it's only specific to Debian.

The "load everything" part was not for cpu drivers but for governors
and helpers that used to sit into drivers/cpufreq alone.
The cpu driver loading part is fairly complex (or yes, messy as you say)
and not too dissimilar than the one from fedora.

-- 
mattia
:wq!

Re: [BUG?] Moving drivers to drivers/cpufreq/ causes all to be loaded

[Jonathan Nieder]

(-cc: ARM/etc people; +cc: Kay)

Dave Jones wrote:

> If we have to move stuff again, we could do drivers/cpufreq/x86/ etc..

Unfortunately the old script used "find", not "ls", so that wouldn't
work. :/

> What we used to do in Fedora grew more and more complex over time.
> Here's the last incarnation: http://fpaste.org/uvDb/

The main difference from Debian seems to be that this script loads the
module corresponding to the chosen governor, while Debian's init
script loads all governor modules early (using a heuristic I would
like to avoid that involves running "find" to list them) and chooses
the governor to use later.

> In an ideal world, we'd auto-load the right module on a hotplug event
> from a cpu, but we're not there yet. I believe Kay is working on that.

Yes, that is what I was hoping for.  Are there patches to test?

The comment at
http://thread.gmane.org/gmane.linux.kernel/796450/focus=796874 also
looks promising.

Thanks much for the help,
Jonathan

Re: [BUG?] Moving drivers to drivers/cpufreq/ causes all to be loaded

[Kay Sievers]

On Sun, Aug 14, 2011 at 19:01, Jonathan Nieder <jrnieder@gmail.com> wrote:
> (-cc: ARM/etc people; +cc: Kay)
>
> Dave Jones wrote:
>
>> If we have to move stuff again, we could do drivers/cpufreq/x86/ etc..
>
> Unfortunately the old script used "find", not "ls", so that wouldn't
> work. :/
>
>> What we used to do in Fedora grew more and more complex over time.
>> Here's the last incarnation: http://fpaste.org/uvDb/
>
> The main difference from Debian seems to be that this script loads the
> module corresponding to the chosen governor, while Debian's init
> script loads all governor modules early (using a heuristic I would
> like to avoid that involves running "find" to list them) and chooses
> the governor to use later.

Right, loading cpufreq drivers from userspace is fragile, and can not
properly work today. Only the kernel itself know which driver to try
to bind in which order. Modular cpufreq kernel modules make no real
sense here with the infrastructure we have available today.

>> In an ideal world, we'd auto-load the right module on a hotplug event
>> from a cpu, but we're not there yet. I believe Kay is working on that.
>
> Yes, that is what I was hoping for.  Are there patches to test?

Yeah, it's on the TODO list, I just get too many 'really broken'
things get sorted on top all time. :) But it will happen in the next
months.

> The comment at
> http://thread.gmane.org/gmane.linux.kernel/796450/focus=796874 also
> looks promising.

That could work, but we better go right for the CPU specific aliases
and do not try to load all of them, even when for completely different
systems. That would just encode another workaround into kernel
modules, which we better solve properly.

Kay

Re: [BUG?] Moving drivers to drivers/cpufreq/ causes all to be loaded

[Kay Sievers]

On Sun, Aug 14, 2011 at 19:01, Jonathan Nieder <jrnieder@gmail.com> wrote:
> (-cc: ARM/etc people; +cc: Kay)
>
> Dave Jones wrote:
>
>> If we have to move stuff again, we could do drivers/cpufreq/x86/ etc..
>
> Unfortunately the old script used "find", not "ls", so that wouldn't
> work. :/
>
>> What we used to do in Fedora grew more and more complex over time.
>> Here's the last incarnation: http://fpaste.org/uvDb/
>
> The main difference from Debian seems to be that this script loads the
> module corresponding to the chosen governor, while Debian's init
> script loads all governor modules early (using a heuristic I would
> like to avoid that involves running "find" to list them) and chooses
> the governor to use later.

Right, loading cpufreq drivers from userspace is fragile, and can not
properly work today. Only the kernel itself know which driver to try
to bind in which order. Modular cpufreq kernel modules make no real
sense here with the infrastructure we have available today.

>> In an ideal world, we'd auto-load the right module on a hotplug event
>> from a cpu, but we're not there yet. I believe Kay is working on that.
>
> Yes, that is what I was hoping for.  Are there patches to test?

Yeah, it's on the TODO list, I just get too many 'really broken'
things get sorted on top all time. :) But it will happen in the next
months.

> The comment at
> http://thread.gmane.org/gmane.linux.kernel/796450/focus=796874 also
> looks promising.

That could work, but we better go right for the CPU specific aliases
and do not try to load all of them, even when for completely different
systems. That would just encode another workaround into kernel
modules, which we better solve properly.

Kay

Re: Status of arch/arm in linux-next

[Jamie Lokier]

Arnd Bergmann wrote:
> On Thursday 21 April 2011, Nicolas Pitre wrote:
> > > if there's commonality between some of the ARM arch drivers, why can't
> > > there be a arch/arm/cpufreq/ dir for the shared code, and do everything there ?
> > 
> > Because usually there isn't.  "ARM" is just a CPU architecture, not a 
> > system architecture.  Everything around the core is different from one 
> > vendor to the next.  And when commonality exists it is much easier to 
> > deal with if it is close together.
> 
> Exactly. To make matters worse, we are starting to see a number of vendors
> that use multiple CPU architectures with the same I/O devices (e.g. Renesas,
> Freescale, Xilinx, TI, ...). Not sure if any of these use the same cpufreq
> register on more than one architecture, but it's quite likely to happen
> at some point.

Can't comment on in-tree SoCs, but out of tree (they use Linux but
don't submit anything upstream as far as I can tell), Sigma Designs
use ARM & MIPS CPU architectures, with the clock/timing registers, irq
registers and more or less everything else being the same among them.

-- Jamie

Status of arch/arm in linux-next

[Jamie Lokier]

Arnd Bergmann wrote:
> On Thursday 21 April 2011, Nicolas Pitre wrote:
> > > if there's commonality between some of the ARM arch drivers, why can't
> > > there be a arch/arm/cpufreq/ dir for the shared code, and do everything there ?
> > 
> > Because usually there isn't.  "ARM" is just a CPU architecture, not a 
> > system architecture.  Everything around the core is different from one 
> > vendor to the next.  And when commonality exists it is much easier to 
> > deal with if it is close together.
> 
> Exactly. To make matters worse, we are starting to see a number of vendors
> that use multiple CPU architectures with the same I/O devices (e.g. Renesas,
> Freescale, Xilinx, TI, ...). Not sure if any of these use the same cpufreq
> register on more than one architecture, but it's quite likely to happen
> at some point.

Can't comment on in-tree SoCs, but out of tree (they use Linux but
don't submit anything upstream as far as I can tell), Sigma Designs
use ARM & MIPS CPU architectures, with the clock/timing registers, irq
registers and more or less everything else being the same among them.

-- Jamie

Re: Status of arch/arm in linux-next

[Dave Jones]

On Tue, Apr 19, 2011 at 06:01:19PM +0200, Arnd Bergmann wrote:
 > > Thinking of it, is it OK to put chip CPUfreq drivers into
 > > drivers/cpufreq/* instead of into the arch/arm/* platform
 > > code as everyone does right now? We could probably
 > > fix that and bring down the diffstat considerably.
 > 
 > That's something to discuss with Dave Jones and other people
 > interested in cpufre. Right now, all cpufreq drivers, including
 > those for other architectures are in arch/.
 > 
 > I think it would be good to have the out of the individual
 > platforms, in particular in order to get better reviews of
 > new cpufreq drivers by people that are interested in them.

The platform drivers are by their nature architecture specific,
so arch/ seems apropos.  drivers/platform/arm/ maybe ?

Though, having arm do something different to every other arch seems
a bit awkward too. Everyone else has their cpufreq platform driver
somewhere under arch/whatever/../cpufreq/..  so changing that
violates the principle of least surprise.

I'm also not convinced that moving them would increase review of changes.

What problem is this solving again ?

	Dave

Status of arch/arm in linux-next

[Dave Jones]

On Tue, Apr 19, 2011 at 06:01:19PM +0200, Arnd Bergmann wrote:
 > > Thinking of it, is it OK to put chip CPUfreq drivers into
 > > drivers/cpufreq/* instead of into the arch/arm/* platform
 > > code as everyone does right now? We could probably
 > > fix that and bring down the diffstat considerably.
 > 
 > That's something to discuss with Dave Jones and other people
 > interested in cpufre. Right now, all cpufreq drivers, including
 > those for other architectures are in arch/.
 > 
 > I think it would be good to have the out of the individual
 > platforms, in particular in order to get better reviews of
 > new cpufreq drivers by people that are interested in them.

The platform drivers are by their nature architecture specific,
so arch/ seems apropos.  drivers/platform/arm/ maybe ?

Though, having arm do something different to every other arch seems
a bit awkward too. Everyone else has their cpufreq platform driver
somewhere under arch/whatever/../cpufreq/..  so changing that
violates the principle of least surprise.

I'm also not convinced that moving them would increase review of changes.

What problem is this solving again ?

	Dave

Re: Status of arch/arm in linux-next

[Arnd Bergmann]

On Tuesday 19 April 2011 18:27:42 Dave Jones wrote:
> On Tue, Apr 19, 2011 at 06:01:19PM +0200, Arnd Bergmann wrote:
>  > > Thinking of it, is it OK to put chip CPUfreq drivers into
>  > > drivers/cpufreq/* instead of into the arch/arm/* platform
>  > > code as everyone does right now? We could probably
>  > > fix that and bring down the diffstat considerably.
>  > 
>  > That's something to discuss with Dave Jones and other people
>  > interested in cpufre. Right now, all cpufreq drivers, including
>  > those for other architectures are in arch/.
>  > 
>  > I think it would be good to have the out of the individual
>  > platforms, in particular in order to get better reviews of
>  > new cpufreq drivers by people that are interested in them.
> 
> The platform drivers are by their nature architecture specific,
> so arch/ seems apropos.  drivers/platform/arm/ maybe ?
> 
> Though, having arm do something different to every other arch seems
> a bit awkward too. Everyone else has their cpufreq platform driver
> somewhere under arch/whatever/../cpufreq/..  so changing that
> violates the principle of least surprise.
> 
> I'm also not convinced that moving them would increase review of changes.
> 
> What problem is this solving again ?

Right now, every subarchitecture in arm implements a number of 
drivers (irq, clocksource, gpio, pci, iommu, cpufreq, ...).
These drivers are frequently copies of other existing ones with
slight modifications or (worse) actually are written independently
for the same IP blocks. In some cases, they are copies of drivers
for stuff that is present in other architectures.

We are discussing on a number of fronts to turn the source code
layout from subarchitecture-centric to subsystem-centric, e.g.
put all gpio drivers into one directory instead of each one
into the directory for its (sub)architecture, in order to improve
the abstractions and code reuse we have between the different
implementations.

	Arnd

Status of arch/arm in linux-next

[Arnd Bergmann]

On Tuesday 19 April 2011 18:27:42 Dave Jones wrote:
> On Tue, Apr 19, 2011 at 06:01:19PM +0200, Arnd Bergmann wrote:
>  > > Thinking of it, is it OK to put chip CPUfreq drivers into
>  > > drivers/cpufreq/* instead of into the arch/arm/* platform
>  > > code as everyone does right now? We could probably
>  > > fix that and bring down the diffstat considerably.
>  > 
>  > That's something to discuss with Dave Jones and other people
>  > interested in cpufre. Right now, all cpufreq drivers, including
>  > those for other architectures are in arch/.
>  > 
>  > I think it would be good to have the out of the individual
>  > platforms, in particular in order to get better reviews of
>  > new cpufreq drivers by people that are interested in them.
> 
> The platform drivers are by their nature architecture specific,
> so arch/ seems apropos.  drivers/platform/arm/ maybe ?
> 
> Though, having arm do something different to every other arch seems
> a bit awkward too. Everyone else has their cpufreq platform driver
> somewhere under arch/whatever/../cpufreq/..  so changing that
> violates the principle of least surprise.
> 
> I'm also not convinced that moving them would increase review of changes.
> 
> What problem is this solving again ?

Right now, every subarchitecture in arm implements a number of 
drivers (irq, clocksource, gpio, pci, iommu, cpufreq, ...).
These drivers are frequently copies of other existing ones with
slight modifications or (worse) actually are written independently
for the same IP blocks. In some cases, they are copies of drivers
for stuff that is present in other architectures.

We are discussing on a number of fronts to turn the source code
layout from subarchitecture-centric to subsystem-centric, e.g.
put all gpio drivers into one directory instead of each one
into the directory for its (sub)architecture, in order to improve
the abstractions and code reuse we have between the different
implementations.

	Arnd

Re: Status of arch/arm in linux-next

[Linus Walleij]

2011/4/19 Dave Jones <davej@redhat.com>:

> The platform drivers are by their nature architecture specific,
> so arch/ seems apropos.  drivers/platform/arm/ maybe ?

I opted for putting stuff in there, it was not popular, it will probably
just cause overpopulation there instead, like drivers/misc is doing
right now :-(

> Though, having arm do something different to every other arch seems
> a bit awkward too. Everyone else has their cpufreq platform driver
> somewhere under arch/whatever/../cpufreq/..  so changing that
> violates the principle of least surprise.
>
> I'm also not convinced that moving them would increase review of changes.
>
> What problem is this solving again ?

Recent complaints from Linus (the other one) about overpopulation
bad code reuse and patch collision churn in the arch/arm/* tree.

If all cpufreq drivers (including the x86 ones!) were under
drivers/cpufreq/* it would mean better review and more
opportunity for consolidation I guess? We could begin the move
with a few ARM architectures. Do you agree?

Yours,
Linus Walleij

Status of arch/arm in linux-next

[Linus Walleij]

2011/4/19 Dave Jones <davej@redhat.com>:

> The platform drivers are by their nature architecture specific,
> so arch/ seems apropos. ?drivers/platform/arm/ maybe ?

I opted for putting stuff in there, it was not popular, it will probably
just cause overpopulation there instead, like drivers/misc is doing
right now :-(

> Though, having arm do something different to every other arch seems
> a bit awkward too. Everyone else has their cpufreq platform driver
> somewhere under arch/whatever/../cpufreq/.. ?so changing that
> violates the principle of least surprise.
>
> I'm also not convinced that moving them would increase review of changes.
>
> What problem is this solving again ?

Recent complaints from Linus (the other one) about overpopulation
bad code reuse and patch collision churn in the arch/arm/* tree.

If all cpufreq drivers (including the x86 ones!) were under
drivers/cpufreq/* it would mean better review and more
opportunity for consolidation I guess? We could begin the move
with a few ARM architectures. Do you agree?

Yours,
Linus Walleij

Status of arch/arm in linux-next

[Linus Walleij]

2011/4/19 Linus Walleij <linus.walleij@linaro.org>:

> The problem is that it is dependent on the PRCMU driver which
> provides the communication mechanism to actually control these
> regulators.
>
> The PRCMU is the Power Reset and Control Management Unit,
> it is a register pages where you send commands to a firmware
> running on its own CPU on the other side, partly using mailboxes.
> The firmware handles things like voltage and power domains
> (modeled as regulators), frequency changes (using CPUfreq),
> idle states (CPUidle and sleep, idling), as well as resetting
> particular memory blocks AND an I2C channel to the AB8500
> chip (driven from drivers/ab8500-core.c indeed) and some
> GPIO configuration.

Reading what I just wrote makes me think this beast may belong
in drivers/mfd.

Samuel, would you say I can push this driver to MFD?

Most MFD things are, like I2C chips and stuff but this one sure
match the title "multifunctional device", just that it's very singleton
and very close to the SoC core.

Yours,
Linus Walleij

Status of arch/arm in linux-next

[Arnd Bergmann]

On Thursday 21 April 2011, Linus Walleij wrote:
> > The PRCMU is the Power Reset and Control Management Unit,
> > it is a register pages where you send commands to a firmware
> > running on its own CPU on the other side, partly using mailboxes.
> > The firmware handles things like voltage and power domains
> > (modeled as regulators), frequency changes (using CPUfreq),
> > idle states (CPUidle and sleep, idling), as well as resetting
> > particular memory blocks AND an I2C channel to the AB8500
> > chip (driven from drivers/ab8500-core.c indeed) and some
> > GPIO configuration.
> 
> Reading what I just wrote makes me think this beast may belong
> in drivers/mfd.
> 
> Samuel, would you say I can push this driver to MFD?
> 
> Most MFD things are, like I2C chips and stuff but this one sure
> match the title "multifunctional device", just that it's very singleton
> and very close to the SoC core.

One property of MFD devices is that they register a fixed set of
child devices (cells) that are each providing separate functionality
and have their own drivers. If that's not the case with prcmu,
I think it should not be an MFD driver.

	Arnd

Status of arch/arm in linux-next

[Linus Walleij]

2011/4/21 Arnd Bergmann <arnd@arndb.de>:

>> Most MFD things are, like I2C chips and stuff but this one sure
>> match the title "multifunctional device", just that it's very singleton
>> and very close to the SoC core.
>
> One property of MFD devices is that they register a fixed set of
> child devices (cells) that are each providing separate functionality
> and have their own drivers. If that's not the case with prcmu,
> I think it should not be an MFD driver.

It's both-and. The power domain regulators and cpufreq are
platform devices and will work nicely as MFD cells spawn
off the PRCMU. (Just that nooned did it that way before.)

The big deviating thing is the clock tree, which also use calls to
the PRCMU. I would argue that the problem is rather that clocks
are not modeled as devices rather than that the PRCMU would
be modeled as an MFD device.

Linus Walleij

Status of arch/arm in linux-next

[Linus Walleij]

2011/4/22 Linus Walleij <linus.walleij@linaro.org>:

> The big deviating thing is the clock tree, which also use calls to
> the PRCMU. I would argue that the problem is rather that clocks
> are not modeled as devices rather than that the PRCMU would
> be modeled as an MFD device.

Which sort of begets the question of whether Russell is OK with
pushing clock tree implementations to drivers/clk/*...

I'll be perfectly happy doing that too, so if I:

- Put the PRCMU driver in drivers/mfd/prcmu-db8500.c
- Put the cpufreq driver in drivers/cpufreq/cpufreq-db8500.c
- Put the clock tree in drivers/clk/clock-db8500.c

My diffstat (which was 50% of the ARM +++ in the next tree)
will most certainly instead go to NEGATIVE in arch/arm/* and
be an appetizing pull target.

Needless to say it'll require some ACK:ing. I have mixed feelings
about this but it does depopulate the ARM tree.

Yours,
Linus Walleij

Status of arch/arm in linux-next

[Samuel Ortiz]

Hi Linus,

On Fri, Apr 22, 2011 at 09:56:41AM +0200, Linus Walleij wrote:
> 2011/4/21 Arnd Bergmann <arnd@arndb.de>:
> 
> >> Most MFD things are, like I2C chips and stuff but this one sure
> >> match the title "multifunctional device", just that it's very singleton
> >> and very close to the SoC core.
> >
> > One property of MFD devices is that they register a fixed set of
> > child devices (cells) that are each providing separate functionality
> > and have their own drivers. If that's not the case with prcmu,
> > I think it should not be an MFD driver.
> 
> It's both-and. The power domain regulators and cpufreq are
> platform devices and will work nicely as MFD cells spawn
> off the PRCMU. (Just that nooned did it that way before.)
That looks like a potential candidate for drivers/mfd/. I'd have to look at
the code to take it or not, but it's probably worth trying.

Cheers,
Samuel.

-- 
Intel Open Source Technology Centre
http://oss.intel.com/

Status of arch/arm in linux-next

[Linus Walleij]

2011/5/2 Samuel Ortiz <sameo@linux.intel.com>:
> Hi Linus,
>
> On Fri, Apr 22, 2011 at 09:56:41AM +0200, Linus Walleij wrote:
>> 2011/4/21 Arnd Bergmann <arnd@arndb.de>:
>>
>> >> Most MFD things are, like I2C chips and stuff but this one sure
>> >> match the title "multifunctional device", just that it's very singleton
>> >> and very close to the SoC core.
>> >
>> > One property of MFD devices is that they register a fixed set of
>> > child devices (cells) that are each providing separate functionality
>> > and have their own drivers. If that's not the case with prcmu,
>> > I think it should not be an MFD driver.
>>
>> It's both-and. The power domain regulators and cpufreq are
>> platform devices and will work nicely as MFD cells spawn
>> off the PRCMU. (Just that nooned did it that way before.)
>
> That looks like a potential candidate for drivers/mfd/. I'd have to look at
> the code to take it or not, but it's probably worth trying.

OK I'll hack something up!

Linus Walleij

Status of arch/arm in linux-next

[Tony Lindgren]

* Arnd Bergmann <arnd@arndb.de> [110419 07:13]:
> On Friday 15 April 2011, Tony Lindgren wrote:
> > > drivers/platform/arm/ux500/*
> > > include/linux/platform/arm/ux500/*
> > > 
> > > Are any of these generally speaking good ideas?
> > 
> > Or maybe drivers/arm?
> > 
> > Anyways, whatever can be done as loadable modules should be done
> > that way. That makes the life for distros much easier ;)
> 
> drivers/arm would just become the next pile of crap if we start that,
> like drivers/misc is starting to look now.
> 
> There are more things that I think should become subsystems
> for themselves, with a proper maintainer, rather than bits
> that simply get copied across all platforms.

Sure that sounds good to me.
 
> ACPI does a lot of things (unfortunately), and I did not get the impression
> that prcmu does the one part we really need, which is complete enumeration
> of all devices. If it did that, it should become a bus_type and replace
> the hardcoded sets of platform devices.

Right, for most SoCs there is no such thing as enumeration of devices.
But passing that information in device tree format from the bootloader 
should sort that issue.

For omaps we have the hwmod bus abstraction for platform bus that could
be made generic potentially. Needs to be converted to use DT data first
though..

Tony

Status of arch/arm in linux-next

[Arnd Bergmann]

On Wednesday 20 April 2011 09:33:13 Tony Lindgren wrote:
> For omaps we have the hwmod bus abstraction for platform bus that could
> be made generic potentially. Needs to be converted to use DT data first
> though..

Yes, we had some discussions during ELC about how the data in hwmod can
be put into DT format. I learned that the files are currently
semi-automatically generated from the hardware specifications, and I've
got a good feeling about being able to generate DT fragments in the
same way.

	Arnd

Status of arch/arm in linux-next

[Martin Guy]

On 14 April 2011 11:44, Russell King - ARM Linux <linux@arm.linux.org.uk> wrote:
> This morning, I looked at linux-next to find out how arch/arm is doing
> for the next merge window.
>
> $ git diff -C --cumulative v2.6.39-rc1... arch
> ?19.7% arch/arm/mach-imx/
> ?19.2% arch/arm/mach-mx3/
> ? 3.4% arch/arm/mach-mxc91231/
> ?18.1% arch/arm/mach-ux500/
> ?74.1% arch/arm/
> ? 3.2% arch/m68k/
> ? 4.0% arch/mips/lantiq/
> ? 6.9% arch/mips/
> ? 3.1% arch/x86/kvm/
> ? 7.6% arch/x86/
> ?100.0% arch/

One reason for high ARM activity is that the arm port has far more
different supported computers and drivers for more different hardware
than any other processor, so more activity in the arm tree than any
other is unlikely to go away unless we stop developing for ARM
platforms.

Another is that most of this is recently-produced hardware, so there
are a lot new drivers to develop.

Lastly, ARM platforms are common and tend to be cheap, so they are the
boards that young, enthusiastic developers are likely to have in front
of them and, since it covers many many machines, they are more likely
to find places where suoport is lacking or can be improved.

Counting the arch-specific C files:
for a in arch/* ; do echo -n "$a    "; find $a -name '*.[ch]' | wc -l;
done | sort -nrk 2
arch/arm	3040
arch/mips	1206
arch/powerpc	1018
arch/x86	762
arch/sh	591
arch/sparc	454
...

That's a total of 30% of the files out of the 10000 files under 25 architecture.
I don't think this "problem" is likely to go away.  At best, we can
try to limit the number of "remove trailing spaces from lines" type of
patches.

   M

Status of arch/arm in linux-next

[Russell King - ARM Linux]

On Fri, Apr 15, 2011 at 04:30:50PM +0200, Martin Guy wrote:
> On 14 April 2011 11:44, Russell King - ARM Linux <linux@arm.linux.org.uk> wrote:
> > This morning, I looked at linux-next to find out how arch/arm is doing
> > for the next merge window.
> >
> > $ git diff -C --cumulative v2.6.39-rc1... arch
> > ?19.7% arch/arm/mach-imx/
> > ?19.2% arch/arm/mach-mx3/
> > ? 3.4% arch/arm/mach-mxc91231/
> > ?18.1% arch/arm/mach-ux500/
> > ?74.1% arch/arm/
> > ? 3.2% arch/m68k/
> > ? 4.0% arch/mips/lantiq/
> > ? 6.9% arch/mips/
> > ? 3.1% arch/x86/kvm/
> > ? 7.6% arch/x86/
> > ?100.0% arch/
> 
> One reason for high ARM activity is that the arm port has far more
> different supported computers and drivers for more different hardware
> than any other processor, so more activity in the arm tree than any
> other is unlikely to go away unless we stop developing for ARM
> platforms.

No, if you read Linus' complaints, this argument doesn't apply.  Why?
Because we should have invented some way to sort this stuff out so that
we had data structures passed into the kernel - or a pre-kernel shim -
which sorted out stuff for the kernel.  You'll notice that Linus said
that using ACPI would be better than what we're currently doing with
platform support.

While I don't agree completely with that, as there's platform specific
functions which can't be handled using that method (unless we start
passing bytecode and have an interpreter in the kernel) Linus has a
point when we end up with massive chunks of platform specific data in
the kernel.

Status of arch/arm in linux-next

[Alexey Zaytsev]

Dear, Lunus.

Could you please just apologize for the pointless diffstat complain,
so we could go on?

Dear Russel.

Please don't take the offense. Linus might be a dickhead at times, and
sometimes he's wrong, but I'm sure he did not mean to hurt you.

(And who the fuck is this guy? No one.)

On Thu, Apr 14, 2011 at 13:44, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> This morning, I looked at linux-next to find out how arch/arm is doing
> for the next merge window.
>
> $ git diff -C --cumulative v2.6.39-rc1... arch
> ?19.7% arch/arm/mach-imx/
> ?19.2% arch/arm/mach-mx3/
> ? 3.4% arch/arm/mach-mxc91231/
> ?18.1% arch/arm/mach-ux500/
> ?74.1% arch/arm/
> ? 3.2% arch/m68k/
> ? 4.0% arch/mips/lantiq/
> ? 6.9% arch/mips/
> ? 3.1% arch/x86/kvm/
> ? 7.6% arch/x86/
> ?100.0% arch/
> $ git diff -C --cumulative v2.6.39-rc1... arch/arm
> ? 4.7% arch/arm/mach-imx/
> ? 5.9% arch/arm/mach-msm/
> ? 6.3% arch/arm/mach-mx3/
> ? 8.8% arch/arm/mach-mxc91231/
> ? 7.6% arch/arm/mach-ux500/include/mach/
> ?46.1% arch/arm/mach-ux500/
> ? 3.6% arch/arm/mach-zynq/
> ? 5.9% arch/arm/plat-mxc/include/mach/
> ? 7.3% arch/arm/plat-mxc/
> ?100.0% arch/arm/
> $ git diff -C --shortstat v2.6.39-rc1... arch/arm
> ?450 files changed, 11973 insertions(+), 5736 deletions(-)
>
> Note that with --cumulative, the %age given includes all child directories
> (so the 74% for arch/arm includes everything below it).
>
> So, almost 75% of changes in arch are down to arch/arm. ?For arch/arm,
> there's 6k new lines introduced - so there's a significant amount of new
> work there.
>
> Folk may want to consider that we're three weeks from Linus' rant, and
> there's little sign of the situation changing. ?It looks to me from
> these statistics that it's business as usual.
>
> Please take a moment to consider how Linus will react to this at the
> next merge window.
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>

Status of arch/arm in linux-next

[Linus Torvalds]

On Mon, Apr 18, 2011 at 8:17 AM, Alexey Zaytsev
<alexey.zaytsev@gmail.com> wrote:
>
> Could you please just apologize for the pointless diffstat complain,
> so we could go on?

Ehh. They aren't pointless, and I'm _this_ close to just stopping
pulling from some people unless things improve.

> Dear Russel.
>
> Please don't take the offense. Linus might be a dickhead at times, and
> sometimes he's wrong, but I'm sure he did not mean to hurt you.

Umm. The "some people" who need to get their shit together was never
Russell (and we've been emailing in private about it). We may not
agree about every detail, but on the whole we're not at all butting
heads.

Why do you think he posted that email with those arm statistics?

It's the _machine/platform_ guys who are trouble.

Hint for anybody on the arm list: look at the dirstat that rmk posted,
and if your "arch/arm/{mach,plat}-xyzzy" shows up a lot, it's quite
possible that I won't be pulling your tree unless the reason it shows
up a lot is because it has a lot of code removed.

People need to realize that the endless amounts of new pointless
platform code is a problem, and since my only recourse is to say "if
you don't seem to try to make an effort to fix it, I won't pull from
you", that is what I'll eventually be doing.

Exactly when I reach that point, I don't know.

                      Linus

Status of arch/arm in linux-next

[Alexey Zaytsev]

On Mon, Apr 18, 2011 at 20:23, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Mon, Apr 18, 2011 at 8:17 AM, Alexey Zaytsev
> <alexey.zaytsev@gmail.com> wrote:
>>
>> Could you please just apologize for the pointless diffstat complain,
>> so we could go on?
>
> Ehh. They aren't pointless, and I'm _this_ close to just stopping
> pulling from some people unless things improve.
>
>> Dear Russel.
>>
>> Please don't take the offense. Linus might be a dickhead at times, and
>> sometimes he's wrong, but I'm sure he did not mean to hurt you.
>
> Umm. The "some people" who need to get their shit together was never
> Russell (and we've been emailing in private about it). We may not
> agree about every detail, but on the whole we're not at all butting
> heads.
>

Ok, I fail at trolling this time.

But being serious now, what's the plan for the new code?
I might get basic support for the Conexant/Ikanos SolosW SOC by the
2.6.41 merge window. Should I just send it in, or is there some
moratorium in place, before the current code gets sorted out?

Status of arch/arm in linux-next

[Linus Torvalds]

On Mon, Apr 18, 2011 at 2:54 PM, Alexey Zaytsev
<alexey.zaytsev@gmail.com> wrote:
>
> But being serious now, what's the plan for the new code?
> I might get basic support for the Conexant/Ikanos SolosW SOC by the
> 2.6.41 merge window. Should I just send it in, or is there some
> moratorium in place, before the current code gets sorted out?

I don't have any set-in-stone plans. But I _am_ going to push back a
lot more on people who ask me to pull stuff that adds a lot of lines
to random platform drivers. If it all looks like "more of the same
crap" (another new irq driver, another stupid clock driver or pin
listing for gpio), I'll probably just say "screw it, they didn't even
try, why should I pull".

Linaro is trying to set up some kind of platform maintainership, we'll
see how that goes. But it will take some time to flesh out.

I personally try to avoid any "hard" rules. I'm going to use common
sense. But my common sense will have a lot of "we can't continue to
add crazy almost-duplicated code forever" in it.

              Linus

Status of arch/arm in linux-next

[Jean-Christophe PLAGNIOL-VILLARD]

On 08:02 Tue 19 Apr     , Linus Torvalds wrote:
> On Mon, Apr 18, 2011 at 2:54 PM, Alexey Zaytsev
> <alexey.zaytsev@gmail.com> wrote:
> >
> > But being serious now, what's the plan for the new code?
> > I might get basic support for the Conexant/Ikanos SolosW SOC by the
> > 2.6.41 merge window. Should I just send it in, or is there some
> > moratorium in place, before the current code gets sorted out?
> 
> I don't have any set-in-stone plans. But I _am_ going to push back a
> lot more on people who ask me to pull stuff that adds a lot of lines
> to random platform drivers. If it all looks like "more of the same
> crap" (another new irq driver, another stupid clock driver or pin
> listing for gpio), I'll probably just say "screw it, they didn't even
> try, why should I pull".
> 
> Linaro is trying to set up some kind of platform maintainership, we'll
> see how that goes. But it will take some time to flesh out.
> 
> I personally try to avoid any "hard" rules. I'm going to use common
> sense. But my common sense will have a lot of "we can't continue to
> add crazy almost-duplicated code forever" in it.
I'm actually working on AT91 cleanup as switch to clkdev and early devices,
etc...
but I do not remove too much line on contrary I may add more

I'll later remove duplicate ressources and refactor the code to allow multiple
soc in the same kernel but this will result in a LOTS of changsets.

Best Regards,
J.