* Re: [GIT PULL] ARM MSM updates for 2.6.35-rc1
2010-06-03 1:20 ` Linus Torvalds
@ 2010-06-03 3:44 ` Michael Ellerman
2010-06-03 4:26 ` Linus Torvalds
2010-06-03 4:45 ` Ben Dooks
` (2 subsequent siblings)
3 siblings, 1 reply; 15+ messages in thread
From: Michael Ellerman @ 2010-06-03 3:44 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Kevin Hilman, Daniel Walker, linux-kernel, linux-arm-msm
[-- Attachment #1: Type: text/plain, Size: 841 bytes --]
On Wed, 2010-06-02 at 18:20 -0700, Linus Torvalds wrote:
>
> Now, I'm actually considering just getting rid of all the 'defconfig'
> files entirely. The x86 model is sane (there's two of them, nobody likely
> uses them), but ARM and POWERPC (and to a lesser config SH and MIPS) have
> turned the whole concept into a disgusting mess.
I agree that the defconfig mechanism is broken (ie. carrying the full
text in tree), but the concept is sane IMHO.
What'd be nice is if the defconfig could just be a delta against a base
config for the architecture - that would make most of them reasonably
small, just turning on/off a few options.
You can sort of do that today, by just storing a delta, but oldconfig
will silently turn off things you have enabled if prereqs change, so
that doesn't really work I think.
cheers
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [GIT PULL] ARM MSM updates for 2.6.35-rc1
2010-06-03 3:44 ` Michael Ellerman
@ 2010-06-03 4:26 ` Linus Torvalds
2010-06-03 16:11 ` Tony Lindgren
2010-06-04 5:34 ` Eric Miao
0 siblings, 2 replies; 15+ messages in thread
From: Linus Torvalds @ 2010-06-03 4:26 UTC (permalink / raw)
To: Michael Ellerman; +Cc: Kevin Hilman, Daniel Walker, linux-kernel, linux-arm-msm
On Thu, 3 Jun 2010, Michael Ellerman wrote:
>
> You can sort of do that today, by just storing a delta, but oldconfig
> will silently turn off things you have enabled if prereqs change, so
> that doesn't really work I think.
I think you can do it today with various hacks. Up to and including
basically doing something that just selects the options you want.
IOW, you could likely have a human-written Kconfig.<platform> file that
just does
define_bool MYPLATFORM y
select .. everything I need ..
include Kconfig.main
or a number of other tricks.
Ingo and the x86 folks (who I really think have done a very good job, and
there really aren't any crazy defconfig files there) have this "make
randconfig" together with scripted requirements so that you can actually
_boot_ the random config, just because the requirements make sure that the
things needed for booting on the test setup are selected.
I forget exactly what the build setup there is (Ingo described it to me
long time ago, but since I don't even want to have a build farm in my
home, I didn't care much).
But we certainly _can_ do a better job than the 'defconfig' thing that was
really never meant for the kind of use it sees in ARM/POWERPC/SH/MIPS, and
that really isn't appropriate for any manual editing (so people just run
"make oldconfig" with tweaking or something, and then use the newly
generated file).
And I suspect that it really is best to just remove the existing defconfig
files. People can see them in the history to pick up what the heck they
did, but no way will any sane model ever look even _remotely_ like them,
so they really aren't a useful basis for going forward.
But don't worry. It didn't happen this merge window, obviously.
Linus
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [GIT PULL] ARM MSM updates for 2.6.35-rc1
2010-06-03 4:26 ` Linus Torvalds
@ 2010-06-03 16:11 ` Tony Lindgren
2010-06-04 5:34 ` Eric Miao
1 sibling, 0 replies; 15+ messages in thread
From: Tony Lindgren @ 2010-06-03 16:11 UTC (permalink / raw)
To: Linus Torvalds
Cc: Michael Ellerman, Kevin Hilman, Daniel Walker, linux-kernel,
linux-arm-msm
* Linus Torvalds <torvalds@linux-foundation.org> [100603 07:25]:
>
>
> On Thu, 3 Jun 2010, Michael Ellerman wrote:
> >
> > You can sort of do that today, by just storing a delta, but oldconfig
> > will silently turn off things you have enabled if prereqs change, so
> > that doesn't really work I think.
>
> I think you can do it today with various hacks. Up to and including
> basically doing something that just selects the options you want.
>
> IOW, you could likely have a human-written Kconfig.<platform> file that
> just does
>
> define_bool MYPLATFORM y
> select .. everything I need ..
>
> include Kconfig.main
>
> or a number of other tricks.
I agree all the defconfigs are a pain just for the omaps alone.
If this is of any help we could now just keep omap3_defconfig for
arch/arm/mach-omap2 and get rid of 23 config files:
$ egrep "CONFIG_ARCH_OMAP[2|3|4]=y" arch/arm/configs/* | grep -v omap3_defconfig | wc -l
23
It needs some more work for omap2 though to boot to userspace as
there are still some known issues with ARMv6 vs ARMv7 support and
VFP2 vs 3 support. Will try to look at fixing those again when
I have a chance.
Then making the multi-omap thing work on all omap1 boards would
cut down another 15 defconfigs, that should be also doable.
To be able to compile in multiple arm architectures we would have
to get rid of the Makefile.boot files and NR_IRQS and then have
some kind of common clock framework at least.
I did some experiments compiling in both mach-omap1 and mach-omap2
a few years back using ARMv5 flags, there were probably other issues
too like some conflicting defines.
Cheers,
Tony
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [GIT PULL] ARM MSM updates for 2.6.35-rc1
2010-06-03 4:26 ` Linus Torvalds
2010-06-03 16:11 ` Tony Lindgren
@ 2010-06-04 5:34 ` Eric Miao
1 sibling, 0 replies; 15+ messages in thread
From: Eric Miao @ 2010-06-04 5:34 UTC (permalink / raw)
To: Linus Torvalds
Cc: Michael Ellerman, Kevin Hilman, Daniel Walker, linux-kernel,
linux-arm-msm
On 06/03/2010 12:26 PM, Linus Torvalds wrote:
>
>
> On Thu, 3 Jun 2010, Michael Ellerman wrote:
>>
>> You can sort of do that today, by just storing a delta, but oldconfig
>> will silently turn off things you have enabled if prereqs change, so
>> that doesn't really work I think.
>
> I think you can do it today with various hacks. Up to and including
> basically doing something that just selects the options you want.
>
> IOW, you could likely have a human-written Kconfig.<platform> file that
> just does
>
> define_bool MYPLATFORM y
> select .. everything I need ..
>
> include Kconfig.main
>
> or a number of other tricks.
>
> Ingo and the x86 folks (who I really think have done a very good job, and
> there really aren't any crazy defconfig files there) have this "make
> randconfig" together with scripted requirements so that you can actually
> _boot_ the random config, just because the requirements make sure that the
> things needed for booting on the test setup are selected.
>
> I forget exactly what the build setup there is (Ingo described it to me
> long time ago, but since I don't even want to have a build farm in my
> home, I didn't care much).
>
> But we certainly _can_ do a better job than the 'defconfig' thing that was
> really never meant for the kind of use it sees in ARM/POWERPC/SH/MIPS, and
> that really isn't appropriate for any manual editing (so people just run
> "make oldconfig" with tweaking or something, and then use the newly
> generated file).
>
It certainly looks a better way to handle this. However, considering the
facts that there are so many platforms out there, and doing a transition
without breaking any of them is a lot work, it's actually easier to just
reduce the number of defconfig at this moment, provided that most ARM
platforms with the same SoC are able to be built into a single kernel.
There are some exceptions though, I'd suggest not to introduce any other
defconfig for these platforms until their problem is solved.
Russell has setup a thread for this issue in linux-arm-kernel ML, so
hopefully there will be a lot patches around to address it.
There are some specific problems with ARM, e.g. some platforms are really
not maintained for a long time, and even no way to find someone or some
machine to test. And even with one defconfig per SoC, there could still
be about > 60 defconfigs there (compared with 178 at this moment).
> And I suspect that it really is best to just remove the existing defconfig
> files. People can see them in the history to pick up what the heck they
> did, but no way will any sane model ever look even _remotely_ like them,
> so they really aren't a useful basis for going forward.
>
> But don't worry. It didn't happen this merge window, obviously.
>
> Linus
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [GIT PULL] ARM MSM updates for 2.6.35-rc1
2010-06-03 1:20 ` Linus Torvalds
2010-06-03 3:44 ` Michael Ellerman
@ 2010-06-03 4:45 ` Ben Dooks
2010-06-03 5:36 ` Ben Dooks
2010-06-04 8:27 ` Vincent Sanders
3 siblings, 0 replies; 15+ messages in thread
From: Ben Dooks @ 2010-06-03 4:45 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Kevin Hilman, Daniel Walker, linux-kernel, linux-arm-msm
On Wed, Jun 02, 2010 at 06:20:09PM -0700, Linus Torvalds wrote:
>
> am, but I'm willing to do it only if I have a feeling that things are
> under control. And I'm not getting that feeling.
>
> - TWO HUNDRED THOUSAND lines of arch/arm is just pure garbage, namely the
> defconfig files. Quite frankly, anybody who calls that anything but
> pure "noise" is just misguided and being stupid.
>
> So yes, I do consider a lot of it "noise". When there's two hundred
> thousand lines of garbage, and it keeps growing without bounds, something
> needs to be done.
>
> Now, I'm actually considering just getting rid of all the 'defconfig'
> files entirely. The x86 model is sane (there's two of them, nobody likely
> uses them), but ARM and POWERPC (and to a lesser config SH and MIPS) have
> turned the whole concept into a disgusting mess.
>
> But while POWERPC has abused that thing too, in _other_ respects it has
> been much less egregious.
unfortunately there are so many different variants of the ARM
architecture that these defconfigs spring up to ensure that a base
compile for each one of them is performed. We run an autobuild which
runs through all these defconfigs and produces a report of what
happened to allow tracking of build regressions at the moment.
> So I can largely fix the defconfig mess on my own (by just using a simple
> oneliner shell script to deletes them all) but that really only fixes one
> particularly annoying symptom - not the underlying issue. We do need
> somebody to maintain the arm platform mess, and actually tries to get some
> infrastructure or something in place so that it doesn't explode.
Someone should defiently cull the older defconfigs for sepcific
machines, many of which probably don't get built for mainline.
~
> > The fact is that ARM-based devices multiply like rabbits, and there is
> > a huge amount of diversity between the various derivatives. Also,
> > support most of these devices has lived out of tree for a long time.
> > Now that we have a relatively direct path which doesn't require any
> > single person to have to understand all the mind-numbing details of
> > all of these ARM-based platforms, it has allowed us to significantly
> > improve the support for these devices upstream. Anything that is
> > common to all devices still goes through RMK.
>
> The thing is, I bet there is way more commonality still to be taken
> advantage of. And even if there isn't, we need somebody who cares, and
> doesn't just mindlessly aggregate all the crud. Somebody with the taste to
> say "ok, that's just too effin ugly, you need to fix things up"
> occasionally.
yes, there is that problem, and many of the big company players have
yet to really see the necessity in this... It has taken a while now to
convince the necessary people at Samsung that simply copying and
modifying existing driver/support is simply not going to fly.
--
Ben (ben@fluff.org, http://www.fluff.org/)
'a smiley only costs 4 bytes'
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [GIT PULL] ARM MSM updates for 2.6.35-rc1
2010-06-03 1:20 ` Linus Torvalds
2010-06-03 3:44 ` Michael Ellerman
2010-06-03 4:45 ` Ben Dooks
@ 2010-06-03 5:36 ` Ben Dooks
2010-06-04 8:27 ` Vincent Sanders
3 siblings, 0 replies; 15+ messages in thread
From: Ben Dooks @ 2010-06-03 5:36 UTC (permalink / raw)
To: Linux Torvalds; +Cc: linux-kernel
[sorry if this message gets unthreaded, my mbox for linux-kernel got
corrupted so i've had to hand edit the headers]
> - x86 has its drivers elsewhere, and they are _discoverable_ and not
> hardcoded to some platform. They have often also been useful (to say
> the least) to other architecture platforms. That's not always truefor
> all of them (we do have drivers/platform/x86, but at least that's
> maintained separately and is nowhere near the mess that is ARM)
There are some common drivers for ARM specific hardware under
arch/arm/common but there's not a lot of that. I have been trying to
ensure that new support re-uses common driver code. I was one of
the people who worked with Piere to break the SDHCI reliance on PCI
so that it could be re-used on platform device based systems such as
the new Samsung SoC. I have also been trying to ensure that new
support is carefully considered.
Unfortunately we often run into the following problems;
- The "Our hacked driver has been customer-tested and patched so often
that it must be the one that gets submitted" argument... trying to get
them to realised that firstly, it isn't of the necessary quality or
has been already done elsewhere is very difficult.
- Often the hardware teams don't tell you where the blocks that have
been included come from some common source. This makes the task of
identifying a common driver difficult (they just make their own
driver from the 'documentation' that the hardware guys hack).
I've been subject to this on at-least one occasion and currently
considering renaming the s3c-hsotg driver to dw-hsotg (it turns
out to be a block that was bought in from Synopsys' DesignWare)
- There is also a problem that as soon as the hardware that has even
a small divergence from the common case that they need a special
driver for it (against, see the SDHCI case where there's a couple
of extra configuration register to control clocks, etc.)
When Harald Welte and I looked into some of these development
practices we started to try and educate the developers about this. As
an example we looked the clock structure we used to modle the clock
trees in the SoCs, and found that in certain of their branches there
where 4 copies of the same code, each with its own subtle bugs...
I have been trying very hard to make sure that these things are kept
under control, but unfortunately it can be a hard task and until it
hits these big companies in the pocket it very difficult to get them
to actually pay for this.
As for the churn under arch/arm, then there are a few reasons for this
and the size of the code involved.
- Firstly the implemetations of the ARM architecture can vary wildly
depending on the implementor (and there can be several different
implementing teams within one company). There is very little direct
recommendation or specification from ARM on how the system around
the core is layed out.
This means that there are a variety of different interupt controlers
involved, and even when a standard one is used (ARM VIC) then the
interrupt routing and base addresses of the control registers change
between SoCs.
There is also a huge variety of memory layouts, the base of SDRAM
and peripheral layout can change between individual SoCs, let alone
families of them.
- Secondly there has been an rush to get as many header files in
defining as many registers as possible for all the hardware, even
if it turns out later to be a common block or not used.
I have been trying to remove as many #defines in headers where the
result is only used once in the code to try and help the spiraling
size of the project.
- Thirdly the core of these devices is also widly different, there is
often far more complexity in how the SoC's bus and clock layouts are
put together.
It is not like PCI where each device has a xMHZ clock and integrated
PLL if needed. There are complex heirarchies of PLLs and clocks,
which then need to be exported to the drivers to use, and often the
clock gating is also integrated into the centralised clock control
units instead of in the device.
Also, since there is often very little support for any dynamic bus
discovery, a lot of the device information is dependant on knowing
which SoC you are running on (unlike PCI where there's a specific
configuration header).
Note, there are moves to try and get device trees supported on ARM
to try and solve the previous paragraph, but we have an awfully big
history to sort out, and it only really pushes that part of the
problem out of the kernel onto the device manfactueres (who can
barely get a bootloader to pass in a couple of registers).
So apart from that, another source for the churn is that when we
do identify common parts, thigns need to get move and older systems
updated to use them. These devices seem to evolve at such a rate it
is difficult to predict what we'll see 6 months down the line.
The next problem is that if we don't get some support in each merge
window, then the size of the task for the next one keeps growing and
growing. It is already a problem just for the Samsung range of SoCs
where we only really have core functionality for most of the SoCs;
There are still drivers needed for some of the Security, Multimedia
and connectivity subsystems to be sorted out.
I have been trying my best to keep control of these issues, but it is
very difficult and it isn't my daytime job :(
--
Ben (ben@fluff.org, http://www.fluff.org/)
'a smiley only costs 4 bytes'
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [GIT PULL] ARM MSM updates for 2.6.35-rc1
2010-06-03 1:20 ` Linus Torvalds
` (2 preceding siblings ...)
2010-06-03 5:36 ` Ben Dooks
@ 2010-06-04 8:27 ` Vincent Sanders
3 siblings, 0 replies; 15+ messages in thread
From: Vincent Sanders @ 2010-06-04 8:27 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Linux Kernel Mailing List
[-- Attachment #1: Type: text/plain, Size: 918 bytes --]
On Wed, Jun 02, 2010 at 06:20:09PM -0700, Linus Torvalds wrote:
>
>
snip
>
> - TWO HUNDRED THOUSAND lines of arch/arm is just pure garbage, namely the
> defconfig files. Quite frankly, anybody who calls that anything but
> pure "noise" is just misguided and being stupid.
>
For some hard data which might be useful in arguments on this you
might want to reference
http://armlinux.simtec.co.uk/reports/kautobuild-5year-report.pdf
http://armlinux.simtec.co.uk/reports/kautobuild-5year-stats.ods
The written report is not complete as I had hoped to have an
announcement about the autobuilder fifth aniverasry soon. Perhaps this
will be useful in keeping some default configurations which give us
maximal coverage.
I provide this purely for information, I do not intend to upset you
futher on this subject, pretty please do not shoot the messenger ;-)
--
Regards Vincent
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread