* Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
@ 2013-08-02 7:53 ` Tony Lindgren
0 siblings, 0 replies; 38+ messages in thread
From: Tony Lindgren @ 2013-08-02 7:53 UTC (permalink / raw)
To: Greg KH; +Cc: ksummit-2013-discuss, linux-kernel, linux-arm-kernel
* Greg KH <greg@kroah.com> [130731 05:39]:
> On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> > Hi all,
> >
> > Probably the biggest kernel data bloat issue is in the ARM land, but
> > it also seems that it's becoming a Linux generic issue too, so I
> > guess it could be discussed in either context.
>
> Why is it specific to ARM? What is so unique to ARM that causes it to
> "bloat"?
I think it has so far showed up on ARM because of no discoverable busses,
but chances are it will be more of a generic problem.
> And what exactly do you mean by "bloat"?
Stuffing data to kernel that should not be in the kernel at all. Or
if the data is needed by kernel, there should be only one set of the
data defined rather than multiple copies of the data built into the
kernel for each SoC or driver variant.
> > Basically the data bloat issue is there for the arch code and drivers
> > and may not show up initially until things have headed the wrong way for
> > too long.
>
> What do you mean by this? You seem to be very vague here.
People are unnecessarily defining registers in kernel for similar devices
over and over again for each new SoC at the arch level and now more and
more at the driver level.
One example of that are device tree based drivers that don't describe
the actual hardware, but instead have a binding that points to an index
of defined registers in the driver.
One way to avoid these kind of bloat issues is to allow drivers to load
data at multiple points: Only abtolutely minimal set of data should be
static, some should only come from the bootloader as a device tree or
ACPI tables, and some is best to be loaded after booting from /lib/firmware.
Regards,
Tony
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
2013-08-02 7:53 ` Tony Lindgren
@ 2013-08-02 8:03 ` Felipe Balbi
-1 siblings, 0 replies; 38+ messages in thread
From: Felipe Balbi @ 2013-08-02 8:03 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Fri, Aug 02, 2013 at 12:53:53AM -0700, Tony Lindgren wrote:
> > > Basically the data bloat issue is there for the arch code and drivers
> > > and may not show up initially until things have headed the wrong way for
> > > too long.
> >
> > What do you mean by this? You seem to be very vague here.
>
> People are unnecessarily defining registers in kernel for similar devices
> over and over again for each new SoC at the arch level and now more and
> more at the driver level.
>
> One example of that are device tree based drivers that don't describe
> the actual hardware, but instead have a binding that points to an index
> of defined registers in the driver.
-ECONFUSED... DT passes only the base address and the size of the
address space. If some versions of the IP have slightly different
register layout, that needs to be treated at the driver, right ?
> One way to avoid these kind of bloat issues is to allow drivers to load
> data at multiple points: Only abtolutely minimal set of data should be
> static, some should only come from the bootloader as a device tree or
> ACPI tables, and some is best to be loaded after booting from /lib/firmware.
why would we put data blobs in /lib/firmware ? I know we have discussed
this at some length before, but I still don't get the idea that, just
because data shouldn't be in the kernel, we would bloat /lib/firmware
with blobs which aren't really firmwares.
It would be like adding ACPI tables to /lib/firmware :-p
--
balbi
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130802/c8963d1a/attachment.sig>
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
@ 2013-08-02 8:03 ` Felipe Balbi
0 siblings, 0 replies; 38+ messages in thread
From: Felipe Balbi @ 2013-08-02 8:03 UTC (permalink / raw)
To: Tony Lindgren
Cc: Greg KH, ksummit-2013-discuss, linux-kernel, linux-arm-kernel
[-- Attachment #1: Type: text/plain, Size: 1530 bytes --]
Hi,
On Fri, Aug 02, 2013 at 12:53:53AM -0700, Tony Lindgren wrote:
> > > Basically the data bloat issue is there for the arch code and drivers
> > > and may not show up initially until things have headed the wrong way for
> > > too long.
> >
> > What do you mean by this? You seem to be very vague here.
>
> People are unnecessarily defining registers in kernel for similar devices
> over and over again for each new SoC at the arch level and now more and
> more at the driver level.
>
> One example of that are device tree based drivers that don't describe
> the actual hardware, but instead have a binding that points to an index
> of defined registers in the driver.
-ECONFUSED... DT passes only the base address and the size of the
address space. If some versions of the IP have slightly different
register layout, that needs to be treated at the driver, right ?
> One way to avoid these kind of bloat issues is to allow drivers to load
> data at multiple points: Only abtolutely minimal set of data should be
> static, some should only come from the bootloader as a device tree or
> ACPI tables, and some is best to be loaded after booting from /lib/firmware.
why would we put data blobs in /lib/firmware ? I know we have discussed
this at some length before, but I still don't get the idea that, just
because data shouldn't be in the kernel, we would bloat /lib/firmware
with blobs which aren't really firmwares.
It would be like adding ACPI tables to /lib/firmware :-p
--
balbi
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
2013-08-02 8:03 ` Felipe Balbi
@ 2013-08-02 8:26 ` Tony Lindgren
-1 siblings, 0 replies; 38+ messages in thread
From: Tony Lindgren @ 2013-08-02 8:26 UTC (permalink / raw)
To: linux-arm-kernel
* Felipe Balbi <balbi@ti.com> [130802 01:11]:
> Hi,
>
> On Fri, Aug 02, 2013 at 12:53:53AM -0700, Tony Lindgren wrote:
> > > > Basically the data bloat issue is there for the arch code and drivers
> > > > and may not show up initially until things have headed the wrong way for
> > > > too long.
> > >
> > > What do you mean by this? You seem to be very vague here.
> >
> > People are unnecessarily defining registers in kernel for similar devices
> > over and over again for each new SoC at the arch level and now more and
> > more at the driver level.
> >
> > One example of that are device tree based drivers that don't describe
> > the actual hardware, but instead have a binding that points to an index
> > of defined registers in the driver.
>
> -ECONFUSED... DT passes only the base address and the size of the
> address space. If some versions of the IP have slightly different
> register layout, that needs to be treated at the driver, right ?
The driver shoud know how to handle various _types_ of registers rather
than contain a data what these registers are. Then the DT should define
the _types_ of registers that the hardware has. Any register names etc
are just debug data that can be handled with userspace tools using debugfs.
> > One way to avoid these kind of bloat issues is to allow drivers to load
> > data at multiple points: Only abtolutely minimal set of data should be
> > static, some should only come from the bootloader as a device tree or
> > ACPI tables, and some is best to be loaded after booting from /lib/firmware.
>
> why would we put data blobs in /lib/firmware ? I know we have discussed
> this at some length before, but I still don't get the idea that, just
> because data shouldn't be in the kernel, we would bloat /lib/firmware
> with blobs which aren't really firmwares.
Because the amount of data coming from the bootloader should be pretty
minimal. And it is for ACPI, and device tree will run into performance
issues trying stuff all the data there.
And a huge amount of the data we have in platform_data, defined registers,
device tree etc is only needed by the kernel for debugging, or is only
needed later on for things like PM.
> It would be like adding ACPI tables to /lib/firmware :-p
I don't think ACPI has this issue, the set of data coming from ACPI
is pretty minimal. But I bet a lot of the ACPI drivers could also load
the debug related data as additional kernel modules or from /lib/firmware
especially if it's somehow SoC or board specific.
Regards,
Tony
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
@ 2013-08-02 8:26 ` Tony Lindgren
0 siblings, 0 replies; 38+ messages in thread
From: Tony Lindgren @ 2013-08-02 8:26 UTC (permalink / raw)
To: Felipe Balbi
Cc: Greg KH, ksummit-2013-discuss, linux-kernel, linux-arm-kernel
* Felipe Balbi <balbi@ti.com> [130802 01:11]:
> Hi,
>
> On Fri, Aug 02, 2013 at 12:53:53AM -0700, Tony Lindgren wrote:
> > > > Basically the data bloat issue is there for the arch code and drivers
> > > > and may not show up initially until things have headed the wrong way for
> > > > too long.
> > >
> > > What do you mean by this? You seem to be very vague here.
> >
> > People are unnecessarily defining registers in kernel for similar devices
> > over and over again for each new SoC at the arch level and now more and
> > more at the driver level.
> >
> > One example of that are device tree based drivers that don't describe
> > the actual hardware, but instead have a binding that points to an index
> > of defined registers in the driver.
>
> -ECONFUSED... DT passes only the base address and the size of the
> address space. If some versions of the IP have slightly different
> register layout, that needs to be treated at the driver, right ?
The driver shoud know how to handle various _types_ of registers rather
than contain a data what these registers are. Then the DT should define
the _types_ of registers that the hardware has. Any register names etc
are just debug data that can be handled with userspace tools using debugfs.
> > One way to avoid these kind of bloat issues is to allow drivers to load
> > data at multiple points: Only abtolutely minimal set of data should be
> > static, some should only come from the bootloader as a device tree or
> > ACPI tables, and some is best to be loaded after booting from /lib/firmware.
>
> why would we put data blobs in /lib/firmware ? I know we have discussed
> this at some length before, but I still don't get the idea that, just
> because data shouldn't be in the kernel, we would bloat /lib/firmware
> with blobs which aren't really firmwares.
Because the amount of data coming from the bootloader should be pretty
minimal. And it is for ACPI, and device tree will run into performance
issues trying stuff all the data there.
And a huge amount of the data we have in platform_data, defined registers,
device tree etc is only needed by the kernel for debugging, or is only
needed later on for things like PM.
> It would be like adding ACPI tables to /lib/firmware :-p
I don't think ACPI has this issue, the set of data coming from ACPI
is pretty minimal. But I bet a lot of the ACPI drivers could also load
the debug related data as additional kernel modules or from /lib/firmware
especially if it's somehow SoC or board specific.
Regards,
Tony
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
2013-08-02 7:53 ` Tony Lindgren
@ 2013-08-02 8:11 ` Greg KH
-1 siblings, 0 replies; 38+ messages in thread
From: Greg KH @ 2013-08-02 8:11 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Aug 02, 2013 at 12:53:53AM -0700, Tony Lindgren wrote:
> * Greg KH <greg@kroah.com> [130731 05:39]:
> > On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> > > Hi all,
> > >
> > > Probably the biggest kernel data bloat issue is in the ARM land, but
> > > it also seems that it's becoming a Linux generic issue too, so I
> > > guess it could be discussed in either context.
> >
> > Why is it specific to ARM? What is so unique to ARM that causes it to
> > "bloat"?
>
> I think it has so far showed up on ARM because of no discoverable busses,
> but chances are it will be more of a generic problem.
>
> > And what exactly do you mean by "bloat"?
>
> Stuffing data to kernel that should not be in the kernel at all. Or
> if the data is needed by kernel, there should be only one set of the
> data defined rather than multiple copies of the data built into the
> kernel for each SoC or driver variant.
>
> > > Basically the data bloat issue is there for the arch code and drivers
> > > and may not show up initially until things have headed the wrong way for
> > > too long.
> >
> > What do you mean by this? You seem to be very vague here.
>
> People are unnecessarily defining registers in kernel for similar devices
> over and over again for each new SoC at the arch level and now more and
> more at the driver level.
>
> One example of that are device tree based drivers that don't describe
> the actual hardware, but instead have a binding that points to an index
> of defined registers in the driver.
Ok, and exactly how much "larger" does something like this cost as a
real number, and as a percentage of the size of the kernel?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
@ 2013-08-02 8:11 ` Greg KH
0 siblings, 0 replies; 38+ messages in thread
From: Greg KH @ 2013-08-02 8:11 UTC (permalink / raw)
To: Tony Lindgren; +Cc: ksummit-2013-discuss, linux-kernel, linux-arm-kernel
On Fri, Aug 02, 2013 at 12:53:53AM -0700, Tony Lindgren wrote:
> * Greg KH <greg@kroah.com> [130731 05:39]:
> > On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> > > Hi all,
> > >
> > > Probably the biggest kernel data bloat issue is in the ARM land, but
> > > it also seems that it's becoming a Linux generic issue too, so I
> > > guess it could be discussed in either context.
> >
> > Why is it specific to ARM? What is so unique to ARM that causes it to
> > "bloat"?
>
> I think it has so far showed up on ARM because of no discoverable busses,
> but chances are it will be more of a generic problem.
>
> > And what exactly do you mean by "bloat"?
>
> Stuffing data to kernel that should not be in the kernel at all. Or
> if the data is needed by kernel, there should be only one set of the
> data defined rather than multiple copies of the data built into the
> kernel for each SoC or driver variant.
>
> > > Basically the data bloat issue is there for the arch code and drivers
> > > and may not show up initially until things have headed the wrong way for
> > > too long.
> >
> > What do you mean by this? You seem to be very vague here.
>
> People are unnecessarily defining registers in kernel for similar devices
> over and over again for each new SoC at the arch level and now more and
> more at the driver level.
>
> One example of that are device tree based drivers that don't describe
> the actual hardware, but instead have a binding that points to an index
> of defined registers in the driver.
Ok, and exactly how much "larger" does something like this cost as a
real number, and as a percentage of the size of the kernel?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
2013-08-02 8:11 ` Greg KH
@ 2013-08-02 8:39 ` Tony Lindgren
-1 siblings, 0 replies; 38+ messages in thread
From: Tony Lindgren @ 2013-08-02 8:39 UTC (permalink / raw)
To: linux-arm-kernel
* Greg KH <greg@kroah.com> [130802 01:16]:
> On Fri, Aug 02, 2013 at 12:53:53AM -0700, Tony Lindgren wrote:
> > * Greg KH <greg@kroah.com> [130731 05:39]:
> > > On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> > > > Hi all,
> > > >
> > > > Probably the biggest kernel data bloat issue is in the ARM land, but
> > > > it also seems that it's becoming a Linux generic issue too, so I
> > > > guess it could be discussed in either context.
> > >
> > > Why is it specific to ARM? What is so unique to ARM that causes it to
> > > "bloat"?
> >
> > I think it has so far showed up on ARM because of no discoverable busses,
> > but chances are it will be more of a generic problem.
> >
> > > And what exactly do you mean by "bloat"?
> >
> > Stuffing data to kernel that should not be in the kernel at all. Or
> > if the data is needed by kernel, there should be only one set of the
> > data defined rather than multiple copies of the data built into the
> > kernel for each SoC or driver variant.
> >
> > > > Basically the data bloat issue is there for the arch code and drivers
> > > > and may not show up initially until things have headed the wrong way for
> > > > too long.
> > >
> > > What do you mean by this? You seem to be very vague here.
> >
> > People are unnecessarily defining registers in kernel for similar devices
> > over and over again for each new SoC at the arch level and now more and
> > more at the driver level.
> >
> > One example of that are device tree based drivers that don't describe
> > the actual hardware, but instead have a binding that points to an index
> > of defined registers in the driver.
>
> Ok, and exactly how much "larger" does something like this cost as a
> real number, and as a percentage of the size of the kernel?
Well one example has been making omap4 SoC booting device tree only, and
that has reduced the built in kernel data for pinmux, board support and
platform init code by something like 6000 lines, with patches posted to
reduce the clock related build in kernel data by about additional 1700
lines. Sure some of that has moved to live under drivers, but mostly defined
in the .dts files.
But I'm afraid quite a bit of stuff in general is now just moved to
drivers without dealing with the data issues properly. So I'm hoping
we could establish some guidelines on doing things that might help other
maintainers to catch and solve similar issues.
Regards,
Tony
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
@ 2013-08-02 8:39 ` Tony Lindgren
0 siblings, 0 replies; 38+ messages in thread
From: Tony Lindgren @ 2013-08-02 8:39 UTC (permalink / raw)
To: Greg KH; +Cc: ksummit-2013-discuss, linux-kernel, linux-arm-kernel
* Greg KH <greg@kroah.com> [130802 01:16]:
> On Fri, Aug 02, 2013 at 12:53:53AM -0700, Tony Lindgren wrote:
> > * Greg KH <greg@kroah.com> [130731 05:39]:
> > > On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> > > > Hi all,
> > > >
> > > > Probably the biggest kernel data bloat issue is in the ARM land, but
> > > > it also seems that it's becoming a Linux generic issue too, so I
> > > > guess it could be discussed in either context.
> > >
> > > Why is it specific to ARM? What is so unique to ARM that causes it to
> > > "bloat"?
> >
> > I think it has so far showed up on ARM because of no discoverable busses,
> > but chances are it will be more of a generic problem.
> >
> > > And what exactly do you mean by "bloat"?
> >
> > Stuffing data to kernel that should not be in the kernel at all. Or
> > if the data is needed by kernel, there should be only one set of the
> > data defined rather than multiple copies of the data built into the
> > kernel for each SoC or driver variant.
> >
> > > > Basically the data bloat issue is there for the arch code and drivers
> > > > and may not show up initially until things have headed the wrong way for
> > > > too long.
> > >
> > > What do you mean by this? You seem to be very vague here.
> >
> > People are unnecessarily defining registers in kernel for similar devices
> > over and over again for each new SoC at the arch level and now more and
> > more at the driver level.
> >
> > One example of that are device tree based drivers that don't describe
> > the actual hardware, but instead have a binding that points to an index
> > of defined registers in the driver.
>
> Ok, and exactly how much "larger" does something like this cost as a
> real number, and as a percentage of the size of the kernel?
Well one example has been making omap4 SoC booting device tree only, and
that has reduced the built in kernel data for pinmux, board support and
platform init code by something like 6000 lines, with patches posted to
reduce the clock related build in kernel data by about additional 1700
lines. Sure some of that has moved to live under drivers, but mostly defined
in the .dts files.
But I'm afraid quite a bit of stuff in general is now just moved to
drivers without dealing with the data issues properly. So I'm hoping
we could establish some guidelines on doing things that might help other
maintainers to catch and solve similar issues.
Regards,
Tony
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
2013-08-02 7:53 ` Tony Lindgren
@ 2013-08-02 12:41 ` Tony Lindgren
-1 siblings, 0 replies; 38+ messages in thread
From: Tony Lindgren @ 2013-08-02 12:41 UTC (permalink / raw)
To: linux-arm-kernel
* Tony Lindgren <tony@atomide.com> [130802 01:00]:
> * Greg KH <greg@kroah.com> [130731 05:39]:
> > On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> > > Hi all,
> > >
> > > Probably the biggest kernel data bloat issue is in the ARM land, but
> > > it also seems that it's becoming a Linux generic issue too, so I
> > > guess it could be discussed in either context.
> >
> > Why is it specific to ARM? What is so unique to ARM that causes it to
> > "bloat"?
>
> I think it has so far showed up on ARM because of no discoverable busses,
> but chances are it will be more of a generic problem.
>
> > And what exactly do you mean by "bloat"?
>
> Stuffing data to kernel that should not be in the kernel at all. Or
> if the data is needed by kernel, there should be only one set of the
> data defined rather than multiple copies of the data built into the
> kernel for each SoC or driver variant.
>
> > > Basically the data bloat issue is there for the arch code and drivers
> > > and may not show up initially until things have headed the wrong way for
> > > too long.
> >
> > What do you mean by this? You seem to be very vague here.
>
> People are unnecessarily defining registers in kernel for similar devices
> over and over again for each new SoC at the arch level and now more and
> more at the driver level.
>
> One example of that are device tree based drivers that don't describe
> the actual hardware, but instead have a binding that points to an index
> of defined registers in the driver.
>
> One way to avoid these kind of bloat issues is to allow drivers to load
> data at multiple points: Only abtolutely minimal set of data should be
> static, some should only come from the bootloader as a device tree or
> ACPI tables, and some is best to be loaded after booting from /lib/firmware.
Oh and thinking about it a bit more, this issue is mostly with the
device drivers implementing frameworks, not the device drivers
using the frameworks. Things like clocks, regulators, muxes etc where
an almost similar instance is repeated tens or hundreds of times for
each SoC.
Regards,
Tony
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
@ 2013-08-02 12:41 ` Tony Lindgren
0 siblings, 0 replies; 38+ messages in thread
From: Tony Lindgren @ 2013-08-02 12:41 UTC (permalink / raw)
To: Greg KH; +Cc: ksummit-2013-discuss, linux-kernel, linux-arm-kernel
* Tony Lindgren <tony@atomide.com> [130802 01:00]:
> * Greg KH <greg@kroah.com> [130731 05:39]:
> > On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> > > Hi all,
> > >
> > > Probably the biggest kernel data bloat issue is in the ARM land, but
> > > it also seems that it's becoming a Linux generic issue too, so I
> > > guess it could be discussed in either context.
> >
> > Why is it specific to ARM? What is so unique to ARM that causes it to
> > "bloat"?
>
> I think it has so far showed up on ARM because of no discoverable busses,
> but chances are it will be more of a generic problem.
>
> > And what exactly do you mean by "bloat"?
>
> Stuffing data to kernel that should not be in the kernel at all. Or
> if the data is needed by kernel, there should be only one set of the
> data defined rather than multiple copies of the data built into the
> kernel for each SoC or driver variant.
>
> > > Basically the data bloat issue is there for the arch code and drivers
> > > and may not show up initially until things have headed the wrong way for
> > > too long.
> >
> > What do you mean by this? You seem to be very vague here.
>
> People are unnecessarily defining registers in kernel for similar devices
> over and over again for each new SoC at the arch level and now more and
> more at the driver level.
>
> One example of that are device tree based drivers that don't describe
> the actual hardware, but instead have a binding that points to an index
> of defined registers in the driver.
>
> One way to avoid these kind of bloat issues is to allow drivers to load
> data at multiple points: Only abtolutely minimal set of data should be
> static, some should only come from the bootloader as a device tree or
> ACPI tables, and some is best to be loaded after booting from /lib/firmware.
Oh and thinking about it a bit more, this issue is mostly with the
device drivers implementing frameworks, not the device drivers
using the frameworks. Things like clocks, regulators, muxes etc where
an almost similar instance is repeated tens or hundreds of times for
each SoC.
Regards,
Tony
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
2013-08-02 12:41 ` Tony Lindgren
@ 2013-08-02 13:24 ` Russell King - ARM Linux
-1 siblings, 0 replies; 38+ messages in thread
From: Russell King - ARM Linux @ 2013-08-02 13:24 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Aug 02, 2013 at 05:41:31AM -0700, Tony Lindgren wrote:
> Oh and thinking about it a bit more, this issue is mostly with the
> device drivers implementing frameworks, not the device drivers
> using the frameworks. Things like clocks, regulators, muxes etc where
> an almost similar instance is repeated tens or hundreds of times for
> each SoC.
That is where it helps to have a strong maintainer for a subsystem who
has the guts to refuse to accept stuff which is similar to existing
implementations and insist that existing implementations are either
adapted or reused.
It's all very well someone coming along and writing a "generic" set of
implementations (like tglx did for the IRQ subsystem) but unless there's
a motivation for people to use the generic stuff (such as... you won't
get your code in if you don't use the provided generics unless you can
provide a very good reasoned argument) then people are just going to
write their own code time and time again.
It's just like how the clocksources have gone. We now have multiple
implementations of how to read a counter which ticks at a specific
rate. You wouldn't think that I wrote drivers/clocksource/mmio.c which
can handle all of these simple 32-bit/16-bit up/down counter cases.
Again, the problem is there is no strong reviewer there who looks over
every addition and says "no, use the generic stuff".
That's the basic problem here: the review, and people saying "no" to
new stuff doing the same as generic stuff.
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
@ 2013-08-02 13:24 ` Russell King - ARM Linux
0 siblings, 0 replies; 38+ messages in thread
From: Russell King - ARM Linux @ 2013-08-02 13:24 UTC (permalink / raw)
To: Tony Lindgren
Cc: Greg KH, ksummit-2013-discuss, linux-kernel, linux-arm-kernel
On Fri, Aug 02, 2013 at 05:41:31AM -0700, Tony Lindgren wrote:
> Oh and thinking about it a bit more, this issue is mostly with the
> device drivers implementing frameworks, not the device drivers
> using the frameworks. Things like clocks, regulators, muxes etc where
> an almost similar instance is repeated tens or hundreds of times for
> each SoC.
That is where it helps to have a strong maintainer for a subsystem who
has the guts to refuse to accept stuff which is similar to existing
implementations and insist that existing implementations are either
adapted or reused.
It's all very well someone coming along and writing a "generic" set of
implementations (like tglx did for the IRQ subsystem) but unless there's
a motivation for people to use the generic stuff (such as... you won't
get your code in if you don't use the provided generics unless you can
provide a very good reasoned argument) then people are just going to
write their own code time and time again.
It's just like how the clocksources have gone. We now have multiple
implementations of how to read a counter which ticks at a specific
rate. You wouldn't think that I wrote drivers/clocksource/mmio.c which
can handle all of these simple 32-bit/16-bit up/down counter cases.
Again, the problem is there is no strong reviewer there who looks over
every addition and says "no, use the generic stuff".
That's the basic problem here: the review, and people saying "no" to
new stuff doing the same as generic stuff.
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
2013-08-02 13:24 ` Russell King - ARM Linux
@ 2013-08-05 6:30 ` Tony Lindgren
-1 siblings, 0 replies; 38+ messages in thread
From: Tony Lindgren @ 2013-08-05 6:30 UTC (permalink / raw)
To: linux-arm-kernel
* Russell King - ARM Linux <linux@arm.linux.org.uk> [130802 06:31]:
> On Fri, Aug 02, 2013 at 05:41:31AM -0700, Tony Lindgren wrote:
> > Oh and thinking about it a bit more, this issue is mostly with the
> > device drivers implementing frameworks, not the device drivers
> > using the frameworks. Things like clocks, regulators, muxes etc where
> > an almost similar instance is repeated tens or hundreds of times for
> > each SoC.
>
> That is where it helps to have a strong maintainer for a subsystem who
> has the guts to refuse to accept stuff which is similar to existing
> implementations and insist that existing implementations are either
> adapted or reused.
>
> It's all very well someone coming along and writing a "generic" set of
> implementations (like tglx did for the IRQ subsystem) but unless there's
> a motivation for people to use the generic stuff (such as... you won't
> get your code in if you don't use the provided generics unless you can
> provide a very good reasoned argument) then people are just going to
> write their own code time and time again.
>
> It's just like how the clocksources have gone. We now have multiple
> implementations of how to read a counter which ticks at a specific
> rate. You wouldn't think that I wrote drivers/clocksource/mmio.c which
> can handle all of these simple 32-bit/16-bit up/down counter cases.
> Again, the problem is there is no strong reviewer there who looks over
> every addition and says "no, use the generic stuff".
>
> That's the basic problem here: the review, and people saying "no" to
> new stuff doing the same as generic stuff.
Right, but to make all that easier I was thinking that we can possibly
provide some generic guidelines and tools for maintainers to avoid some
of these issues.
The data issues should be pretty easy to spot based on the size of the
driver, or looking at the object file with size command for the data to
text ratio etc.
Regards,
Tony
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
@ 2013-08-05 6:30 ` Tony Lindgren
0 siblings, 0 replies; 38+ messages in thread
From: Tony Lindgren @ 2013-08-05 6:30 UTC (permalink / raw)
To: Russell King - ARM Linux
Cc: Greg KH, ksummit-2013-discuss, linux-kernel, linux-arm-kernel
* Russell King - ARM Linux <linux@arm.linux.org.uk> [130802 06:31]:
> On Fri, Aug 02, 2013 at 05:41:31AM -0700, Tony Lindgren wrote:
> > Oh and thinking about it a bit more, this issue is mostly with the
> > device drivers implementing frameworks, not the device drivers
> > using the frameworks. Things like clocks, regulators, muxes etc where
> > an almost similar instance is repeated tens or hundreds of times for
> > each SoC.
>
> That is where it helps to have a strong maintainer for a subsystem who
> has the guts to refuse to accept stuff which is similar to existing
> implementations and insist that existing implementations are either
> adapted or reused.
>
> It's all very well someone coming along and writing a "generic" set of
> implementations (like tglx did for the IRQ subsystem) but unless there's
> a motivation for people to use the generic stuff (such as... you won't
> get your code in if you don't use the provided generics unless you can
> provide a very good reasoned argument) then people are just going to
> write their own code time and time again.
>
> It's just like how the clocksources have gone. We now have multiple
> implementations of how to read a counter which ticks at a specific
> rate. You wouldn't think that I wrote drivers/clocksource/mmio.c which
> can handle all of these simple 32-bit/16-bit up/down counter cases.
> Again, the problem is there is no strong reviewer there who looks over
> every addition and says "no, use the generic stuff".
>
> That's the basic problem here: the review, and people saying "no" to
> new stuff doing the same as generic stuff.
Right, but to make all that easier I was thinking that we can possibly
provide some generic guidelines and tools for maintainers to avoid some
of these issues.
The data issues should be pretty easy to spot based on the size of the
driver, or looking at the object file with size command for the data to
text ratio etc.
Regards,
Tony
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
2013-08-02 7:53 ` Tony Lindgren
@ 2013-08-02 19:57 ` Mike Turquette
-1 siblings, 0 replies; 38+ messages in thread
From: Mike Turquette @ 2013-08-02 19:57 UTC (permalink / raw)
To: linux-arm-kernel
Quoting Tony Lindgren (2013-08-02 00:53:53)
> People are unnecessarily defining registers in kernel for similar devices
> over and over again for each new SoC at the arch level and now more and
> more at the driver level.
>
> One example of that are device tree based drivers that don't describe
> the actual hardware, but instead have a binding that points to an index
> of defined registers in the driver.
Apologies for possibly hijacking this thread, but this issue keeps me up
at night. People use DT for different things and have different ideas
about its Purpose In The World.
Tony wants to move data out of the kernel, which was the impetus behind
the OMAP DT clock patches that describes every single clock in a DT node
(around 250 clocks). This create very large dts, but reduces the clock
drivers to pure logic, no data.
Other folks are motivated only to get rid of board files and platform
data hacks. They keep all of the clock data in the clock driver and
instead use DT only as a way to hook up clocks to devices via a simple
mapping, thus describing how individual boards or SoC variants are set
up outside of the kernel source. dts remains small, essentially just an
array to map bindings but all of the clock data remains in the kernel.
Both approaches have their merits and drawbacks. Is it possible to
gather consensus on selecting a single approach? For the clock subsystem
I've accepted drivers and DT bindings which do either. I simply do not
have the DT experience or sensibilities to draw a line in the sand...
and maybe I should not draw a line in the sand and just let people pick
whichever approach they prefer (which maintains the status quo).
If the ARM Summit figures out all the answers then please let me know
what they are :-)
Thanks,
Mike
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
@ 2013-08-02 19:57 ` Mike Turquette
0 siblings, 0 replies; 38+ messages in thread
From: Mike Turquette @ 2013-08-02 19:57 UTC (permalink / raw)
To: Tony Lindgren, Greg KH
Cc: ksummit-2013-discuss, linux-kernel, linux-arm-kernel
Quoting Tony Lindgren (2013-08-02 00:53:53)
> People are unnecessarily defining registers in kernel for similar devices
> over and over again for each new SoC at the arch level and now more and
> more at the driver level.
>
> One example of that are device tree based drivers that don't describe
> the actual hardware, but instead have a binding that points to an index
> of defined registers in the driver.
Apologies for possibly hijacking this thread, but this issue keeps me up
at night. People use DT for different things and have different ideas
about its Purpose In The World.
Tony wants to move data out of the kernel, which was the impetus behind
the OMAP DT clock patches that describes every single clock in a DT node
(around 250 clocks). This create very large dts, but reduces the clock
drivers to pure logic, no data.
Other folks are motivated only to get rid of board files and platform
data hacks. They keep all of the clock data in the clock driver and
instead use DT only as a way to hook up clocks to devices via a simple
mapping, thus describing how individual boards or SoC variants are set
up outside of the kernel source. dts remains small, essentially just an
array to map bindings but all of the clock data remains in the kernel.
Both approaches have their merits and drawbacks. Is it possible to
gather consensus on selecting a single approach? For the clock subsystem
I've accepted drivers and DT bindings which do either. I simply do not
have the DT experience or sensibilities to draw a line in the sand...
and maybe I should not draw a line in the sand and just let people pick
whichever approach they prefer (which maintains the status quo).
If the ARM Summit figures out all the answers then please let me know
what they are :-)
Thanks,
Mike
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
2013-08-02 19:57 ` Mike Turquette
@ 2013-08-05 6:36 ` Tony Lindgren
-1 siblings, 0 replies; 38+ messages in thread
From: Tony Lindgren @ 2013-08-05 6:36 UTC (permalink / raw)
To: linux-arm-kernel
* Mike Turquette <mturquette@linaro.org> [130802 13:04]:
> Quoting Tony Lindgren (2013-08-02 00:53:53)
> > People are unnecessarily defining registers in kernel for similar devices
> > over and over again for each new SoC at the arch level and now more and
> > more at the driver level.
> >
> > One example of that are device tree based drivers that don't describe
> > the actual hardware, but instead have a binding that points to an index
> > of defined registers in the driver.
>
> Apologies for possibly hijacking this thread, but this issue keeps me up
> at night. People use DT for different things and have different ideas
> about its Purpose In The World.
>
> Tony wants to move data out of the kernel, which was the impetus behind
> the OMAP DT clock patches that describes every single clock in a DT node
> (around 250 clocks). This create very large dts, but reduces the clock
> drivers to pure logic, no data.
Yes that should eventually allow "booting new hardware with old
kernels".. But ideally with clocks we should support any combination of
DT defined clocks and /lib/firmware defined clocks should be allowed to
avoid bloating the DT files. We should be able to boot to user space
with a pretty minimal set of clocks, and deal with the PM related settings
based on SoC specific data from /lib/firmware.
> Other folks are motivated only to get rid of board files and platform
> data hacks. They keep all of the clock data in the clock driver and
> instead use DT only as a way to hook up clocks to devices via a simple
> mapping, thus describing how individual boards or SoC variants are set
> up outside of the kernel source. dts remains small, essentially just an
> array to map bindings but all of the clock data remains in the kernel.
Right, that just moves the real problem to drivers and then it will
eventually suck up all your maintainer time with constant churn to patch
that data for various SoC revisions :)
> Both approaches have their merits and drawbacks. Is it possible to
> gather consensus on selecting a single approach? For the clock subsystem
> I've accepted drivers and DT bindings which do either. I simply do not
> have the DT experience or sensibilities to draw a line in the sand...
> and maybe I should not draw a line in the sand and just let people pick
> whichever approach they prefer (which maintains the status quo).
>
> If the ARM Summit figures out all the answers then please let me know
> what they are :-)
Well I think it's been discussed many times earlier and it should be
clear that Linus wants this kind of data out of the kernel. So maybe
we could establish some kind of set of standards for the maintainers
to follow.
Regards,
Tony
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it
@ 2013-08-05 6:36 ` Tony Lindgren
0 siblings, 0 replies; 38+ messages in thread
From: Tony Lindgren @ 2013-08-05 6:36 UTC (permalink / raw)
To: Mike Turquette
Cc: Greg KH, ksummit-2013-discuss, linux-kernel, linux-arm-kernel
* Mike Turquette <mturquette@linaro.org> [130802 13:04]:
> Quoting Tony Lindgren (2013-08-02 00:53:53)
> > People are unnecessarily defining registers in kernel for similar devices
> > over and over again for each new SoC at the arch level and now more and
> > more at the driver level.
> >
> > One example of that are device tree based drivers that don't describe
> > the actual hardware, but instead have a binding that points to an index
> > of defined registers in the driver.
>
> Apologies for possibly hijacking this thread, but this issue keeps me up
> at night. People use DT for different things and have different ideas
> about its Purpose In The World.
>
> Tony wants to move data out of the kernel, which was the impetus behind
> the OMAP DT clock patches that describes every single clock in a DT node
> (around 250 clocks). This create very large dts, but reduces the clock
> drivers to pure logic, no data.
Yes that should eventually allow "booting new hardware with old
kernels".. But ideally with clocks we should support any combination of
DT defined clocks and /lib/firmware defined clocks should be allowed to
avoid bloating the DT files. We should be able to boot to user space
with a pretty minimal set of clocks, and deal with the PM related settings
based on SoC specific data from /lib/firmware.
> Other folks are motivated only to get rid of board files and platform
> data hacks. They keep all of the clock data in the clock driver and
> instead use DT only as a way to hook up clocks to devices via a simple
> mapping, thus describing how individual boards or SoC variants are set
> up outside of the kernel source. dts remains small, essentially just an
> array to map bindings but all of the clock data remains in the kernel.
Right, that just moves the real problem to drivers and then it will
eventually suck up all your maintainer time with constant churn to patch
that data for various SoC revisions :)
> Both approaches have their merits and drawbacks. Is it possible to
> gather consensus on selecting a single approach? For the clock subsystem
> I've accepted drivers and DT bindings which do either. I simply do not
> have the DT experience or sensibilities to draw a line in the sand...
> and maybe I should not draw a line in the sand and just let people pick
> whichever approach they prefer (which maintains the status quo).
>
> If the ARM Summit figures out all the answers then please let me know
> what they are :-)
Well I think it's been discussed many times earlier and it should be
clear that Linus wants this kind of data out of the kernel. So maybe
we could establish some kind of set of standards for the maintainers
to follow.
Regards,
Tony
^ permalink raw reply [flat|nested] 38+ messages in thread