[PATCH] console - Add configurable support for console charset translation

[PATCH] console - Add configurable support for console charset translation

[Tim Bird]

With CONSOLE_TRANSLATIONS turned off, this saves about 6K
on my kernel configured for an ARM development board (OMAP
5912 OSK).  In embedded products I'm familiar with,
console translations are not needed.

This was taken from the Linux-tiny project and updated slightly
for 2.6.25.

  drivers/char/consolemap.c |   81 ++++++++++++++++++++++++++++++++++++++++++++++
  drivers/char/vt.c         |    4 ++
  init/Kconfig              |    7 +++
  3 files changed, 92 insertions(+)

Signed-off-by: Tim Bird <tim.bird@am.sony.com>

--- a/drivers/char/consolemap.c
+++ b/drivers/char/consolemap.c
@@ -22,6 +22,8 @@
  #include <linux/consolemap.h>
  #include <linux/vt_kern.h>

+#ifdef CONFIG_CONSOLE_TRANSLATIONS
+
  static unsigned short translations[][256] = {
    /* 8-bit Latin-1 mapped to Unicode -- trivial mapping */
    {
@@ -742,3 +744,82 @@ console_map_init(void)
  }

  EXPORT_SYMBOL(con_copy_unimap);
+
+#else
+
+u16 inverse_translate(struct vc_data *conp, int glyph, int use_unicode)
+{
+	return glyph;
+}
+
+unsigned short *set_translate(int m, struct vc_data *vc)
+{
+	return NULL;
+}
+
+int con_set_trans_old(unsigned char *arg)
+{
+	return 0;
+}
+
+int con_get_trans_old(unsigned char *arg)
+{
+	return -EINVAL;
+}
+
+int con_set_trans_new(ushort *arg)
+{
+	return 0;
+}
+
+int con_get_trans_new(ushort *arg)
+{
+	return -EINVAL;
+}
+
+int con_clear_unimap(struct vc_data *vc, struct unimapinit *ui)
+{
+	return 0;
+}
+
+int con_set_unimap(struct vc_data *vc, ushort ct, struct unipair *list)
+{
+	return 0;
+}
+
+int con_set_default_unimap(struct vc_data *vc)
+{
+	return 0;
+}
+
+int con_copy_unimap(struct vc_data *d, struct vc_data *s)
+{
+	return 0;
+}
+
+int con_get_unimap(struct vc_data *vc, ushort ct, ushort *uct,
+		   struct unipair *list)
+{
+	return -EINVAL;
+}
+
+void con_free_unimap(struct vc_data *vc) { }
+
+int conv_uni_to_pc(struct vc_data *conp, long ucs)
+{
+	return ucs > 0xff ? -1: ucs;
+}
+
+void __init console_map_init(void) { }
+
+u32 conv_8bit_to_uni(unsigned char c)
+{
+	return c;
+}
+
+int conv_uni_to_8bit(u32 uni)
+{
+	return (int)uni & 0xff;
+}
+
+#endif
--- a/drivers/char/vt.c
+++ b/drivers/char/vt.c
@@ -2198,7 +2198,11 @@ rescan_last_byte:
  			c = 0xfffd;
  		    tc = c;
  		} else {	/* no utf or alternate charset mode */
+#ifdef CONFIG_CONSOLE_TRANSLATIONS
  		    tc = vc->vc_translate[vc->vc_toggle_meta ? (c | 0x80) : c];
+#else
+		    tc = c;
+#endif
  		}

                  /* If the original code was a control character we
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -758,6 +758,13 @@ config PROC_PAGE_MONITOR
  	  /proc/kpagecount, and /proc/kpageflags. Disabling these
            interfaces will reduce the size of the kernel by approximately 4kb.

+config CONSOLE_TRANSLATIONS
+	default y
+	bool "Enable character translations in console" if EMBEDDED
+	help
+	  This enables support for font mapping and Unicode translation
+          on virtual consoles.
+
  endmenu		# General setup

  config SLABINFO

Re: [PATCH] console - Add configurable support for console charset translation

[H. Peter Anvin]

Tim Bird wrote:
> With CONSOLE_TRANSLATIONS turned off, this saves about 6K
> on my kernel configured for an ARM development board (OMAP
> 5912 OSK).  In embedded products I'm familiar with,
> console translations are not needed.

On most embedded products I'm familiar with, you wouldn't have virtual 
consoles at all...?

	-hpa

Re: [PATCH] console - Add configurable support for console charset translation

[Jamie Lokier]

H. Peter Anvin wrote:
> Tim Bird wrote:
> >With CONSOLE_TRANSLATIONS turned off, this saves about 6K
> >on my kernel configured for an ARM development board (OMAP
> >5912 OSK).  In embedded products I'm familiar with,
> >console translations are not needed.
> 
> On most embedded products I'm familiar with, you wouldn't have virtual 
> consoles at all...?

On anything with a framebuffer, VCs are quite handy for debugging.
Saving 6k won't save the world on those devices, but each little bit
helps - especially due to no-MMU fragmentation.

-- Jamie

Re: [PATCH] console - Add configurable support for console charset translation

[Matt Mackall]

On Mon, 2008-06-02 at 18:34 -0700, H. Peter Anvin wrote:
> Tim Bird wrote:
> > With CONSOLE_TRANSLATIONS turned off, this saves about 6K
> > on my kernel configured for an ARM development board (OMAP
> > 5912 OSK).  In embedded products I'm familiar with,
> > console translations are not needed.
> 
> On most embedded products I'm familiar with, you wouldn't have virtual 
> consoles at all...?

Actually, lots have frame buffers these days.

-- 
Mathematics is the supreme nostalgia of our time.

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Mon, 2008-06-02 at 15:37 -0700, Tim Bird wrote:
> With CONSOLE_TRANSLATIONS turned off, this saves about 6K
> on my kernel configured for an ARM development board (OMAP
> 5912 OSK).  In embedded products I'm familiar with,
> console translations are not needed.

I'd have been more inclined to put the #ifdef and the empty functions
into <linux/vt_kern.h>. Then you don't need to provide real stub
functions -- they can be inlined at the caller.

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Mon, 2008-06-02 at 15:37 -0700, Tim Bird wrote:
> With CONSOLE_TRANSLATIONS turned off, this saves about 6K
> on my kernel configured for an ARM development board (OMAP
> 5912 OSK).  In embedded products I'm familiar with,
> console translations are not needed.
> 
> This was taken from the Linux-tiny project and updated slightly
> for 2.6.25.

I prefer it like this... we can drop consolemap.o and
consolemap_deftbl.o from the build completely. It saves 7.2KiB on a
ppc32 build here.

diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig
index 595a925..f740190 100644
--- a/drivers/char/Kconfig
+++ b/drivers/char/Kconfig
@@ -36,6 +36,13 @@ config VT
 	  If unsure, say Y, or else you won't be able to do much with your new
 	  shiny Linux system :-)
 
+config CONSOLE_TRANSLATIONS
+	default y
+	bool "Enable character translations in console" if EMBEDDED
+	---help---
+	  This enables support for font mapping and Unicode translation
+	  on virtual consoles.
+
 config VT_CONSOLE
 	bool "Support for console on virtual terminal" if EMBEDDED
 	depends on VT
diff --git a/drivers/char/Makefile b/drivers/char/Makefile
index 4c1c584..6ef173c 100644
--- a/drivers/char/Makefile
+++ b/drivers/char/Makefile
@@ -12,8 +12,8 @@ obj-y	 += mem.o random.o tty_io.o n_tty.o tty_ioctl.o
 obj-$(CONFIG_LEGACY_PTYS)	+= pty.o
 obj-$(CONFIG_UNIX98_PTYS)	+= pty.o
 obj-y				+= misc.o
-obj-$(CONFIG_VT)		+= vt_ioctl.o vc_screen.o consolemap.o \
-				   consolemap_deftbl.o selection.o keyboard.o
+obj-$(CONFIG_VT)		+= vt_ioctl.o vc_screen.o selection.o keyboard.o
+obj-$(CONFIG_CONSOLE_TRANSLATIONS) += consolemap.o consolemap_deftbl.o
 obj-$(CONFIG_HW_CONSOLE)	+= vt.o defkeymap.o
 obj-$(CONFIG_AUDIT)		+= tty_audit.o
 obj-$(CONFIG_MAGIC_SYSRQ)	+= sysrq.o
diff --git a/include/linux/consolemap.h b/include/linux/consolemap.h
index e2bf7e5..c4811da 100644
--- a/include/linux/consolemap.h
+++ b/include/linux/consolemap.h
@@ -3,6 +3,9 @@
  *
  * Interface between console.c, selection.c  and consolemap.c
  */
+#ifndef __LINUX_CONSOLEMAP_H__
+#define __LINUX_CONSOLEMAP_H__
+
 #define LAT1_MAP 0
 #define GRAF_MAP 1
 #define IBMPC_MAP 2
@@ -10,6 +13,7 @@
 
 #include <linux/types.h>
 
+#ifdef CONFIG_CONSOLE_TRANSLATIONS
 struct vc_data;
 
 extern u16 inverse_translate(struct vc_data *conp, int glyph, int use_unicode);
@@ -18,3 +22,13 @@ extern int conv_uni_to_pc(struct vc_data *conp, long ucs);
 extern u32 conv_8bit_to_uni(unsigned char c);
 extern int conv_uni_to_8bit(u32 uni);
 void console_map_init(void);
+#else
+#define inverse_translate(conp, glyph, uni) ((uint16_t)glyph)
+#define set_translate(m, vc) ((unsigned short *)NULL)
+#define conv_uni_to_pc(conp, ucs) ((int) (ucs > 0xff ? -1: ucs))
+#define conv_8bit_to_uni(c) ((uint32_t)(c))
+#define conv_uni_to_8bit(c) ((int) ((c) & 0xff))
+#define console_map_init(c) do { ; } while (0)
+#endif /* CONFIG_CONSOLE_TRANSLATIONS */
+
+#endif /* __LINUX_CONSOLEMAP_H__ */
diff --git a/include/linux/vt_kern.h b/include/linux/vt_kern.h
index 9448ffb..81ed155 100644
--- a/include/linux/vt_kern.h
+++ b/include/linux/vt_kern.h
@@ -12,6 +12,7 @@
 #include <linux/mutex.h>
 #include <linux/console_struct.h>
 #include <linux/mm.h>
+#include <linux/consolemap.h>
 
 /*
  * Presently, a lot of graphics programs do not restore the contents of
@@ -54,6 +55,7 @@ void redraw_screen(struct vc_data *vc, int is_switch);
 struct tty_struct;
 int tioclinux(struct tty_struct *tty, unsigned long arg);
 
+#ifdef CONFIG_CONSOLE_TRANSLATIONS
 /* consolemap.c */
 
 struct unimapinit;
@@ -70,6 +72,18 @@ int con_set_default_unimap(struct vc_data *vc);
 void con_free_unimap(struct vc_data *vc);
 void con_protect_unimap(struct vc_data *vc, int rdonly);
 int con_copy_unimap(struct vc_data *dst_vc, struct vc_data *src_vc);
+#else
+#define con_set_trans_old(arg) (0)
+#define con_get_trans_old(arg) (-EINVAL)
+#define con_set_trans_new(arg) (0)
+#define con_get_trans_new(arg) (-EINVAL)
+#define con_clear_unimap(vc, ui) (0)
+#define con_set_unimap(vc, ct, list) (0)
+#define con_set_default_unimap(vc) (0)
+#define con_copy_unimap(d, s) (0)
+#define con_get_unimap(vc, ct, uct, list) (-EINVAL)
+#define con_free_unimap(vc) do { ; } while (0)
+#endif
 
 /* vt.c */
 int vt_waitactive(int vt);

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Holger Schurig]

Maybe change the Kconfig entry so that:

* with CONFIG_EMBEDDED=y it's configuration
* with CONFIG_EMBEDDED=n it's on by default

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Tue, 2008-06-03 at 16:06 +0200, Holger Schurig wrote:
> Maybe change the Kconfig entry so that:
> 
> * with CONFIG_EMBEDDED=y it's configuration
> * with CONFIG_EMBEDDED=n it's on by default

That's what Tim's patch did (although I moved it to drivers/char/Kconfig
and made it depend on CONFIG_VT).

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Tim Bird]

David Woodhouse wrote:
> On Mon, 2008-06-02 at 15:37 -0700, Tim Bird wrote:
>> With CONSOLE_TRANSLATIONS turned off, this saves about 6K
>> on my kernel configured for an ARM development board (OMAP
>> 5912 OSK).  In embedded products I'm familiar with,
>> console translations are not needed.
>>
>> This was taken from the Linux-tiny project and updated slightly
>> for 2.6.25.
> 
> I prefer it like this... we can drop consolemap.o and
> consolemap_deftbl.o from the build completely. It saves 7.2KiB on a
> ppc32 build here.
> 
> diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig
> index 595a925..f740190 100644
> --- a/drivers/char/Kconfig
> +++ b/drivers/char/Kconfig
> ...

This is clearly an improvement. But it is missing this part of the
original patch:

--- a/drivers/char/vt.c
+++ b/drivers/char/vt.c
@@ -2198,7 +2198,11 @@ rescan_last_byte:
 			c = 0xfffd;
 		    tc = c;
 		} else {	/* no utf or alternate charset mode */
+#ifdef CONFIG_CONSOLE_TRANSLATIONS
 		    tc = vc->vc_translate[vc->vc_toggle_meta ? (c | 0x80) : c];
+#else
+		    tc = c;
+#endif
 		}

                 /* If the original code was a control character we

With the set_translate function stubbed, and the actual translation
operation left intact, I think the code might have problems.

I ran your patch fine on my OSK board here, but I must not have hit a
character translation case.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Tue, 2008-06-03 at 17:01 -0700, Tim Bird wrote:
> This is clearly an improvement. But it is missing this part of the
> original patch:

Oops, well spotted. I've updated the patch in the git tree; thanks.

(that's what comes of applying patches by hand -- I _knew_ I had to do
that hunk, but managed to forget about it within the space of about 30
seconds when I was doing the other parts. I plead stupidity)

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Josh Boyer]

On Wed, 04 Jun 2008 01:16:37 +0100
David Woodhouse <dwmw2@infradead.org> wrote:

> On Tue, 2008-06-03 at 17:01 -0700, Tim Bird wrote:
> > This is clearly an improvement. But it is missing this part of the
> > original patch:
> 
> Oops, well spotted. I've updated the patch in the git tree; thanks.

What git tree?

josh

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Tue, 2008-06-03 at 20:03 -0500, Josh Boyer wrote:
> On Wed, 04 Jun 2008 01:16:37 +0100
> David Woodhouse <dwmw2@infradead.org> wrote:
> 
> > On Tue, 2008-06-03 at 17:01 -0700, Tim Bird wrote:
> > > This is clearly an improvement. But it is missing this part of the
> > > original patch:
> > 
> > Oops, well spotted. I've updated the patch in the git tree; thanks.
> 
> What git tree?

 git.infradead.org/embedded-2.6.git

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Josh Boyer]

On Wed, 04 Jun 2008 02:05:40 +0100
David Woodhouse <dwmw2@infradead.org> wrote:

> On Tue, 2008-06-03 at 20:03 -0500, Josh Boyer wrote:
> > On Wed, 04 Jun 2008 01:16:37 +0100
> > David Woodhouse <dwmw2@infradead.org> wrote:
> > 
> > > On Tue, 2008-06-03 at 17:01 -0700, Tim Bird wrote:
> > > > This is clearly an improvement. But it is missing this part of the
> > > > original patch:
> > > 
> > > Oops, well spotted. I've updated the patch in the git tree; thanks.
> > 
> > What git tree?
> 
>  git.infradead.org/embedded-2.6.git

Do you have plans to get that in -mm, or linux-next?

(Please say yes).

josh

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Tue, 2008-06-03 at 22:13 -0500, Josh Boyer wrote:
> 
> >  git.infradead.org/embedded-2.6.git
> 
> Do you have plans to get that in -mm, or linux-next?

Should be already there in today's linux-next.

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Adrian Bunk]

On Wed, Jun 04, 2008 at 10:16:22AM +0100, David Woodhouse wrote:
> On Tue, 2008-06-03 at 22:13 -0500, Josh Boyer wrote:
> > 
> > >  git.infradead.org/embedded-2.6.git
> > 
> > Do you have plans to get that in -mm, or linux-next?
> 
> Should be already there in today's linux-next.

Does this imply you want Linus to merge from this tree?

How does this tree fit into the usual maintainership inside the Linux 
kernel?

> dwmw2

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Wed, 2008-06-04 at 13:07 +0300, Adrian Bunk wrote:
> On Wed, Jun 04, 2008 at 10:16:22AM +0100, David Woodhouse wrote:
> > On Tue, 2008-06-03 at 22:13 -0500, Josh Boyer wrote:
> > > 
> > > >  git.infradead.org/embedded-2.6.git
> > > 
> > > Do you have plans to get that in -mm, or linux-next?
> > 
> > Should be already there in today's linux-next.
> 
> Does this imply you want Linus to merge from this tree?

Yes.

> How does this tree fit into the usual maintainership inside the Linux 
> kernel?

Well, I was trying to avoid creating it -- because largely it doesn't.
I was hoping that everything could go upstream through other routes.
But Tim's patch really was a candidate for handling it myself (unless
I'm just going to punt everything to Andrew), so I capitulated.

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Tim Bird]

David Woodhouse wrote:
> On Tue, 2008-06-03 at 17:01 -0700, Tim Bird wrote:
>> This is clearly an improvement. But it is missing this part of the
>> original patch:
> 
> Oops, well spotted. I've updated the patch in the git tree; thanks.
> 
> (that's what comes of applying patches by hand -- I _knew_ I had to do
> that hunk, but managed to forget about it within the space of about 30
> seconds when I was doing the other parts. I plead stupidity)

We can chalk it up to my stupidity for sending a patch
that couldn't be applied automatically...
Thanks very much for the improvements!
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: [PATCH] console - Add configurable support for console charset translation

[Alan Cox]

> This is clearly an improvement. But it is missing this part of the
> original patch:
> 
> --- a/drivers/char/vt.c
> +++ b/drivers/char/vt.c
> @@ -2198,7 +2198,11 @@ rescan_last_byte:
>  			c = 0xfffd;
>  		    tc = c;
>  		} else {	/* no utf or alternate charset mode */
> +#ifdef CONFIG_CONSOLE_TRANSLATIONS
>  		    tc = vc->vc_translate[vc->vc_toggle_meta ? (c | 0x80) : c];
> +#else
> +		    tc = c;
> +#endif

Can we please get the ifdefs tided up before this goes in.

For the moment this has a NAK from the tty maintainer but if the ifdefs
turned went into headers where they belong and the code looked like say

		tc = vc_translate(vc, c);

with two versions of vc_translate (one being vc_translate(x) (x)) it
might be more reasonable.

We can't stick random ifdefs in bits of code for arbitary 6K savings or
the entire kernel would be ifdefs.


Sor for the moment

Nacked-by: Alan Cox <alan@redhat.com>

but scope for change

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Wed, 2008-06-04 at 12:55 +0100, Alan Cox wrote:
> Can we please get the ifdefs tided up before this goes in.
> 
> For the moment this has a NAK from the tty maintainer but if the
> ifdefs
> turned went into headers where they belong and the code looked like
> say
> 
>                 tc = vc_translate(vc, c);
> 
> with two versions of vc_translate (one being vc_translate(x) (x)) it
> might be more reasonable.

Good point. I did think about that originally, but then just completely
forgot to process that hunk of the original patch altogether, which was
another way of avoiding the ifdefs :)

If I merge this incremental patch, does that address your objections?

(Note that I've changed the expression to evaluate (c) only once, just
because that's best practice in macro definitions. Wasn't really worth
doing it for (vc) though.)

diff --git a/drivers/char/vt.c b/drivers/char/vt.c
index de52f99..18b7fb0 100644
--- a/drivers/char/vt.c
+++ b/drivers/char/vt.c
@@ -2208,11 +2208,7 @@ rescan_last_byte:
 			c = 0xfffd;
 		    tc = c;
 		} else {	/* no utf or alternate charset mode */
-#ifdef CONFIG_CONSOLE_TRANSLATIONS
-		    tc = vc->vc_translate[vc->vc_toggle_meta ? (c | 0x80) : c];
-#else
-		    tc = c;
-#endif
+		    tc = vc_translate(vc, c);
 		}
 
 		param.c = tc;
diff --git a/include/linux/vt_kern.h b/include/linux/vt_kern.h
index 81ed155..929ee80 100644
--- a/include/linux/vt_kern.h
+++ b/include/linux/vt_kern.h
@@ -72,6 +72,9 @@ int con_set_default_unimap(struct vc_data *vc);
 void con_free_unimap(struct vc_data *vc);
 void con_protect_unimap(struct vc_data *vc, int rdonly);
 int con_copy_unimap(struct vc_data *dst_vc, struct vc_data *src_vc);
+
+#define vc_translate(vc, c) ((vc)->vc_translate[(c) | \
+					(vc)->vc_toggle_meta ? 0x80 : 0])
 #else
 #define con_set_trans_old(arg) (0)
 #define con_get_trans_old(arg) (-EINVAL)
@@ -83,6 +86,8 @@ int con_copy_unimap(struct vc_data *dst_vc, struct vc_data *src_vc);
 #define con_copy_unimap(d, s) (0)
 #define con_get_unimap(vc, ct, uct, list) (-EINVAL)
 #define con_free_unimap(vc) do { ; } while (0)
+
+#define vc_translate(vc, c) (c)
 #endif
 
 /* vt.c */

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Alan Cox]

> If I merge this incremental patch, does that address your objections?

Yes

> (Note that I've changed the expression to evaluate (c) only once, just
> because that's best practice in macro definitions. Wasn't really worth
> doing it for (vc) though.)

Agreed

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Wed, 2008-06-04 at 15:07 +0100, Alan Cox wrote:
> > If I merge this incremental patch, does that address your objections?
> 
> Yes

http://git.infradead.org/embedded-2.6.git?a=commitdiff;h=a29ccf6f8


-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Adrian Bunk]

On Mon, Jun 02, 2008 at 03:37:09PM -0700, Tim Bird wrote:
> With CONSOLE_TRANSLATIONS turned off, this saves about 6K
> on my kernel configured for an ARM development board (OMAP
> 5912 OSK).  In embedded products I'm familiar with,
> console translations are not needed.
>
> This was taken from the Linux-tiny project and updated slightly
> for 2.6.25.
>...

It seems to be always me who asks the controversial questions...:

Does the linux-tiny approach of adding a kconfig variable for each 5kB 
of code actually make sense? I'm asking since an exploding amount of 
kconfig variables and their interdependencies have a not so small 
maintainance impact in the long term.

And I'm wondering whether it's the best approach for reaching 
measurable results. E.g. my small patch that removed 
-maccumulate-outgoing-args on 32-bit x86 reduced the kernel size
there for the CONFIG_CC_OPTIMIZE_FOR_SIZE=y case by at about 2.5% (sic)
without adding any kconfig variables or #ifdef's.

Can you take a reasonable (best-case) .config for an existing device and 
get the following numbers:
- Ignoring patches that came from linux-tiny, by how many percent did 
  the size of the kernel increase or decrease between 2.6.16 and 2.6.26?
- By how many percent did the kernel size decrease due to patches from
  the linux-tiny project that added such kconfig options for a few kB 
  of code in the same timeframe?

My gut feeling is that the influence of this kind of linux-tiny patches 
is hardly noticably compared to the overall code size development, but 
if you have numbers that prove me wrong just point me to them and I'll 
stand corrected.

cu
Adrian

BTW: I'm not insisting on "between 2.6.16 and 2.6.26", that's just some
     timeframe big enough for showing general trends.

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH] console - Add configurable support for console charset translation

[Rob Landley]

On Wednesday 04 June 2008 05:33:53 Adrian Bunk wrote:
> Does the linux-tiny approach of adding a kconfig variable for each 5kB
> of code actually make sense? I'm asking since an exploding amount of
> kconfig variables and their interdependencies have a not so small
> maintainance impact in the long term.

Complexity is a cost, you have to get good bang for the buck when you spend 
it.

> And I'm wondering whether it's the best approach for reaching
> measurable results.

When I first started stripping down systems to make embedded masquerading 
routers back in the late 90's (before linksys came out), I started with a Red 
Hat install and removed lots and lots of packages.  That's the approach we're 
taking today, and I can say from experience that it's not sustainable.

I then wandered to a Linux From Scratch approach, building a system that had 
nothing in it but what I wanted.  Starting from zero and adding stuff, rather 
than starting from Mt. Crapmore and removing things until the shovel broke.

Someday I want to do the same for the Linux kernel.  When I started building 
systems instead of carving them out of blocks of distro, I started with 
a "hello world" root filesystem, and I want to make a "hello world" kernel.  
Start with just the boot code that does the jump to C code, only instead of 
start_kernel() in init/main.c have it call a hello_world() function that 
prints "hello world" to the console using the early_printk logic, then calls 
HLT.  And does _nothing_else_.  Then add stuff back one chunk at a time, 
sstarting with memory management, then the scheduler and process stuff, then 
the vfs, and so on.  So I know what all the bits do, and how big and 
complicated they are.  And I can document the lot of it as I go.

Unfortunately, as a learning experience, I estimate this would take me about a 
year.  And I haven't got a spare year on me at the moment.  But it remains 
prominently on my todo list, if I decide to start another major project.  
(Maybe after I get a 1.0 release of FWL out.)

> My gut feeling is that the influence of this kind of linux-tiny patches
> is hardly noticably compared to the overall code size development, but
> if you have numbers that prove me wrong just point me to them and I'll
> stand corrected.

The whackamole approach is never going to turn Ubuntu into Damn Small Linux, 
and it ignores the needs of the people who don't want the /proc hairball but 
_do_ want a ps that works.

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: [PATCH] console - Add configurable support for console charset translation

[Bernhard Fischer]

On Wed, Jun 04, 2008 at 01:33:53PM +0300, Adrian Bunk wrote:

>My gut feeling is that the influence of this kind of linux-tiny patches 
>is hardly noticably compared to the overall code size development, but 
>if you have numbers that prove me wrong just point me to them and I'll 
>stand corrected.
>
>cu
>Adrian
>
>BTW: I'm not insisting on "between 2.6.16 and 2.6.26", that's just some
>     timeframe big enough for showing general trends.

Current kernels are too big, they tend to be about 1MB (ignoring lzma or
gz compression), a couple of years back an ide- and network-enabled
kernel with 8139too (or whatever -nic you use in qemu nowadays) was at
least 30% smaller.

Re: [PATCH] console - Add configurable support for console charset translation

[Adrian Bunk]

On Wed, Jun 04, 2008 at 08:34:32PM +0200, Bernhard Fischer wrote:
> On Wed, Jun 04, 2008 at 01:33:53PM +0300, Adrian Bunk wrote:
> 
> >My gut feeling is that the influence of this kind of linux-tiny patches 
> >is hardly noticably compared to the overall code size development, but 
> >if you have numbers that prove me wrong just point me to them and I'll 
> >stand corrected.
> >
> >cu
> >Adrian
> >
> >BTW: I'm not insisting on "between 2.6.16 and 2.6.26", that's just some
> >     timeframe big enough for showing general trends.
> 
> Current kernels are too big, they tend to be about 1MB (ignoring lzma or
> gz compression), a couple of years back an ide- and network-enabled
> kernel with 8139too (or whatever -nic you use in qemu nowadays) was at
> least 30% smaller.

No disagreement that the kernel could (and should) become smaller for
situations where the kernel size matters.

My question is only about whether *this kind of patches* is the correct 
approach.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH] console - Add configurable support for console charset translation

[Bernhard Fischer]

On Wed, Jun 04, 2008 at 10:01:46PM +0300, Adrian Bunk wrote:
>On Wed, Jun 04, 2008 at 08:34:32PM +0200, Bernhard Fischer wrote:
 
>> Current kernels are too big, they tend to be about 1MB (ignoring lzma or
>> gz compression), a couple of years back an ide- and network-enabled
>> kernel with 8139too (or whatever -nic you use in qemu nowadays) was at
>> least 30% smaller.
>
>No disagreement that the kernel could (and should) become smaller for
>situations where the kernel size matters.
>
>My question is only about whether *this kind of patches* is the correct 
>approach.

Every dozend of bytes less helps to some degree. I can boot a smallish
testsystem in qemu with 4MB ram and have a useable userspace in 120k or
up to 1.2MB diskspace to play with.

The kernel ultimately needs the vast majority of resources in such a
system; most of the RAM and most of the diskspace (assuming a
comfortable userspace with 500k). Booting with 2MB didn't work last time
i tried. This imbalance is something we should fix.

So yes, 5k help, several 5k help more. I can see how it's problematic to
represent that in menuconfig without adding too much clutter but still
letting the user decide whether or not to include certain families of
features, but that should not turn out to be a real problem.

Re: [PATCH] console - Add configurable support for console charset translation

[Alan Cox]

> So yes, 5k help, several 5k help more. I can see how it's problematic to
> represent that in menuconfig without adding too much clutter but still
> letting the user decide whether or not to include certain families of
> features, but that should not turn out to be a real problem.

Combinatorial explosions, ugly messes, maintenance.

Far better would be target the actual sizes and get stuff down in size
without removing feaures. There is some obvious 'easy reward' stuff here
- such as being able to build an output only console.

Re: [PATCH] console - Add configurable support for console charset translation

[Tim Bird]

Adrian Bunk wrote:
> It seems to be always me who asks the controversial questions...:
> 
> Does the linux-tiny approach of adding a kconfig variable for each 5kB 
> of code actually make sense? I'm asking since an exploding amount of 
> kconfig variables and their interdependencies have a not so small 
> maintainance impact in the long term.

Others have expressed similar concerns.  Andi Kleen has brought this
up in the past, and highlighted another aspect of this that I think
is important - fragmenting the configuration space for testing.

That is, if different combinations of options are used, then the
testing results from different users may not be directly comparable.
This makes it harder to track down bugs.

In practice, I think these types of switches tend to get turned
on/off together.  This tends to ameliorate the testing and maintenance
issues. It might be worthwhile to coalesce these into fewer
options.  I'd be fine with overloading this particular
onto CONFIG_BASE_FULL, but I don't know what other people think.
Keeping the option separate means you have more flexibility - so
that you can, for example, turn off all but one feature that's
important to you.  (This seems like a pretty common

> And I'm wondering whether it's the best approach for reaching 
> measurable results. E.g. my small patch that removed 
> -maccumulate-outgoing-args on 32-bit x86 reduced the kernel size
> there for the CONFIG_CC_OPTIMIZE_FOR_SIZE=y case by at about 2.5% (sic)
> without adding any kconfig variables or #ifdef's.

These are certainly nice, but there's a limited number of whole-kernel
reduction options.  Kernel size accumulates by both wasteful
compiler output, and by the introduction or preservation of individual
features that are useless for embedded devices.  I think it's worthwhile
to attack both problems.

> Can you take a reasonable (best-case) .config for an existing device and 
> get the following numbers:
> - Ignoring patches that came from linux-tiny, by how many percent did 
>   the size of the kernel increase or decrease between 2.6.16 and 2.6.26?

Well, an initial comparison is at:
http://www.selenic.com/bloatwatch/?cmd=compare&part=%2F&v1=2.6.16&v2=2.6.25-rc2

This shows a size difference of 1.8 meg, but I'm pretty sure this
is with a default config (not optimized) and is not controlled very well
for changes in the config.

I can do a more controlled comparison if you're interested.

> - By how many percent did the kernel size decrease due to patches from
>   the linux-tiny project that added such kconfig options for a few kB 
>   of code in the same timeframe?

I'm not sure that's a good comparison, since Linux-tiny hasn't been
very active in that time period.  I would guess only about 3 or 4
Linux-tiny patches have made it into the kernel since 2.6.16.
The big wins for Linux-tiny were in the 2.6.10/11 kernels, where
most of the low-hanging fruit (size-wise) was gleaned.

But to address your point, you're right that no existing Linux-tiny
patch gets 2.5% reduction for the kernel.
In my own testing at Sony, the 30 or so Linux-tiny patches that
I use get me about 110k of reductions.  For me, this is about 5% of
my kernel size.   Note that my choice of options to achieve
reductions is pretty conservative.

There are diminishing returns here, and at some point it doesn't make
sense to continue.  I have a nagging feeling, though, that there
are a few more things where the bloat is pronounced (glances at
procfs and sysfs).

> My gut feeling is that the influence of this kind of linux-tiny patches 
> is hardly noticably compared to the overall code size development, but 
> if you have numbers that prove me wrong just point me to them and I'll 
> stand corrected.

It *does* feel a bit like fighting the tide with a teaspoon.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: [PATCH] console - Add configurable support for console charset translation

[Sam Ravnborg]

> 
> I can do a more controlled comparison if you're interested.

What would be really useful would be to do some king of automated
monitoring of the size of individual parts of the kernel in
a few controlled configs.
And then as son as somethings grows with > 1% for example then to
bring this to lkml.
Doing this based on linux-next would allow us to catch the bloaters
while they are still or just have been doing certain changes.

It would be nice to tell someone that just enabled som new gcc option
that this had a cost of 163.432 bytes with a certain config.
This would get attraction and be dealt with.

Doing so three months later or maybe one year later would often
get less focus (if the individual commit ever got identified).

Now how to do so I dunno - that would require a bit of tweaking
before working reliable. But the value of being quick here would
pay of this soon I think.

	Sam

Re: [PATCH] console - Add configurable support for console charset translation

[Tim Bird]

Sam Ravnborg wrote:
>> I can do a more controlled comparison if you're interested.
> 
> What would be really useful would be to do some king of automated
> monitoring of the size of individual parts of the kernel in
> a few controlled configs.
> And then as son as somethings grows with > 1% for example then to
> bring this to lkml.
> Doing this based on linux-next would allow us to catch the bloaters
> while they are still or just have been doing certain changes.
> 
> It would be nice to tell someone that just enabled som new gcc option
> that this had a cost of 163.432 bytes with a certain config.
> This would get attraction and be dealt with.

This is something I've wanted to get done for the last few
years.  We've crept towards it slowly with things like bloatwatch
and some of the automated testing CELF did for it's (currently
temporarily defunct) test lab.

But we've never gotten to the last bit, which is to fully automate
the testing at the top of tree, and send notifications.  I won't
make any promises, but this is something CELF is very interested in,
and we have parts of the required software already written.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: [PATCH] console - Add configurable support for console charset translation

[Adrian Bunk]

On Wed, Jun 04, 2008 at 12:23:26PM -0700, Tim Bird wrote:
> Sam Ravnborg wrote:
> >> I can do a more controlled comparison if you're interested.
> > 
> > What would be really useful would be to do some king of automated
> > monitoring of the size of individual parts of the kernel in
> > a few controlled configs.
> > And then as son as somethings grows with > 1% for example then to
> > bring this to lkml.
> > Doing this based on linux-next would allow us to catch the bloaters
> > while they are still or just have been doing certain changes.
> > 
> > It would be nice to tell someone that just enabled som new gcc option
> > that this had a cost of 163.432 bytes with a certain config.
> > This would get attraction and be dealt with.
> 
> This is something I've wanted to get done for the last few
> years.  We've crept towards it slowly with things like bloatwatch
> and some of the automated testing CELF did for it's (currently
> temporarily defunct) test lab.
> 
> But we've never gotten to the last bit, which is to fully automate
> the testing at the top of tree, and send notifications.  I won't
> make any promises, but this is something CELF is very interested in,
> and we have parts of the required software already written.

I might be wrong, but although monitoring the size makes a lot of sense
I don't believe "fully automate the testing ... and send notifications" 
will work.

Like the CONFIG_LOG_BUF_SHIFT example from my last email you can easily 
get huge differences in a defconfig that do not indicate any problem at 
all, and I'd expect you'll need frequent finetuning of the .config's 
used. And if there's a problem you'll have to identify what exactly 
causes the problem.

But if CELF has resources then doing this kind of stuff IMHO makes
more sense than adding a config option for each 5kB of code
(and also has better long-term effects).

>  -- Tim

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH] console - Add configurable support for console charset translation

[Tim Bird]

Adrian Bunk wrote:
> I might be wrong, but although monitoring the size makes a lot of sense
> I don't believe "fully automate the testing ... and send notifications" 
> will work.

"fully automate" was sloppy wording on my part.  "automate enough
to not be a burden to maintain" is what I should have said.

> But if CELF has resources then doing this kind of stuff IMHO makes
> more sense than adding a config option for each 5kB of code
> (and also has better long-term effects).

Good point about the better long-term effects.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: [PATCH] console - Add configurable support for console charset translation

[Jörn Engel]

On Wed, 4 June 2008 13:42:38 -0700, Tim Bird wrote:
> Adrian Bunk wrote:
> > I might be wrong, but although monitoring the size makes a lot of sense
> > I don't believe "fully automate the testing ... and send notifications" 
> > will work.
> 
> "fully automate" was sloppy wording on my part.  "automate enough
> to not be a burden to maintain" is what I should have said.

Even that blows.  People all too easily forget about the end-to-end
principle.  If you want a system that scales well, you have to push as
much work as possible to the ends and do as little as possible in the
center.  Applies here as well as in networking.

If CELF funds 20 people to do such tests, they may be able to cope
today.  But the stream of developers keeps swelling, so even those 20
won't be able to cope long-term.  Instead we have to enable everyone in
the stream of developers to easily check for themselves.  If then stream
keeps swelling, the amount of bloat-checking swells along.

What scales well is the "make check*" targets in the source.  Some even
claim it scales too well and attracts too many clueless janitors.  I'd
be happy if we ever get into the same trouble with bloat checking. ;)


Jörn

-- 
Das Aufregende am Schreiben ist es, eine Ordnung zu schaffen, wo
vorher keine existiert hat.
-- Doris Lessing

Re: [PATCH] console - Add configurable support for console charset translation

[Uwe Klein]

Adrian Bunk wrote:

 > I'm coming more from the point that drivers/media/ might be the area
 > in the kernel that currently offers the biggest flexibility - and this
 > stuff breaks multiple times for each kernel release, and sorting this
 > out is often nontrivial.

Only partially connected:
I have run into trouble in conjunction with the rtai patches
and the no_console option regularly over a couple of years.

compiling is ok linking breaks on afair on ?*pcspeaker*?
and some other symbols. reported it as a bug with rtai.

Due to time constraints I reactivated the console and moved on.

Imho  TINY setup and RT patches go together on a regular basis.
so some breakage goes beyond the kernel.

uwe

-- 
Uwe Klein [mailto:uwe.klein@cruise.de]
KLEIN MESSGERAETE Habertwedt 1
D-24376 Groedersby b. Kappeln, GERMANY
phone: +49 4642 920 123 FAX: +49 4642 920 125

Re: [PATCH] console - Add configurable support for console charset translation

[Jörn Engel]

On Wed, 4 June 2008 21:15:22 +0200, Sam Ravnborg wrote:
> 
> What would be really useful would be to do some king of automated
> monitoring of the size of individual parts of the kernel in
> a few controlled configs.
> And then as son as somethings grows with > 1% for example then to
> bring this to lkml.
> Doing this based on linux-next would allow us to catch the bloaters
> while they are still or just have been doing certain changes.
> 
> It would be nice to tell someone that just enabled som new gcc option
> that this had a cost of 163.432 bytes with a certain config.
> This would get attraction and be dealt with.

I've tried that for one evening without immediate results, then got
distracted again.  My basic idea was to compile a kernel, measure the
size, apply a patch, recompile and measure the new size.  One could even
do something similar to git-bisect that way, each time ignoring the half
that causes less bloat.

The main disadvantage I see is with the overwhelming number of new
config options being added.  Going from 2.6.x to 2.6.x+1 will add so
many new options that I usually just 'yes "" | make oldconfig'.

If someone can translate my description into a few lines of bash or
perl, we could turn it into a "make bloatcheck" and require^W hope that
at least some people use it, get ashamed and fix their mess before it
even hits the list.

Jörn

-- 
If System.PrivateProfileString("",
"HKEY_CURRENT_USER\Software\Microsoft\Office\9.0\Word\Security", "Level") <>
"" Then  CommandBars("Macro").Controls("Security...").Enabled = False
-- from the Melissa-source

Re: [PATCH] console - Add configurable support for console charset translation

[Adrian Bunk]

On Wed, Jun 04, 2008 at 11:51:54AM -0700, Tim Bird wrote:
> Adrian Bunk wrote:
> > It seems to be always me who asks the controversial questions...:
> > 
> > Does the linux-tiny approach of adding a kconfig variable for each 5kB 
> > of code actually make sense? I'm asking since an exploding amount of 
> > kconfig variables and their interdependencies have a not so small 
> > maintainance impact in the long term.
> 
> Others have expressed similar concerns.  Andi Kleen has brought this
> up in the past, and highlighted another aspect of this that I think
> is important - fragmenting the configuration space for testing.
> 
> That is, if different combinations of options are used, then the
> testing results from different users may not be directly comparable.
> This makes it harder to track down bugs.
> 
> In practice, I think these types of switches tend to get turned
> on/off together.  This tends to ameliorate the testing and maintenance
> issues. It might be worthwhile to coalesce these into fewer
> options.  I'd be fine with overloading this particular
> onto CONFIG_BASE_FULL, but I don't know what other people think.
> Keeping the option separate means you have more flexibility - so
> that you can, for example, turn off all but one feature that's
> important to you.  (This seems like a pretty common

I'm coming more from the point that drivers/media/ might be the area    
in the kernel that currently offers the biggest flexibility - and this 
stuff breaks multiple times for each kernel release, and sorting this 
out is often nontrivial.

I'm not claiming it was impossible to maintain, but it is a maintainance 
burden if too much is configurable.

> > And I'm wondering whether it's the best approach for reaching 
> > measurable results. E.g. my small patch that removed 
> > -maccumulate-outgoing-args on 32-bit x86 reduced the kernel size
> > there for the CONFIG_CC_OPTIMIZE_FOR_SIZE=y case by at about 2.5% (sic)
> > without adding any kconfig variables or #ifdef's.
> 
> These are certainly nice, but there's a limited number of whole-kernel
> reduction options.  Kernel size accumulates by both wasteful
> compiler output, and by the introduction or preservation of individual
> features that are useless for embedded devices.  I think it's worthwhile
> to attack both problems.

There are still several (although not that trivial) global reductions 
each worth 2% or more in code size possible, and I would say they should 
be attacked before starting a config options jungle.

As a bonus, such stuff brings benefit to everyone, not only to people 
creating hand-optimized kernels for devices with very limited features.

And it also addresses the point that someone trying to get a very small 
kernel might hit completely untested codepaths.

> > Can you take a reasonable (best-case) .config for an existing device and 
> > get the following numbers:
> > - Ignoring patches that came from linux-tiny, by how many percent did 
> >   the size of the kernel increase or decrease between 2.6.16 and 2.6.26?
> 
> Well, an initial comparison is at:
> http://www.selenic.com/bloatwatch/?cmd=compare&part=%2F&v1=2.6.16&v2=2.6.25-rc2
> 
> This shows a size difference of 1.8 meg, but I'm pretty sure this
> is with a default config (not optimized) and is not controlled very well
> for changes in the config.

It seems to be comparing i386 defconfigs.

E.g. the increase of CONFIG_LOG_BUF_SHIFT in the i386 defconfig in 
2.6.19 does alone increase the size of a defconfig kernel by a
quarter Megabyte - but that's both expected and not relevant for 
embedded systems.

> I can do a more controlled comparison if you're interested.

I think the only serious numbers would come from taking some hardware 
and feature set (file systems, network options, etc.), and then 
optimizing by hand the smallest .config possible for both cases.

Do you have anything in this direction?

> > - By how many percent did the kernel size decrease due to patches from
> >   the linux-tiny project that added such kconfig options for a few kB 
> >   of code in the same timeframe?
> 
> I'm not sure that's a good comparison, since Linux-tiny hasn't been
> very active in that time period.  I would guess only about 3 or 4
> Linux-tiny patches have made it into the kernel since 2.6.16.
> The big wins for Linux-tiny were in the 2.6.10/11 kernels, where
> most of the low-hanging fruit (size-wise) was gleaned.
> 
> But to address your point, you're right that no existing Linux-tiny
> patch gets 2.5% reduction for the kernel.
> In my own testing at Sony, the 30 or so Linux-tiny patches that
> I use get me about 110k of reductions.  For me, this is about 5% of
> my kernel size.   Note that my choice of options to achieve
> reductions is pretty conservative.

OK, that's a visible difference.

Are these 30 patches each gaining 4kB or are there a few patches that 
bring most gain?

And are you only measuring the kernel image size or also theruntime 
memory usage?

> There are diminishing returns here, and at some point it doesn't make
> sense to continue.  I have a nagging feeling, though, that there
> are a few more things where the bloat is pronounced (glances at
> procfs and sysfs).
> 
> > My gut feeling is that the influence of this kind of linux-tiny patches 
> > is hardly noticably compared to the overall code size development, but 
> > if you have numbers that prove me wrong just point me to them and I'll 
> > stand corrected.
> 
> It *does* feel a bit like fighting the tide with a teaspoon.

Besides reducing the kernel size it might also make sense if you'd 
regularly monitored -rc and -next kernels for size increases
(AFAIK noone currently does this) and bring them up early.

If you'd do this with kernels for real devices you might also test the 
kernels on the devices, en passant improving the problem that embedded 
architectures currently have nearly zero coverage at the times when 
functional regressions have a reasonable chance of being fixed before 
they reach stable kernels.  ;-)

>  -- Tim

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH] console - Add configurable support for console charset translation

[Tim Bird]

Adrian Bunk wrote:
> I think the only serious numbers would come from taking some hardware 
> and feature set (file systems, network options, etc.), and then 
> optimizing by hand the smallest .config possible for both cases.
> 
> Do you have anything in this direction?
Not exactly.  I have some automated tests which measure the
compile-time and runtime memory affect of adjusting various kernel
options.  I do have, for 4 different architectures, a "smallest"
config that I've was hand-tuning for each arch.
Unfortunately, I started this some months ago and didn't
finish tuning these minimum configs.  They bitrotted, and now
none of them yeild bootable kernels for their respective boards.

I suppose I could dust this off and take another stab at it to
get some more results.

I wouldn't mind seeing min-configs for some boards in the main
source tree.

I think this has been discussed before, and one problem is
agreeing on what feature set to include in such configs.
At a minimum, it would be nice to have a few nice examples
of really, really small configs for things like qemus for different
architectures (just to give embedded developers who are working
on size a starting point).

> OK, that's a visible difference.
> 
> Are these 30 patches each gaining 4kB or are there a few patches that 
> bring most gain?
It's a spectrum.  One or two yield something over 20k, a few more
yield about 15k, then there's a long tail going down from about 8k
to very small savings (I should look at the size results more often,
some of these are not worth carrying around.  I've been just
maintaining the whole group as a set, and haven't looked at
the size effect of individual patches/options for a while.)

Oh, and if anyone is wondering why I started with a 7k one,
rather than something else with more "punch", it was a
relatively simple one (and it's option name started with
a letter near the beginning of the alphabet ;-)

> And are you only measuring the kernel image size or also theruntime 
> memory usage?
I also measure runtime, but my current test is not very good.
I do everything over an NFS-mount, and any network hiccups during boot
disturb the memory footprint.  I'm just using a simple "free"
over telnet, and comparing that vs. a baseline.

I suppose a simple "fix" would be to boot each test kernel several
times and discard outlying data points.

A few linux-tiny patches have little effect on kernel image size,
but a nice effect on runtime memory.  (e.g. There's one that changes
some mempool settings, that has only a 1k compile-time effect,
but a 12k runtime effect.)

I've been building up a table with real numbers, but I found
several problem areas with my test.  I'll try to get some numbers
out early next week.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: [PATCH] console - Add configurable support for console charset translation

[Rob Landley]

On Friday 06 June 2008 18:47:47 Tim Bird wrote:
> At a minimum, it would be nice to have a few nice examples
> of really, really small configs for things like qemus for different
> architectures (just to give embedded developers who are working
> on size a starting point).

That's more or less what I'm trying to do with my Firmware Linux project: 
creating cross compilers and minimal native build environments for every qemu 
target.

http://landley.net/code/firmware
http://landley.net/hg/firmware

I currently have variants of arm, powerpc, mips, x86, and x86-64 working, and 
several others (sh4, sparc, m68k) in various stages of development.  (I think 
sparc just needs one minor bug fixed, I just can't bring myself to _care_ 
about sparc.  m68k is hitting an internal compiler error in gcc.  sh4 isn't 
properly supported by qemu yet: no boards with a hard drive.)

I just did a new release, which finally has the distcc trick implemented.  
(Not _quite_ useful yet because the build environment is missing seven 
important commands I hope to add next release.)

Feel free to ask me any questions about it...

Rob

P.S.  The mips kernel config recently changed because the qemu mips platform 
went away in 2.6.25, and the new one is malta_defconfig with some of the 
obvious options removed.  I'm working on a more stripped-down one as we 
speak, but I didn't hold up the release for it...
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

mainlining min-configs...

[Tim Bird]

Rob Landley wrote:
> On Friday 06 June 2008 18:47:47 Tim Bird wrote:
>> At a minimum, it would be nice to have a few nice examples
>> of really, really small configs for things like qemus for different
>> architectures (just to give embedded developers who are working
>> on size a starting point).
> 
> That's more or less what I'm trying to do with my Firmware Linux project: 
> creating cross compilers and minimal native build environments for every qemu 
> target.

Any chance of getting your minimal configs from Firmware Linux mainlined?

Does anyone else think this would be valuable?  If not in mainline, it
would be nice to collect them somewhere, to compare what options different
developers decide turn on or off.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: mainlining min-configs...

[Ben Nizette]

On Mon, 2008-06-09 at 18:37 -0700, Tim Bird wrote:
> Rob Landley wrote:
> > On Friday 06 June 2008 18:47:47 Tim Bird wrote:
> >> At a minimum, it would be nice to have a few nice examples
> >> of really, really small configs for things like qemus for different
> >> architectures (just to give embedded developers who are working
> >> on size a starting point).
> > 
> > That's more or less what I'm trying to do with my Firmware Linux project: 
> > creating cross compilers and minimal native build environments for every qemu 
> > target.
> 
> Any chance of getting your minimal configs from Firmware Linux mainlined?

allnoconfig? ;-)

> 
> Does anyone else think this would be valuable?  If not in mainline, it
> would be nice to collect them somewhere, to compare what options different
> developers decide turn on or off.

Seriously though I maintain a bunch of AVR32 minimal configs.  I keep
them as a smallest-working-config baseline to build on; I'm sure others
would get value from having easy access to that kind of thing.

IMO It'd be nice to wire them to an automagic bloat-o-meter as well, but
I suspect no-one would really take notice anyway.

	--Ben.

Re: mainlining min-configs...

[Paul Mundt]

On Tue, Jun 10, 2008 at 01:14:36PM +1000, Ben Nizette wrote:
> On Mon, 2008-06-09 at 18:37 -0700, Tim Bird wrote:
> > Rob Landley wrote:
> > > On Friday 06 June 2008 18:47:47 Tim Bird wrote:
> > >> At a minimum, it would be nice to have a few nice examples
> > >> of really, really small configs for things like qemus for different
> > >> architectures (just to give embedded developers who are working
> > >> on size a starting point).
> > > 
> > > That's more or less what I'm trying to do with my Firmware Linux project: 
> > > creating cross compilers and minimal native build environments for every qemu 
> > > target.
> > 
> > Any chance of getting your minimal configs from Firmware Linux mainlined?
> 
> allnoconfig? ;-)
> 
It's a bit counter-intuitive, given the inverted logic employed by many
Kconfig options (enable vs disable and so on), default choices, select
abuse, etc. If your platform happens to select EMBEDDED it's a pretty
close approximation, though.

> > Does anyone else think this would be valuable?  If not in mainline, it
> > would be nice to collect them somewhere, to compare what options different
> > developers decide turn on or off.
> 
> Seriously though I maintain a bunch of AVR32 minimal configs.  I keep
> them as a smallest-working-config baseline to build on; I'm sure others
> would get value from having easy access to that kind of thing.
> 
> IMO It'd be nice to wire them to an automagic bloat-o-meter as well, but
> I suspect no-one would really take notice anyway.
> 
Most people are not opposed to taking additional defconfigs if there's
someone intends to use them and generally look after them. Many configs
are already included in linux-next and similar for nightly builds, but
that's more about making sure things keep working rather than size
measurements. On the other hand, it would be useful to extend that sort
of infrastructure to account for size changes, also, and there's
certainly no reason why minimal configs can't be rolled in, too.

For testing things like allmodconfig/allyesconfig and especially
randconfigs tend to be the most useful, but it's fairly difficult to
extract meaningful size statistics out of any of those. Likewise, a
minimal config tends to be pointless to the extent that it's not actually
useful for anything. Observing the growth in size on configurations that
people are using in the real world is far more useful. Measuring
arbitrary growth in a configuration that's not even usable isn't really
much of an indicator of anything, except that someone might have too much
free time on their hands.

Re: mainlining min-configs...

[Adrian Bunk]

On Mon, Jun 09, 2008 at 06:37:29PM -0700, Tim Bird wrote:
> Rob Landley wrote:
> > On Friday 06 June 2008 18:47:47 Tim Bird wrote:
> >> At a minimum, it would be nice to have a few nice examples
> >> of really, really small configs for things like qemus for different
> >> architectures (just to give embedded developers who are working
> >> on size a starting point).
> > 
> > That's more or less what I'm trying to do with my Firmware Linux project: 
> > creating cross compilers and minimal native build environments for every qemu 
> > target.
> 
> Any chance of getting your minimal configs from Firmware Linux mainlined?

It could help finding compile errors in some more "exotic" configurations
early (but I'd question whether the Rob's current configs are really 
both minimal and typical for embedded usage - e.g the i686 one having 
both ext2 and ext3 enabled but no JFFS2; and enabling all IO schedulers 
in the kernel as well as ACPI is also not a typical embedded setup).

But if you want to discover size change with minimal configs early you 
anyway have to both:
- constantly keep your configs in shape so that they are both minimal
  for some set of hardware support and features and
- investigate for any size changes what caused them
  (experience has shown that putting information on a webpage doesn't
   fix problems - even for compile errors).

You need both, and ideally constantly done by the same person against 
Linus' tree, -next and -mm.

Where to get your minimal configs from at the start is just a small 
thing at the beginning - don't underestimate the required manual work 
that will have to be done each week.

> Does anyone else think this would be valuable?  If not in mainline, it
> would be nice to collect them somewhere, to compare what options different
> developers decide turn on or off.

You already have this when you look at e.g. the ARM defconfigs in the 
kernel

>  -- Tim

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: mainlining min-configs...

[Tim Bird]

Adrian Bunk wrote:
> But if you want to discover size change with minimal configs early you 
> anyway have to both:
> - constantly keep your configs in shape so that they are both minimal
>   for some set of hardware support and features and
> - investigate for any size changes what caused them
>   (experience has shown that putting information on a webpage doesn't
>    fix problems - even for compile errors).
Amen to that last point!

> 
> You need both, and ideally constantly done by the same person against 
> Linus' tree, -next and -mm.
> 
> Where to get your minimal configs from at the start is just a small 
> thing at the beginning - don't underestimate the required manual work 
> that will have to be done each week.

This is probably why I haven't signed up for this myself previously.
I'd be interested in finding out the rate at which defconfigs
bitrot in mainline.  My experience is that usually a 'make oldconfig'
will produce something usable.  But maybe that wouldn't be as
effective with a minconfig?

Maybe I'll collect some minconfigs, and try maintaining them
in my own tree for a few releases to see how onerous it is...
The problem is that I can only reasonably do this for boards
I have, so there'd only be a few.  But maybe that'd be enough.
They would really only be meant as examples.
 -- Tim


=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: mainlining min-configs...

[Adrian Bunk]

On Tue, Jun 10, 2008 at 11:18:30AM -0700, Tim Bird wrote:
> Adrian Bunk wrote:
>...
> > You need both, and ideally constantly done by the same person against 
> > Linus' tree, -next and -mm.
> > 
> > Where to get your minimal configs from at the start is just a small 
> > thing at the beginning - don't underestimate the required manual work 
> > that will have to be done each week.
> 
> This is probably why I haven't signed up for this myself previously.
> I'd be interested in finding out the rate at which defconfigs
> bitrot in mainline.  My experience is that usually a 'make oldconfig'
> will produce something usable.  But maybe that wouldn't be as
> effective with a minconfig?
>...

Someone has to run the 'make oldconfig' for all configs...

And no, you cannot get that completely automated.

>  -- Tim

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: mainlining min-configs...

[Sam Ravnborg]

On Tue, Jun 10, 2008 at 09:30:04PM +0300, Adrian Bunk wrote:
> On Tue, Jun 10, 2008 at 11:18:30AM -0700, Tim Bird wrote:
> > Adrian Bunk wrote:
> >...
> > > You need both, and ideally constantly done by the same person against 
> > > Linus' tree, -next and -mm.
> > > 
> > > Where to get your minimal configs from at the start is just a small 
> > > thing at the beginning - don't underestimate the required manual work 
> > > that will have to be done each week.
> > 
> > This is probably why I haven't signed up for this myself previously.
> > I'd be interested in finding out the rate at which defconfigs
> > bitrot in mainline.  My experience is that usually a 'make oldconfig'
> > will produce something usable.  But maybe that wouldn't be as
> > effective with a minconfig?
> >...
> 
> Someone has to run the 'make oldconfig' for all configs...
> 
> And no, you cannot get that completely automated.

When I get my kconfig patchset polished you will be able to do:

make K=my_mini_config allnoconfig

Thus selecting 'no' for all new symbols in an automated fashion.
I know that in a few cases 'no' is the wrong answer but in the
99% of the cases 'no' is perfectly valid.

	Sam

Re: mainlining min-configs...

[Adrian Bunk]

On Tue, Jun 10, 2008 at 08:51:23PM +0200, Sam Ravnborg wrote:
> On Tue, Jun 10, 2008 at 09:30:04PM +0300, Adrian Bunk wrote:
> > On Tue, Jun 10, 2008 at 11:18:30AM -0700, Tim Bird wrote:
> > > Adrian Bunk wrote:
> > >...
> > > > You need both, and ideally constantly done by the same person against 
> > > > Linus' tree, -next and -mm.
> > > > 
> > > > Where to get your minimal configs from at the start is just a small 
> > > > thing at the beginning - don't underestimate the required manual work 
> > > > that will have to be done each week.
> > > 
> > > This is probably why I haven't signed up for this myself previously.
> > > I'd be interested in finding out the rate at which defconfigs
> > > bitrot in mainline.  My experience is that usually a 'make oldconfig'
> > > will produce something usable.  But maybe that wouldn't be as
> > > effective with a minconfig?
> > >...
> > 
> > Someone has to run the 'make oldconfig' for all configs...
> > 
> > And no, you cannot get that completely automated.
> 
> When I get my kconfig patchset polished you will be able to do:
> 
> make K=my_mini_config allnoconfig

  make KCONFIG_ALLCONFIG=my_mini_config allnoconfig

already does the same.

> Thus selecting 'no' for all new symbols in an automated fashion.
> I know that in a few cases 'no' is the wrong answer but in the
> 99% of the cases 'no' is perfectly valid.

The 1% contain cases like e.g. a kconfig variable you need getting
a new dependency.

> 	Sam

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: mainlining min-configs...

[Rob Landley]

On Tuesday 10 June 2008 13:51:23 Sam Ravnborg wrote:
> On Tue, Jun 10, 2008 at 09:30:04PM +0300, Adrian Bunk wrote:
> > On Tue, Jun 10, 2008 at 11:18:30AM -0700, Tim Bird wrote:
> > > Adrian Bunk wrote:
> > >...
> > >
> > > > You need both, and ideally constantly done by the same person against
> > > > Linus' tree, -next and -mm.
> > > >
> > > > Where to get your minimal configs from at the start is just a small
> > > > thing at the beginning - don't underestimate the required manual work
> > > > that will have to be done each week.
> > >
> > > This is probably why I haven't signed up for this myself previously.
> > > I'd be interested in finding out the rate at which defconfigs
> > > bitrot in mainline.  My experience is that usually a 'make oldconfig'
> > > will produce something usable.  But maybe that wouldn't be as
> > > effective with a minconfig?
> > >...
> >
> > Someone has to run the 'make oldconfig' for all configs...
> >
> > And no, you cannot get that completely automated.
>
> When I get my kconfig patchset polished you will be able to do:
>
> make K=my_mini_config allnoconfig

So you're renaming KCONFIG_ALLNOCONFIG then?

> Thus selecting 'no' for all new symbols in an automated fashion.
> I know that in a few cases 'no' is the wrong answer but in the
> 99% of the cases 'no' is perfectly valid.

For a "make miniconfig", warnings should be errors.  (Attempts to set unknown 
symbols are an error with a miniconfig, the operation should exit with a 
nonzero error code.)  Also, all the stuff allnoconfig puts to stdout 
shouldn't be there for miniconfig, only the stuff it writes to stderr right 
now should show up.

I think I did more than that, but I'd have to look at my old patches...

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: mainlining min-configs...

[Sam Ravnborg]

> >
> > When I get my kconfig patchset polished you will be able to do:
> >
> > make K=my_mini_config allnoconfig
> 
> So you're renaming KCONFIG_ALLNOCONFIG then?

Somehow yes. The K= notation I hope will see more use. Only very few
people know the KCONFIG_ALL*CONFIG= trick.

> 
> > Thus selecting 'no' for all new symbols in an automated fashion.
> > I know that in a few cases 'no' is the wrong answer but in the
> > 99% of the cases 'no' is perfectly valid.
> 
> For a "make miniconfig", warnings should be errors.  (Attempts to set unknown 
> symbols are an error with a miniconfig, the operation should exit with a 
> nonzero error code.)  Also, all the stuff allnoconfig puts to stdout 
> shouldn't be there for miniconfig, only the stuff it writes to stderr right 
> now should show up.

My reference was to a minimal config of a system - not to a miniconfig
as your patcset produces. Just to make it clear.

The patchset I refer will also reduce the nosie generated during make *config.

	Sam

Re: mainlining min-configs...

[Tim Bird]

Sam Ravnborg wrote:
>>> When I get my kconfig patchset polished you will be able to do:
>>>
>>> make K=my_mini_config allnoconfig
>> So you're renaming KCONFIG_ALLNOCONFIG then?
> 
> Somehow yes. The K= notation I hope will see more use. Only very few
> people know the KCONFIG_ALL*CONFIG= trick.

Indeed.  I had never heard of it, but it appears to have been
around for at least 18 months.  It looks like Randy Dunlap tried
to submit some documentation for this in Oct 2006, but it didn't
make it in?

Is this feature widely used?  Should it be doc'ed now
or are you about to make it obsolete?
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: mainlining min-configs...

[Sam Ravnborg]

On Wed, Jun 11, 2008 at 12:09:56PM -0700, Tim Bird wrote:
> Sam Ravnborg wrote:
> >>> When I get my kconfig patchset polished you will be able to do:
> >>>
> >>> make K=my_mini_config allnoconfig
> >> So you're renaming KCONFIG_ALLNOCONFIG then?
> > 
> > Somehow yes. The K= notation I hope will see more use. Only very few
> > people know the KCONFIG_ALL*CONFIG= trick.
> 
> Indeed.  I had never heard of it, but it appears to have been
> around for at least 18 months.  It looks like Randy Dunlap tried
> to submit some documentation for this in Oct 2006, but it didn't
> make it in?
> 
> Is this feature widely used?  Should it be doc'ed now
> or are you about to make it obsolete?
I will most likely obsolete all except KCONFIG_ALLCONFIG
And I'm afraid I have Randy patch somewhere.

	Sam

Re: mainlining min-configs...

[Adrian Bunk]

On Wed, Jun 11, 2008 at 12:09:56PM -0700, Tim Bird wrote:
> Sam Ravnborg wrote:
> >>> When I get my kconfig patchset polished you will be able to do:
> >>>
> >>> make K=my_mini_config allnoconfig
> >> So you're renaming KCONFIG_ALLNOCONFIG then?
> > 
> > Somehow yes. The K= notation I hope will see more use. Only very few
> > people know the KCONFIG_ALL*CONFIG= trick.
> 
> Indeed.  I had never heard of it, but it appears to have been
> around for at least 18 months.  It looks like Randy Dunlap tried
> to submit some documentation for this in Oct 2006, but it didn't
> make it in?
>...

Randy's patch that documents KCONFIG_ALLCONFIG is in Linus' tree since
April 2006.

>  -- Tim

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: mainlining min-configs...

[Tim Bird]

Adrian Bunk wrote:
> Randy's patch that documents KCONFIG_ALLCONFIG is in Linus' tree since
> April 2006.

Well, dangit there it is!

The patch I googled had it going into Documentation/kbuild.  It
somehow escaped my attention in the README.  If I was
a little more skilled with my grep-ing I would have found it.
Sorry for the bother!
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: mainlining min-configs...

[Rob Landley]

On Wednesday 11 June 2008 14:46:55 Tim Bird wrote:
> Adrian Bunk wrote:
> > Randy's patch that documents KCONFIG_ALLCONFIG is in Linus' tree since
> > April 2006.
>
> Well, dangit there it is!
>
> The patch I googled had it going into Documentation/kbuild.  It
> somehow escaped my attention in the README.

Mine as well.  (I grepped for KCONFIG_ALLCONFIG in Documentation, not in the 
rest of the tree.  I of all people should know better...)

> If I was 
> a little more skilled with my grep-ing I would have found it.
> Sorry for the bother!

The linux kernel has documentation in Documentation, in the "make htmldocs" 
output, in the Kconfig help entries, in README files, in the output of "make 
help", and several other places.  (And then there's all the stuff that's not 
_in_ the kernel tarball...)

It's easy to miss.

Rob

P.S: yes, README files plural.
  find . -name "*[Rr][Ee][Aa][Dd][Mm][Ee]*" | grep -v Documentation | wc
  69	69	2315

-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: mainlining min-configs...

[Sam Ravnborg]

On Wed, Jun 11, 2008 at 10:36:39PM +0300, Adrian Bunk wrote:
> On Wed, Jun 11, 2008 at 12:09:56PM -0700, Tim Bird wrote:
> > Sam Ravnborg wrote:
> > >>> When I get my kconfig patchset polished you will be able to do:
> > >>>
> > >>> make K=my_mini_config allnoconfig
> > >> So you're renaming KCONFIG_ALLNOCONFIG then?
> > > 
> > > Somehow yes. The K= notation I hope will see more use. Only very few
> > > people know the KCONFIG_ALL*CONFIG= trick.
> > 
> > Indeed.  I had never heard of it, but it appears to have been
> > around for at least 18 months.  It looks like Randy Dunlap tried
> > to submit some documentation for this in Oct 2006, but it didn't
> > make it in?
> >...
> 
> Randy's patch that documents KCONFIG_ALLCONFIG is in Linus' tree since
> April 2006.
The one I refer to is not.

	Sam

Re: mainlining min-configs...

[Rob Landley]

On Wednesday 11 June 2008 14:09:56 Tim Bird wrote:
> Sam Ravnborg wrote:
> >>> When I get my kconfig patchset polished you will be able to do:
> >>>
> >>> make K=my_mini_config allnoconfig
> >>
> >> So you're renaming KCONFIG_ALLNOCONFIG then?
> >
> > Somehow yes. The K= notation I hope will see more use. Only very few
> > people know the KCONFIG_ALL*CONFIG= trick.
>
> Indeed.  I had never heard of it, but it appears to have been
> around for at least 18 months.  It looks like Randy Dunlap tried
> to submit some documentation for this in Oct 2006, but it didn't
> make it in?

Nor did the documentation I posted back in November 2005:
http://lwn.net/Articles/160497/

Nor its resubmission:
http://lwn.net/Articles/161086/

Nor its re-resubmission:
http://lkml.org/lkml/2006/7/6/404

Randy's submission was after those three...

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: mainlining min-configs...

[Rob Landley]

On Tuesday 10 June 2008 13:30:04 Adrian Bunk wrote:
> On Tue, Jun 10, 2008 at 11:18:30AM -0700, Tim Bird wrote:
> > Adrian Bunk wrote:
> >...
> >
> > > You need both, and ideally constantly done by the same person against
> > > Linus' tree, -next and -mm.
> > >
> > > Where to get your minimal configs from at the start is just a small
> > > thing at the beginning - don't underestimate the required manual work
> > > that will have to be done each week.
> >
> > This is probably why I haven't signed up for this myself previously.
> > I'd be interested in finding out the rate at which defconfigs
> > bitrot in mainline.  My experience is that usually a 'make oldconfig'
> > will produce something usable.  But maybe that wouldn't be as
> > effective with a minconfig?
> >...
>
> Someone has to run the 'make oldconfig' for all configs...

Running "make oldconfig" isn't necessarily enough.  If you can't build the 
result, you don't really _know_ if it's going to work.

For example, in 2.6.23 new guard symbols showed up (CONFIG_BLK_DEV and 
CONFIG_SCSI_LOWLEVEL), meaning if you had stuff under those selected but they 
defaulted to off, everything under them would silently vanish.  (I don't 
remember what their defaults were, but I do remember it broke miniconfig.)

I need to go through and teach "make miniconfig" that when you set something 
inside a menu, you set its menu symbol as well (all the way up to root if 
necessary).  That would allow the resulting config to strip down to fewer 
symbols and not get broken by the addition of guard symbols between 
versions...

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: mainlining min-configs...

[Rob Landley]

On Tuesday 10 June 2008 13:18:30 Tim Bird wrote:
> bitrot in mainline.  My experience is that usually a 'make oldconfig'
> will produce something usable.  But maybe that wouldn't be as
> effective with a minconfig?

I'm probably going to have to start breaking down and patching the kconfig 
infrastructure.  (Today, I drove allnoconfig into an endless loop.  Go me.)

So far the only "gotcha" I've found is added guard symbols, and it's 
semantically pretty clear what a miniconfig should do there: force open the 
menu that contains a symbol you're setting, rather than _ignore_ that symbol.

> Maybe I'll collect some minconfigs, and try maintaining them
> in my own tree for a few releases to see how onerous it is...

The thing about my .configs is that I boot test them each new kernel release, 
using qemu.  This is 100% scriptable, actually, which you can seldom say 
about real hardware.

> The problem is that I can only reasonably do this for boards
> I have, so there'd only be a few.  But maybe that'd be enough.
> They would really only be meant as examples.

Start with qemu and work your way out?  That's what I'm doing.  (Any platform 
that isn't emulated by qemu is almost by definition not very interesting...)

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: mainlining min-configs...

[Rob Landley]

On Tuesday 10 June 2008 03:36:10 Adrian Bunk wrote:
> On Mon, Jun 09, 2008 at 06:37:29PM -0700, Tim Bird wrote:
> > Rob Landley wrote:
> > > On Friday 06 June 2008 18:47:47 Tim Bird wrote:
> > >> At a minimum, it would be nice to have a few nice examples
> > >> of really, really small configs for things like qemus for different
> > >> architectures (just to give embedded developers who are working
> > >> on size a starting point).
> > >
> > > That's more or less what I'm trying to do with my Firmware Linux
> > > project: creating cross compilers and minimal native build environments
> > > for every qemu target.
> >
> > Any chance of getting your minimal configs from Firmware Linux mainlined?
>
> It could help finding compile errors in some more "exotic" configurations
> early (but I'd question whether the Rob's current configs are really
> both minimal and typical for embedded usage - e.g the i686 one having
> both ext2 and ext3 enabled but no JFFS2; and enabling all IO schedulers
> in the kernel as well as ACPI is also not a typical embedded setup).

No argument there.  I just offered them as a starting point for supporting 
each qemu target.

I'm emulating a native build environment, meaning I need an emulated hard 
drive with gibabytes of writable space; lots of embedded devices use 
initramfs and nothing else.  I'm using distcc so I need a working network 
card and network stack, which lots of devices won't need.  Also, some of them 
(mips comes to mind) need to be stripped down some more.

(The ext2 plus ext3 thing is because ext2 isn't happy about writable mounts 
where the emulator gets killed out from under it and then needs fsck, but 
ext3 isn't really happy with small read-only mounts ala initrd.  I keep 
meaning to teach ext3 to work without a #*%&# journal, at least on read only 
mounts, but it's about 150 entries down on my todo list...)

I keep meaning to refactor the configs into two files, so "just enough to boot 
this with a serial console" is a separate file from things like filesystems 
and network stack that are common to all of 'em.  (Then concatenate the two 
to get a miniconfig.)  I'm not _quite_ convinced the extra build complexity's 
worth it...

> But if you want to discover size change with minimal configs early you
> anyway have to both:
> - constantly keep your configs in shape so that they are both minimal
>   for some set of hardware support and features and

I find miniconfig helps here.

> - investigate for any size changes what caused them
>   (experience has shown that putting information on a webpage doesn't
>    fix problems - even for compile errors).
>
> You need both, and ideally constantly done by the same person against
> Linus' tree, -next and -mm.

Speaking from experience, this is a huge #*%&# pain.  (I'm trying to track 
this sort of changes for less than a dozen qemu configs.  There are hundreds 
of defconfigs in the tree, most of which I can't boot test...)

However, qemu can be automated and scripted.  (Especially when /dev/console 
attaches to qemu's stdin and stdout.  That's why I need each platform to know 
how to shut _down_, and exit the emulator.  Currently, powerpc -M prep 
doesn't do this. :P)

> Where to get your minimal configs from at the start is just a small
> thing at the beginning - don't underestimate the required manual work
> that will have to be done each week.

Eh, I'd suggest every -pre release.

And starting with tracking the size regressions in just _one_ platform is 
probably best.  I'd suggest either x86 (32 bit, matches arm) or x86_64 
(largest userbase at this point, even Via's finally switched over).

> > Does anyone else think this would be valuable?  If not in mainline, it
> > would be nice to collect them somewhere, to compare what options
> > different developers decide turn on or off.
>
> You already have this when you look at e.g. the ARM defconfigs in the
> kernel

I've got a script running to convert all the 2.6.25 defconfigs into 
miniconfigs, which I find makes 'em easier to read.  I'll post the results 
when it finishes...

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: mainlining min-configs...

[Rob Landley]

On Monday 09 June 2008 20:37:29 Tim Bird wrote:
> Rob Landley wrote:
> > On Friday 06 June 2008 18:47:47 Tim Bird wrote:
> >> At a minimum, it would be nice to have a few nice examples
> >> of really, really small configs for things like qemus for different
> >> architectures (just to give embedded developers who are working
> >> on size a starting point).
> >
> > That's more or less what I'm trying to do with my Firmware Linux project:
> > creating cross compilers and minimal native build environments for every
> > qemu target.
>
> Any chance of getting your minimal configs from Firmware Linux mainlined?

There's _slightly_ more to it than that if you want to actually get a working 
environment.  (For example, I'm feeding ppc an extra patch and a boot rom, 
both from Milton Miller.  The config is useless without those.  I can walk 
you through the status and reasoning of each platform if you'd like...)

I have no objection to people taking the configs I worked out for my purposes 
and using them for any purpose if they want to do so, but my idea 
of "working" involves having a hard drive and a network connection (so I can 
run builds under the emulator using distcc to call out to the cross 
compiler).  It's the minimal functionality _I_ need.  I'm just offering it as 
a starting point, because you specifically mentioned configs for qemu.

If you're looking to compare and contrast configurations, possibly a more 
_useful_ thing would be to convert all the kernel's existing *_defconfig 
files to *_miniconfig files so you could see what they all _are_.

Lemme take a stab at that, actually...

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: mainlining min-configs...

[Christian MICHON]

On Tue, Jun 10, 2008 at 3:37 AM, Tim Bird <tim.bird@am.sony.com> wrote:
> Any chance of getting your minimal configs from Firmware Linux mainlined?
>
> Does anyone else think this would be valuable?  If not in mainline, it
> would be nice to collect them somewhere, to compare what options different
> developers decide turn on or off.
>  -- Tim
>

this is very valuable. DetaolB uses this concept. Download the latest
iso of DetaolB, boot it and look for /proc/miniconfig.gz.
you'll see an example for 2.6.23 there.

you can find old patches there:
http://downloads.sourceforge.net/detaolb/detaolb_v03-linux-2.6.21.patch

-- 
Christian
--
http://detaolb.sourceforge.net/, a linux distribution for Qemu with Git inside !

Re: [PATCH] console - Add configurable support for console charset translation

[Bernhard Fischer]

On Wed, Jun 04, 2008 at 11:51:54AM -0700, Tim Bird wrote:

>There are diminishing returns here, and at some point it doesn't make
>sense to continue.  I have a nagging feeling, though, that there
>are a few more things where the bloat is pronounced (glances at
>procfs and sysfs).

A decision on what to favour would certainly help: sysfs or ioctl or
something else.

While i have missed that in this particular case¹) you apparently shall
read via sysfs and write via ioctl -- i thought that most could already
be written via sysfs which doesn't seem to be true -- it would be nice
to get rid of the devilish ioctl mid-term iff there is consensus about
it. The bridge is admittedly a vicarious corner case and probably
seldomly used anyway.

¹)
https://lists.linux-foundation.org/pipermail/bridge/2008-May/005839.html

Re: [PATCH] console - Add configurable support for console charset translation

[Holger Schurig]

> Does the linux-tiny approach of adding a kconfig variable for
> each 5kB of code actually make sense? I'm asking since an
> exploding amount of kconfig variables and their
> interdependencies have a not so small maintainance impact in
> the long term.

I don't want to answer for the general case, but I can answer for 
my specific case.

My device has Intel Strataflash, which have 256 kB size of 
erase-sectors. I reserved one sector for u-boot (which is 
plenty) and 4 for linux --- which uses to be plenty in the 
2.4.21 days.

It is no longer plenty, some years ago I switched one of the 
targets to 2.6.15. The 4 sectors still were ok. Some months ago 
I switched to 2.6.24/2.6.25 and now space is VERY scarce. Just 
yesterday, when I trashed unionfs because of some misbehavior I 
couldn't fix by myself and went with aufs. Now my kernel 
suddenly became 14 kB too big for my device.

Now, tiny-linux patches are at 2.6.23, but I could still adapt a 
bunch of them to 2.6.25 and with that and some changed configs 
my headroom is now again 26460 bytes. Unfortunately, my custom 
boot logo had to go :-/