[PATCH] console - Add configurable support for console charset translation

[PATCH] console - Add configurable support for console charset translation

[Tim Bird]

With CONSOLE_TRANSLATIONS turned off, this saves about 6K
on my kernel configured for an ARM development board (OMAP
5912 OSK).  In embedded products I'm familiar with,
console translations are not needed.

This was taken from the Linux-tiny project and updated slightly
for 2.6.25.

  drivers/char/consolemap.c |   81 ++++++++++++++++++++++++++++++++++++++++++++++
  drivers/char/vt.c         |    4 ++
  init/Kconfig              |    7 +++
  3 files changed, 92 insertions(+)

Signed-off-by: Tim Bird <tim.bird@am.sony.com>

--- a/drivers/char/consolemap.c
+++ b/drivers/char/consolemap.c
@@ -22,6 +22,8 @@
  #include <linux/consolemap.h>
  #include <linux/vt_kern.h>

+#ifdef CONFIG_CONSOLE_TRANSLATIONS
+
  static unsigned short translations[][256] = {
    /* 8-bit Latin-1 mapped to Unicode -- trivial mapping */
    {
@@ -742,3 +744,82 @@ console_map_init(void)
  }

  EXPORT_SYMBOL(con_copy_unimap);
+
+#else
+
+u16 inverse_translate(struct vc_data *conp, int glyph, int use_unicode)
+{
+	return glyph;
+}
+
+unsigned short *set_translate(int m, struct vc_data *vc)
+{
+	return NULL;
+}
+
+int con_set_trans_old(unsigned char *arg)
+{
+	return 0;
+}
+
+int con_get_trans_old(unsigned char *arg)
+{
+	return -EINVAL;
+}
+
+int con_set_trans_new(ushort *arg)
+{
+	return 0;
+}
+
+int con_get_trans_new(ushort *arg)
+{
+	return -EINVAL;
+}
+
+int con_clear_unimap(struct vc_data *vc, struct unimapinit *ui)
+{
+	return 0;
+}
+
+int con_set_unimap(struct vc_data *vc, ushort ct, struct unipair *list)
+{
+	return 0;
+}
+
+int con_set_default_unimap(struct vc_data *vc)
+{
+	return 0;
+}
+
+int con_copy_unimap(struct vc_data *d, struct vc_data *s)
+{
+	return 0;
+}
+
+int con_get_unimap(struct vc_data *vc, ushort ct, ushort *uct,
+		   struct unipair *list)
+{
+	return -EINVAL;
+}
+
+void con_free_unimap(struct vc_data *vc) { }
+
+int conv_uni_to_pc(struct vc_data *conp, long ucs)
+{
+	return ucs > 0xff ? -1: ucs;
+}
+
+void __init console_map_init(void) { }
+
+u32 conv_8bit_to_uni(unsigned char c)
+{
+	return c;
+}
+
+int conv_uni_to_8bit(u32 uni)
+{
+	return (int)uni & 0xff;
+}
+
+#endif
--- a/drivers/char/vt.c
+++ b/drivers/char/vt.c
@@ -2198,7 +2198,11 @@ rescan_last_byte:
  			c = 0xfffd;
  		    tc = c;
  		} else {	/* no utf or alternate charset mode */
+#ifdef CONFIG_CONSOLE_TRANSLATIONS
  		    tc = vc->vc_translate[vc->vc_toggle_meta ? (c | 0x80) : c];
+#else
+		    tc = c;
+#endif
  		}

                  /* If the original code was a control character we
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -758,6 +758,13 @@ config PROC_PAGE_MONITOR
  	  /proc/kpagecount, and /proc/kpageflags. Disabling these
            interfaces will reduce the size of the kernel by approximately 4kb.

+config CONSOLE_TRANSLATIONS
+	default y
+	bool "Enable character translations in console" if EMBEDDED
+	help
+	  This enables support for font mapping and Unicode translation
+          on virtual consoles.
+
  endmenu		# General setup

  config SLABINFO

Re: [PATCH] console - Add configurable support for console charset translation

[H. Peter Anvin]

Tim Bird wrote:
> With CONSOLE_TRANSLATIONS turned off, this saves about 6K
> on my kernel configured for an ARM development board (OMAP
> 5912 OSK).  In embedded products I'm familiar with,
> console translations are not needed.

On most embedded products I'm familiar with, you wouldn't have virtual 
consoles at all...?

	-hpa

Re: [PATCH] console - Add configurable support for console charset translation

[Jamie Lokier]

H. Peter Anvin wrote:
> Tim Bird wrote:
> >With CONSOLE_TRANSLATIONS turned off, this saves about 6K
> >on my kernel configured for an ARM development board (OMAP
> >5912 OSK).  In embedded products I'm familiar with,
> >console translations are not needed.
> 
> On most embedded products I'm familiar with, you wouldn't have virtual 
> consoles at all...?

On anything with a framebuffer, VCs are quite handy for debugging.
Saving 6k won't save the world on those devices, but each little bit
helps - especially due to no-MMU fragmentation.

-- Jamie

Re: [PATCH] console - Add configurable support for console charset translation

[Matt Mackall]

On Mon, 2008-06-02 at 18:34 -0700, H. Peter Anvin wrote:
> Tim Bird wrote:
> > With CONSOLE_TRANSLATIONS turned off, this saves about 6K
> > on my kernel configured for an ARM development board (OMAP
> > 5912 OSK).  In embedded products I'm familiar with,
> > console translations are not needed.
> 
> On most embedded products I'm familiar with, you wouldn't have virtual 
> consoles at all...?

Actually, lots have frame buffers these days.

-- 
Mathematics is the supreme nostalgia of our time.

Re: [PATCH] console - Add configurable support for console charset translation

[Rob Landley]

On Tuesday 03 June 2008 10:46:04 Matt Mackall wrote:
> On Mon, 2008-06-02 at 18:34 -0700, H. Peter Anvin wrote:
> > Tim Bird wrote:
> > > With CONSOLE_TRANSLATIONS turned off, this saves about 6K
> > > on my kernel configured for an ARM development board (OMAP
> > > 5912 OSK).  In embedded products I'm familiar with,
> > > console translations are not needed.
> >
> > On most embedded products I'm familiar with, you wouldn't have virtual
> > consoles at all...?
>
> Actually, lots have frame buffers these days.

Cell phones, for instance.

My research is out of date now, but back in 2007 I tried to give a sense of
scale at my cross compiling tutorial at OLS.  A cut and paste from my
(18 month old) notes:

--------------------------------------------------------------------------

According to http://eetimes.eu/showArticle.jhtml?articleID=199702110

  Arm shipped the processors in 250 million "smart phones" in 2006 (83%
  market share in that niche).

According to ARM Inc. quarterly results for Q1 2007:
  http://media.corporate-ir.net/media_files/irol/19/197211/press/Q12007EarningsRelease.pdf

  In the first quarter of 2007, the licensees of ARM Inc. shipped 724 million
  ARM processors.  In one quarter.  (ARM Inc. collected $45 million in license
  fees, which is an average of 6.2 cents/processor.)

In a May 23 2006 presentation to ARM investors, ARM Inc. estimated its 2006
market share at 80-90% of the cell phone market, but only 21% of the larger
market.  (The embedded world is big, folks.)

  Drew highlighted five key growth areas for ARM between now and 2010...

  [The first four are] set-top-box, high-definition television and DVD systems,
  solid-state and hard disk drive storage, automotive electronics and 32-bit
  microcontrollers.  In these areas in 2006, ARM had market shares of 14, 20,
  5 and 13 percent respectively...  They are also market sectors that will
  represent a total available annual market of nearly a billion cores or more
  each in 2010. The biggest annual market opportunities are likely to be
  automotive and 32-bit microcontrollers at 2.0 billion and 1.9 billion cores
  each.

  ARM's fifth key target market is the smart phone, where it is already
  enjoying success. In this area the company shipped 250 million units in 2006
  and has a market share of 83 percent, according to its own estimates.

--------------------------------------------------------------------------

(And that's why Apple did the iPhone.)

Speaking of set-top boxes (another thing with a framebuffer), Netflix is
transitioning its business model to video-on-demand, with a $99 set top box
that runs Linux on Arm:
http://www.linuxdevices.com/news/NS8633598605.html

Here's the CEO of netflix saying the DVD-by-mail business will peak within
the next 5 years:
http://www.reuters.com/article/marketsNews/idINN2843042120080528?rpc=44

And the manufacturer almost immediately went to a 10 day wait on delivery of
new devices:
http://forums.roku.com/viewtopic.php?t=16722

(And that's why Apple did Apple TV.)

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: [PATCH] console - Add configurable support for console charset translation

[H. Peter Anvin]

Rob Landley wrote:
>> Actually, lots have frame buffers these days.
> 
> Cell phones, for instance.

Sure, but do you want to use them as consoles?

	-hpa

Re: [PATCH] console - Add configurable support for console charset translation

[Paul Mundt]

On Tue, Jun 03, 2008 at 03:37:23PM -0700, H. Peter Anvin wrote:
> Rob Landley wrote:
> >>Actually, lots have frame buffers these days.
> >
> >Cell phones, for instance.
> 
> Sure, but do you want to use them as consoles?
> 
Unless your name is Pavel, no one actually wants a console on their
phone. So no, just framebuffers, as is to be expected :-)

Re: [PATCH] console - Add configurable support for console charset translation

[Rob Landley]

On Tuesday 03 June 2008 21:56:48 Paul Mundt wrote:
> On Tue, Jun 03, 2008 at 03:37:23PM -0700, H. Peter Anvin wrote:
> > Rob Landley wrote:
> > >>Actually, lots have frame buffers these days.
> > >
> > >Cell phones, for instance.
> >
> > Sure, but do you want to use them as consoles?
>
> Unless your name is Pavel, no one actually wants a console on their
> phone. So no, just framebuffers, as is to be expected :-)

Actually if you ever need to diagnose early boot stuff on _any_ platform, you 
do need a console.  But it can be serial or netconsole, as long as that 
works...

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: [PATCH] console - Add configurable support for console charset translation

[Bill Gatliff]

Rob Landley wrote:
> On Tuesday 03 June 2008 21:56:48 Paul Mundt wrote:
>> On Tue, Jun 03, 2008 at 03:37:23PM -0700, H. Peter Anvin wrote:
>>> Rob Landley wrote:
>>>>> Actually, lots have frame buffers these days.
>>>> Cell phones, for instance.
>>> Sure, but do you want to use them as consoles?
>> Unless your name is Pavel, no one actually wants a console on their
>> phone. So no, just framebuffers, as is to be expected :-)
> 
> Actually if you ever need to diagnose early boot stuff on _any_ platform, you 
> do need a console.  But it can be serial or netconsole, as long as that 
> works...

... who says I _always_ want it to look and feel like a "cell phone"?

Some days, I'd prefer that my cell phone look like something else.  Anything
else.  Especially something that will actually work _for_ me instead of the
other way around.

But barring that, something throw-able.  Like a chair.  :)



b.g.
-- 
Bill Gatliff
bgat@billgatliff.com

Re: [PATCH] console - Add configurable support for console charset translation

[Rob Landley]

On Wednesday 04 June 2008 12:42:06 Bill Gatliff wrote:
> Rob Landley wrote:
> > On Tuesday 03 June 2008 21:56:48 Paul Mundt wrote:
> >> On Tue, Jun 03, 2008 at 03:37:23PM -0700, H. Peter Anvin wrote:
> >>> Rob Landley wrote:
> >>>>> Actually, lots have frame buffers these days.
> >>>>
> >>>> Cell phones, for instance.
> >>>
> >>> Sure, but do you want to use them as consoles?
> >>
> >> Unless your name is Pavel, no one actually wants a console on their
> >> phone. So no, just framebuffers, as is to be expected :-)
> >
> > Actually if you ever need to diagnose early boot stuff on _any_ platform,
> > you do need a console.  But it can be serial or netconsole, as long as
> > that works...
>
> ... who says I _always_ want it to look and feel like a "cell phone"?

With the little bluetooth earpieces, cellphone "look and feel" is decidedly 
mutable these days.

(Ok, cell phones started out with the star-tac flip open phone, a fairly 
obvious rip-off from Star Trek TOS.  We now have the banana in Uhura's ear, 
running off of bluetooth.  Now I'm waiting for the communicator badges.  It's 
only a matter of time...)

That said, the cell phone and PDA spaces are converging, and thingies like the 
iPhone and Nokia 810 are the next logical step beyond laptops.  (If you want 
to hook them up to a big screen and keyboard, your "docking station" can just 
be USB.  If it wasn't for the need to provide power to the suckers, it could 
be bluetooth.)

You only need to carry around one pack of electronics with you, and everybody 
has a cell phone these days.  The rest is a question of user interface and 
letting Moore's Law crank long enough...

Rob

P.S. I agree with what Peter seems to be saying, that the old "vga console" 
code is a side issue.  /dev/console doesn't even provide a controlling tty, 
and the PTY code shouldn't really have anything in commonwith the old virtual 
console stuff except an API.  (Honestly it just attaches a cursor position, 
screen size, and some signals to a pipe.  The rest of the stuff it does like 
setting serial port hardware rates and beeping the speaker probably should 
have been done via ioctl on /dev nodes or something.)
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: [PATCH] console - Add configurable support for console charset translation

[Bill Gatliff]

Rob Landley wrote:
> Now I'm waiting for the communicator badges.  It's only a matter of time...)

Working on it...  :)


b.g.
-- 
Bill Gatliff
bgat@billgatliff.com

Re: [PATCH] console - Add configurable support for console charset translation

[Paul Mundt]

On Wed, Jun 04, 2008 at 12:36:11PM -0500, Rob Landley wrote:
> Actually if you ever need to diagnose early boot stuff on _any_ platform, you 
> do need a console.  But it can be serial or netconsole, as long as that 
> works...
> 
Except for the minor fact that most early boot debugging happens long
before the console subsystem is even available..

At the risk of perpetuating the stupidity of this thread.. If you ship a
device to a customer expecting them to debug it for you, you are likewise
not likely to be very commercially successful, either. Devices are not
shipping with consoles, period. If you disagree with this, you've
obviously never shipped a device.

Re: [PATCH] console - Add configurable support for console charset translation

[Rob Landley]

On Wednesday 04 June 2008 19:46:30 Paul Mundt wrote:
> On Wed, Jun 04, 2008 at 12:36:11PM -0500, Rob Landley wrote:
> > Actually if you ever need to diagnose early boot stuff on _any_ platform,
> > you do need a console.  But it can be serial or netconsole, as long as
> > that works...
>
> Except for the minor fact that most early boot debugging happens long
> before the console subsystem is even available..

Isn't that why CONFIG_EARLY_PRINTK was written?  (And I mentioned Linus's hack 
using the RTC to see how far the _really_ early stuff got.)

> At the risk of perpetuating the stupidity of this thread..

I plead the fifth.

> If you ship a 
> device to a customer expecting them to debug it for you, you are likewise
> not likely to be very commercially successful, either.

There are such things as field servicable devices where companies either send 
people out into the wild or get hardware brought in for service.

However, looking at the message you're replying to, I was talking about during 
development.  (Remember how the early linksys boxes didn't have the serial 
port physically wired up to the outside, but if you busted out a soldering 
iron you got a shell prompt on it without even installing new software?  They 
didn't change it after they got it working because they didn't want to 
re-validate their image?  Yeah, that kind of stuff gets shipped.)

Sure if the device in the field doesn't boot, it's cheaper to just replace it, 
unless you need to get data off the sucker (in which case they bring it in).  
But the defective units returned to the factory sometimes get diagnosed so 
they can reduce the future return rate (or resell 'em used if it was pilot 
error), so once again it helps if it's possible to service 'em after the 
fact.  Depends on the company.

> Devices are not shipping with consoles, period.

Earlier I was trying to distinguish between /dev/console (no controlling tty), 
virtual terminals (tied to old VGA hardware although possibly usable through 
the framebuffer, I'm unclear on this), the tty layer (which is what I 
initially thought the patch was aimed at, but it seems to be the vga VT 
stuff), and having a bitmapped display (may be GUI only).  Four separate 
things, I've lost track of which we're talking about here.

When you imply it's stupid to think anyone will ever have a console on an 
embedded system (because we all know the embedded world is far more uniform 
than crazy diverse things like the desktop space), which of these are _you_ 
referring to?

My cable modem has a serial port that gives me a login prompt if I plug into 
it, and I watched the guy at sprint bring one up last time I broke my cell 
phone so he could get my number list off it.  That kind of console may be 
useful out in the real world, and that's the kind of console I was talking 
about in the message you replied to.

I've learned that "I don't do that, therefore nobody ever will" is not always 
the world's greatest assumption.  Somebody out there may want a console, and 
they may not want the overhead of internationalizing it because although 
their customers speak mandarin, their field service people do not.

Somebody out there may also want to do something I consider a bad idea.  
(Compared to shipping a device with Windows CE on it, any quibble I have 
about Linux configuration is a rounding error.)

> If you disagree with this, you've 
> obviously never shipped a device.

Since I haven't shipped only _one_ device, and since I used to own a tuxscreen 
phone which may actually meet _all_ the above definitions of "console", I 
guess it's ok for me to disagree with this?

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: [PATCH] console - Add configurable support for console charset translation

[H. Peter Anvin]

Rob Landley wrote:
>> Except for the minor fact that most early boot debugging happens long
>> before the console subsystem is even available..
> 
> Isn't that why CONFIG_EARLY_PRINTK was written?  (And I mentioned Linus's hack 
> using the RTC to see how far the _really_ early stuff got.)

Don't think that works with frame buffers.

> Earlier I was trying to distinguish between /dev/console (no controlling tty), 
> virtual terminals (tied to old VGA hardware although possibly usable through 
> the framebuffer, I'm unclear on this), the tty layer (which is what I 
> initially thought the patch was aimed at, but it seems to be the vga VT 
> stuff), and having a bitmapped display (may be GUI only).  Four separate 
> things, I've lost track of which we're talking about here.
> 
> When you imply it's stupid to think anyone will ever have a console on an 
> embedded system (because we all know the embedded world is far more uniform 
> than crazy diverse things like the desktop space), which of these are _you_ 
> referring to?

I'm talking about the virtual console stuff, which is *not* inherently 
tied to VGA.

The issue is whether it makes sense to have virtual console capability 
on a device which is so constrained that you want a *subset* of the 
functionality in place.

	-hpa

Re: [PATCH] console - Add configurable support for console charset translation

[Jamie Lokier]

H. Peter Anvin wrote:
> >When you imply it's stupid to think anyone will ever have a console on an 
> >embedded system (because we all know the embedded world is far more 
> >uniform than crazy diverse things like the desktop space), which of these 
> >are _you_ referring to?
> 
> I'm talking about the virtual console stuff, which is *not* inherently 
> tied to VGA.
> 
> The issue is whether it makes sense to have virtual console capability 
> on a device which is so constrained that you want a *subset* of the 
> functionality in place.

The uClinux devices I ship do have that.  System console is the serial
port on a board header.  But field engineers are instructed to plug in
a USB keyboard and Alt-F2 to get a diagnostic screen (we call it a
"shell" ;-).  Occasionally even customers with specific requirements
get to use that.

We had to patch the kernel to get the combination of virtual terminal
capability on a framebuffer, but system console messages not to the
framebuffer.  The uClinux (2.4.x) disables system console on a virtual
terminal, but due to some confusion(*), it has the side effect that
you can't attach a VT to a framebuffer at all.  Several howtos provide
a "con2fb" program which didn't work with uClinux.  We had to fix that.

(*) - the word "console" is used for four different concepts in the
kernel, sometimes in the same code, and it seems to have confused
someone.(**)  In the following, CONFIG_VT_CONSOLE should not control
take_over_console(); the two "console" are not the same concept:

#ifdef CONFIG_VT_CONSOLE
    take_over_console(&fb_con, first_fb_vc, last_fb_vc, fbcon_is_default);
#endif

(**) So I suggest using "virtual terminal" consistently in the code,
and when talking about them, would be an improvement.

-- Jamie

Re: [PATCH] console - Add configurable support for console charset translation

[Geert Uytterhoeven]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1559 bytes --]

On Mon, 9 Jun 2008, Jamie Lokier wrote:
> The uClinux devices I ship do have that.  System console is the serial
> port on a board header.  But field engineers are instructed to plug in
> a USB keyboard and Alt-F2 to get a diagnostic screen (we call it a
> "shell" ;-).  Occasionally even customers with specific requirements
> get to use that.
> 
> We had to patch the kernel to get the combination of virtual terminal
> capability on a framebuffer, but system console messages not to the
> framebuffer.  The uClinux (2.4.x) disables system console on a virtual
> terminal, but due to some confusion(*), it has the side effect that
> you can't attach a VT to a framebuffer at all.  Several howtos provide
> a "con2fb" program which didn't work with uClinux.  We had to fix that.

I think this is due to using 2.4.

In 2.6, al you have to do is pass `console=ttyS0' (replace `ttyS0' by your
serial port device) to the kernel, and system console messages will no longer
appear on the frame buffer (unless you add an explicit `console=tty0').
With kind regards,

Geert Uytterhoeven
Software Architect

Sony Techsoft Centre
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium

Phone:    +32 (0)2 700 8453
Fax:      +32 (0)2 700 8622
E-mail:   Geert.Uytterhoeven@sonycom.com
Internet: http://www.sony-europe.com/

Sony Technology and Software Centre Europe
A division of Sony Service Centre (Europe) N.V.
Registered office: Technologielaan 7 · B-1840 Londerzeel · Belgium
VAT BE 0413.825.160 · RPR Brussels
Fortis 293-0376800-10 GEBA-BE-BB

Re: [PATCH] console - Add configurable support for console charset translation

[Rob Landley]

On Tuesday 03 June 2008 17:37:23 H. Peter Anvin wrote:
> Rob Landley wrote:
> >> Actually, lots have frame buffers these days.
> >
> > Cell phones, for instance.
>
> Sure, but do you want to use them as consoles?

A friend of mine hacked his iPhone and got shell prompt on it, and it's quite 
usable with the touchscreen keyboard.  (I very much want a linux box with 
that form factor, battery life, connectivity, at least that much memory...)

And hooking a set top box up to an HDTV gives you better potential resolution 
than my laptop currently has, on a biiiiiig screen.  They already display 
text for program listings and recorded program menus and such, plus you can 
add a mouse and keyboard via bluetooth or usb if you like...

> 	-hpa

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Mon, 2008-06-02 at 15:37 -0700, Tim Bird wrote:
> With CONSOLE_TRANSLATIONS turned off, this saves about 6K
> on my kernel configured for an ARM development board (OMAP
> 5912 OSK).  In embedded products I'm familiar with,
> console translations are not needed.

I'd have been more inclined to put the #ifdef and the empty functions
into <linux/vt_kern.h>. Then you don't need to provide real stub
functions -- they can be inlined at the caller.

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Mon, 2008-06-02 at 15:37 -0700, Tim Bird wrote:
> With CONSOLE_TRANSLATIONS turned off, this saves about 6K
> on my kernel configured for an ARM development board (OMAP
> 5912 OSK).  In embedded products I'm familiar with,
> console translations are not needed.
> 
> This was taken from the Linux-tiny project and updated slightly
> for 2.6.25.

I prefer it like this... we can drop consolemap.o and
consolemap_deftbl.o from the build completely. It saves 7.2KiB on a
ppc32 build here.

diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig
index 595a925..f740190 100644
--- a/drivers/char/Kconfig
+++ b/drivers/char/Kconfig
@@ -36,6 +36,13 @@ config VT
 	  If unsure, say Y, or else you won't be able to do much with your new
 	  shiny Linux system :-)
 
+config CONSOLE_TRANSLATIONS
+	default y
+	bool "Enable character translations in console" if EMBEDDED
+	---help---
+	  This enables support for font mapping and Unicode translation
+	  on virtual consoles.
+
 config VT_CONSOLE
 	bool "Support for console on virtual terminal" if EMBEDDED
 	depends on VT
diff --git a/drivers/char/Makefile b/drivers/char/Makefile
index 4c1c584..6ef173c 100644
--- a/drivers/char/Makefile
+++ b/drivers/char/Makefile
@@ -12,8 +12,8 @@ obj-y	 += mem.o random.o tty_io.o n_tty.o tty_ioctl.o
 obj-$(CONFIG_LEGACY_PTYS)	+= pty.o
 obj-$(CONFIG_UNIX98_PTYS)	+= pty.o
 obj-y				+= misc.o
-obj-$(CONFIG_VT)		+= vt_ioctl.o vc_screen.o consolemap.o \
-				   consolemap_deftbl.o selection.o keyboard.o
+obj-$(CONFIG_VT)		+= vt_ioctl.o vc_screen.o selection.o keyboard.o
+obj-$(CONFIG_CONSOLE_TRANSLATIONS) += consolemap.o consolemap_deftbl.o
 obj-$(CONFIG_HW_CONSOLE)	+= vt.o defkeymap.o
 obj-$(CONFIG_AUDIT)		+= tty_audit.o
 obj-$(CONFIG_MAGIC_SYSRQ)	+= sysrq.o
diff --git a/include/linux/consolemap.h b/include/linux/consolemap.h
index e2bf7e5..c4811da 100644
--- a/include/linux/consolemap.h
+++ b/include/linux/consolemap.h
@@ -3,6 +3,9 @@
  *
  * Interface between console.c, selection.c  and consolemap.c
  */
+#ifndef __LINUX_CONSOLEMAP_H__
+#define __LINUX_CONSOLEMAP_H__
+
 #define LAT1_MAP 0
 #define GRAF_MAP 1
 #define IBMPC_MAP 2
@@ -10,6 +13,7 @@
 
 #include <linux/types.h>
 
+#ifdef CONFIG_CONSOLE_TRANSLATIONS
 struct vc_data;
 
 extern u16 inverse_translate(struct vc_data *conp, int glyph, int use_unicode);
@@ -18,3 +22,13 @@ extern int conv_uni_to_pc(struct vc_data *conp, long ucs);
 extern u32 conv_8bit_to_uni(unsigned char c);
 extern int conv_uni_to_8bit(u32 uni);
 void console_map_init(void);
+#else
+#define inverse_translate(conp, glyph, uni) ((uint16_t)glyph)
+#define set_translate(m, vc) ((unsigned short *)NULL)
+#define conv_uni_to_pc(conp, ucs) ((int) (ucs > 0xff ? -1: ucs))
+#define conv_8bit_to_uni(c) ((uint32_t)(c))
+#define conv_uni_to_8bit(c) ((int) ((c) & 0xff))
+#define console_map_init(c) do { ; } while (0)
+#endif /* CONFIG_CONSOLE_TRANSLATIONS */
+
+#endif /* __LINUX_CONSOLEMAP_H__ */
diff --git a/include/linux/vt_kern.h b/include/linux/vt_kern.h
index 9448ffb..81ed155 100644
--- a/include/linux/vt_kern.h
+++ b/include/linux/vt_kern.h
@@ -12,6 +12,7 @@
 #include <linux/mutex.h>
 #include <linux/console_struct.h>
 #include <linux/mm.h>
+#include <linux/consolemap.h>
 
 /*
  * Presently, a lot of graphics programs do not restore the contents of
@@ -54,6 +55,7 @@ void redraw_screen(struct vc_data *vc, int is_switch);
 struct tty_struct;
 int tioclinux(struct tty_struct *tty, unsigned long arg);
 
+#ifdef CONFIG_CONSOLE_TRANSLATIONS
 /* consolemap.c */
 
 struct unimapinit;
@@ -70,6 +72,18 @@ int con_set_default_unimap(struct vc_data *vc);
 void con_free_unimap(struct vc_data *vc);
 void con_protect_unimap(struct vc_data *vc, int rdonly);
 int con_copy_unimap(struct vc_data *dst_vc, struct vc_data *src_vc);
+#else
+#define con_set_trans_old(arg) (0)
+#define con_get_trans_old(arg) (-EINVAL)
+#define con_set_trans_new(arg) (0)
+#define con_get_trans_new(arg) (-EINVAL)
+#define con_clear_unimap(vc, ui) (0)
+#define con_set_unimap(vc, ct, list) (0)
+#define con_set_default_unimap(vc) (0)
+#define con_copy_unimap(d, s) (0)
+#define con_get_unimap(vc, ct, uct, list) (-EINVAL)
+#define con_free_unimap(vc) do { ; } while (0)
+#endif
 
 /* vt.c */
 int vt_waitactive(int vt);

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Holger Schurig]

Maybe change the Kconfig entry so that:

* with CONFIG_EMBEDDED=y it's configuration
* with CONFIG_EMBEDDED=n it's on by default

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Tue, 2008-06-03 at 16:06 +0200, Holger Schurig wrote:
> Maybe change the Kconfig entry so that:
> 
> * with CONFIG_EMBEDDED=y it's configuration
> * with CONFIG_EMBEDDED=n it's on by default

That's what Tim's patch did (although I moved it to drivers/char/Kconfig
and made it depend on CONFIG_VT).

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Tim Bird]

David Woodhouse wrote:
> On Mon, 2008-06-02 at 15:37 -0700, Tim Bird wrote:
>> With CONSOLE_TRANSLATIONS turned off, this saves about 6K
>> on my kernel configured for an ARM development board (OMAP
>> 5912 OSK).  In embedded products I'm familiar with,
>> console translations are not needed.
>>
>> This was taken from the Linux-tiny project and updated slightly
>> for 2.6.25.
> 
> I prefer it like this... we can drop consolemap.o and
> consolemap_deftbl.o from the build completely. It saves 7.2KiB on a
> ppc32 build here.
> 
> diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig
> index 595a925..f740190 100644
> --- a/drivers/char/Kconfig
> +++ b/drivers/char/Kconfig
> ...

This is clearly an improvement. But it is missing this part of the
original patch:

--- a/drivers/char/vt.c
+++ b/drivers/char/vt.c
@@ -2198,7 +2198,11 @@ rescan_last_byte:
 			c = 0xfffd;
 		    tc = c;
 		} else {	/* no utf or alternate charset mode */
+#ifdef CONFIG_CONSOLE_TRANSLATIONS
 		    tc = vc->vc_translate[vc->vc_toggle_meta ? (c | 0x80) : c];
+#else
+		    tc = c;
+#endif
 		}

                 /* If the original code was a control character we

With the set_translate function stubbed, and the actual translation
operation left intact, I think the code might have problems.

I ran your patch fine on my OSK board here, but I must not have hit a
character translation case.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Tue, 2008-06-03 at 17:01 -0700, Tim Bird wrote:
> This is clearly an improvement. But it is missing this part of the
> original patch:

Oops, well spotted. I've updated the patch in the git tree; thanks.

(that's what comes of applying patches by hand -- I _knew_ I had to do
that hunk, but managed to forget about it within the space of about 30
seconds when I was doing the other parts. I plead stupidity)

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Josh Boyer]

On Wed, 04 Jun 2008 01:16:37 +0100
David Woodhouse <dwmw2@infradead.org> wrote:

> On Tue, 2008-06-03 at 17:01 -0700, Tim Bird wrote:
> > This is clearly an improvement. But it is missing this part of the
> > original patch:
> 
> Oops, well spotted. I've updated the patch in the git tree; thanks.

What git tree?

josh

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Tue, 2008-06-03 at 20:03 -0500, Josh Boyer wrote:
> On Wed, 04 Jun 2008 01:16:37 +0100
> David Woodhouse <dwmw2@infradead.org> wrote:
> 
> > On Tue, 2008-06-03 at 17:01 -0700, Tim Bird wrote:
> > > This is clearly an improvement. But it is missing this part of the
> > > original patch:
> > 
> > Oops, well spotted. I've updated the patch in the git tree; thanks.
> 
> What git tree?

 git.infradead.org/embedded-2.6.git

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Josh Boyer]

On Wed, 04 Jun 2008 02:05:40 +0100
David Woodhouse <dwmw2@infradead.org> wrote:

> On Tue, 2008-06-03 at 20:03 -0500, Josh Boyer wrote:
> > On Wed, 04 Jun 2008 01:16:37 +0100
> > David Woodhouse <dwmw2@infradead.org> wrote:
> > 
> > > On Tue, 2008-06-03 at 17:01 -0700, Tim Bird wrote:
> > > > This is clearly an improvement. But it is missing this part of the
> > > > original patch:
> > > 
> > > Oops, well spotted. I've updated the patch in the git tree; thanks.
> > 
> > What git tree?
> 
>  git.infradead.org/embedded-2.6.git

Do you have plans to get that in -mm, or linux-next?

(Please say yes).

josh

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Tue, 2008-06-03 at 22:13 -0500, Josh Boyer wrote:
> 
> >  git.infradead.org/embedded-2.6.git
> 
> Do you have plans to get that in -mm, or linux-next?

Should be already there in today's linux-next.

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Adrian Bunk]

On Wed, Jun 04, 2008 at 10:16:22AM +0100, David Woodhouse wrote:
> On Tue, 2008-06-03 at 22:13 -0500, Josh Boyer wrote:
> > 
> > >  git.infradead.org/embedded-2.6.git
> > 
> > Do you have plans to get that in -mm, or linux-next?
> 
> Should be already there in today's linux-next.

Does this imply you want Linus to merge from this tree?

How does this tree fit into the usual maintainership inside the Linux 
kernel?

> dwmw2

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Wed, 2008-06-04 at 13:07 +0300, Adrian Bunk wrote:
> On Wed, Jun 04, 2008 at 10:16:22AM +0100, David Woodhouse wrote:
> > On Tue, 2008-06-03 at 22:13 -0500, Josh Boyer wrote:
> > > 
> > > >  git.infradead.org/embedded-2.6.git
> > > 
> > > Do you have plans to get that in -mm, or linux-next?
> > 
> > Should be already there in today's linux-next.
> 
> Does this imply you want Linus to merge from this tree?

Yes.

> How does this tree fit into the usual maintainership inside the Linux 
> kernel?

Well, I was trying to avoid creating it -- because largely it doesn't.
I was hoping that everything could go upstream through other routes.
But Tim's patch really was a candidate for handling it myself (unless
I'm just going to punt everything to Andrew), so I capitulated.

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Tim Bird]

David Woodhouse wrote:
> On Tue, 2008-06-03 at 17:01 -0700, Tim Bird wrote:
>> This is clearly an improvement. But it is missing this part of the
>> original patch:
> 
> Oops, well spotted. I've updated the patch in the git tree; thanks.
> 
> (that's what comes of applying patches by hand -- I _knew_ I had to do
> that hunk, but managed to forget about it within the space of about 30
> seconds when I was doing the other parts. I plead stupidity)

We can chalk it up to my stupidity for sending a patch
that couldn't be applied automatically...
Thanks very much for the improvements!
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: [PATCH] console - Add configurable support for console charset translation

[Alan Cox]

> This is clearly an improvement. But it is missing this part of the
> original patch:
> 
> --- a/drivers/char/vt.c
> +++ b/drivers/char/vt.c
> @@ -2198,7 +2198,11 @@ rescan_last_byte:
>  			c = 0xfffd;
>  		    tc = c;
>  		} else {	/* no utf or alternate charset mode */
> +#ifdef CONFIG_CONSOLE_TRANSLATIONS
>  		    tc = vc->vc_translate[vc->vc_toggle_meta ? (c | 0x80) : c];
> +#else
> +		    tc = c;
> +#endif

Can we please get the ifdefs tided up before this goes in.

For the moment this has a NAK from the tty maintainer but if the ifdefs
turned went into headers where they belong and the code looked like say

		tc = vc_translate(vc, c);

with two versions of vc_translate (one being vc_translate(x) (x)) it
might be more reasonable.

We can't stick random ifdefs in bits of code for arbitary 6K savings or
the entire kernel would be ifdefs.


Sor for the moment

Nacked-by: Alan Cox <alan@redhat.com>

but scope for change

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Wed, 2008-06-04 at 12:55 +0100, Alan Cox wrote:
> Can we please get the ifdefs tided up before this goes in.
> 
> For the moment this has a NAK from the tty maintainer but if the
> ifdefs
> turned went into headers where they belong and the code looked like
> say
> 
>                 tc = vc_translate(vc, c);
> 
> with two versions of vc_translate (one being vc_translate(x) (x)) it
> might be more reasonable.

Good point. I did think about that originally, but then just completely
forgot to process that hunk of the original patch altogether, which was
another way of avoiding the ifdefs :)

If I merge this incremental patch, does that address your objections?

(Note that I've changed the expression to evaluate (c) only once, just
because that's best practice in macro definitions. Wasn't really worth
doing it for (vc) though.)

diff --git a/drivers/char/vt.c b/drivers/char/vt.c
index de52f99..18b7fb0 100644
--- a/drivers/char/vt.c
+++ b/drivers/char/vt.c
@@ -2208,11 +2208,7 @@ rescan_last_byte:
 			c = 0xfffd;
 		    tc = c;
 		} else {	/* no utf or alternate charset mode */
-#ifdef CONFIG_CONSOLE_TRANSLATIONS
-		    tc = vc->vc_translate[vc->vc_toggle_meta ? (c | 0x80) : c];
-#else
-		    tc = c;
-#endif
+		    tc = vc_translate(vc, c);
 		}
 
 		param.c = tc;
diff --git a/include/linux/vt_kern.h b/include/linux/vt_kern.h
index 81ed155..929ee80 100644
--- a/include/linux/vt_kern.h
+++ b/include/linux/vt_kern.h
@@ -72,6 +72,9 @@ int con_set_default_unimap(struct vc_data *vc);
 void con_free_unimap(struct vc_data *vc);
 void con_protect_unimap(struct vc_data *vc, int rdonly);
 int con_copy_unimap(struct vc_data *dst_vc, struct vc_data *src_vc);
+
+#define vc_translate(vc, c) ((vc)->vc_translate[(c) | \
+					(vc)->vc_toggle_meta ? 0x80 : 0])
 #else
 #define con_set_trans_old(arg) (0)
 #define con_get_trans_old(arg) (-EINVAL)
@@ -83,6 +86,8 @@ int con_copy_unimap(struct vc_data *dst_vc, struct vc_data *src_vc);
 #define con_copy_unimap(d, s) (0)
 #define con_get_unimap(vc, ct, uct, list) (-EINVAL)
 #define con_free_unimap(vc) do { ; } while (0)
+
+#define vc_translate(vc, c) (c)
 #endif
 
 /* vt.c */

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Alan Cox]

> If I merge this incremental patch, does that address your objections?

Yes

> (Note that I've changed the expression to evaluate (c) only once, just
> because that's best practice in macro definitions. Wasn't really worth
> doing it for (vc) though.)

Agreed

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Wed, 2008-06-04 at 15:07 +0100, Alan Cox wrote:
> > If I merge this incremental patch, does that address your objections?
> 
> Yes

http://git.infradead.org/embedded-2.6.git?a=commitdiff;h=a29ccf6f8


-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Rob Landley]

On Monday 02 June 2008 17:37:09 Tim Bird wrote:
> With CONSOLE_TRANSLATIONS turned off, this saves about 6K
> on my kernel configured for an ARM development board (OMAP
> 5912 OSK).  In embedded products I'm familiar with,
> console translations are not needed.
>
> This was taken from the Linux-tiny project and updated slightly
> for 2.6.25.

Er, what _is_ the relationship between linux-tiny and linux-embedded lists 
now?

I've been on linux-tiny for years, but it's been mostly dead since Matt moved 
on (not one message in the whole of may, 4 the month before that), and this 
seems the more logical place to discuss kernel patches that might actually 
want to go upstream someday anyway.

Just curious...

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: [PATCH] console - Add configurable support for console charset translation

[David Woodhouse]

On Tue, 2008-06-03 at 16:18 -0500, Rob Landley wrote:
> Er, what _is_ the relationship between linux-tiny and linux-embedded lists 
> now?

There's been a bunch of embedded-related lists, and one of the reasons
for doing linux-embedded@vger (rather than @lists.infradead.org as would
have been my first inclination) is to set up an 'official' one that
people can congregate on.

-- 
dwmw2

Re: [PATCH] console - Add configurable support for console charset translation

[Matt Mackall]

On Tue, 2008-06-03 at 16:18 -0500, Rob Landley wrote:
> On Monday 02 June 2008 17:37:09 Tim Bird wrote:
> > With CONSOLE_TRANSLATIONS turned off, this saves about 6K
> > on my kernel configured for an ARM development board (OMAP
> > 5912 OSK).  In embedded products I'm familiar with,
> > console translations are not needed.
> >
> > This was taken from the Linux-tiny project and updated slightly
> > for 2.6.25.
> 
> Er, what _is_ the relationship between linux-tiny and linux-embedded lists 
> now?
> 
> I've been on linux-tiny for years, but it's been mostly dead since Matt moved 
> on (not one message in the whole of may, 4 the month before that), and this 
> seems the more logical place to discuss kernel patches that might actually 
> want to go upstream someday anyway.

Linux-embedded is the place to be, folks. It's intended to be the
catch-all list for embedded kernel work. When traffic there picks up a
bit more, I'll start bouncing linux-tiny mail.

-- 
Mathematics is the supreme nostalgia of our time.

Re: [PATCH] console - Add configurable support for console charset translation

[Holger Schurig]

> Linux-embedded is the place to be, folks.

This is the first time I've heard about linux-embedded.

Re: [PATCH] console - Add configurable support for console charset translation

[Holger Schurig]

> Linux-embedded is the place to be, folks.

Another thing that I just noticed: According to

http://vger.kernel.org/vger-lists.html#linux-embedded

there is no mailing list archive for this list.

Re: [PATCH] console - Add configurable support for console charset translation

[Dave Hylands]

> Another thing that I just noticed: According to
>
> http://vger.kernel.org/vger-lists.html#linux-embedded

Mike Frysinger setup the following on May 27:
<http://news.gmane.org/gmane.linux.kernel.embedded>

-- 
Dave Hylands
Vancouver, BC, Canada
http://www.DaveHylands.com/

Re: [PATCH] console - Add configurable support for console charset translation

[Randy Dunlap]

On Wed, 4 Jun 2008 00:36:48 -0700 Dave Hylands wrote:

> > Another thing that I just noticed: According to
> >
> > http://vger.kernel.org/vger-lists.html#linux-embedded
> 
> Mike Frysinger setup the following on May 27:
> <http://news.gmane.org/gmane.linux.kernel.embedded>


It's also available at  http://marc.info/?l=linux-embedded


---
~Randy
<quote:>
"It's the Government of the United States." ... The largest, and yet
the least efficient, producer of computer software in the world.

Re: [PATCH] console - Add configurable support for console charset translation

[Rob Landley]

On Wednesday 04 June 2008 02:04:13 Holger Schurig wrote:
> > Linux-embedded is the place to be, folks.
>
> Another thing that I just noticed: According to
>
> http://vger.kernel.org/vger-lists.html#linux-embedded
>
> there is no mailing list archive for this list.

Yes, but according to the comments on http://lwn.net/Articles/283749/ there 
is.  Unfortunately, there seems to be no way to beat a threaded user 
interface out of it.

I have a complete mbox going back to the beginning (thanks Randy!) and am 
pondering putting up an archive in the format I like (threaded index, grouped 
monthly) this weekend.  Depends on whether or not I can convince mailman to 
archive a list it isn't hosting, or if I get bored enough to write my own 
indexer in python...

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

linux-embedded archives [was Re: [PATCH] console - Add configurable support for console charset translation]

[T Ziomek]

On Wed, Jun 04, 2008 at 12:19:53PM -0500, Rob Landley wrote:
> On Wednesday 04 June 2008 02:04:13 Holger Schurig wrote:
> > > Linux-embedded is the place to be, folks.
> >
> > Another thing that I just noticed: According to
> >
> > http://vger.kernel.org/vger-lists.html#linux-embedded
> >
> > there is no mailing list archive for this list.
> 
> Yes, but according to the comments on http://lwn.net/Articles/283749/ there 
> is.  Unfortunately, there seems to be no way to beat a threaded user 
> interface out of it.

Somebody updated the http://vger.kernel.org/ today; it now lists 3 archives
(mail-archive.com, gmane.org and marc.info).

Tom
-- 
  /"\  ASCII Ribbon Campaign   |
  \ /                          |   Email to user 'CTZ001'
   X        Against HTML       |             at 'email.mot.com'
  / \     in e-mail & news     |

Re: linux-embedded archives [was Re: [PATCH] console - Add configurable support for console charset translation]

[Rob Landley]

On Wednesday 04 June 2008 12:26:17 T Ziomek wrote:
> On Wed, Jun 04, 2008 at 12:19:53PM -0500, Rob Landley wrote:
> > On Wednesday 04 June 2008 02:04:13 Holger Schurig wrote:
> > > > Linux-embedded is the place to be, folks.
> > >
> > > Another thing that I just noticed: According to
> > >
> > > http://vger.kernel.org/vger-lists.html#linux-embedded
> > >
> > > there is no mailing list archive for this list.
> >
> > Yes, but according to the comments on http://lwn.net/Articles/283749/
> > there is.  Unfortunately, there seems to be no way to beat a threaded
> > user interface out of it.
>
> Somebody updated the http://vger.kernel.org/ today; it now lists 3 archives
> (mail-archive.com, gmane.org and marc.info).

So mail-archive.com has no obvious way to browse historical messages by date, 
gmane's interface is a mix of frames and javascript, and marc can't show an 
index with nested replies to threads. 

Well, at least they're broken in different ways...

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: [PATCH] console - Add configurable support for console charset translation

[Adrian Bunk]

On Mon, Jun 02, 2008 at 03:37:09PM -0700, Tim Bird wrote:
> With CONSOLE_TRANSLATIONS turned off, this saves about 6K
> on my kernel configured for an ARM development board (OMAP
> 5912 OSK).  In embedded products I'm familiar with,
> console translations are not needed.
>
> This was taken from the Linux-tiny project and updated slightly
> for 2.6.25.
>...

It seems to be always me who asks the controversial questions...:

Does the linux-tiny approach of adding a kconfig variable for each 5kB 
of code actually make sense? I'm asking since an exploding amount of 
kconfig variables and their interdependencies have a not so small 
maintainance impact in the long term.

And I'm wondering whether it's the best approach for reaching 
measurable results. E.g. my small patch that removed 
-maccumulate-outgoing-args on 32-bit x86 reduced the kernel size
there for the CONFIG_CC_OPTIMIZE_FOR_SIZE=y case by at about 2.5% (sic)
without adding any kconfig variables or #ifdef's.

Can you take a reasonable (best-case) .config for an existing device and 
get the following numbers:
- Ignoring patches that came from linux-tiny, by how many percent did 
  the size of the kernel increase or decrease between 2.6.16 and 2.6.26?
- By how many percent did the kernel size decrease due to patches from
  the linux-tiny project that added such kconfig options for a few kB 
  of code in the same timeframe?

My gut feeling is that the influence of this kind of linux-tiny patches 
is hardly noticably compared to the overall code size development, but 
if you have numbers that prove me wrong just point me to them and I'll 
stand corrected.

cu
Adrian

BTW: I'm not insisting on "between 2.6.16 and 2.6.26", that's just some
     timeframe big enough for showing general trends.

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH] console - Add configurable support for console charset translation

[Rob Landley]

On Wednesday 04 June 2008 05:33:53 Adrian Bunk wrote:
> Does the linux-tiny approach of adding a kconfig variable for each 5kB
> of code actually make sense? I'm asking since an exploding amount of
> kconfig variables and their interdependencies have a not so small
> maintainance impact in the long term.

Complexity is a cost, you have to get good bang for the buck when you spend 
it.

> And I'm wondering whether it's the best approach for reaching
> measurable results.

When I first started stripping down systems to make embedded masquerading 
routers back in the late 90's (before linksys came out), I started with a Red 
Hat install and removed lots and lots of packages.  That's the approach we're 
taking today, and I can say from experience that it's not sustainable.

I then wandered to a Linux From Scratch approach, building a system that had 
nothing in it but what I wanted.  Starting from zero and adding stuff, rather 
than starting from Mt. Crapmore and removing things until the shovel broke.

Someday I want to do the same for the Linux kernel.  When I started building 
systems instead of carving them out of blocks of distro, I started with 
a "hello world" root filesystem, and I want to make a "hello world" kernel.  
Start with just the boot code that does the jump to C code, only instead of 
start_kernel() in init/main.c have it call a hello_world() function that 
prints "hello world" to the console using the early_printk logic, then calls 
HLT.  And does _nothing_else_.  Then add stuff back one chunk at a time, 
sstarting with memory management, then the scheduler and process stuff, then 
the vfs, and so on.  So I know what all the bits do, and how big and 
complicated they are.  And I can document the lot of it as I go.

Unfortunately, as a learning experience, I estimate this would take me about a 
year.  And I haven't got a spare year on me at the moment.  But it remains 
prominently on my todo list, if I decide to start another major project.  
(Maybe after I get a 1.0 release of FWL out.)

> My gut feeling is that the influence of this kind of linux-tiny patches
> is hardly noticably compared to the overall code size development, but
> if you have numbers that prove me wrong just point me to them and I'll
> stand corrected.

The whackamole approach is never going to turn Ubuntu into Damn Small Linux, 
and it ignores the needs of the people who don't want the /proc hairball but 
_do_ want a ps that works.

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: [PATCH] console - Add configurable support for console charset translation

[Bernhard Fischer]

On Wed, Jun 04, 2008 at 01:33:53PM +0300, Adrian Bunk wrote:

>My gut feeling is that the influence of this kind of linux-tiny patches 
>is hardly noticably compared to the overall code size development, but 
>if you have numbers that prove me wrong just point me to them and I'll 
>stand corrected.
>
>cu
>Adrian
>
>BTW: I'm not insisting on "between 2.6.16 and 2.6.26", that's just some
>     timeframe big enough for showing general trends.

Current kernels are too big, they tend to be about 1MB (ignoring lzma or
gz compression), a couple of years back an ide- and network-enabled
kernel with 8139too (or whatever -nic you use in qemu nowadays) was at
least 30% smaller.

Re: [PATCH] console - Add configurable support for console charset translation

[Adrian Bunk]

On Wed, Jun 04, 2008 at 08:34:32PM +0200, Bernhard Fischer wrote:
> On Wed, Jun 04, 2008 at 01:33:53PM +0300, Adrian Bunk wrote:
> 
> >My gut feeling is that the influence of this kind of linux-tiny patches 
> >is hardly noticably compared to the overall code size development, but 
> >if you have numbers that prove me wrong just point me to them and I'll 
> >stand corrected.
> >
> >cu
> >Adrian
> >
> >BTW: I'm not insisting on "between 2.6.16 and 2.6.26", that's just some
> >     timeframe big enough for showing general trends.
> 
> Current kernels are too big, they tend to be about 1MB (ignoring lzma or
> gz compression), a couple of years back an ide- and network-enabled
> kernel with 8139too (or whatever -nic you use in qemu nowadays) was at
> least 30% smaller.

No disagreement that the kernel could (and should) become smaller for
situations where the kernel size matters.

My question is only about whether *this kind of patches* is the correct 
approach.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH] console - Add configurable support for console charset translation

[Bernhard Fischer]

On Wed, Jun 04, 2008 at 10:01:46PM +0300, Adrian Bunk wrote:
>On Wed, Jun 04, 2008 at 08:34:32PM +0200, Bernhard Fischer wrote:
 
>> Current kernels are too big, they tend to be about 1MB (ignoring lzma or
>> gz compression), a couple of years back an ide- and network-enabled
>> kernel with 8139too (or whatever -nic you use in qemu nowadays) was at
>> least 30% smaller.
>
>No disagreement that the kernel could (and should) become smaller for
>situations where the kernel size matters.
>
>My question is only about whether *this kind of patches* is the correct 
>approach.

Every dozend of bytes less helps to some degree. I can boot a smallish
testsystem in qemu with 4MB ram and have a useable userspace in 120k or
up to 1.2MB diskspace to play with.

The kernel ultimately needs the vast majority of resources in such a
system; most of the RAM and most of the diskspace (assuming a
comfortable userspace with 500k). Booting with 2MB didn't work last time
i tried. This imbalance is something we should fix.

So yes, 5k help, several 5k help more. I can see how it's problematic to
represent that in menuconfig without adding too much clutter but still
letting the user decide whether or not to include certain families of
features, but that should not turn out to be a real problem.

Re: [PATCH] console - Add configurable support for console charset translation

[Alan Cox]

> So yes, 5k help, several 5k help more. I can see how it's problematic to
> represent that in menuconfig without adding too much clutter but still
> letting the user decide whether or not to include certain families of
> features, but that should not turn out to be a real problem.

Combinatorial explosions, ugly messes, maintenance.

Far better would be target the actual sizes and get stuff down in size
without removing feaures. There is some obvious 'easy reward' stuff here
- such as being able to build an output only console.

Re: [PATCH] console - Add configurable support for console charset translation

[Tim Bird]

Adrian Bunk wrote:
> It seems to be always me who asks the controversial questions...:
> 
> Does the linux-tiny approach of adding a kconfig variable for each 5kB 
> of code actually make sense? I'm asking since an exploding amount of 
> kconfig variables and their interdependencies have a not so small 
> maintainance impact in the long term.

Others have expressed similar concerns.  Andi Kleen has brought this
up in the past, and highlighted another aspect of this that I think
is important - fragmenting the configuration space for testing.

That is, if different combinations of options are used, then the
testing results from different users may not be directly comparable.
This makes it harder to track down bugs.

In practice, I think these types of switches tend to get turned
on/off together.  This tends to ameliorate the testing and maintenance
issues. It might be worthwhile to coalesce these into fewer
options.  I'd be fine with overloading this particular
onto CONFIG_BASE_FULL, but I don't know what other people think.
Keeping the option separate means you have more flexibility - so
that you can, for example, turn off all but one feature that's
important to you.  (This seems like a pretty common

> And I'm wondering whether it's the best approach for reaching 
> measurable results. E.g. my small patch that removed 
> -maccumulate-outgoing-args on 32-bit x86 reduced the kernel size
> there for the CONFIG_CC_OPTIMIZE_FOR_SIZE=y case by at about 2.5% (sic)
> without adding any kconfig variables or #ifdef's.

These are certainly nice, but there's a limited number of whole-kernel
reduction options.  Kernel size accumulates by both wasteful
compiler output, and by the introduction or preservation of individual
features that are useless for embedded devices.  I think it's worthwhile
to attack both problems.

> Can you take a reasonable (best-case) .config for an existing device and 
> get the following numbers:
> - Ignoring patches that came from linux-tiny, by how many percent did 
>   the size of the kernel increase or decrease between 2.6.16 and 2.6.26?

Well, an initial comparison is at:
http://www.selenic.com/bloatwatch/?cmd=compare&part=%2F&v1=2.6.16&v2=2.6.25-rc2

This shows a size difference of 1.8 meg, but I'm pretty sure this
is with a default config (not optimized) and is not controlled very well
for changes in the config.

I can do a more controlled comparison if you're interested.

> - By how many percent did the kernel size decrease due to patches from
>   the linux-tiny project that added such kconfig options for a few kB 
>   of code in the same timeframe?

I'm not sure that's a good comparison, since Linux-tiny hasn't been
very active in that time period.  I would guess only about 3 or 4
Linux-tiny patches have made it into the kernel since 2.6.16.
The big wins for Linux-tiny were in the 2.6.10/11 kernels, where
most of the low-hanging fruit (size-wise) was gleaned.

But to address your point, you're right that no existing Linux-tiny
patch gets 2.5% reduction for the kernel.
In my own testing at Sony, the 30 or so Linux-tiny patches that
I use get me about 110k of reductions.  For me, this is about 5% of
my kernel size.   Note that my choice of options to achieve
reductions is pretty conservative.

There are diminishing returns here, and at some point it doesn't make
sense to continue.  I have a nagging feeling, though, that there
are a few more things where the bloat is pronounced (glances at
procfs and sysfs).

> My gut feeling is that the influence of this kind of linux-tiny patches 
> is hardly noticably compared to the overall code size development, but 
> if you have numbers that prove me wrong just point me to them and I'll 
> stand corrected.

It *does* feel a bit like fighting the tide with a teaspoon.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: [PATCH] console - Add configurable support for console charset translation

[Sam Ravnborg]

> 
> I can do a more controlled comparison if you're interested.

What would be really useful would be to do some king of automated
monitoring of the size of individual parts of the kernel in
a few controlled configs.
And then as son as somethings grows with > 1% for example then to
bring this to lkml.
Doing this based on linux-next would allow us to catch the bloaters
while they are still or just have been doing certain changes.

It would be nice to tell someone that just enabled som new gcc option
that this had a cost of 163.432 bytes with a certain config.
This would get attraction and be dealt with.

Doing so three months later or maybe one year later would often
get less focus (if the individual commit ever got identified).

Now how to do so I dunno - that would require a bit of tweaking
before working reliable. But the value of being quick here would
pay of this soon I think.

	Sam

Re: [PATCH] console - Add configurable support for console charset translation

[Tim Bird]

Sam Ravnborg wrote:
>> I can do a more controlled comparison if you're interested.
> 
> What would be really useful would be to do some king of automated
> monitoring of the size of individual parts of the kernel in
> a few controlled configs.
> And then as son as somethings grows with > 1% for example then to
> bring this to lkml.
> Doing this based on linux-next would allow us to catch the bloaters
> while they are still or just have been doing certain changes.
> 
> It would be nice to tell someone that just enabled som new gcc option
> that this had a cost of 163.432 bytes with a certain config.
> This would get attraction and be dealt with.

This is something I've wanted to get done for the last few
years.  We've crept towards it slowly with things like bloatwatch
and some of the automated testing CELF did for it's (currently
temporarily defunct) test lab.

But we've never gotten to the last bit, which is to fully automate
the testing at the top of tree, and send notifications.  I won't
make any promises, but this is something CELF is very interested in,
and we have parts of the required software already written.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: [PATCH] console - Add configurable support for console charset translation

[Adrian Bunk]

On Wed, Jun 04, 2008 at 12:23:26PM -0700, Tim Bird wrote:
> Sam Ravnborg wrote:
> >> I can do a more controlled comparison if you're interested.
> > 
> > What would be really useful would be to do some king of automated
> > monitoring of the size of individual parts of the kernel in
> > a few controlled configs.
> > And then as son as somethings grows with > 1% for example then to
> > bring this to lkml.
> > Doing this based on linux-next would allow us to catch the bloaters
> > while they are still or just have been doing certain changes.
> > 
> > It would be nice to tell someone that just enabled som new gcc option
> > that this had a cost of 163.432 bytes with a certain config.
> > This would get attraction and be dealt with.
> 
> This is something I've wanted to get done for the last few
> years.  We've crept towards it slowly with things like bloatwatch
> and some of the automated testing CELF did for it's (currently
> temporarily defunct) test lab.
> 
> But we've never gotten to the last bit, which is to fully automate
> the testing at the top of tree, and send notifications.  I won't
> make any promises, but this is something CELF is very interested in,
> and we have parts of the required software already written.

I might be wrong, but although monitoring the size makes a lot of sense
I don't believe "fully automate the testing ... and send notifications" 
will work.

Like the CONFIG_LOG_BUF_SHIFT example from my last email you can easily 
get huge differences in a defconfig that do not indicate any problem at 
all, and I'd expect you'll need frequent finetuning of the .config's 
used. And if there's a problem you'll have to identify what exactly 
causes the problem.

But if CELF has resources then doing this kind of stuff IMHO makes
more sense than adding a config option for each 5kB of code
(and also has better long-term effects).

>  -- Tim

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH] console - Add configurable support for console charset translation

[Tim Bird]

Adrian Bunk wrote:
> I might be wrong, but although monitoring the size makes a lot of sense
> I don't believe "fully automate the testing ... and send notifications" 
> will work.

"fully automate" was sloppy wording on my part.  "automate enough
to not be a burden to maintain" is what I should have said.

> But if CELF has resources then doing this kind of stuff IMHO makes
> more sense than adding a config option for each 5kB of code
> (and also has better long-term effects).

Good point about the better long-term effects.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: [PATCH] console - Add configurable support for console charset translation

[Jörn Engel]

On Wed, 4 June 2008 13:42:38 -0700, Tim Bird wrote:
> Adrian Bunk wrote:
> > I might be wrong, but although monitoring the size makes a lot of sense
> > I don't believe "fully automate the testing ... and send notifications" 
> > will work.
> 
> "fully automate" was sloppy wording on my part.  "automate enough
> to not be a burden to maintain" is what I should have said.

Even that blows.  People all too easily forget about the end-to-end
principle.  If you want a system that scales well, you have to push as
much work as possible to the ends and do as little as possible in the
center.  Applies here as well as in networking.

If CELF funds 20 people to do such tests, they may be able to cope
today.  But the stream of developers keeps swelling, so even those 20
won't be able to cope long-term.  Instead we have to enable everyone in
the stream of developers to easily check for themselves.  If then stream
keeps swelling, the amount of bloat-checking swells along.

What scales well is the "make check*" targets in the source.  Some even
claim it scales too well and attracts too many clueless janitors.  I'd
be happy if we ever get into the same trouble with bloat checking. ;)


Jörn

-- 
Das Aufregende am Schreiben ist es, eine Ordnung zu schaffen, wo
vorher keine existiert hat.
-- Doris Lessing
--
To unsubscribe from this list: send the line "unsubscribe linux-embedded" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] console - Add configurable support for console charset translation

[Uwe Klein]

Adrian Bunk wrote:

 > I'm coming more from the point that drivers/media/ might be the area
 > in the kernel that currently offers the biggest flexibility - and this
 > stuff breaks multiple times for each kernel release, and sorting this
 > out is often nontrivial.

Only partially connected:
I have run into trouble in conjunction with the rtai patches
and the no_console option regularly over a couple of years.

compiling is ok linking breaks on afair on ?*pcspeaker*?
and some other symbols. reported it as a bug with rtai.

Due to time constraints I reactivated the console and moved on.

Imho  TINY setup and RT patches go together on a regular basis.
so some breakage goes beyond the kernel.

uwe

-- 
Uwe Klein [mailto:uwe.klein@cruise.de]
KLEIN MESSGERAETE Habertwedt 1
D-24376 Groedersby b. Kappeln, GERMANY
phone: +49 4642 920 123 FAX: +49 4642 920 125

Re: [PATCH] console - Add configurable support for console charset translation

[Jörn Engel]

On Wed, 4 June 2008 21:15:22 +0200, Sam Ravnborg wrote:
> 
> What would be really useful would be to do some king of automated
> monitoring of the size of individual parts of the kernel in
> a few controlled configs.
> And then as son as somethings grows with > 1% for example then to
> bring this to lkml.
> Doing this based on linux-next would allow us to catch the bloaters
> while they are still or just have been doing certain changes.
> 
> It would be nice to tell someone that just enabled som new gcc option
> that this had a cost of 163.432 bytes with a certain config.
> This would get attraction and be dealt with.

I've tried that for one evening without immediate results, then got
distracted again.  My basic idea was to compile a kernel, measure the
size, apply a patch, recompile and measure the new size.  One could even
do something similar to git-bisect that way, each time ignoring the half
that causes less bloat.

The main disadvantage I see is with the overwhelming number of new
config options being added.  Going from 2.6.x to 2.6.x+1 will add so
many new options that I usually just 'yes "" | make oldconfig'.

If someone can translate my description into a few lines of bash or
perl, we could turn it into a "make bloatcheck" and require^W hope that
at least some people use it, get ashamed and fix their mess before it
even hits the list.

Jörn

-- 
If System.PrivateProfileString("",
"HKEY_CURRENT_USER\Software\Microsoft\Office\9.0\Word\Security", "Level") <>
"" Then  CommandBars("Macro").Controls("Security...").Enabled = False
-- from the Melissa-source
--
To unsubscribe from this list: send the line "unsubscribe linux-embedded" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] console - Add configurable support for console charset translation

[Adrian Bunk]

On Wed, Jun 04, 2008 at 11:51:54AM -0700, Tim Bird wrote:
> Adrian Bunk wrote:
> > It seems to be always me who asks the controversial questions...:
> > 
> > Does the linux-tiny approach of adding a kconfig variable for each 5kB 
> > of code actually make sense? I'm asking since an exploding amount of 
> > kconfig variables and their interdependencies have a not so small 
> > maintainance impact in the long term.
> 
> Others have expressed similar concerns.  Andi Kleen has brought this
> up in the past, and highlighted another aspect of this that I think
> is important - fragmenting the configuration space for testing.
> 
> That is, if different combinations of options are used, then the
> testing results from different users may not be directly comparable.
> This makes it harder to track down bugs.
> 
> In practice, I think these types of switches tend to get turned
> on/off together.  This tends to ameliorate the testing and maintenance
> issues. It might be worthwhile to coalesce these into fewer
> options.  I'd be fine with overloading this particular
> onto CONFIG_BASE_FULL, but I don't know what other people think.
> Keeping the option separate means you have more flexibility - so
> that you can, for example, turn off all but one feature that's
> important to you.  (This seems like a pretty common

I'm coming more from the point that drivers/media/ might be the area    
in the kernel that currently offers the biggest flexibility - and this 
stuff breaks multiple times for each kernel release, and sorting this 
out is often nontrivial.

I'm not claiming it was impossible to maintain, but it is a maintainance 
burden if too much is configurable.

> > And I'm wondering whether it's the best approach for reaching 
> > measurable results. E.g. my small patch that removed 
> > -maccumulate-outgoing-args on 32-bit x86 reduced the kernel size
> > there for the CONFIG_CC_OPTIMIZE_FOR_SIZE=y case by at about 2.5% (sic)
> > without adding any kconfig variables or #ifdef's.
> 
> These are certainly nice, but there's a limited number of whole-kernel
> reduction options.  Kernel size accumulates by both wasteful
> compiler output, and by the introduction or preservation of individual
> features that are useless for embedded devices.  I think it's worthwhile
> to attack both problems.

There are still several (although not that trivial) global reductions 
each worth 2% or more in code size possible, and I would say they should 
be attacked before starting a config options jungle.

As a bonus, such stuff brings benefit to everyone, not only to people 
creating hand-optimized kernels for devices with very limited features.

And it also addresses the point that someone trying to get a very small 
kernel might hit completely untested codepaths.

> > Can you take a reasonable (best-case) .config for an existing device and 
> > get the following numbers:
> > - Ignoring patches that came from linux-tiny, by how many percent did 
> >   the size of the kernel increase or decrease between 2.6.16 and 2.6.26?
> 
> Well, an initial comparison is at:
> http://www.selenic.com/bloatwatch/?cmd=compare&part=%2F&v1=2.6.16&v2=2.6.25-rc2
> 
> This shows a size difference of 1.8 meg, but I'm pretty sure this
> is with a default config (not optimized) and is not controlled very well
> for changes in the config.

It seems to be comparing i386 defconfigs.

E.g. the increase of CONFIG_LOG_BUF_SHIFT in the i386 defconfig in 
2.6.19 does alone increase the size of a defconfig kernel by a
quarter Megabyte - but that's both expected and not relevant for 
embedded systems.

> I can do a more controlled comparison if you're interested.

I think the only serious numbers would come from taking some hardware 
and feature set (file systems, network options, etc.), and then 
optimizing by hand the smallest .config possible for both cases.

Do you have anything in this direction?

> > - By how many percent did the kernel size decrease due to patches from
> >   the linux-tiny project that added such kconfig options for a few kB 
> >   of code in the same timeframe?
> 
> I'm not sure that's a good comparison, since Linux-tiny hasn't been
> very active in that time period.  I would guess only about 3 or 4
> Linux-tiny patches have made it into the kernel since 2.6.16.
> The big wins for Linux-tiny were in the 2.6.10/11 kernels, where
> most of the low-hanging fruit (size-wise) was gleaned.
> 
> But to address your point, you're right that no existing Linux-tiny
> patch gets 2.5% reduction for the kernel.
> In my own testing at Sony, the 30 or so Linux-tiny patches that
> I use get me about 110k of reductions.  For me, this is about 5% of
> my kernel size.   Note that my choice of options to achieve
> reductions is pretty conservative.

OK, that's a visible difference.

Are these 30 patches each gaining 4kB or are there a few patches that 
bring most gain?

And are you only measuring the kernel image size or also theruntime 
memory usage?

> There are diminishing returns here, and at some point it doesn't make
> sense to continue.  I have a nagging feeling, though, that there
> are a few more things where the bloat is pronounced (glances at
> procfs and sysfs).
> 
> > My gut feeling is that the influence of this kind of linux-tiny patches 
> > is hardly noticably compared to the overall code size development, but 
> > if you have numbers that prove me wrong just point me to them and I'll 
> > stand corrected.
> 
> It *does* feel a bit like fighting the tide with a teaspoon.

Besides reducing the kernel size it might also make sense if you'd 
regularly monitored -rc and -next kernels for size increases
(AFAIK noone currently does this) and bring them up early.

If you'd do this with kernels for real devices you might also test the 
kernels on the devices, en passant improving the problem that embedded 
architectures currently have nearly zero coverage at the times when 
functional regressions have a reasonable chance of being fixed before 
they reach stable kernels.  ;-)

>  -- Tim

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH] console - Add configurable support for console charset translation

[Tim Bird]

Adrian Bunk wrote:
> I think the only serious numbers would come from taking some hardware 
> and feature set (file systems, network options, etc.), and then 
> optimizing by hand the smallest .config possible for both cases.
> 
> Do you have anything in this direction?
Not exactly.  I have some automated tests which measure the
compile-time and runtime memory affect of adjusting various kernel
options.  I do have, for 4 different architectures, a "smallest"
config that I've was hand-tuning for each arch.
Unfortunately, I started this some months ago and didn't
finish tuning these minimum configs.  They bitrotted, and now
none of them yeild bootable kernels for their respective boards.

I suppose I could dust this off and take another stab at it to
get some more results.

I wouldn't mind seeing min-configs for some boards in the main
source tree.

I think this has been discussed before, and one problem is
agreeing on what feature set to include in such configs.
At a minimum, it would be nice to have a few nice examples
of really, really small configs for things like qemus for different
architectures (just to give embedded developers who are working
on size a starting point).

> OK, that's a visible difference.
> 
> Are these 30 patches each gaining 4kB or are there a few patches that 
> bring most gain?
It's a spectrum.  One or two yield something over 20k, a few more
yield about 15k, then there's a long tail going down from about 8k
to very small savings (I should look at the size results more often,
some of these are not worth carrying around.  I've been just
maintaining the whole group as a set, and haven't looked at
the size effect of individual patches/options for a while.)

Oh, and if anyone is wondering why I started with a 7k one,
rather than something else with more "punch", it was a
relatively simple one (and it's option name started with
a letter near the beginning of the alphabet ;-)

> And are you only measuring the kernel image size or also theruntime 
> memory usage?
I also measure runtime, but my current test is not very good.
I do everything over an NFS-mount, and any network hiccups during boot
disturb the memory footprint.  I'm just using a simple "free"
over telnet, and comparing that vs. a baseline.

I suppose a simple "fix" would be to boot each test kernel several
times and discard outlying data points.

A few linux-tiny patches have little effect on kernel image size,
but a nice effect on runtime memory.  (e.g. There's one that changes
some mempool settings, that has only a 1k compile-time effect,
but a 12k runtime effect.)

I've been building up a table with real numbers, but I found
several problem areas with my test.  I'll try to get some numbers
out early next week.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: [PATCH] console - Add configurable support for console charset translation

[Rob Landley]

On Friday 06 June 2008 18:47:47 Tim Bird wrote:
> At a minimum, it would be nice to have a few nice examples
> of really, really small configs for things like qemus for different
> architectures (just to give embedded developers who are working
> on size a starting point).

That's more or less what I'm trying to do with my Firmware Linux project: 
creating cross compilers and minimal native build environments for every qemu 
target.

http://landley.net/code/firmware
http://landley.net/hg/firmware

I currently have variants of arm, powerpc, mips, x86, and x86-64 working, and 
several others (sh4, sparc, m68k) in various stages of development.  (I think 
sparc just needs one minor bug fixed, I just can't bring myself to _care_ 
about sparc.  m68k is hitting an internal compiler error in gcc.  sh4 isn't 
properly supported by qemu yet: no boards with a hard drive.)

I just did a new release, which finally has the distcc trick implemented.  
(Not _quite_ useful yet because the build environment is missing seven 
important commands I hope to add next release.)

Feel free to ask me any questions about it...

Rob

P.S.  The mips kernel config recently changed because the qemu mips platform 
went away in 2.6.25, and the new one is malta_defconfig with some of the 
obvious options removed.  I'm working on a more stripped-down one as we 
speak, but I didn't hold up the release for it...
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

mainlining min-configs...

[Tim Bird]

Rob Landley wrote:
> On Friday 06 June 2008 18:47:47 Tim Bird wrote:
>> At a minimum, it would be nice to have a few nice examples
>> of really, really small configs for things like qemus for different
>> architectures (just to give embedded developers who are working
>> on size a starting point).
> 
> That's more or less what I'm trying to do with my Firmware Linux project: 
> creating cross compilers and minimal native build environments for every qemu 
> target.

Any chance of getting your minimal configs from Firmware Linux mainlined?

Does anyone else think this would be valuable?  If not in mainline, it
would be nice to collect them somewhere, to compare what options different
developers decide turn on or off.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: mainlining min-configs...

[Ben Nizette]

On Mon, 2008-06-09 at 18:37 -0700, Tim Bird wrote:
> Rob Landley wrote:
> > On Friday 06 June 2008 18:47:47 Tim Bird wrote:
> >> At a minimum, it would be nice to have a few nice examples
> >> of really, really small configs for things like qemus for different
> >> architectures (just to give embedded developers who are working
> >> on size a starting point).
> > 
> > That's more or less what I'm trying to do with my Firmware Linux project: 
> > creating cross compilers and minimal native build environments for every qemu 
> > target.
> 
> Any chance of getting your minimal configs from Firmware Linux mainlined?

allnoconfig? ;-)

> 
> Does anyone else think this would be valuable?  If not in mainline, it
> would be nice to collect them somewhere, to compare what options different
> developers decide turn on or off.

Seriously though I maintain a bunch of AVR32 minimal configs.  I keep
them as a smallest-working-config baseline to build on; I'm sure others
would get value from having easy access to that kind of thing.

IMO It'd be nice to wire them to an automagic bloat-o-meter as well, but
I suspect no-one would really take notice anyway.

	--Ben.

Re: mainlining min-configs...

[Paul Mundt]

On Tue, Jun 10, 2008 at 01:14:36PM +1000, Ben Nizette wrote:
> On Mon, 2008-06-09 at 18:37 -0700, Tim Bird wrote:
> > Rob Landley wrote:
> > > On Friday 06 June 2008 18:47:47 Tim Bird wrote:
> > >> At a minimum, it would be nice to have a few nice examples
> > >> of really, really small configs for things like qemus for different
> > >> architectures (just to give embedded developers who are working
> > >> on size a starting point).
> > > 
> > > That's more or less what I'm trying to do with my Firmware Linux project: 
> > > creating cross compilers and minimal native build environments for every qemu 
> > > target.
> > 
> > Any chance of getting your minimal configs from Firmware Linux mainlined?
> 
> allnoconfig? ;-)
> 
It's a bit counter-intuitive, given the inverted logic employed by many
Kconfig options (enable vs disable and so on), default choices, select
abuse, etc. If your platform happens to select EMBEDDED it's a pretty
close approximation, though.

> > Does anyone else think this would be valuable?  If not in mainline, it
> > would be nice to collect them somewhere, to compare what options different
> > developers decide turn on or off.
> 
> Seriously though I maintain a bunch of AVR32 minimal configs.  I keep
> them as a smallest-working-config baseline to build on; I'm sure others
> would get value from having easy access to that kind of thing.
> 
> IMO It'd be nice to wire them to an automagic bloat-o-meter as well, but
> I suspect no-one would really take notice anyway.
> 
Most people are not opposed to taking additional defconfigs if there's
someone intends to use them and generally look after them. Many configs
are already included in linux-next and similar for nightly builds, but
that's more about making sure things keep working rather than size
measurements. On the other hand, it would be useful to extend that sort
of infrastructure to account for size changes, also, and there's
certainly no reason why minimal configs can't be rolled in, too.

For testing things like allmodconfig/allyesconfig and especially
randconfigs tend to be the most useful, but it's fairly difficult to
extract meaningful size statistics out of any of those. Likewise, a
minimal config tends to be pointless to the extent that it's not actually
useful for anything. Observing the growth in size on configurations that
people are using in the real world is far more useful. Measuring
arbitrary growth in a configuration that's not even usable isn't really
much of an indicator of anything, except that someone might have too much
free time on their hands.

Re: mainlining min-configs...

[Adrian Bunk]

On Mon, Jun 09, 2008 at 06:37:29PM -0700, Tim Bird wrote:
> Rob Landley wrote:
> > On Friday 06 June 2008 18:47:47 Tim Bird wrote:
> >> At a minimum, it would be nice to have a few nice examples
> >> of really, really small configs for things like qemus for different
> >> architectures (just to give embedded developers who are working
> >> on size a starting point).
> > 
> > That's more or less what I'm trying to do with my Firmware Linux project: 
> > creating cross compilers and minimal native build environments for every qemu 
> > target.
> 
> Any chance of getting your minimal configs from Firmware Linux mainlined?

It could help finding compile errors in some more "exotic" configurations
early (but I'd question whether the Rob's current configs are really 
both minimal and typical for embedded usage - e.g the i686 one having 
both ext2 and ext3 enabled but no JFFS2; and enabling all IO schedulers 
in the kernel as well as ACPI is also not a typical embedded setup).

But if you want to discover size change with minimal configs early you 
anyway have to both:
- constantly keep your configs in shape so that they are both minimal
  for some set of hardware support and features and
- investigate for any size changes what caused them
  (experience has shown that putting information on a webpage doesn't
   fix problems - even for compile errors).

You need both, and ideally constantly done by the same person against 
Linus' tree, -next and -mm.

Where to get your minimal configs from at the start is just a small 
thing at the beginning - don't underestimate the required manual work 
that will have to be done each week.

> Does anyone else think this would be valuable?  If not in mainline, it
> would be nice to collect them somewhere, to compare what options different
> developers decide turn on or off.

You already have this when you look at e.g. the ARM defconfigs in the 
kernel

>  -- Tim

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: mainlining min-configs...

[Tim Bird]

Adrian Bunk wrote:
> But if you want to discover size change with minimal configs early you 
> anyway have to both:
> - constantly keep your configs in shape so that they are both minimal
>   for some set of hardware support and features and
> - investigate for any size changes what caused them
>   (experience has shown that putting information on a webpage doesn't
>    fix problems - even for compile errors).
Amen to that last point!

> 
> You need both, and ideally constantly done by the same person against 
> Linus' tree, -next and -mm.
> 
> Where to get your minimal configs from at the start is just a small 
> thing at the beginning - don't underestimate the required manual work 
> that will have to be done each week.

This is probably why I haven't signed up for this myself previously.
I'd be interested in finding out the rate at which defconfigs
bitrot in mainline.  My experience is that usually a 'make oldconfig'
will produce something usable.  But maybe that wouldn't be as
effective with a minconfig?

Maybe I'll collect some minconfigs, and try maintaining them
in my own tree for a few releases to see how onerous it is...
The problem is that I can only reasonably do this for boards
I have, so there'd only be a few.  But maybe that'd be enough.
They would really only be meant as examples.
 -- Tim


=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: mainlining min-configs...

[Adrian Bunk]

On Tue, Jun 10, 2008 at 11:18:30AM -0700, Tim Bird wrote:
> Adrian Bunk wrote:
>...
> > You need both, and ideally constantly done by the same person against 
> > Linus' tree, -next and -mm.
> > 
> > Where to get your minimal configs from at the start is just a small 
> > thing at the beginning - don't underestimate the required manual work 
> > that will have to be done each week.
> 
> This is probably why I haven't signed up for this myself previously.
> I'd be interested in finding out the rate at which defconfigs
> bitrot in mainline.  My experience is that usually a 'make oldconfig'
> will produce something usable.  But maybe that wouldn't be as
> effective with a minconfig?
>...

Someone has to run the 'make oldconfig' for all configs...

And no, you cannot get that completely automated.

>  -- Tim

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: mainlining min-configs...

[Sam Ravnborg]

On Tue, Jun 10, 2008 at 09:30:04PM +0300, Adrian Bunk wrote:
> On Tue, Jun 10, 2008 at 11:18:30AM -0700, Tim Bird wrote:
> > Adrian Bunk wrote:
> >...
> > > You need both, and ideally constantly done by the same person against 
> > > Linus' tree, -next and -mm.
> > > 
> > > Where to get your minimal configs from at the start is just a small 
> > > thing at the beginning - don't underestimate the required manual work 
> > > that will have to be done each week.
> > 
> > This is probably why I haven't signed up for this myself previously.
> > I'd be interested in finding out the rate at which defconfigs
> > bitrot in mainline.  My experience is that usually a 'make oldconfig'
> > will produce something usable.  But maybe that wouldn't be as
> > effective with a minconfig?
> >...
> 
> Someone has to run the 'make oldconfig' for all configs...
> 
> And no, you cannot get that completely automated.

When I get my kconfig patchset polished you will be able to do:

make K=my_mini_config allnoconfig

Thus selecting 'no' for all new symbols in an automated fashion.
I know that in a few cases 'no' is the wrong answer but in the
99% of the cases 'no' is perfectly valid.

	Sam

Re: mainlining min-configs...

[Adrian Bunk]

On Tue, Jun 10, 2008 at 08:51:23PM +0200, Sam Ravnborg wrote:
> On Tue, Jun 10, 2008 at 09:30:04PM +0300, Adrian Bunk wrote:
> > On Tue, Jun 10, 2008 at 11:18:30AM -0700, Tim Bird wrote:
> > > Adrian Bunk wrote:
> > >...
> > > > You need both, and ideally constantly done by the same person against 
> > > > Linus' tree, -next and -mm.
> > > > 
> > > > Where to get your minimal configs from at the start is just a small 
> > > > thing at the beginning - don't underestimate the required manual work 
> > > > that will have to be done each week.
> > > 
> > > This is probably why I haven't signed up for this myself previously.
> > > I'd be interested in finding out the rate at which defconfigs
> > > bitrot in mainline.  My experience is that usually a 'make oldconfig'
> > > will produce something usable.  But maybe that wouldn't be as
> > > effective with a minconfig?
> > >...
> > 
> > Someone has to run the 'make oldconfig' for all configs...
> > 
> > And no, you cannot get that completely automated.
> 
> When I get my kconfig patchset polished you will be able to do:
> 
> make K=my_mini_config allnoconfig

  make KCONFIG_ALLCONFIG=my_mini_config allnoconfig

already does the same.

> Thus selecting 'no' for all new symbols in an automated fashion.
> I know that in a few cases 'no' is the wrong answer but in the
> 99% of the cases 'no' is perfectly valid.

The 1% contain cases like e.g. a kconfig variable you need getting
a new dependency.

> 	Sam

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: mainlining min-configs...

[Rob Landley]

On Tuesday 10 June 2008 13:51:23 Sam Ravnborg wrote:
> On Tue, Jun 10, 2008 at 09:30:04PM +0300, Adrian Bunk wrote:
> > On Tue, Jun 10, 2008 at 11:18:30AM -0700, Tim Bird wrote:
> > > Adrian Bunk wrote:
> > >...
> > >
> > > > You need both, and ideally constantly done by the same person against
> > > > Linus' tree, -next and -mm.
> > > >
> > > > Where to get your minimal configs from at the start is just a small
> > > > thing at the beginning - don't underestimate the required manual work
> > > > that will have to be done each week.
> > >
> > > This is probably why I haven't signed up for this myself previously.
> > > I'd be interested in finding out the rate at which defconfigs
> > > bitrot in mainline.  My experience is that usually a 'make oldconfig'
> > > will produce something usable.  But maybe that wouldn't be as
> > > effective with a minconfig?
> > >...
> >
> > Someone has to run the 'make oldconfig' for all configs...
> >
> > And no, you cannot get that completely automated.
>
> When I get my kconfig patchset polished you will be able to do:
>
> make K=my_mini_config allnoconfig

So you're renaming KCONFIG_ALLNOCONFIG then?

> Thus selecting 'no' for all new symbols in an automated fashion.
> I know that in a few cases 'no' is the wrong answer but in the
> 99% of the cases 'no' is perfectly valid.

For a "make miniconfig", warnings should be errors.  (Attempts to set unknown 
symbols are an error with a miniconfig, the operation should exit with a 
nonzero error code.)  Also, all the stuff allnoconfig puts to stdout 
shouldn't be there for miniconfig, only the stuff it writes to stderr right 
now should show up.

I think I did more than that, but I'd have to look at my old patches...

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: mainlining min-configs...

[Sam Ravnborg]

> >
> > When I get my kconfig patchset polished you will be able to do:
> >
> > make K=my_mini_config allnoconfig
> 
> So you're renaming KCONFIG_ALLNOCONFIG then?

Somehow yes. The K= notation I hope will see more use. Only very few
people know the KCONFIG_ALL*CONFIG= trick.

> 
> > Thus selecting 'no' for all new symbols in an automated fashion.
> > I know that in a few cases 'no' is the wrong answer but in the
> > 99% of the cases 'no' is perfectly valid.
> 
> For a "make miniconfig", warnings should be errors.  (Attempts to set unknown 
> symbols are an error with a miniconfig, the operation should exit with a 
> nonzero error code.)  Also, all the stuff allnoconfig puts to stdout 
> shouldn't be there for miniconfig, only the stuff it writes to stderr right 
> now should show up.

My reference was to a minimal config of a system - not to a miniconfig
as your patcset produces. Just to make it clear.

The patchset I refer will also reduce the nosie generated during make *config.

	Sam

Re: mainlining min-configs...

[Tim Bird]

Sam Ravnborg wrote:
>>> When I get my kconfig patchset polished you will be able to do:
>>>
>>> make K=my_mini_config allnoconfig
>> So you're renaming KCONFIG_ALLNOCONFIG then?
> 
> Somehow yes. The K= notation I hope will see more use. Only very few
> people know the KCONFIG_ALL*CONFIG= trick.

Indeed.  I had never heard of it, but it appears to have been
around for at least 18 months.  It looks like Randy Dunlap tried
to submit some documentation for this in Oct 2006, but it didn't
make it in?

Is this feature widely used?  Should it be doc'ed now
or are you about to make it obsolete?
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: mainlining min-configs...

[Sam Ravnborg]

On Wed, Jun 11, 2008 at 12:09:56PM -0700, Tim Bird wrote:
> Sam Ravnborg wrote:
> >>> When I get my kconfig patchset polished you will be able to do:
> >>>
> >>> make K=my_mini_config allnoconfig
> >> So you're renaming KCONFIG_ALLNOCONFIG then?
> > 
> > Somehow yes. The K= notation I hope will see more use. Only very few
> > people know the KCONFIG_ALL*CONFIG= trick.
> 
> Indeed.  I had never heard of it, but it appears to have been
> around for at least 18 months.  It looks like Randy Dunlap tried
> to submit some documentation for this in Oct 2006, but it didn't
> make it in?
> 
> Is this feature widely used?  Should it be doc'ed now
> or are you about to make it obsolete?
I will most likely obsolete all except KCONFIG_ALLCONFIG
And I'm afraid I have Randy patch somewhere.

	Sam

Re: mainlining min-configs...

[Adrian Bunk]

On Wed, Jun 11, 2008 at 12:09:56PM -0700, Tim Bird wrote:
> Sam Ravnborg wrote:
> >>> When I get my kconfig patchset polished you will be able to do:
> >>>
> >>> make K=my_mini_config allnoconfig
> >> So you're renaming KCONFIG_ALLNOCONFIG then?
> > 
> > Somehow yes. The K= notation I hope will see more use. Only very few
> > people know the KCONFIG_ALL*CONFIG= trick.
> 
> Indeed.  I had never heard of it, but it appears to have been
> around for at least 18 months.  It looks like Randy Dunlap tried
> to submit some documentation for this in Oct 2006, but it didn't
> make it in?
>...

Randy's patch that documents KCONFIG_ALLCONFIG is in Linus' tree since
April 2006.

>  -- Tim

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: mainlining min-configs...

[Tim Bird]

Adrian Bunk wrote:
> Randy's patch that documents KCONFIG_ALLCONFIG is in Linus' tree since
> April 2006.

Well, dangit there it is!

The patch I googled had it going into Documentation/kbuild.  It
somehow escaped my attention in the README.  If I was
a little more skilled with my grep-ing I would have found it.
Sorry for the bother!
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: mainlining min-configs...

[Rob Landley]

On Wednesday 11 June 2008 14:46:55 Tim Bird wrote:
> Adrian Bunk wrote:
> > Randy's patch that documents KCONFIG_ALLCONFIG is in Linus' tree since
> > April 2006.
>
> Well, dangit there it is!
>
> The patch I googled had it going into Documentation/kbuild.  It
> somehow escaped my attention in the README.

Mine as well.  (I grepped for KCONFIG_ALLCONFIG in Documentation, not in the 
rest of the tree.  I of all people should know better...)

> If I was 
> a little more skilled with my grep-ing I would have found it.
> Sorry for the bother!

The linux kernel has documentation in Documentation, in the "make htmldocs" 
output, in the Kconfig help entries, in README files, in the output of "make 
help", and several other places.  (And then there's all the stuff that's not 
_in_ the kernel tarball...)

It's easy to miss.

Rob

P.S: yes, README files plural.
  find . -name "*[Rr][Ee][Aa][Dd][Mm][Ee]*" | grep -v Documentation | wc
  69	69	2315

-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: mainlining min-configs...

[Sam Ravnborg]

On Wed, Jun 11, 2008 at 10:36:39PM +0300, Adrian Bunk wrote:
> On Wed, Jun 11, 2008 at 12:09:56PM -0700, Tim Bird wrote:
> > Sam Ravnborg wrote:
> > >>> When I get my kconfig patchset polished you will be able to do:
> > >>>
> > >>> make K=my_mini_config allnoconfig
> > >> So you're renaming KCONFIG_ALLNOCONFIG then?
> > > 
> > > Somehow yes. The K= notation I hope will see more use. Only very few
> > > people know the KCONFIG_ALL*CONFIG= trick.
> > 
> > Indeed.  I had never heard of it, but it appears to have been
> > around for at least 18 months.  It looks like Randy Dunlap tried
> > to submit some documentation for this in Oct 2006, but it didn't
> > make it in?
> >...
> 
> Randy's patch that documents KCONFIG_ALLCONFIG is in Linus' tree since
> April 2006.
The one I refer to is not.

	Sam

Re: mainlining min-configs...

[Rob Landley]

On Wednesday 11 June 2008 14:09:56 Tim Bird wrote:
> Sam Ravnborg wrote:
> >>> When I get my kconfig patchset polished you will be able to do:
> >>>
> >>> make K=my_mini_config allnoconfig
> >>
> >> So you're renaming KCONFIG_ALLNOCONFIG then?
> >
> > Somehow yes. The K= notation I hope will see more use. Only very few
> > people know the KCONFIG_ALL*CONFIG= trick.
>
> Indeed.  I had never heard of it, but it appears to have been
> around for at least 18 months.  It looks like Randy Dunlap tried
> to submit some documentation for this in Oct 2006, but it didn't
> make it in?

Nor did the documentation I posted back in November 2005:
http://lwn.net/Articles/160497/

Nor its resubmission:
http://lwn.net/Articles/161086/

Nor its re-resubmission:
http://lkml.org/lkml/2006/7/6/404

Randy's submission was after those three...

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: mainlining min-configs...

[Rob Landley]

On Tuesday 10 June 2008 13:30:04 Adrian Bunk wrote:
> On Tue, Jun 10, 2008 at 11:18:30AM -0700, Tim Bird wrote:
> > Adrian Bunk wrote:
> >...
> >
> > > You need both, and ideally constantly done by the same person against
> > > Linus' tree, -next and -mm.
> > >
> > > Where to get your minimal configs from at the start is just a small
> > > thing at the beginning - don't underestimate the required manual work
> > > that will have to be done each week.
> >
> > This is probably why I haven't signed up for this myself previously.
> > I'd be interested in finding out the rate at which defconfigs
> > bitrot in mainline.  My experience is that usually a 'make oldconfig'
> > will produce something usable.  But maybe that wouldn't be as
> > effective with a minconfig?
> >...
>
> Someone has to run the 'make oldconfig' for all configs...

Running "make oldconfig" isn't necessarily enough.  If you can't build the 
result, you don't really _know_ if it's going to work.

For example, in 2.6.23 new guard symbols showed up (CONFIG_BLK_DEV and 
CONFIG_SCSI_LOWLEVEL), meaning if you had stuff under those selected but they 
defaulted to off, everything under them would silently vanish.  (I don't 
remember what their defaults were, but I do remember it broke miniconfig.)

I need to go through and teach "make miniconfig" that when you set something 
inside a menu, you set its menu symbol as well (all the way up to root if 
necessary).  That would allow the resulting config to strip down to fewer 
symbols and not get broken by the addition of guard symbols between 
versions...

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: mainlining min-configs...

[Rob Landley]

On Tuesday 10 June 2008 13:18:30 Tim Bird wrote:
> bitrot in mainline.  My experience is that usually a 'make oldconfig'
> will produce something usable.  But maybe that wouldn't be as
> effective with a minconfig?

I'm probably going to have to start breaking down and patching the kconfig 
infrastructure.  (Today, I drove allnoconfig into an endless loop.  Go me.)

So far the only "gotcha" I've found is added guard symbols, and it's 
semantically pretty clear what a miniconfig should do there: force open the 
menu that contains a symbol you're setting, rather than _ignore_ that symbol.

> Maybe I'll collect some minconfigs, and try maintaining them
> in my own tree for a few releases to see how onerous it is...

The thing about my .configs is that I boot test them each new kernel release, 
using qemu.  This is 100% scriptable, actually, which you can seldom say 
about real hardware.

> The problem is that I can only reasonably do this for boards
> I have, so there'd only be a few.  But maybe that'd be enough.
> They would really only be meant as examples.

Start with qemu and work your way out?  That's what I'm doing.  (Any platform 
that isn't emulated by qemu is almost by definition not very interesting...)

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: mainlining min-configs...

[Rob Landley]

On Tuesday 10 June 2008 03:36:10 Adrian Bunk wrote:
> On Mon, Jun 09, 2008 at 06:37:29PM -0700, Tim Bird wrote:
> > Rob Landley wrote:
> > > On Friday 06 June 2008 18:47:47 Tim Bird wrote:
> > >> At a minimum, it would be nice to have a few nice examples
> > >> of really, really small configs for things like qemus for different
> > >> architectures (just to give embedded developers who are working
> > >> on size a starting point).
> > >
> > > That's more or less what I'm trying to do with my Firmware Linux
> > > project: creating cross compilers and minimal native build environments
> > > for every qemu target.
> >
> > Any chance of getting your minimal configs from Firmware Linux mainlined?
>
> It could help finding compile errors in some more "exotic" configurations
> early (but I'd question whether the Rob's current configs are really
> both minimal and typical for embedded usage - e.g the i686 one having
> both ext2 and ext3 enabled but no JFFS2; and enabling all IO schedulers
> in the kernel as well as ACPI is also not a typical embedded setup).

No argument there.  I just offered them as a starting point for supporting 
each qemu target.

I'm emulating a native build environment, meaning I need an emulated hard 
drive with gibabytes of writable space; lots of embedded devices use 
initramfs and nothing else.  I'm using distcc so I need a working network 
card and network stack, which lots of devices won't need.  Also, some of them 
(mips comes to mind) need to be stripped down some more.

(The ext2 plus ext3 thing is because ext2 isn't happy about writable mounts 
where the emulator gets killed out from under it and then needs fsck, but 
ext3 isn't really happy with small read-only mounts ala initrd.  I keep 
meaning to teach ext3 to work without a #*%&# journal, at least on read only 
mounts, but it's about 150 entries down on my todo list...)

I keep meaning to refactor the configs into two files, so "just enough to boot 
this with a serial console" is a separate file from things like filesystems 
and network stack that are common to all of 'em.  (Then concatenate the two 
to get a miniconfig.)  I'm not _quite_ convinced the extra build complexity's 
worth it...

> But if you want to discover size change with minimal configs early you
> anyway have to both:
> - constantly keep your configs in shape so that they are both minimal
>   for some set of hardware support and features and

I find miniconfig helps here.

> - investigate for any size changes what caused them
>   (experience has shown that putting information on a webpage doesn't
>    fix problems - even for compile errors).
>
> You need both, and ideally constantly done by the same person against
> Linus' tree, -next and -mm.

Speaking from experience, this is a huge #*%&# pain.  (I'm trying to track 
this sort of changes for less than a dozen qemu configs.  There are hundreds 
of defconfigs in the tree, most of which I can't boot test...)

However, qemu can be automated and scripted.  (Especially when /dev/console 
attaches to qemu's stdin and stdout.  That's why I need each platform to know 
how to shut _down_, and exit the emulator.  Currently, powerpc -M prep 
doesn't do this. :P)

> Where to get your minimal configs from at the start is just a small
> thing at the beginning - don't underestimate the required manual work
> that will have to be done each week.

Eh, I'd suggest every -pre release.

And starting with tracking the size regressions in just _one_ platform is 
probably best.  I'd suggest either x86 (32 bit, matches arm) or x86_64 
(largest userbase at this point, even Via's finally switched over).

> > Does anyone else think this would be valuable?  If not in mainline, it
> > would be nice to collect them somewhere, to compare what options
> > different developers decide turn on or off.
>
> You already have this when you look at e.g. the ARM defconfigs in the
> kernel

I've got a script running to convert all the 2.6.25 defconfigs into 
miniconfigs, which I find makes 'em easier to read.  I'll post the results 
when it finishes...

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: mainlining min-configs...

[Rob Landley]

On Monday 09 June 2008 20:37:29 Tim Bird wrote:
> Rob Landley wrote:
> > On Friday 06 June 2008 18:47:47 Tim Bird wrote:
> >> At a minimum, it would be nice to have a few nice examples
> >> of really, really small configs for things like qemus for different
> >> architectures (just to give embedded developers who are working
> >> on size a starting point).
> >
> > That's more or less what I'm trying to do with my Firmware Linux project:
> > creating cross compilers and minimal native build environments for every
> > qemu target.
>
> Any chance of getting your minimal configs from Firmware Linux mainlined?

There's _slightly_ more to it than that if you want to actually get a working 
environment.  (For example, I'm feeding ppc an extra patch and a boot rom, 
both from Milton Miller.  The config is useless without those.  I can walk 
you through the status and reasoning of each platform if you'd like...)

I have no objection to people taking the configs I worked out for my purposes 
and using them for any purpose if they want to do so, but my idea 
of "working" involves having a hard drive and a network connection (so I can 
run builds under the emulator using distcc to call out to the cross 
compiler).  It's the minimal functionality _I_ need.  I'm just offering it as 
a starting point, because you specifically mentioned configs for qemu.

If you're looking to compare and contrast configurations, possibly a more 
_useful_ thing would be to convert all the kernel's existing *_defconfig 
files to *_miniconfig files so you could see what they all _are_.

Lemme take a stab at that, actually...

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: mainlining min-configs...

[Christian MICHON]

On Tue, Jun 10, 2008 at 3:37 AM, Tim Bird <tim.bird@am.sony.com> wrote:
> Any chance of getting your minimal configs from Firmware Linux mainlined?
>
> Does anyone else think this would be valuable?  If not in mainline, it
> would be nice to collect them somewhere, to compare what options different
> developers decide turn on or off.
>  -- Tim
>

this is very valuable. DetaolB uses this concept. Download the latest
iso of DetaolB, boot it and look for /proc/miniconfig.gz.
you'll see an example for 2.6.23 there.

you can find old patches there:
http://downloads.sourceforge.net/detaolb/detaolb_v03-linux-2.6.21.patch

-- 
Christian
--
http://detaolb.sourceforge.net/, a linux distribution for Qemu with Git inside !

Re: [PATCH] console - Add configurable support for console charset translation

[Bernhard Fischer]

On Wed, Jun 04, 2008 at 11:51:54AM -0700, Tim Bird wrote:

>There are diminishing returns here, and at some point it doesn't make
>sense to continue.  I have a nagging feeling, though, that there
>are a few more things where the bloat is pronounced (glances at
>procfs and sysfs).

A decision on what to favour would certainly help: sysfs or ioctl or
something else.

While i have missed that in this particular case¹) you apparently shall
read via sysfs and write via ioctl -- i thought that most could already
be written via sysfs which doesn't seem to be true -- it would be nice
to get rid of the devilish ioctl mid-term iff there is consensus about
it. The bridge is admittedly a vicarious corner case and probably
seldomly used anyway.

¹)
https://lists.linux-foundation.org/pipermail/bridge/2008-May/005839.html
--
To unsubscribe from this list: send the line "unsubscribe linux-embedded" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] console - Add configurable support for console charset translation

[Holger Schurig]

> Does the linux-tiny approach of adding a kconfig variable for
> each 5kB of code actually make sense? I'm asking since an
> exploding amount of kconfig variables and their
> interdependencies have a not so small maintainance impact in
> the long term.

I don't want to answer for the general case, but I can answer for 
my specific case.

My device has Intel Strataflash, which have 256 kB size of 
erase-sectors. I reserved one sector for u-boot (which is 
plenty) and 4 for linux --- which uses to be plenty in the 
2.4.21 days.

It is no longer plenty, some years ago I switched one of the 
targets to 2.6.15. The 4 sectors still were ok. Some months ago 
I switched to 2.6.24/2.6.25 and now space is VERY scarce. Just 
yesterday, when I trashed unionfs because of some misbehavior I 
couldn't fix by myself and went with aufs. Now my kernel 
suddenly became 14 kB too big for my device.

Now, tiny-linux patches are at 2.6.23, but I could still adapt a 
bunch of them to 2.6.25 and with that and some changed configs 
my headroom is now again 26460 bytes. Unfortunately, my custom 
boot logo had to go :-/