/proc/config reducing kernel image size

/proc/config reducing kernel image size

[Tom Fredrik Blenning Klaussen]

[-- Attachment #1: Type: text/plain, Size: 763 bytes --]

I've written a small patch that reduces the size of /proc/config.

There is no point in storing all the comments and unused options in the
kernel image. This typically reduces the config size to about 1/5th
before compressing, and to about 1/4th after compressing.

I've also added the configuration option of how you want to compress it.

In order to do the comment stripping I was forced to add a small script
to the scripts, section, this has to be made executable. However if it
is possible, the script should be skipped altogether, but I can't figure
out how to do the proper escaping of the special characters in the
kernel/Makefile. So if anyone could help me with that, I would be
grateful.

I'm interessted in comments on the concept.

Sincerely
-- 
BFG

[-- Attachment #2: linux-2.6.8.1-config-compress-patch.diff --]
[-- Type: text/x-patch, Size: 6697 bytes --]

diff -ruN /usr/src/linux-2.6.8.1/init/Kconfig ./init/Kconfig
--- /usr/src/linux-2.6.8.1/init/Kconfig	2004-08-31 13:11:45.000000000 +0200
+++ ./init/Kconfig	2004-09-12 19:51:06.000000000 +0200
@@ -197,6 +197,8 @@
 	  agent" (/sbin/hotplug) to load modules and set up software needed
 	  to use devices as you hotplug them.
 
+menu "Kernel .config support"
+
 config IKCONFIG
 	bool "Kernel .config support"
 	---help---
@@ -209,13 +211,71 @@
 	  It can also be extracted from a running kernel by reading
 	  /proc/config.gz if enabled (below).
 
-config IKCONFIG_PROC
-	bool "Enable access to .config through /proc/config.gz"
-	depends on IKCONFIG && PROC_FS
+config IKCONFIG_STRIP
+	bool "Strip comments and unused options"
+	depends on IKCONFIG
+	---help---
+	  This option strips all comments and options that increases 
+          human readability, but are unnecessary for the parsing 
+          process.
+
+choice
+        prompt "Config compression method"
+	depends on IKCONFIG
+        default IKCONFIG_COMPRESS_NONE
+	---help---
+	  Choose which kind of compression that will be used for the 
+          kernel copy of the config file.
+
+config IKCONFIG_COMPRESS_NONE
+	bool "None"
+	---help---
+	  This option disables config file compression.
+
+
+config IKCONFIG_COMPRESS_GZIP
+	bool "Gzip"
+	---help---
+	  This option enables gzip config file compression.
+
+          Gzip is the standard compression utility that can be found on 
+          any UNIX machine.
+
+config IKCONFIG_COMPRESS_BZIP2
+	bool "Bzip2 (RECOMMENDED)"
 	---help---
-	  This option enables access to the kernel configuration file
-	  through /proc/config.gz.
+	  This option enables bzip2 config file compression.
+
+          Bzip2 is designed for text compression, and is slightly better 
+          than gzip for this purpose.
 
+endchoice
+
+config IKCONFIG_PROC
+        bool "Enable access to .config through /proc/config"
+        depends on IKCONFIG_COMPRESS_NONE && PROC_FS
+        ---help---
+          This option enables access to the kernel configuration file
+          through /proc/config.
+
+
+config IKCONFIG_PROC
+        bool "Enable access to .config through /proc/config.gz"
+        depends on IKCONFIG_COMPRESS_GZIP && PROC_FS
+        ---help---
+          This option enables access to the kernel configuration file
+          through /proc/config.gz.
+
+
+config IKCONFIG_PROC
+        bool "Enable access to .config through /proc/config.bz2"
+        depends on IKCONFIG_COMPRESS_BZIP2 && PROC_FS
+        ---help---
+          This option enables access to the kernel configuration file
+          through /proc/config.bz2.
+
+
+endmenu
 
 menuconfig EMBEDDED
 	bool "Configure standard kernel features (for small systems)"
diff -ruN /usr/src/linux-2.6.8.1/kernel/Makefile ./kernel/Makefile
--- /usr/src/linux-2.6.8.1/kernel/Makefile	2004-08-31 13:11:45.000000000 +0200
+++ ./kernel/Makefile	2004-09-12 20:08:02.000000000 +0200
@@ -19,6 +19,8 @@
 obj-$(CONFIG_BSD_PROCESS_ACCT) += acct.o
 obj-$(CONFIG_COMPAT) += compat.o
 obj-$(CONFIG_IKCONFIG) += configs.o
+#Is this strictly needed? CONFIG_IKCONFIG_PROC should only be defined 
+# if CONFIG_IKCONFIG is defined.
 obj-$(CONFIG_IKCONFIG_PROC) += configs.o
 obj-$(CONFIG_STOP_MACHINE) += stop_machine.o
 obj-$(CONFIG_AUDIT) += audit.o
@@ -35,14 +37,48 @@
 
 $(obj)/configs.o: $(obj)/config_data.h
 
-# config_data.h contains the same information as ikconfig.h but gzipped.
-# Info from config_data can be extracted from /proc/config*
-targets += config_data.gz
-$(obj)/config_data.gz: .config FORCE
-	$(call if_changed,gzip)
 
-quiet_cmd_ikconfiggz = IKCFG   $@
-      cmd_ikconfiggz = (echo "const char kernel_config_data[] = MAGIC_START"; cat $< | scripts/bin2c; echo "MAGIC_END;") > $@
+#define the stripping command, basically strip or copy
+ifdef CONFIG_IKCONFIG_STRIP
+quiet_cmd_ikconfig_strip = STRIP   $@
+	cmd_ikconfig_strip = scripts/stripCommBlank.sh < $< > $@
+else
+quiet_cmd_ikconfig_strip = COPY    $@
+	cmd_ikconfig_strip = cp $< $@
+endif
+
+# config_data_in contains the (stripped?) config file
+targets += config_data_in
+$(obj)/config_data_in: .config FORCE
+	$(call if_changed,ikconfig_strip)
+
+
+#define the compression command, none,gzip or bzip2
+ifdef CONFIG_IKCONFIG_COMPRESS_NONE
+quiet_cmd_ikconfig_compress = COPY    $@
+	cmd_ikconfig_compress = cp $< $@
+endif
+
+ifdef CONFIG_IKCONFIG_COMPRESS_GZIP
+quiet_cmd_ikconfig_compress = GZIP    $@
+	cmd_ikconfig_compress = gzip --best < $< > $@
+endif
+
+ifdef CONFIG_IKCONFIG_COMPRESS_BZIP2
+quiet_cmd_ikconfig_compress = BZIP2   $@
+	cmd_ikconfig_compress = bzip2 --best < $< > $@
+endif
+
+
+# config_data_compressed contains the (compressed?) config file
+targets += config_data_compressed
+$(obj)/config_data_compressed: $(obj)/config_data_in FORCE
+	$(call if_changed,ikconfig_compress)
+
+quiet_cmd_ikconfig = IKCFG   $@
+      cmd_ikconfig = (echo "const char kernel_config_data[] = MAGIC_START"; cat $< | scripts/bin2c; echo "MAGIC_END;") > $@
+
+# Info from config_data can be extracted from /proc/config*
 targets += config_data.h
-$(obj)/config_data.h: $(obj)/config_data.gz FORCE
-	$(call if_changed,ikconfiggz)
+$(obj)/config_data.h: $(obj)/config_data_compressed FORCE
+	$(call if_changed,ikconfig)
diff -ruN /usr/src/linux-2.6.8.1/kernel/configs.c ./kernel/configs.c
--- /usr/src/linux-2.6.8.1/kernel/configs.c	2004-08-31 13:11:45.000000000 +0200
+++ ./kernel/configs.c	2004-09-12 19:51:06.000000000 +0200
@@ -55,6 +55,16 @@
 
 #ifdef CONFIG_IKCONFIG_PROC
 
+#ifdef CONFIG_IKCONFIG_COMPRESS_NONE
+#define PROC_CONFIG_FILENAME "config"
+#elif CONFIG_IKCONFIG_COMPRESS_GZIP
+#define PROC_CONFIG_FILENAME "config.gz"
+#elif CONFIG_IKCONFIG_COMPRESS_BZIP2
+#define PROC_CONFIG_FILENAME "config.bz2"
+#else
+#error Proc config filename could not be defined
+#endif
+
 /**************************************************/
 /* globals and useful constants                   */
 
@@ -89,7 +99,7 @@
 	struct proc_dir_entry *entry;
 
 	/* create the current config file */
-	entry = create_proc_entry("config.gz", S_IFREG | S_IRUGO,
+	entry = create_proc_entry(PROC_CONFIG_FILENAME, S_IFREG | S_IRUGO,
 				  &proc_root);
 	if (!entry)
 		return -ENOMEM;
diff -ruN /usr/src/linux-2.6.8.1/scripts/stripCommBlank.sh ./scripts/stripCommBlank.sh
--- /usr/src/linux-2.6.8.1/scripts/stripCommBlank.sh	1970-01-01 01:00:00.000000000 +0100
+++ ./scripts/stripCommBlank.sh	2004-09-12 19:51:06.000000000 +0200
@@ -0,0 +1,2 @@
+#!/bin/sh
+egrep -v '^#|^[:space:]*$'

Re: /proc/config reducing kernel image size

[DervishD]

    Hi Tom :)

 * Tom Fredrik Blenning Klaussen <bfg-kernel@blenning.no> dixit:
> There is no point in storing all the comments and unused options in the
> kernel image. This typically reduces the config size to about 1/5th
> before compressing, and to about 1/4th after compressing.

    I'm with you in that there is no point in storing the comments,
but I disagree about the unused options. Storing the unused options
as comments is more useful than it seems ;)

    Look at this example. You want to know if you have 'CONFIG_PNP'
enabled, so you do something like 'grep CONFIG_PMP /proc/config' (the
typo PNP->PMP is intended here). Of course that commands doesn't
print anything due to the typo. If you store the disabled options as
comments and a grep fails, you probably mispelled the config option,
or you're referring to a config option not present in your old
kernel, but if you remove them and you mispell the config option
there is no (automatic) way of knowing if you made a typo or if the
option is disabled. Any automatic search can, potentially, give you a
false negative.

    I'm not really sure about it, but I think that the unset options
are left as comments for the sake of automation. The space saving
doesn't (IMHO) worth the pain.

> I've also added the configuration option of how you want to compress it.

    Compression is always welcome, I suppose ;) Thanks for the idea.

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: /proc/config reducing kernel image size

[Tom Fredrik Blenning Klaussen]

On Tue, 2004-09-14 at 19:26, DervishD wrote:
>     Hi Tom :)
> 
>  * Tom Fredrik Blenning Klaussen <bfg-kernel@blenning.no> dixit:
> > There is no point in storing all the comments and unused options in the
> > kernel image. This typically reduces the config size to about 1/5th
> > before compressing, and to about 1/4th after compressing.
> 
>     I'm with you in that there is no point in storing the comments,
> but I disagree about the unused options. Storing the unused options
> as comments is more useful than it seems ;)

This is why I added a config option.

>     Look at this example. You want to know if you have 'CONFIG_PNP'
> enabled, so you do something like 'grep CONFIG_PMP /proc/config' (the
> typo PNP->PMP is intended here). Of course that commands doesn't
> print anything due to the typo. If you store the disabled options as
> comments and a grep fails, you probably mispelled the config option,
> or you're referring to a config option not present in your old
> kernel, but if you remove them and you mispell the config option
> there is no (automatic) way of knowing if you made a typo or if the
> option is disabled. Any automatic search can, potentially, give you a
> false negative.

A small util could easily be written to do the same job, just comparing
it to the default config. Although I agree it's much easier to do a
search like this when it is in it's original form.

>     I'm not really sure about it, but I think that the unset options
> are left as comments for the sake of automation. The space saving
> doesn't (IMHO) worth the pain.

I'm not sure either, but I don't know of any programs that uses this.
Putting this config file inside the same source tree as it was compiled
with, and then just starting and stopping menuconfig will restore it to
it's original form.

> > I've also added the configuration option of how you want to compress it.
> 
>     Compression is always welcome, I suppose ;) Thanks for the idea.

Sincerely
-- 
BFG

Re: /proc/config reducing kernel image size

[DervishD]

    Hi Tom :)

 * Tom Fredrik Blenning Klaussen <bfg-kernel@blenning.no> dixit:
> > > There is no point in storing all the comments and unused options in the
> > > kernel image. This typically reduces the config size to about 1/5th
> > > before compressing, and to about 1/4th after compressing.
> >     I'm with you in that there is no point in storing the comments,
> > but I disagree about the unused options. Storing the unused options
> > as comments is more useful than it seems ;)
> This is why I added a config option.

    But removing the comments is a good idea. Even reformatting the
contents, or something like that.
 
> >     I'm not really sure about it, but I think that the unset options
> > are left as comments for the sake of automation. The space saving
> > doesn't (IMHO) worth the pain.
> I'm not sure either, but I don't know of any programs that uses this.

    Neither do I.

> Putting this config file inside the same source tree as it was compiled
> with, and then just starting and stopping menuconfig will restore it to
> it's original form.

    That's true.

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: /proc/config reducing kernel image size

[Herbert Poetzl]

On Tue, Sep 14, 2004 at 08:01:24PM +0200, DervishD wrote:
>     Hi Tom :)
> 
>  * Tom Fredrik Blenning Klaussen <bfg-kernel@blenning.no> dixit:
> > > > There is no point in storing all the comments and unused options in the
> > > > kernel image. This typically reduces the config size to about 1/5th
> > > > before compressing, and to about 1/4th after compressing.
> > >     I'm with you in that there is no point in storing the comments,
> > > but I disagree about the unused options. Storing the unused options
> > > as comments is more useful than it seems ;)
> > This is why I added a config option.
> 
>     But removing the comments is a good idea. Even reformatting the
> contents, or something like that.
>  
> > >     I'm not really sure about it, but I think that the unset options
> > > are left as comments for the sake of automation. The space saving
> > > doesn't (IMHO) worth the pain.
> > I'm not sure either, but I don't know of any programs that uses this.
> 
>     Neither do I.

well, the kernel config uses it, to decide if 
some option is known, or has to be defaulted/asked

try to remove one of those 'comments' from a config
and run 'make oldconfig', you'll get asked for this
specific option ...

best,
Herbert

> > Putting this config file inside the same source tree as it was compiled
> > with, and then just starting and stopping menuconfig will restore it to
> > it's original form.
> 
>     That's true.
> 
>     Raúl Núñez de Arenas Coronado
> 
> -- 
> Linux Registered User 88736
> http://www.pleyades.net & http://raul.pleyades.net/
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: /proc/config reducing kernel image size

[Norberto Bensa]

Tom Fredrik Blenning Klaussen wrote:
> There is no point in storing all the comments and unused options in the

Try this:

    mv .config .config-saveme
    grep -v ^# .config-saveme  > .config
    make oldconfig

Do you get the exact same kernel (hint: diff .config .config-saveme) ?


Regards,
Norberto

Re: /proc/config reducing kernel image size

[Tom Fredrik Blenning Klaussen]

On Tue, 2004-09-14 at 19:44, Norberto Bensa wrote:
> Tom Fredrik Blenning Klaussen wrote:
> > There is no point in storing all the comments and unused options in the
> 
> Try this:
> 
>     mv .config .config-saveme
>     grep -v ^# .config-saveme  > .config
>     make oldconfig
>
> Do you get the exact same kernel (hint: diff .config .config-saveme) ?

You're right.
Am I correct when I say that when an option is not present, it assumes
it's default?

Sincerely
-- 
BFG

Re: /proc/config reducing kernel image size

[Norberto Bensa]

Tom Fredrik Blenning Klaussen wrote:
> On Tue, 2004-09-14 at 19:44, Norberto Bensa wrote:
> > Try this:
>
> You're right.
> Am I correct when I say that when an option is not present, it assumes
> it's default?

Exactly.

Best regards,
Norberto