/proc/config idea

/proc/config idea

[Ian Soboroff]

[sorry this doesn't have proper References: headers, i read the list
off the hypermail archive.]

there was some discussion of whether the kernel should emit a
/proc/config or some such for purposes of bug reporting, but that
seems to be a lot of bloat.

instead, why not try to point to a canonical location for a config
copy (we already basically do this with ksymoops and System.map), and
instead have a /proc/config-hash which emits a (precomputed) MD5 hash
of the .config file it was compiled with?

this way, you could check possible configs (Debian for example likes
to stash a copy in /boot, like System.map) and also know if they were
the right ones.

the one problem that comes to mind right now is modules, which needn't
correspond to a full kernel .config.

anyway, my $0.02.
ian

-- 
----
Ian Soboroff                                       ian@cs.umbc.edu
University of MD Baltimore County      http://www.cs.umbc.edu/~ian

Re: /proc/config idea

[Jeremy Jackson]

Ian Soboroff wrote:

> [sorry this doesn't have proper References: headers, i read the list
> off the hypermail archive.]
>
> there was some discussion of whether the kernel should emit a
> /proc/config or some such for purposes of bug reporting, but that
> seems to be a lot of bloat.
>
> instead, why not try to point to a canonical location for a config
> copy (we already basically do this with ksymoops and System.map), and
> instead have a /proc/config-hash which emits a (precomputed) MD5 hash
> of the .config file it was compiled with?
>
> this way, you could check possible configs (Debian for example likes
> to stash a copy in /boot, like System.map) and also know if they were

Yes, I like this.  I do this manually, it allows reproducability, and
incremental
modifications, tracing how that kernel on that problem system was made...

I think the ultimate would be to put all of .config (gzipped?) in a new ELF
section without the Loadable attribute...  I wish System.map was the same.
The you're guaranteed you know how a kernel on disk was configured.

To correlate a running kernel to one on disk (vmlinuz) you have LILO...
it appends an environment variable to the kernel command line with
the name of the file it booted.  This is not infallable, since LILO maps
disk sectors, only using the filesystem at map install time.

Permaps an md5sum of the .text ELF section would conclusively
link the in-core kernel with an on-disk vmlinuz?  Shouldn't be hard
to do with objcopy and /proc/kmem?

>
> the right ones.
>
> the one problem that comes to mind right now is modules, which needn't
> correspond to a full kernel .config.

Well it's a step...

Comments anyone?

Re: /proc/config idea

[Brian Gerst]

Jeremy Jackson wrote:
> 
> Ian Soboroff wrote:
> 
> > [sorry this doesn't have proper References: headers, i read the list
> > off the hypermail archive.]
> >
> > there was some discussion of whether the kernel should emit a
> > /proc/config or some such for purposes of bug reporting, but that
> > seems to be a lot of bloat.
> >
> > instead, why not try to point to a canonical location for a config
> > copy (we already basically do this with ksymoops and System.map), and
> > instead have a /proc/config-hash which emits a (precomputed) MD5 hash
> > of the .config file it was compiled with?
> >
> > this way, you could check possible configs (Debian for example likes
> > to stash a copy in /boot, like System.map) and also know if they were
> 
> Yes, I like this.  I do this manually, it allows reproducability, and
> incremental
> modifications, tracing how that kernel on that problem system was made...
> 
> I think the ultimate would be to put all of .config (gzipped?) in a new ELF
> section without the Loadable attribute...  I wish System.map was the same.
> The you're guaranteed you know how a kernel on disk was configured.
> 
> To correlate a running kernel to one on disk (vmlinuz) you have LILO...
> it appends an environment variable to the kernel command line with
> the name of the file it booted.  This is not infallable, since LILO maps
> disk sectors, only using the filesystem at map install time.
> 
> Permaps an md5sum of the .text ELF section would conclusively
> link the in-core kernel with an on-disk vmlinuz?  Shouldn't be hard
> to do with objcopy and /proc/kmem?

Except that the bootable kernel (bzImage, at least on x86) is not in ELF
format.  It is a compressed binary image.  The vmlinux file is ELF, but
you run into the same issue of correlating the file with the running
kernel.

--

				Brian Gerst

Re: /proc/config idea

[Bart Trojanowski]

I see benefit in having the .config file in the kernel.  It being a non
loadable elf section would be a plus.  However I also see merit to having
it available as a proc entry.

Say that we decide to go with /proc/config.  In that case I think that
Jeremy is right on with the compressing of the info.

linux-2.4.3# cat .config | grep ^CONFIG_ | wc -c
   2885

linux-2.4.3# cat .config | grep ^CONFIG_ | gzip | wc -c
    874

While 3k is not a lot, >1k would be even better.  The /proc/config could
just unzip the compressed config stored in the kernel on the fly.

This would reduce the 'bloat'.  Of course this functionality would be
configurable and maybe off by default.  Although if it's not available on a
default kernel of distribution X, Y and Z there is little merit in it as
install scripts for 3rd party drivers could not reliably use it.

Cheers,
Bart.

-- 
	WebSig: http://www.jukie.net/~bart/sig/

Re: /proc/config idea

[Jeff Garzik]

Jeremy Jackson wrote:
> Yes, I like this.  I do this manually, it allows reproducability, and
> incremental
> modifications, tracing how that kernel on that problem system was made...
> 
> I think the ultimate would be to put all of .config (gzipped?) in a new ELF
> section without the Loadable attribute...  I wish System.map was the same.
> The you're guaranteed you know how a kernel on disk was configured.
> 
> To correlate a running kernel to one on disk (vmlinuz) you have LILO...
> it appends an environment variable to the kernel command line with
> the name of the file it booted.  This is not infallable, since LILO maps
> disk sectors, only using the filesystem at map install time.
> 
> Permaps an md5sum of the .text ELF section would conclusively
> link the in-core kernel with an on-disk vmlinuz?  Shouldn't be hard
> to do with objcopy and /proc/kmem?
[...]
> Comments anyone?

Instead of doing all this stuff in the kernel, you could simply update
symlinks to properly installed files at boot time.

Putting _files_ in the kernel is plain silly.  This is unreclaimable
memory, folks.  There is no need to special case an operation as simple
as reading a file.  [I think this about firmware images too, but that's
another thread]

-- 
Jeff Garzik       | May you have warm words on a cold evening,
Building 1024     | a full moon on a dark night,
MandrakeSoft      | and a smooth road all the way to your door.

Re: /proc/config idea

[Jeremy Jackson]

Jeff Garzik wrote:

> Jeremy Jackson wrote:
> > Yes, I like this.  I do this manually, it allows reproducability, and
> > incremental
> > modifications, tracing how that kernel on that problem system was made...
> >
> > I think the ultimate would be to put all of .config (gzipped?) in a new ELF
> > section without the Loadable attribute...  I wish System.map was the same.
> > The you're guaranteed you know how a kernel on disk was configured.
> >
> > To correlate a running kernel to one on disk (vmlinuz) you have LILO...
> > it appends an environment variable to the kernel command line with
> > the name of the file it booted.  This is not infallable, since LILO maps
> > disk sectors, only using the filesystem at map install time.
> >
> > Permaps an md5sum of the .text ELF section would conclusively
> > link the in-core kernel with an on-disk vmlinuz?  Shouldn't be hard
> > to do with objcopy and /proc/kmem?
> [...]
> > Comments anyone?
>
> Instead of doing all this stuff in the kernel, you could simply update
> symlinks to properly installed files at boot time.
>
> Putting _files_ in the kernel is plain silly.  This is unreclaimable

not in the kernel in an ELF section, marked not loadable.  it never leaves the
disk.
as someone else posted, it's ~900 bytes gzipped (if you want to have it in core)

unfortunately (in the LILO case) x86 doesn't boot an ELF image... (oops)

>
> memory, folks.  There is no need to special case an operation as simple

If you have a lot of kernels around, which Config-2.4.3 applies to kernel 2.4.3
given 5 to choose from...the idea (same for System.map) is that it being in the
same
file they can't be confused.  Kinda like forks under Mac (but let's not go there
now)

>
> as reading a file.  [I think this about firmware images too, but that's
> another thread]

Re: /proc/config idea

[Jeff Garzik]

Jeremy Jackson wrote:
> If you have a lot of kernels around, which Config-2.4.3 applies to kernel 2.4.3
> given 5 to choose from...the idea (same for System.map) is that it being in the
> same
> file they can't be confused.  Kinda like forks under Mac (but let's not go there
> now)

The same applies to kernel modules too.  Are you going to put all those
in the kernel image too?

If it's a file, read it from a filesystem after the kernel has booted. 
There is no need to special case this stuff.

-- 
Jeff Garzik       | May you have warm words on a cold evening,
Building 1024     | a full moon on a dark night,
MandrakeSoft      | and a smooth road all the way to your door.

Re: /proc/config idea

[David Lang]

a module for 2.4.3 will work for any 2.4.3 kernel that supports modules
at all (except for the SMP vs UP issue) so it's not the same thing as
trying to figure out which if the 2.4.3 kernels matches what you are
running.

David Lang

On Mon, 2 Apr 2001, Jeff Garzik wrote:

> Date: Mon, 02 Apr 2001 20:49:09 -0400
> From: Jeff Garzik <jgarzik@mandrakesoft.com>
> To: Jeremy Jackson <jerj@coplanar.net>
> Cc: Ian Soboroff <ian@cs.umbc.edu>, linux-kernel@vger.kernel.org
> Subject: Re: /proc/config idea
>
> Jeremy Jackson wrote:
> > If you have a lot of kernels around, which Config-2.4.3 applies to kernel 2.4.3
> > given 5 to choose from...the idea (same for System.map) is that it being in the
> > same
> > file they can't be confused.  Kinda like forks under Mac (but let's not go there
> > now)
>
> The same applies to kernel modules too.  Are you going to put all those
> in the kernel image too?
>
> If it's a file, read it from a filesystem after the kernel has booted.
> There is no need to special case this stuff.
>
> --
> Jeff Garzik       | May you have warm words on a cold evening,
> Building 1024     | a full moon on a dark night,
> MandrakeSoft      | and a smooth road all the way to your door.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: /proc/config idea

[Olaf Titz]

> a module for 2.4.3 will work for any 2.4.3 kernel that supports modules
> at all (except for the SMP vs UP issue) so it's not the same thing as

No, no, no. This is absolutely and dangerously wrong.

A module depends on the kernel configuration, because it may access
internal data structures of the kernel which change with
configuration. It also depends on the compiler version and exact
options used, because certain structures in the kernel are compiler
dependent. Loading a module which doesn't fit the running kernel
_exactly_ is the easiest way to get a hard kernel crash.

SMP is only a special case of this general problem. CONFIG_MODVERSIONS
is designed to take care of this dependency, and it really ought to be
non-optional.

Olaf

Re: /proc/config idea

[Alan Cox]

> a module for 2.4.3 will work for any 2.4.3 kernel that supports modules
> at all (except for the SMP vs UP issue) so it's not the same thing as
> trying to figure out which if the 2.4.3 kernels matches what you are
> running.

Nope. The 2.4 kernel ABI depends upon a mixture of config options including the
cpu type, as well as the compiler version being used. The API is intended to
be constant throughout 2.4 (but isnt yet totally solid due to bug fixing
activity). We don't care about the ABI because we are source code based

Re: /proc/config idea

[Jeremy Jackson]

Alan Cox wrote:

> > a module for 2.4.3 will work for any 2.4.3 kernel that supports modules
> > at all (except for the SMP vs UP issue) so it's not the same thing as
> > trying to figure out which if the 2.4.3 kernels matches what you are
> > running.
>
> Nope. The 2.4 kernel ABI depends upon a mixture of config options including the
> cpu type, as well as the compiler version being used. The API is intended to
> be constant throughout 2.4 (but isnt yet totally solid due to bug fixing
> activity). We don't care about the ABI because we are source code based
>

Is it possible to identify *all* the dependencies and include symbols (or by some
other
method) have these dependencies checked by insmod?  It would be simply
smashing to have it all inherently bullet proof. (i know never say never, but
lower maintenance then or simpler for users or something)

Re: /proc/config idea

[Alan Cox]

> method) have these dependencies checked by insmod?  It would be simply
> smashing to have it all inherently bullet proof. (i know never say never, but
> lower maintenance then or simpler for users or something)

The goal of modversions is to do this. It isnt perfect. Keith Owens was talking
about several ideas to improve it at the 2.5 kickoff conference

Re: /proc/config idea

[David Lang]

yes it's unreclaimable memory and that's why we want to minimize how much
is used.

on the other hand there is a factor of reliability in the kernel knowing
what options were used to compile it that you just cannot match with a
seperate file, or even with it a part of the on-disk image that is thrown
out when it gets loaded into memory.

if the distro/sysadmin _always_ installs the kernel the 'right way' then
the difference isn't nessasarily that large, but if you want reliability
on any system it may be worth loosing a page or so of memory (hasn't
someone said that the data can be compressed to <1K?) make it so that you
need a common external tool to use the data and deliver it from the kernel
in compressed form and you don't even need to put the decompression
routine in the kernel (cat /proc/sys/kernel/config |gunzip >config)

David Lang

 On Mon, 2 Apr 2001, Jeff Garzik
wrote:

> Date: Mon, 02 Apr 2001 20:23:28 -0400
> From: Jeff Garzik <jgarzik@mandrakesoft.com>
> To: Jeremy Jackson <jerj@coplanar.net>
> Cc: Ian Soboroff <ian@cs.umbc.edu>, linux-kernel@vger.kernel.org
> Subject: Re: /proc/config idea
>
> Jeremy Jackson wrote:
> > Yes, I like this.  I do this manually, it allows reproducability, and
> > incremental
> > modifications, tracing how that kernel on that problem system was made...
> >
> > I think the ultimate would be to put all of .config (gzipped?) in a new ELF
> > section without the Loadable attribute...  I wish System.map was the same.
> > The you're guaranteed you know how a kernel on disk was configured.
> >
> > To correlate a running kernel to one on disk (vmlinuz) you have LILO...
> > it appends an environment variable to the kernel command line with
> > the name of the file it booted.  This is not infallable, since LILO maps
> > disk sectors, only using the filesystem at map install time.
> >
> > Permaps an md5sum of the .text ELF section would conclusively
> > link the in-core kernel with an on-disk vmlinuz?  Shouldn't be hard
> > to do with objcopy and /proc/kmem?
> [...]
> > Comments anyone?
>
> Instead of doing all this stuff in the kernel, you could simply update
> symlinks to properly installed files at boot time.
>
> Putting _files_ in the kernel is plain silly.  This is unreclaimable
> memory, folks.  There is no need to special case an operation as simple
> as reading a file.  [I think this about firmware images too, but that's
> another thread]
>
> --
> Jeff Garzik       | May you have warm words on a cold evening,
> Building 1024     | a full moon on a dark night,
> MandrakeSoft      | and a smooth road all the way to your door.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: /proc/config idea

[Michal Jaegermann]

On Mon, Apr 02, 2001 at 05:39:19PM -0700, David Lang wrote:
> 
> if the distro/sysadmin _always_ installs the kernel the 'right way' then
> the difference isn't nessasarily that large, but if you want reliability
> on any system it may be worth loosing a page or so of memory (hasn't
> someone said that the data can be compressed to <1K?)

After throwing away in a Makefile rule all "is not set" lines, as they
are trivially recoverable with 'make oldconfig', what is left for an
avarege kernel compresses to something like 500 bytes.  Quite a bit
of space left on this one page if you need more extensive .config.
'zcat /proc/config.gz' works just fine.

As most kernels around are NOT installed "the right way" I found that in
practice separating configuration information from a kernel image is not
even close to be semi-reliable on a longer run.  Those who say
"installation script", and similar things, assume that people compile
kernels for themselves.  This is undoubtely true for folks on this list;
this does not start to approximate the situation in general and, it
seems, that we really want it that way. :-)

BTW - /sbin/installkernel, as seen in practice, is not even correct for
a general case with x86; not to mention other architectures.  Writing
something like /var/log/config from "init data" during a bootup could be
another solution which does not take any kernel memory and still keeps
all this information attached to a kernel image itself.  OTOH we have
all these tons of strings which show in /proc/pci output and somehow
these do not cause such huge opposition.  Yes, I know that 'lspci' was
supposed to replace that; but it did not.

  Michal

Re: /proc/config idea

[J . A . Magallon]

On 04.03 David Lang wrote:
> 
> if the distro/sysadmin _always_ installs the kernel the 'right way' then
> the difference isn't nessasarily that large, but if you want reliability
> on any system it may be worth loosing a page or so of memory (hasn't
> someone said that the data can be compressed to <1K?) make it so that you
> need a common external tool to use the data and deliver it from the kernel
> in compressed form and you don't even need to put the decompression
> routine in the kernel (cat /proc/sys/kernel/config |gunzip >config)
> 

Just my 2 cents...

If this has not been done for System.map, that is a much more important
info for debug and oops, and the de facto standard is to put it aside
kernel with some standadr naming, lets use the same method for config.

-- 
J.A. Magallon                                          #  Let the source
mailto:jamagallon@able.es                              #  be with you, Luke... 

Linux werewolf 2.4.3 #2 SMP Fri Mar 30 15:42:05 CEST 2001 i686

Re: /proc/config idea

[Ben Ford]

J . A . Magallon wrote:

> On 04.03 David Lang wrote:
> 
>> if the distro/sysadmin _always_ installs the kernel the 'right way' then
>> the difference isn't nessasarily that large, but if you want reliability
>> on any system it may be worth loosing a page or so of memory (hasn't
>> someone said that the data can be compressed to <1K?) make it so that you
>> need a common external tool to use the data and deliver it from the kernel
>> in compressed form and you don't even need to put the decompression
>> routine in the kernel (cat /proc/sys/kernel/config |gunzip >config)
>> 
> 
> Just my 2 cents...
> 
> If this has not been done for System.map, that is a much more important
> info for debug and oops, and the de facto standard is to put it aside
> kernel with some standadr naming, lets use the same method for config.
> 
That would be great and all, but can you tell me how to do it when I 
have 3 or 4 different compiles of the same kernel version?

-b

Re: /proc/config idea

[Alan Cox]

> That would be great and all, but can you tell me how to do it when I 
> have 3 or 4 different compiles of the same kernel version?

Each compile you do already gets assigned a version #, if thats not enough then
add your own id string

Re: /proc/config idea

[J . A . Magallon]

On 04.03 Ben Ford wrote:
> J . A . Magallon wrote:
> > 
> > If this has not been done for System.map, that is a much more important
> > info for debug and oops, and the de facto standard is to put it aside
> > kernel with some standadr naming, lets use the same method for config.
> > 
> That would be great and all, but can you tell me how to do it when I 
> have 3 or 4 different compiles of the same kernel version?
> 

Just like the Alan Cox for 2.4 or Andrea Arcangeli for 2.2. Lets say you
have 2.4.2-ac27. For each of your compiles, set EXTRAVERSION to -ac27-bf1,
-ac27-bf2, etc. Your files will be:
vmlinuz-2.4.2-ac27-bfX
System.map-2.4.2-ac27-bfX
config-2.4.2-ac27-bfX

-- 
J.A. Magallon                                          #  Let the source
mailto:jamagallon@able.es                              #  be with you, Luke... 

Linux werewolf 2.4.3 #2 SMP Fri Mar 30 15:42:05 CEST 2001 i686

Re: /proc/config idea

[Mike Castle]

On Tue, Apr 03, 2001 at 09:12:18PM +0200, J . A . Magallon wrote:
> Just like the Alan Cox for 2.4 or Andrea Arcangeli for 2.2. Lets say you
> have 2.4.2-ac27. For each of your compiles, set EXTRAVERSION to -ac27-bf1,
> -ac27-bf2, etc. Your files will be:

Some patches, such as the RAID patches, sets up EXTRAVERSION to a specific
value.

I do with the make file also had a USERVERSION that would be hands off for
anyone but the builder.

mrc
-- 
       Mike Castle       Life is like a clock:  You can work constantly
  dalgoda@ix.netcom.com  and be right all the time, or not work at all
www.netcom.com/~dalgoda/ and be right at least twice a day.  -- mrc
    We are all of us living in the shadow of Manhattan.  -- Watchmen

Re: /proc/config idea

[GOMBAS Gabor]

On Tue, Apr 03, 2001 at 12:30:14PM -0700, Mike Castle wrote:

> Some patches, such as the RAID patches, sets up EXTRAVERSION to a specific
> value.

- If you apply such a patch first, and after that you edit EXTRAVERSION,
your value will be used - no problem.

- If you edit EXTRAVERSION before applying such a patch, the specific hunk
trying to change the EXTRAVERSION will be rejected - again no problem.
Actually this is already the case when you apply 2 patches trying to set
EXTRAVERSION to 2 different values...

> I do with the make file also had a USERVERSION that would be hands off for
> anyone but the builder.

It is not needed.

Gabor

-- 
Gabor Gombas                                       Eotvos Lorand University
E-mail: gombasg@inf.elte.hu                        Hungary

Re: /proc/config idea

[Ben Ford]

J . A . Magallon wrote:

> On 04.03 Ben Ford wrote:
> 
>> J . A . Magallon wrote:
>> 
>>> If this has not been done for System.map, that is a much more important
>>> info for debug and oops, and the de facto standard is to put it aside
>>> kernel with some standadr naming, lets use the same method for config.
>>> 
>> That would be great and all, but can you tell me how to do it when I 
>> have 3 or 4 different compiles of the same kernel version?
>> 
> 
> Just like the Alan Cox for 2.4 or Andrea Arcangeli for 2.2. Lets say you
> have 2.4.2-ac27. For each of your compiles, set EXTRAVERSION to -ac27-bf1,
> -ac27-bf2, etc. Your files will be:
> vmlinuz-2.4.2-ac27-bfX
> System.map-2.4.2-ac27-bfX
> config-2.4.2-ac27-bfX
> 
Many thanks, I didn't know that.