CONFIG_* used by user-space to figure out whether a feature is on/off

CONFIG_* used by user-space to figure out whether a feature is on/off

[Konrad Rzeszutek Wilk]

Hey Linus,

I am hoping you can help me draw an understanding and a line in sand whether:
 a) Tools should not depend on /proc/config.gz to figure out whether
    a kernel has some CONFIG_X=y feature.

 b) If they are OK to do so, what do we do when certain CONFIG_X options
    get reworked/removed. Would they be considered regressions? Aka
    is this similar to 'you shall not break user-space'?


Irrespective of that, do you have any ideas of how a user-space program (say GRUB)
can figure out whether the configuration stanze it generates is supported by
the kernel. If you don't want to answer this question - since this might
open a can of worms you prefer not to deal with - that is absolutly OK.

Folks have been tossing ideas such as:
 - Let the user deal with it and if it does not boot - oh well.
 - readelf or objdump.
 - use /boot/config-<kernel>-<version> as most (all?) distros stick that in
   there.

Thanks!

Re: CONFIG_* used by user-space to figure out whether a feature is on/off

[Greg KH]

On Mon, Jul 15, 2013 at 11:40:50AM -0400, Konrad Rzeszutek Wilk wrote:
> Hey Linus,
> 
> I am hoping you can help me draw an understanding and a line in sand whether:
>  a) Tools should not depend on /proc/config.gz to figure out whether
>     a kernel has some CONFIG_X=y feature.
> 
>  b) If they are OK to do so, what do we do when certain CONFIG_X options
>     get reworked/removed. Would they be considered regressions? Aka
>     is this similar to 'you shall not break user-space'?

CONFIG_ values never get exported to userspace, you need to dig to find
them, and they don't "mean" anything to userspace.

You should test for the functionality of the kernel, not a CONFIG
option, in my opinion.

I've been working with some userspace tools that were blindly looking at
kernel version numbers (i.e. docker), and that too is not a good idea as
distros backport features and fixes to older kernel versions, so you
can't "rely" on that either.

> Irrespective of that, do you have any ideas of how a user-space program (say GRUB)
> can figure out whether the configuration stanze it generates is supported by
> the kernel. If you don't want to answer this question - since this might
> open a can of worms you prefer not to deal with - that is absolutly OK.

Why does grub need to care about the kernel configuration?  Other
bootloaders sure don't.  A bootloader just needs to load a blob and pass
control over to it, no need to care what is in that blob at all, right?

What am I missing here?

thanks,

gre gk-h

Re: CONFIG_* used by user-space to figure out whether a feature is on/off

[Konrad Rzeszutek Wilk]

On Mon, Jul 15, 2013 at 10:02:44AM -0700, Greg KH wrote:
> On Mon, Jul 15, 2013 at 11:40:50AM -0400, Konrad Rzeszutek Wilk wrote:
> > Hey Linus,
> > 
> > I am hoping you can help me draw an understanding and a line in sand whether:
> >  a) Tools should not depend on /proc/config.gz to figure out whether
> >     a kernel has some CONFIG_X=y feature.
> > 
> >  b) If they are OK to do so, what do we do when certain CONFIG_X options
> >     get reworked/removed. Would they be considered regressions? Aka
> >     is this similar to 'you shall not break user-space'?
> 
> CONFIG_ values never get exported to userspace, you need to dig to find
> them, and they don't "mean" anything to userspace.
> 
> You should test for the functionality of the kernel, not a CONFIG
> option, in my opinion.
> 
> I've been working with some userspace tools that were blindly looking at
> kernel version numbers (i.e. docker), and that too is not a good idea as
> distros backport features and fixes to older kernel versions, so you
> can't "rely" on that either.

OK.
> 
> > Irrespective of that, do you have any ideas of how a user-space program (say GRUB)
> > can figure out whether the configuration stanze it generates is supported by
> > the kernel. If you don't want to answer this question - since this might
> > open a can of worms you prefer not to deal with - that is absolutly OK.
> 
> Why does grub need to care about the kernel configuration?  Other
> bootloaders sure don't.  A bootloader just needs to load a blob and pass
> control over to it, no need to care what is in that blob at all, right?

If life was that simple.. This particular issue was with inhibiting certain
config stanzas if certain features were not built in the kernel. Here is the
bug that started it:
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=633127

And this thread has more details:
 https://groups.google.com/forum/#!topic/linux.kernel/9VQqxBr-NOE

Make sure you get your favorite beverage and food substance when reading it.
> 
> What am I missing here?

I am trying to make sure that the line is drawn so that when a patch is
proposed for grub2 and a discussion starts it is the right fix and not
a half-way fix that only fixes part of the problem.

> 
> thanks,
> 
> gre gk-h

Re: CONFIG_* used by user-space to figure out whether a feature is on/off

[Linus Torvalds]

On Mon, Jul 15, 2013 at 10:24 AM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>
> If life was that simple.. This particular issue was with inhibiting certain
> config stanzas if certain features were not built in the kernel. Here is the
> bug that started it:
>   http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=633127

Can somebody please open a grub/xen ticket that says all of this is
just too confusing and broken, and that it should just be removed
entirely instead of expanded upon?

This is a grub bug. It really is that simple. Treat it as one.

I don't want to break user space applications, but this isn't a
"user-space app". This is "broken kernel support", and when that
happens, we call them out for being buggy and don't support them
(example: all the gcc bugs that have caused us problems over the
years).

                 Linus

Re: CONFIG_* used by user-space to figure out whether a feature is on/off

[Konrad Rzeszutek Wilk]

On Mon, Jul 15, 2013 at 10:46:17AM -0700, Linus Torvalds wrote:
> On Mon, Jul 15, 2013 at 10:24 AM, Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com> wrote:
> >
> > If life was that simple.. This particular issue was with inhibiting certain
> > config stanzas if certain features were not built in the kernel. Here is the
> > bug that started it:
> >   http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=633127
> 
> Can somebody please open a grub/xen ticket that says all of this is
> just too confusing and broken, and that it should just be removed
> entirely instead of expanded upon?
> 
> This is a grub bug. It really is that simple. Treat it as one.
> 
> I don't want to break user space applications, but this isn't a
> "user-space app". This is "broken kernel support", and when that
> happens, we call them out for being buggy and don't support them
> (example: all the gcc bugs that have caused us problems over the
> years).

How about I make you proud and post a patch. See:

 http://article.gmane.org/gmane.comp.boot-loaders.grub.devel/20332 
> 
>                  Linus

Re: CONFIG_* used by user-space to figure out whether a feature is on/off

[Paul Bolle]

On Mon, 2013-07-15 at 14:02 -0400, Konrad Rzeszutek Wilk wrote:
> How about I make you proud and post a patch. See:
> 
>  http://article.gmane.org/gmane.comp.boot-loaders.grub.devel/20332 

And you're now also taking Michael's patch, that triggered this
discussion, aren't you?


Paul Bolle

Re: CONFIG_* used by user-space to figure out whether a feature is on/off

[Konrad Rzeszutek Wilk]

On Mon, Jul 15, 2013 at 08:25:43PM +0200, Paul Bolle wrote:
> On Mon, 2013-07-15 at 14:02 -0400, Konrad Rzeszutek Wilk wrote:
> > How about I make you proud and post a patch. See:
> > 
> >  http://article.gmane.org/gmane.comp.boot-loaders.grub.devel/20332 
> 
> And you're now also taking Michael's patch, that triggered this
> discussion, aren't you?

I believe there is a better way of doing this that I had enumerated
in the thread (see "xen: remove unused Kconfig parameter"). That was to
remove the CONFIG_XEN_PRIVILIGED and XEN_CONFIG_DOM0 and instead have an
CONFIG_XEN_HARDWARE_DOMAIN and CONFIG_XEN_CONTROL_DOMAIN which reflect the
reality a lot better.

P.S.
That idea had been percolating a bit - and it started way back with Andrew Jones
proposing an Kconfig patch a year ago - see:
http://lists.xen.org/archives/html/xen-devel/2012-01/msg00500.html for some
backstory.

Re: CONFIG_* used by user-space to figure out whether a feature is on/off

[Paul Bolle]

On Mon, 2013-07-15 at 15:03 -0400, Konrad Rzeszutek Wilk wrote:
> On Mon, Jul 15, 2013 at 08:25:43PM +0200, Paul Bolle wrote:
> > And you're now also taking Michael's patch, that triggered this
> > discussion, aren't you?
> 
> I believe there is a better way of doing this that I had enumerated
> in the thread (see "xen: remove unused Kconfig parameter"). That was to
> remove the CONFIG_XEN_PRIVILIGED and XEN_CONFIG_DOM0 and instead have an
> CONFIG_XEN_HARDWARE_DOMAIN and CONFIG_XEN_CONTROL_DOMAIN which reflect the
> reality a lot better.

But, whatever you choose to do in the future, that needn't stop you from
dropping XEN_PRIVILIGED_GUEST now. It's basically a trivial cleanup.


Paul Bolle

Re: CONFIG_* used by user-space to figure out whether a feature is on/off

[Linus Torvalds]

On Mon, Jul 15, 2013 at 8:40 AM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>
> I am hoping you can help me draw an understanding and a line in sand whether:
>  a) Tools should not depend on /proc/config.gz to figure out whether
>     a kernel has some CONFIG_X=y feature.

Well, /proc/config.gz is better than some crazy saved-off config file,
since it at least is guaranteed to match the kernel you're running,
but it's still a completely crazy idea. Not the least because it's not
at all guaranteed to be there, and even if it's there, we'll rename
config options without caring one whit. It's meant for "make
oldconfig" style stuff, nothing more. Any user program that depends on
it is broken by design.

>  b) If they are OK to do so, what do we do when certain CONFIG_X options
>     get reworked/removed. Would they be considered regressions? Aka
>     is this similar to 'you shall not break user-space'?

Absolutely not. If you depend on any config file, you're broken by
definition. The only thing that can depend on the config file is the
kernel tree itself, and even then we happily break that at any time
(ie "make oldconfig" is meant to give an _approximation_ of the old
config, but if some config option gets renamed, the old value is
thrown away without question, and the new name is asked about).

> Irrespective of that, do you have any ideas of how a user-space program (say GRUB)
> can figure out whether the configuration stanze it generates is supported by
> the kernel. If you don't want to answer this question - since this might
> open a can of worms you prefer not to deal with - that is absolutly OK.

I think grub should stop trying to be clever. Quite frankly, from my
own experience, grub has become too clever by half, and become harder
to use and configure as a result. Just don't do it.

If you want to have grub Xen options for the kernel, make them grub
options. In the grub config file. And if that option isn't there, just
boot it as a native kernel. That had better work anyway, and is a hell
of a lot more flexible and stable anyway. Don't try to be clever, and
certainly don't try to parse some random config file that may or may
not even match the kernel you're booting.

                 Linus

Re: CONFIG_* used by user-space to figure out whether a feature is on/off

[Raymond Jennings]

On Mon, 2013-07-15 at 10:17 -0700, Linus Torvalds wrote:
> On Mon, Jul 15, 2013 at 8:40 AM, Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com> wrote:
> >
> > I am hoping you can help me draw an understanding and a line in sand whether:
> >  a) Tools should not depend on /proc/config.gz to figure out whether
> >     a kernel has some CONFIG_X=y feature.
> 
> Well, /proc/config.gz is better than some crazy saved-off config file,
> since it at least is guaranteed to match the kernel you're running,
> but it's still a completely crazy idea. Not the least because it's not
> at all guaranteed to be there, and even if it's there, we'll rename
> config options without caring one whit. It's meant for "make
> oldconfig" style stuff, nothing more. Any user program that depends on
> it is broken by design.
> 
> >  b) If they are OK to do so, what do we do when certain CONFIG_X options
> >     get reworked/removed. Would they be considered regressions? Aka
> >     is this similar to 'you shall not break user-space'?
> 
> Absolutely not. If you depend on any config file, you're broken by
> definition. The only thing that can depend on the config file is the
> kernel tree itself, and even then we happily break that at any time
> (ie "make oldconfig" is meant to give an _approximation_ of the old
> config, but if some config option gets renamed, the old value is
> thrown away without question, and the new name is asked about).
> 
> > Irrespective of that, do you have any ideas of how a user-space program (say GRUB)
> > can figure out whether the configuration stanze it generates is supported by
> > the kernel.

I'd like to point out that Google Chrome also makes use of CONFIG_ tests
to detect support for namespaces and pid containers and stuff.

> If you don't want to answer this question - since this might
> > open a can of worms you prefer not to deal with - that is absolutly OK.
> 
> I think grub should stop trying to be clever. Quite frankly, from my
> own experience, grub has become too clever by half, and become harder
> to use and configure as a result. Just don't do it.
> 
> If you want to have grub Xen options for the kernel, make them grub
> options. In the grub config file. And if that option isn't there, just
> boot it as a native kernel. That had better work anyway, and is a hell
> of a lot more flexible and stable anyway. Don't try to be clever, and
> certainly don't try to parse some random config file that may or may
> not even match the kernel you're booting.
> 
>                  Linus
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: CONFIG_* used by user-space to figure out whether a feature is on/off

[Linus Torvalds]

On Mon, Jul 15, 2013 at 5:46 PM, Raymond Jennings <shentino@gmail.com> wrote:
>
> I'd like to point out that Google Chrome also makes use of CONFIG_ tests
> to detect support for namespaces and pid containers and stuff.

Hmm. It must work fine despite that. Because I run self-compiled
kernels and there are no config files to be found by user apps.
Neither in /proc nor in /boot. And I'm using chrome to write this.

              Linus

Re: CONFIG_* used by user-space to figure out whether a feature is on/off

[Raymond Jennings]

On Mon, 2013-07-15 at 17:53 -0700, Linus Torvalds wrote:
> On Mon, Jul 15, 2013 at 5:46 PM, Raymond Jennings <shentino@gmail.com> wrote:
> >
> > I'd like to point out that Google Chrome also makes use of CONFIG_ tests
> > to detect support for namespaces and pid containers and stuff.
> 
> Hmm. It must work fine despite that. Because I run self-compiled
> kernels and there are no config files to be found by user apps.
> Neither in /proc nor in /boot. And I'm using chrome to write this.
> 
>               Linus

It could be the quirks of my package manager though.

I run Gentoo, so it's entirely possible that the ebuild is doing all the
bitching, but chrome itself just fails gracefully and falls back to not
using those features if it can't find them.

I imagine though that the same stuff applying to applications in general
should also apply to installers.

Anyway I just wanted to highlight that it's not just the xen stuff
that's peeking at kernel config.