clarifying details about "is not set" lines in kernel config fragment files

clarifying details about "is not set" lines in kernel config fragment files

[Robert P. J. Day]

  oddly, in the current YP kernel dev manual, i don't see the phrase
"is not set" explained anywhere in the entire doc, which is weird
since it seems like it would be important.

  first, just to confirm *i* know what it's used for, it's not just a
comment -- it's used explicitly by the kernel config process to
stipulate that we are aware of a kernel config setting, and we don't
want it. that is, if we have a kernel .config file used as the basis
of a "make *config" command, and there is no mention whatever of a
reachable config variable, the config process will stop and ask us
about it. am i right so far?

  this means that, once the kernel config process completes and the
.config file is generated, every single *available* and *settable*
kernel config option should be listed in the .config file, to avoid
being asked about it the next time we do a kernel configuration.

  oh, and about that "reachable" thing, we don't need a line regarding
every single kernel config setting, just the ones that are available
to be set given the current config settings. how am i doing so far?

  given all that, my understanding is that, when one is writing kernel
config fragments, it is necessary to explicitly deselect ("# ... is
not set") *only* those kernel config options that would be selected
based on other config settings we selected. do i have that right?

  for instance, here's kernel Kbuild file net/Kconfig:


    ...
    menuconfig NETFILTER
        bool "Network packet filtering framework (Netfilter)"
        ... snip ...
    if NETFILTER

    config NETFILTER_DEBUG
        bool "Network packet filtering debugging"
        depends on NETFILTER
        help
          You can say Y here if you want to get additional messages useful in
          debugging the netfilter code.


as a test, i added a single-line frgament file:

  CONFIG_NETFILTER=y

however, even though NETFILTER_DEBUG was then selectable, because it
wasn't *explicitly* selected, what i got in my final .config file was:

    ...
    CONFIG_NETFILTER=y
    # CONFIG_NETFILTER_DEBUG is not set
    ...

however, because the very next Kconfig stanza was "default y":

    config NETFILTER_ADVANCED
        bool "Advanced netfilter configuration"
        depends on NETFILTER
        default y

i *did* get the line in the final .config:

    CONFIG_NETFILTER_ADVANCED=y

does all this sound right? i always thought it was pretty
straightforward, but i'm willing to be convinced i never really
understood it.

  and it does seem odd that it's not described anywhere in the YP
kernel dev manual.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: clarifying details about "is not set" lines in kernel config fragment files

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 4717 bytes --]

On Tue, May 3, 2016 at 1:04 PM, Robert P. J. Day <rpjday@crashcourse.ca>
wrote:

>
>   oddly, in the current YP kernel dev manual, i don't see the phrase
> "is not set" explained anywhere in the entire doc, which is weird
> since it seems like it would be important.
>

The manuals don't cover the mechanics of how the mainline kernel is
configured.
Just the same way the don't cover the configuration language for any number
of packages in the system.


>
>   first, just to confirm *i* know what it's used for, it's not just a
> comment -- it's used explicitly by the kernel config process to
> stipulate that we are aware of a kernel config setting, and we don't
> want it. that is, if we have a kernel .config file used as the basis
> of a "make *config" command, and there is no mention whatever of a
> reachable config variable, the config process will stop and ask us
> about it. am i right so far?
>

Also correct. That's more upstream kernel configuration detail, not
something we duplicate.


>
>   this means that, once the kernel config process completes and the
> .config file is generated, every single *available* and *settable*
> kernel config option should be listed in the .config file, to avoid
> being asked about it the next time we do a kernel configuration.
>
>   oh, and about that "reachable" thing, we don't need a line regarding
> every single kernel config setting, just the ones that are available
> to be set given the current config settings. how am i doing so far?
>
>
Yep, and that means the final .config varies quite a bit, based on very
small
deltas in some config option.


>   given all that, my understanding is that, when one is writing kernel
> config fragments, it is necessary to explicitly deselect ("# ... is
> not set") *only* those kernel config options that would be selected
> based on other config settings we selected. do i have that right?
>

Nope.

If a Kconfig actually has "select FOO", you can't turn it off "FOO" no
matter how
hard you try. You need to turn off the option that has the select itself,
i.e. the
parent/containing option.

So you use # CONFIG_FOO is not set, to turn off your typical boolean or
tristate option, that may have a default 'y', or be otherwise enabled when
you
don't want it.


>
>   for instance, here's kernel Kbuild file net/Kconfig:
>
>
>     ...
>     menuconfig NETFILTER
>         bool "Network packet filtering framework (Netfilter)"
>         ... snip ...
>     if NETFILTER
>
>     config NETFILTER_DEBUG
>         bool "Network packet filtering debugging"
>         depends on NETFILTER
>         help
>           You can say Y here if you want to get additional messages useful
> in
>           debugging the netfilter code.
>
>
> as a test, i added a single-line frgament file:
>
>   CONFIG_NETFILTER=y
>
> however, even though NETFILTER_DEBUG was then selectable, because it
> wasn't *explicitly* selected, what i got in my final .config file was:
>
>     ...
>     CONFIG_NETFILTER=y
>     # CONFIG_NETFILTER_DEBUG is not set
>     ...
>
> however, because the very next Kconfig stanza was "default y":
>
>     config NETFILTER_ADVANCED
>         bool "Advanced netfilter configuration"
>         depends on NETFILTER
>         default y
>
> i *did* get the line in the final .config:
>
>     CONFIG_NETFILTER_ADVANCED=y
>
> does all this sound right? i always thought it was pretty
> straightforward, but i'm willing to be convinced i never really
> understood it.
>

Those are some relatively simple examples, but yes, pretty straight
forward. But no
one has ever claimed Kconfig to be simple or easy to fully grok.


>
>   and it does seem odd that it's not described anywhere in the YP
> kernel dev manual.
>

See above. We don't document what is already described in the mainline
kernel's
Kconfig specification.

Bruce


>
> rday
>
> --
>
> ========================================================================
> Robert P. J. Day                                 Ottawa, Ontario, CANADA
>                         http://crashcourse.ca
>
> Twitter:                                       http://twitter.com/rpjday
> LinkedIn:                               http://ca.linkedin.com/in/rpjday
> ========================================================================
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"

[-- Attachment #2: Type: text/html, Size: 7061 bytes --]

Re: clarifying details about "is not set" lines in kernel config fragment files

[Robert P. J. Day]

[-- Attachment #1: Type: text/plain, Size: 1639 bytes --]

On Tue, 3 May 2016, Bruce Ashfield wrote:

> On Tue, May 3, 2016 at 1:04 PM, Robert P. J. Day <rpjday@crashcourse.ca> wrote:
>
>         oddly, in the current YP kernel dev manual, i don't see the
>       phrase "is not set" explained anywhere in the entire doc,
>       which is weird since it seems like it would be important.
>
> The manuals don't cover the mechanics of how the mainline kernel is
> configured. Just the same way the don't cover the configuration
> language for any number of packages in the system.

  assuming i'm understanding your position, that's where i'm going to
*strongly* disagree. people who work with the mainline kernel in a
non-OE environment don't use kernel config fragment files. period.
AFAIK (and correct me if i'm wrong), .cfg kernel config fragment files
are exclusively an OE/YP thing, so there is no reason for normal
kernel developers to understand how they work.

  knowing how regular kernel configuration works, and how kernel
config fragment files are processed to aid in that process, are two
*entirely* different things, and how those files are processed is,
IMHO, most assuredly the responsibility of the OE/YP docs to explain.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: clarifying details about "is not set" lines in kernel config fragment files

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 1311 bytes --]

On Tue, May 03, 2016 at 02:27:38PM -0400, Robert P. J. Day wrote:
> On Tue, 3 May 2016, Bruce Ashfield wrote:
> 
> > On Tue, May 3, 2016 at 1:04 PM, Robert P. J. Day <rpjday@crashcourse.ca> wrote:
> >
> >         oddly, in the current YP kernel dev manual, i don't see the
> >       phrase "is not set" explained anywhere in the entire doc,
> >       which is weird since it seems like it would be important.
> >
> > The manuals don't cover the mechanics of how the mainline kernel is
> > configured. Just the same way the don't cover the configuration
> > language for any number of packages in the system.
> 
>   assuming i'm understanding your position, that's where i'm going to
> *strongly* disagree. people who work with the mainline kernel in a
> non-OE environment don't use kernel config fragment files. period.
> AFAIK (and correct me if i'm wrong), .cfg kernel config fragment files
> are exclusively an OE/YP thing, so there is no reason for normal
> kernel developers to understand how they work.

Or maybe to be more clear, does the manual spell out that these .cfg
files are Kconfig fragments and follow the normal syntax rules Kconfig
uses?  If not, it really should.  If so, maybe it should also contain a
reference to the external authority on this syntax?

-- 
Tom

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: clarifying details about "is not set" lines in kernel config fragment files

[Robert P. J. Day]

[-- Attachment #1: Type: text/plain, Size: 2211 bytes --]

On Tue, 3 May 2016, Tom Rini wrote:

> On Tue, May 03, 2016 at 02:27:38PM -0400, Robert P. J. Day wrote:
> > On Tue, 3 May 2016, Bruce Ashfield wrote:
> >
> > > On Tue, May 3, 2016 at 1:04 PM, Robert P. J. Day <rpjday@crashcourse.ca> wrote:
> > >
> > >         oddly, in the current YP kernel dev manual, i don't see the
> > >       phrase "is not set" explained anywhere in the entire doc,
> > >       which is weird since it seems like it would be important.
> > >
> > > The manuals don't cover the mechanics of how the mainline kernel
> > > is configured. Just the same way the don't cover the
> > > configuration language for any number of packages in the system.
> >
> >   assuming i'm understanding your position, that's where i'm going
> > to *strongly* disagree. people who work with the mainline kernel
> > in a non-OE environment don't use kernel config fragment files.
> > period. AFAIK (and correct me if i'm wrong), .cfg kernel config
> > fragment files are exclusively an OE/YP thing, so there is no
> > reason for normal kernel developers to understand how they work.
>
> Or maybe to be more clear, does the manual spell out that these .cfg
> files are Kconfig fragments and follow the normal syntax rules
> Kconfig uses?  If not, it really should.  If so, maybe it should
> also contain a reference to the external authority on this syntax?

  saying they're regular Kconfig fragments might be technically
correct, but still doesn't explain the *possibilities* as to what can
go into them, and the consequences (or necessity) of adding your own
"is not set" lines, and why you'd want to do that.

  i don't see a long-winded explanation -- no more than a few
paragraphs would suffice to make it clear what developers can do.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: clarifying details about "is not set" lines in kernel config fragment files

[Robert P. J. Day]

[-- Attachment #1: Type: text/plain, Size: 3639 bytes --]

On Tue, 3 May 2016, Tom Rini wrote:

> On Tue, May 03, 2016 at 02:27:38PM -0400, Robert P. J. Day wrote:
> > On Tue, 3 May 2016, Bruce Ashfield wrote:
> >
> > > On Tue, May 3, 2016 at 1:04 PM, Robert P. J. Day <rpjday@crashcourse.ca> wrote:
> > >
> > >         oddly, in the current YP kernel dev manual, i don't see the
> > >       phrase "is not set" explained anywhere in the entire doc,
> > >       which is weird since it seems like it would be important.
> > >
> > > The manuals don't cover the mechanics of how the mainline kernel
> > > is configured. Just the same way the don't cover the
> > > configuration language for any number of packages in the system.
> >
> >   assuming i'm understanding your position, that's where i'm going
> > to *strongly* disagree. people who work with the mainline kernel
> > in a non-OE environment don't use kernel config fragment files.
> > period. AFAIK (and correct me if i'm wrong), .cfg kernel config
> > fragment files are exclusively an OE/YP thing, so there is no
> > reason for normal kernel developers to understand how they work.
>
> Or maybe to be more clear, does the manual spell out that these .cfg
> files are Kconfig fragments and follow the normal syntax rules
> Kconfig uses?  If not, it really should.  If so, maybe it should
> also contain a reference to the external authority on this syntax?

  "1.1 Overview" of the current kernel dev manual states:

"In particular, the kernel tools allow you to generate configuration
fragments that specify only what you must, and nothing more.
Configuration fragments only need to contain the highest level visible
CONFIG options as presented by the Linux kernel menuconfig system."

  so that opening bit really doesn't provide all the detail one needs,
*especially* if one is a newcomer. (and, yes, i'm sure experienced
kernel hackers are aware of kernel config fragment processing but,
seriously, is anyone trying to make the case you need to be an
experienced kernel hacker to use YP? please tell me that's not the
position you're taking.)

  further on, one reads:

"Note:

"The build system applies the configurations from the defconfig file
before applying any subsequent configuration fragments. The final
kernel configuration is a combination of the configurations in the
defconfig file and any configuration fragments you provide. You need
to realize that if you have any configuration fragments, the build
system applies these on top of and after applying the existing
defconfig file configurations."

  so ... lots of mention of "applying" fragments with no explanation
of what "applying" means. i'm not suggesting a comprehensive tutorial
on the application of kernel config fragments in the manual, but the
idea that one can talk about fragments and have *no* mention of the
value of an "# ... is not set" fragment line and when to use it
strikes me as absurd.

  i think a *lot* of potential confusion could be averted with at most
3 or 4 paragraphs briefly describing what goes into a fragment, and
why it's there. or you could stand on principle and leave confused
readers to fend for themselves. tough decision there.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: clarifying details about "is not set" lines in kernel config fragment files

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 2852 bytes --]

On Tue, May 3, 2016 at 2:27 PM, Robert P. J. Day <rpjday@crashcourse.ca>
wrote:

> On Tue, 3 May 2016, Bruce Ashfield wrote:
>
> > On Tue, May 3, 2016 at 1:04 PM, Robert P. J. Day <rpjday@crashcourse.ca>
> wrote:
> >
> >         oddly, in the current YP kernel dev manual, i don't see the
> >       phrase "is not set" explained anywhere in the entire doc,
> >       which is weird since it seems like it would be important.
> >
> > The manuals don't cover the mechanics of how the mainline kernel is
> > configured. Just the same way the don't cover the configuration
> > language for any number of packages in the system.
>
>   assuming i'm understanding your position, that's where i'm going to
> *strongly* disagree. people who work with the mainline kernel in a
> non-OE environment don't use kernel config fragment files. period.
> AFAIK (and correct me if i'm wrong), .cfg kernel config fragment files
> are exclusively an OE/YP thing, so there is no reason for normal
> kernel developers to understand how they work.
>

Nothing about the options is unique configuration fragments.

We contributed the merge_config script to the mainline kernel ages ago, so
the
rudimentary support for fragments is carried along with all recent kernel
trees. At the most basic level, it is a concatenation of a bunch of
configs, the
extra processing around features, the audit, etc, are part of what we do in
Yocto,
but the slamming together of the options and feeding them to the kernel
configuration system .. is not unique.

kernel developers have their workflow, and they stick to it. There's no
attempt
to change that .. the reason I mention that, is that people working with
the
mainline kernel can use fragments if they like .. but also can happily
ignore
them as well :D


>
>   knowing how regular kernel configuration works, and how kernel
> config fragment files are processed to aid in that process, are two
> *entirely* different things, and how those files are processed is,
> IMHO, most assuredly the responsibility of the OE/YP docs to explain.
>

You referenced # is not set ... that has nothing to do with fragments, and
works
the same way if you are manually editing a .config and then re-running make
"old config".

Bruce


>
> rday
>
> --
>
> ========================================================================
> Robert P. J. Day                                 Ottawa, Ontario, CANADA
>                         http://crashcourse.ca
>
> Twitter:                                       http://twitter.com/rpjday
> LinkedIn:                               http://ca.linkedin.com/in/rpjday
> ========================================================================
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"

[-- Attachment #2: Type: text/html, Size: 4228 bytes --]

Re: clarifying details about "is not set" lines in kernel config fragment files

[Robert P. J. Day]

  one more question about fragments which i'm fairly sure i know the
answer to, but will be embarrassed about if i'm wrong.

  if i want to set a kernel option in a fragment, i'm fairly certain i
need to *explicitly* set all options it depends on, correct?
otherwise, that option is left unselected.

  is there any warning in a log file that mentions that happening?

rday

p.s. oh, and i think someone mentioned this earlier ... what happens
if i explicitly set one option, then explicitly "not set" another
option that is "select"ed by that first option? who wins?

  ok, back to work ...

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: clarifying details about "is not set" lines in kernel config fragment files

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 2024 bytes --]

On Wed, May 4, 2016 at 6:41 AM, Robert P. J. Day <rpjday@crashcourse.ca>
wrote:

>
>   one more question about fragments which i'm fairly sure i know the
> answer to, but will be embarrassed about if i'm wrong.
>
>   if i want to set a kernel option in a fragment, i'm fairly certain i
> need to *explicitly* set all options it depends on, correct?
> otherwise, that option is left unselected.
>

Correct.


>
>   is there any warning in a log file that mentions that happening?
>
>
The kconfig audit will indicate that the value didn't make it into the
final .config, and
if the option has been tagged "hardware", then that warning is dumped to
the screen
for the developer to see.

More audit information is available within the kernel build itself, and
IIRC that is
covered in the more detailed guides.

Right now, it can't tell you what was missing .. but there's no a library
that I can
potentially use to get that information, and I've raised an enhancement bug
for 2.2
to not only output why an option didn't make they final .config .. but why
it was
dropped.


> rday
>
> p.s. oh, and i think someone mentioned this earlier ... what happens
> if i explicitly set one option, then explicitly "not set" another
> option that is "select"ed by that first option? who wins?
>
>
Last through the gate. Ordering matters. But the "redefinition" of the value
is logged in the audit output

Bruce


>   ok, back to work ...
>
> --
>
> ========================================================================
> Robert P. J. Day                                 Ottawa, Ontario, CANADA
>                         http://crashcourse.ca
>
> Twitter:                                       http://twitter.com/rpjday
> LinkedIn:                               http://ca.linkedin.com/in/rpjday
> ========================================================================
>
>
>


-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"

[-- Attachment #2: Type: text/html, Size: 3429 bytes --]

Re: clarifying details about "is not set" lines in kernel config fragment files

[Denys Dmytriyenko]

On Tue, May 03, 2016 at 02:27:38PM -0400, Robert P. J. Day wrote:
> On Tue, 3 May 2016, Bruce Ashfield wrote:
> 
> > On Tue, May 3, 2016 at 1:04 PM, Robert P. J. Day <rpjday@crashcourse.ca> wrote:
> >
> >         oddly, in the current YP kernel dev manual, i don't see the
> >       phrase "is not set" explained anywhere in the entire doc,
> >       which is weird since it seems like it would be important.
> >
> > The manuals don't cover the mechanics of how the mainline kernel is
> > configured. Just the same way the don't cover the configuration
> > language for any number of packages in the system.
> 
>   assuming i'm understanding your position, that's where i'm going to
> *strongly* disagree. people who work with the mainline kernel in a
> non-OE environment don't use kernel config fragment files. period.
> AFAIK (and correct me if i'm wrong), .cfg kernel config fragment files
> are exclusively an OE/YP thing, so there is no reason for normal
> kernel developers to understand how they work.

You are *strongly* wrong! I know many kernel developers using this:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/scripts/kconfig/merge_config.sh


>   knowing how regular kernel configuration works, and how kernel
> config fragment files are processed to aid in that process, are two
> *entirely* different things, and how those files are processed is,
> IMHO, most assuredly the responsibility of the OE/YP docs to explain.
> 
> rday
> 
> -- 
> 
> ========================================================================
> Robert P. J. Day                                 Ottawa, Ontario, CANADA
>                         http://crashcourse.ca
> 
> Twitter:                                       http://twitter.com/rpjday
> LinkedIn:                               http://ca.linkedin.com/in/rpjday
> ========================================================================

> -- 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: clarifying details about "is not set" lines in kernel config fragment files

[Robert P. J. Day]

[-- Attachment #1: Type: text/plain, Size: 1812 bytes --]

On Tue, 3 May 2016, Denys Dmytriyenko wrote:

> On Tue, May 03, 2016 at 02:27:38PM -0400, Robert P. J. Day wrote:
> > On Tue, 3 May 2016, Bruce Ashfield wrote:
> >
> > > On Tue, May 3, 2016 at 1:04 PM, Robert P. J. Day <rpjday@crashcourse.ca> wrote:
> > >
> > >         oddly, in the current YP kernel dev manual, i don't see the
> > >       phrase "is not set" explained anywhere in the entire doc,
> > >       which is weird since it seems like it would be important.
> > >
> > > The manuals don't cover the mechanics of how the mainline kernel is
> > > configured. Just the same way the don't cover the configuration
> > > language for any number of packages in the system.
> >
> >   assuming i'm understanding your position, that's where i'm going to
> > *strongly* disagree. people who work with the mainline kernel in a
> > non-OE environment don't use kernel config fragment files. period.
> > AFAIK (and correct me if i'm wrong), .cfg kernel config fragment files
> > are exclusively an OE/YP thing, so there is no reason for normal
> > kernel developers to understand how they work.
>
> You are *strongly* wrong! I know many kernel developers using this:
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/scripts/kconfig/merge_config.sh

  ah, i was unaware of that. ok, a short reference to that would work
just fine.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================