[Buildroot] Incorrect kernel configuration produced from _defconfig file

[Buildroot] Incorrect kernel configuration produced from _defconfig file

[Patrick Doyle]

I am using the linux-at91 kernel (from
git://github.com/linux4sam/linux-at91.git) with a custom
linux_defconfig file that contains, among other things:

CONFIG_MODULES=y
CONFIG_USB_GADGET=y
CONFIG_USB_GADGETFS=y

When I build my kernel using buildroot, I get a .config file that has
CONFIG_USB_GADGETFS=m.

I can reproduce the steps that buildroot executes and see this problem
manually with:

echo CONFIG_MODULES=y > .config
echo CONFIG_USB_GADGET=y >> .config
echo CONFIG_USB_GADGETFS=y >> .config
yes "" | make -k ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- oldconfig
grep CONFIG_USB_GADGET .config

Is there a project specific post-kernel-configuration hook I can apply
to fix my .config file?

Does anybody know why "make oldconfig" does this?  Am I missing
another configuration item?

I suppose I could just disable modules in my kernel, which, arguably,
is the right answer for this application, but I spent 1/2 day
yesterday being confused by this, and it's time for me to reach out
for some help.

On a slightly related, but totally off-subject topic, is there a more
modern way to implement a userspace gadget device driver than using
gadgetfs?  That could be an alternative solution as well.  (We have an
existing product that communicates with a host that uses libusb.  The
embedded Linux side uses a gadgetfs user space driver.)

Thanks for all of the tips you keep feeding me.  I really appreciate
them.  Expect to see (a very minor) new package showing up from me on
this list shortly (for the Atmel WILC1000 firmware).

--wpd

[Buildroot] Incorrect kernel configuration produced from _defconfig file

[Thomas Petazzoni]

Hello,

On Thu, 25 Feb 2016 08:22:31 -0500, Patrick Doyle wrote:
> I am using the linux-at91 kernel (from
> git://github.com/linux4sam/linux-at91.git) with a custom
> linux_defconfig file that contains, among other things:
> 
> CONFIG_MODULES=y
> CONFIG_USB_GADGET=y
> CONFIG_USB_GADGETFS=y
> 
> When I build my kernel using buildroot, I get a .config file that has
> CONFIG_USB_GADGETFS=m.
> 
> I can reproduce the steps that buildroot executes and see this problem
> manually with:
> 
> echo CONFIG_MODULES=y > .config
> echo CONFIG_USB_GADGET=y >> .config
> echo CONFIG_USB_GADGETFS=y >> .config
> yes "" | make -k ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- oldconfig

This is due to this command, which doesn't do completely the right
thing.

The CONFIG_USB_GADGETFS option is somewhat odd, in that it is part of a
choice...endchoice block that itself is a tristate option.

If you use olddefconfig instead of the yes "" | ... oldconfig hack,
then it works fine.

See:

$ cat defconfig 
CONFIG_MODULES=y
CONFIG_USB_GADGET=y
CONFIG_USB_GADGETFS=y
$ cat defconfig > .config
$ make ARCH=arm olddefconfig
  GEN     ./Makefile
scripts/kconfig/conf  --olddefconfig Kconfig
#
# configuration written to .config
#
$ grep GADGETFS .config
CONFIG_USB_GADGETFS=y

But:

$ cat defconfig > .config
$ yes "" | make ARCH=arm oldconfig
$ grep GADGETFS .config
CONFIG_USB_GADGETFS=m

> Does anybody know why "make oldconfig" does this?  Am I missing
> another configuration item?

From "oldconfig" point of view, the default value for the tristate
mentioned above is "M":

  USB Gadget Drivers [M/y/?] (NEW)

And therefore the yes "" | make oldconfig hack enables this "USB Gadget
Drivers" option as a module. But since this option has no name, we
cannot set it in the defconfig file.

I've Cc'ed Yann on this aspect, and also my colleague Alex who already
saw this problem in the past IIRC.

> On a slightly related, but totally off-subject topic, is there a more
> modern way to implement a userspace gadget device driver than using
> gadgetfs?  That could be an alternative solution as well.  (We have an
> existing product that communicates with a host that uses libusb.  The
> embedded Linux side uses a gadgetfs user space driver.)

Look at functionfs:

config USB_CONFIGFS_F_FS
        bool "Function filesystem (FunctionFS)"
        depends on USB_CONFIGFS
        select USB_F_FS
        help
          The Function Filesystem (FunctionFS) lets one create USB
          composite functions in user space in the same way GadgetFS
          lets one create USB gadgets in user space.  This allows creation
          of composite gadgets such that some of the functions are
          implemented in kernel space (for instance Ethernet, serial or
          mass storage) and other are implemented in user space.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] Incorrect kernel configuration produced from _defconfig file

[Alexandre Belloni]

Hi,

On 25/02/2016 at 17:04:43 +0100, Thomas Petazzoni wrote :
> I've Cc'ed Yann on this aspect, and also my colleague Alex who already
> saw this problem in the past IIRC.
> 

Yes, that is how I solved it for OE with Yann's advice:
http://git.openembedded.org/openembedded-core/commit/?id=9b75f6a5786ff7b2e6219d78b38f0032f100c660

oldnoconfig is now an alias for olddefconfig but appeared earlier so it
was used to do the right thing with more kernel version.

-- 
Alexandre Belloni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] Incorrect kernel configuration produced from _defconfig file

[Patrick Doyle]

On Thu, Feb 25, 2016 at 11:14 AM, Alexandre Belloni
<alexandre.belloni@free-electrons.com> wrote:
> Hi,
>
> On 25/02/2016 at 17:04:43 +0100, Thomas Petazzoni wrote :
>> I've Cc'ed Yann on this aspect, and also my colleague Alex who already
>> saw this problem in the past IIRC.
>>
>
> Yes, that is how I solved it for OE with Yann's advice:
> http://git.openembedded.org/openembedded-core/commit/?id=9b75f6a5786ff7b2e6219d78b38f0032f100c660
>
> oldnoconfig is now an alias for olddefconfig but appeared earlier so it
> was used to do the right thing with more kernel version.
>
So this sounds like a change in buildroot is warranted if we want to
change this behavior.  I could submit a patch to buildroot that mimics
Alexandre's Yocto patch.  Is that the next step here?

--wpd

[Buildroot] Incorrect kernel configuration produced from _defconfig file

[Alexandre Belloni]

On 25/02/2016 at 11:24:31 -0500, Patrick Doyle wrote :
> On Thu, Feb 25, 2016 at 11:14 AM, Alexandre Belloni
> <alexandre.belloni@free-electrons.com> wrote:
> > Hi,
> >
> > On 25/02/2016 at 17:04:43 +0100, Thomas Petazzoni wrote :
> >> I've Cc'ed Yann on this aspect, and also my colleague Alex who already
> >> saw this problem in the past IIRC.
> >>
> >
> > Yes, that is how I solved it for OE with Yann's advice:
> > http://git.openembedded.org/openembedded-core/commit/?id=9b75f6a5786ff7b2e6219d78b38f0032f100c660
> >
> > oldnoconfig is now an alias for olddefconfig but appeared earlier so it
> > was used to do the right thing with more kernel version.
> >
> So this sounds like a change in buildroot is warranted if we want to
> change this behavior.  I could submit a patch to buildroot that mimics
> Alexandre's Yocto patch.  Is that the next step here?
> 

Personally, I'd drop the yes "" | part of it and use only make
olddefconfig but the rules are also used for busybox et al. so you'd
have to check that all have a recent enough kconfig

-- 
Alexandre Belloni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] Incorrect kernel configuration produced from _defconfig file

[Peter Korsgaard]

>>>>> "Patrick" == Patrick Doyle <wpdster@gmail.com> writes:

Hi,

>> Yes, that is how I solved it for OE with Yann's advice:
 >> http://git.openembedded.org/openembedded-core/commit/?id=9b75f6a5786ff7b2e6219d78b38f0032f100c660
 >> 
 >> oldnoconfig is now an alias for olddefconfig but appeared earlier so it
 >> was used to do the right thing with more kernel version.
 >> 
 > So this sounds like a change in buildroot is warranted if we want to
 > change this behavior.  I could submit a patch to buildroot that mimics
 > Alexandre's Yocto patch.  Is that the next step here?

Yes, it sounds like it. It seems like oldnoconfig was added in commit
ef61ca88c511154d6bead23c08f9a021cfdfeb01 (2.6.36), so it should work for
anything more recent than that (and otherwise it falls back to what we
have now).

-- 
Bye, Peter Korsgaard

[Buildroot] Incorrect kernel configuration produced from _defconfig file

[Thomas Petazzoni]

Dear Alexandre Belloni,

On Thu, 25 Feb 2016 17:50:32 +0100, Alexandre Belloni wrote:

> Personally, I'd drop the yes "" | part of it and use only make
> olddefconfig but the rules are also used for busybox et al. so you'd
> have to check that all have a recent enough kconfig

I think the reason we still use the yes "" trick is that we have a
generic kconfig infrastructure, and it should work for all kconfig
based packages. Including old kernel version (when was
olddefconfig introduced?) or Busybox (which has an horribly old copy of
the kconfig code).

So switching to olddefconfig might not be that easy. But maybe this is
something that can be made conditional in the pkg-kconfig
infrastructure. Like use olddefconfig if available, and if not fall
back to the yes "" | make oldconfig trick.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] Incorrect kernel configuration produced from _defconfig file

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 > Dear Alexandre Belloni,
 > On Thu, 25 Feb 2016 17:50:32 +0100, Alexandre Belloni wrote:

 >> Personally, I'd drop the yes "" | part of it and use only make
 >> olddefconfig but the rules are also used for busybox et al. so you'd
 >> have to check that all have a recent enough kconfig

 > I think the reason we still use the yes "" trick is that we have a
 > generic kconfig infrastructure, and it should work for all kconfig
 > based packages. Including old kernel version (when was
 > olddefconfig introduced?) or Busybox (which has an horribly old copy of
 > the kconfig code).

oldnoconfig was introduced (renamed) just before 2.6.36
(ef61ca88c511154d6bead23c08f9a021cfdfeb01) and olddefconfig was
introduced during the 3.7 cycle (fb16d8912db5268f29706010ecafff74b971c58d).

With the recent discussion about if we would rely on devtmpfs or not
(which was added in the 2.6.32 cycle), I don't think we can rely on any
of them without the fallback code.

The fact that oldnoconfig == olddefconfig is also kernel specific, but
from a quick look it doesn't seem to cause any conflicts:

 - busybox doesn't have it
 - barebox, uClibc and uboot have it with similar behaviour as the kernel

 > So switching to olddefconfig might not be that easy. But maybe this is
 > something that can be made conditional in the pkg-kconfig
 > infrastructure. Like use olddefconfig if available, and if not fall
 > back to the yes "" | make oldconfig trick.

Why not just do it like in OE ? run the yes "" .. thing if make
oldnoconfig fails?

-- 
Bye, Peter Korsgaard

[Buildroot] Incorrect kernel configuration produced from _defconfig file

[Thomas Petazzoni]

Hello,

On Thu, 25 Feb 2016 22:33:30 +0100, Peter Korsgaard wrote:

> oldnoconfig was introduced (renamed) just before 2.6.36
> (ef61ca88c511154d6bead23c08f9a021cfdfeb01) and olddefconfig was
> introduced during the 3.7 cycle (fb16d8912db5268f29706010ecafff74b971c58d).
> 
> With the recent discussion about if we would rely on devtmpfs or not
> (which was added in the 2.6.32 cycle), I don't think we can rely on any
> of them without the fallback code.
> 
> The fact that oldnoconfig == olddefconfig is also kernel specific, but
> from a quick look it doesn't seem to cause any conflicts:
> 
>  - busybox doesn't have it
>  - barebox, uClibc and uboot have it with similar behaviour as the kernel

It is somewhat odd that oldnoconfig and olddefconfig are doing the
same. If they are the same, why do they have two different names?

I would have expected oldnoconfig to say "no" to all new options, and
olddefconfig to say "use the default" to all new options.

>  > So switching to olddefconfig might not be that easy. But maybe this is
>  > something that can be made conditional in the pkg-kconfig
>  > infrastructure. Like use olddefconfig if available, and if not fall
>  > back to the yes "" | make oldconfig trick.
> 
> Why not just do it like in OE ? run the yes "" .. thing if make
> oldnoconfig fails?

Yes, sure, this seems a good idea. This way we have a good/sane
behavior for the packages using a modern kconfig code base, and
fallback to a working solution for old stuff like busybox.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] Incorrect kernel configuration produced from _defconfig file

[Alexandre Belloni]

On 25/02/2016 at 23:06:03 +0100, Thomas Petazzoni wrote :
> Hello,
> 
> On Thu, 25 Feb 2016 22:33:30 +0100, Peter Korsgaard wrote:
> 
> > oldnoconfig was introduced (renamed) just before 2.6.36
> > (ef61ca88c511154d6bead23c08f9a021cfdfeb01) and olddefconfig was
> > introduced during the 3.7 cycle (fb16d8912db5268f29706010ecafff74b971c58d).
> > 
> > With the recent discussion about if we would rely on devtmpfs or not
> > (which was added in the 2.6.32 cycle), I don't think we can rely on any
> > of them without the fallback code.
> > 
> > The fact that oldnoconfig == olddefconfig is also kernel specific, but
> > from a quick look it doesn't seem to cause any conflicts:
> > 
> >  - busybox doesn't have it
> >  - barebox, uClibc and uboot have it with similar behaviour as the kernel
> 
> It is somewhat odd that oldnoconfig and olddefconfig are doing the
> same. If they are the same, why do they have two different names?
> 
> I would have expected oldnoconfig to say "no" to all new options, and
> olddefconfig to say "use the default" to all new options.
> 

That's exactly why it has been renamed:

scripts/kconfig/Makefile:
# oldnoconfig is an alias of olddefconfig, because people already are dependent
# on its behavior (sets new symbols to their default value but not 'n') with the
# counter-intuitive name.
oldnoconfig: olddefconfig

scripts/kconfig/conf.c:
	/*
	 * oldnoconfig is an alias of olddefconfig, because people already
	 * are dependent on its behavior(sets new symbols to their default
	 * value but not 'n') with the counter-intuitive name.
	 */
	{"oldnoconfig",     no_argument,       NULL, olddefconfig},

See fb16d8912db5268f29706010ecafff74b971c58d and
67d34a6a391369269a2e5dba8a5f42cc4cd50231


-- 
Alexandre Belloni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com