[Buildroot] [Bug 8156] New: pkg-kconfig infra broken for *-update-{config, defconfig}

[Buildroot] [Bug 8156] New: pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

           Summary: pkg-kconfig infra broken for
                    *-update-{config,defconfig}
           Product: buildroot
           Version: 2015.05
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: Other
        AssignedTo: unassigned at buildroot.uclibc.org
        ReportedBy: thomas.petazzoni at free-electrons.com
                CC: buildroot at uclibc.org
   Estimated Hours: 0.0


The pkg-kconfig infrastructure seems to be broken in its implementation of the
*-update-{config,defconfig}, for example linux-update-config and
linux-update-defconfig.

Here is a scenario with the kernel that shows the problem:

 1/ Run "make menuconfig"

 2/ Use ARM, enable the Linux kernel, say version 4.0, and use omap2plus as the
defconfig.

 3/ Run "make linux-menuconfig", enable some random option.

 4/ Go back to "make menuconfig", and switch from using the omap2plus defconfig
to a custom config file, say "linux.config"

 5/ Now run "make linux-update-config"

The expected behavior is that Buildroot should copy
output/build/linux-4.0/.config to linux.config.

However, Buildroot tries to do the opposite: it tries to copy linux.config to
output/build/linux-4.0/.config:

thomas at skate:~/projets/buildroot (master)$ LANG=C make linux-update-config
/usr/bin/install -m 0644 linux.config
/home/thomas/projets/buildroot/output/build/linux-4.0/.config
/usr/bin/install: cannot stat 'linux.config': No such file or directory

This is probably a bug in the recent changes of the pkg-config infrastructure.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 8156] pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

Yann E. MORIN <yann.morin.1998@free.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |yann.morin.1998 at free.fr
         AssignedTo|unassigned at buildroot.uclibc |yann.morin.1998 at free.fr
                   |.org                        |

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 8156] pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

--- Comment #1 from Yann E. MORIN <yann.morin.1998@free.fr> 2015-06-03 19:54:28 UTC ---
Sorry, it works for me here, strictly following your step-by-step
procedure from a previously clean tree, and tested both the in-tree
and out-of-tree builds, both with a relative path (like "linux.config")
and an absolute path (like "$(O)/linux.config").

However, let's look at your traces:

    $ make linux-update-config
    /usr/bin/install -m 0644 linux.config
    /home/thomas/projets/buildroot/output/build/linux-4.0/.config
    /usr/bin/install: cannot stat 'linux.config': No such file or directory

The 'install' command is run when there is not yet a .config file
for the package (linux in this case). Saving back the defconfig file
is done with a plain 'cp' (followed by a touch).

So, it seems in your case, Linux is not configured, and there is no
/home/thomas/projets/buildroot/output/build/linux-4.0/.config file.

Regards,
Yann E. MORIN.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 8156] pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

--- Comment #2 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2015-06-04 15:10:36 UTC ---
I'm a bit confused. On the bug report, you say that you cannot reproduce, but
on IRC, you were apparently saying that you could reproduce and that it wasn't
indeed a use case that hadn't been planned.

Who should I trust? The y_morin on IRC, or the Yann E. Morin in the bug tracker
? :-)

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 8156] pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

--- Comment #3 from Yann E. MORIN <yann.morin.1998@free.fr> 2015-06-04 17:12:53 UTC ---
Thomas,

> I'm a bit confused. On the bug report, you say that you cannot reproduce,
> but on IRC, you were apparently saying that you could reproduce and that
> it wasn't indeed a use case that hadn't been planned.

Well, I first did a quick review of the code, on which I reported on
IRC. That led me thinking it was indeed broken.

Then I actually tested your use-case, and was not able to reproduce.

Sorry for the confusion. I stand by the comment I made here on the
tracker: I can not reproduce, and I can't see why it is broken.

So, let's look at the code (after partially expanding variables in
package/pkg-kconfig.mk); we have this dependency chain:

    .PHONY: linux-update-config linux-savedefconfig

    linux-update-config: linux-savedefconfig

    linux-savedefconfig: $(LINUX_DIR)/.stamp_kconfig_fixup_done

    $(LINUX_DIR)/.stamp_kconfig_fixup_done: $(LINUX_DIR)/.config

    $(LINUX_DIR)/.config: $(LINUX_KCONFIG_FILE)

So in your case:

  - $(LINUX_KCONFIG_FILE) initially points to the bundled defconfig
    file, and the dependency chain is satisfied;

  - then you change it to point to an arbitrary filename, and the
    dependency chain is now broken, because that file does not yet
    exist;

  - but (because linux-update-config and linux-savedefconfig are .PHONY?)
    it is not a problem for make; at least it is not for my make-3.81.

What's your make version?

> Who should I trust? The y_morin on IRC, or the Yann E. Morin in the bug
> tracker ? :-)

Eh! ;-)

Regards,
Yann E. MORIN.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 8156] pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

--- Comment #4 from Yann E. MORIN <yann.morin.1998@free.fr> 2015-06-04 17:30:32 UTC ---
Thomas, All,

OK, I was using master to do thechecks, while you were probably stuck
on 2015.02, right? So, I had the kconfig fragment stuff coming into
play, and that hid the problem (but see below).

And in the end, I looked more in-depth at what was going on, and it is
even worse than what you reported: 'make linux-update-config' ends
without error, but the resulting file is garbage (does not match the
actual configuration).

So, indeed, there is a problem: errors are not caught, and that leaves
the user with the impresion everything was OK, when the resulting file
is broken.

I'll work on that.

Trust the y_morin on IRC, he was right after all (but not for the good
reason!).

Regards,
Yann E. MORIN.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 8156] pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

--- Comment #5 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2015-06-04 18:07:46 UTC ---
I am indeed using 2015.02. Glad to read that you have reproduced the problem
:-)

I now trust both y_morin on IRC and Yann E. Morin in the bug tracker :-)

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 8156] pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

--- Comment #6 from Yann E. MORIN <yann.morin.1998@free.fr> 2015-06-04 21:21:42 UTC ---
Thomas, All,

So, we have a bunch of pkg-kconfig issues introduced with the pkg-kconfig
infra itself, but also further bugs hiding those first ones, later bugs
introduced with the support for kconfig fragments.

For a starter, merge-config.sh does not bail out if the input file(s) do
not exist. Once that is fixed, your problem indeed re-appears. And I'm
afraid there's more going on behind the scene... :-(

I'm working on that, but that's far from trivial... :-(

Regards,
Yann E. MORIN.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 8156] pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

--- Comment #7 from Yann E. MORIN <yann.morin.1998@free.fr> 2015-06-04 22:07:02 UTC ---
Created attachment 6046
  --> https://bugs.busybox.net/attachment.cgi?id=6046
Rough tentative for a fix

Thomas,

This is definitely not the final patch, but a tentative
to get your use-case working.

This will have to be refined for a final solution (like,
splitting the merge-config.sh fix to a separate patch).

Regards,
Yann E. MORIN.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 8156] pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

Yann E. MORIN <yann.morin.1998@free.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
   Target Milestone|---                         |2015.08

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 8156] pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

--- Comment #8 from Floris Bos <bos@je-eigen-domein.nl> 2015-06-04 22:20:42 UTC ---
>[...] later bugs introduced with the support for kconfig fragments.
>For a starter, merge-config.sh does not bail out if the input file(s) do
>not exist.

But shouldn't make bail out with a "No rule to make target name-of-file" type
of error first because the input files do are listed as a dependency for
.config?

$$($(2)_DIR)/.config: $$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_FRAGMENT_FILES)

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 8156] pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

--- Comment #9 from Yann E. MORIN <yann.morin.1998@free.fr> 2015-06-04 22:37:40 UTC ---
Floris,

(In reply to comment #8)
> But shouldn't make bail out with a "No rule to make target name-of-file" type
> of error first because the input files do are listed as a dependency for
> .config?
> 
> $$($(2)_DIR)/.config: $$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_FRAGMENT_FILES)

Yep, I would have expected so, too. But it does not...

I'm using make-3.81, and it does not error-out. I've installed a
make-4.0, and it does not break either.

Regards,
Yann E. MORIN.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 8156] pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

--- Comment #10 from Floris Bos <bos@je-eigen-domein.nl> 2015-06-04 22:45:08 UTC ---
(In reply to comment #9)
> Floris,
> 
> (In reply to comment #8)
> > But shouldn't make bail out with a "No rule to make target name-of-file" type
> > of error first because the input files do are listed as a dependency for
> > .config?
> > 
> > $$($(2)_DIR)/.config: $$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_FRAGMENT_FILES)
> 
> Yep, I would have expected so, too. But it does not...
> 
> I'm using make-3.81, and it does not error-out. I've installed a
> make-4.0, and it does not break either.

Strange, that dependency rule did work for me when I submitted and tested the
kconfig patch.

And seems it worked for Arnout as well:
http://lists.busybox.net/pipermail/buildroot/2015-May/127584.html

But that's when running make normally with a fragment file missing that do was
specified. Haven't tested the exact scenario reported in this bug, and looked
what might make it different.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 8156] pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

--- Comment #11 from Yann E. MORIN <yann.morin.1998@free.fr> 2015-06-06 10:09:01 UTC ---
Floris, All,

(In reply to comment #10)
> (In reply to comment #9)
> > Floris,
> > 
> > (In reply to comment #8)
> > > But shouldn't make bail out with a "No rule to make target name-of-file" type
> > > of error first because the input files do are listed as a dependency for
> > > .config?
> > > 
> > > $$($(2)_DIR)/.config: $$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_FRAGMENT_FILES)
> > 
> > Yep, I would have expected so, too. But it does not...
> > 
> > I'm using make-3.81, and it does not error-out. I've installed a
> > make-4.0, and it does not break either.
> 
> Strange, that dependency rule did work for me when I submitted and tested the
> kconfig patch.

Nah, it did not work, because it is not the same thing.

What is not working is checking the _base_ kconfig file; a missing base
kconfig file is not caught, but missing fragments are properly caught
(but that's a problem, too).

The problem is that the base kconfig file has a dependency, but no rule,
and in this case is a non-existent file, so "make imagines this target
to have been updated whenever its rule is run. This implies that all
targets depending on this one will always have their recipe run."

    https://www.gnu.org/software/make/manual/make.html#Force-Targets

And this is exactly what happens here... Sigh...

I've already fixed that.

Now, I'm also saying that detecting missing fragments is an issue,
because they do not depend on the package being extracted first (like
the base kconfig file does), in case they are bundled in the package.

I'm fixing that now.

> And seems it worked for Arnout as well:
> http://lists.busybox.net/pipermail/buildroot/2015-May/127584.html
> 
> But that's when running make normally with a fragment file missing that do was
> specified. Haven't tested the exact scenario reported in this bug, and looked
> what might make it different.

Don't worry, it's not your fault, the bug was pre-existing the kconfig
fragments, but that's what exposed it.

Regards,
Yann E. MORIN.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 8156] pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

--- Comment #12 from Yann E. MORIN <yann.morin.1998@free.fr> 2015-06-06 11:58:26 UTC ---
Thomas, Floris, All,

Patch series just posted:
    http://lists.busybox.net/pipermail/buildroot/2015-June/129644.html

Regards,
Yann E. MORIN.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 8156] pkg-kconfig infra broken for *-update-{config, defconfig}

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=8156

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED

--- Comment #13 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2015-07-07 15:46:26 UTC ---
Patches have been merged.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.