[Buildroot] [PATCH v2 0/4] Handle generated kconfig defconfigs

[Buildroot] [PATCH v2 0/4] Handle generated kconfig defconfigs

[Sam Bobroff]

See the first patch in this set for a description of the issue.


Changes v1 -> v2:
This version incorporates various improvements suggested by Arnout
Vandecappelle <arnout@mind.be>.

* The initial fix is now separated from it's first use.

* Some make error messages are now reported via $(error ...) which is simpler
  and provides better (IMHO) feedback than echo and exit and there are a few
  other minor makefile tweaks.

* Two other places (in boot/) now make use of the new method.

Please note that the changes to the boot targets are not tested (I included
them because Arnout indicated that they needed this fix), although they look
simple enough. I had a quick look at testing them but couldn't work out how to
do it. I'll test them if someone can give me instructions (including which
defconfigs are missing config files and which versions of the packages they
occur in if they differ from the default buildroot versions) or they could be
dropped.


Sam Bobroff (4):
  Handle kconfig generated defconfigs
  package/linux: Handle generated defconfigs
  boot/at91bootstrap3: Handled generated defconfigs
  boot/barebox: Handle generated defconfigs

 boot/at91bootstrap3/at91bootstrap3.mk |  9 ++++-----
 boot/barebox/barebox.mk               | 10 ++++------
 linux/linux.mk                        | 10 ++++------
 package/pkg-kconfig.mk                | 26 ++++++++++++++++++++------
 4 files changed, 32 insertions(+), 23 deletions(-)

-- 
2.1.4

[Buildroot] [PATCH v2 1/4] Handle kconfig generated defconfigs

[Sam Bobroff]

As of Linux kernel commit ea4d1a8 "powerpc/configs: Replace
pseries_le_defconfig with a Makefile target using merge_config" some
kconfig defconfigs (one so far: "pseries_le_defconfig") can be
generated using "make <defconfig>" and they do not exist on disk as a
single kconfig file.

This causes buildroot's build to fail for those configs with:
'arch/powerpc/configs/pseries_le_defconfig' for 'linux' does not exist

To handle this case and keep the makefile steps as simple as possible,
introduce a new package variable, *_KCONFIG_DEFCONFIG, that can be
used to indicate that a defconfig is being used, rather than a file.

This allows the rule that generates the .config file to use either the
provided file (by copying) or a generated defconfig (by running "make
<defconfig>") as it's starting point.  merge_config.sh can then
be run the same way in either case, using .config as both input and
output.

Note that merge_config.sh is now modifying .config in-place but this
is safe because it uses a temporary copy while making changes.

This patch introduces the new variable but does not make use of it.
Use of the new variable will be introduced in separate patches.

Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
---
 package/pkg-kconfig.mk | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/package/pkg-kconfig.mk b/package/pkg-kconfig.mk
index 44c74a7..47bb550 100644
--- a/package/pkg-kconfig.mk
+++ b/package/pkg-kconfig.mk
@@ -63,8 +63,14 @@ $$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_FRAGMENT_FILES): | $(1)-patch
 # full .config first. We use 'make oldconfig' because this can be safely
 # done even when the package does not support defconfigs.
 $$($(2)_DIR)/.config: $$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_FRAGMENT_FILES)
+	if [ ! -z "$$($(2)_KCONFIG_DEFCONFIG)" ] ; then \
+		$$($(2)_MAKE_ENV) $$(MAKE) -C $$($(2)_DIR) \
+			$$($(2)_KCONFIG_OPTS) $$($(2)_KCONFIG_DEFCONFIG)_defconfig ; \
+	else \
+		cp $$($(2)_KCONFIG_FILE) $$(@) ; \
+	fi
 	support/kconfig/merge_config.sh -m -O $$(@D) \
-		$$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_FRAGMENT_FILES)
+		$$(@) $$($(2)_KCONFIG_FRAGMENT_FILES)
 	$$(Q)yes "" | $$($(2)_MAKE_ENV) $$(MAKE) -C $$($(2)_DIR) \
 		$$($(2)_KCONFIG_OPTS) oldconfig
 
@@ -89,10 +95,10 @@ $$($(2)_TARGET_CONFIGURE): $$($(2)_DIR)/.stamp_kconfig_fixup_done
 # already called above, so we can effectively use this variable.
 ifeq ($$($$($(2)_KCONFIG_VAR)),y)
 
-# FOO_KCONFIG_FILE is required
 ifeq ($$(BR_BUILDING),y)
-ifeq ($$($(2)_KCONFIG_FILE),)
-$$(error Internal error: no value specified for $(2)_KCONFIG_FILE)
+# Either FOO_KCONFIG_FILE or FOO_KCONFIG_DEFCONFIG is required
+ifeq ($$($(2)_KCONFIG_FILE)$$($(2)_KCONFIG_DEFCONFIG),)
+$$(error Internal error: no value specified for $(2)_KCONFIG_FILE or $(2)_KCONFIG_DEFCONFIG)
 endif
 endif
 
@@ -159,7 +165,9 @@ $(1)-savedefconfig: $(1)-check-configuration-done
 # cp and 'touch --reference' is used for symmetry with $(1)-update-defconfig.
 $(1)-update-config: $(1)-check-configuration-done
 	@$$(if $$($(2)_KCONFIG_FRAGMENT_FILES), \
-		echo "Unable to perform $(1)-update-config when fragment files are set"; exit 1)
+		$$(error Unable to perform $(1)-update-config when fragment files are set))
+	@$$(if $$($(2)_KCONFIG_DEFCONFIG), \
+		$$(error Unable to perform $(1)-update-config when using an in-tree defconfig))
 	cp -f $$($(2)_DIR)/.config $$($(2)_KCONFIG_FILE)
 	touch --reference $$($(2)_DIR)/.config $$($(2)_KCONFIG_FILE)
 
@@ -169,7 +177,9 @@ $(1)-update-config: $(1)-check-configuration-done
 # we copy.
 $(1)-update-defconfig: $(1)-savedefconfig
 	@$$(if $$($(2)_KCONFIG_FRAGMENT_FILES), \
-		echo "Unable to perform $(1)-update-defconfig when fragment files are set"; exit 1)
+		$$(error Unable to perform $(1)-update-defconfig when fragment files are set))
+	@$$(if $$($(2)_KCONFIG_DEFCONFIG), \
+		$$(error Unable to perform $(1)-update-defconfig when using an in-tree defconfig"))
 	cp -f $$($(2)_DIR)/defconfig $$($(2)_KCONFIG_FILE)
 	touch --reference $$($(2)_DIR)/.config $$($(2)_KCONFIG_FILE)
 
-- 
2.1.4

[Buildroot] [PATCH v2 2/4] package/linux: Handle generated defconfigs

[Sam Bobroff]

Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
---
 linux/linux.mk         | 10 ++++------
 package/pkg-kconfig.mk |  8 ++++++--
 2 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/linux/linux.mk b/linux/linux.mk
index 48f9c74..2f6e884 100644
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -164,12 +164,10 @@ endef
 LINUX_POST_PATCH_HOOKS += LINUX_APPLY_LOCAL_PATCHES
 
 ifeq ($(BR2_LINUX_KERNEL_USE_DEFCONFIG),y)
-KERNEL_SOURCE_CONFIG = $(KERNEL_ARCH_PATH)/configs/$(call qstrip,$(BR2_LINUX_KERNEL_DEFCONFIG))_defconfig
+LINUX_KCONFIG_DEFCONFIG = $(call qstrip,$(BR2_LINUX_KERNEL_DEFCONFIG))
 else ifeq ($(BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG),y)
-KERNEL_SOURCE_CONFIG = $(call qstrip,$(BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE))
+LINUX_KCONFIG_FILE = $(call qstrip,$(BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE))
 endif
-
-LINUX_KCONFIG_FILE = $(KERNEL_SOURCE_CONFIG)
 LINUX_KCONFIG_FRAGMENT_FILES = $(call qstrip,$(BR2_LINUX_KERNEL_CONFIG_FRAGMENT_FILES))
 LINUX_KCONFIG_EDITORS = menuconfig xconfig gconfig nconfig
 LINUX_KCONFIG_OPTS = $(LINUX_MAKE_FLAGS)
@@ -359,13 +357,13 @@ LINUX_POST_INSTALL_TARGET_HOOKS += $(foreach tool,$(LINUX_TOOLS),\
 ifeq ($(BR_BUILDING),y)
 
 ifeq ($(BR2_LINUX_KERNEL_USE_DEFCONFIG),y)
-ifeq ($(call qstrip,$(BR2_LINUX_KERNEL_DEFCONFIG)),)
+ifeq ($(LINUX_KCONFIG_DEFCONFIG),)
 $(error No kernel defconfig name specified, check your BR2_LINUX_KERNEL_DEFCONFIG setting)
 endif
 endif
 
 ifeq ($(BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG),y)
-ifeq ($(call qstrip,$(BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE)),)
+ifeq ($(LINUX_KCONFIG_FILE),)
 $(error No kernel configuration file specified, check your BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE setting)
 endif
 endif
diff --git a/package/pkg-kconfig.mk b/package/pkg-kconfig.mk
index 47bb550..ed8ec07 100644
--- a/package/pkg-kconfig.mk
+++ b/package/pkg-kconfig.mk
@@ -96,10 +96,14 @@ $$($(2)_TARGET_CONFIGURE): $$($(2)_DIR)/.stamp_kconfig_fixup_done
 ifeq ($$($$($(2)_KCONFIG_VAR)),y)
 
 ifeq ($$(BR_BUILDING),y)
-# Either FOO_KCONFIG_FILE or FOO_KCONFIG_DEFCONFIG is required
-ifeq ($$($(2)_KCONFIG_FILE)$$($(2)_KCONFIG_DEFCONFIG),)
+# Either FOO_KCONFIG_FILE or FOO_KCONFIG_DEFCONFIG is required...
+ifeq ($$(or $$($(2)_KCONFIG_FILE),$$($(2)_KCONFIG_DEFCONFIG)),)
 $$(error Internal error: no value specified for $(2)_KCONFIG_FILE or $(2)_KCONFIG_DEFCONFIG)
 endif
+# ... but not both:
+ifneq ($$(and $$($(2)_KCONFIG_FILE),$$($(2)_KCONFIG_DEFCONFIG)),)
+$$(error Internal error: $(2)_KCONFIG_FILE and $(2)_KCONFIG_DEFCONFIG are mutually exclusive but but both are defined)
+endif
 endif
 
 # For the configurators, we do want to use the system-provided host
-- 
2.1.4

[Buildroot] [PATCH v2 3/4] boot/at91bootstrap3: Handled generated defconfigs

[Sam Bobroff]

Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
---
 boot/at91bootstrap3/at91bootstrap3.mk | 9 ++++-----
 package/pkg-kconfig.mk                | 2 +-
 2 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/boot/at91bootstrap3/at91bootstrap3.mk b/boot/at91bootstrap3/at91bootstrap3.mk
index fa67ea6..1910d46 100644
--- a/boot/at91bootstrap3/at91bootstrap3.mk
+++ b/boot/at91bootstrap3/at91bootstrap3.mk
@@ -41,12 +41,11 @@ define AT91BOOTSTRAP3_INSTALL_IMAGES_CMDS
 endef
 
 ifeq ($(BR2_TARGET_AT91BOOTSTRAP3_USE_DEFCONFIG),y)
-AT91BOOTSTRAP3_SOURCE_CONFIG = $(AT91BOOTSTRAP3_DIR)/board/*/$(call qstrip,$(BR2_TARGET_AT91BOOTSTRAP3_DEFCONFIG))_defconfig
+AT91BOOTSTRAP3_KCONFIG_DEFCONFIG = $(call qstrip,$(BR2_TARGET_AT91BOOTSTRAP3_DEFCONFIG))
 else ifeq ($(BR2_TARGET_AT91BOOTSTRAP3_USE_CUSTOM_CONFIG),y)
-AT91BOOTSTRAP3_SOURCE_CONFIG = $(call qstrip,$(BR2_TARGET_AT91BOOTSTRAP3_CUSTOM_CONFIG_FILE))
+AT91BOOTSTRAP3_KCONFIG_FILE = $(call qstrip,$(BR2_TARGET_AT91BOOTSTRAP3_CUSTOM_CONFIG_FILE))
 endif
 
-AT91BOOTSTRAP3_KCONFIG_FILE = $(AT91BOOTSTRAP3_SOURCE_CONFIG)
 AT91BOOTSTRAP3_KCONFIG_EDITORS = menuconfig xconfig gconfig
 AT91BOOTSTRAP3_KCONFIG_OPTS = $(AT91BOOTSTRAP3_MAKE_OPTS)
 
@@ -55,13 +54,13 @@ AT91BOOTSTRAP3_KCONFIG_OPTS = $(AT91BOOTSTRAP3_MAKE_OPTS)
 ifeq ($(BR_BUILDING),y)
 
 ifeq ($(BR2_TARGET_AT91BOOTSTRAP3_USE_DEFCONFIG),y)
-ifeq ($(call qstrip,$(BR2_TARGET_AT91BOOTSTRAP3_DEFCONFIG)),)
+ifeq ($(AT91BOOTSTRAP3_KCONFIG_DEFCONFIG),)
 $(error No at91bootstrap3 defconfig name specified, check your BR2_TARGET_AT91BOOTSTRAP3_DEFCONFIG setting)
 endif
 endif
 
 ifeq ($(BR2_TARGET_AT91BOOTSTRAP3_USE_CUSTOM_CONFIG),y)
-ifeq ($(call qstrip,$(BR2_TARGET_AT91BOOTSTRAP3_CUSTOM_CONFIG_FILE)),)
+ifeq ($(AT91BOOTSTRAP3_KCONFIG_FILE),)
 $(error No at91bootstrap3 configuration file specified, check your BR2_TARGET_AT91BOOTSTRAP3_CUSTOM_CONFIG_FILE setting)
 endif
 endif
diff --git a/package/pkg-kconfig.mk b/package/pkg-kconfig.mk
index ed8ec07..cf0c523 100644
--- a/package/pkg-kconfig.mk
+++ b/package/pkg-kconfig.mk
@@ -183,7 +183,7 @@ $(1)-update-defconfig: $(1)-savedefconfig
 	@$$(if $$($(2)_KCONFIG_FRAGMENT_FILES), \
 		$$(error Unable to perform $(1)-update-defconfig when fragment files are set))
 	@$$(if $$($(2)_KCONFIG_DEFCONFIG), \
-		$$(error Unable to perform $(1)-update-defconfig when using an in-tree defconfig"))
+		$$(error Unable to perform $(1)-update-defconfig when using an in-tree defconfig))
 	cp -f $$($(2)_DIR)/defconfig $$($(2)_KCONFIG_FILE)
 	touch --reference $$($(2)_DIR)/.config $$($(2)_KCONFIG_FILE)
 
-- 
2.1.4

[Buildroot] [PATCH v2 4/4] boot/barebox: Handle generated defconfigs

[Sam Bobroff]

Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
---
 boot/barebox/barebox.mk | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/boot/barebox/barebox.mk b/boot/barebox/barebox.mk
index e45976d..018b79c 100644
--- a/boot/barebox/barebox.mk
+++ b/boot/barebox/barebox.mk
@@ -57,13 +57,11 @@ BAREBOX_MAKE_FLAGS = ARCH=$(BAREBOX_ARCH) CROSS_COMPILE="$(CCACHE) \
 BAREBOX_MAKE_ENV = $(TARGET_MAKE_ENV)
 
 ifeq ($(BR2_TARGET_BAREBOX_USE_DEFCONFIG),y)
-BAREBOX_SOURCE_CONFIG = $(BAREBOX_DIR)/arch/$(BAREBOX_ARCH)/configs/$(call qstrip,\
-	$(BR2_TARGET_BAREBOX_BOARD_DEFCONFIG))_defconfig
+BAREBOX_KCONFIG_DEFCONFIG = $(call qstrip,$(BR2_TARGET_BAREBOX_BOARD_DEFCONFIG))
 else ifeq ($(BR2_TARGET_BAREBOX_USE_CUSTOM_CONFIG),y)
-BAREBOX_SOURCE_CONFIG = $(call qstrip,$(BR2_TARGET_BAREBOX_CUSTOM_CONFIG_FILE))
+BAREBOX_KCONFIG_FILE = $(call qstrip,$(BR2_TARGET_BAREBOX_CUSTOM_CONFIG_FILE))
 endif
 
-BAREBOX_KCONFIG_FILE = $(BAREBOX_SOURCE_CONFIG)
 BAREBOX_KCONFIG_FRAGMENT_FILES = $(call qstrip,$(BR2_TARGET_BAREBOX_CONFIG_FRAGMENT_FILES))
 BAREBOX_KCONFIG_EDITORS = menuconfig xconfig gconfig nconfig
 BAREBOX_KCONFIG_OPTS = $(BAREBOX_MAKE_FLAGS)
@@ -112,8 +110,8 @@ endif
 # Checks to give errors that the user can understand
 # Must be before we call to kconfig-package
 ifeq ($(BR2_TARGET_BAREBOX)$(BR_BUILDING),yy)
-ifeq ($(BAREBOX_SOURCE_CONFIG),)
-$(error No Barebox config file. Check your BR2_TARGET_BAREBOX_BOARD_DEFCONFIG or BR2_TARGET_BAREBOX_CUSTOM_CONFIG_FILE settings)
+ifeq ($(or $(BAREBOX_KCONFIG_FILE),$(BAREBOX_KCONFIG_DEFCONFIG)),)
+$(error No Barebox config. Check your BR2_TARGET_BAREBOX_BOARD_DEFCONFIG or BR2_TARGET_BAREBOX_CUSTOM_CONFIG_FILE settings)
 endif
 endif
 
-- 
2.1.4

[Buildroot] [PATCH v2 0/4] Handle generated kconfig defconfigs

[Yann E. MORIN]

Sam, All,

On 2015-09-02 13:54 +1000, Sam Bobroff spake thusly:
[--SNIP--]
> Sam Bobroff (4):
>   Handle kconfig generated defconfigs
>   package/linux: Handle generated defconfigs
>   boot/at91bootstrap3: Handled generated defconfigs
>   boot/barebox: Handle generated defconfigs

We've discussed this series of yours a while ago on IRC, and we decided
I would adopt it to clean it and respin it. I've now done that, and just
resent an updated version of your patches:

    https://patchwork.ozlabs.org/patch/554822/
    https://patchwork.ozlabs.org/patch/554823/
    https://patchwork.ozlabs.org/patch/554824/
    https://patchwork.ozlabs.org/patch/554825/
    https://patchwork.ozlabs.org/patch/554826/

Of course, those patches are still attributed to you as the author.
You ar ewelcome to review them and give your opinion and this new
iteration.

Thank you very much for your initial submission! :-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v2 0/4] Handle generated kconfig defconfigs

[Sam Bobroff]

On Wed, Dec 09, 2015 at 10:53:42PM +0100, Yann E. MORIN wrote:
> Sam, All,
> 
> On 2015-09-02 13:54 +1000, Sam Bobroff spake thusly:
> [--SNIP--]
> > Sam Bobroff (4):
> >   Handle kconfig generated defconfigs
> >   package/linux: Handle generated defconfigs
> >   boot/at91bootstrap3: Handled generated defconfigs
> >   boot/barebox: Handle generated defconfigs
> 
> We've discussed this series of yours a while ago on IRC, and we decided
> I would adopt it to clean it and respin it. I've now done that, and just
> resent an updated version of your patches:
> 
>     https://patchwork.ozlabs.org/patch/554822/
>     https://patchwork.ozlabs.org/patch/554823/
>     https://patchwork.ozlabs.org/patch/554824/
>     https://patchwork.ozlabs.org/patch/554825/
>     https://patchwork.ozlabs.org/patch/554826/
> 
> Of course, those patches are still attributed to you as the author.
> You ar ewelcome to review them and give your opinion and this new
> iteration.
> 
> Thank you very much for your initial submission! :-)
> 
> Regards,
> Yann E. MORIN.

Hi Yann,

Looks good to me, thanks for cleaning it up :-)

One thing that bothered me was that I wasn't able to test the non-kernel parts
of it. Has someone been able to do that?

Cheers,
Sam.