[Buildroot] [PATCH RFC] kconfig: allow pre-seending randpackageconfig

[Buildroot] [PATCH RFC] kconfig: allow pre-seending randpackageconfig

[Yann E. MORIN]

NOTE: this is only a proof of concept, for initial feedback.

kconfig is not able to properly randomise choices when a base .config
file is provided.

Sicne we do have quite a few choices that selects one package or
another, and we do want to sometime build them, we need to be able to
preseed those packages.

Add a new environment variable that contains the path to a file
containing options to be preseeded to randpackageconfig.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

---
 Makefile | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Makefile b/Makefile
index 1af51de..eeaf7d3 100644
--- a/Makefile
+++ b/Makefile
@@ -751,6 +751,9 @@ randpackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@grep '^config BR2_PACKAGE_' Config.in.legacy | \
 		while read config pkg; do \
 		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
+ifneq ($(BR2_RAND_PRESEED_CONFIG),)
+	@cat $(BR2_RAND_PRESEED_CONFIG) >> $(CONFIG_DIR)/.config.nopkg
+endif
 	@$(COMMON_CONFIG_ENV) \
 		KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
 		$< --randconfig $(CONFIG_CONFIG_IN)
-- 
1.9.1

[Buildroot] [PATCH RFC] kconfig: allow pre-seending randpackageconfig

[Thomas De Schampheleire]

Hi Yann,

On Sun, Jul 20, 2014 at 1:25 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> NOTE: this is only a proof of concept, for initial feedback.
>

Commit title: s/pre-seending/pre-seeding/

> kconfig is not able to properly randomise choices when a base .config
> file is provided.
>
> Sicne we do have quite a few choices that selects one package or
> another, and we do want to sometime build them, we need to be able to

Since
sometimes

> preseed those packages.

Note that I don't find the above sentence extremely clear,
content-wise, so rewriting this may be good for a subsequent revision.

>
> Add a new environment variable that contains the path to a file
> containing options to be preseeded to randpackageconfig.
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

I'm fine with this idea.
We should add at least a comment in the Makefile to document this
though, just reading the target recipe is not trivial.

>
> ---
>  Makefile | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index 1af51de..eeaf7d3 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -751,6 +751,9 @@ randpackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>         @grep '^config BR2_PACKAGE_' Config.in.legacy | \
>                 while read config pkg; do \
>                 echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
> +ifneq ($(BR2_RAND_PRESEED_CONFIG),)
> +       @cat $(BR2_RAND_PRESEED_CONFIG) >> $(CONFIG_DIR)/.config.nopkg
> +endif
>         @$(COMMON_CONFIG_ENV) \
>                 KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
>                 $< --randconfig $(CONFIG_CONFIG_IN)
> --
> 1.9.1


Best regards,
Thomas

[Buildroot] [PATCH] randconfig: seed with BR2_RAND_PRESEED_CONFIG

[Arnout Vandecappelle]

Currently, randconfig is completely unusable. Instead, we use
randpackageconfig, with two use cases: in the autobuilders, and when
doing manual testing of the validity of depends/select chains. In both
cases, we actually want the possibility to explicitly select some
packages or to set a string value or to randomise between a choice or
to test bootloaders and kernels - all of this is currently impossible.

With this patch, we can use randconfig instead of randpackageconfig,
and provide it with a seed file specified with BR2_RAND_PRESEED_CONFIG.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 Makefile | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index c0dcec8..79b12c6 100644
--- a/Makefile
+++ b/Makefile
@@ -727,7 +727,16 @@ oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 
 randconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
-	@$(COMMON_CONFIG_ENV) $< --randconfig $(CONFIG_CONFIG_IN)
+ifneq ($(BR2_RAND_PRESEED_CONFIG),)
+	@cp $(BR2_RAND_PRESEED_CONFIG) $(CONFIG_DIR)/.config.nopkg
+endif
+	@grep '^config BR2_PACKAGE_' Config.in.legacy | \
+		while read config pkg; do \
+		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
+	@$(COMMON_CONFIG_ENV) \
+		KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
+		$< --randconfig $(CONFIG_CONFIG_IN)
+	@rm -f $(CONFIG_DIR)/.config.nopkg
 
 allyesconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
@@ -852,7 +861,8 @@ help:
 	@echo '  oldconfig              - resolve any unresolved symbols in .config'
 	@echo '  silentoldconfig        - Same as oldconfig, but quietly, additionally update deps'
 	@echo '  olddefconfig           - Same as silentoldconfig but sets new symbols to their default value'
-	@echo '  randconfig             - New config with random answer to all options'
+	@echo '  randconfig             - New config with random answer to all options -'
+	@echo '                             options can be forced by setting BR2_RAND_PRESEED_CONFIG'
 	@echo '  defconfig              - New config with default answer to all options'
 	@echo '                             BR2_DEFCONFIG, if set, is used as input'
 	@echo '  savedefconfig          - Save current config as ./defconfig (minimal config)'
-- 
2.1.1

[Buildroot] [PATCH v2 1/2] randconfig, allyesconfig: fix handling of legacy options

[Arnout Vandecappelle]

Randconfig and allyesconfig should never select legacy options, so use
the same trick as allpackageyesconfig and randpackageconfig to remove
them.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 Makefile | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 907a0fc..5bcaa77 100644
--- a/Makefile
+++ b/Makefile
@@ -731,11 +731,25 @@ oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 
 randconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
-	@$(COMMON_CONFIG_ENV) $< --randconfig $(CONFIG_CONFIG_IN)
+	@rm -f $(CONFIG_DIR)/.config.nopkg
+	@grep '^config ' Config.in.legacy | \
+		while read config pkg; do \
+		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
+	@$(COMMON_CONFIG_ENV) \
+		KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
+		$< --randconfig $(CONFIG_CONFIG_IN)
+	@rm -f $(CONFIG_DIR)/.config.nopkg
 
 allyesconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
-	@$(COMMON_CONFIG_ENV) $< --allyesconfig $(CONFIG_CONFIG_IN)
+	@rm -f $(CONFIG_DIR)/.config.nopkg
+	@grep '^config ' Config.in.legacy | \
+		while read config pkg; do \
+		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
+	@$(COMMON_CONFIG_ENV) \
+		KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
+		$< --allyesconfig $(CONFIG_CONFIG_IN)
+	@rm -f $(CONFIG_DIR)/.config.nopkg
 
 allnoconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
-- 
2.1.1

[Buildroot] [PATCH v2 2/2] randconfig: seed with BR2_RAND_PRESEED_CONFIG_FILES

[Arnout Vandecappelle]

Currently, randconfig is completely unusable. Instead, we use
randpackageconfig, with two use cases: in the autobuilders, and when
doing manual testing of the validity of depends/select chains. In both
cases, we actually want the possibility to explicitly select some
packages or to set a string value or to randomise between a choice or
to test bootloaders and kernels - all of this is currently impossible.

This patch adds a BR2_RAND_PRESEED_CONFIG_FILES environment variable
that can be set to a (set of) files with config options that should be
set or unset. randconfig and randpackageconfig will force these
values.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
v2:
 - split off the fix for legacy
 - apply to both randconfig and randpackageconfig
 - rename to BR2_RAND_PRESEED_CONFIG_FILES because it supports
   multiple files

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 Makefile | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 5bcaa77..fb99de1 100644
--- a/Makefile
+++ b/Makefile
@@ -732,6 +732,9 @@ oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 randconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@rm -f $(CONFIG_DIR)/.config.nopkg
+ifneq ($(BR2_RAND_PRESEED_CONFIG_FILES),)
+	@cp $(BR2_RAND_PRESEED_CONFIG_FILES) $(CONFIG_DIR)/.config.nopkg
+endif
 	@grep '^config ' Config.in.legacy | \
 		while read config pkg; do \
 		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
@@ -758,6 +761,9 @@ allnoconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 randpackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@grep -v BR2_PACKAGE_ $(BR2_CONFIG) > $(CONFIG_DIR)/.config.nopkg
+ifneq ($(BR2_RAND_PRESEED_CONFIG_FILES),)
+	@grep BR2_PACKAGE_ $(BR2_RAND_PRESEED_CONFIG_FILES) >> $(CONFIG_DIR)/.config.nopkg
+endif
 	@grep '^config BR2_PACKAGE_' Config.in.legacy | \
 		while read config pkg; do \
 		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
@@ -870,13 +876,15 @@ help:
 	@echo '  oldconfig              - resolve any unresolved symbols in .config'
 	@echo '  silentoldconfig        - Same as oldconfig, but quietly, additionally update deps'
 	@echo '  olddefconfig           - Same as silentoldconfig but sets new symbols to their default value'
-	@echo '  randconfig             - New config with random answer to all options'
+	@echo '  randconfig             - New config with random answer to all options -'
+	@echo '                             options can be forced by setting BR2_RAND_PRESEED_CONFIG_FILES'
 	@echo '  defconfig              - New config with default answer to all options'
 	@echo '                             BR2_DEFCONFIG, if set, is used as input'
 	@echo '  savedefconfig          - Save current config as ./defconfig (minimal config)'
 	@echo '  allyesconfig           - New config where all options are accepted with yes'
 	@echo '  allnoconfig            - New config where all options are answered with no'
-	@echo '  randpackageconfig      - New config with random answer to package options'
+	@echo '  randpackageconfig      - New config with random answer to package options -'
+	@echo '                             options can be forced by setting BR2_RAND_PRESEED_CONFIG_FILES'
 	@echo '  allyespackageconfig    - New config where pkg options are accepted with yes'
 	@echo '  allnopackageconfig     - New config where package options are answered with no'
 ifeq ($(BR2_PACKAGE_BUSYBOX),y)
-- 
2.1.1

[Buildroot] [PATCH v2 1/2] randconfig, allyesconfig: fix handling of legacy options

[Yann E. MORIN]

Arnout, All,

On 2014-10-12 20:21 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:
> Randconfig and allyesconfig should never select legacy options, so use
> the same trick as allpackageyesconfig and randpackageconfig to remove
> them.
> 
> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Regards,
Yann E. MORIN.

> ---
>  Makefile | 18 ++++++++++++++++--
>  1 file changed, 16 insertions(+), 2 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 907a0fc..5bcaa77 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -731,11 +731,25 @@ oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  
>  randconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
> -	@$(COMMON_CONFIG_ENV) $< --randconfig $(CONFIG_CONFIG_IN)
> +	@rm -f $(CONFIG_DIR)/.config.nopkg
> +	@grep '^config ' Config.in.legacy | \
> +		while read config pkg; do \
> +		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
> +	@$(COMMON_CONFIG_ENV) \
> +		KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
> +		$< --randconfig $(CONFIG_CONFIG_IN)
> +	@rm -f $(CONFIG_DIR)/.config.nopkg
>  
>  allyesconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
> -	@$(COMMON_CONFIG_ENV) $< --allyesconfig $(CONFIG_CONFIG_IN)
> +	@rm -f $(CONFIG_DIR)/.config.nopkg
> +	@grep '^config ' Config.in.legacy | \
> +		while read config pkg; do \
> +		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
> +	@$(COMMON_CONFIG_ENV) \
> +		KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
> +		$< --allyesconfig $(CONFIG_CONFIG_IN)
> +	@rm -f $(CONFIG_DIR)/.config.nopkg
>  
>  allnoconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
> -- 
> 2.1.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v2 2/2] randconfig: seed with BR2_RAND_PRESEED_CONFIG_FILES

[Yann E. MORIN]

Arnout, All,

On 2014-10-12 20:21 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:
> Currently, randconfig is completely unusable. Instead, we use
> randpackageconfig, with two use cases: in the autobuilders, and when
> doing manual testing of the validity of depends/select chains. In both
> cases, we actually want the possibility to explicitly select some
> packages or to set a string value or to randomise between a choice or
> to test bootloaders and kernels - all of this is currently impossible.
> 
> This patch adds a BR2_RAND_PRESEED_CONFIG_FILES environment variable
> that can be set to a (set of) files with config options that should be
> set or unset. randconfig and randpackageconfig will force these
> values.
> 
> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
> v2:
>  - split off the fix for legacy
>  - apply to both randconfig and randpackageconfig
>  - rename to BR2_RAND_PRESEED_CONFIG_FILES because it supports
>    multiple files
> 
> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> ---
>  Makefile | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 5bcaa77..fb99de1 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -732,6 +732,9 @@ oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  randconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@rm -f $(CONFIG_DIR)/.config.nopkg
> +ifneq ($(BR2_RAND_PRESEED_CONFIG_FILES),)
> +	@cp $(BR2_RAND_PRESEED_CONFIG_FILES) $(CONFIG_DIR)/.config.nopkg
> +endif

I guess you meant something like:
    @cat $(BR2_RAND_PRESEED_CONFIG_FILES) $>(CONFIG_DIR)/.config.nopkg

Here's what I get with cp:
    $ cp foo bar buz
    cp: target ?buz? is not a directory

>  	@grep '^config ' Config.in.legacy | \
>  		while read config pkg; do \
>  		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
> @@ -758,6 +761,9 @@ allnoconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  randpackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@grep -v BR2_PACKAGE_ $(BR2_CONFIG) > $(CONFIG_DIR)/.config.nopkg
> +ifneq ($(BR2_RAND_PRESEED_CONFIG_FILES),)
> +	@grep BR2_PACKAGE_ $(BR2_RAND_PRESEED_CONFIG_FILES) >> $(CONFIG_DIR)/.config.nopkg
> +endif

Why do you explicitly grep for packages here?

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v2 2/2] randconfig: seed with BR2_RAND_PRESEED_CONFIG_FILES

[Arnout Vandecappelle]

On 14/10/14 09:32, Yann E. MORIN wrote:
> Arnout, All,
>
> On 2014-10-12 20:21 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:
[snip]
> > diff --git a/Makefile b/Makefile
> > index 5bcaa77..fb99de1 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -732,6 +732,9 @@ oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> >  randconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> >      @mkdir -p $(BUILD_DIR)/buildroot-config
> >      @rm -f $(CONFIG_DIR)/.config.nopkg
> > +ifneq ($(BR2_RAND_PRESEED_CONFIG_FILES),)
> > +    @cp $(BR2_RAND_PRESEED_CONFIG_FILES) $(CONFIG_DIR)/.config.nopkg
> > +endif
>
> I guess you meant something like:
>     @cat $(BR2_RAND_PRESEED_CONFIG_FILES) $>(CONFIG_DIR)/.config.nopkg
>
> Here's what I get with cp:
>     $ cp foo bar buz
>     cp: target ?buz? is not a directory

 Well spotted, thanks. I changed _FILE into _FILES at the last moment
but never tested it.



>
> >      @grep '^config ' Config.in.legacy | \
> >          while read config pkg; do \
> >          echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
> > @@ -758,6 +761,9 @@ allnoconfig: $(BUILD_DIR)/buildroot-config/conf
> outputmakefile
> >  randpackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> >      @mkdir -p $(BUILD_DIR)/buildroot-config
> >      @grep -v BR2_PACKAGE_ $(BR2_CONFIG) > $(CONFIG_DIR)/.config.nopkg
> > +ifneq ($(BR2_RAND_PRESEED_CONFIG_FILES),)
> > +    @grep BR2_PACKAGE_ $(BR2_RAND_PRESEED_CONFIG_FILES) >>
> $(CONFIG_DIR)/.config.nopkg
> > +endif
>
> Why do you explicitly grep for packages here?

 .config.nopkg will already contain the config of everything else (cfr.
the line above). So if you keep the full preseed file, you'll get duplicate
definitions in the .config.nopkg.

 Of course, if you do

cp foo .config
make randpackageconfig BR2_RAND_PRESEED_CONFIG_FILES=bar

then the .config only contains whatever you had in foo, which is presumably
a toolchain config fragment. But that use case is better handled with

make randconfig BR2_RAND_PRESEED_CONFIG_FILES="foo bar"


 Regards,
 Arnout


-- 
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F