* [PATCH] kconfig: Add merge_config.sh script
@ 2011-10-24 22:48 John Stultz
2011-10-24 23:05 ` Darren Hart
0 siblings, 1 reply; 13+ messages in thread
From: John Stultz @ 2011-10-24 22:48 UTC (permalink / raw)
To: LKML
Cc: John Stultz, Sam Ravnborg, gthelen, tartler, Dmitry Fink,
Darren Hart, Eric B Munson, Bruce Ashfield, Michal Marek,
linux-kbuild
No changes here from the last time, but since I didn't get any
feedback last time, I wanted to send this out for possible inclusion.
After noticing almost every distro has their own method of managing
config fragments, I went looking at some best practices, and wanted
to try to consolidate some of the different approaches so this fairly
simple infrastructure can be shared (and new distros/build systems
don't have to implement yet another config fragment merge script).
This script is most influenced by the Windriver tools used in
the Yocto Project, reusing some portions found there.
This script merges multiple config fragments, warning on any
overrided values. It then sets any unspecified values to their
default, then finally checks to make sure no specified value was
dropped due to unsatisfied dependencies.
I'm sure this implementation won't work for everyone, and I expect
it will need to evolve to adapt for various use cases. But I think
its a reasonable starting point.
v2:
* Reworked to use alldefconfig instead of the proposed
olddefconfig as suggested by Sam Ravnborg.
v3:
* Script improvements from Dmitri.
* allnoconfig option from Darren
* pre-make exit option from Darren
* lots of other fixes/cleanups from Darren.
* Fix final check to not compain about config values in comments
Please let me know if you have any comments or thoughts!
CC: Sam Ravnborg <sam@ravnborg.org>
CC: gthelen@google.com
CC: tartler@cs.fau.de
CC: Dmitry Fink <Dmitry.Fink@palm.com>
CC: Darren Hart <dvhart@linux.intel.com>
CC: Eric B Munson <ebmunson@us.ibm.com>
CC: Bruce Ashfield <Bruce.Ashfield@windriver.com>
CC: Michal Marek <mmarek@suse.cz>
CC: linux-kbuild@vger.kernel.org
Signed-off-by: John Stultz <john.stultz@linaro.org>
---
scripts/kconfig/merge_config.sh | 117 +++++++++++++++++++++++++++++++++++++++
1 files changed, 117 insertions(+), 0 deletions(-)
create mode 100755 scripts/kconfig/merge_config.sh
diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh
new file mode 100755
index 0000000..890276b
--- /dev/null
+++ b/scripts/kconfig/merge_config.sh
@@ -0,0 +1,117 @@
+#!/bin/sh
+# merge_config.sh - Takes a list of config fragment values, and merges
+# them one by one. Provides warnings on overridden values, and specified
+# values that did not make it to the resulting .config file (due to missed
+# dependencies or config symbol removal).
+#
+# Portions reused from kconf_check and generate_cfg:
+# http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/kconf_check
+# http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/generate_cfg
+#
+# Copyright (c) 2009-2010 Wind River Systems, Inc.
+# Copyright 2011 Linaro
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+# See the GNU General Public License for more details.
+
+clean_up() {
+ rm -f $TMP_FILE
+ exit
+}
+trap clean_up SIGHUP SIGINT SIGTERM
+
+usage() {
+ echo "Usage: $0 [OPTIONS] [CONFIG [...]]"
+ echo " -h display this help text"
+ echo " -m only merge the fragments, do not execute the make command"
+ echo " -n use allnoconfig instead of alldefconfig"
+}
+
+MAKE=true
+ALLTARGET=alldefconfig
+
+while true; do
+ case $1 in
+ "-n")
+ ALLTARGET=allnoconfig
+ shift
+ continue
+ ;;
+ "-m")
+ MAKE=false
+ shift
+ continue
+ ;;
+ "-h")
+ usage
+ exit
+ ;;
+ *)
+ break
+ ;;
+ esac
+done
+
+
+
+MERGE_LIST=$*
+SED_CONFIG_EXP="s/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/p"
+TMP_FILE=$(mktemp ./.tmp.config.XXXXXXXXXX)
+
+# Merge files, printing warnings on overrided values
+for MERGE_FILE in $MERGE_LIST ; do
+ echo "Merging $MERGE_FILE"
+ CFG_LIST=$(sed -n "$SED_CONFIG_EXP" $MERGE_FILE)
+
+ for CFG in $CFG_LIST ; do
+ grep -q -w $CFG $TMP_FILE
+ if [ $? -eq 0 ] ; then
+ PREV_VAL=$(grep -w $CFG $TMP_FILE)
+ NEW_VAL=$(grep -w $CFG $MERGE_FILE)
+ if [ "x$PREV_VAL" != "x$NEW_VAL" ] ; then
+ echo Value of $CFG is redefined by fragment $MERGE_FILE:
+ echo Previous value: $PREV_VAL
+ echo New value: $NEW_VAL
+ echo
+ fi
+ sed -i "/$CFG[ =]/d" $TMP_FILE
+ fi
+ done
+ cat $MERGE_FILE >> $TMP_FILE
+done
+
+if [ "$MAKE" = "false" ]; then
+ cp $TMP_FILE .config
+ echo "#"
+ echo "# merged configuration written to .config (needs make)"
+ echo "#"
+ clean_up
+ exit
+fi
+
+# Use the merged file as the starting point for:
+# alldefconfig: Fills in any missing symbols with Kconfig default
+# allnoconfig: Fills in any missing symbols with # CONFIG_* is not set
+make KCONFIG_ALLCONFIG=$TMP_FILE $ALLTARGET
+
+
+# Check all specified config values took (might have missed-dependency issues)
+for CFG in $(sed -n "$SED_CONFIG_EXP" $TMP_FILE); do
+
+ REQUESTED_VAL=$(sed -n "$SED_CONFIG_EXP" $TMP_FILE | grep -w -e "$CFG")
+ ACTUAL_VAL=$(sed -n "$SED_CONFIG_EXP" .config | grep -w -e "$CFG")
+ if [ "x$REQUESTED_VAL" != "x$ACTUAL_VAL" ] ; then
+ echo "Value requested for $CFG not in final .config"
+ echo "Requested value: $REQUESTED_VAL"
+ echo "Actual value: $ACTUAL_VAL"
+ echo ""
+ fi
+done
+
+clean_up
--
1.7.3.2.146.gca209
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [PATCH] kconfig: Add merge_config.sh script
2011-10-24 22:48 [PATCH] kconfig: Add merge_config.sh script John Stultz
@ 2011-10-24 23:05 ` Darren Hart
0 siblings, 0 replies; 13+ messages in thread
From: Darren Hart @ 2011-10-24 23:05 UTC (permalink / raw)
To: John Stultz
Cc: LKML, Sam Ravnborg, gthelen, tartler, Dmitry Fink, Eric B Munson,
Bruce Ashfield, Michal Marek, linux-kbuild
On 10/25/2011 12:48 AM, John Stultz wrote:
> No changes here from the last time, but since I didn't get any
> feedback last time, I wanted to send this out for possible inclusion.
>
> After noticing almost every distro has their own method of managing
> config fragments, I went looking at some best practices, and wanted
> to try to consolidate some of the different approaches so this fairly
> simple infrastructure can be shared (and new distros/build systems
> don't have to implement yet another config fragment merge script).
>
> This script is most influenced by the Windriver tools used in
> the Yocto Project, reusing some portions found there.
This adds some additional features to what we currently do in the Yocto
Project now and we would like to just reuse what is upstream rather than
rolling our own. This would be good to have upstream from our perspective.
--
Darren
>
> This script merges multiple config fragments, warning on any
> overrided values. It then sets any unspecified values to their
> default, then finally checks to make sure no specified value was
> dropped due to unsatisfied dependencies.
>
> I'm sure this implementation won't work for everyone, and I expect
> it will need to evolve to adapt for various use cases. But I think
> its a reasonable starting point.
>
> v2:
> * Reworked to use alldefconfig instead of the proposed
> olddefconfig as suggested by Sam Ravnborg.
>
> v3:
> * Script improvements from Dmitri.
> * allnoconfig option from Darren
> * pre-make exit option from Darren
> * lots of other fixes/cleanups from Darren.
> * Fix final check to not compain about config values in comments
>
> Please let me know if you have any comments or thoughts!
>
> CC: Sam Ravnborg <sam@ravnborg.org>
> CC: gthelen@google.com
> CC: tartler@cs.fau.de
> CC: Dmitry Fink <Dmitry.Fink@palm.com>
> CC: Darren Hart <dvhart@linux.intel.com>
> CC: Eric B Munson <ebmunson@us.ibm.com>
> CC: Bruce Ashfield <Bruce.Ashfield@windriver.com>
> CC: Michal Marek <mmarek@suse.cz>
> CC: linux-kbuild@vger.kernel.org
> Signed-off-by: John Stultz <john.stultz@linaro.org>
> ---
> scripts/kconfig/merge_config.sh | 117 +++++++++++++++++++++++++++++++++++++++
> 1 files changed, 117 insertions(+), 0 deletions(-)
> create mode 100755 scripts/kconfig/merge_config.sh
>
> diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh
> new file mode 100755
> index 0000000..890276b
> --- /dev/null
> +++ b/scripts/kconfig/merge_config.sh
> @@ -0,0 +1,117 @@
> +#!/bin/sh
> +# merge_config.sh - Takes a list of config fragment values, and merges
> +# them one by one. Provides warnings on overridden values, and specified
> +# values that did not make it to the resulting .config file (due to missed
> +# dependencies or config symbol removal).
> +#
> +# Portions reused from kconf_check and generate_cfg:
> +# http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/kconf_check
> +# http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/generate_cfg
> +#
> +# Copyright (c) 2009-2010 Wind River Systems, Inc.
> +# Copyright 2011 Linaro
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License version 2 as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> +# See the GNU General Public License for more details.
> +
> +clean_up() {
> + rm -f $TMP_FILE
> + exit
> +}
> +trap clean_up SIGHUP SIGINT SIGTERM
> +
> +usage() {
> + echo "Usage: $0 [OPTIONS] [CONFIG [...]]"
> + echo " -h display this help text"
> + echo " -m only merge the fragments, do not execute the make command"
> + echo " -n use allnoconfig instead of alldefconfig"
> +}
> +
> +MAKE=true
> +ALLTARGET=alldefconfig
> +
> +while true; do
> + case $1 in
> + "-n")
> + ALLTARGET=allnoconfig
> + shift
> + continue
> + ;;
> + "-m")
> + MAKE=false
> + shift
> + continue
> + ;;
> + "-h")
> + usage
> + exit
> + ;;
> + *)
> + break
> + ;;
> + esac
> +done
> +
> +
> +
> +MERGE_LIST=$*
> +SED_CONFIG_EXP="s/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/p"
> +TMP_FILE=$(mktemp ./.tmp.config.XXXXXXXXXX)
> +
> +# Merge files, printing warnings on overrided values
> +for MERGE_FILE in $MERGE_LIST ; do
> + echo "Merging $MERGE_FILE"
> + CFG_LIST=$(sed -n "$SED_CONFIG_EXP" $MERGE_FILE)
> +
> + for CFG in $CFG_LIST ; do
> + grep -q -w $CFG $TMP_FILE
> + if [ $? -eq 0 ] ; then
> + PREV_VAL=$(grep -w $CFG $TMP_FILE)
> + NEW_VAL=$(grep -w $CFG $MERGE_FILE)
> + if [ "x$PREV_VAL" != "x$NEW_VAL" ] ; then
> + echo Value of $CFG is redefined by fragment $MERGE_FILE:
> + echo Previous value: $PREV_VAL
> + echo New value: $NEW_VAL
> + echo
> + fi
> + sed -i "/$CFG[ =]/d" $TMP_FILE
> + fi
> + done
> + cat $MERGE_FILE >> $TMP_FILE
> +done
> +
> +if [ "$MAKE" = "false" ]; then
> + cp $TMP_FILE .config
> + echo "#"
> + echo "# merged configuration written to .config (needs make)"
> + echo "#"
> + clean_up
> + exit
> +fi
> +
> +# Use the merged file as the starting point for:
> +# alldefconfig: Fills in any missing symbols with Kconfig default
> +# allnoconfig: Fills in any missing symbols with # CONFIG_* is not set
> +make KCONFIG_ALLCONFIG=$TMP_FILE $ALLTARGET
> +
> +
> +# Check all specified config values took (might have missed-dependency issues)
> +for CFG in $(sed -n "$SED_CONFIG_EXP" $TMP_FILE); do
> +
> + REQUESTED_VAL=$(sed -n "$SED_CONFIG_EXP" $TMP_FILE | grep -w -e "$CFG")
> + ACTUAL_VAL=$(sed -n "$SED_CONFIG_EXP" .config | grep -w -e "$CFG")
> + if [ "x$REQUESTED_VAL" != "x$ACTUAL_VAL" ] ; then
> + echo "Value requested for $CFG not in final .config"
> + echo "Requested value: $REQUESTED_VAL"
> + echo "Actual value: $ACTUAL_VAL"
> + echo ""
> + fi
> +done
> +
> +clean_up
--
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH] kconfig: Add merge_config.sh script
@ 2011-10-04 23:45 John Stultz
0 siblings, 0 replies; 13+ messages in thread
From: John Stultz @ 2011-10-04 23:45 UTC (permalink / raw)
To: LKML
Cc: John Stultz, Sam Ravnborg, gthelen, tartler, Dmitry Fink,
Darren Hart, Eric B Munson, Bruce Ashfield, Michal Marek,
linux-kbuild
After noticing almost every distro has their own method of managing
config fragments, I went looking at some best practices, and wanted
to try to consolidate some of the different approaches so this fairly
simple infrastructure can be shared (and new distros/build systems
don't have to implement yet another config fragment merge script).
This script is most influenced by the Windriver tools used in
the Yocto Project, reusing some portions found there.
This script merges multiple config fragments, warning on any
overrided values. It then sets any unspecified values to their
default, then finally checks to make sure no specified value was
dropped due to unsatisfied dependencies.
I'm sure this implementation won't work for everyone, and I expect
it will need to evolve to adapt for various use cases. But I think
its a reasonable starting point.
v2:
* Reworked to use alldefconfig instead of the proposed
olddefconfig as suggested by Sam Ravnborg.
v3:
* Script improvements from Dmitri.
* allnoconfig option from Darren
* pre-make exit option from Darren
* lots of other fixes/cleanups from Darren.
* Fix final check to not compain about config values in comments
Please let me know if you have any comments or thoughts!
CC: Sam Ravnborg <sam@ravnborg.org>
CC: gthelen@google.com
CC: tartler@cs.fau.de
CC: Dmitry Fink <Dmitry.Fink@palm.com>
CC: Darren Hart <dvhart@linux.intel.com>
CC: Eric B Munson <ebmunson@us.ibm.com>
CC: Bruce Ashfield <Bruce.Ashfield@windriver.com>
CC: Michal Marek <mmarek@suse.cz>
CC: linux-kbuild@vger.kernel.org
Signed-off-by: John Stultz <john.stultz@linaro.org>
---
scripts/kconfig/merge_config.sh | 117 +++++++++++++++++++++++++++++++++++++++
1 files changed, 117 insertions(+), 0 deletions(-)
create mode 100755 scripts/kconfig/merge_config.sh
diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh
new file mode 100755
index 0000000..890276b
--- /dev/null
+++ b/scripts/kconfig/merge_config.sh
@@ -0,0 +1,117 @@
+#!/bin/sh
+# merge_config.sh - Takes a list of config fragment values, and merges
+# them one by one. Provides warnings on overridden values, and specified
+# values that did not make it to the resulting .config file (due to missed
+# dependencies or config symbol removal).
+#
+# Portions reused from kconf_check and generate_cfg:
+# http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/kconf_check
+# http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/generate_cfg
+#
+# Copyright (c) 2009-2010 Wind River Systems, Inc.
+# Copyright 2011 Linaro
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+# See the GNU General Public License for more details.
+
+clean_up() {
+ rm -f $TMP_FILE
+ exit
+}
+trap clean_up SIGHUP SIGINT SIGTERM
+
+usage() {
+ echo "Usage: $0 [OPTIONS] [CONFIG [...]]"
+ echo " -h display this help text"
+ echo " -m only merge the fragments, do not execute the make command"
+ echo " -n use allnoconfig instead of alldefconfig"
+}
+
+MAKE=true
+ALLTARGET=alldefconfig
+
+while true; do
+ case $1 in
+ "-n")
+ ALLTARGET=allnoconfig
+ shift
+ continue
+ ;;
+ "-m")
+ MAKE=false
+ shift
+ continue
+ ;;
+ "-h")
+ usage
+ exit
+ ;;
+ *)
+ break
+ ;;
+ esac
+done
+
+
+
+MERGE_LIST=$*
+SED_CONFIG_EXP="s/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/p"
+TMP_FILE=$(mktemp ./.tmp.config.XXXXXXXXXX)
+
+# Merge files, printing warnings on overrided values
+for MERGE_FILE in $MERGE_LIST ; do
+ echo "Merging $MERGE_FILE"
+ CFG_LIST=$(sed -n "$SED_CONFIG_EXP" $MERGE_FILE)
+
+ for CFG in $CFG_LIST ; do
+ grep -q -w $CFG $TMP_FILE
+ if [ $? -eq 0 ] ; then
+ PREV_VAL=$(grep -w $CFG $TMP_FILE)
+ NEW_VAL=$(grep -w $CFG $MERGE_FILE)
+ if [ "x$PREV_VAL" != "x$NEW_VAL" ] ; then
+ echo Value of $CFG is redefined by fragment $MERGE_FILE:
+ echo Previous value: $PREV_VAL
+ echo New value: $NEW_VAL
+ echo
+ fi
+ sed -i "/$CFG[ =]/d" $TMP_FILE
+ fi
+ done
+ cat $MERGE_FILE >> $TMP_FILE
+done
+
+if [ "$MAKE" = "false" ]; then
+ cp $TMP_FILE .config
+ echo "#"
+ echo "# merged configuration written to .config (needs make)"
+ echo "#"
+ clean_up
+ exit
+fi
+
+# Use the merged file as the starting point for:
+# alldefconfig: Fills in any missing symbols with Kconfig default
+# allnoconfig: Fills in any missing symbols with # CONFIG_* is not set
+make KCONFIG_ALLCONFIG=$TMP_FILE $ALLTARGET
+
+
+# Check all specified config values took (might have missed-dependency issues)
+for CFG in $(sed -n "$SED_CONFIG_EXP" $TMP_FILE); do
+
+ REQUESTED_VAL=$(sed -n "$SED_CONFIG_EXP" $TMP_FILE | grep -w -e "$CFG")
+ ACTUAL_VAL=$(sed -n "$SED_CONFIG_EXP" .config | grep -w -e "$CFG")
+ if [ "x$REQUESTED_VAL" != "x$ACTUAL_VAL" ] ; then
+ echo "Value requested for $CFG not in final .config"
+ echo "Requested value: $REQUESTED_VAL"
+ echo "Actual value: $ACTUAL_VAL"
+ echo ""
+ fi
+done
+
+clean_up
--
1.7.3.2.146.gca209
^ permalink raw reply related [flat|nested] 13+ messages in thread* [PATCH] kconfig: Add merge_config.sh script
@ 2011-09-21 5:22 John Stultz
2011-09-21 6:44 ` Richard Cochran
` (3 more replies)
0 siblings, 4 replies; 13+ messages in thread
From: John Stultz @ 2011-09-21 5:22 UTC (permalink / raw)
To: lkml
Cc: John Stultz, Sam Ravnborg, gthelen, tartler, Dmitry Fink,
Darren Hart, Eric B Munson, Bruce Ashfield, Michal Marek,
linux-kbuild
After noticing almost every distro has their own method of managing
config fragments, I went looking at some best practices, and wanted
to try to consolidate some of the different approaches so this fairly
simple infrastructure can be shared (and new distros/build systems
don't have to implement yet another config fragment merge script).
This script is most influenced by the Windriver tools used in
the Yocto Project, reusing some portions found there.
This script merges multiple config fragments, warning on any
overrided values. It then sets any unspecified values to their
default, then finally checks to make sure no specified value was
dropped due to unsatisfied dependencies.
I'm sure I'm sure this implementation won't work for everyone, and
I expect it will need to evolve to adapt for various use cases.
But I think its a reasonable starting point.
v2: Reworked to use alldefconfig instead of the proposed
olddefconfig as suggested by Sam Ravnborg.
Thanks to Darren Hart for early review and feedback!
Please let me know if you have any comments or thoughts!
CC: Sam Ravnborg <sam@ravnborg.org>
CC: gthelen@google.com
CC: tartler@cs.fau.de
CC: Dmitry Fink <Dmitry.Fink@palm.com>
CC: Darren Hart <dvhart@linux.intel.com>
CC: Eric B Munson <ebmunson@us.ibm.com>
CC: Bruce Ashfield <Bruce.Ashfield@windriver.com>
CC: Michal Marek <mmarek@suse.cz>
CC: linux-kbuild@vger.kernel.org
Signed-off-by: John Stultz <john.stultz@linaro.org>
---
scripts/kconfig/merge_config.sh | 71 +++++++++++++++++++++++++++++++++++++++
1 files changed, 71 insertions(+), 0 deletions(-)
create mode 100755 scripts/kconfig/merge_config.sh
diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh
new file mode 100755
index 0000000..fda0139
--- /dev/null
+++ b/scripts/kconfig/merge_config.sh
@@ -0,0 +1,71 @@
+#!/bin/sh
+# merge_config.sh - Takes a list of config fragment values, and merges
+# them one by one. Provides warnings on overridden values, and specified
+# values that did not make it to the resulting .config file (due to missed
+# dependencies or config symbol removal).
+#
+# Portions reused from kconf_check and generate_cfg:
+# http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/kconf_check
+# http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/generate_cfg
+#
+# Copyright (c) 2009-2010 Wind River Systems, Inc.
+# Copyright 2011 Linaro
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+# See the GNU General Public License for more details.
+
+MERGE_LIST=$*
+
+
+TMP_FILE=$(mktemp ./.tmp.config.XXXXXXXXXX)
+
+# Merge files, printing warnings on overrided values
+for MERGE_FILE in $MERGE_LIST ; do
+ echo "Merging $MERGE_FILE"
+ CFG_LIST=`cat $MERGE_FILE | \
+ sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
+ for CFG in $CFG_LIST ; do
+ grep -q -w $CFG $TMP_FILE
+ if [ $? == 0 ] ; then
+ PREV_VAL=`grep -w $CFG $TMP_FILE`
+ NEW_VAL=`grep -w $CFG $MERGE_FILE`
+ if [ "x$PREV_VAL" != "x$NEW_VAL" ] ; then
+ echo Value of $CFG is redefined by fragment $MERGE_FILE:
+ echo Previous value: `grep -w $CFG $TMP_FILE`
+ echo New value: `grep -w $CFG $MERGE_FILE`
+ echo
+ fi
+ sed -i "/$CFG[ =]/d" $TMP_FILE
+ fi
+ done
+ cat $MERGE_FILE >> $TMP_FILE
+done
+
+
+# Use the merged file as the starting point for alldefconfig
+# (Fills in any missing symbols with Kconfig default)
+make KCONFIG_ALLCONFIG=$TMP_FILE alldefconfig
+
+# Check all specified config values took (might have missed-dependency issues)
+cat $TMP_FILE | while read line; do
+ CFG=`echo $line | \
+ sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
+
+ REQUESTED_VAL=`grep -w -e "$CFG" $TMP_FILE`
+ ACTUAL_VAL=`grep -w -e "$CFG" .config`
+ if [ "x$REQUESTED_VAL" != "x$ACTUAL_VAL" ] ; then
+ echo "Value requested for $CFG not in final .config"
+ echo "Requested value: $REQUESTED_VAL"
+ echo "Actual value: $ACTUAL_VAL"
+ echo ""
+ fi
+done
+
+# Cleanup
+rm $TMP_FILE
--
1.7.3.2.146.gca209
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [PATCH] kconfig: Add merge_config.sh script
2011-09-21 5:22 John Stultz
@ 2011-09-21 6:44 ` Richard Cochran
2011-09-21 15:18 ` John Stultz
2011-09-22 20:05 ` Sam Ravnborg
2011-09-21 12:10 ` Michal Marek
` (2 subsequent siblings)
3 siblings, 2 replies; 13+ messages in thread
From: Richard Cochran @ 2011-09-21 6:44 UTC (permalink / raw)
To: John Stultz
Cc: lkml, Sam Ravnborg, gthelen, tartler, Dmitry Fink, Darren Hart,
Eric B Munson, Bruce Ashfield, Michal Marek, linux-kbuild
On Tue, Sep 20, 2011 at 10:22:41PM -0700, John Stultz wrote:
> After noticing almost every distro has their own method of managing
> config fragments, I went looking at some best practices, and wanted
> to try to consolidate some of the different approaches so this fairly
> simple infrastructure can be shared (and new distros/build systems
> don't have to implement yet another config fragment merge script).
John,
I have been wanting to make or find a way to start with an allnoconfig
and enable a short list of config options (and the options that are
needed to allow them). I don't want the other defaults at all.
Can I use your script for this? If not, is there another way to go
about it?
Thanks,
Richard
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH] kconfig: Add merge_config.sh script
2011-09-21 6:44 ` Richard Cochran
@ 2011-09-21 15:18 ` John Stultz
2011-09-21 15:28 ` Darren Hart
2011-09-22 20:05 ` Sam Ravnborg
1 sibling, 1 reply; 13+ messages in thread
From: John Stultz @ 2011-09-21 15:18 UTC (permalink / raw)
To: Richard Cochran
Cc: lkml, Sam Ravnborg, gthelen, tartler, Dmitry Fink, Darren Hart,
Eric B Munson, Bruce Ashfield, Michal Marek, linux-kbuild
On Wed, 2011-09-21 at 08:44 +0200, Richard Cochran wrote:
> On Tue, Sep 20, 2011 at 10:22:41PM -0700, John Stultz wrote:
> > After noticing almost every distro has their own method of managing
> > config fragments, I went looking at some best practices, and wanted
> > to try to consolidate some of the different approaches so this fairly
> > simple infrastructure can be shared (and new distros/build systems
> > don't have to implement yet another config fragment merge script).
>
> John,
>
> I have been wanting to make or find a way to start with an allnoconfig
> and enable a short list of config options (and the options that are
> needed to allow them). I don't want the other defaults at all.
"make oldnoconfig" provides something close to this, I think.
> Can I use your script for this? If not, is there another way to go
> about it?
I guess my script could take an option to use oldnoconfig instead of the
alldefconfig method.
thanks
-john
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] kconfig: Add merge_config.sh script
2011-09-21 15:18 ` John Stultz
@ 2011-09-21 15:28 ` Darren Hart
0 siblings, 0 replies; 13+ messages in thread
From: Darren Hart @ 2011-09-21 15:28 UTC (permalink / raw)
To: John Stultz
Cc: Richard Cochran, lkml, Sam Ravnborg, gthelen, tartler,
Dmitry Fink, Eric B Munson, Bruce Ashfield, Michal Marek,
linux-kbuild
On 09/21/2011 08:18 AM, John Stultz wrote:
> On Wed, 2011-09-21 at 08:44 +0200, Richard Cochran wrote:
>> On Tue, Sep 20, 2011 at 10:22:41PM -0700, John Stultz wrote:
>>> After noticing almost every distro has their own method of managing
>>> config fragments, I went looking at some best practices, and wanted
>>> to try to consolidate some of the different approaches so this fairly
>>> simple infrastructure can be shared (and new distros/build systems
>>> don't have to implement yet another config fragment merge script).
>>
>> John,
>>
>> I have been wanting to make or find a way to start with an allnoconfig
>> and enable a short list of config options (and the options that are
>> needed to allow them). I don't want the other defaults at all.
>
> "make oldnoconfig" provides something close to this, I think.
>
>> Can I use your script for this? If not, is there another way to go
>> about it?
>
> I guess my script could take an option to use oldnoconfig instead of the
> alldefconfig method.
I had considered suggesting this as well. It would be useful for
assembling a bare minimum configuration.
--
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] kconfig: Add merge_config.sh script
2011-09-21 6:44 ` Richard Cochran
2011-09-21 15:18 ` John Stultz
@ 2011-09-22 20:05 ` Sam Ravnborg
1 sibling, 0 replies; 13+ messages in thread
From: Sam Ravnborg @ 2011-09-22 20:05 UTC (permalink / raw)
To: Richard Cochran
Cc: John Stultz, lkml, gthelen, tartler, Dmitry Fink, Darren Hart,
Eric B Munson, Bruce Ashfield, Michal Marek, linux-kbuild
On Wed, Sep 21, 2011 at 08:44:23AM +0200, Richard Cochran wrote:
> On Tue, Sep 20, 2011 at 10:22:41PM -0700, John Stultz wrote:
> > After noticing almost every distro has their own method of managing
> > config fragments, I went looking at some best practices, and wanted
> > to try to consolidate some of the different approaches so this fairly
> > simple infrastructure can be shared (and new distros/build systems
> > don't have to implement yet another config fragment merge script).
>
> John,
>
> I have been wanting to make or find a way to start with an allnoconfig
> and enable a short list of config options (and the options that are
> needed to allow them). I don't want the other defaults at all.
>From Documentation/kbuild/kconfig.txt:
===================================================================
This 'KCONFIG_ALLCONFIG' file is a config file which contains
(usually a subset of all) preset config symbols. These variable
settings are still subject to normal dependency checks.
Examples:
KCONFIG_ALLCONFIG=custom-notebook.config make allnoconfig
or
KCONFIG_ALLCONFIG=mini.config make allnoconfig
or
make KCONFIG_ALLCONFIG=mini.config allnoconfig
These examples will disable most options (allnoconfig) but enable or
disable the options that are explicitly listed in the specified
mini-config files.
===================================================================
I think this should do what you want.
Sam
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] kconfig: Add merge_config.sh script
2011-09-21 5:22 John Stultz
2011-09-21 6:44 ` Richard Cochran
@ 2011-09-21 12:10 ` Michal Marek
2011-09-21 15:36 ` John Stultz
2011-09-21 21:42 ` Dmitry Fink (Palm GBU)
2011-09-22 1:20 ` Dmitry Fink (Palm GBU)
3 siblings, 1 reply; 13+ messages in thread
From: Michal Marek @ 2011-09-21 12:10 UTC (permalink / raw)
To: John Stultz
Cc: lkml, Sam Ravnborg, gthelen, tartler, Dmitry Fink, Darren Hart,
Eric B Munson, Bruce Ashfield, linux-kbuild
On 21.9.2011 07:22, John Stultz wrote:
> After noticing almost every distro has their own method of managing
> config fragments, I went looking at some best practices, and wanted
> to try to consolidate some of the different approaches so this fairly
> simple infrastructure can be shared (and new distros/build systems
> don't have to implement yet another config fragment merge script).
>
> This script is most influenced by the Windriver tools used in
> the Yocto Project, reusing some portions found there.
>
> This script merges multiple config fragments, warning on any
> overrided values. It then sets any unspecified values to their
> default, then finally checks to make sure no specified value was
> dropped due to unsatisfied dependencies.
>
> I'm sure I'm sure this implementation won't work for everyone, and
> I expect it will need to evolve to adapt for various use cases.
> But I think its a reasonable starting point.
>
> v2: Reworked to use alldefconfig instead of the proposed
> olddefconfig as suggested by Sam Ravnborg.
>
> Thanks to Darren Hart for early review and feedback!
>
> Please let me know if you have any comments or thoughts!
>
> CC: Sam Ravnborg <sam@ravnborg.org>
> CC: gthelen@google.com
> CC: tartler@cs.fau.de
> CC: Dmitry Fink <Dmitry.Fink@palm.com>
> CC: Darren Hart <dvhart@linux.intel.com>
> CC: Eric B Munson <ebmunson@us.ibm.com>
> CC: Bruce Ashfield <Bruce.Ashfield@windriver.com>
> CC: Michal Marek <mmarek@suse.cz>
> CC: linux-kbuild@vger.kernel.org
> Signed-off-by: John Stultz <john.stultz@linaro.org>
> ---
> scripts/kconfig/merge_config.sh | 71 +++++++++++++++++++++++++++++++++++++++
> 1 files changed, 71 insertions(+), 0 deletions(-)
> create mode 100755 scripts/kconfig/merge_config.sh
>
> diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh
> new file mode 100755
> index 0000000..fda0139
> --- /dev/null
> +++ b/scripts/kconfig/merge_config.sh
> @@ -0,0 +1,71 @@
> +#!/bin/sh
> +# merge_config.sh - Takes a list of config fragment values, and merges
> +# them one by one. Provides warnings on overridden values, and specified
> +# values that did not make it to the resulting .config file (due to missed
> +# dependencies or config symbol removal).
> +#
> +# Portions reused from kconf_check and generate_cfg:
> +# http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/kconf_check
> +# http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/generate_cfg
> +#
> +# Copyright (c) 2009-2010 Wind River Systems, Inc.
> +# Copyright 2011 Linaro
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License version 2 as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> +# See the GNU General Public License for more details.
> +
> +MERGE_LIST=$*
> +
> +
> +TMP_FILE=$(mktemp ./.tmp.config.XXXXXXXXXX)
> +
> +# Merge files, printing warnings on overrided values
> +for MERGE_FILE in $MERGE_LIST ; do
> + echo "Merging $MERGE_FILE"
> + CFG_LIST=`cat $MERGE_FILE | \
> + sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
> + for CFG in $CFG_LIST ; do
> + grep -q -w $CFG $TMP_FILE
> + if [ $? == 0 ] ; then
> + PREV_VAL=`grep -w $CFG $TMP_FILE`
> + NEW_VAL=`grep -w $CFG $MERGE_FILE`
> + if [ "x$PREV_VAL" != "x$NEW_VAL" ] ; then
> + echo Value of $CFG is redefined by fragment $MERGE_FILE:
> + echo Previous value: `grep -w $CFG $TMP_FILE`
> + echo New value: `grep -w $CFG $MERGE_FILE`
kconfig already warns when a symbol appears multiple times in .config.
If really wanted, a verbose mode that show the previous and new value
could be added as well.
> + echo
> + fi
> + sed -i "/$CFG[ =]/d" $TMP_FILE
> + fi
> + done
> + cat $MERGE_FILE >> $TMP_FILE
> +done
> +
> +
> +# Use the merged file as the starting point for alldefconfig
> +# (Fills in any missing symbols with Kconfig default)
> +make KCONFIG_ALLCONFIG=$TMP_FILE alldefconfig
> +
> +# Check all specified config values took (might have missed-dependency issues)
> +cat $TMP_FILE | while read line; do
> + CFG=`echo $line | \
> + sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
> +
> + REQUESTED_VAL=`grep -w -e "$CFG" $TMP_FILE`
> + ACTUAL_VAL=`grep -w -e "$CFG" .config`
> + if [ "x$REQUESTED_VAL" != "x$ACTUAL_VAL" ] ; then
> + echo "Value requested for $CFG not in final .config"
> + echo "Requested value: $REQUESTED_VAL"
> + echo "Actual value: $ACTUAL_VAL"
> + echo ""
This requested vs. actual diff would be nice to have in kconfig I think.
The the merge script could be reduced to a simple cat(1) invocation.
Michal
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH] kconfig: Add merge_config.sh script
2011-09-21 12:10 ` Michal Marek
@ 2011-09-21 15:36 ` John Stultz
0 siblings, 0 replies; 13+ messages in thread
From: John Stultz @ 2011-09-21 15:36 UTC (permalink / raw)
To: Michal Marek
Cc: lkml, Sam Ravnborg, gthelen, tartler, Dmitry Fink, Darren Hart,
Eric B Munson, Bruce Ashfield, linux-kbuild
On Wed, 2011-09-21 at 14:10 +0200, Michal Marek wrote:
> On 21.9.2011 07:22, John Stultz wrote:
> > After noticing almost every distro has their own method of managing
> > config fragments, I went looking at some best practices, and wanted
> > to try to consolidate some of the different approaches so this fairly
> > simple infrastructure can be shared (and new distros/build systems
> > don't have to implement yet another config fragment merge script).
> >
> > This script is most influenced by the Windriver tools used in
> > the Yocto Project, reusing some portions found there.
> >
> > This script merges multiple config fragments, warning on any
> > overrided values. It then sets any unspecified values to their
> > default, then finally checks to make sure no specified value was
> > dropped due to unsatisfied dependencies.
> >
> > I'm sure I'm sure this implementation won't work for everyone, and
> > I expect it will need to evolve to adapt for various use cases.
> > But I think its a reasonable starting point.
> >
> > v2: Reworked to use alldefconfig instead of the proposed
> > olddefconfig as suggested by Sam Ravnborg.
> >
> > Thanks to Darren Hart for early review and feedback!
> >
> > Please let me know if you have any comments or thoughts!
> >
> > CC: Sam Ravnborg <sam@ravnborg.org>
> > CC: gthelen@google.com
> > CC: tartler@cs.fau.de
> > CC: Dmitry Fink <Dmitry.Fink@palm.com>
> > CC: Darren Hart <dvhart@linux.intel.com>
> > CC: Eric B Munson <ebmunson@us.ibm.com>
> > CC: Bruce Ashfield <Bruce.Ashfield@windriver.com>
> > CC: Michal Marek <mmarek@suse.cz>
> > CC: linux-kbuild@vger.kernel.org
> > Signed-off-by: John Stultz <john.stultz@linaro.org>
> > ---
> > scripts/kconfig/merge_config.sh | 71 +++++++++++++++++++++++++++++++++++++++
> > 1 files changed, 71 insertions(+), 0 deletions(-)
> > create mode 100755 scripts/kconfig/merge_config.sh
> >
> > diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh
> > new file mode 100755
> > index 0000000..fda0139
> > --- /dev/null
> > +++ b/scripts/kconfig/merge_config.sh
> > @@ -0,0 +1,71 @@
> > +#!/bin/sh
> > +# merge_config.sh - Takes a list of config fragment values, and merges
> > +# them one by one. Provides warnings on overridden values, and specified
> > +# values that did not make it to the resulting .config file (due to missed
> > +# dependencies or config symbol removal).
> > +#
> > +# Portions reused from kconf_check and generate_cfg:
> > +# http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/kconf_check
> > +# http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/generate_cfg
> > +#
> > +# Copyright (c) 2009-2010 Wind River Systems, Inc.
> > +# Copyright 2011 Linaro
> > +#
> > +# This program is free software; you can redistribute it and/or modify
> > +# it under the terms of the GNU General Public License version 2 as
> > +# published by the Free Software Foundation.
> > +#
> > +# This program is distributed in the hope that it will be useful,
> > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> > +# See the GNU General Public License for more details.
> > +
> > +MERGE_LIST=$*
> > +
> > +
> > +TMP_FILE=$(mktemp ./.tmp.config.XXXXXXXXXX)
> > +
> > +# Merge files, printing warnings on overrided values
> > +for MERGE_FILE in $MERGE_LIST ; do
> > + echo "Merging $MERGE_FILE"
> > + CFG_LIST=`cat $MERGE_FILE | \
> > + sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
> > + for CFG in $CFG_LIST ; do
> > + grep -q -w $CFG $TMP_FILE
> > + if [ $? == 0 ] ; then
> > + PREV_VAL=`grep -w $CFG $TMP_FILE`
> > + NEW_VAL=`grep -w $CFG $MERGE_FILE`
> > + if [ "x$PREV_VAL" != "x$NEW_VAL" ] ; then
> > + echo Value of $CFG is redefined by fragment $MERGE_FILE:
> > + echo Previous value: `grep -w $CFG $TMP_FILE`
> > + echo New value: `grep -w $CFG $MERGE_FILE`
>
> kconfig already warns when a symbol appears multiple times in .config.
> If really wanted, a verbose mode that show the previous and new value
> could be added as well.
Indeed. Unfortunately the override messages at evaluation time becomes
less informative since it cannot detect from which of the fragments the
duplication/override came from.
> > +# Use the merged file as the starting point for alldefconfig
> > +# (Fills in any missing symbols with Kconfig default)
> > +make KCONFIG_ALLCONFIG=$TMP_FILE alldefconfig
> > +
> > +# Check all specified config values took (might have missed-dependency issues)
> > +cat $TMP_FILE | while read line; do
> > + CFG=`echo $line | \
> > + sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
> > +
> > + REQUESTED_VAL=`grep -w -e "$CFG" $TMP_FILE`
> > + ACTUAL_VAL=`grep -w -e "$CFG" .config`
> > + if [ "x$REQUESTED_VAL" != "x$ACTUAL_VAL" ] ; then
> > + echo "Value requested for $CFG not in final .config"
> > + echo "Requested value: $REQUESTED_VAL"
> > + echo "Actual value: $ACTUAL_VAL"
> > + echo ""
>
> This requested vs. actual diff would be nice to have in kconfig I think.
> The the merge script could be reduced to a simple cat(1) invocation.
I actually started with a simple cat call, but that has limitations:
Specifically when checking what was requested and what took in
the .config, we don't want to flag explicitly overrided values (for
instance in file A: CONFIG_FOO=y, in file B:# CONFIG_FOO is not set, we
don't want to complain that CONFIG_FOO=y isn't set.) That's why we do
the de-duplication first, and then compare the deduplicated input with
the output.
thanks
-john
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] kconfig: Add merge_config.sh script
2011-09-21 5:22 John Stultz
2011-09-21 6:44 ` Richard Cochran
2011-09-21 12:10 ` Michal Marek
@ 2011-09-21 21:42 ` Dmitry Fink (Palm GBU)
2011-09-22 1:20 ` Dmitry Fink (Palm GBU)
3 siblings, 0 replies; 13+ messages in thread
From: Dmitry Fink (Palm GBU) @ 2011-09-21 21:42 UTC (permalink / raw)
To: John Stultz, lkml
Cc: Sam Ravnborg, gthelen@google.com, tartler@cs.fau.de, Darren Hart,
Eric B Munson, Bruce Ashfield, Michal Marek,
linux-kbuild@vger.kernel.org
Hi John,
I think you may want to apply the following fix:
--- a/scripts/kconfig/merge_config.sh
+++ b/scripts/kconfig/merge_config.sh
@@ -32,7 +32,7 @@ for MERGE_FILE in $MERGE_LIST ; do
sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
for CFG in $CFG_LIST ; do
grep -q -w $CFG $TMP_FILE
- if [ $? == 0 ] ; then
+ if [ $? -eq 0 ] ; then
or just change it single "=" sign, "==" seems to be a bashism and failed
on ubuntu's /bin/sh.
You also have a minor typo in the commit msg:)
"I'm sure I'm sure this implementation won't work for everyone, and"
^ ^
Regards,
Dmitry
On 9/20/11 10:22 PM, "John Stultz" <john.stultz@linaro.org> wrote:
>After noticing almost every distro has their own method of managing
>config fragments, I went looking at some best practices, and wanted
>to try to consolidate some of the different approaches so this fairly
>simple infrastructure can be shared (and new distros/build systems
>don't have to implement yet another config fragment merge script).
>
>This script is most influenced by the Windriver tools used in
>the Yocto Project, reusing some portions found there.
>
>This script merges multiple config fragments, warning on any
>overrided values. It then sets any unspecified values to their
>default, then finally checks to make sure no specified value was
>dropped due to unsatisfied dependencies.
>
>I'm sure I'm sure this implementation won't work for everyone, and
>I expect it will need to evolve to adapt for various use cases.
>But I think its a reasonable starting point.
>
>v2: Reworked to use alldefconfig instead of the proposed
>olddefconfig as suggested by Sam Ravnborg.
>
>Thanks to Darren Hart for early review and feedback!
>
>Please let me know if you have any comments or thoughts!
>
>CC: Sam Ravnborg <sam@ravnborg.org>
>CC: gthelen@google.com
>CC: tartler@cs.fau.de
>CC: Dmitry Fink <Dmitry.Fink@palm.com>
>CC: Darren Hart <dvhart@linux.intel.com>
>CC: Eric B Munson <ebmunson@us.ibm.com>
>CC: Bruce Ashfield <Bruce.Ashfield@windriver.com>
>CC: Michal Marek <mmarek@suse.cz>
>CC: linux-kbuild@vger.kernel.org
>Signed-off-by: John Stultz <john.stultz@linaro.org>
>---
> scripts/kconfig/merge_config.sh | 71
>+++++++++++++++++++++++++++++++++++++++
> 1 files changed, 71 insertions(+), 0 deletions(-)
> create mode 100755 scripts/kconfig/merge_config.sh
>
>diff --git a/scripts/kconfig/merge_config.sh
>b/scripts/kconfig/merge_config.sh
>new file mode 100755
>index 0000000..fda0139
>--- /dev/null
>+++ b/scripts/kconfig/merge_config.sh
>@@ -0,0 +1,71 @@
>+#!/bin/sh
>+# merge_config.sh - Takes a list of config fragment values, and merges
>+# them one by one. Provides warnings on overridden values, and specified
>+# values that did not make it to the resulting .config file (due to
>missed
>+# dependencies or config symbol removal).
>+#
>+# Portions reused from kconf_check and generate_cfg:
>+#
>http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/kc
>onf_check
>+#
>http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/ge
>nerate_cfg
>+#
>+# Copyright (c) 2009-2010 Wind River Systems, Inc.
>+# Copyright 2011 Linaro
>+#
>+# This program is free software; you can redistribute it and/or modify
>+# it under the terms of the GNU General Public License version 2 as
>+# published by the Free Software Foundation.
>+#
>+# This program is distributed in the hope that it will be useful,
>+# but WITHOUT ANY WARRANTY; without even the implied warranty of
>+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
>+# See the GNU General Public License for more details.
>+
>+MERGE_LIST=$*
>+
>+
>+TMP_FILE=$(mktemp ./.tmp.config.XXXXXXXXXX)
>+
>+# Merge files, printing warnings on overrided values
>+for MERGE_FILE in $MERGE_LIST ; do
>+ echo "Merging $MERGE_FILE"
>+ CFG_LIST=`cat $MERGE_FILE | \
>+ sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
>+ for CFG in $CFG_LIST ; do
>+ grep -q -w $CFG $TMP_FILE
>+ if [ $? == 0 ] ; then
>+ PREV_VAL=`grep -w $CFG $TMP_FILE`
>+ NEW_VAL=`grep -w $CFG $MERGE_FILE`
>+ if [ "x$PREV_VAL" != "x$NEW_VAL" ] ; then
>+ echo Value of $CFG is redefined by fragment $MERGE_FILE:
>+ echo Previous value: `grep -w $CFG $TMP_FILE`
>+ echo New value: `grep -w $CFG $MERGE_FILE`
>+ echo
>+ fi
>+ sed -i "/$CFG[ =]/d" $TMP_FILE
>+ fi
>+ done
>+ cat $MERGE_FILE >> $TMP_FILE
>+done
>+
>+
>+# Use the merged file as the starting point for alldefconfig
>+# (Fills in any missing symbols with Kconfig default)
>+make KCONFIG_ALLCONFIG=$TMP_FILE alldefconfig
>+
>+# Check all specified config values took (might have missed-dependency
>issues)
>+cat $TMP_FILE | while read line; do
>+ CFG=`echo $line | \
>+ sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
>+
>+ REQUESTED_VAL=`grep -w -e "$CFG" $TMP_FILE`
>+ ACTUAL_VAL=`grep -w -e "$CFG" .config`
>+ if [ "x$REQUESTED_VAL" != "x$ACTUAL_VAL" ] ; then
>+ echo "Value requested for $CFG not in final .config"
>+ echo "Requested value: $REQUESTED_VAL"
>+ echo "Actual value: $ACTUAL_VAL"
>+ echo ""
>+ fi
>+done
>+
>+# Cleanup
>+rm $TMP_FILE
>--
>1.7.3.2.146.gca209
>
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH] kconfig: Add merge_config.sh script
2011-09-21 5:22 John Stultz
` (2 preceding siblings ...)
2011-09-21 21:42 ` Dmitry Fink (Palm GBU)
@ 2011-09-22 1:20 ` Dmitry Fink (Palm GBU)
2011-09-22 16:18 ` Arnaud Lacombe
3 siblings, 1 reply; 13+ messages in thread
From: Dmitry Fink (Palm GBU) @ 2011-09-22 1:20 UTC (permalink / raw)
To: John Stultz, lkml
Cc: Sam Ravnborg, gthelen@google.com, tartler@cs.fau.de, Darren Hart,
Eric B Munson, Bruce Ashfield, Michal Marek,
linux-kbuild@vger.kernel.org
Hi, John
Here is another suggestion:
--- a/scripts/kconfig/merge_config.sh
+++ b/scripts/kconfig/merge_config.sh
@@ -28,11 +28,11 @@ TMP_FILE=$(mktemp ./.tmp.config.XXXXXXXXXX)
# Merge files, printing warnings on overrided values
for MERGE_FILE in $MERGE_LIST ; do
echo "Merging $MERGE_FILE"
- CFG_LIST=`cat $MERGE_FILE | \
+ CFG_LIST=`cat $MERGE_FILE | grep -e "CONFIG_[a-zA-Z0-9_]*" | \
sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
for CFG in $CFG_LIST ; do
grep -q -w $CFG $TMP_FILE
@@ -53,7 +53,7 @@ done
make KCONFIG_ALLCONFIG=$TMP_FILE alldefconfig
# Check all specified config values took (might have missed-dependency
issues)
-cat $TMP_FILE | while read line; do
+cat $TMP_FILE | grep -e "CONFIG_[a-zA-Z0-9_]*" | while read line; do
CFG=`echo $line | \
sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
The original script chokes on fragments that include empty lines or section
comments (and think we do want them in fragments). There might be a nicer
way to
handle it, this is the quickest fix I found, feel free to consider a
different
approach.
Dmitry
On 9/20/11 10:22 PM, "John Stultz" <john.stultz@linaro.org> wrote:
>After noticing almost every distro has their own method of managing
>config fragments, I went looking at some best practices, and wanted
>to try to consolidate some of the different approaches so this fairly
>simple infrastructure can be shared (and new distros/build systems
>don't have to implement yet another config fragment merge script).
>
>This script is most influenced by the Windriver tools used in
>the Yocto Project, reusing some portions found there.
>
>This script merges multiple config fragments, warning on any
>overrided values. It then sets any unspecified values to their
>default, then finally checks to make sure no specified value was
>dropped due to unsatisfied dependencies.
>
>I'm sure I'm sure this implementation won't work for everyone, and
>I expect it will need to evolve to adapt for various use cases.
>But I think its a reasonable starting point.
>
>v2: Reworked to use alldefconfig instead of the proposed
>olddefconfig as suggested by Sam Ravnborg.
>
>Thanks to Darren Hart for early review and feedback!
>
>Please let me know if you have any comments or thoughts!
>
>CC: Sam Ravnborg <sam@ravnborg.org>
>CC: gthelen@google.com
>CC: tartler@cs.fau.de
>CC: Dmitry Fink <Dmitry.Fink@palm.com>
>CC: Darren Hart <dvhart@linux.intel.com>
>CC: Eric B Munson <ebmunson@us.ibm.com>
>CC: Bruce Ashfield <Bruce.Ashfield@windriver.com>
>CC: Michal Marek <mmarek@suse.cz>
>CC: linux-kbuild@vger.kernel.org
>Signed-off-by: John Stultz <john.stultz@linaro.org>
>---
> scripts/kconfig/merge_config.sh | 71
>+++++++++++++++++++++++++++++++++++++++
> 1 files changed, 71 insertions(+), 0 deletions(-)
> create mode 100755 scripts/kconfig/merge_config.sh
>
>diff --git a/scripts/kconfig/merge_config.sh
>b/scripts/kconfig/merge_config.sh
>new file mode 100755
>index 0000000..fda0139
>--- /dev/null
>+++ b/scripts/kconfig/merge_config.sh
>@@ -0,0 +1,71 @@
>+#!/bin/sh
>+# merge_config.sh - Takes a list of config fragment values, and merges
>+# them one by one. Provides warnings on overridden values, and specified
>+# values that did not make it to the resulting .config file (due to
>missed
>+# dependencies or config symbol removal).
>+#
>+# Portions reused from kconf_check and generate_cfg:
>+#
>http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/kc
>onf_check
>+#
>http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/ge
>nerate_cfg
>+#
>+# Copyright (c) 2009-2010 Wind River Systems, Inc.
>+# Copyright 2011 Linaro
>+#
>+# This program is free software; you can redistribute it and/or modify
>+# it under the terms of the GNU General Public License version 2 as
>+# published by the Free Software Foundation.
>+#
>+# This program is distributed in the hope that it will be useful,
>+# but WITHOUT ANY WARRANTY; without even the implied warranty of
>+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
>+# See the GNU General Public License for more details.
>+
>+MERGE_LIST=$*
>+
>+
>+TMP_FILE=$(mktemp ./.tmp.config.XXXXXXXXXX)
>+
>+# Merge files, printing warnings on overrided values
>+for MERGE_FILE in $MERGE_LIST ; do
>+ echo "Merging $MERGE_FILE"
>+ CFG_LIST=`cat $MERGE_FILE | \
>+ sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
>+ for CFG in $CFG_LIST ; do
>+ grep -q -w $CFG $TMP_FILE
>+ if [ $? == 0 ] ; then
>+ PREV_VAL=`grep -w $CFG $TMP_FILE`
>+ NEW_VAL=`grep -w $CFG $MERGE_FILE`
>+ if [ "x$PREV_VAL" != "x$NEW_VAL" ] ; then
>+ echo Value of $CFG is redefined by fragment $MERGE_FILE:
>+ echo Previous value: `grep -w $CFG $TMP_FILE`
>+ echo New value: `grep -w $CFG $MERGE_FILE`
>+ echo
>+ fi
>+ sed -i "/$CFG[ =]/d" $TMP_FILE
>+ fi
>+ done
>+ cat $MERGE_FILE >> $TMP_FILE
>+done
>+
>+
>+# Use the merged file as the starting point for alldefconfig
>+# (Fills in any missing symbols with Kconfig default)
>+make KCONFIG_ALLCONFIG=$TMP_FILE alldefconfig
>+
>+# Check all specified config values took (might have missed-dependency
>issues)
>+cat $TMP_FILE | while read line; do
>+ CFG=`echo $line | \
>+ sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
>+
>+ REQUESTED_VAL=`grep -w -e "$CFG" $TMP_FILE`
>+ ACTUAL_VAL=`grep -w -e "$CFG" .config`
>+ if [ "x$REQUESTED_VAL" != "x$ACTUAL_VAL" ] ; then
>+ echo "Value requested for $CFG not in final .config"
>+ echo "Requested value: $REQUESTED_VAL"
>+ echo "Actual value: $ACTUAL_VAL"
>+ echo ""
>+ fi
>+done
>+
>+# Cleanup
>+rm $TMP_FILE
>--
>1.7.3.2.146.gca209
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH] kconfig: Add merge_config.sh script
2011-09-22 1:20 ` Dmitry Fink (Palm GBU)
@ 2011-09-22 16:18 ` Arnaud Lacombe
0 siblings, 0 replies; 13+ messages in thread
From: Arnaud Lacombe @ 2011-09-22 16:18 UTC (permalink / raw)
To: Dmitry Fink (Palm GBU)
Cc: John Stultz, lkml, Sam Ravnborg, gthelen@google.com,
tartler@cs.fau.de, Darren Hart, Eric B Munson, Bruce Ashfield,
Michal Marek, linux-kbuild@vger.kernel.org
Hi,
On Wed, Sep 21, 2011 at 9:20 PM, Dmitry Fink (Palm GBU)
<Dmitry.Fink@palm.com> wrote:
> Hi, John
> Here is another suggestion:
>
> --- a/scripts/kconfig/merge_config.sh
> +++ b/scripts/kconfig/merge_config.sh
> @@ -28,11 +28,11 @@ TMP_FILE=$(mktemp ./.tmp.config.XXXXXXXXXX)
> # Merge files, printing warnings on overrided values
> for MERGE_FILE in $MERGE_LIST ; do
> echo "Merging $MERGE_FILE"
> - CFG_LIST=`cat $MERGE_FILE | \
> + CFG_LIST=`cat $MERGE_FILE | grep -e "CONFIG_[a-zA-Z0-9_]*" | \
> sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
3 process, when only 1 is really needed, the following:
CFG_LIST=`sed '/CONFIG_[a-zA-Z0-9_]*/!d; s/^\(#
\)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/' $MERGE_FILE`
should do the job; however, it's completely untested.
- Arnaud
> for CFG in $CFG_LIST ; do
> grep -q -w $CFG $TMP_FILE
> @@ -53,7 +53,7 @@ done
> make KCONFIG_ALLCONFIG=$TMP_FILE alldefconfig
>
> # Check all specified config values took (might have missed-dependency
> issues)
> -cat $TMP_FILE | while read line; do
> +cat $TMP_FILE | grep -e "CONFIG_[a-zA-Z0-9_]*" | while read line; do
> CFG=`echo $line | \
> sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
>
> The original script chokes on fragments that include empty lines or section
> comments (and think we do want them in fragments). There might be a nicer
> way to
> handle it, this is the quickest fix I found, feel free to consider a
> different
> approach.
>
> Dmitry
>
>
> On 9/20/11 10:22 PM, "John Stultz" <john.stultz@linaro.org> wrote:
>
>>After noticing almost every distro has their own method of managing
>>config fragments, I went looking at some best practices, and wanted
>>to try to consolidate some of the different approaches so this fairly
>>simple infrastructure can be shared (and new distros/build systems
>>don't have to implement yet another config fragment merge script).
>>
>>This script is most influenced by the Windriver tools used in
>>the Yocto Project, reusing some portions found there.
>>
>>This script merges multiple config fragments, warning on any
>>overrided values. It then sets any unspecified values to their
>>default, then finally checks to make sure no specified value was
>>dropped due to unsatisfied dependencies.
>>
>>I'm sure I'm sure this implementation won't work for everyone, and
>>I expect it will need to evolve to adapt for various use cases.
>>But I think its a reasonable starting point.
>>
>>v2: Reworked to use alldefconfig instead of the proposed
>>olddefconfig as suggested by Sam Ravnborg.
>>
>>Thanks to Darren Hart for early review and feedback!
>>
>>Please let me know if you have any comments or thoughts!
>>
>>CC: Sam Ravnborg <sam@ravnborg.org>
>>CC: gthelen@google.com
>>CC: tartler@cs.fau.de
>>CC: Dmitry Fink <Dmitry.Fink@palm.com>
>>CC: Darren Hart <dvhart@linux.intel.com>
>>CC: Eric B Munson <ebmunson@us.ibm.com>
>>CC: Bruce Ashfield <Bruce.Ashfield@windriver.com>
>>CC: Michal Marek <mmarek@suse.cz>
>>CC: linux-kbuild@vger.kernel.org
>>Signed-off-by: John Stultz <john.stultz@linaro.org>
>>---
>> scripts/kconfig/merge_config.sh | 71
>>+++++++++++++++++++++++++++++++++++++++
>> 1 files changed, 71 insertions(+), 0 deletions(-)
>> create mode 100755 scripts/kconfig/merge_config.sh
>>
>>diff --git a/scripts/kconfig/merge_config.sh
>>b/scripts/kconfig/merge_config.sh
>>new file mode 100755
>>index 0000000..fda0139
>>--- /dev/null
>>+++ b/scripts/kconfig/merge_config.sh
>>@@ -0,0 +1,71 @@
>>+#!/bin/sh
>>+# merge_config.sh - Takes a list of config fragment values, and merges
>>+# them one by one. Provides warnings on overridden values, and specified
>>+# values that did not make it to the resulting .config file (due to
>>missed
>>+# dependencies or config symbol removal).
>>+#
>>+# Portions reused from kconf_check and generate_cfg:
>>+#
>>http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/kc
>>onf_check
>>+#
>>http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/ge
>>nerate_cfg
>>+#
>>+# Copyright (c) 2009-2010 Wind River Systems, Inc.
>>+# Copyright 2011 Linaro
>>+#
>>+# This program is free software; you can redistribute it and/or modify
>>+# it under the terms of the GNU General Public License version 2 as
>>+# published by the Free Software Foundation.
>>+#
>>+# This program is distributed in the hope that it will be useful,
>>+# but WITHOUT ANY WARRANTY; without even the implied warranty of
>>+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
>>+# See the GNU General Public License for more details.
>>+
>>+MERGE_LIST=$*
>>+
>>+
>>+TMP_FILE=$(mktemp ./.tmp.config.XXXXXXXXXX)
>>+
>>+# Merge files, printing warnings on overrided values
>>+for MERGE_FILE in $MERGE_LIST ; do
>>+ echo "Merging $MERGE_FILE"
>>+ CFG_LIST=`cat $MERGE_FILE | \
>>+ sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
>>+ for CFG in $CFG_LIST ; do
>>+ grep -q -w $CFG $TMP_FILE
>>+ if [ $? == 0 ] ; then
>>+ PREV_VAL=`grep -w $CFG $TMP_FILE`
>>+ NEW_VAL=`grep -w $CFG $MERGE_FILE`
>>+ if [ "x$PREV_VAL" != "x$NEW_VAL" ] ; then
>>+ echo Value of $CFG is redefined by fragment $MERGE_FILE:
>>+ echo Previous value: `grep -w $CFG $TMP_FILE`
>>+ echo New value: `grep -w $CFG $MERGE_FILE`
>>+ echo
>>+ fi
>>+ sed -i "/$CFG[ =]/d" $TMP_FILE
>>+ fi
>>+ done
>>+ cat $MERGE_FILE >> $TMP_FILE
>>+done
>>+
>>+
>>+# Use the merged file as the starting point for alldefconfig
>>+# (Fills in any missing symbols with Kconfig default)
>>+make KCONFIG_ALLCONFIG=$TMP_FILE alldefconfig
>>+
>>+# Check all specified config values took (might have missed-dependency
>>issues)
>>+cat $TMP_FILE | while read line; do
>>+ CFG=`echo $line | \
>>+ sed 's/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/'`
>>+
>>+ REQUESTED_VAL=`grep -w -e "$CFG" $TMP_FILE`
>>+ ACTUAL_VAL=`grep -w -e "$CFG" .config`
>>+ if [ "x$REQUESTED_VAL" != "x$ACTUAL_VAL" ] ; then
>>+ echo "Value requested for $CFG not in final .config"
>>+ echo "Requested value: $REQUESTED_VAL"
>>+ echo "Actual value: $ACTUAL_VAL"
>>+ echo ""
>>+ fi
>>+done
>>+
>>+# Cleanup
>>+rm $TMP_FILE
>>--
>>1.7.3.2.146.gca209
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2011-10-24 23:05 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-10-24 22:48 [PATCH] kconfig: Add merge_config.sh script John Stultz
2011-10-24 23:05 ` Darren Hart
-- strict thread matches above, loose matches on Subject: below --
2011-10-04 23:45 John Stultz
2011-09-21 5:22 John Stultz
2011-09-21 6:44 ` Richard Cochran
2011-09-21 15:18 ` John Stultz
2011-09-21 15:28 ` Darren Hart
2011-09-22 20:05 ` Sam Ravnborg
2011-09-21 12:10 ` Michal Marek
2011-09-21 15:36 ` John Stultz
2011-09-21 21:42 ` Dmitry Fink (Palm GBU)
2011-09-22 1:20 ` Dmitry Fink (Palm GBU)
2011-09-22 16:18 ` Arnaud Lacombe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).