Re: [PATCH] Miniconfig revisited (2/3)

Re: [PATCH] Miniconfig revisited (2/3)

[Rob Landley]

[-- Attachment #1: Type: text/plain, Size: 1540 bytes --]

On Saturday 08 July 2006 12:21 pm, Horst von Brand wrote:
> > +LENGTH=$(cat .config | wc -l)
>
> Why the cat(1)? "wc -l < .config" is the same

I believe it was originally a longer pipeline, and I can't wc the file 
directly or it'll spit out the filename, but < is indeed one less process 
spawned per loop.

> > +# Loop through all lines in the file
> > +I=1
> > +while true
> > +do
> > +  if [ $I -gt $LENGTH ]
> > +  then
> > +    break
> > +  fi
>
> Could do it with:
>
>   for I in $(seq 1 $LENGTH); do
>     ...
>   done

Makes a 1/10th of a second difference to the final runtime, but if you prefer 
that...

> or just plain read the lines
>
> > +
> > +  echo -n -e "\r"$I/$LENGTH lines $(cat "$OUTPUT" | wc -c) bytes
>
> Again, unnecessary cat(1).

Yup, a valid cleanup.

The thing is, the script's going to be dog slow no matter what I do, and most 
of the slowness isn't in the script, it's the config infrastructure 
re-parsing the config file 1500 times or so for defconfig.  (The script's 
about 3% of the runtime overhead, and the two largest chunks of that are 
printing out the progress indicator and the sed invocation right above the 
make.)  So I didn't put that much effort into optimizing it.

Someday I'd like to go in and make kconfig spit out a miniconfig directly, but 
kconfig's not really set up to do that, and touching Roman Zippel's code 
would give him the opportunity to say no again.

Anyway, here's a version with your fixes.  Thanks for the review,

Rob
-- 
Never bet against the cheap plastic solution.

[-- Attachment #2: shrinkconfig2.patch --]
[-- Type: text/x-diff, Size: 2038 bytes --]

--- linux-2.6.17.1/scripts/shrinkconfig	2006-07-09 12:40:38.000000000 -0400
+++ linux-2.6.17.new/scripts/shrinkconfig	2006-07-09 12:12:32.000000000 -0400
@@ -0,0 +1,79 @@
+#!/bin/bash
+
+# shrinkconfig copyright 2006 by Rob Landley <rob@landley.net>
+# Licensed under the GNU General Public License version 2.
+
+if [ $# -ne 1 ]
+then
+  echo "Turns current .config into a miniconfig file."
+  echo "Usage: shrinkconfig mini.config"
+  exit 1
+fi
+
+if [ ! -f .config ]
+then
+  echo "Need a .config file to shrink."
+  exit 1
+fi
+LENGTH=$(wc -l < .config)
+
+OUTPUT="$1"
+cp .config "$OUTPUT"
+if [ $? -ne 0 ]
+then
+  echo "Couldn't create $OUTPUT"
+  exit 1
+fi
+
+# If we get interrupted, clean up the mess
+
+KERNELOUTPUT=""
+
+function cleanup
+{
+  echo
+  echo "Interrupted."
+  [ ! -z "$KERNELOUTPUT" ] && rm -rf "$KERNELOUTPUT"
+  rm "$OUTPUT"
+  exit 1
+}
+
+trap cleanup HUP INT QUIT TERM
+
+# Since the "O=" argument to make doesn't work recursively, we need to jump
+# through a few hoops to avoid overwriting the .config that we're shrinking.
+
+# If we're building out of tree, we'll have absolute paths to source and build
+# directories in the Makefile.
+
+KERNELSRC=$(sed -n -e 's/KERNELSRC[^/]*:=[^/]*//p' Makefile)
+[ -z "$KERNELSRC" ] && KERNELSRC=$(pwd)
+KERNELOUTPUT=`pwd`/.config.minitemp
+
+mkdir -p "$KERNELOUTPUT" || exit 1
+
+echo "Shrinking .config to $OUTPUT..."
+
+for I in $(seq 1 $LENGTH)
+do
+  echo -n -e "\r"$I/$LENGTH lines $(wc -c < "$OUTPUT") bytes
+
+  sed -n "${I}!p" "$OUTPUT" > "$KERNELOUTPUT"/.config.test
+  # Do a config with this file
+  make -C "$KERNELSRC" O="$KERNELOUTPUT" allnoconfig KCONFIG_ALLCONFIG="$KERNELOUTPUT"/.config.test > /dev/null
+
+  # Compare.  The date changes, so expect a small difference each time.
+  D=$(diff "$KERNELOUTPUT"/.config .config | wc -l)
+  if [ $D -eq 4 ]
+  then
+    mv "$KERNELOUTPUT"/.config.test "$OUTPUT"
+    LENGTH=$[$LENGTH-1]
+  else
+    I=$[$I + 1]
+  fi
+done
+
+rm -rf "$KERNELOUTPUT"
+
+# One extra echo to preserve status line.
+echo

Re: [PATCH] Miniconfig revisited (2/3)

[Horst von Brand]

Rob Landley <rob@landley.net> wrote:
> On Saturday 08 July 2006 12:21 pm, Horst von Brand wrote:

[...]

> > > +# Loop through all lines in the file
> > > +I=1
> > > +while true
> > > +do
> > > +  if [ $I -gt $LENGTH ]
> > > +  then
> > > +    break
> > > +  fi
> >
> > Could do it with:
> >
> >   for I in $(seq 1 $LENGTH); do
> >     ...
> >   done

> Makes a 1/10th of a second difference to the final runtime, but if you
> prefer that...

I prefer it for readability.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

[PATCH] Miniconfig revisited (0/3)

[Rob Landley]

Ok, here's a rewrite of the mini.config patch that's just a makefile tweak to 
add the "make miniconfig" target, the miniconfig generator script, and some 
documentation.  No C code is touched by this patch series.

At some point I'd like to replace the guts of scripts/shrinkconfig with 
internal changes to kconfig that spit out a mini.config directly.  The script 
shrinks the current defconfig from 1584 lines to 116, but it takes 28 minutes 
to do so on my laptop.  However, in the past six months I haven't gotten 
around to tackling that and it won't creep up my to-do list sitting 
unsubmitted on my hard drive, so...

Rob
-- 
Never bet against the cheap plastic solution.

[PATCH] Miniconfig revisited (2/3)

[Rob Landley]

Add scripts/shrinkconfig to create a miniconfig from an existing .config.

Signed-off-by: Rob Landley <rob@landley.net>

diff -ur linux-2.6.17.1/scripts/shrinkconfig linux-2.6.17.new/scripts/shrinkconfig
--- linux-2.6.17.1/scripts/shrinkconfig	2006-07-06 16:34:39.000000000 -0400
+++ linux-2.6.17.new/scripts/shrinkconfig	2006-07-06 15:54:40.000000000 -0400
@@ -0,0 +1,86 @@
+#!/bin/bash
+
+# shrinkconfig copyright 2006 by Rob Landley <rob@landley.net>
+# Licensed under the GNU General Public License version 2.
+
+if [ $# -ne 1 ]
+then
+  echo "Turns current .config into a miniconfig file."
+  echo "Usage: shrinkconfig mini.config"
+  exit 1
+fi
+
+if [ ! -f .config ]
+then
+  echo "Need a .config file to shrink."
+  exit 1
+fi
+LENGTH=$(cat .config | wc -l)
+
+OUTPUT="$1"
+cp .config "$OUTPUT"
+if [ $? -ne 0 ]
+then
+  echo "Couldn't create $OUTPUT"
+  exit 1
+fi
+
+# If we get interrupted, clean up the mess
+
+KERNELOUTPUT=""
+
+function cleanup
+{
+  echo
+  echo "Interrupted."
+  [ ! -z "$KERNELOUTPUT" ] && rm -rf "$KERNELOUTPUT"
+  rm "$OUTPUT"
+  exit 1
+}
+
+trap cleanup HUP INT QUIT TERM
+
+# Since the "O=" argument to make doesn't work recursively, we need to jump
+# through a few hoops to avoid overwriting the .config that we're shrinking.
+
+# If we're building out of tree, we'll have absolute paths to source and build
+# directories in the Makefile.
+
+KERNELSRC=$(sed -n -e 's/KERNELSRC[^/]*:=[^/]*//p' Makefile)
+[ -z "$KERNELSRC" ] && KERNELSRC=$(pwd)
+KERNELOUTPUT=`pwd`/.config.minitemp
+
+mkdir -p "$KERNELOUTPUT" || exit 1
+
+echo "Shrinking .config to $OUTPUT..."
+
+# Loop through all lines in the file 
+I=1
+while true
+do
+  if [ $I -gt $LENGTH ]
+  then
+    break
+  fi
+
+  echo -n -e "\r"$I/$LENGTH lines $(cat "$OUTPUT" | wc -c) bytes
+
+  sed -n "${I}!p" "$OUTPUT" > "$KERNELOUTPUT"/.config.test
+  # Do a config with this file
+  make -C "$KERNELSRC" O="$KERNELOUTPUT" allnoconfig KCONFIG_ALLCONFIG="$KERNELOUTPUT"/.config.test > /dev/null
+
+  # Compare.  The date changes, so expect a small difference each time.
+  D=$(diff "$KERNELOUTPUT"/.config .config | wc -l)
+  if [ $D -eq 4 ]
+  then
+    mv "$KERNELOUTPUT"/.config.test "$OUTPUT"
+    LENGTH=$[$LENGTH-1]
+  else
+    I=$[$I + 1]
+  fi
+done
+
+rm -rf "$KERNELOUTPUT"
+
+# One extra echo to preserve status line.
+echo

-- 
Never bet against the cheap plastic solution.