Re: [PATCH] kconfig: simplification of scripts/extract-ikconfig

Re: [PATCH] kconfig: simplification of scripts/extract-ikconfig

[Amerigo Wang]

On Mon, Jun 29, 2009 at 01:36:13PM +0200, Dick Streefland wrote:
>On Monday 2009-06-29 10:51, Amerigo Wang wrote:
>| Does this work?
>| My quick test shows no...
>| 
>| I still can't get the right offset number with this trick.
>| 
>| P.S. My grep is 2.5.1.
>
>I've tested the script with grep 2.5.1, and it works for me. Can you be
>more specific about what you think is wrong with the offsets? You need
>to be aware of the fact that because "tr" replaces the first character
>of the pattern by a newline, an offset will be 1 higher than the start
>of the original pattern. The resulting offsets are therefore 1-based
>instead of 0-based, which is exactly what "tail" expects for the "-c"
>option.

Hi,

I appologize for my delay, I totally missed this thread, sorry.

I just tested it, it works! So,

Tested-by: WANG Cong <xiyou.wangcong@gmail.com>


But, I still don't understand the behavior of 'grep -abo',
please try the following commands:

echo -e '\x01\x02\x03\x04' > test.bin
od -x test.bin
tr $'\x01\x02\x03\x04' '1234' < test.bin | grep -abo '4'


Am I missing something here??

Thanks.

Re: [PATCH] kconfig: simplification of scripts/extract-ikconfig

[Dick Streefland]

On Thursday 2009-07-09 10:38, Amerigo Wang wrote:
| I appologize for my delay, I totally missed this thread, sorry.
| 
| I just tested it, it works! So,
| 
| Tested-by: WANG Cong <xiyou.wangcong@gmail.com>

Great, thanks.

| But, I still don't understand the behavior of 'grep -abo',
| please try the following commands:
| 
| echo -e '\x01\x02\x03\x04' > test.bin
| od -x test.bin
| tr $'\x01\x02\x03\x04' '1234' < test.bin | grep -abo '4'
| 
| Am I missing something here??

The output with grep-2.5.3 is 3:4, which means that the string "4" was
found at offset 3 (0-based). With grep-2.5.1, you get 0:4, which means
that the line containing the string "4" has offset 0. The "tr" commands
in the script ensure that the patterns we are looking for are always at
the start of a line, so that it doesn't matter which version of grep is
installed.

-- 
Dick

Re: [PATCH] kconfig: simplification of scripts/extract-ikconfig

[Amerigo Wang]

On Thu, Jul 09, 2009 at 10:26:51AM +0200, Dick Streefland wrote:
>On Thursday 2009-07-09 10:38, Amerigo Wang wrote:
>| I appologize for my delay, I totally missed this thread, sorry.
>| 
>| I just tested it, it works! So,
>| 
>| Tested-by: WANG Cong <xiyou.wangcong@gmail.com>
>
>Great, thanks.
>
>| But, I still don't understand the behavior of 'grep -abo',
>| please try the following commands:
>| 
>| echo -e '\x01\x02\x03\x04' > test.bin
>| od -x test.bin
>| tr $'\x01\x02\x03\x04' '1234' < test.bin | grep -abo '4'
>| 
>| Am I missing something here??
>
>The output with grep-2.5.3 is 3:4, which means that the string "4" was
>found at offset 3 (0-based). With grep-2.5.1, you get 0:4, which means
>that the line containing the string "4" has offset 0. The "tr" commands
>in the script ensure that the patterns we are looking for are always at
>the start of a line, so that it doesn't matter which version of grep is
>installed.

Hmm, I see, it is really a _big_ change. :)

Have you checked other version of grep? I mean lower version, of course,
does your script work with them too?

Thank you!

Re: [PATCH] kconfig: simplification of scripts/extract-ikconfig

[Dick Streefland]

On Friday 2009-07-10 15:55, Amerigo Wang wrote:
| Have you checked other version of grep? I mean lower version, of course,
| does your script work with them too?

The script works with all grep versions since 2.5, released 2002-03-13,
where the -o option was added. The config extraction feature was
introduced in Linux 2.6.8, released 2004-08-14, so I don't think this
dependency will be a problem.

-- 
Dick

Re: [PATCH] kconfig: simplification of scripts/extract-ikconfig

[Sam Ravnborg]

On Fri, Jul 10, 2009 at 11:54:32AM +0200, Dick Streefland wrote:
> On Friday 2009-07-10 15:55, Amerigo Wang wrote:
> | Have you checked other version of grep? I mean lower version, of course,
> | does your script work with them too?
> 
> The script works with all grep versions since 2.5, released 2002-03-13,
> where the -o option was added. The config extraction feature was
> introduced in Linux 2.6.8, released 2004-08-14, so I don't think this
> dependency will be a problem.

Dick - I lost the original patch.
Can you please resend.

In the changelog please include the information that
you tested this with relevant grep versions.

	Sam

Re: [PATCH] kconfig: simplification of scripts/extract-ikconfig

[Dick Streefland]

On Friday 2009-07-17 23:17, Sam Ravnborg wrote:
| Dick - I lost the original patch.
| Can you please resend.
| 
| In the changelog please include the information that
| you tested this with relevant grep versions.

Sure, here it is:

======================================================================

I've rewritten the extract-ikconfig script to extract the kernel
configuration from a kernel compiled with CONFIG_IKCONFIG. The main
motivation for the rewrite was to remove the dependency on the
external C program binoffset.c, which is compiled on the initial run.

The binoffset executable is invoked with a relative path, which means
that the old script can only be run from the top of the kernel tree,
and only when you have write permission in the scripts directory.
The new script uses tr/grep/tail/zcat only, and can be invoked from
anywhere. The binoffset.c program has been removed. This script
requires GNU grep 2.5 (released 2002-03-13) or higher, because the -o
option was introduced in that version.

Signed-off-by: Dick Streefland <dick@streefland.net>
---

 Documentation/dontdiff   |    1 -
 scripts/.gitignore       |    1 -
 scripts/binoffset.c      |  163 ----------------------------------------------
 scripts/extract-ikconfig |  113 ++++++++++++-------------------
 4 files changed, 44 insertions(+), 234 deletions(-)

diff --git a/Documentation/dontdiff b/Documentation/dontdiff
index 88519da..6fef118 100644
--- a/Documentation/dontdiff
+++ b/Documentation/dontdiff
@@ -68,7 +68,6 @@ autoconf.h*
 bbootsect
 bin2c
 binkernel.spec
-binoffset
 bootsect
 bounds.h
 bsetup
diff --git a/scripts/.gitignore b/scripts/.gitignore
index 52cab46..c5d5db5 100644
--- a/scripts/.gitignore
+++ b/scripts/.gitignore
@@ -6,5 +6,4 @@ kallsyms
 pnmtologo
 bin2c
 unifdef
-binoffset
 ihex2fw
diff --git a/scripts/binoffset.c b/scripts/binoffset.c
deleted file mode 100644
index 1a2e39b..0000000
--- a/scripts/binoffset.c
+++ /dev/null
@@ -1,163 +0,0 @@
-/***************************************************************************
- * binoffset.c
- * (C) 2002 Randy Dunlap <rdunlap@xenotime.net>
-
-#   This program is free software; you can redistribute it and/or modify
-#   it under the terms of the GNU General Public License as published by
-#   the Free Software Foundation; either version 2 of the License, or
-#   (at your option) any later version.
-#
-#   This program is distributed in the hope that it will be useful,
-#   but WITHOUT ANY WARRANTY; without even the implied warranty of
-#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-#   GNU General Public License for more details.
-#
-#   You should have received a copy of the GNU General Public License
-#   along with this program; if not, write to the Free Software
-#   Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
-
-# binoffset.c:
-# - searches a (binary) file for a specified (binary) pattern
-# - returns the offset of the located pattern or ~0 if not found
-# - exits with exit status 0 normally or non-0 if pattern is not found
-#   or any other error occurs.
-
-****************************************************************/
-
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <errno.h>
-#include <unistd.h>
-#include <fcntl.h>
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <sys/mman.h>
-
-#define VERSION		"0.1"
-#define BUF_SIZE	(16 * 1024)
-#define PAT_SIZE	100
-
-char		*progname;
-char		*inputname;
-int		inputfd;
-unsigned int	bix;			/* buf index */
-unsigned char	patterns [PAT_SIZE] = {0}; /* byte-sized pattern array */
-int		pat_len;		/* actual number of pattern bytes */
-unsigned char	*madr;			/* mmap address */
-size_t		filesize;
-int		num_matches = 0;
-off_t		firstloc = 0;
-
-void usage (void)
-{
-	fprintf (stderr, "%s ver. %s\n", progname, VERSION);
-	fprintf (stderr, "usage:  %s filename pattern_bytes\n",
-			progname);
-	fprintf (stderr, "        [prints location of pattern_bytes in file]\n");
-	exit (1);
-}
-
-void get_pattern (int pat_count, char *pats [])
-{
-	int ix, err, tmp;
-
-#ifdef DEBUG
-	fprintf (stderr,"get_pattern: count = %d\n", pat_count);
-	for (ix = 0; ix < pat_count; ix++)
-		fprintf (stderr, "  pat # %d:  [%s]\n", ix, pats[ix]);
-#endif
-
-	for (ix = 0; ix < pat_count; ix++) {
-		tmp = 0;
-		err = sscanf (pats[ix], "%5i", &tmp);
-		if (err != 1 || tmp > 0xff) {
-			fprintf (stderr, "pattern or value error in pattern # %d [%s]\n",
-					ix, pats[ix]);
-			usage ();
-		}
-		patterns [ix] = tmp;
-	}
-	pat_len = pat_count;
-}
-
-void search_pattern (void)
-{
-	for (bix = 0; bix < filesize; bix++) {
-		if (madr[bix] == patterns[0]) {
-			if (memcmp (&madr[bix], patterns, pat_len) == 0) {
-				if (num_matches == 0)
-					firstloc = bix;
-				num_matches++;
-			}
-		}
-	}
-}
-
-#ifdef NOTDEF
-size_t get_filesize (int fd)
-{
-	off_t end_off = lseek (fd, 0, SEEK_END);
-	lseek (fd, 0, SEEK_SET);
-	return (size_t) end_off;
-}
-#endif
-
-size_t get_filesize (int fd)
-{
-	int err;
-	struct stat stat;
-
-	err = fstat (fd, &stat);
-	fprintf (stderr, "filesize: %ld\n", err < 0 ? (long)err : stat.st_size);
-	if (err < 0)
-		return err;
-	return (size_t) stat.st_size;
-}
-
-int main (int argc, char *argv [])
-{
-	progname = argv[0];
-
-	if (argc < 3)
-		usage ();
-
-	get_pattern (argc - 2, argv + 2);
-
-	inputname = argv[1];
-
-	inputfd = open (inputname, O_RDONLY);
-	if (inputfd == -1) {
-		fprintf (stderr, "%s: cannot open '%s'\n",
-				progname, inputname);
-		exit (3);
-	}
-
-	filesize = get_filesize (inputfd);
-
-	madr = mmap (0, filesize, PROT_READ, MAP_PRIVATE, inputfd, 0);
-	if (madr == MAP_FAILED) {
-		fprintf (stderr, "mmap error = %d\n", errno);
-		close (inputfd);
-		exit (4);
-	}
-
-	search_pattern ();
-
-	if (munmap (madr, filesize))
-		fprintf (stderr, "munmap error = %d\n", errno);
-
-	if (close (inputfd))
-		fprintf (stderr, "%s: error %d closing '%s'\n",
-				progname, errno, inputname);
-
-	fprintf (stderr, "number of pattern matches = %d\n", num_matches);
-	if (num_matches == 0)
-		firstloc = ~0;
-	printf ("%ld\n", firstloc);
-	fprintf (stderr, "%ld\n", firstloc);
-
-	exit (num_matches ? 0 : 2);
-}
-
-/* end binoffset.c */
diff --git a/scripts/extract-ikconfig b/scripts/extract-ikconfig
index 72997c3..37f30d3 100755
--- a/scripts/extract-ikconfig
+++ b/scripts/extract-ikconfig
@@ -1,78 +1,53 @@
 #!/bin/sh
-# extracts .config info from a [b]zImage file
-# uses: binoffset (new), dd, zcat, strings, grep
-# $arg1 is [b]zImage filename
-
-binoffset="./scripts/binoffset"
-test -e $binoffset || cc -o $binoffset ./scripts/binoffset.c || exit 1
-
-IKCFG_ST="0x49 0x4b 0x43 0x46 0x47 0x5f 0x53 0x54"
-IKCFG_ED="0x49 0x4b 0x43 0x46 0x47 0x5f 0x45 0x44"
-dump_config() {
-    file="$1"
-
-    start=`$binoffset $file $IKCFG_ST 2>/dev/null`
-    [ "$?" != "0" ] && start="-1"
-    if [ "$start" -eq "-1" ]; then
-	return
-    fi
-    end=`$binoffset $file $IKCFG_ED 2>/dev/null`
-
-    start=`expr $start + 8`
-    size=`expr $end - $start`
-
-    dd if="$file" ibs=1 skip="$start" count="$size" 2>/dev/null | zcat
-
-    clean_up
-    exit 0
-}
-
-
-usage()
+# ----------------------------------------------------------------------
+# extract-ikconfig - Extract the .config file from a kernel image
+#
+# This will only work when the kernel was compiled with CONFIG_IKCONFIG.
+#
+# The obscure use of the "tr" filter is to work around older versions of
+# "grep" that report the byte offset of the line instead of the pattern.
+#
+# (c) 2009, Dick Streefland <dick@streefland.net>
+# Licensed under the terms of the GNU General Public License.
+# ----------------------------------------------------------------------
+
+gz1='\037\213\010'
+gz2='01'
+cf1='IKCFG_ST\037\213\010'
+cf2='0123456789'
+
+dump_config()
 {
-	echo "  usage: extract-ikconfig [b]zImage_filename"
-}
-
-clean_up()
-{
-	if [ "$TMPFILE" != "" ]; then
-		rm -f $TMPFILE
+	if	pos=`tr "$cf1\n$cf2" "\n$cf2=" < "$1" | grep -abo "^$cf2"`
+	then
+		pos=${pos%%:*}
+		tail -c+$(($pos+8)) "$1" | zcat -q
+		exit 0
 	fi
 }
 
-if [ $# -lt 1 ]
+# Check invocation:
+me=${0##*/}
+img=$1
+if	[ $# -ne 1 -o ! -s "$img" ]
 then
-	usage
-	exit 1
+	echo "Usage: $me <kernel-image>" >&2
+	exit 2
 fi
 
-TMPFILE=`mktemp -t ikconfig-XXXXXX` || exit 1
-image="$1"
-
-# vmlinux: Attempt to dump the configuration from the file directly
-dump_config "$image"
-
-GZHDR1="0x1f 0x8b 0x08 0x00"
-GZHDR2="0x1f 0x8b 0x08 0x08"
-
-# vmlinux.gz: Check for a compressed images
-off=`$binoffset "$image" $GZHDR1 2>/dev/null`
-[ "$?" != "0" ] && off="-1"
-if [ "$off" -eq "-1" ]; then
-	off=`$binoffset "$image" $GZHDR2 2>/dev/null`
-	[ "$?" != "0" ] && off="-1"
-fi
-if [ "$off" -eq "0" ]; then
-	zcat <"$image" >"$TMPFILE"
-	dump_config "$TMPFILE"
-elif [ "$off" -ne "-1" ]; then
-	(dd ibs="$off" skip=1 count=0 && dd bs=512k) <"$image" 2>/dev/null | \
-		zcat >"$TMPFILE"
-	dump_config "$TMPFILE"
-fi
-
-echo "ERROR: Unable to extract kernel configuration information."
-echo "       This kernel image may not have the config info."
-
-clean_up
+# Initial attempt for uncompressed images or objects:
+dump_config "$img"
+
+# That didn't work, so decompress and try again:
+tmp=/tmp/ikconfig$$
+trap "rm -f $tmp" 0
+for	pos in `tr "$gz1\n$gz2" "\n$gz2=" < "$img" | grep -abo "^$gz2"`
+do
+	pos=${pos%%:*}
+	tail -c+$pos "$img" | zcat 2> /dev/null > $tmp
+	dump_config $tmp
+done
+
+# Bail out:
+echo "$me: Cannot find kernel config." >&2
 exit 1

Re: [PATCH] kconfig: simplification of scripts/extract-ikconfig

[Sam Ravnborg]

On Sat, Jul 18, 2009 at 01:08:59AM +0200, Dick Streefland wrote:
> On Friday 2009-07-17 23:17, Sam Ravnborg wrote:
> | Dick - I lost the original patch.
> | Can you please resend.
> | 
> | In the changelog please include the information that
> | you tested this with relevant grep versions.
> 
> Sure, here it is:
> 
> ======================================================================
> 
> I've rewritten the extract-ikconfig script to extract the kernel
> configuration from a kernel compiled with CONFIG_IKCONFIG. The main
> motivation for the rewrite was to remove the dependency on the
> external C program binoffset.c, which is compiled on the initial run.
> 
> The binoffset executable is invoked with a relative path, which means
> that the old script can only be run from the top of the kernel tree,
> and only when you have write permission in the scripts directory.
> The new script uses tr/grep/tail/zcat only, and can be invoked from
> anywhere. The binoffset.c program has been removed. This script
> requires GNU grep 2.5 (released 2002-03-13) or higher, because the -o
> option was introduced in that version.
> 
> Signed-off-by: Dick Streefland <dick@streefland.net>

Applied to kbuild-next.git.

	Sam

[PATCH] kconfig: simplification of scripts/extract-ikconfig

[Dick Streefland]

I've rewritten the extract-ikconfig script to extract the kernel
configuration from a kernel compiled with CONFIG_IKCONFIG. The main
motivation for the rewrite was to remove the dependency on the
external C program binoffset.c, which is compiled on the initial run.

The binoffset executable is invoked with a relative path, which means
that the old script can only be run from the top of the kernel tree,
and only when you have write permission in the scripts directory.

The new script uses tr/grep/tail/zcat only, and can be invoked from
anywhere. The binoffset.c program has been removed. This script
requires GNU grep 2.5 (released 2002-03-13) or higher, because the -o
option was introduced in that version.

Signed-off-by: Dick Streefland <dick@streefland.net>
Tested-by: Steven Rostedt <rostedt@goodmis.org>
Acked-by: Steven Rostedt <rostedt@goodmis.org>

---

 Documentation/dontdiff   |    1 -
 scripts/.gitignore       |    1 -
 scripts/binoffset.c      |  163 ----------------------------------------------
 scripts/extract-ikconfig |  127 ++++++++++++-----------------------
 4 files changed, 44 insertions(+), 248 deletions(-)

diff --git a/Documentation/dontdiff b/Documentation/dontdiff
index e1efc40..bc01a5a 100644
--- a/Documentation/dontdiff
+++ b/Documentation/dontdiff
@@ -68,7 +68,6 @@ autoconf.h*
 bbootsect
 bin2c
 binkernel.spec
-binoffset
 bootsect
 bounds.h
 bsetup
diff --git a/scripts/.gitignore b/scripts/.gitignore
index 52cab46..c5d5db5 100644
--- a/scripts/.gitignore
+++ b/scripts/.gitignore
@@ -6,5 +6,4 @@ kallsyms
 pnmtologo
 bin2c
 unifdef
-binoffset
 ihex2fw
diff --git a/scripts/binoffset.c b/scripts/binoffset.c
deleted file mode 100644
index 1a2e39b..0000000
--- a/scripts/binoffset.c
+++ /dev/null
@@ -1,163 +0,0 @@
-/***************************************************************************
- * binoffset.c
- * (C) 2002 Randy Dunlap <rdunlap@xenotime.net>
-
-#   This program is free software; you can redistribute it and/or modify
-#   it under the terms of the GNU General Public License as published by
-#   the Free Software Foundation; either version 2 of the License, or
-#   (at your option) any later version.
-#
-#   This program is distributed in the hope that it will be useful,
-#   but WITHOUT ANY WARRANTY; without even the implied warranty of
-#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-#   GNU General Public License for more details.
-#
-#   You should have received a copy of the GNU General Public License
-#   along with this program; if not, write to the Free Software
-#   Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
-
-# binoffset.c:
-# - searches a (binary) file for a specified (binary) pattern
-# - returns the offset of the located pattern or ~0 if not found
-# - exits with exit status 0 normally or non-0 if pattern is not found
-#   or any other error occurs.
-
-****************************************************************/
-
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <errno.h>
-#include <unistd.h>
-#include <fcntl.h>
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <sys/mman.h>
-
-#define VERSION		"0.1"
-#define BUF_SIZE	(16 * 1024)
-#define PAT_SIZE	100
-
-char		*progname;
-char		*inputname;
-int		inputfd;
-unsigned int	bix;			/* buf index */
-unsigned char	patterns [PAT_SIZE] = {0}; /* byte-sized pattern array */
-int		pat_len;		/* actual number of pattern bytes */
-unsigned char	*madr;			/* mmap address */
-size_t		filesize;
-int		num_matches = 0;
-off_t		firstloc = 0;
-
-void usage (void)
-{
-	fprintf (stderr, "%s ver. %s\n", progname, VERSION);
-	fprintf (stderr, "usage:  %s filename pattern_bytes\n",
-			progname);
-	fprintf (stderr, "        [prints location of pattern_bytes in file]\n");
-	exit (1);
-}
-
-void get_pattern (int pat_count, char *pats [])
-{
-	int ix, err, tmp;
-
-#ifdef DEBUG
-	fprintf (stderr,"get_pattern: count = %d\n", pat_count);
-	for (ix = 0; ix < pat_count; ix++)
-		fprintf (stderr, "  pat # %d:  [%s]\n", ix, pats[ix]);
-#endif
-
-	for (ix = 0; ix < pat_count; ix++) {
-		tmp = 0;
-		err = sscanf (pats[ix], "%5i", &tmp);
-		if (err != 1 || tmp > 0xff) {
-			fprintf (stderr, "pattern or value error in pattern # %d [%s]\n",
-					ix, pats[ix]);
-			usage ();
-		}
-		patterns [ix] = tmp;
-	}
-	pat_len = pat_count;
-}
-
-void search_pattern (void)
-{
-	for (bix = 0; bix < filesize; bix++) {
-		if (madr[bix] == patterns[0]) {
-			if (memcmp (&madr[bix], patterns, pat_len) == 0) {
-				if (num_matches == 0)
-					firstloc = bix;
-				num_matches++;
-			}
-		}
-	}
-}
-
-#ifdef NOTDEF
-size_t get_filesize (int fd)
-{
-	off_t end_off = lseek (fd, 0, SEEK_END);
-	lseek (fd, 0, SEEK_SET);
-	return (size_t) end_off;
-}
-#endif
-
-size_t get_filesize (int fd)
-{
-	int err;
-	struct stat stat;
-
-	err = fstat (fd, &stat);
-	fprintf (stderr, "filesize: %ld\n", err < 0 ? (long)err : stat.st_size);
-	if (err < 0)
-		return err;
-	return (size_t) stat.st_size;
-}
-
-int main (int argc, char *argv [])
-{
-	progname = argv[0];
-
-	if (argc < 3)
-		usage ();
-
-	get_pattern (argc - 2, argv + 2);
-
-	inputname = argv[1];
-
-	inputfd = open (inputname, O_RDONLY);
-	if (inputfd == -1) {
-		fprintf (stderr, "%s: cannot open '%s'\n",
-				progname, inputname);
-		exit (3);
-	}
-
-	filesize = get_filesize (inputfd);
-
-	madr = mmap (0, filesize, PROT_READ, MAP_PRIVATE, inputfd, 0);
-	if (madr == MAP_FAILED) {
-		fprintf (stderr, "mmap error = %d\n", errno);
-		close (inputfd);
-		exit (4);
-	}
-
-	search_pattern ();
-
-	if (munmap (madr, filesize))
-		fprintf (stderr, "munmap error = %d\n", errno);
-
-	if (close (inputfd))
-		fprintf (stderr, "%s: error %d closing '%s'\n",
-				progname, errno, inputname);
-
-	fprintf (stderr, "number of pattern matches = %d\n", num_matches);
-	if (num_matches == 0)
-		firstloc = ~0;
-	printf ("%ld\n", firstloc);
-	fprintf (stderr, "%ld\n", firstloc);
-
-	exit (num_matches ? 0 : 2);
-}
-
-/* end binoffset.c */
diff --git a/scripts/extract-ikconfig b/scripts/extract-ikconfig
index de233ff..37f30d3 100755
--- a/scripts/extract-ikconfig
+++ b/scripts/extract-ikconfig
@@ -1,92 +1,53 @@
 #!/bin/sh
-# extracts .config info from a [b]zImage file
-# uses: binoffset (new), dd, zcat, strings, grep
-# $arg1 is [b]zImage filename
-
-binoffset="./scripts/binoffset"
-test -e $binoffset || cc -o $binoffset ./scripts/binoffset.c || exit 1
-
-IKCFG_ST="0x49 0x4b 0x43 0x46 0x47 0x5f 0x53 0x54"
-IKCFG_ED="0x49 0x4b 0x43 0x46 0x47 0x5f 0x45 0x44"
-dump_config() {
-    file="$1"
-
-    start=`$binoffset $file $IKCFG_ST 2>/dev/null`
-    [ "$?" != "0" ] && start="-1"
-    if [ "$start" -eq "-1" ]; then
-	return
-    fi
-    end=`$binoffset $file $IKCFG_ED 2>/dev/null`
-    [ "$?" != "0" ] && end="-1"
-    if [ "$end" -eq "-1" ]; then
-	return
-    fi
-
-    start=`expr $start + 8`
-    size=`expr $end - $start`
-
-    dd if="$file" ibs=1 skip="$start" count="$size" 2>/dev/null | zcat
-
-    clean_up
-    exit 0
-}
-
-
-usage()
-{
-	echo "  usage: extract-ikconfig [b]zImage_filename"
-}
-
-clean_up()
+# ----------------------------------------------------------------------
+# extract-ikconfig - Extract the .config file from a kernel image
+#
+# This will only work when the kernel was compiled with CONFIG_IKCONFIG.
+#
+# The obscure use of the "tr" filter is to work around older versions of
+# "grep" that report the byte offset of the line instead of the pattern.
+#
+# (c) 2009, Dick Streefland <dick@streefland.net>
+# Licensed under the terms of the GNU General Public License.
+# ----------------------------------------------------------------------
+
+gz1='\037\213\010'
+gz2='01'
+cf1='IKCFG_ST\037\213\010'
+cf2='0123456789'
+
+dump_config()
 {
-	if [ "$TMPFILE" != "" ]; then
-		rm -f $TMPFILE
+	if	pos=`tr "$cf1\n$cf2" "\n$cf2=" < "$1" | grep -abo "^$cf2"`
+	then
+		pos=${pos%%:*}
+		tail -c+$(($pos+8)) "$1" | zcat -q
+		exit 0
 	fi
 }
 
-if [ $# -lt 1 ]
+# Check invocation:
+me=${0##*/}
+img=$1
+if	[ $# -ne 1 -o ! -s "$img" ]
 then
-	usage
-	exit 1
+	echo "Usage: $me <kernel-image>" >&2
+	exit 2
 fi
 
-TMPFILE=`mktemp -t ikconfig-XXXXXX` || exit 1
-image="$1"
-
-# vmlinux: Attempt to dump the configuration from the file directly
-dump_config "$image"
-
-GZHDR1="0x1f 0x8b 0x08 0x00"
-GZHDR2="0x1f 0x8b 0x08 0x08"
-
-ELFHDR="0x7f 0x45 0x4c 0x46"
-
-# vmlinux.gz: Check for a compressed images
-off=`$binoffset "$image" $GZHDR1 2>/dev/null`
-[ "$?" != "0" ] && off="-1"
-if [ "$off" -eq "-1" ]; then
-	off=`$binoffset "$image" $GZHDR2 2>/dev/null`
-	[ "$?" != "0" ] && off="-1"
-fi
-if [ "$off" -eq "0" ]; then
-	zcat <"$image" >"$TMPFILE"
-	dump_config "$TMPFILE"
-elif [ "$off" -ne "-1" ]; then
-	(dd ibs="$off" skip=1 count=0 && dd bs=512k) <"$image" 2>/dev/null | \
-		zcat >"$TMPFILE"
-	dump_config "$TMPFILE"
-
-# check if this is simply an ELF file
-else
-	off=`$binoffset "$image" $ELFHDR 2>/dev/null`
-	[ "$?" != "0" ] && off="-1"
-	if [ "$off" -eq "0" ]; then
-		dump_config "$image"
-	fi
-fi
-
-echo "ERROR: Unable to extract kernel configuration information."
-echo "       This kernel image may not have the config info."
-
-clean_up
+# Initial attempt for uncompressed images or objects:
+dump_config "$img"
+
+# That didn't work, so decompress and try again:
+tmp=/tmp/ikconfig$$
+trap "rm -f $tmp" 0
+for	pos in `tr "$gz1\n$gz2" "\n$gz2=" < "$img" | grep -abo "^$gz2"`
+do
+	pos=${pos%%:*}
+	tail -c+$pos "$img" | zcat 2> /dev/null > $tmp
+	dump_config $tmp
+done
+
+# Bail out:
+echo "$me: Cannot find kernel config." >&2
 exit 1

Re: [PATCH] kconfig: simplification of scripts/extract-ikconfig

[Steven Rostedt]

On Tue, 2009-10-06 at 22:35 +0200, Dick Streefland wrote:
> I've rewritten the extract-ikconfig script to extract the kernel
> configuration from a kernel compiled with CONFIG_IKCONFIG. The main
> motivation for the rewrite was to remove the dependency on the
> external C program binoffset.c, which is compiled on the initial run.
> 
> The binoffset executable is invoked with a relative path, which means
> that the old script can only be run from the top of the kernel tree,
> and only when you have write permission in the scripts directory.
> 
> The new script uses tr/grep/tail/zcat only, and can be invoked from
> anywhere. The binoffset.c program has been removed. This script
> requires GNU grep 2.5 (released 2002-03-13) or higher, because the -o
> option was introduced in that version.

Thanks Dick!

Now the question is, is this something that we should try to get into
the 32 release, or let this version go through linux-next for another
release?

-- Steve

> 
> Signed-off-by: Dick Streefland <dick@streefland.net>
> Tested-by: Steven Rostedt <rostedt@goodmis.org>
> Acked-by: Steven Rostedt <rostedt@goodmis.org>
> 
> ---
> 
>  Documentation/dontdiff   |    1 -
>  scripts/.gitignore       |    1 -
>  scripts/binoffset.c      |  163 ----------------------------------------------
>  scripts/extract-ikconfig |  127 ++++++++++++-----------------------
>  4 files changed, 44 insertions(+), 248 deletions(-)
> 
> diff --git a/Documentation/dontdiff b/Documentation/dontdiff
> index e1efc40..bc01a5a 100644
> --- a/Documentation/dontdiff
> +++ b/Documentation/dontdiff
> @@ -68,7 +68,6 @@ autoconf.h*
>  bbootsect
>  bin2c
>  binkernel.spec
> -binoffset
>  bootsect
>  bounds.h
>  bsetup
> diff --git a/scripts/.gitignore b/scripts/.gitignore
> index 52cab46..c5d5db5 100644
> --- a/scripts/.gitignore
> +++ b/scripts/.gitignore
> @@ -6,5 +6,4 @@ kallsyms
>  pnmtologo
>  bin2c
>  unifdef
> -binoffset
>  ihex2fw
> diff --git a/scripts/binoffset.c b/scripts/binoffset.c
> deleted file mode 100644
> index 1a2e39b..0000000
> --- a/scripts/binoffset.c
> +++ /dev/null
> @@ -1,163 +0,0 @@
> -/***************************************************************************
> - * binoffset.c
> - * (C) 2002 Randy Dunlap <rdunlap@xenotime.net>
> -
> -#   This program is free software; you can redistribute it and/or modify
> -#   it under the terms of the GNU General Public License as published by
> -#   the Free Software Foundation; either version 2 of the License, or
> -#   (at your option) any later version.
> -#
> -#   This program is distributed in the hope that it will be useful,
> -#   but WITHOUT ANY WARRANTY; without even the implied warranty of
> -#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> -#   GNU General Public License for more details.
> -#
> -#   You should have received a copy of the GNU General Public License
> -#   along with this program; if not, write to the Free Software
> -#   Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
> -
> -# binoffset.c:
> -# - searches a (binary) file for a specified (binary) pattern
> -# - returns the offset of the located pattern or ~0 if not found
> -# - exits with exit status 0 normally or non-0 if pattern is not found
> -#   or any other error occurs.
> -
> -****************************************************************/
> -
> -#include <stdio.h>
> -#include <stdlib.h>
> -#include <string.h>
> -#include <errno.h>
> -#include <unistd.h>
> -#include <fcntl.h>
> -#include <sys/types.h>
> -#include <sys/stat.h>
> -#include <sys/mman.h>
> -
> -#define VERSION		"0.1"
> -#define BUF_SIZE	(16 * 1024)
> -#define PAT_SIZE	100
> -
> -char		*progname;
> -char		*inputname;
> -int		inputfd;
> -unsigned int	bix;			/* buf index */
> -unsigned char	patterns [PAT_SIZE] = {0}; /* byte-sized pattern array */
> -int		pat_len;		/* actual number of pattern bytes */
> -unsigned char	*madr;			/* mmap address */
> -size_t		filesize;
> -int		num_matches = 0;
> -off_t		firstloc = 0;
> -
> -void usage (void)
> -{
> -	fprintf (stderr, "%s ver. %s\n", progname, VERSION);
> -	fprintf (stderr, "usage:  %s filename pattern_bytes\n",
> -			progname);
> -	fprintf (stderr, "        [prints location of pattern_bytes in file]\n");
> -	exit (1);
> -}
> -
> -void get_pattern (int pat_count, char *pats [])
> -{
> -	int ix, err, tmp;
> -
> -#ifdef DEBUG
> -	fprintf (stderr,"get_pattern: count = %d\n", pat_count);
> -	for (ix = 0; ix < pat_count; ix++)
> -		fprintf (stderr, "  pat # %d:  [%s]\n", ix, pats[ix]);
> -#endif
> -
> -	for (ix = 0; ix < pat_count; ix++) {
> -		tmp = 0;
> -		err = sscanf (pats[ix], "%5i", &tmp);
> -		if (err != 1 || tmp > 0xff) {
> -			fprintf (stderr, "pattern or value error in pattern # %d [%s]\n",
> -					ix, pats[ix]);
> -			usage ();
> -		}
> -		patterns [ix] = tmp;
> -	}
> -	pat_len = pat_count;
> -}
> -
> -void search_pattern (void)
> -{
> -	for (bix = 0; bix < filesize; bix++) {
> -		if (madr[bix] == patterns[0]) {
> -			if (memcmp (&madr[bix], patterns, pat_len) == 0) {
> -				if (num_matches == 0)
> -					firstloc = bix;
> -				num_matches++;
> -			}
> -		}
> -	}
> -}
> -
> -#ifdef NOTDEF
> -size_t get_filesize (int fd)
> -{
> -	off_t end_off = lseek (fd, 0, SEEK_END);
> -	lseek (fd, 0, SEEK_SET);
> -	return (size_t) end_off;
> -}
> -#endif
> -
> -size_t get_filesize (int fd)
> -{
> -	int err;
> -	struct stat stat;
> -
> -	err = fstat (fd, &stat);
> -	fprintf (stderr, "filesize: %ld\n", err < 0 ? (long)err : stat.st_size);
> -	if (err < 0)
> -		return err;
> -	return (size_t) stat.st_size;
> -}
> -
> -int main (int argc, char *argv [])
> -{
> -	progname = argv[0];
> -
> -	if (argc < 3)
> -		usage ();
> -
> -	get_pattern (argc - 2, argv + 2);
> -
> -	inputname = argv[1];
> -
> -	inputfd = open (inputname, O_RDONLY);
> -	if (inputfd == -1) {
> -		fprintf (stderr, "%s: cannot open '%s'\n",
> -				progname, inputname);
> -		exit (3);
> -	}
> -
> -	filesize = get_filesize (inputfd);
> -
> -	madr = mmap (0, filesize, PROT_READ, MAP_PRIVATE, inputfd, 0);
> -	if (madr == MAP_FAILED) {
> -		fprintf (stderr, "mmap error = %d\n", errno);
> -		close (inputfd);
> -		exit (4);
> -	}
> -
> -	search_pattern ();
> -
> -	if (munmap (madr, filesize))
> -		fprintf (stderr, "munmap error = %d\n", errno);
> -
> -	if (close (inputfd))
> -		fprintf (stderr, "%s: error %d closing '%s'\n",
> -				progname, errno, inputname);
> -
> -	fprintf (stderr, "number of pattern matches = %d\n", num_matches);
> -	if (num_matches == 0)
> -		firstloc = ~0;
> -	printf ("%ld\n", firstloc);
> -	fprintf (stderr, "%ld\n", firstloc);
> -
> -	exit (num_matches ? 0 : 2);
> -}
> -
> -/* end binoffset.c */
> diff --git a/scripts/extract-ikconfig b/scripts/extract-ikconfig
> index de233ff..37f30d3 100755
> --- a/scripts/extract-ikconfig
> +++ b/scripts/extract-ikconfig
> @@ -1,92 +1,53 @@
>  #!/bin/sh
> -# extracts .config info from a [b]zImage file
> -# uses: binoffset (new), dd, zcat, strings, grep
> -# $arg1 is [b]zImage filename
> -
> -binoffset="./scripts/binoffset"
> -test -e $binoffset || cc -o $binoffset ./scripts/binoffset.c || exit 1
> -
> -IKCFG_ST="0x49 0x4b 0x43 0x46 0x47 0x5f 0x53 0x54"
> -IKCFG_ED="0x49 0x4b 0x43 0x46 0x47 0x5f 0x45 0x44"
> -dump_config() {
> -    file="$1"
> -
> -    start=`$binoffset $file $IKCFG_ST 2>/dev/null`
> -    [ "$?" != "0" ] && start="-1"
> -    if [ "$start" -eq "-1" ]; then
> -	return
> -    fi
> -    end=`$binoffset $file $IKCFG_ED 2>/dev/null`
> -    [ "$?" != "0" ] && end="-1"
> -    if [ "$end" -eq "-1" ]; then
> -	return
> -    fi
> -
> -    start=`expr $start + 8`
> -    size=`expr $end - $start`
> -
> -    dd if="$file" ibs=1 skip="$start" count="$size" 2>/dev/null | zcat
> -
> -    clean_up
> -    exit 0
> -}
> -
> -
> -usage()
> -{
> -	echo "  usage: extract-ikconfig [b]zImage_filename"
> -}
> -
> -clean_up()
> +# ----------------------------------------------------------------------
> +# extract-ikconfig - Extract the .config file from a kernel image
> +#
> +# This will only work when the kernel was compiled with CONFIG_IKCONFIG.
> +#
> +# The obscure use of the "tr" filter is to work around older versions of
> +# "grep" that report the byte offset of the line instead of the pattern.
> +#
> +# (c) 2009, Dick Streefland <dick@streefland.net>
> +# Licensed under the terms of the GNU General Public License.
> +# ----------------------------------------------------------------------
> +
> +gz1='\037\213\010'
> +gz2='01'
> +cf1='IKCFG_ST\037\213\010'
> +cf2='0123456789'
> +
> +dump_config()
>  {
> -	if [ "$TMPFILE" != "" ]; then
> -		rm -f $TMPFILE
> +	if	pos=`tr "$cf1\n$cf2" "\n$cf2=" < "$1" | grep -abo "^$cf2"`
> +	then
> +		pos=${pos%%:*}
> +		tail -c+$(($pos+8)) "$1" | zcat -q
> +		exit 0
>  	fi
>  }
>  
> -if [ $# -lt 1 ]
> +# Check invocation:
> +me=${0##*/}
> +img=$1
> +if	[ $# -ne 1 -o ! -s "$img" ]
>  then
> -	usage
> -	exit 1
> +	echo "Usage: $me <kernel-image>" >&2
> +	exit 2
>  fi
>  
> -TMPFILE=`mktemp -t ikconfig-XXXXXX` || exit 1
> -image="$1"
> -
> -# vmlinux: Attempt to dump the configuration from the file directly
> -dump_config "$image"
> -
> -GZHDR1="0x1f 0x8b 0x08 0x00"
> -GZHDR2="0x1f 0x8b 0x08 0x08"
> -
> -ELFHDR="0x7f 0x45 0x4c 0x46"
> -
> -# vmlinux.gz: Check for a compressed images
> -off=`$binoffset "$image" $GZHDR1 2>/dev/null`
> -[ "$?" != "0" ] && off="-1"
> -if [ "$off" -eq "-1" ]; then
> -	off=`$binoffset "$image" $GZHDR2 2>/dev/null`
> -	[ "$?" != "0" ] && off="-1"
> -fi
> -if [ "$off" -eq "0" ]; then
> -	zcat <"$image" >"$TMPFILE"
> -	dump_config "$TMPFILE"
> -elif [ "$off" -ne "-1" ]; then
> -	(dd ibs="$off" skip=1 count=0 && dd bs=512k) <"$image" 2>/dev/null | \
> -		zcat >"$TMPFILE"
> -	dump_config "$TMPFILE"
> -
> -# check if this is simply an ELF file
> -else
> -	off=`$binoffset "$image" $ELFHDR 2>/dev/null`
> -	[ "$?" != "0" ] && off="-1"
> -	if [ "$off" -eq "0" ]; then
> -		dump_config "$image"
> -	fi
> -fi
> -
> -echo "ERROR: Unable to extract kernel configuration information."
> -echo "       This kernel image may not have the config info."
> -
> -clean_up
> +# Initial attempt for uncompressed images or objects:
> +dump_config "$img"
> +
> +# That didn't work, so decompress and try again:
> +tmp=/tmp/ikconfig$$
> +trap "rm -f $tmp" 0
> +for	pos in `tr "$gz1\n$gz2" "\n$gz2=" < "$img" | grep -abo "^$gz2"`
> +do
> +	pos=${pos%%:*}
> +	tail -c+$pos "$img" | zcat 2> /dev/null > $tmp
> +	dump_config $tmp
> +done
> +
> +# Bail out:
> +echo "$me: Cannot find kernel config." >&2
>  exit 1

Re: [PATCH] kconfig: simplification of scripts/extract-ikconfig

[Dick Streefland]

On Tuesday 2009-10-06 22:11, Steven Rostedt wrote:
| Now the question is, is this something that we should try to get into
| the 32 release, or let this version go through linux-next for another
| release?

I don't have a strong preference, as long as it gets into the kernel
tree in the end. I realize that it is a bit late in the 32 release
cycle, but on the other hand this script is not used in the kernel
build process.

-- 
Dick

Re: [PATCH] kconfig: simplification of scripts/extract-ikconfig

[Steven Rostedt]

On Wed, 2009-10-07 at 10:41 +0200, Dick Streefland wrote:
> On Tuesday 2009-10-06 22:11, Steven Rostedt wrote:
> | Now the question is, is this something that we should try to get into
> | the 32 release, or let this version go through linux-next for another
> | release?
> 
> I don't have a strong preference, as long as it gets into the kernel
> tree in the end. I realize that it is a bit late in the 32 release
> cycle, but on the other hand this script is not used in the kernel
> build process.

I'm sure it is fine, but since we are after the merge window, I'll push
this off to linux-next.

Thanks,

-- Steve