[PATCH v2 0/2] scripts: headers_install: avoid some reprocessing

[PATCH v2 0/2] scripts: headers_install: avoid some reprocessing

[David Disseldorp]

scripts/headers_install.sh is relatively inefficient: it parses/modifies
headers via unifdef and three sed invocations, with sed output
post-processed via grep in one case (user-space CONFIG_ leaks).

These patches attempt to improve simplicity and efficiency by merging
two sed calls and processing user-space CONFIG_ leak filters inline.

Output is unaffected, aside from one minor error string change:
 error: $INFILE: leak CONFIG_X to user-space
becomes...
 error: $INFILE:CONFIG_X leak to user-space

License and CONFIG leak checks have been manually tested via error
injection.

Changes since v1 RFC:
- accept $INFILE prefixes from e.g. make headers_install O=build_dir
  as reported by Oliver Sang / kernel test robot
- drop extra config filter variable and do it all inline in sed

David Disseldorp (2):
      scripts: headers_install: filter ignored configs via sed
      scripts: headers_install: avoid extra sed call for license check

 scripts/headers_install.sh | 74 ++++++++++++++++----------------------
 1 file changed, 31 insertions(+), 43 deletions(-)

[PATCH v2 1/2] scripts: headers_install: filter ignored configs via sed

[David Disseldorp]

The sed script currently prints any CONFIG_ entries carried in installed
headers. A subsequent shell script parses this output to check whether
the found CONFIG_ values should be ignored or not.

Drop the unnecessary sed output post-processing and instead skip over
ignored CONFIG_ values as part of initial processing.

INFILE may carry a prefix for e.g. make headers_install O=build_dir .
Therefore, don't anchor the <file-name>:<CONFIG-option> match against
the beginning of INFILE.

Signed-off-by: David Disseldorp <ddiss@suse.de>
---
 scripts/headers_install.sh | 55 +++++++++++++-------------------------
 1 file changed, 19 insertions(+), 36 deletions(-)

diff --git a/scripts/headers_install.sh b/scripts/headers_install.sh
index 0e4e939efc940..2181abd1c9b70 100755
--- a/scripts/headers_install.sh
+++ b/scripts/headers_install.sh
@@ -42,7 +42,7 @@ scripts/unifdef -U__KERNEL__ -D__EXPORTED_HEADERS__ $TMPFILE > $OUTFILE
 [ $? -gt 1 ] && exit 1
 
 # Remove /* ... */ style comments, and find CONFIG_ references in code
-configs=$(sed -e '
+sed -e '
 :comment
 	s:/\*[^*][^*]*:/*:
 	s:/\*\*\**\([^/]\):/*\1:
@@ -53,48 +53,31 @@ configs=$(sed -e '
 	N
 	b comment
 :print
+	# The entries in the following list do not result in an error.
+	# Please do not add a new entry. This list is only for existing ones.
+	# The list will be reduced gradually, and deleted eventually.
+	#
+	# The format is s@<file-name>:<CONFIG-option>\n@@ in each line.
+	s@arch/arc/include/uapi/asm/swab.h:CONFIG_ARC_HAS_SWAPE\n@@
+	s@arch/arm/include/uapi/asm/ptrace.h:CONFIG_CPU_ENDIAN_BE8\n@@
+	s@arch/nios2/include/uapi/asm/swab.h:CONFIG_NIOS2_CI_SWAB_NO\n@@
+	s@arch/nios2/include/uapi/asm/swab.h:CONFIG_NIOS2_CI_SWAB_SUPPORT\n@@
+	s@arch/x86/include/uapi/asm/auxvec.h:CONFIG_IA32_EMULATION\n@@
+	s@arch/x86/include/uapi/asm/auxvec.h:CONFIG_X86_64\n@@
+
+	# Jump if any of the above filters applied, otherwise error out.
+	t check
+	s@^\(.*\)\n.*@error: \1 leak to user-space@
 	P
-	D
+	Q2
 :check
-	s:^\(CONFIG_[[:alnum:]_]*\):\1\n:
+	s@^\(CONFIG_[[:alnum:]_]*\)@'"$INFILE"':\1\n@
 	t print
 	s:^[[:alnum:]_][[:alnum:]_]*::
 	s:^[^[:alnum:]_][^[:alnum:]_]*::
 	t check
 	d
-' $OUTFILE)
-
-# The entries in the following list do not result in an error.
-# Please do not add a new entry. This list is only for existing ones.
-# The list will be reduced gradually, and deleted eventually. (hopefully)
-#
-# The format is <file-name>:<CONFIG-option> in each line.
-config_leak_ignores="
-arch/arc/include/uapi/asm/swab.h:CONFIG_ARC_HAS_SWAPE
-arch/arm/include/uapi/asm/ptrace.h:CONFIG_CPU_ENDIAN_BE8
-arch/nios2/include/uapi/asm/swab.h:CONFIG_NIOS2_CI_SWAB_NO
-arch/nios2/include/uapi/asm/swab.h:CONFIG_NIOS2_CI_SWAB_SUPPORT
-arch/x86/include/uapi/asm/auxvec.h:CONFIG_IA32_EMULATION
-arch/x86/include/uapi/asm/auxvec.h:CONFIG_X86_64
-"
-
-for c in $configs
-do
-	leak_error=1
-
-	for ignore in $config_leak_ignores
-	do
-		if echo "$INFILE:$c" | grep -q "$ignore$"; then
-			leak_error=
-			break
-		fi
-	done
-
-	if [ "$leak_error" = 1 ]; then
-		echo "error: $INFILE: leak $c to user-space" >&2
-		exit 1
-	fi
-done
+' $OUTFILE >&2 || exit 1
 
 rm -f $TMPFILE
 trap - EXIT
-- 
2.51.0

[PATCH v2 2/2] scripts: headers_install: avoid extra sed call for license check

[David Disseldorp]

headers_install runs a sed script to check that any GPL variant
SPDX-License-Identifier line carries a "WITH Linux-syscall-note".
A subsequent sed invocation then handles removal of a few things that
aren't desired in installed headers (e.g. __attribute_const__).

Combine these two sed scripts to avoid re-processing the same file.
License check errors, as opposed to write errors, are indicated via a
special sed exit status of 9.

Signed-off-by: David Disseldorp <ddiss@suse.de>
---
 scripts/headers_install.sh | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/scripts/headers_install.sh b/scripts/headers_install.sh
index 2181abd1c9b70..ef952cbbb8fde 100755
--- a/scripts/headers_install.sh
+++ b/scripts/headers_install.sh
@@ -23,20 +23,25 @@ TMPFILE=$OUTFILE.tmp
 
 trap 'rm -f $OUTFILE $TMPFILE' EXIT
 
-# SPDX-License-Identifier with GPL variants must have "WITH Linux-syscall-note"
-if [ -n "$(sed -n -e "/SPDX-License-Identifier:.*GPL-/{/WITH Linux-syscall-note/!p}" $INFILE)" ]; then
-	echo "error: $INFILE: missing \"WITH Linux-syscall-note\" for SPDX-License-Identifier" >&2
-	exit 1
-fi
-
+# returns 9 if GPL SPDX-License-Identifier omits "WITH Linux-syscall-note"
 sed -E -e '
+	/SPDX-License-Identifier:.*GPL-/{/WITH Linux-syscall-note/! Q9}
 	s/([[:space:](])(__user|__force|__iomem)[[:space:]]/\1/g
 	s/__attribute_const__([[:space:]]|$)/\1/g
 	s@^#include <linux/compiler.h>@@
 	s/(^|[^a-zA-Z0-9])__packed([^a-zA-Z0-9_]|$)/\1__attribute__((packed))\2/g
 	s/(^|[[:space:](])(inline|asm|volatile)([[:space:](]|$)/\1__\2__\3/g
 	s@#(ifndef|define|endif[[:space:]]*/[*])[[:space:]]*_UAPI@#\1 @
-' $INFILE > $TMPFILE || exit 1
+' $INFILE > $TMPFILE
+case $? in
+9)
+	echo "error: $INFILE: missing \"WITH Linux-syscall-note\" for SPDX-License-Identifier" >&2
+	exit 1
+	;;
+1)
+	exit 1
+	;;
+esac
 
 scripts/unifdef -U__KERNEL__ -D__EXPORTED_HEADERS__ $TMPFILE > $OUTFILE
 [ $? -gt 1 ] && exit 1
-- 
2.51.0

Re: [PATCH v2 0/2] scripts: headers_install: avoid some reprocessing

[David Disseldorp]

Here are the perf (perf stat -r 10) numbers I see with / without the v2
patches applied atop v6.19-rc4:

  $ cat bench.sh 
  touch scripts/headers_install.sh
  make O=/tmp/build_dir headers_install

  $ cat bench_j8.sh 
  touch scripts/headers_install.sh
  make -j8 O=/tmp/build_dir headers_install

        |    bench.sh        |   bench_j8.sh
--------+--------------------+---------------------
before  |  41.176 (+-0.28%)  |  2.58672 (+-0.28%)
--------+--------------------+---------------------
after   |  34.811 (+-1.35%)  |  2.33096 (+-0.39%)
--------+--------------------+---------------------

Re: [PATCH v2 1/2] scripts: headers_install: filter ignored configs via sed

[Thomas Weißschuh]

Hi David,

On Thu, Jan 08, 2026 at 07:39:42AM +1100, David Disseldorp wrote:
> The sed script currently prints any CONFIG_ entries carried in installed
> headers. A subsequent shell script parses this output to check whether
> the found CONFIG_ values should be ignored or not.
> Drop the unnecessary sed output post-processing and instead skip over
> ignored CONFIG_ values as part of initial processing.

The exlusion logic is going to go away completely soon:
https://lore.kernel.org/lkml/20260112-headers_install-config-leak-v2-0-4d8084444603@linutronix.de/
(My series is probably going through the asm-generic tree)

Our patches will conflict.


Thomas

Re: [PATCH v2 1/2] scripts: headers_install: filter ignored configs via sed

[David Disseldorp]

Hi Thomas,

On Mon, 12 Jan 2026 11:51:41 +0100, Thomas Weißschuh wrote:

> Hi David,
> 
> On Thu, Jan 08, 2026 at 07:39:42AM +1100, David Disseldorp wrote:
> > The sed script currently prints any CONFIG_ entries carried in installed
> > headers. A subsequent shell script parses this output to check whether
> > the found CONFIG_ values should be ignored or not.
> > Drop the unnecessary sed output post-processing and instead skip over
> > ignored CONFIG_ values as part of initial processing.  
> 
> The exlusion logic is going to go away completely soon:
> https://lore.kernel.org/lkml/20260112-headers_install-config-leak-v2-0-4d8084444603@linutronix.de/
> (My series is probably going through the asm-generic tree)
> 
> Our patches will conflict.

Looks good, I'm happy to drop this patch in favour of your removals.
Patch v2 2/2 should still be relevant and apply relatively cleanly.

Thanks, David

Re: [PATCH v2 2/2] scripts: headers_install: avoid extra sed call for license check

[Nicolas Schier]

[Cc+=nathan]

On Thu, Jan 08, 2026 at 07:39:43AM +1100, David Disseldorp wrote:
> headers_install runs a sed script to check that any GPL variant
> SPDX-License-Identifier line carries a "WITH Linux-syscall-note".
> A subsequent sed invocation then handles removal of a few things that
> aren't desired in installed headers (e.g. __attribute_const__).
> 
> Combine these two sed scripts to avoid re-processing the same file.
> License check errors, as opposed to write errors, are indicated via a
> special sed exit status of 9.
> 
> Signed-off-by: David Disseldorp <ddiss@suse.de>
> ---
>  scripts/headers_install.sh | 19 ++++++++++++-------
>  1 file changed, 12 insertions(+), 7 deletions(-)
> 
> diff --git a/scripts/headers_install.sh b/scripts/headers_install.sh
> index 2181abd1c9b70..ef952cbbb8fde 100755
> --- a/scripts/headers_install.sh
> +++ b/scripts/headers_install.sh
> @@ -23,20 +23,25 @@ TMPFILE=$OUTFILE.tmp
>  
>  trap 'rm -f $OUTFILE $TMPFILE' EXIT
>  
> -# SPDX-License-Identifier with GPL variants must have "WITH Linux-syscall-note"
> -if [ -n "$(sed -n -e "/SPDX-License-Identifier:.*GPL-/{/WITH Linux-syscall-note/!p}" $INFILE)" ]; then
> -	echo "error: $INFILE: missing \"WITH Linux-syscall-note\" for SPDX-License-Identifier" >&2
> -	exit 1
> -fi
> -
> +# returns 9 if GPL SPDX-License-Identifier omits "WITH Linux-syscall-note"
>  sed -E -e '
> +	/SPDX-License-Identifier:.*GPL-/{/WITH Linux-syscall-note/! Q9}

thanks, I like the idea of combining the sed calls.  According to the
manual 'Q' (or 'q' with argument) is a GNU extension (non-POSIX).

May we expect GNU sed on all system running headers_install?

>  	s/([[:space:](])(__user|__force|__iomem)[[:space:]]/\1/g
>  	s/__attribute_const__([[:space:]]|$)/\1/g
>  	s@^#include <linux/compiler.h>@@
>  	s/(^|[^a-zA-Z0-9])__packed([^a-zA-Z0-9_]|$)/\1__attribute__((packed))\2/g
>  	s/(^|[[:space:](])(inline|asm|volatile)([[:space:](]|$)/\1__\2__\3/g
>  	s@#(ifndef|define|endif[[:space:]]*/[*])[[:space:]]*_UAPI@#\1 @
> -' $INFILE > $TMPFILE || exit 1
> +' $INFILE > $TMPFILE
> +case $? in
> +9)
> +	echo "error: $INFILE: missing \"WITH Linux-syscall-note\" for SPDX-License-Identifier" >&2
> +	exit 1
> +	;;
> +1)
> +	exit 1
> +	;;
> +esac

This silently ignores all other exit codes.  What about this:

9)
...
	;;
0)	;;
*)
	exit 1
	;;
esac


Kind regards,
Nicolas

Re: [PATCH v2 2/2] scripts: headers_install: avoid extra sed call for license check

[David Disseldorp]

Thanks for the review Nicolas...

On Wed, 21 Jan 2026 12:34:49 +0100, Nicolas Schier wrote:

> [Cc+=nathan]
> 
> On Thu, Jan 08, 2026 at 07:39:43AM +1100, David Disseldorp wrote:
> > headers_install runs a sed script to check that any GPL variant
> > SPDX-License-Identifier line carries a "WITH Linux-syscall-note".
> > A subsequent sed invocation then handles removal of a few things that
> > aren't desired in installed headers (e.g. __attribute_const__).
> > 
> > Combine these two sed scripts to avoid re-processing the same file.
> > License check errors, as opposed to write errors, are indicated via a
> > special sed exit status of 9.
> > 
> > Signed-off-by: David Disseldorp <ddiss@suse.de>
> > ---
> >  scripts/headers_install.sh | 19 ++++++++++++-------
> >  1 file changed, 12 insertions(+), 7 deletions(-)
> > 
> > diff --git a/scripts/headers_install.sh b/scripts/headers_install.sh
> > index 2181abd1c9b70..ef952cbbb8fde 100755
> > --- a/scripts/headers_install.sh
> > +++ b/scripts/headers_install.sh
> > @@ -23,20 +23,25 @@ TMPFILE=$OUTFILE.tmp
> >  
> >  trap 'rm -f $OUTFILE $TMPFILE' EXIT
> >  
> > -# SPDX-License-Identifier with GPL variants must have "WITH Linux-syscall-note"
> > -if [ -n "$(sed -n -e "/SPDX-License-Identifier:.*GPL-/{/WITH Linux-syscall-note/!p}" $INFILE)" ]; then
> > -	echo "error: $INFILE: missing \"WITH Linux-syscall-note\" for SPDX-License-Identifier" >&2
> > -	exit 1
> > -fi
> > -
> > +# returns 9 if GPL SPDX-License-Identifier omits "WITH Linux-syscall-note"
> >  sed -E -e '
> > +	/SPDX-License-Identifier:.*GPL-/{/WITH Linux-syscall-note/! Q9}  
> 
> thanks, I like the idea of combining the sed calls.  According to the
> manual 'Q' (or 'q' with argument) is a GNU extension (non-POSIX).
> 
> May we expect GNU sed on all system running headers_install?

Good catch. 
I suspect there might already be some GNU sed specific code, based on
LKL's explicit use of gsed for builds on BSD hosts:
https://github.com/lkl/linux/commit/84f39699a6d1013d1866b8637977804e515d8d7e
I don't have any proof for mainline though (yet).

> >  	s/([[:space:](])(__user|__force|__iomem)[[:space:]]/\1/g
> >  	s/__attribute_const__([[:space:]]|$)/\1/g
> >  	s@^#include <linux/compiler.h>@@
> >  	s/(^|[^a-zA-Z0-9])__packed([^a-zA-Z0-9_]|$)/\1__attribute__((packed))\2/g
> >  	s/(^|[[:space:](])(inline|asm|volatile)([[:space:](]|$)/\1__\2__\3/g
> >  	s@#(ifndef|define|endif[[:space:]]*/[*])[[:space:]]*_UAPI@#\1 @
> > -' $INFILE > $TMPFILE || exit 1
> > +' $INFILE > $TMPFILE
> > +case $? in
> > +9)
> > +	echo "error: $INFILE: missing \"WITH Linux-syscall-note\" for SPDX-License-Identifier" >&2
> > +	exit 1
> > +	;;
> > +1)
> > +	exit 1
> > +	;;
> > +esac  
> 
> This silently ignores all other exit codes.  What about this:
> 
> 9)
> ...
> 	;;
> 0)	;;
> *)
> 	exit 1
> 	;;
> esac

Ack, I see that 2 and 4 returns are also possible. Your fix looks good
to me.
I'll squash it in and resend if I find any other GNU sed specific code.
Otherwise this change can probably be dropped.

Cheers, David