linux-doc.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 00/15] kbuild: refactor headers_install and support compile-test of UAPI headers
@ 2019-06-04 10:13 Masahiro Yamada
  2019-06-04 10:13 ` [PATCH 01/15] kbuild: remove headers_{install,check}_all Masahiro Yamada
                   ` (3 more replies)
  0 siblings, 4 replies; 9+ messages in thread
From: Masahiro Yamada @ 2019-06-04 10:13 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Randy Dunlap, Arnd Bergmann, David Howells, Sam Ravnborg,
	Jani Nikula, Masahiro Yamada, linux-doc, Jonathan Corbet,
	linux-riscv, Daniel Borkmann, linux-s390, Greentime Hu,
	Helge Deller, Vineet Gupta, Palmer Dabbelt, Christian Borntraeger,
	linux-snps-arc, Song Liu, James E.J. Bottomley, Yonghong Song,
	Benjamin Herrenschmidt, Michal Marek, Paul Mackerras,
	Martin KaFai Lau, Albert Ou, Vasily Gorbik, linux-parisc,
	Alexei Starovoitov, Michael Ellerman, Heiko Carstens,
	linux-kernel, Vincent Chen, netdev, bpf, linuxppc-dev


Multiple people have suggested to compile-test UAPI headers.

Currently, Kbuild provides simple sanity checks by headers_check
but they are not enough to catch bugs.

The most recent patch I know is David Howells' work:
https://patchwork.kernel.org/patch/10590203/

I agree that we need better tests for UAPI headers,
but I want to integrate it in a clean way.

The idea that has been in my mind is to compile each header
to make sure the selfcontainedness.

Recently, Jani Nikula proposed a new syntax 'header-test-y'.
https://patchwork.kernel.org/patch/10947005/

So, I implemented UAPI compile-testing on top of that.

When adding a new feature, cleaning the code first is a
good practice.

[1] Remove headers_install_all

This target installs UAPI headers of all architectures
in a single tree.
It does not make sense to compile test of headers from
multiple arches at the same time. Hence, removed.

[2] Split header installation into 'make headers' and 'make headers_install'

To compile-test UAPI headers, we need a work-directory somewhere
to save objects and .*.cmd files.

usr/include/ will be the work-directory.

Since we cannot pollute the final destination of headers_install,

I split the header installation into two stages.

'make headers' will build up
the ready-to-install headers in usr/include,
which will be also used as a work-directory for the compile-test.

'make headers_install' will copy headers
from usr/include to $(INSTALL_HDR_PATH)/include.

[3] Support compile-test of UAPI headers

This is implemented in usr/include/Makefile


Jani Nikula (1):
  kbuild: add support for ensuring headers are self-contained

Masahiro Yamada (14):
  kbuild: remove headers_{install,check}_all
  kbuild: remove stale dependency between Documentation/ and
    headers_install
  kbuild: make gdb_script depend on prepare0 instead of prepare
  kbuild: fix Kconfig prompt of CONFIG_HEADERS_CHECK
  kbuild: add CONFIG_HEADERS_INSTALL and loosen the dependency of
    samples
  kbuild: remove build_unifdef target in scripts/Makefile
  kbuild: build all prerequisite of headers_install simultaneously
  kbuild: add 'headers' target to build up ready-to-install uapi headers
  kbuild: re-implement Makefile.headersinst without directory descending
  kbuild: move hdr-inst shorthand to top Makefile
  kbuild: simplify scripts/headers_install.sh
  kbuild: deb-pkg: do not run headers_check
  fixup: kbuild: add support for ensuring headers are self-contained
  kbuild: compile test UAPI headers to ensure they are self-contained

 Documentation/kbuild/headers_install.txt |   7 --
 Documentation/kbuild/makefiles.txt       |  13 ++-
 Makefile                                 |  56 +++++-----
 arch/arc/configs/tb10x_defconfig         |   1 +
 arch/nds32/configs/defconfig             |   1 +
 arch/parisc/configs/a500_defconfig       |   1 +
 arch/parisc/configs/b180_defconfig       |   1 +
 arch/parisc/configs/c3000_defconfig      |   1 +
 arch/parisc/configs/default_defconfig    |   1 +
 arch/powerpc/configs/ppc6xx_defconfig    |   1 +
 arch/s390/configs/debug_defconfig        |   1 +
 include/uapi/{linux => }/Kbuild          |   6 +-
 init/Kconfig                             |  20 ++++
 lib/Kconfig.debug                        |  25 +++--
 samples/Kconfig                          |  14 ++-
 samples/Makefile                         |   4 +-
 scripts/Kbuild.include                   |   6 --
 scripts/Makefile                         |   5 -
 scripts/Makefile.build                   |   9 ++
 scripts/Makefile.headersinst             | 132 ++++++++++-------------
 scripts/Makefile.lib                     |   3 +
 scripts/cc-system-headers.sh             |   8 ++
 scripts/headers.sh                       |  29 -----
 scripts/headers_install.sh               |  48 ++++-----
 scripts/package/builddeb                 |   2 +-
 usr/.gitignore                           |   1 -
 usr/Makefile                             |   2 +
 usr/include/.gitignore                   |   3 +
 usr/include/Makefile                     | 132 +++++++++++++++++++++++
 29 files changed, 329 insertions(+), 204 deletions(-)
 rename include/uapi/{linux => }/Kbuild (77%)
 create mode 100755 scripts/cc-system-headers.sh
 delete mode 100755 scripts/headers.sh
 create mode 100644 usr/include/.gitignore
 create mode 100644 usr/include/Makefile

-- 
2.17.1


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH 01/15] kbuild: remove headers_{install,check}_all
  2019-06-04 10:13 [PATCH 00/15] kbuild: refactor headers_install and support compile-test of UAPI headers Masahiro Yamada
@ 2019-06-04 10:13 ` Masahiro Yamada
  2019-06-04 17:57   ` Sam Ravnborg
  2019-06-04 10:14 ` [PATCH 13/15] kbuild: add support for ensuring headers are self-contained Masahiro Yamada
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 9+ messages in thread
From: Masahiro Yamada @ 2019-06-04 10:13 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Randy Dunlap, Arnd Bergmann, David Howells, Sam Ravnborg,
	Jani Nikula, Masahiro Yamada, linux-doc, linux-kernel,
	Jonathan Corbet, Michal Marek

headers_install_all does not make much sense any more because different
architectures export different set of uapi/linux/ headers. As you see
in include/uapi/linux/Kbuild, the installation of a.out.h, kvm.h, and
kvm_para.h is arch-dependent. So, headers_install_all repeats the
installation/removal of them.

If somebody really thinks it is useful to do headers_install for all
architectures, it would be possible by small shell-scripting, but the
top Makefile do not have to provide entry targets just for that purpose.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

 Documentation/kbuild/headers_install.txt |  7 ------
 Documentation/kbuild/makefiles.txt       |  6 +----
 Makefile                                 | 15 ++----------
 scripts/headers.sh                       | 29 ------------------------
 4 files changed, 3 insertions(+), 54 deletions(-)
 delete mode 100755 scripts/headers.sh

diff --git a/Documentation/kbuild/headers_install.txt b/Documentation/kbuild/headers_install.txt
index f0153adb95e2..f07e34eab70b 100644
--- a/Documentation/kbuild/headers_install.txt
+++ b/Documentation/kbuild/headers_install.txt
@@ -39,12 +39,5 @@ INSTALL_HDR_PATH indicates where to install the headers. It defaults to
 An 'include' directory is automatically created inside INSTALL_HDR_PATH and
 headers are installed in 'INSTALL_HDR_PATH/include'.
 
-The command "make headers_install_all" exports headers for all architectures
-simultaneously.  (This is mostly of interest to distribution maintainers,
-who create an architecture-independent tarball from the resulting include
-directory.)  You also can use HDR_ARCH_LIST to specify list of architectures.
-Remember to provide the appropriate linux/asm directory via "mv" or "ln -s"
-before building a C library with headers exported this way.
-
 The kernel header export infrastructure is maintained by David Woodhouse
 <dwmw2@infradead.org>.
diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
index d65ad5746f94..bac301a73a86 100644
--- a/Documentation/kbuild/makefiles.txt
+++ b/Documentation/kbuild/makefiles.txt
@@ -897,11 +897,7 @@ When kbuild executes, the following steps are followed (roughly):
 --- 6.2 Add prerequisites to archheaders:
 
 	The archheaders: rule is used to generate header files that
-	may be installed into user space by "make header_install" or
-	"make headers_install_all".  In order to support
-	"make headers_install_all", this target has to be able to run
-	on an unconfigured tree, or a tree configured for another
-	architecture.
+	may be installed into user space by "make header_install".
 
 	It is run before "make archprepare" when run on the
 	architecture itself.
diff --git a/Makefile b/Makefile
index 004d67a4405f..7745bdd84861 100644
--- a/Makefile
+++ b/Makefile
@@ -1181,34 +1181,23 @@ headerdep:
 #Default location for installed headers
 export INSTALL_HDR_PATH = $(objtree)/usr
 
-# If we do an all arch process set dst to include/arch-$(SRCARCH)
-hdr-dst = $(if $(KBUILD_HEADERS), dst=include/arch-$(SRCARCH), dst=include)
-
 PHONY += archheaders archscripts
 
 PHONY += __headers
 __headers: $(version_h) scripts_basic uapi-asm-generic archheaders archscripts
 	$(Q)$(MAKE) $(build)=scripts build_unifdef
 
-PHONY += headers_install_all
-headers_install_all:
-	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/headers.sh install
-
 PHONY += headers_install
 headers_install: __headers
 	$(if $(wildcard $(srctree)/arch/$(SRCARCH)/include/uapi/asm/Kbuild),, \
 	  $(error Headers not exportable for the $(SRCARCH) architecture))
 	$(Q)$(MAKE) $(hdr-inst)=include/uapi dst=include
-	$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi $(hdr-dst)
-
-PHONY += headers_check_all
-headers_check_all: headers_install_all
-	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/headers.sh check
+	$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi dst=include
 
 PHONY += headers_check
 headers_check: headers_install
 	$(Q)$(MAKE) $(hdr-inst)=include/uapi dst=include HDRCHECK=1
-	$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi $(hdr-dst) HDRCHECK=1
+	$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi dst=include HDRCHECK=1
 
 ifdef CONFIG_HEADERS_CHECK
 all: headers_check
diff --git a/scripts/headers.sh b/scripts/headers.sh
deleted file mode 100755
index e0f883eb39a2..000000000000
--- a/scripts/headers.sh
+++ /dev/null
@@ -1,29 +0,0 @@
-#!/bin/sh
-# SPDX-License-Identifier: GPL-2.0
-# Run headers_$1 command for all suitable architectures
-
-# Stop on error
-set -e
-
-do_command()
-{
-	if [ -f ${srctree}/arch/$2/include/asm/Kbuild ]; then
-		make ARCH=$2 KBUILD_HEADERS=$1 headers_$1
-	else
-		printf "Ignoring arch: %s\n" ${arch}
-	fi
-}
-
-archs=${HDR_ARCH_LIST:-$(ls ${srctree}/arch)}
-
-for arch in ${archs}; do
-	case ${arch} in
-	um)        # no userspace export
-		;;
-	*)
-		if [ -d ${srctree}/arch/${arch} ]; then
-			do_command $1 ${arch}
-		fi
-		;;
-	esac
-done
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 13/15] kbuild: add support for ensuring headers are self-contained
  2019-06-04 10:13 [PATCH 00/15] kbuild: refactor headers_install and support compile-test of UAPI headers Masahiro Yamada
  2019-06-04 10:13 ` [PATCH 01/15] kbuild: remove headers_{install,check}_all Masahiro Yamada
@ 2019-06-04 10:14 ` Masahiro Yamada
  2019-06-04 12:42   ` [PATCH v2] " Jani Nikula
  2019-06-05  2:37 ` [PATCH 00/15] kbuild: refactor headers_install and support compile-test of UAPI headers Masahiro Yamada
  2019-06-18 15:46 ` Masahiro Yamada
  3 siblings, 1 reply; 9+ messages in thread
From: Masahiro Yamada @ 2019-06-04 10:14 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Randy Dunlap, Arnd Bergmann, David Howells, Sam Ravnborg,
	Jani Nikula, Chris Wilson, Masahiro Yamada, Michal Marek,
	linux-doc, linux-kernel, Jonathan Corbet

From: Jani Nikula <jani.nikula@intel.com>

Sometimes it's useful to be able to explicitly ensure certain headers
remain self-contained, i.e. that they are compilable as standalone
units, by including and/or forward declaring everything they depend on.

Add special target header-test-y where individual Makefiles can add
headers to be tested if CONFIG_HEADER_TEST is enabled. This will
generate a dummy C file per header that gets built as part of extra-y.

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Michal Marek <michal.lkml@markovi.net>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

 Documentation/kbuild/makefiles.txt |  7 +++++++
 init/Kconfig                       |  9 +++++++++
 scripts/Makefile.build             | 10 ++++++++++
 scripts/Makefile.lib               |  3 +++
 4 files changed, 29 insertions(+)

diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
index bac301a73a86..ca4b24ec0399 100644
--- a/Documentation/kbuild/makefiles.txt
+++ b/Documentation/kbuild/makefiles.txt
@@ -1018,6 +1018,13 @@ When kbuild executes, the following steps are followed (roughly):
 	In this example, extra-y is used to list object files that
 	shall be built, but shall not be linked as part of built-in.a.
 
+    header-test-y
+
+	header-test-y specifies headers (*.h) in the current directory that
+	should be compile tested to ensure they are self-contained,
+	i.e. compilable as standalone units. If CONFIG_HEADER_TEST is enabled,
+	this autogenerates dummy sources to include the headers, and builds them
+	as part of extra-y.
 
 --- 6.7 Commands useful for building a boot image
 
diff --git a/init/Kconfig b/init/Kconfig
index 36894c9fb420..02d8897b91fb 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -96,6 +96,15 @@ config COMPILE_TEST
 	  here. If you are a user/distributor, say N here to exclude useless
 	  drivers to be distributed.
 
+config HEADER_TEST
+	bool "Compile test headers that should be standalone compilable"
+	help
+	  Compile test headers listed in header-test-y target to ensure they are
+	  self-contained, i.e. compilable as standalone units.
+
+	  If you are a developer or tester and want to ensure the requested
+	  headers are self-contained, say Y here. Otherwise, choose N.
+
 config LOCALVERSION
 	string "Local version - append to kernel release"
 	help
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index ae9cf740633e..2b4d56483c2e 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -294,6 +294,16 @@ quiet_cmd_cc_lst_c = MKLST   $@
 $(obj)/%.lst: $(src)/%.c FORCE
 	$(call if_changed_dep,cc_lst_c)
 
+# Dummy C sources for header test (header-test-y target)
+# ---------------------------------------------------------------------------
+
+quiet_cmd_header_test = HDRTEST $@
+      cmd_header_test = echo "\#include \"$(<F)\"" > $@
+
+# FIXME: would be nice to be able to limit this implicit rule to header-test-y
+$(obj)/%.header_test.c: $(src)/%.h FORCE
+	$(call if_changed,header_test)
+
 # Compile assembler sources (.S)
 # ---------------------------------------------------------------------------
 
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index f1f38c8cdc74..60a739a22b9c 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -66,6 +66,9 @@ extra-y += $(patsubst %.dtb,%.dt.yaml, $(dtb-y))
 extra-$(CONFIG_OF_ALL_DTBS) += $(patsubst %.dtb,%.dt.yaml, $(dtb-))
 endif
 
+# Test self-contained headers
+extra-$(CONFIG_HEADER_TEST) += $(patsubst %.h,%.header_test.o,$(header-test-y))
+
 # Add subdir path
 
 extra-y		:= $(addprefix $(obj)/,$(extra-y))
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH v2] kbuild: add support for ensuring headers are self-contained
  2019-06-04 10:14 ` [PATCH 13/15] kbuild: add support for ensuring headers are self-contained Masahiro Yamada
@ 2019-06-04 12:42   ` Jani Nikula
  2019-06-04 17:25     ` Sam Ravnborg
  0 siblings, 1 reply; 9+ messages in thread
From: Jani Nikula @ 2019-06-04 12:42 UTC (permalink / raw)
  To: Masahiro Yamada, linux-kbuild
  Cc: Randy Dunlap, Arnd Bergmann, David Howells, Sam Ravnborg,
	Jani Nikula, Chris Wilson, Michal Marek, linux-doc, linux-kernel,
	Jonathan Corbet

Sometimes it's useful to be able to explicitly ensure certain headers
remain self-contained, i.e. that they are compilable as standalone
units, by including and/or forward declaring everything they depend on.

Add special target header-test-y where individual Makefiles can add
headers to be tested if CONFIG_HEADER_TEST is enabled. This will
generate a dummy C file per header that gets built as part of extra-y.

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>

---

v2: changes suggested by Masahiro:

http://mid.mail-archive.com/CAK7LNAT=nB0=at0X4OnHVKB=y7WwHGm4LXkrQnCw9HpjB5LooA@mail.gmail.com
---
 .gitignore                         | 1 +
 Documentation/dontdiff             | 1 +
 Documentation/kbuild/makefiles.txt | 7 +++++++
 Makefile                           | 1 +
 init/Kconfig                       | 9 +++++++++
 scripts/Makefile.build             | 9 +++++++++
 scripts/Makefile.lib               | 3 +++
 7 files changed, 31 insertions(+)

diff --git a/.gitignore b/.gitignore
index 7587ef56b92d..4bb60f0fa23b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -22,6 +22,7 @@
 *.elf
 *.gcno
 *.gz
+*.hdrtest.c
 *.i
 *.ko
 *.lex.c
diff --git a/Documentation/dontdiff b/Documentation/dontdiff
index 5eba889ea84d..554dfe4883d2 100644
--- a/Documentation/dontdiff
+++ b/Documentation/dontdiff
@@ -19,6 +19,7 @@
 *.grep
 *.grp
 *.gz
+*.hdrtest.c
 *.html
 *.i
 *.jpeg
diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
index d65ad5746f94..a31e54bd9ddd 100644
--- a/Documentation/kbuild/makefiles.txt
+++ b/Documentation/kbuild/makefiles.txt
@@ -1022,6 +1022,13 @@ When kbuild executes, the following steps are followed (roughly):
 	In this example, extra-y is used to list object files that
 	shall be built, but shall not be linked as part of built-in.a.
 
+    header-test-y
+
+	header-test-y specifies headers (*.h) in the current directory that
+	should be compile tested to ensure they are self-contained,
+	i.e. compilable as standalone units. If CONFIG_HEADER_TEST is enabled,
+	this autogenerates dummy sources to include the headers, and builds them
+	as part of extra-y.
 
 --- 6.7 Commands useful for building a boot image
 
diff --git a/Makefile b/Makefile
index 11358153d8f2..b347be697fbb 100644
--- a/Makefile
+++ b/Makefile
@@ -1646,6 +1646,7 @@ clean: $(clean-dirs)
 		-o -name '*.dwo' -o -name '*.lst' \
 		-o -name '*.su'  \
 		-o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \
+		-o -name '*.hdrtest.c' \
 		-o -name '*.lex.c' -o -name '*.tab.[ch]' \
 		-o -name '*.asn1.[ch]' \
 		-o -name '*.symtypes' -o -name 'modules.order' \
diff --git a/init/Kconfig b/init/Kconfig
index 36894c9fb420..02d8897b91fb 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -96,6 +96,15 @@ config COMPILE_TEST
 	  here. If you are a user/distributor, say N here to exclude useless
 	  drivers to be distributed.
 
+config HEADER_TEST
+	bool "Compile test headers that should be standalone compilable"
+	help
+	  Compile test headers listed in header-test-y target to ensure they are
+	  self-contained, i.e. compilable as standalone units.
+
+	  If you are a developer or tester and want to ensure the requested
+	  headers are self-contained, say Y here. Otherwise, choose N.
+
 config LOCALVERSION
 	string "Local version - append to kernel release"
 	help
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index ae9cf740633e..ee0319560513 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -294,6 +294,15 @@ quiet_cmd_cc_lst_c = MKLST   $@
 $(obj)/%.lst: $(src)/%.c FORCE
 	$(call if_changed_dep,cc_lst_c)
 
+# Dummy C sources for header test (header-test-y target)
+# ---------------------------------------------------------------------------
+
+quiet_cmd_header_test = HDRTEST $@
+      cmd_header_test = echo "\#include \"$*.h\"" > $@
+
+$(obj)/%.hdrtest.c:
+	$(call cmd,header_test)
+
 # Compile assembler sources (.S)
 # ---------------------------------------------------------------------------
 
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index f1f38c8cdc74..3e630fcaffd1 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -66,6 +66,9 @@ extra-y += $(patsubst %.dtb,%.dt.yaml, $(dtb-y))
 extra-$(CONFIG_OF_ALL_DTBS) += $(patsubst %.dtb,%.dt.yaml, $(dtb-))
 endif
 
+# Test self-contained headers
+extra-$(CONFIG_HEADER_TEST) += $(patsubst %.h,%.hdrtest.o,$(header-test-y))
+
 # Add subdir path
 
 extra-y		:= $(addprefix $(obj)/,$(extra-y))
-- 
2.20.1


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [PATCH v2] kbuild: add support for ensuring headers are self-contained
  2019-06-04 12:42   ` [PATCH v2] " Jani Nikula
@ 2019-06-04 17:25     ` Sam Ravnborg
  2019-06-18 15:45       ` Masahiro Yamada
  0 siblings, 1 reply; 9+ messages in thread
From: Sam Ravnborg @ 2019-06-04 17:25 UTC (permalink / raw)
  To: Jani Nikula
  Cc: Masahiro Yamada, linux-kbuild, Randy Dunlap, Arnd Bergmann,
	David Howells, Chris Wilson, Michal Marek, linux-doc,
	linux-kernel, Jonathan Corbet

On Tue, Jun 04, 2019 at 03:42:48PM +0300, Jani Nikula wrote:
> Sometimes it's useful to be able to explicitly ensure certain headers
> remain self-contained, i.e. that they are compilable as standalone
> units, by including and/or forward declaring everything they depend on.
> 
> Add special target header-test-y where individual Makefiles can add
> headers to be tested if CONFIG_HEADER_TEST is enabled. This will
> generate a dummy C file per header that gets built as part of extra-y.
> 
> Cc: Chris Wilson <chris@chris-wilson.co.uk>
> Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
> Cc: Michal Marek <michal.lkml@markovi.net>
> Cc: Sam Ravnborg <sam@ravnborg.org>
> Signed-off-by: Jani Nikula <jani.nikula@intel.com>

Looks good, thanks.

Reviewed-by: Sam Ravnborg <sam@ravnborg.org>


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 01/15] kbuild: remove headers_{install,check}_all
  2019-06-04 10:13 ` [PATCH 01/15] kbuild: remove headers_{install,check}_all Masahiro Yamada
@ 2019-06-04 17:57   ` Sam Ravnborg
  0 siblings, 0 replies; 9+ messages in thread
From: Sam Ravnborg @ 2019-06-04 17:57 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: linux-kbuild, Randy Dunlap, Arnd Bergmann, David Howells,
	Jani Nikula, linux-doc, linux-kernel, Jonathan Corbet,
	Michal Marek

On Tue, Jun 04, 2019 at 07:13:55PM +0900, Masahiro Yamada wrote:
> headers_install_all does not make much sense any more because different
> architectures export different set of uapi/linux/ headers. As you see
> in include/uapi/linux/Kbuild, the installation of a.out.h, kvm.h, and
> kvm_para.h is arch-dependent. So, headers_install_all repeats the
> installation/removal of them.
> 
> If somebody really thinks it is useful to do headers_install for all
> architectures, it would be possible by small shell-scripting, but the
> top Makefile do not have to provide entry targets just for that purpose.
Agreed, good to see this be dropped.
Acked-by: Sam Ravnborg <sam@ravnborg.org>

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 00/15] kbuild: refactor headers_install and support compile-test of UAPI headers
  2019-06-04 10:13 [PATCH 00/15] kbuild: refactor headers_install and support compile-test of UAPI headers Masahiro Yamada
  2019-06-04 10:13 ` [PATCH 01/15] kbuild: remove headers_{install,check}_all Masahiro Yamada
  2019-06-04 10:14 ` [PATCH 13/15] kbuild: add support for ensuring headers are self-contained Masahiro Yamada
@ 2019-06-05  2:37 ` Masahiro Yamada
  2019-06-18 15:46 ` Masahiro Yamada
  3 siblings, 0 replies; 9+ messages in thread
From: Masahiro Yamada @ 2019-06-05  2:37 UTC (permalink / raw)
  To: Linux Kbuild mailing list
  Cc: Song Liu, open list:DOCUMENTATION, Benjamin Herrenschmidt,
	Palmer Dabbelt, Heiko Carstens, Alexei Starovoitov, David Howells,
	Paul Mackerras, linux-riscv, Vincent Chen, Sam Ravnborg,
	linux-s390, Vasily Gorbik, Daniel Borkmann, Jonathan Corbet,
	Michael Ellerman, Helge Deller, Christian Borntraeger,
	Yonghong Song, arcml, Albert Ou, Arnd Bergmann, Jani Nikula,
	Greentime Hu, James E.J. Bottomley, Michal Marek, linux-parisc,
	Vineet Gupta, Randy Dunlap, Linux Kernel Mailing List, Networking,
	bpf, linuxppc-dev, Martin KaFai Lau

On Tue, Jun 4, 2019 at 7:15 PM Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
>
>
> Multiple people have suggested to compile-test UAPI headers.
>
> Currently, Kbuild provides simple sanity checks by headers_check
> but they are not enough to catch bugs.
>
> The most recent patch I know is David Howells' work:
> https://patchwork.kernel.org/patch/10590203/
>
> I agree that we need better tests for UAPI headers,
> but I want to integrate it in a clean way.
>
> The idea that has been in my mind is to compile each header
> to make sure the selfcontainedness.


For convenience, I pushed this series at

git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
uapi-header-test-v1

(13/15 was replaced with v2)


If you want to test it quickly, please check-out it, then

  $ make -j8 allmodconfig usr/

(As I noted in the commit log, you need to use
a compiler that provides <stdlib.h>, <sys/time.h>, etc.)


-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH v2] kbuild: add support for ensuring headers are self-contained
  2019-06-04 17:25     ` Sam Ravnborg
@ 2019-06-18 15:45       ` Masahiro Yamada
  0 siblings, 0 replies; 9+ messages in thread
From: Masahiro Yamada @ 2019-06-18 15:45 UTC (permalink / raw)
  To: Sam Ravnborg, Jani Nikula
  Cc: Linux Kbuild mailing list, Randy Dunlap, Arnd Bergmann,
	David Howells, Chris Wilson, Michal Marek,
	open list:DOCUMENTATION, Linux Kernel Mailing List,
	Jonathan Corbet

On Wed, Jun 5, 2019 at 2:26 AM Sam Ravnborg <sam@ravnborg.org> wrote:
>
> On Tue, Jun 04, 2019 at 03:42:48PM +0300, Jani Nikula wrote:
> > Sometimes it's useful to be able to explicitly ensure certain headers
> > remain self-contained, i.e. that they are compilable as standalone
> > units, by including and/or forward declaring everything they depend on.
> >
> > Add special target header-test-y where individual Makefiles can add
> > headers to be tested if CONFIG_HEADER_TEST is enabled. This will
> > generate a dummy C file per header that gets built as part of extra-y.
> >
> > Cc: Chris Wilson <chris@chris-wilson.co.uk>
> > Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
> > Cc: Michal Marek <michal.lkml@markovi.net>
> > Cc: Sam Ravnborg <sam@ravnborg.org>
> > Signed-off-by: Jani Nikula <jani.nikula@intel.com>
>
> Looks good, thanks.
>
> Reviewed-by: Sam Ravnborg <sam@ravnborg.org>
>

Applied to linux-kbuild.
Thanks.


-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 00/15] kbuild: refactor headers_install and support compile-test of UAPI headers
  2019-06-04 10:13 [PATCH 00/15] kbuild: refactor headers_install and support compile-test of UAPI headers Masahiro Yamada
                   ` (2 preceding siblings ...)
  2019-06-05  2:37 ` [PATCH 00/15] kbuild: refactor headers_install and support compile-test of UAPI headers Masahiro Yamada
@ 2019-06-18 15:46 ` Masahiro Yamada
  3 siblings, 0 replies; 9+ messages in thread
From: Masahiro Yamada @ 2019-06-18 15:46 UTC (permalink / raw)
  To: Linux Kbuild mailing list
  Cc: Song Liu, open list:DOCUMENTATION, Benjamin Herrenschmidt,
	Palmer Dabbelt, Heiko Carstens, Alexei Starovoitov, David Howells,
	Paul Mackerras, linux-riscv, Vincent Chen, Sam Ravnborg,
	linux-s390, Vasily Gorbik, Daniel Borkmann, Jonathan Corbet,
	Michael Ellerman, Helge Deller, Christian Borntraeger,
	Yonghong Song, arcml, Albert Ou, Arnd Bergmann, Jani Nikula,
	Greentime Hu, James E.J. Bottomley, Michal Marek, linux-parisc,
	Vineet Gupta, Randy Dunlap, Linux Kernel Mailing List, Networking,
	bpf, linuxppc-dev, Martin KaFai Lau

On Tue, Jun 4, 2019 at 7:15 PM Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
>
>
> Multiple people have suggested to compile-test UAPI headers.
>
> Currently, Kbuild provides simple sanity checks by headers_check
> but they are not enough to catch bugs.
>
> The most recent patch I know is David Howells' work:
> https://patchwork.kernel.org/patch/10590203/
>
> I agree that we need better tests for UAPI headers,
> but I want to integrate it in a clean way.
>
> The idea that has been in my mind is to compile each header
> to make sure the selfcontainedness.
>
> Recently, Jani Nikula proposed a new syntax 'header-test-y'.
> https://patchwork.kernel.org/patch/10947005/
>
> So, I implemented UAPI compile-testing on top of that.
>
> When adding a new feature, cleaning the code first is a
> good practice.
>
> [1] Remove headers_install_all
>
> This target installs UAPI headers of all architectures
> in a single tree.
> It does not make sense to compile test of headers from
> multiple arches at the same time. Hence, removed.
>
> [2] Split header installation into 'make headers' and 'make headers_install'
>
> To compile-test UAPI headers, we need a work-directory somewhere
> to save objects and .*.cmd files.
>
> usr/include/ will be the work-directory.
>
> Since we cannot pollute the final destination of headers_install,
>
> I split the header installation into two stages.
>
> 'make headers' will build up
> the ready-to-install headers in usr/include,
> which will be also used as a work-directory for the compile-test.
>
> 'make headers_install' will copy headers
> from usr/include to $(INSTALL_HDR_PATH)/include.
>
> [3] Support compile-test of UAPI headers
>
> This is implemented in usr/include/Makefile
>
>
> Jani Nikula (1):
>   kbuild: add support for ensuring headers are self-contained
>
> Masahiro Yamada (14):
>   kbuild: remove headers_{install,check}_all
>   kbuild: remove stale dependency between Documentation/ and
>     headers_install
>   kbuild: make gdb_script depend on prepare0 instead of prepare
>   kbuild: fix Kconfig prompt of CONFIG_HEADERS_CHECK
>   kbuild: add CONFIG_HEADERS_INSTALL and loosen the dependency of
>     samples
>   kbuild: remove build_unifdef target in scripts/Makefile
>   kbuild: build all prerequisite of headers_install simultaneously
>   kbuild: add 'headers' target to build up ready-to-install uapi headers
>   kbuild: re-implement Makefile.headersinst without directory descending
>   kbuild: move hdr-inst shorthand to top Makefile
>   kbuild: simplify scripts/headers_install.sh
>   kbuild: deb-pkg: do not run headers_check
>   fixup: kbuild: add support for ensuring headers are self-contained
>   kbuild: compile test UAPI headers to ensure they are self-contained

Series, applied to linux-kbuild.


>  Documentation/kbuild/headers_install.txt |   7 --
>  Documentation/kbuild/makefiles.txt       |  13 ++-
>  Makefile                                 |  56 +++++-----
>  arch/arc/configs/tb10x_defconfig         |   1 +
>  arch/nds32/configs/defconfig             |   1 +
>  arch/parisc/configs/a500_defconfig       |   1 +
>  arch/parisc/configs/b180_defconfig       |   1 +
>  arch/parisc/configs/c3000_defconfig      |   1 +
>  arch/parisc/configs/default_defconfig    |   1 +
>  arch/powerpc/configs/ppc6xx_defconfig    |   1 +
>  arch/s390/configs/debug_defconfig        |   1 +
>  include/uapi/{linux => }/Kbuild          |   6 +-
>  init/Kconfig                             |  20 ++++
>  lib/Kconfig.debug                        |  25 +++--
>  samples/Kconfig                          |  14 ++-
>  samples/Makefile                         |   4 +-
>  scripts/Kbuild.include                   |   6 --
>  scripts/Makefile                         |   5 -
>  scripts/Makefile.build                   |   9 ++
>  scripts/Makefile.headersinst             | 132 ++++++++++-------------
>  scripts/Makefile.lib                     |   3 +
>  scripts/cc-system-headers.sh             |   8 ++
>  scripts/headers.sh                       |  29 -----
>  scripts/headers_install.sh               |  48 ++++-----
>  scripts/package/builddeb                 |   2 +-
>  usr/.gitignore                           |   1 -
>  usr/Makefile                             |   2 +
>  usr/include/.gitignore                   |   3 +
>  usr/include/Makefile                     | 132 +++++++++++++++++++++++
>  29 files changed, 329 insertions(+), 204 deletions(-)
>  rename include/uapi/{linux => }/Kbuild (77%)
>  create mode 100755 scripts/cc-system-headers.sh
>  delete mode 100755 scripts/headers.sh
>  create mode 100644 usr/include/.gitignore
>  create mode 100644 usr/include/Makefile
>
> --
> 2.17.1
>
>
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2019-06-18 15:46 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-06-04 10:13 [PATCH 00/15] kbuild: refactor headers_install and support compile-test of UAPI headers Masahiro Yamada
2019-06-04 10:13 ` [PATCH 01/15] kbuild: remove headers_{install,check}_all Masahiro Yamada
2019-06-04 17:57   ` Sam Ravnborg
2019-06-04 10:14 ` [PATCH 13/15] kbuild: add support for ensuring headers are self-contained Masahiro Yamada
2019-06-04 12:42   ` [PATCH v2] " Jani Nikula
2019-06-04 17:25     ` Sam Ravnborg
2019-06-18 15:45       ` Masahiro Yamada
2019-06-05  2:37 ` [PATCH 00/15] kbuild: refactor headers_install and support compile-test of UAPI headers Masahiro Yamada
2019-06-18 15:46 ` Masahiro Yamada

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).