[Buildroot] [PATCH 00/23] Support building the Linux kernel out of tree with LINUX_OVERRIDE_SRCDIR

* [Buildroot] [PATCH 00/23] Support building the Linux kernel out of tree with LINUX_OVERRIDE_SRCDIR
@ 2026-10-23 21:47 Florian Fainelli via buildroot
  2026-10-23 21:47 ` [Buildroot] [PATCH 01/23] package: prepare removal of <pkg>_DIR Florian Fainelli via buildroot
                   ` (8 more replies)
  0 siblings, 9 replies; 47+ messages in thread
From: Florian Fainelli via buildroot @ 2026-10-23 21:47 UTC (permalink / raw)
  To: buildroot
  Cc: Eric Le Bihan, Gary Bisson, Jérôme Pouiller,
	Florian Fainelli, Bernd Kuhls, Thomas Perale, Maxime Chevallier,
	Refik Tuzakli, Francois Perrad, Thomas Petazzoni,
	Arnout Vandecappelle, Petr Vorel, Spenser Gilliland,
	Giulio Benetti, Pascal de Bruijn, Luca Ceresoli, Romain Naour,
	Mahyar Koshkouei

Hello,

This series lets a package that is built from an OVERRIDE_SRCDIR be built
out of tree, i.e. without rsync'ing its sources into output/build/<pkg>-custom/,
and enables it for the Linux kernel.

For a large source tree such as the kernel, the rsync step is slow, doubles
the disk usage, and leaves the developer's tree and the build directory out
of sync. With this series, with LINUX_OVERRIDE_SRCDIR set, the kernel is
configured and built with O=output/build/linux-custom and the sources in
the overridden directory are never written to.

Most of the groundwork comes from the out-of-tree work Thomas Petazzoni
and Yann E. Morin did in 2015-2017, which was never merged. It has been
rebased on current master, and extended to cover the custom logo, custom
device tree files, and linux-tools. Their authorship is preserved.

Series overview
===============

Patches 1-8, infra preparation:
  - introduce <pkg>_SRCDIR (where the sources are) and <pkg>_BUILDDIR
    (where the package is built, and where the stamp files live), both
    defaulting to <pkg>_DIR, so nothing changes for existing packages;
  - fix up the users of LINUX_DIR that need the build directory
    (boot-wrapper-aarch64, mxs-bootlets, the linux extensions), and run
    the kernel version probing from the build directory;
  - introduce <pkg>_SRCSUBDIR (<pkg>_SRCDIR/<pkg>_SUBDIR) and convert the
    infras (cargo, hare, luarocks, meson, perl, waf) and gcc-final to it.

Patch 9, the kernel:
  - add <pkg>_SUPPORTS_OUT_OF_TREE. When it is YES and <pkg>_OVERRIDE_SRCDIR
    is set, the rsync step is skipped, the stamp files go in the build
    directory and kconfig is called with -C $(SRCDIR) O=$(BUILDDIR);
  - opt linux in, in the same patch. Kbuild requires a clean source tree
    for O=: a tree with a stale .config, include/config or
    arch/*/include/generated is rejected by kbuild itself.

Patches 10-19, host/target consistency. When the sources are shared between
the host and the target variant of a package, they must not be patched
differently, so:
  - sync the host and target patch hooks, and fix omniorb, lua and perl
    which did different things between the two variants;
  - have host packages inherit the extra downloads and apply the libtool
    patch to host libtool;
  - warn about packages that autoreconf without the libtool patch;
  - replace $(PKG)_DIR with $(@D) where this is possible.

Patches 20-21, the parts that write to the kernel sources:
  - the custom logo (BR2_LINUX_KERNEL_CUSTOM_LOGO_PATH) and custom device
    tree files (BR2_LINUX_KERNEL_CUSTOM_DTS_PATH) are copied into the kernel
    sources. For an out-of-tree build, linux is then built from a "shadow"
    tree: real directories whose files are symbolic links to the original
    tree, created by the new support/scripts/symlink-tree. Files are
    replaced rather than written through the links, so the overridden
    tree is never modified;
  - linux-tools checks which tools are available in $(LINUX_SRCDIR)/tools,
    and builds them in a shadow of that directory, because several of the
    kernel tools build in place, and some (mm, tmon) do not support
    OUTPUT/O= at all. This also covers the PCI endpoint selftests path.

Patches 22-23:
  - packages that read kernel headers from the source tree use
    $(LINUX_SRCDIR) instead of $(LINUX_DIR);
  - document the feature and the new variables in the manual.

Things this series intentionally does not do
============================================

  - Only linux opts in. Other packages are unchanged and still rsync.
    A package opts in with <pkg>_SUPPORTS_OUT_OF_TREE = YES.
  - The "hooks sanity check" and "host inherits the target hooks" patches
    of the original series are not included: they do not work with the
    infras that register per-name hooks (e.g. golang).
  - The Python 2 and asterisk patches of the original series are dropped:
    those packages are gone or no longer need them.

Testing
=======

All of the following build and, where noted, boot, with the series
applied on top of master (fd9c348c227e):

  - raspberrypi4_64_defconfig, stock (in-tree kernel);
  - raspberrypi4_64_defconfig with the Raspberry Pi kernel built out of tree
    through LINUX_OVERRIDE_SRCDIR: the Image differs from the in-tree one
    only by the embedded build timestamp, the dtbs are identical;
  - qemu_x86_64 and qemu_arm_vexpress, stock, and with an out-of-tree
    kernel; the out-of-tree ones boot to a login prompt in QEMU;
  - a wide package set on arm64 (autotools, cmake, meson, perl, python,
    cargo, golang, waf, luarocks, hare, kernel modules, asterisk, ffmpeg,
    libvpx, u-boot, and the linux-tools gpio, iio, perf, pci, cpupower,
    rtla, tmon, mm, usbtools), both with an in-tree and an out-of-tree
    kernel: the content of target/ is identical;
  - a custom logo and a custom device tree file with a #include, out of
    tree, with the overridden tree left unmodified;
  - the same package set in the Buildroot CI docker image, offline;
  - every patch of the series builds, tested on a small package set.

Known issues, not caused by this series and identical with and without it:
xtables-addons does not build with the Raspberry Pi 6.12 kernel, and
some linux-tools selftests (mm) fail to link. I have not tested the
selftests in the docker image, as the container has no python yaml.

Not tested: the other kernel extensions (rtai, xenomai), which are
adapted but not built, and booting on real hardware.

Changes
=======

This is the first version.

Regards,
Florian

Florian Fainelli (6):
  package: use <pkg>_SRCSUBDIR where <pkg>_SRCDIR used to include
    _SUBDIR
  linux: build out of tree when LINUX_OVERRIDE_SRCDIR is set
  linux: support custom logo and devicetree files in out-of-tree builds
  package/linux-tools: support an out-of-tree kernel build
  package: read kernel headers from LINUX_SRCDIR
  docs/manual: document out-of-tree package builds and the new srcdir
    variables

Thomas Petazzoni (7):
  package: prepare removal of <pkg>_DIR
  infra: introduce a global SRC_DIR variable
  packages: sync host and target patch hooks
  omniorb: avoid target-only patch hook
  lua: rework post-patch hook usage
  package/libtool: use post-patch hooks
  package/{ffmpeg,libvpx}: use $(@D) instead of $(pkg_SRCDIR) when
    possible

Yann E. MORIN (10):
  package: fix <pkg>_SRCDIR and <pkg>_BUILDDIR values
  boot/boot-wrapper-aarch64: needs LINUX_BUILDDIR
  boot/mxs-bootlets: needs (BAREBOX,LINUX,UBOOT)_BUILDDIR
  linux/extensions: point to proper _SRCDIR and @D
  linux/linux: LINUX_VERSION_PROBED should run from build directory
  infra: host packages inherit the target extra downloads
  package/libtool: also apply libtool-patch to host libtool
  package/perl: apply the perl-cross even for the host variant
  infra/autotools: warn for package that autoreconf without libtool
    patch
  packages: replace $(PKG)_DIR with @D where possible

 Makefile                                      |  1 +
 .../boot-wrapper-aarch64.mk                   |  8 +-
 boot/mxs-bootlets/mxs-bootlets.mk             |  6 +-
 docs/manual/adding-packages-generic.adoc      | 11 +++
 docs/manual/using-buildroot-development.adoc  | 30 ++++++++
 linux/linux-ext-rtai.mk                       |  8 +-
 linux/linux-ext-xenomai.mk                    |  4 +-
 linux/linux.mk                                | 40 ++++++++--
 package/a10disp/a10disp.mk                    |  2 +-
 package/android-tools/android-tools.mk        |  1 -
 package/binutils/binutils.mk                  |  1 -
 package/ffmpeg/ffmpeg.mk                      |  2 +-
 .../imx-alsa-plugins/imx-alsa-plugins.mk      |  2 +-
 package/freescale-imx/imx-lib/imx-lib.mk      |  2 +-
 package/gcc/gcc-final/gcc-final.mk            |  2 +-
 package/gdb/gdb.mk                            |  1 -
 package/gstreamer1/gst1-imx/gst1-imx.mk       |  2 +-
 package/icu/icu.mk                            |  1 +
 package/libtool/libtool.mk                    |  5 +-
 package/libvpx/libvpx.mk                      |  2 +-
 package/linux-backports/linux-backports.mk    |  2 +-
 package/linux-tools/linux-tool-cpupower.mk.in |  8 +-
 package/linux-tools/linux-tool-gpio.mk.in     |  6 +-
 package/linux-tools/linux-tool-hv.mk.in       |  6 +-
 package/linux-tools/linux-tool-iio.mk.in      |  6 +-
 package/linux-tools/linux-tool-mm.mk.in       | 10 +--
 package/linux-tools/linux-tool-pci.mk.in      | 12 +--
 package/linux-tools/linux-tool-perf.mk.in     | 29 +++++---
 package/linux-tools/linux-tool-rtla.mk.in     | 10 +--
 .../linux-tools/linux-tool-selftests.mk.in    | 15 ++--
 package/linux-tools/linux-tool-tmon.mk.in     |  8 +-
 package/linux-tools/linux-tool-usbtools.mk.in |  6 +-
 package/linux-tools/linux-tools.mk            | 20 ++++-
 package/lua/lua.mk                            |  9 ++-
 package/ncftp/ncftp.mk                        |  2 +-
 .../omniorb/0002-optional-appl-services.patch | 29 ++++++++
 package/omniorb/omniorb.mk                    | 26 +++----
 package/pciutils/pciutils.mk                  |  2 +-
 package/perl/perl.mk                          |  2 +
 package/php/php.mk                            |  2 +-
 package/pkg-autotools.mk                      | 28 +++----
 package/pkg-cargo.mk                          |  8 +-
 package/pkg-cmake.mk                          |  8 +-
 package/pkg-generic.mk                        | 74 ++++++++++++-------
 package/pkg-hare.mk                           |  8 +-
 package/pkg-kconfig.mk                        | 48 ++++++------
 package/pkg-luarocks.mk                       | 10 +--
 package/pkg-meson.mk                          | 26 +++----
 package/pkg-perl.mk                           | 12 +--
 package/pkg-waf.mk                            |  8 +-
 .../xdriver_xf86-video-imx.mk                 |  2 +-
 support/scripts/symlink-tree                  | 58 +++++++++++++++
 52 files changed, 422 insertions(+), 209 deletions(-)
 create mode 100644 package/omniorb/0002-optional-appl-services.patch
 create mode 100755 support/scripts/symlink-tree

-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 47+ messages in thread