Re: [RFC PATCH 00/11] output a valid shell script when running 'make -n'

[Vegard Nossum <vegard.nossum@oracle.com>]

Hi,

I didn't receive a single comment on this patch series since I submitted
it a month ago, but I understand it's been busy with conferences and the
merge window.

I've rebased it on latest mainline (including the kbuild-6.12 merge) and
there's just one tiny trivial conflict. Can/should I wait for -rc1 and
resubmit it for inclusion then?

Thanks,


Vegard

On 19/08/2024 18:02, Vegard Nossum wrote:
> This patch series lets 'make -n' output a shell script that can be
> used to build the kernel without any further use of make. For example:
> 
>      make defconfig
> 
>      # ensure some build prerequisites are built
>      make prepare
> 
>      # generate build script
>      make -n | tee build.sh
> 
>      # excecute build script
>      bash -eux build.sh
> 
> The purpose of this is to take a step towards defeating the insertion of
> backdoors at build time (see [1]). Some of the benefits of separating the
> build script from the build system are:
> 
>   - we can invoke make in a restricted environment (e.g. mostly read-only
>     kernel tree),
> 
>   - we have an audit log of the exact commands that run during the build
>     process; although it's true that the build script wouldn't be useful
>     for either production or development builds (as it doesn't support
>     incremental rebuilds or parallel builds), it would allow you to
>     rebuild an existing kernel and compare the resulting binary for
>     discrepancies to the original build,
> 
>   - the audit log can be stored (e.g. in git) and changes to it over time
>     can themselves be audited (e.g. by looking at diffs),
> 
>   - there's a lot fewer places to hide malicious code in a straight-line
>     shell script that makes minimal use of variables and helper functions.
>     You also cannot inject fragments of Makefile code through environment
>     variables (again, see [1]).
> 
> Alternative ways to achieve some of the same things would be:
> 
>   - the existing compile_commands.json infrastructure; unfortunately this
>     does not include most of the steps performed during a build (such as
>     linking vmlinux) and does not really allow you to reproduce/verify the
>     full build,
> 
>   - simply running something like strace -f -e trace=execve make; however,
>     this also does not result in something that can be easily played back;
>     at the very least it would need to be heavily filtered and processed
>     to account for data passed in environment variables and things like
>     temporary files used by the compiler.
> 
> This implementation works as follows:
> 
>   - 'make -n' (AKA --dry-run) by default prints out the commands that make
>     runs; this output is modified to be usable as a shell script,
> 
>   - we output 'make() { :; }' at the start of the script in order to make
>     all 'make' invocations in the resulting build script no-ops (GNU Make
>     will actually execute -- and print -- all recipe lines that include
>     $(MAKE), even when invoked with -n).
> 
>   - we simplify the makefile rules in some cases to make the shell script
>     more readable; for example, we don't need the logic that extracts
>     dependencies from .c files (since that is only used by 'make' itself
>     when determining what to rebuild) or the logic that generates .cmd
>     files,
> 
> This patch is WIP and may not produce a working shell script in all
> circumstances. For example, while plain 'make -n' works for me, other
> make targets (e.g. 'make -n htmldocs') are not at all guaranteed to
> produce meaningful output; certain kernel configs may also not work,
> especially those that rely on external tools like e.g. Rust.
> 
> [1]: https://www.openwall.com/lists/oss-security/2024/04/17/3
> [2]: https://www.gnu.org/software/make/manual/make.html#Testing-Flags
> 
> 
> Vegard
> 
> ---
> 
> Vegard Nossum (11):
>    kbuild: ignore .config rule for make --always-make
>    kbuild: document some prerequisites
>    kbuild: pass KERNELVERSION and LOCALVERSION explicitly to
>      setlocalversion
>    kbuild: don't execute .ko recipe in --dry-run mode
>    kbuild: execute modules.order recipe in --dry-run mode
>    kbuild: set $dry_run when running in --dry-run mode
>    kbuild: define 'make' as a no-op in --dry-run mode
>    kbuild: make link-vmlinux.sh respect $dry_run
>    kbuild: simplify commands in --dry-run mode
>    kbuild: don't test for file presence in --dry-run mode
>    kbuild: suppress echoing of commands in --dry-run mode
> 
>   Makefile                          | 28 +++++++++++++++++---
>   arch/x86/boot/compressed/Makefile |  6 +++++
>   scripts/Kbuild.include            | 27 +++++++++++++++++++
>   scripts/Makefile.build            |  2 +-
>   scripts/Makefile.modfinal         |  9 +++++--
>   scripts/Makefile.modpost          |  8 ++++--
>   scripts/Makefile.vmlinux          | 22 ++++++++++++++--
>   scripts/Makefile.vmlinux_o        |  3 +++
>   scripts/link-vmlinux.sh           | 44 ++++++++++++++++++++-----------
>   9 files changed, 123 insertions(+), 26 deletions(-)
>