Re: [PATCH v2 07/13] kbuild: do not create built-in.a.symversions or lib.a.symversions

Kees Cook
2021-08-31T17:36:35Z

On Tue, Aug 31, 2021 at 04:39:58PM +0900, Masahiro Yamada wrote:
> Merge all *.o.symversions in scripts/link-vmlinux.sh instead of
> incrementally merging them in the unit of built-in.a or lib.a.
>
> This is a preparation for further code cleanups.
>
> The initial patch version was implemented in a shell script, but it
> was slow due to the slowness of the 'cat' command [1]. This version
> was implemented in Perl.
>
> [1]: https://lore.kernel.org/lkml/CAK7LNATyNAu6sa-TT9JXy=BXr5d2Q5K-sp-mVXXtJDuJyi6_bA@mail.gmail.com/
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

My earlier questions about speed here were actually with regard to
non-LTO builds. But it sounds like it's not a problem there either.

Reviewed-by: Kees Cook <keescook@chromium.org>

> ---
>
> scripts/Makefile.build | 10 ++------
> scripts/link-vmlinux.sh | 9 ++-----
> scripts/merge-symvers.pl | 52 ++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 56 insertions(+), 15 deletions(-)
> create mode 100644 scripts/merge-symvers.pl
>
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index afc906cd7256..3ad1b1227371 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -434,11 +434,8 @@ endif
> quiet_cmd_ar_builtin = AR $@
> cmd_ar_builtin = rm -f $@; $(AR) cDPrST $@ $(real-prereqs)
>
> -quiet_cmd_ar_and_symver = AR $@
> - cmd_ar_and_symver = $(cmd_update_lto_symversions); $(cmd_ar_builtin)
> -
> $(obj)/built-in.a: $(real-obj-y) FORCE
> - $(call if_changed,ar_and_symver)
> + $(call if_changed,ar_builtin)
>
> #
> # Rule to create modules.order file
> @@ -458,11 +455,8 @@ $(obj)/modules.order: $(obj-m) FORCE
> #
> # Rule to compile a set of .o files into one .a file (with symbol table)
> #
> -quiet_cmd_ar_lib = AR $@
> - cmd_ar_lib = $(cmd_update_lto_symversions); $(cmd_ar)
> -
> $(obj)/lib.a: $(lib-y) FORCE
> - $(call if_changed,ar_lib)
> + $(call if_changed,ar)
>
> # NOTE:
> # Do not replace $(filter %.o,^) with $(real-prereqs). When a single object
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index d74cee5c4326..0cc6a03f2cb1 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -57,13 +57,8 @@ gen_initcalls()
> gen_symversions()
> {
> info GEN .tmp_symversions.lds
> - rm -f .tmp_symversions.lds
> -
> - for o in ${KBUILD_VMLINUX_OBJS} ${KBUILD_VMLINUX_LIBS}; do
> - if [ -f ${o}.symversions ]; then
> - cat ${o}.symversions >> .tmp_symversions.lds
> - fi
> - done
> + ${PERL} scripts/merge-symvers.pl -a ${AR} -o .tmp_symversions.lds \
> + ${KBUILD_VMLINUX_OBJS} ${KBUILD_VMLINUX_LIBS}
> }
>
> # Link of vmlinux.o used for section mismatch analysis
> diff --git a/scripts/merge-symvers.pl b/scripts/merge-symvers.pl
> new file mode 100644
> index 000000000000..0bd092d24eff
> --- /dev/null
> +++ b/scripts/merge-symvers.pl
> @@ -0,0 +1,52 @@
> +#!/usr/bin/env perl
> +# SPDX-License-Identifier: GPL-2.0-only
> +
> +use autodie;
> +use strict;
> +use warnings;
> +use Getopt::Long 'GetOptions';
> +
> +my $ar;
> +my $output;
> +
> +GetOptions(
> + 'a|ar=s' => \$ar,
> + 'o|output=s' => \$output,
> +);
> +
> +# Collect all objects
> +my @objects;
> +
> +foreach (@ARGV) {
> + if (/\.o$/) {
> + # Some objects (head-y) are linked to vmlinux directly.
> + push(@objects, $_);
> + } elsif (/\.a$/) {
> + # Most of built-in objects are contained in built-in.a or lib.a.
> + # Use 'ar -t' to get the list of the contained objects.
> + $_ = `$ar -t $_`;
> + push(@objects, split(/\n/));
> + } else {
> + die "$_: unknown file type\n";
> + }
> +}
> +
> +open(my $out_fh, '>', "$output");
> +
> +foreach (@objects) {
> + # The symbol CRCs for foo/bar/baz.o is output to foo/bar/baz.o.symversions
> + s/(.*)/$1.symversions/;
> +
> + if (! -e $_) {
> + # .symversions does not exist if the object does not contain
> + # EXPORT_SYMBOL at all. Skip it.
> + next;
> + }
> +
> + open(my $in_fh, '<', "$_");
> + # Concatenate all the existing *.symversions files.
> + print $out_fh do { local $/; <$in_fh> };
> + close $in_fh;
> +}
> +
> +close $out_fh;
> --
> 2.30.2
>

--
Kees Cook