Re: # Toplevel Makefile doesn't process module build correctly on recursive make calls

[Marcos Paulo de Souza <mpdesouza@suse.com>]

On Sat, 14 Oct 2023 17:35:55 +0900 Masahiro Yamada <masahiroy@kernel.org> wrote:

> On Tue, Oct 10, 2023 at 5:43 AM Marcos Paulo de Souza <mpdesouza@suse.de> wrote:
> >
> > Hi all,
> >
> > I found an issue while moving the livepatch kselftest modules to be built on the
> > fly, instead of building them on kernel building.
> >
> > If, for some reason, there is a recursive make invocation that starts from the
> > top level Makefile and in the leaf Makefile it tries to build a module (using M=
> > in the make invocation), it doesn't produce the module. This happens because the
> > toplevel Makefile checks for M= only once. This is controlled by the
> > sub_make_done variable, which is exported after checking the command line
> > options are passed to the top level Makefile. Once this variable is set it's
> > the M= setting is never checked again on the recursive call.
> >
> > This can be observed when cleaning the bpf kselftest dir. When calling
> >
> >         $ make TARGETS="bpf" SKIP_TARGETS="" kselftest-clean
> >
> > What happens:
> >
> >         1. It checks for some command line settings (like M=) was passed (it wasn't),
> >         set some definitions and exports sub_make_done.
> >
> >         2. Jump into tools/testing/selftests/bpf, and calls the clean target.
> >
> >         3. The clean target is overwritten to remove some files and then jump to
> >         bpf_testmod dir and call clean there
> >
> >         4. On bpf_testmod/Makefile, the clean target will execute
> >                 $(Q)make -C $(KDIR) M=$(BPF_TESTMOD_DIR) clean
> >
> >         5. The KDIR is to toplevel dir. The top Makefile will check that sub_make_done was
> >         already set, ignoring the M= setting.
> >
> >         6. As M= wasn't checked, KBUILD_EXTMOD isn't set, and the clean target applies
> >         to the kernel as a whole, making it clean all generated code/objects and
> >         everything.
> >
> > One way to avoid it is to call "unexport sub_make_done" on
> > tools/testing/selftests/bpf/bpf_testmod/Makefile before processing the all
> > target, forcing the toplevel Makefile to process the M=, producing the module
> > file correctly.
> >
> > If the M=dir points to /lib/modules/.../build, then it fails with "m2c: No such
> > file", which I already reported here[1]. At the time this problem was treated
> > like a problem with kselftest infrastructure.
> >
> > Important: The process works fine if the initial make invocation is targeted to a
> > different directory (using -C), since it doesn't goes through the toplevel
> > Makefile, and sub_make_done variable is not set.
> >
> > I attached a minimal reproducer, that can be used to better understand the
> > problem. The "make testmod" and "make testmod-clean" have the same effect that
> > can be seem with the bpf kselftests. There is a unexport call commented on
> > test-mods/Makefile, and once that is called the process works as expected.
> >
> > Is there a better way to fix this? Is this really a problem, or am I missing
> > something?
> 
> 
> Or, using KBUILD_EXTMOD will work too.

So it worked until 13b25489b6f8bd73ed65f07928f7c27a481f1820 ("kbuild: change working
directory to external module directory with M="). Right now, it shows the same
error as reported before:

m2c    -o /home/mpdesouza/git/linux/scripts/Makefile.build -e /home/mpdesouza/git/linux/scripts/Makefile.build /home/mpdesouza/git/linux/scripts/Makefile.build.mod
make[6]: m2c: No such file or directory

To reproduce, it's just execute the livepatch ksefltests:

$ make kselftest TARGETS=livepatch KDIR=<kernel source top dir>

Is there something that we can do in this case to solve this issue?

I'm quite surprised that no bots found this problem yet.

> 
> 
> 
> 
> 
> --
> Best Regards
> Masahiro Yamada

/*
 * !REMOVE THIS COMMENT BLOCK BEFORE SENDING THIS MAIL!
 * below are signatures added by "hkml".
 * edit signatures below, or use "hkml signature".
 */

Sent using hkml (https://github.com/sjp38/hackermail)