[PATCH RFC] Makefile: Export absolute srctree path for out-of-tree builds

[PATCH RFC] Makefile: Export absolute srctree path for out-of-tree builds

[Li Zhijian]

Fixes an issue where out-of-tree kselftest builds fail when building
the BPF and bpftools components. The failure occurs because the top-level
Makefile passes a relative srctree path ('..') to its sub-Makefiles, which
leads to errors in locating necessary files.

For example, the following error is encountered:

```
$ make V=1 O=$build/ TARGETS=hid kselftest-all
...
make -C ../tools/testing/selftests all
make[4]: Entering directory '/path/to/linux/tools/testing/selftests/hid'
make  -C /path/to/linux/tools/testing/selftests/../../../tools/lib/bpf OUTPUT=/path/to/linux/O/kselftest/hid/tools/build/libbpf/ \
            EXTRA_CFLAGS='-g -O0'                                      \
            DESTDIR=/path/to/linux/O/kselftest/hid/tools prefix= all install_headers
make[5]: Entering directory '/path/to/linux/tools/lib/bpf'
...
make[5]: Entering directory '/path/to/linux/tools/bpf/bpftool'
Makefile:127: ../tools/build/Makefile.feature: No such file or directory
make[5]: *** No rule to make target '../tools/build/Makefile.feature'.  Stop.
```

To resolve this, the srctree is exported as an absolute path (abs_srctree)
when performing an out-of-tree build. This ensures that all sub-Makefiles
have the correct path to the source tree, preventing directory resolution
errors.

Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
---
Request for Additional Testing

We welcome all contributors and CI systems to test this change thoroughly.
In theory, this change should not affect in-tree builds. However, to ensure
stability and compatibility, we encourage testing across different
configurations.

What has been tested?
- out-of-tree kernel build
- out-of-tree kselftest-all
---
 Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index e5b8a8832c0c..36e65806bb5e 100644
--- a/Makefile
+++ b/Makefile
@@ -275,7 +275,8 @@ else ifeq ($(srcroot)/,$(dir $(CURDIR)))
     srcroot := ..
 endif
 
-export srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot))
+srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot))
+export srctree := $(if $(building_out_of_srctree),$(abs_srctree),$(srctree))
 
 ifdef building_out_of_srctree
 export VPATH := $(srcroot)
-- 
2.44.0

Re: [PATCH RFC] Makefile: Export absolute srctree path for out-of-tree builds

[Quentin Monnet]

2024-12-17 11:10 UTC+0800 ~ Li Zhijian <lizhijian@fujitsu.com>
> Fixes an issue where out-of-tree kselftest builds fail when building
> the BPF and bpftools components. The failure occurs because the top-level
> Makefile passes a relative srctree path ('..') to its sub-Makefiles, which
> leads to errors in locating necessary files.
> 
> For example, the following error is encountered:
> 
> ```
> $ make V=1 O=$build/ TARGETS=hid kselftest-all
> ...
> make -C ../tools/testing/selftests all
> make[4]: Entering directory '/path/to/linux/tools/testing/selftests/hid'
> make  -C /path/to/linux/tools/testing/selftests/../../../tools/lib/bpf OUTPUT=/path/to/linux/O/kselftest/hid/tools/build/libbpf/ \
>             EXTRA_CFLAGS='-g -O0'                                      \
>             DESTDIR=/path/to/linux/O/kselftest/hid/tools prefix= all install_headers
> make[5]: Entering directory '/path/to/linux/tools/lib/bpf'
> ...
> make[5]: Entering directory '/path/to/linux/tools/bpf/bpftool'
> Makefile:127: ../tools/build/Makefile.feature: No such file or directory
> make[5]: *** No rule to make target '../tools/build/Makefile.feature'.  Stop.
> ```


Another condition to reproduce the failure above is to have have 
$(srcroot) in the Makefile set to "." or "..", for example when your
$build is located right under the root of the repo [0].

[0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Makefile?h=v6.13-rc3#n273


> To resolve this, the srctree is exported as an absolute path (abs_srctree)
> when performing an out-of-tree build. This ensures that all sub-Makefiles
> have the correct path to the source tree, preventing directory resolution
> errors.
> 
> Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
> ---
> Request for Additional Testing
> 
> We welcome all contributors and CI systems to test this change thoroughly.
> In theory, this change should not affect in-tree builds. However, to ensure
> stability and compatibility, we encourage testing across different
> configurations.
> 
> What has been tested?
> - out-of-tree kernel build
> - out-of-tree kselftest-all
> ---
>  Makefile | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/Makefile b/Makefile
> index e5b8a8832c0c..36e65806bb5e 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -275,7 +275,8 @@ else ifeq ($(srcroot)/,$(dir $(CURDIR)))
>      srcroot := ..
>  endif
>  
> -export srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot))
> +srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot))
> +export srctree := $(if $(building_out_of_srctree),$(abs_srctree),$(srctree))
>  
>  ifdef building_out_of_srctree
>  export VPATH := $(srcroot)


The patch does fix the issue in the case you reported.

Maybe I'd write the change differently, though; rather than potentially
overwriting $(srctree) before exporting it, let's unroll with ifdef's to
make it clearer?

	ifdef building_out_of_srctree
	    srctree := $(abs_srctree)
	else ifdef KBUILD_EXTMOD
	    srctree := $(abs_srctree)
	else
	    srctree := $(srcroot)
	endif
	export srctree

(not tested)

Thanks,
Quentin

Re: [PATCH RFC] Makefile: Export absolute srctree path for out-of-tree builds

[Zhijian Li (Fujitsu)]

On 21/12/2024 01:14, Quentin Monnet wrote:
> 2024-12-17 11:10 UTC+0800 ~ Li Zhijian <lizhijian@fujitsu.com>
>> Fixes an issue where out-of-tree kselftest builds fail when building
>> the BPF and bpftools components. The failure occurs because the top-level
>> Makefile passes a relative srctree path ('..') to its sub-Makefiles, which
>> leads to errors in locating necessary files.
>>
>> For example, the following error is encountered:
>>
>> ```
>> $ make V=1 O=$build/ TARGETS=hid kselftest-all
>> ...
>> make -C ../tools/testing/selftests all
>> make[4]: Entering directory '/path/to/linux/tools/testing/selftests/hid'
>> make  -C /path/to/linux/tools/testing/selftests/../../../tools/lib/bpf OUTPUT=/path/to/linux/O/kselftest/hid/tools/build/libbpf/ \
>>              EXTRA_CFLAGS='-g -O0'                                      \
>>              DESTDIR=/path/to/linux/O/kselftest/hid/tools prefix= all install_headers
>> make[5]: Entering directory '/path/to/linux/tools/lib/bpf'
>> ...
>> make[5]: Entering directory '/path/to/linux/tools/bpf/bpftool'
>> Makefile:127: ../tools/build/Makefile.feature: No such file or directory
>> make[5]: *** No rule to make target '../tools/build/Makefile.feature'.  Stop.
>> ```
> 
> 
> Another condition to reproduce the failure above is to have have
> $(srcroot) in the Makefile set to "." or "..", for example when your
> $build is located right under the root of the repo [0].
> 
> [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Makefile?h=v6.13-rc3#n273
> 
> 
>> To resolve this, the srctree is exported as an absolute path (abs_srctree)
>> when performing an out-of-tree build. This ensures that all sub-Makefiles
>> have the correct path to the source tree, preventing directory resolution
>> errors.
>>
>> Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
>> ---
>> Request for Additional Testing
>>
>> We welcome all contributors and CI systems to test this change thoroughly.
>> In theory, this change should not affect in-tree builds. However, to ensure
>> stability and compatibility, we encourage testing across different
>> configurations.
>>
>> What has been tested?
>> - out-of-tree kernel build
>> - out-of-tree kselftest-all
>> ---
>>   Makefile | 3 ++-
>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/Makefile b/Makefile
>> index e5b8a8832c0c..36e65806bb5e 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -275,7 +275,8 @@ else ifeq ($(srcroot)/,$(dir $(CURDIR)))
>>       srcroot := ..
>>   endif
>>   
>> -export srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot))
>> +srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot))
>> +export srctree := $(if $(building_out_of_srctree),$(abs_srctree),$(srctree))
>>   
>>   ifdef building_out_of_srctree
>>   export VPATH := $(srcroot)
> 
> 
> The patch does fix the issue in the case you reported.
> 
> Maybe I'd write the change differently, though; rather than potentially
> overwriting $(srctree) before exporting it, let's unroll with ifdef's to
> make it clearer?

Yes, your code looks better.
However, this "absolute source tree" approach was not approved.:)

Thank you!!



> 
> 	ifdef building_out_of_srctree
> 	    srctree := $(abs_srctree)
> 	else ifdef KBUILD_EXTMOD
> 	    srctree := $(abs_srctree)
> 	else
> 	    srctree := $(srcroot)
> 	endif
> 	export srctree
> 
> (not tested)
> 
> Thanks,
> Quentin

Re: [PATCH RFC] Makefile: Export absolute srctree path for out-of-tree builds

[Masahiro Yamada]

On Tue, Dec 17, 2024 at 12:10 PM Li Zhijian <lizhijian@fujitsu.com> wrote:
>
> Fixes an issue where out-of-tree kselftest builds fail when building
> the BPF and bpftools components. The failure occurs because the top-level
> Makefile passes a relative srctree path ('..') to its sub-Makefiles, which
> leads to errors in locating necessary files.
>
> For example, the following error is encountered:
>
> ```
> $ make V=1 O=$build/ TARGETS=hid kselftest-all
> ...
> make -C ../tools/testing/selftests all
> make[4]: Entering directory '/path/to/linux/tools/testing/selftests/hid'
> make  -C /path/to/linux/tools/testing/selftests/../../../tools/lib/bpf OUTPUT=/path/to/linux/O/kselftest/hid/tools/build/libbpf/ \
>             EXTRA_CFLAGS='-g -O0'                                      \
>             DESTDIR=/path/to/linux/O/kselftest/hid/tools prefix= all install_headers
> make[5]: Entering directory '/path/to/linux/tools/lib/bpf'
> ...
> make[5]: Entering directory '/path/to/linux/tools/bpf/bpftool'
> Makefile:127: ../tools/build/Makefile.feature: No such file or directory
> make[5]: *** No rule to make target '../tools/build/Makefile.feature'.  Stop.
> ```
>
> To resolve this, the srctree is exported as an absolute path (abs_srctree)
> when performing an out-of-tree build. This ensures that all sub-Makefiles
> have the correct path to the source tree, preventing directory resolution
> errors.

NACK.
This negates 9da0763bdd82572be243fcf5161734f11568960f

This is a recurring topic [1] because kselftest adopts a completely
different build system.
If kselftest cannot do this correctly, please do not hook it to the
top-Makefile.

[1] https://lore.kernel.org/linux-kbuild/cover.1657614127.git.guillaume.tucker@collabora.com/



> Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
> ---
> Request for Additional Testing
>
> We welcome all contributors and CI systems to test this change thoroughly.

This is NACKed. I recommend not wasting CI system resources.






> In theory, this change should not affect in-tree builds. However, to ensure
> stability and compatibility, we encourage testing across different
> configurations.
>
> What has been tested?
> - out-of-tree kernel build
> - out-of-tree kselftest-all
> ---
>  Makefile | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/Makefile b/Makefile
> index e5b8a8832c0c..36e65806bb5e 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -275,7 +275,8 @@ else ifeq ($(srcroot)/,$(dir $(CURDIR)))
>      srcroot := ..
>  endif
>
> -export srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot))
> +srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot))
> +export srctree := $(if $(building_out_of_srctree),$(abs_srctree),$(srctree))
>
>  ifdef building_out_of_srctree
>  export VPATH := $(srcroot)
> --
> 2.44.0
>
>


-- 
Best Regards
Masahiro Yamada

Re: [PATCH RFC] Makefile: Export absolute srctree path for out-of-tree builds

[Zhijian Li (Fujitsu)]

On 21/12/2024 11:41, Masahiro Yamada wrote:
> On Tue, Dec 17, 2024 at 12:10 PM Li Zhijian <lizhijian@fujitsu.com> wrote:
>>
>> Fixes an issue where out-of-tree kselftest builds fail when building
>> the BPF and bpftools components. The failure occurs because the top-level
>> Makefile passes a relative srctree path ('..') to its sub-Makefiles, which
>> leads to errors in locating necessary files.
>>
>> For example, the following error is encountered:
>>
>> ```
>> $ make V=1 O=$build/ TARGETS=hid kselftest-all
>> ...
>> make -C ../tools/testing/selftests all
>> make[4]: Entering directory '/path/to/linux/tools/testing/selftests/hid'
>> make  -C /path/to/linux/tools/testing/selftests/../../../tools/lib/bpf OUTPUT=/path/to/linux/O/kselftest/hid/tools/build/libbpf/ \
>>              EXTRA_CFLAGS='-g -O0'                                      \
>>              DESTDIR=/path/to/linux/O/kselftest/hid/tools prefix= all install_headers
>> make[5]: Entering directory '/path/to/linux/tools/lib/bpf'
>> ...
>> make[5]: Entering directory '/path/to/linux/tools/bpf/bpftool'
>> Makefile:127: ../tools/build/Makefile.feature: No such file or directory
>> make[5]: *** No rule to make target '../tools/build/Makefile.feature'.  Stop.
>> ```
>>
>> To resolve this, the srctree is exported as an absolute path (abs_srctree)
>> when performing an out-of-tree build. This ensures that all sub-Makefiles
>> have the correct path to the source tree, preventing directory resolution
>> errors.
> 
> NACK.
> This negates 9da0763bdd82572be243fcf5161734f11568960f
> 
> This is a recurring topic [1] because kselftest adopts a completely
> different build system.
> If kselftest cannot do this correctly, please do not hook it to the
> top-Makefile.
> 

Hi Masahiro,

Thank you for your feedback
I understand and agree with your points, thanks again for these information.


> [1] https://lore.kernel.org/linux-kbuild/cover.1657614127.git.guillaume.tucker@collabora.com/
> 
> 
> 
>> Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
>> ---
>> Request for Additional Testing
>>
>> We welcome all contributors and CI systems to test this change thoroughly.
> 
> This is NACKed. I recommend not wasting CI system resources.


It sounds reasonable.
It makes sense to avoid using CI resources on this patch given the current circumstances.


Thanks
Zhijian

> 
> 
> 
> 
> 
> 
>> In theory, this change should not affect in-tree builds. However, to ensure
>> stability and compatibility, we encourage testing across different
>> configurations.
>>
>> What has been tested?
>> - out-of-tree kernel build
>> - out-of-tree kselftest-all
>> ---
>>   Makefile | 3 ++-
>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/Makefile b/Makefile
>> index e5b8a8832c0c..36e65806bb5e 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -275,7 +275,8 @@ else ifeq ($(srcroot)/,$(dir $(CURDIR)))
>>       srcroot := ..
>>   endif
>>
>> -export srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot))
>> +srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot))
>> +export srctree := $(if $(building_out_of_srctree),$(abs_srctree),$(srctree))
>>
>>   ifdef building_out_of_srctree
>>   export VPATH := $(srcroot)
>> --
>> 2.44.0
>>
>>
> 
>