[PATCH v5] selftests: fix prepending $(OUTPUT) to $(TEST_PROGS)

[PATCH v5] selftests: fix prepending $(OUTPUT) to $(TEST_PROGS)

[Ilya Leoshkevich]

Currently the following command produces an error message:

    linux# make kselftest TARGETS=bpf O=/mnt/linux-build
    # selftests: bpf: test_libbpf.sh
    # ./test_libbpf.sh: line 23: ./test_libbpf_open: No such file or directory
    # test_libbpf: failed at file test_l4lb.o
    # selftests: test_libbpf [FAILED]

The error message might not affect the return code of make, therefore
one needs to grep make output in order to detect it.

This is not the only instance of the same underlying problem; any test
with more than one element in $(TEST_PROGS) fails the same way. Another
example:

    linux# make O=/mnt/linux-build TARGETS=splice kselftest
    [...]
    # ./short_splice_read.sh: 15: ./splice_read: not found
    # FAIL: /sys/module/test_module/sections/.init.text 2
    not ok 2 selftests: splice: short_splice_read.sh # exit=1

The current logic prepends $(OUTPUT) only to the first member of
$(TEST_PROGS). After that, run_one() does

   cd `dirname $TEST`

For all tests except the first one, `dirname $TEST` is ., which means
they cannot access the files generated in $(OUTPUT).

Fix by using $(addprefix) to prepend $(OUTPUT)/ to each member of
$(TEST_PROGS).

Fixes: 1a940687e424 ("selftests: lib.mk: copy test scripts and test files for make O=dir run")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---

v1->v2:
- Append / to $(OUTPUT).
- Use $(addprefix) instead of $(foreach).

v2->v3:
- Split the patch in two.
- Improve the commit message.

v3: https://lore.kernel.org/linux-kselftest/20191024121347.22189-1-iii@linux.ibm.com/
v3->v4:
- Drop the first patch.
- Add a note regarding make return code to the commit message.

v4: https://lore.kernel.org/linux-kselftest/20191115150428.61131-1-iii@linux.ibm.com/
v4->v5:
- Add another reproducer to the commit message.

 tools/testing/selftests/lib.mk | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
index a5ce26d548e4..be17462fe146 100644
--- a/tools/testing/selftests/lib.mk
+++ b/tools/testing/selftests/lib.mk
@@ -74,7 +74,8 @@ ifdef building_out_of_srctree
 		rsync -aq $(TEST_PROGS) $(TEST_PROGS_EXTENDED) $(TEST_FILES) $(OUTPUT); \
 	fi
 	@if [ "X$(TEST_PROGS)" != "X" ]; then \
-		$(call RUN_TESTS, $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS) $(OUTPUT)/$(TEST_PROGS)) ; \
+		$(call RUN_TESTS, $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS) \
+				  $(addprefix $(OUTPUT)/,$(TEST_PROGS))) ; \
 	else \
 		$(call RUN_TESTS, $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS)); \
 	fi
-- 
2.29.2

Re: [PATCH v5] selftests: fix prepending $(OUTPUT) to $(TEST_PROGS)

[Shuah Khan]

On 3/2/21 5:44 PM, Ilya Leoshkevich wrote:
> Currently the following command produces an error message:
> 
>      linux# make kselftest TARGETS=bpf O=/mnt/linux-build
>      # selftests: bpf: test_libbpf.sh
>      # ./test_libbpf.sh: line 23: ./test_libbpf_open: No such file or directory
>      # test_libbpf: failed at file test_l4lb.o
>      # selftests: test_libbpf [FAILED]
> 
> The error message might not affect the return code of make, therefore
> one needs to grep make output in order to detect it.
> 
> This is not the only instance of the same underlying problem; any test
> with more than one element in $(TEST_PROGS) fails the same way. Another
> example:
> 
>      linux# make O=/mnt/linux-build TARGETS=splice kselftest
>      [...]
>      # ./short_splice_read.sh: 15: ./splice_read: not found
>      # FAIL: /sys/module/test_module/sections/.init.text 2
>      not ok 2 selftests: splice: short_splice_read.sh # exit=1
> 
> The current logic prepends $(OUTPUT) only to the first member of
> $(TEST_PROGS). After that, run_one() does
> 
>     cd `dirname $TEST`
> 
> For all tests except the first one, `dirname $TEST` is ., which means
> they cannot access the files generated in $(OUTPUT).
> 
> Fix by using $(addprefix) to prepend $(OUTPUT)/ to each member of
> $(TEST_PROGS).
> 
> Fixes: 1a940687e424 ("selftests: lib.mk: copy test scripts and test files for make O=dir run")
> Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
> ---
> 
> v1->v2:
> - Append / to $(OUTPUT).
> - Use $(addprefix) instead of $(foreach).
> 
> v2->v3:
> - Split the patch in two.
> - Improve the commit message.
> 
> v3: https://lore.kernel.org/linux-kselftest/20191024121347.22189-1-iii@linux.ibm.com/
> v3->v4:
> - Drop the first patch.
> - Add a note regarding make return code to the commit message.
> 
> v4: https://lore.kernel.org/linux-kselftest/20191115150428.61131-1-iii@linux.ibm.com/
> v4->v5:
> - Add another reproducer to the commit message.
> 
>   tools/testing/selftests/lib.mk | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
> index a5ce26d548e4..be17462fe146 100644
> --- a/tools/testing/selftests/lib.mk
> +++ b/tools/testing/selftests/lib.mk
> @@ -74,7 +74,8 @@ ifdef building_out_of_srctree
>   		rsync -aq $(TEST_PROGS) $(TEST_PROGS_EXTENDED) $(TEST_FILES) $(OUTPUT); \
>   	fi
>   	@if [ "X$(TEST_PROGS)" != "X" ]; then \
> -		$(call RUN_TESTS, $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS) $(OUTPUT)/$(TEST_PROGS)) ; \
> +		$(call RUN_TESTS, $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS) \
> +				  $(addprefix $(OUTPUT)/,$(TEST_PROGS))) ; \
>   	else \
>   		$(call RUN_TESTS, $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS)); \
>   	fi
> 

Thank you. Will apply it for 5.13-rc1

thanks,
-- Shuah