* [PATCH v3 01/11] selftests/mm: fix uffd-stress unused function warning
2023-06-06 7:16 [PATCH v3 00/11] A minor flurry of selftest/mm fixes John Hubbard
@ 2023-06-06 7:16 ` John Hubbard
2023-06-06 7:46 ` Muhammad Usama Anjum
2023-06-06 7:16 ` [PATCH v3 02/11] selftests/mm: fix unused variable warnings in hugetlb-madvise.c, migration.c John Hubbard
` (9 subsequent siblings)
10 siblings, 1 reply; 39+ messages in thread
From: John Hubbard @ 2023-06-06 7:16 UTC (permalink / raw)
To: Andrew Morton
Cc: David Hildenbrand, Peter Xu, Shuah Khan, Nathan Chancellor,
linux-mm, linux-kselftest, LKML, John Hubbard
uffd_minor_feature() was unused. Remove it in order to fix the
associated clang build warning.
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
tools/testing/selftests/mm/uffd-stress.c | 10 ----------
1 file changed, 10 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index f1ad9eef1c3a..995ff13e74c7 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -88,16 +88,6 @@ static void uffd_stats_reset(struct uffd_args *args, unsigned long n_cpus)
}
}
-static inline uint64_t uffd_minor_feature(void)
-{
- if (test_type == TEST_HUGETLB && map_shared)
- return UFFD_FEATURE_MINOR_HUGETLBFS;
- else if (test_type == TEST_SHMEM)
- return UFFD_FEATURE_MINOR_SHMEM;
- else
- return 0;
-}
-
static void *locking_thread(void *arg)
{
unsigned long cpu = (unsigned long) arg;
--
2.40.1
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH v3 01/11] selftests/mm: fix uffd-stress unused function warning
2023-06-06 7:16 ` [PATCH v3 01/11] selftests/mm: fix uffd-stress unused function warning John Hubbard
@ 2023-06-06 7:46 ` Muhammad Usama Anjum
0 siblings, 0 replies; 39+ messages in thread
From: Muhammad Usama Anjum @ 2023-06-06 7:46 UTC (permalink / raw)
To: John Hubbard, Andrew Morton
Cc: Muhammad Usama Anjum, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML
On 6/6/23 12:16 PM, John Hubbard wrote:
> uffd_minor_feature() was unused. Remove it in order to fix the
> associated clang build warning.
>
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Reviewed-by: Peter Xu <peterx@redhat.com>
> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Tested-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
> ---
> tools/testing/selftests/mm/uffd-stress.c | 10 ----------
> 1 file changed, 10 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
> index f1ad9eef1c3a..995ff13e74c7 100644
> --- a/tools/testing/selftests/mm/uffd-stress.c
> +++ b/tools/testing/selftests/mm/uffd-stress.c
> @@ -88,16 +88,6 @@ static void uffd_stats_reset(struct uffd_args *args, unsigned long n_cpus)
> }
> }
>
> -static inline uint64_t uffd_minor_feature(void)
> -{
> - if (test_type == TEST_HUGETLB && map_shared)
> - return UFFD_FEATURE_MINOR_HUGETLBFS;
> - else if (test_type == TEST_SHMEM)
> - return UFFD_FEATURE_MINOR_SHMEM;
> - else
> - return 0;
> -}
> -
> static void *locking_thread(void *arg)
> {
> unsigned long cpu = (unsigned long) arg;
--
BR,
Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH v3 02/11] selftests/mm: fix unused variable warnings in hugetlb-madvise.c, migration.c
2023-06-06 7:16 [PATCH v3 00/11] A minor flurry of selftest/mm fixes John Hubbard
2023-06-06 7:16 ` [PATCH v3 01/11] selftests/mm: fix uffd-stress unused function warning John Hubbard
@ 2023-06-06 7:16 ` John Hubbard
2023-06-06 7:48 ` Muhammad Usama Anjum
2023-06-06 7:16 ` [PATCH v3 03/11] selftests/mm: fix "warning: expression which evaluates to zero..." in mlock2-tests.c John Hubbard
` (8 subsequent siblings)
10 siblings, 1 reply; 39+ messages in thread
From: John Hubbard @ 2023-06-06 7:16 UTC (permalink / raw)
To: Andrew Morton
Cc: David Hildenbrand, Peter Xu, Shuah Khan, Nathan Chancellor,
linux-mm, linux-kselftest, LKML, John Hubbard
Dummy variables are required in order to make these two (similar)
routines work, so in both cases, declare the variables as volatile in
order to avoid the clang compiler warning.
Furthermore, in order to ensure that each test actually does what is
intended, add an asm volatile invocation (thanks to David Hildenbrand
for the suggestion), with a clarifying comment so that it survives
future maintenance.
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
tools/testing/selftests/mm/hugetlb-madvise.c | 8 ++++++--
tools/testing/selftests/mm/migration.c | 5 ++++-
2 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/tools/testing/selftests/mm/hugetlb-madvise.c b/tools/testing/selftests/mm/hugetlb-madvise.c
index 28426e30d9bc..d55322df4b73 100644
--- a/tools/testing/selftests/mm/hugetlb-madvise.c
+++ b/tools/testing/selftests/mm/hugetlb-madvise.c
@@ -65,11 +65,15 @@ void write_fault_pages(void *addr, unsigned long nr_pages)
void read_fault_pages(void *addr, unsigned long nr_pages)
{
- unsigned long dummy = 0;
+ volatile unsigned long dummy = 0;
unsigned long i;
- for (i = 0; i < nr_pages; i++)
+ for (i = 0; i < nr_pages; i++) {
dummy += *((unsigned long *)(addr + (i * huge_page_size)));
+
+ /* Prevent the compiler from optimizing out the entire loop: */
+ asm volatile("" : "+r" (dummy));
+ }
}
int main(int argc, char **argv)
diff --git a/tools/testing/selftests/mm/migration.c b/tools/testing/selftests/mm/migration.c
index 1cec8425e3ca..379581567f27 100644
--- a/tools/testing/selftests/mm/migration.c
+++ b/tools/testing/selftests/mm/migration.c
@@ -95,12 +95,15 @@ int migrate(uint64_t *ptr, int n1, int n2)
void *access_mem(void *ptr)
{
- uint64_t y = 0;
+ volatile uint64_t y = 0;
volatile uint64_t *x = ptr;
while (1) {
pthread_testcancel();
y += *x;
+
+ /* Prevent the compiler from optimizing out the writes to y: */
+ asm volatile("" : "+r" (y));
}
return NULL;
--
2.40.1
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH v3 02/11] selftests/mm: fix unused variable warnings in hugetlb-madvise.c, migration.c
2023-06-06 7:16 ` [PATCH v3 02/11] selftests/mm: fix unused variable warnings in hugetlb-madvise.c, migration.c John Hubbard
@ 2023-06-06 7:48 ` Muhammad Usama Anjum
0 siblings, 0 replies; 39+ messages in thread
From: Muhammad Usama Anjum @ 2023-06-06 7:48 UTC (permalink / raw)
To: John Hubbard, Andrew Morton
Cc: Muhammad Usama Anjum, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML
On 6/6/23 12:16 PM, John Hubbard wrote:
> Dummy variables are required in order to make these two (similar)
> routines work, so in both cases, declare the variables as volatile in
> order to avoid the clang compiler warning.
>
> Furthermore, in order to ensure that each test actually does what is
> intended, add an asm volatile invocation (thanks to David Hildenbrand
> for the suggestion), with a clarifying comment so that it survives
> future maintenance.
>
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Reviewed-by: Peter Xu <peterx@redhat.com>
> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Tested-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
> ---
> tools/testing/selftests/mm/hugetlb-madvise.c | 8 ++++++--
> tools/testing/selftests/mm/migration.c | 5 ++++-
> 2 files changed, 10 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/hugetlb-madvise.c b/tools/testing/selftests/mm/hugetlb-madvise.c
> index 28426e30d9bc..d55322df4b73 100644
> --- a/tools/testing/selftests/mm/hugetlb-madvise.c
> +++ b/tools/testing/selftests/mm/hugetlb-madvise.c
> @@ -65,11 +65,15 @@ void write_fault_pages(void *addr, unsigned long nr_pages)
>
> void read_fault_pages(void *addr, unsigned long nr_pages)
> {
> - unsigned long dummy = 0;
> + volatile unsigned long dummy = 0;
> unsigned long i;
>
> - for (i = 0; i < nr_pages; i++)
> + for (i = 0; i < nr_pages; i++) {
> dummy += *((unsigned long *)(addr + (i * huge_page_size)));
> +
> + /* Prevent the compiler from optimizing out the entire loop: */
> + asm volatile("" : "+r" (dummy));
> + }
> }
>
> int main(int argc, char **argv)
> diff --git a/tools/testing/selftests/mm/migration.c b/tools/testing/selftests/mm/migration.c
> index 1cec8425e3ca..379581567f27 100644
> --- a/tools/testing/selftests/mm/migration.c
> +++ b/tools/testing/selftests/mm/migration.c
> @@ -95,12 +95,15 @@ int migrate(uint64_t *ptr, int n1, int n2)
>
> void *access_mem(void *ptr)
> {
> - uint64_t y = 0;
> + volatile uint64_t y = 0;
> volatile uint64_t *x = ptr;
>
> while (1) {
> pthread_testcancel();
> y += *x;
> +
> + /* Prevent the compiler from optimizing out the writes to y: */
> + asm volatile("" : "+r" (y));
> }
>
> return NULL;
--
BR,
Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH v3 03/11] selftests/mm: fix "warning: expression which evaluates to zero..." in mlock2-tests.c
2023-06-06 7:16 [PATCH v3 00/11] A minor flurry of selftest/mm fixes John Hubbard
2023-06-06 7:16 ` [PATCH v3 01/11] selftests/mm: fix uffd-stress unused function warning John Hubbard
2023-06-06 7:16 ` [PATCH v3 02/11] selftests/mm: fix unused variable warnings in hugetlb-madvise.c, migration.c John Hubbard
@ 2023-06-06 7:16 ` John Hubbard
2023-06-06 7:49 ` Muhammad Usama Anjum
2023-06-06 7:16 ` [PATCH v3 04/11] selftests/mm: fix invocation of tests that are run via shell scripts John Hubbard
` (7 subsequent siblings)
10 siblings, 1 reply; 39+ messages in thread
From: John Hubbard @ 2023-06-06 7:16 UTC (permalink / raw)
To: Andrew Morton
Cc: David Hildenbrand, Peter Xu, Shuah Khan, Nathan Chancellor,
linux-mm, linux-kselftest, LKML, John Hubbard
The stop variable is a char*, and the code was assigning a char value to
it. This was generating a warning when compiling with clang.
However, as both David and Peter pointed out, stop is not even used
after the problematic assignment to a char type. So just delete that
line entirely.
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
tools/testing/selftests/mm/mlock2-tests.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/tools/testing/selftests/mm/mlock2-tests.c b/tools/testing/selftests/mm/mlock2-tests.c
index 11b2301f3aa3..80cddc0de206 100644
--- a/tools/testing/selftests/mm/mlock2-tests.c
+++ b/tools/testing/selftests/mm/mlock2-tests.c
@@ -50,7 +50,6 @@ static int get_vm_area(unsigned long addr, struct vm_boundaries *area)
printf("cannot parse /proc/self/maps\n");
goto out;
}
- stop = '\0';
sscanf(line, "%lx", &start);
sscanf(end_addr, "%lx", &end);
--
2.40.1
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH v3 03/11] selftests/mm: fix "warning: expression which evaluates to zero..." in mlock2-tests.c
2023-06-06 7:16 ` [PATCH v3 03/11] selftests/mm: fix "warning: expression which evaluates to zero..." in mlock2-tests.c John Hubbard
@ 2023-06-06 7:49 ` Muhammad Usama Anjum
0 siblings, 0 replies; 39+ messages in thread
From: Muhammad Usama Anjum @ 2023-06-06 7:49 UTC (permalink / raw)
To: John Hubbard, Andrew Morton
Cc: Muhammad Usama Anjum, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML
On 6/6/23 12:16 PM, John Hubbard wrote:
> The stop variable is a char*, and the code was assigning a char value to
> it. This was generating a warning when compiling with clang.
>
> However, as both David and Peter pointed out, stop is not even used
> after the problematic assignment to a char type. So just delete that
> line entirely.
>
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Reviewed-by: Peter Xu <peterx@redhat.com>
> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Tested-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
> ---
> tools/testing/selftests/mm/mlock2-tests.c | 1 -
> 1 file changed, 1 deletion(-)
>
> diff --git a/tools/testing/selftests/mm/mlock2-tests.c b/tools/testing/selftests/mm/mlock2-tests.c
> index 11b2301f3aa3..80cddc0de206 100644
> --- a/tools/testing/selftests/mm/mlock2-tests.c
> +++ b/tools/testing/selftests/mm/mlock2-tests.c
> @@ -50,7 +50,6 @@ static int get_vm_area(unsigned long addr, struct vm_boundaries *area)
> printf("cannot parse /proc/self/maps\n");
> goto out;
> }
> - stop = '\0';
>
> sscanf(line, "%lx", &start);
> sscanf(end_addr, "%lx", &end);
--
BR,
Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH v3 04/11] selftests/mm: fix invocation of tests that are run via shell scripts
2023-06-06 7:16 [PATCH v3 00/11] A minor flurry of selftest/mm fixes John Hubbard
` (2 preceding siblings ...)
2023-06-06 7:16 ` [PATCH v3 03/11] selftests/mm: fix "warning: expression which evaluates to zero..." in mlock2-tests.c John Hubbard
@ 2023-06-06 7:16 ` John Hubbard
2023-06-06 7:51 ` Muhammad Usama Anjum
2023-06-06 7:16 ` [PATCH v3 05/11] selftests/mm: .gitignore: add mkdirty, va_high_addr_switch John Hubbard
` (6 subsequent siblings)
10 siblings, 1 reply; 39+ messages in thread
From: John Hubbard @ 2023-06-06 7:16 UTC (permalink / raw)
To: Andrew Morton
Cc: David Hildenbrand, Peter Xu, Shuah Khan, Nathan Chancellor,
linux-mm, linux-kselftest, LKML, John Hubbard
We cannot depend upon git to reliably retain the executable bit on shell
scripts, or so I was told several years ago while working on this same
run_vmtests.sh script. And sure enough, things such as test_hmm.sh are
lately failing to run, due to lacking execute permissions.
Fix this by explicitly adding "bash" to each of the shell script
invocations. Leave fixing the overall approach to another day.
Acked-by: David Hildenbrand <david@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
tools/testing/selftests/mm/run_vmtests.sh | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index 4893eb60d96d..8f81432e4bac 100644
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -242,18 +242,18 @@ if [ $VADDR64 -ne 0 ]; then
if [ "$ARCH" == "$ARCH_ARM64" ]; then
echo 6 > /proc/sys/vm/nr_hugepages
fi
- CATEGORY="hugevm" run_test ./va_high_addr_switch.sh
+ CATEGORY="hugevm" run_test bash ./va_high_addr_switch.sh
if [ "$ARCH" == "$ARCH_ARM64" ]; then
echo $prev_nr_hugepages > /proc/sys/vm/nr_hugepages
fi
fi # VADDR64
# vmalloc stability smoke test
-CATEGORY="vmalloc" run_test ./test_vmalloc.sh smoke
+CATEGORY="vmalloc" run_test bash ./test_vmalloc.sh smoke
CATEGORY="mremap" run_test ./mremap_dontunmap
-CATEGORY="hmm" run_test ./test_hmm.sh smoke
+CATEGORY="hmm" run_test bash ./test_hmm.sh smoke
# MADV_POPULATE_READ and MADV_POPULATE_WRITE tests
CATEGORY="madv_populate" run_test ./madv_populate
--
2.40.1
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH v3 04/11] selftests/mm: fix invocation of tests that are run via shell scripts
2023-06-06 7:16 ` [PATCH v3 04/11] selftests/mm: fix invocation of tests that are run via shell scripts John Hubbard
@ 2023-06-06 7:51 ` Muhammad Usama Anjum
0 siblings, 0 replies; 39+ messages in thread
From: Muhammad Usama Anjum @ 2023-06-06 7:51 UTC (permalink / raw)
To: John Hubbard, Andrew Morton
Cc: Muhammad Usama Anjum, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML
On 6/6/23 12:16 PM, John Hubbard wrote:
> We cannot depend upon git to reliably retain the executable bit on shell
> scripts, or so I was told several years ago while working on this same
> run_vmtests.sh script. And sure enough, things such as test_hmm.sh are
> lately failing to run, due to lacking execute permissions.
>
> Fix this by explicitly adding "bash" to each of the shell script
> invocations. Leave fixing the overall approach to another day.
>
> Acked-by: David Hildenbrand <david@redhat.com>
> Cc: Peter Xu <peterx@redhat.com>
> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Tested-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
> ---
> tools/testing/selftests/mm/run_vmtests.sh | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index 4893eb60d96d..8f81432e4bac 100644
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -242,18 +242,18 @@ if [ $VADDR64 -ne 0 ]; then
> if [ "$ARCH" == "$ARCH_ARM64" ]; then
> echo 6 > /proc/sys/vm/nr_hugepages
> fi
> - CATEGORY="hugevm" run_test ./va_high_addr_switch.sh
> + CATEGORY="hugevm" run_test bash ./va_high_addr_switch.sh
> if [ "$ARCH" == "$ARCH_ARM64" ]; then
> echo $prev_nr_hugepages > /proc/sys/vm/nr_hugepages
> fi
> fi # VADDR64
>
> # vmalloc stability smoke test
> -CATEGORY="vmalloc" run_test ./test_vmalloc.sh smoke
> +CATEGORY="vmalloc" run_test bash ./test_vmalloc.sh smoke
>
> CATEGORY="mremap" run_test ./mremap_dontunmap
>
> -CATEGORY="hmm" run_test ./test_hmm.sh smoke
> +CATEGORY="hmm" run_test bash ./test_hmm.sh smoke
>
> # MADV_POPULATE_READ and MADV_POPULATE_WRITE tests
> CATEGORY="madv_populate" run_test ./madv_populate
--
BR,
Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH v3 05/11] selftests/mm: .gitignore: add mkdirty, va_high_addr_switch
2023-06-06 7:16 [PATCH v3 00/11] A minor flurry of selftest/mm fixes John Hubbard
` (3 preceding siblings ...)
2023-06-06 7:16 ` [PATCH v3 04/11] selftests/mm: fix invocation of tests that are run via shell scripts John Hubbard
@ 2023-06-06 7:16 ` John Hubbard
2023-06-06 7:52 ` Muhammad Usama Anjum
2023-06-06 7:16 ` [PATCH v3 06/11] selftests/mm: fix two -Wformat-security warnings in uffd builds John Hubbard
` (5 subsequent siblings)
10 siblings, 1 reply; 39+ messages in thread
From: John Hubbard @ 2023-06-06 7:16 UTC (permalink / raw)
To: Andrew Morton
Cc: David Hildenbrand, Peter Xu, Shuah Khan, Nathan Chancellor,
linux-mm, linux-kselftest, LKML, John Hubbard
These new build products were left out of .gitignore, so add them now.
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
tools/testing/selftests/mm/.gitignore | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tools/testing/selftests/mm/.gitignore b/tools/testing/selftests/mm/.gitignore
index 8917455f4f51..ab215303d8e9 100644
--- a/tools/testing/selftests/mm/.gitignore
+++ b/tools/testing/selftests/mm/.gitignore
@@ -39,3 +39,5 @@ local_config.h
local_config.mk
ksm_functional_tests
mdwe_test
+mkdirty
+va_high_addr_switch
\ No newline at end of file
--
2.40.1
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH v3 05/11] selftests/mm: .gitignore: add mkdirty, va_high_addr_switch
2023-06-06 7:16 ` [PATCH v3 05/11] selftests/mm: .gitignore: add mkdirty, va_high_addr_switch John Hubbard
@ 2023-06-06 7:52 ` Muhammad Usama Anjum
0 siblings, 0 replies; 39+ messages in thread
From: Muhammad Usama Anjum @ 2023-06-06 7:52 UTC (permalink / raw)
To: John Hubbard, Andrew Morton
Cc: Muhammad Usama Anjum, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML
On 6/6/23 12:16 PM, John Hubbard wrote:
> These new build products were left out of .gitignore, so add them now.
I'd added to the instructions that object files should be added to
.gitignore. But still sometimes people forget.
>
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Reviewed-by: Peter Xu <peterx@redhat.com>
> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Tested-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
> ---
> tools/testing/selftests/mm/.gitignore | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/tools/testing/selftests/mm/.gitignore b/tools/testing/selftests/mm/.gitignore
> index 8917455f4f51..ab215303d8e9 100644
> --- a/tools/testing/selftests/mm/.gitignore
> +++ b/tools/testing/selftests/mm/.gitignore
> @@ -39,3 +39,5 @@ local_config.h
> local_config.mk
> ksm_functional_tests
> mdwe_test
> +mkdirty
> +va_high_addr_switch
> \ No newline at end of file
--
BR,
Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH v3 06/11] selftests/mm: fix two -Wformat-security warnings in uffd builds
2023-06-06 7:16 [PATCH v3 00/11] A minor flurry of selftest/mm fixes John Hubbard
` (4 preceding siblings ...)
2023-06-06 7:16 ` [PATCH v3 05/11] selftests/mm: .gitignore: add mkdirty, va_high_addr_switch John Hubbard
@ 2023-06-06 7:16 ` John Hubbard
2023-06-06 7:54 ` Muhammad Usama Anjum
2023-06-06 7:16 ` [PATCH v3 07/11] selftests/mm: fix a "possibly uninitialized" warning in pkey-x86.h John Hubbard
` (4 subsequent siblings)
10 siblings, 1 reply; 39+ messages in thread
From: John Hubbard @ 2023-06-06 7:16 UTC (permalink / raw)
To: Andrew Morton
Cc: David Hildenbrand, Peter Xu, Shuah Khan, Nathan Chancellor,
linux-mm, linux-kselftest, LKML, John Hubbard
The uffd tests generate two compile time warnings from clang's
-Wformat-security setting. These trigger at the call sites for
uffd_test_start() and uffd_test_skip().
1) Fix the uffd_test_start() issue by removing the intermediate
test_name variable (thanks to David Hildenbrand for showing how to do
this).
2) Fix the uffd_test_skip() issue by observing that there is no need for
a macro and a variable args approach, because all callers of
uffd_test_skip() pass in a simple char* string, without any format
specifiers. So just change uffd_test_skip() into a regular C function.
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
tools/testing/selftests/mm/uffd-unit-tests.c | 16 ++++++----------
1 file changed, 6 insertions(+), 10 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index 269c86768a02..04d91f144d1c 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -109,12 +109,11 @@ static void uffd_test_pass(void)
ksft_inc_fail_cnt(); \
} while (0)
-#define uffd_test_skip(...) do { \
- printf("skipped [reason: "); \
- printf(__VA_ARGS__); \
- printf("]\n"); \
- ksft_inc_xskip_cnt(); \
- } while (0)
+static void uffd_test_skip(const char *message)
+{
+ printf("skipped [reason: %s]\n", message);
+ ksft_inc_xskip_cnt();
+}
/*
* Returns 1 if specific userfaultfd supported, 0 otherwise. Note, we'll
@@ -1149,7 +1148,6 @@ int main(int argc, char *argv[])
uffd_test_case_t *test;
mem_type_t *mem_type;
uffd_test_args_t args;
- char test_name[128];
const char *errmsg;
int has_uffd, opt;
int i, j;
@@ -1192,10 +1190,8 @@ int main(int argc, char *argv[])
mem_type = &mem_types[j];
if (!(test->mem_targets & mem_type->mem_flag))
continue;
- snprintf(test_name, sizeof(test_name),
- "%s on %s", test->name, mem_type->name);
- uffd_test_start(test_name);
+ uffd_test_start("%s on %s", test->name, mem_type->name);
if (!uffd_feature_supported(test)) {
uffd_test_skip("feature missing");
continue;
--
2.40.1
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH v3 06/11] selftests/mm: fix two -Wformat-security warnings in uffd builds
2023-06-06 7:16 ` [PATCH v3 06/11] selftests/mm: fix two -Wformat-security warnings in uffd builds John Hubbard
@ 2023-06-06 7:54 ` Muhammad Usama Anjum
0 siblings, 0 replies; 39+ messages in thread
From: Muhammad Usama Anjum @ 2023-06-06 7:54 UTC (permalink / raw)
To: John Hubbard, Andrew Morton
Cc: Muhammad Usama Anjum, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML
On 6/6/23 12:16 PM, John Hubbard wrote:
> The uffd tests generate two compile time warnings from clang's
> -Wformat-security setting. These trigger at the call sites for
> uffd_test_start() and uffd_test_skip().
>
> 1) Fix the uffd_test_start() issue by removing the intermediate
> test_name variable (thanks to David Hildenbrand for showing how to do
> this).
>
> 2) Fix the uffd_test_skip() issue by observing that there is no need for
> a macro and a variable args approach, because all callers of
> uffd_test_skip() pass in a simple char* string, without any format
> specifiers. So just change uffd_test_skip() into a regular C function.
>
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Reviewed-by: Peter Xu <peterx@redhat.com>
> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Tested-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
> ---
> tools/testing/selftests/mm/uffd-unit-tests.c | 16 ++++++----------
> 1 file changed, 6 insertions(+), 10 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
> index 269c86768a02..04d91f144d1c 100644
> --- a/tools/testing/selftests/mm/uffd-unit-tests.c
> +++ b/tools/testing/selftests/mm/uffd-unit-tests.c
> @@ -109,12 +109,11 @@ static void uffd_test_pass(void)
> ksft_inc_fail_cnt(); \
> } while (0)
>
> -#define uffd_test_skip(...) do { \
> - printf("skipped [reason: "); \
> - printf(__VA_ARGS__); \
> - printf("]\n"); \
> - ksft_inc_xskip_cnt(); \
> - } while (0)
> +static void uffd_test_skip(const char *message)
> +{
> + printf("skipped [reason: %s]\n", message);
> + ksft_inc_xskip_cnt();
> +}
>
> /*
> * Returns 1 if specific userfaultfd supported, 0 otherwise. Note, we'll
> @@ -1149,7 +1148,6 @@ int main(int argc, char *argv[])
> uffd_test_case_t *test;
> mem_type_t *mem_type;
> uffd_test_args_t args;
> - char test_name[128];
> const char *errmsg;
> int has_uffd, opt;
> int i, j;
> @@ -1192,10 +1190,8 @@ int main(int argc, char *argv[])
> mem_type = &mem_types[j];
> if (!(test->mem_targets & mem_type->mem_flag))
> continue;
> - snprintf(test_name, sizeof(test_name),
> - "%s on %s", test->name, mem_type->name);
>
> - uffd_test_start(test_name);
> + uffd_test_start("%s on %s", test->name, mem_type->name);
> if (!uffd_feature_supported(test)) {
> uffd_test_skip("feature missing");
> continue;
--
BR,
Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH v3 07/11] selftests/mm: fix a "possibly uninitialized" warning in pkey-x86.h
2023-06-06 7:16 [PATCH v3 00/11] A minor flurry of selftest/mm fixes John Hubbard
` (5 preceding siblings ...)
2023-06-06 7:16 ` [PATCH v3 06/11] selftests/mm: fix two -Wformat-security warnings in uffd builds John Hubbard
@ 2023-06-06 7:16 ` John Hubbard
2023-06-06 7:55 ` Muhammad Usama Anjum
2023-06-06 7:16 ` [PATCH v3 08/11] selftests/mm: fix build failures due to missing MADV_COLLAPSE John Hubbard
` (3 subsequent siblings)
10 siblings, 1 reply; 39+ messages in thread
From: John Hubbard @ 2023-06-06 7:16 UTC (permalink / raw)
To: Andrew Morton
Cc: David Hildenbrand, Peter Xu, Shuah Khan, Nathan Chancellor,
linux-mm, linux-kselftest, LKML, John Hubbard
This fixes a real bug, too, because xstate_size() was assuming that
the stack variable xstate_size was initialized to zero. That's not
guaranteed nor even especially likely.
Reviewed-by: David Hildenbrand <david@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
tools/testing/selftests/mm/pkey-x86.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/mm/pkey-x86.h b/tools/testing/selftests/mm/pkey-x86.h
index 72c14cd3ddc7..e32ae8a1cd99 100644
--- a/tools/testing/selftests/mm/pkey-x86.h
+++ b/tools/testing/selftests/mm/pkey-x86.h
@@ -132,7 +132,7 @@ int pkey_reg_xstate_offset(void)
unsigned int ecx;
unsigned int edx;
int xstate_offset;
- int xstate_size;
+ int xstate_size = 0;
unsigned long XSTATE_CPUID = 0xd;
int leaf;
--
2.40.1
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH v3 07/11] selftests/mm: fix a "possibly uninitialized" warning in pkey-x86.h
2023-06-06 7:16 ` [PATCH v3 07/11] selftests/mm: fix a "possibly uninitialized" warning in pkey-x86.h John Hubbard
@ 2023-06-06 7:55 ` Muhammad Usama Anjum
0 siblings, 0 replies; 39+ messages in thread
From: Muhammad Usama Anjum @ 2023-06-06 7:55 UTC (permalink / raw)
To: John Hubbard, Andrew Morton
Cc: Muhammad Usama Anjum, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML
On 6/6/23 12:16 PM, John Hubbard wrote:
> This fixes a real bug, too, because xstate_size() was assuming that
> the stack variable xstate_size was initialized to zero. That's not
> guaranteed nor even especially likely.
>
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Cc: Peter Xu <peterx@redhat.com>
> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Tested-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
> ---
> tools/testing/selftests/mm/pkey-x86.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/testing/selftests/mm/pkey-x86.h b/tools/testing/selftests/mm/pkey-x86.h
> index 72c14cd3ddc7..e32ae8a1cd99 100644
> --- a/tools/testing/selftests/mm/pkey-x86.h
> +++ b/tools/testing/selftests/mm/pkey-x86.h
> @@ -132,7 +132,7 @@ int pkey_reg_xstate_offset(void)
> unsigned int ecx;
> unsigned int edx;
> int xstate_offset;
> - int xstate_size;
> + int xstate_size = 0;
> unsigned long XSTATE_CPUID = 0xd;
> int leaf;
>
--
BR,
Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH v3 08/11] selftests/mm: fix build failures due to missing MADV_COLLAPSE
2023-06-06 7:16 [PATCH v3 00/11] A minor flurry of selftest/mm fixes John Hubbard
` (6 preceding siblings ...)
2023-06-06 7:16 ` [PATCH v3 07/11] selftests/mm: fix a "possibly uninitialized" warning in pkey-x86.h John Hubbard
@ 2023-06-06 7:16 ` John Hubbard
2023-06-06 7:55 ` Muhammad Usama Anjum
2023-06-06 7:16 ` [PATCH v3 09/11] selftests/mm: move certain uffd*() routines from vm_util.c to uffd-common.c John Hubbard
` (2 subsequent siblings)
10 siblings, 1 reply; 39+ messages in thread
From: John Hubbard @ 2023-06-06 7:16 UTC (permalink / raw)
To: Andrew Morton
Cc: David Hildenbrand, Peter Xu, Shuah Khan, Nathan Chancellor,
linux-mm, linux-kselftest, LKML, John Hubbard,
Muhammad Usama Anjum
MADV_PAGEOUT, MADV_POPULATE_READ, MADV_COLLAPSE are conditionally
defined as necessary. However, that was being done in .c files, and a
new build failure came up that would have been automatically avoided had
these been in a common header file.
So consolidate and move them all to vm_util.h, which fixes the build
failure.
An alternative approach from Muhammad Usama Anjum was: rely on "make
headers" being required, and include asm-generic/mman-common.h. This
works in the sense that it builds, but it still generates warnings about
duplicate MADV_* symbols, and the goal here is to get a fully clean (no
warnings) build here.
Reviewed-by: David Hildenbrand <david@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
tools/testing/selftests/mm/cow.c | 7 -------
tools/testing/selftests/mm/khugepaged.c | 10 ----------
tools/testing/selftests/mm/vm_util.h | 10 ++++++++++
3 files changed, 10 insertions(+), 17 deletions(-)
diff --git a/tools/testing/selftests/mm/cow.c b/tools/testing/selftests/mm/cow.c
index dc9d6fe86028..8882b05ec9c8 100644
--- a/tools/testing/selftests/mm/cow.c
+++ b/tools/testing/selftests/mm/cow.c
@@ -30,13 +30,6 @@
#include "../kselftest.h"
#include "vm_util.h"
-#ifndef MADV_PAGEOUT
-#define MADV_PAGEOUT 21
-#endif
-#ifndef MADV_COLLAPSE
-#define MADV_COLLAPSE 25
-#endif
-
static size_t pagesize;
static int pagemap_fd;
static size_t thpsize;
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c
index 97adc0f34f9c..e88ee039d0eb 100644
--- a/tools/testing/selftests/mm/khugepaged.c
+++ b/tools/testing/selftests/mm/khugepaged.c
@@ -22,16 +22,6 @@
#include "vm_util.h"
-#ifndef MADV_PAGEOUT
-#define MADV_PAGEOUT 21
-#endif
-#ifndef MADV_POPULATE_READ
-#define MADV_POPULATE_READ 22
-#endif
-#ifndef MADV_COLLAPSE
-#define MADV_COLLAPSE 25
-#endif
-
#define BASE_ADDR ((void *)(1UL << 30))
static unsigned long hpage_pmd_size;
static unsigned long page_size;
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index b950bd16083a..07f39ed2efba 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -63,3 +63,13 @@ int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,
#define PAGEMAP_PRESENT(ent) (((ent) & (1ull << 63)) != 0)
#define PAGEMAP_PFN(ent) ((ent) & ((1ull << 55) - 1))
+
+#ifndef MADV_PAGEOUT
+#define MADV_PAGEOUT 21
+#endif
+#ifndef MADV_POPULATE_READ
+#define MADV_POPULATE_READ 22
+#endif
+#ifndef MADV_COLLAPSE
+#define MADV_COLLAPSE 25
+#endif
--
2.40.1
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH v3 08/11] selftests/mm: fix build failures due to missing MADV_COLLAPSE
2023-06-06 7:16 ` [PATCH v3 08/11] selftests/mm: fix build failures due to missing MADV_COLLAPSE John Hubbard
@ 2023-06-06 7:55 ` Muhammad Usama Anjum
0 siblings, 0 replies; 39+ messages in thread
From: Muhammad Usama Anjum @ 2023-06-06 7:55 UTC (permalink / raw)
To: John Hubbard, Andrew Morton
Cc: Muhammad Usama Anjum, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML
On 6/6/23 12:16 PM, John Hubbard wrote:
> MADV_PAGEOUT, MADV_POPULATE_READ, MADV_COLLAPSE are conditionally
> defined as necessary. However, that was being done in .c files, and a
> new build failure came up that would have been automatically avoided had
> these been in a common header file.
>
> So consolidate and move them all to vm_util.h, which fixes the build
> failure.
>
> An alternative approach from Muhammad Usama Anjum was: rely on "make
> headers" being required, and include asm-generic/mman-common.h. This
> works in the sense that it builds, but it still generates warnings about
> duplicate MADV_* symbols, and the goal here is to get a fully clean (no
> warnings) build here.
>
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Cc: Peter Xu <peterx@redhat.com>
> Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Tested-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
> ---
> tools/testing/selftests/mm/cow.c | 7 -------
> tools/testing/selftests/mm/khugepaged.c | 10 ----------
> tools/testing/selftests/mm/vm_util.h | 10 ++++++++++
> 3 files changed, 10 insertions(+), 17 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/cow.c b/tools/testing/selftests/mm/cow.c
> index dc9d6fe86028..8882b05ec9c8 100644
> --- a/tools/testing/selftests/mm/cow.c
> +++ b/tools/testing/selftests/mm/cow.c
> @@ -30,13 +30,6 @@
> #include "../kselftest.h"
> #include "vm_util.h"
>
> -#ifndef MADV_PAGEOUT
> -#define MADV_PAGEOUT 21
> -#endif
> -#ifndef MADV_COLLAPSE
> -#define MADV_COLLAPSE 25
> -#endif
> -
> static size_t pagesize;
> static int pagemap_fd;
> static size_t thpsize;
> diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c
> index 97adc0f34f9c..e88ee039d0eb 100644
> --- a/tools/testing/selftests/mm/khugepaged.c
> +++ b/tools/testing/selftests/mm/khugepaged.c
> @@ -22,16 +22,6 @@
>
> #include "vm_util.h"
>
> -#ifndef MADV_PAGEOUT
> -#define MADV_PAGEOUT 21
> -#endif
> -#ifndef MADV_POPULATE_READ
> -#define MADV_POPULATE_READ 22
> -#endif
> -#ifndef MADV_COLLAPSE
> -#define MADV_COLLAPSE 25
> -#endif
> -
> #define BASE_ADDR ((void *)(1UL << 30))
> static unsigned long hpage_pmd_size;
> static unsigned long page_size;
> diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
> index b950bd16083a..07f39ed2efba 100644
> --- a/tools/testing/selftests/mm/vm_util.h
> +++ b/tools/testing/selftests/mm/vm_util.h
> @@ -63,3 +63,13 @@ int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,
>
> #define PAGEMAP_PRESENT(ent) (((ent) & (1ull << 63)) != 0)
> #define PAGEMAP_PFN(ent) ((ent) & ((1ull << 55) - 1))
> +
> +#ifndef MADV_PAGEOUT
> +#define MADV_PAGEOUT 21
> +#endif
> +#ifndef MADV_POPULATE_READ
> +#define MADV_POPULATE_READ 22
> +#endif
> +#ifndef MADV_COLLAPSE
> +#define MADV_COLLAPSE 25
> +#endif
--
BR,
Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH v3 09/11] selftests/mm: move certain uffd*() routines from vm_util.c to uffd-common.c
2023-06-06 7:16 [PATCH v3 00/11] A minor flurry of selftest/mm fixes John Hubbard
` (7 preceding siblings ...)
2023-06-06 7:16 ` [PATCH v3 08/11] selftests/mm: fix build failures due to missing MADV_COLLAPSE John Hubbard
@ 2023-06-06 7:16 ` John Hubbard
2023-06-06 7:56 ` Muhammad Usama Anjum
2023-06-06 7:16 ` [PATCH v3 10/11] Documentation: kselftest: "make headers" is a prerequisite John Hubbard
2023-06-06 7:16 ` [PATCH v3 11/11] selftests: error out if kernel header files are not yet built John Hubbard
10 siblings, 1 reply; 39+ messages in thread
From: John Hubbard @ 2023-06-06 7:16 UTC (permalink / raw)
To: Andrew Morton
Cc: David Hildenbrand, Peter Xu, Shuah Khan, Nathan Chancellor,
linux-mm, linux-kselftest, LKML, John Hubbard
There are only three uffd*() routines that are used outside of the uffd
selftests. Leave these in vm_util.c, where they are available to any mm
selftest program:
uffd_register()
uffd_unregister()
uffd_register_with_ioctls().
A few other uffd*() routines, however, are only used by the uffd-focused
tests found in uffd-stress.c and uffd-unit-tests.c. Move those routines
into uffd-common.c.
Cc: Peter Xu <peterx@redhat.com>
Acked-by: David Hildenbrand <david@redhat.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
tools/testing/selftests/mm/uffd-common.c | 59 ++++++++++++++++++++++++
tools/testing/selftests/mm/uffd-common.h | 5 ++
tools/testing/selftests/mm/vm_util.c | 59 ------------------------
tools/testing/selftests/mm/vm_util.h | 4 --
4 files changed, 64 insertions(+), 63 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index 61c6250adf93..ba20d7504022 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -616,3 +616,62 @@ int copy_page(int ufd, unsigned long offset, bool wp)
{
return __copy_page(ufd, offset, false, wp);
}
+
+int uffd_open_dev(unsigned int flags)
+{
+ int fd, uffd;
+
+ fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
+ if (fd < 0)
+ return fd;
+ uffd = ioctl(fd, USERFAULTFD_IOC_NEW, flags);
+ close(fd);
+
+ return uffd;
+}
+
+int uffd_open_sys(unsigned int flags)
+{
+#ifdef __NR_userfaultfd
+ return syscall(__NR_userfaultfd, flags);
+#else
+ return -1;
+#endif
+}
+
+int uffd_open(unsigned int flags)
+{
+ int uffd = uffd_open_sys(flags);
+
+ if (uffd < 0)
+ uffd = uffd_open_dev(flags);
+
+ return uffd;
+}
+
+int uffd_get_features(uint64_t *features)
+{
+ struct uffdio_api uffdio_api = { .api = UFFD_API, .features = 0 };
+ /*
+ * This should by default work in most kernels; the feature list
+ * will be the same no matter what we pass in here.
+ */
+ int fd = uffd_open(UFFD_USER_MODE_ONLY);
+
+ if (fd < 0)
+ /* Maybe the kernel is older than user-only mode? */
+ fd = uffd_open(0);
+
+ if (fd < 0)
+ return fd;
+
+ if (ioctl(fd, UFFDIO_API, &uffdio_api)) {
+ close(fd);
+ return -errno;
+ }
+
+ *features = uffdio_api.features;
+ close(fd);
+
+ return 0;
+}
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index 6068f2346b86..197f5262fe0d 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -110,6 +110,11 @@ int __copy_page(int ufd, unsigned long offset, bool retry, bool wp);
int copy_page(int ufd, unsigned long offset, bool wp);
void *uffd_poll_thread(void *arg);
+int uffd_open_dev(unsigned int flags);
+int uffd_open_sys(unsigned int flags);
+int uffd_open(unsigned int flags);
+int uffd_get_features(uint64_t *features);
+
#define TEST_ANON 1
#define TEST_HUGETLB 2
#define TEST_SHMEM 3
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 9b06a5034808..681277615839 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -242,62 +242,3 @@ int uffd_unregister(int uffd, void *addr, uint64_t len)
return ret;
}
-
-int uffd_open_dev(unsigned int flags)
-{
- int fd, uffd;
-
- fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
- if (fd < 0)
- return fd;
- uffd = ioctl(fd, USERFAULTFD_IOC_NEW, flags);
- close(fd);
-
- return uffd;
-}
-
-int uffd_open_sys(unsigned int flags)
-{
-#ifdef __NR_userfaultfd
- return syscall(__NR_userfaultfd, flags);
-#else
- return -1;
-#endif
-}
-
-int uffd_open(unsigned int flags)
-{
- int uffd = uffd_open_sys(flags);
-
- if (uffd < 0)
- uffd = uffd_open_dev(flags);
-
- return uffd;
-}
-
-int uffd_get_features(uint64_t *features)
-{
- struct uffdio_api uffdio_api = { .api = UFFD_API, .features = 0 };
- /*
- * This should by default work in most kernels; the feature list
- * will be the same no matter what we pass in here.
- */
- int fd = uffd_open(UFFD_USER_MODE_ONLY);
-
- if (fd < 0)
- /* Maybe the kernel is older than user-only mode? */
- fd = uffd_open(0);
-
- if (fd < 0)
- return fd;
-
- if (ioctl(fd, UFFDIO_API, &uffdio_api)) {
- close(fd);
- return -errno;
- }
-
- *features = uffdio_api.features;
- close(fd);
-
- return 0;
-}
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index 07f39ed2efba..c2d4ff798b91 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -48,10 +48,6 @@ unsigned long default_huge_page_size(void);
int uffd_register(int uffd, void *addr, uint64_t len,
bool miss, bool wp, bool minor);
int uffd_unregister(int uffd, void *addr, uint64_t len);
-int uffd_open_dev(unsigned int flags);
-int uffd_open_sys(unsigned int flags);
-int uffd_open(unsigned int flags);
-int uffd_get_features(uint64_t *features);
int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,
bool miss, bool wp, bool minor, uint64_t *ioctls);
--
2.40.1
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH v3 09/11] selftests/mm: move certain uffd*() routines from vm_util.c to uffd-common.c
2023-06-06 7:16 ` [PATCH v3 09/11] selftests/mm: move certain uffd*() routines from vm_util.c to uffd-common.c John Hubbard
@ 2023-06-06 7:56 ` Muhammad Usama Anjum
0 siblings, 0 replies; 39+ messages in thread
From: Muhammad Usama Anjum @ 2023-06-06 7:56 UTC (permalink / raw)
To: John Hubbard, Andrew Morton
Cc: Muhammad Usama Anjum, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML
On 6/6/23 12:16 PM, John Hubbard wrote:
> There are only three uffd*() routines that are used outside of the uffd
> selftests. Leave these in vm_util.c, where they are available to any mm
> selftest program:
>
> uffd_register()
> uffd_unregister()
> uffd_register_with_ioctls().
>
> A few other uffd*() routines, however, are only used by the uffd-focused
> tests found in uffd-stress.c and uffd-unit-tests.c. Move those routines
> into uffd-common.c.
>
> Cc: Peter Xu <peterx@redhat.com>
> Acked-by: David Hildenbrand <david@redhat.com>
> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Tested-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
> ---
> tools/testing/selftests/mm/uffd-common.c | 59 ++++++++++++++++++++++++
> tools/testing/selftests/mm/uffd-common.h | 5 ++
> tools/testing/selftests/mm/vm_util.c | 59 ------------------------
> tools/testing/selftests/mm/vm_util.h | 4 --
> 4 files changed, 64 insertions(+), 63 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
> index 61c6250adf93..ba20d7504022 100644
> --- a/tools/testing/selftests/mm/uffd-common.c
> +++ b/tools/testing/selftests/mm/uffd-common.c
> @@ -616,3 +616,62 @@ int copy_page(int ufd, unsigned long offset, bool wp)
> {
> return __copy_page(ufd, offset, false, wp);
> }
> +
> +int uffd_open_dev(unsigned int flags)
> +{
> + int fd, uffd;
> +
> + fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
> + if (fd < 0)
> + return fd;
> + uffd = ioctl(fd, USERFAULTFD_IOC_NEW, flags);
> + close(fd);
> +
> + return uffd;
> +}
> +
> +int uffd_open_sys(unsigned int flags)
> +{
> +#ifdef __NR_userfaultfd
> + return syscall(__NR_userfaultfd, flags);
> +#else
> + return -1;
> +#endif
> +}
> +
> +int uffd_open(unsigned int flags)
> +{
> + int uffd = uffd_open_sys(flags);
> +
> + if (uffd < 0)
> + uffd = uffd_open_dev(flags);
> +
> + return uffd;
> +}
> +
> +int uffd_get_features(uint64_t *features)
> +{
> + struct uffdio_api uffdio_api = { .api = UFFD_API, .features = 0 };
> + /*
> + * This should by default work in most kernels; the feature list
> + * will be the same no matter what we pass in here.
> + */
> + int fd = uffd_open(UFFD_USER_MODE_ONLY);
> +
> + if (fd < 0)
> + /* Maybe the kernel is older than user-only mode? */
> + fd = uffd_open(0);
> +
> + if (fd < 0)
> + return fd;
> +
> + if (ioctl(fd, UFFDIO_API, &uffdio_api)) {
> + close(fd);
> + return -errno;
> + }
> +
> + *features = uffdio_api.features;
> + close(fd);
> +
> + return 0;
> +}
> diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
> index 6068f2346b86..197f5262fe0d 100644
> --- a/tools/testing/selftests/mm/uffd-common.h
> +++ b/tools/testing/selftests/mm/uffd-common.h
> @@ -110,6 +110,11 @@ int __copy_page(int ufd, unsigned long offset, bool retry, bool wp);
> int copy_page(int ufd, unsigned long offset, bool wp);
> void *uffd_poll_thread(void *arg);
>
> +int uffd_open_dev(unsigned int flags);
> +int uffd_open_sys(unsigned int flags);
> +int uffd_open(unsigned int flags);
> +int uffd_get_features(uint64_t *features);
> +
> #define TEST_ANON 1
> #define TEST_HUGETLB 2
> #define TEST_SHMEM 3
> diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
> index 9b06a5034808..681277615839 100644
> --- a/tools/testing/selftests/mm/vm_util.c
> +++ b/tools/testing/selftests/mm/vm_util.c
> @@ -242,62 +242,3 @@ int uffd_unregister(int uffd, void *addr, uint64_t len)
>
> return ret;
> }
> -
> -int uffd_open_dev(unsigned int flags)
> -{
> - int fd, uffd;
> -
> - fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
> - if (fd < 0)
> - return fd;
> - uffd = ioctl(fd, USERFAULTFD_IOC_NEW, flags);
> - close(fd);
> -
> - return uffd;
> -}
> -
> -int uffd_open_sys(unsigned int flags)
> -{
> -#ifdef __NR_userfaultfd
> - return syscall(__NR_userfaultfd, flags);
> -#else
> - return -1;
> -#endif
> -}
> -
> -int uffd_open(unsigned int flags)
> -{
> - int uffd = uffd_open_sys(flags);
> -
> - if (uffd < 0)
> - uffd = uffd_open_dev(flags);
> -
> - return uffd;
> -}
> -
> -int uffd_get_features(uint64_t *features)
> -{
> - struct uffdio_api uffdio_api = { .api = UFFD_API, .features = 0 };
> - /*
> - * This should by default work in most kernels; the feature list
> - * will be the same no matter what we pass in here.
> - */
> - int fd = uffd_open(UFFD_USER_MODE_ONLY);
> -
> - if (fd < 0)
> - /* Maybe the kernel is older than user-only mode? */
> - fd = uffd_open(0);
> -
> - if (fd < 0)
> - return fd;
> -
> - if (ioctl(fd, UFFDIO_API, &uffdio_api)) {
> - close(fd);
> - return -errno;
> - }
> -
> - *features = uffdio_api.features;
> - close(fd);
> -
> - return 0;
> -}
> diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
> index 07f39ed2efba..c2d4ff798b91 100644
> --- a/tools/testing/selftests/mm/vm_util.h
> +++ b/tools/testing/selftests/mm/vm_util.h
> @@ -48,10 +48,6 @@ unsigned long default_huge_page_size(void);
> int uffd_register(int uffd, void *addr, uint64_t len,
> bool miss, bool wp, bool minor);
> int uffd_unregister(int uffd, void *addr, uint64_t len);
> -int uffd_open_dev(unsigned int flags);
> -int uffd_open_sys(unsigned int flags);
> -int uffd_open(unsigned int flags);
> -int uffd_get_features(uint64_t *features);
> int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,
> bool miss, bool wp, bool minor, uint64_t *ioctls);
>
--
BR,
Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH v3 10/11] Documentation: kselftest: "make headers" is a prerequisite
2023-06-06 7:16 [PATCH v3 00/11] A minor flurry of selftest/mm fixes John Hubbard
` (8 preceding siblings ...)
2023-06-06 7:16 ` [PATCH v3 09/11] selftests/mm: move certain uffd*() routines from vm_util.c to uffd-common.c John Hubbard
@ 2023-06-06 7:16 ` John Hubbard
2023-06-06 7:57 ` Muhammad Usama Anjum
2023-07-10 14:20 ` Mark Brown
2023-06-06 7:16 ` [PATCH v3 11/11] selftests: error out if kernel header files are not yet built John Hubbard
10 siblings, 2 replies; 39+ messages in thread
From: John Hubbard @ 2023-06-06 7:16 UTC (permalink / raw)
To: Andrew Morton
Cc: David Hildenbrand, Peter Xu, Shuah Khan, Nathan Chancellor,
linux-mm, linux-kselftest, LKML, John Hubbard,
Muhammad Usama Anjum, Jonathan Corbet, linux-doc
As per a discussion with Muhammad Usama Anjum [1], the following is how
one is supposed to build selftests:
make headers && make -C tools/testing/selftests/mm
However, that's not yet documented anywhere. So add it to
Documentation/dev-tools/kselftest.rst .
[1] https://lore.kernel.org/all/bf910fa5-0c96-3707-cce4-5bcc656b6274@collabora.com/
Cc: Peter Xu <peterx@redhat.com>
Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: linux-doc@vger.kernel.org
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
Documentation/dev-tools/kselftest.rst | 1 +
1 file changed, 1 insertion(+)
diff --git a/Documentation/dev-tools/kselftest.rst b/Documentation/dev-tools/kselftest.rst
index 12b575b76b20..6e35d042199c 100644
--- a/Documentation/dev-tools/kselftest.rst
+++ b/Documentation/dev-tools/kselftest.rst
@@ -36,6 +36,7 @@ Running the selftests (hotplug tests are run in limited mode)
To build the tests::
+ $ make headers
$ make -C tools/testing/selftests
To run the tests::
--
2.40.1
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH v3 10/11] Documentation: kselftest: "make headers" is a prerequisite
2023-06-06 7:16 ` [PATCH v3 10/11] Documentation: kselftest: "make headers" is a prerequisite John Hubbard
@ 2023-06-06 7:57 ` Muhammad Usama Anjum
2023-07-10 14:20 ` Mark Brown
1 sibling, 0 replies; 39+ messages in thread
From: Muhammad Usama Anjum @ 2023-06-06 7:57 UTC (permalink / raw)
To: John Hubbard, Andrew Morton
Cc: Muhammad Usama Anjum, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Jonathan Corbet, linux-doc
On 6/6/23 12:16 PM, John Hubbard wrote:
> As per a discussion with Muhammad Usama Anjum [1], the following is how
> one is supposed to build selftests:
>
> make headers && make -C tools/testing/selftests/mm
>
> However, that's not yet documented anywhere. So add it to
> Documentation/dev-tools/kselftest.rst .
>
> [1] https://lore.kernel.org/all/bf910fa5-0c96-3707-cce4-5bcc656b6274@collabora.com/
>
> Cc: Peter Xu <peterx@redhat.com>
> Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: linux-doc@vger.kernel.org
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
> ---
> Documentation/dev-tools/kselftest.rst | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/Documentation/dev-tools/kselftest.rst b/Documentation/dev-tools/kselftest.rst
> index 12b575b76b20..6e35d042199c 100644
> --- a/Documentation/dev-tools/kselftest.rst
> +++ b/Documentation/dev-tools/kselftest.rst
> @@ -36,6 +36,7 @@ Running the selftests (hotplug tests are run in limited mode)
>
> To build the tests::
>
> + $ make headers
> $ make -C tools/testing/selftests
>
> To run the tests::
--
BR,
Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v3 10/11] Documentation: kselftest: "make headers" is a prerequisite
2023-06-06 7:16 ` [PATCH v3 10/11] Documentation: kselftest: "make headers" is a prerequisite John Hubbard
2023-06-06 7:57 ` Muhammad Usama Anjum
@ 2023-07-10 14:20 ` Mark Brown
1 sibling, 0 replies; 39+ messages in thread
From: Mark Brown @ 2023-07-10 14:20 UTC (permalink / raw)
To: John Hubbard
Cc: Andrew Morton, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Muhammad Usama Anjum, Jonathan Corbet, linux-doc
[-- Attachment #1: Type: text/plain, Size: 556 bytes --]
On Tue, Jun 06, 2023 at 12:16:36AM -0700, John Hubbard wrote:
> As per a discussion with Muhammad Usama Anjum [1], the following is how
> one is supposed to build selftests:
>
> make headers && make -C tools/testing/selftests/mm
>
> However, that's not yet documented anywhere. So add it to
> Documentation/dev-tools/kselftest.rst .
This is breaking the arm64 selftests, I've sent a revert:
https://lore.kernel.org/linux-kselftest/20230710-kselftest-fix-arm64-v1-1-48e872844f25@kernel.org/T/#u
(logs included in the above patch.)
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-06-06 7:16 [PATCH v3 00/11] A minor flurry of selftest/mm fixes John Hubbard
` (9 preceding siblings ...)
2023-06-06 7:16 ` [PATCH v3 10/11] Documentation: kselftest: "make headers" is a prerequisite John Hubbard
@ 2023-06-06 7:16 ` John Hubbard
2023-06-06 7:38 ` Muhammad Usama Anjum
` (3 more replies)
10 siblings, 4 replies; 39+ messages in thread
From: John Hubbard @ 2023-06-06 7:16 UTC (permalink / raw)
To: Andrew Morton
Cc: David Hildenbrand, Peter Xu, Shuah Khan, Nathan Chancellor,
linux-mm, linux-kselftest, LKML, John Hubbard,
Muhammad Usama Anjum, Jonathan Corbet, linux-doc
As per a discussion with Muhammad Usama Anjum [1], the following is how
one is supposed to build selftests:
make headers && make -C tools/testing/selftests/mm
Change the selftest build system's lib.mk to fail out with a helpful
message if that prerequisite "make headers" has not been done yet.
[1] https://lore.kernel.org/all/bf910fa5-0c96-3707-cce4-5bcc656b6274@collabora.com/
Cc: David Hildenbrand <david@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: linux-doc@vger.kernel.org
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
tools/testing/selftests/lib.mk | 36 +++++++++++++++++++++++++++++++---
1 file changed, 33 insertions(+), 3 deletions(-)
diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
index 05400462c779..b8ea03b9a015 100644
--- a/tools/testing/selftests/lib.mk
+++ b/tools/testing/selftests/lib.mk
@@ -44,10 +44,22 @@ endif
selfdir = $(realpath $(dir $(filter %/lib.mk,$(MAKEFILE_LIST))))
top_srcdir = $(selfdir)/../../..
-ifeq ($(KHDR_INCLUDES),)
-KHDR_INCLUDES := -isystem $(top_srcdir)/usr/include
+ifneq ($(KBUILD_OUTPUT),)
+ # Make's built-in functions such as $(abspath ...), $(realpath ...) cannot
+ # expand a shell special character '~'. We use a somewhat tedious way here.
+ abs_objtree := $(shell cd $(top_srcdir) && mkdir -p $(KBUILD_OUTPUT) && cd $(KBUILD_OUTPUT) && pwd)
+ $(if $(abs_objtree),, \
+ $(error failed to create output directory "$(KBUILD_OUTPUT)"))
+ # $(realpath ...) resolves symlinks
+ abs_objtree := $(realpath $(abs_objtree))
+ KHDR_DIR := ${abs_objtree}/usr/include
+else
+ abs_srctree := $(shell cd $(top_srcdir) && pwd)
+ KHDR_DIR := ${abs_srctree}/usr/include
endif
+KHDR_INCLUDES := -isystem $(KHDR_DIR)
+
# The following are built by lib.mk common compile rules.
# TEST_CUSTOM_PROGS should be used by tests that require
# custom build rule and prevent common build rule use.
@@ -58,7 +70,25 @@ TEST_GEN_PROGS := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS))
TEST_GEN_PROGS_EXTENDED := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS_EXTENDED))
TEST_GEN_FILES := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_FILES))
-all: $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES)
+all: kernel_header_files $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) \
+ $(TEST_GEN_FILES)
+
+kernel_header_files:
+ @ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null; \
+ if [ $$? -ne 0 ]; then \
+ RED='\033[1;31m'; \
+ NOCOLOR='\033[0m'; \
+ echo; \
+ echo -e "$${RED}error$${NOCOLOR}: missing kernel header files."; \
+ echo "Please run this and try again:"; \
+ echo; \
+ echo " cd $(top_srcdir)"; \
+ echo " make headers"; \
+ echo; \
+ exit 1; \
+ fi
+
+.PHONY: kernel_header_files
define RUN_TESTS
BASE_DIR="$(selfdir)"; \
--
2.40.1
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-06-06 7:16 ` [PATCH v3 11/11] selftests: error out if kernel header files are not yet built John Hubbard
@ 2023-06-06 7:38 ` Muhammad Usama Anjum
2023-06-06 20:10 ` John Hubbard
2023-06-06 7:57 ` Muhammad Usama Anjum
` (2 subsequent siblings)
3 siblings, 1 reply; 39+ messages in thread
From: Muhammad Usama Anjum @ 2023-06-06 7:38 UTC (permalink / raw)
To: John Hubbard, Andrew Morton
Cc: Muhammad Usama Anjum, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Jonathan Corbet, linux-doc
[-- Attachment #1: Type: text/plain, Size: 4749 bytes --]
On 6/6/23 12:16 PM, John Hubbard wrote:
> As per a discussion with Muhammad Usama Anjum [1], the following is how
> one is supposed to build selftests:
>
> make headers && make -C tools/testing/selftests/mm
>
> Change the selftest build system's lib.mk to fail out with a helpful
> message if that prerequisite "make headers" has not been done yet.
>
> [1] https://lore.kernel.org/all/bf910fa5-0c96-3707-cce4-5bcc656b6274@collabora.com/
>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: Peter Xu <peterx@redhat.com>
> Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: linux-doc@vger.kernel.org
> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
> ---
> tools/testing/selftests/lib.mk | 36 +++++++++++++++++++++++++++++++---
> 1 file changed, 33 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
> index 05400462c779..b8ea03b9a015 100644
> --- a/tools/testing/selftests/lib.mk
> +++ b/tools/testing/selftests/lib.mk
> @@ -44,10 +44,22 @@ endif
> selfdir = $(realpath $(dir $(filter %/lib.mk,$(MAKEFILE_LIST))))
> top_srcdir = $(selfdir)/../../..
>
> -ifeq ($(KHDR_INCLUDES),)
> -KHDR_INCLUDES := -isystem $(top_srcdir)/usr/include
> +ifneq ($(KBUILD_OUTPUT),)
> + # Make's built-in functions such as $(abspath ...), $(realpath ...) cannot
> + # expand a shell special character '~'. We use a somewhat tedious way here.
> + abs_objtree := $(shell cd $(top_srcdir) && mkdir -p $(KBUILD_OUTPUT) && cd $(KBUILD_OUTPUT) && pwd)
> + $(if $(abs_objtree),, \
> + $(error failed to create output directory "$(KBUILD_OUTPUT)"))
> + # $(realpath ...) resolves symlinks
> + abs_objtree := $(realpath $(abs_objtree))
> + KHDR_DIR := ${abs_objtree}/usr/include
> +else
> + abs_srctree := $(shell cd $(top_srcdir) && pwd)
> + KHDR_DIR := ${abs_srctree}/usr/include
> endif
>
> +KHDR_INCLUDES := -isystem $(KHDR_DIR)
> +
> # The following are built by lib.mk common compile rules.
> # TEST_CUSTOM_PROGS should be used by tests that require
> # custom build rule and prevent common build rule use.
> @@ -58,7 +70,25 @@ TEST_GEN_PROGS := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS))
> TEST_GEN_PROGS_EXTENDED := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS_EXTENDED))
> TEST_GEN_FILES := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_FILES))
>
> -all: $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES)
> +all: kernel_header_files $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) \
> + $(TEST_GEN_FILES)
> +
> +kernel_header_files:
> + @ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null; \
> + if [ $$? -ne 0 ]; then \
> + RED='\033[1;31m'; \
> + NOCOLOR='\033[0m'; \
> + echo; \
> + echo -e "$${RED}error$${NOCOLOR}: missing kernel header files."; \
> + echo "Please run this and try again:"; \
> + echo; \
> + echo " cd $(top_srcdir)"; \
> + echo " make headers"; \
> + echo; \
> + exit 1; \
> + fi
Thank you for adding this. This is outputting error for every selftest
directory. We should try to make it even better by just aborting the
Make-ing process the first time headers aren't detected. We can do this now
or later, fine by me.
make[1]: Entering directory
'/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/futex'
-e error: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: *** [../lib.mk:77: kernel_header_files] Error 1
make[1]: Leaving directory
'/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/futex'
make[1]: Entering directory
'/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/gpio'
-e error: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: *** [../lib.mk:77: kernel_header_files] Error 1
make[1]: Leaving directory
'/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/gpio'
m
Complete error log file is attached.
> +
> +.PHONY: kernel_header_files
>
> define RUN_TESTS
> BASE_DIR="$(selfdir)"; \
--
BR,
Muhammad Usama Anjum
[-- Attachment #2: log --]
[-- Type: text/plain, Size: 33996 bytes --]
make: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/alsa'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/alsa'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/amd-pstate'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/amd-pstate'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/arm64'
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/arm64'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/breakpoints'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/breakpoints'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cachestat'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cachestat'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/capabilities'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/capabilities'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cgroup'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cgroup'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/clone3'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/clone3'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/core'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/core'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cpufreq'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cpufreq'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cpu-hotplug'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cpu-hotplug'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/damon'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/damon'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/dma-buf'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/dma-buf'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/s390x/uvdevice'
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/s390x/uvdevice'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/net/bonding'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/net/bonding'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/net/team'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/net/team'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/efivarfs'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/efivarfs'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/exec'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/exec'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems/binderfs'
make[1]: 'binderfs_test' is up to date.
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems/binderfs'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems/epoll'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems/epoll'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems/fat'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems/fat'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/firmware'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/firmware'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/fpu'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/fpu'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ftrace'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ftrace'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/futex'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/futex'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/gpio'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/gpio'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/hid'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/hid'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/intel_pstate'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/intel_pstate'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/iommu'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/iommu'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ipc'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ipc'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ir'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ir'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/kcmp'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/kcmp'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/kexec'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/kexec'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/kvm'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/kvm'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/landlock'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/landlock'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/lib'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/lib'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/livepatch'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/livepatch'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/lkdtm'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/lkdtm'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/membarrier'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/membarrier'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/memfd'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/memfd'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/memory-hotplug'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/memory-hotplug'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mincore'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mincore'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mount'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mount'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mount_setattr'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mount_setattr'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/move_mount_set_group'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/move_mount_set_group'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mqueue'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mqueue'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/nci'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/nci'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/af_unix'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/af_unix'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/forwarding'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/forwarding'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/hsr'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/hsr'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/mptcp'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/mptcp'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/openvswitch'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/openvswitch'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/netfilter'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/netfilter'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/nsfs'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/nsfs'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/pidfd'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/pidfd'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/pid_namespace'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/pid_namespace'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/powerpc'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/powerpc'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/prctl'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/prctl'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/proc'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/proc'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/pstore'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/pstore'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ptrace'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ptrace'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/openat2'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/openat2'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/resctrl'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/resctrl'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/riscv'
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/riscv'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/rlimits'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/rlimits'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/rseq'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/rseq'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/rtc'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/rtc'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/seccomp'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/seccomp'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sgx'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sgx'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sigaltstack'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sigaltstack'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/size'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/size'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sparc64'
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sparc64'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/splice'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/splice'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/static_keys'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/static_keys'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sync'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sync'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/syscall_user_dispatch'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/syscall_user_dispatch'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sysctl'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sysctl'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tc-testing'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tc-testing'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tdx'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tdx'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/timens'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/timens'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/timers'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/timers'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tmpfs'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tmpfs'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tpm2'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tpm2'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/user'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/user'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/vDSO'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/vDSO'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mm'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mm'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/x86'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/x86'
make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/zram'
-e ^[[1;31merror^[[0m: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/zram'
make: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests'
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-06-06 7:38 ` Muhammad Usama Anjum
@ 2023-06-06 20:10 ` John Hubbard
2023-06-07 5:37 ` Muhammad Usama Anjum
0 siblings, 1 reply; 39+ messages in thread
From: John Hubbard @ 2023-06-06 20:10 UTC (permalink / raw)
To: Muhammad Usama Anjum, Andrew Morton
Cc: David Hildenbrand, Peter Xu, Shuah Khan, Nathan Chancellor,
linux-mm, linux-kselftest, LKML, Jonathan Corbet, linux-doc
[-- Attachment #1: Type: text/plain, Size: 4091 bytes --]
On 6/6/23 00:38, Muhammad Usama Anjum wrote:
...
>> +kernel_header_files:
>> + @ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null; \
>> + if [ $$? -ne 0 ]; then \
>> + RED='\033[1;31m'; \
>> + NOCOLOR='\033[0m'; \
>> + echo; \
>> + echo -e "$${RED}error$${NOCOLOR}: missing kernel header files."; \
>> + echo "Please run this and try again:"; \
>> + echo; \
>> + echo " cd $(top_srcdir)"; \
>> + echo " make headers"; \
>> + echo; \
>> + exit 1; \
>> + fi
> Thank you for adding this. This is outputting error for every selftest
> directory. We should try to make it even better by just aborting the
> Make-ing process the first time headers aren't detected. We can do this now
> or later, fine by me.
>
OK, I see. Yes, this can be improved by adding the same mechanism to the
selftests/Makefile, that is in selftests/mm/Makefile.
I'd like to keep both, because as I mentioned earlier, mm folks like to
run just that one Makefile, sometimes, and selftests/mm/Makefile is not
invoking the top level Makefile. Rather, it includes lib.mk--which the
top level Makefile does *not* include.
Arguably, using includes instead of recursive Make, would improve this
framework: reduce duplication such as the above. But that's a larger
project and just food for thought at this point.
Anyway, this works nicely on my system, and I'll attach it as a patch
also in case you want to try it out. What do you think of this:
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 90a62cf75008..bdca160063d8 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -144,10 +144,12 @@ ifneq ($(KBUILD_OUTPUT),)
abs_objtree := $(realpath $(abs_objtree))
BUILD := $(abs_objtree)/kselftest
KHDR_INCLUDES := -isystem ${abs_objtree}/usr/include
+ KHDR_DIR := ${abs_objtree}/usr/include
else
BUILD := $(CURDIR)
abs_srctree := $(shell cd $(top_srcdir) && pwd)
KHDR_INCLUDES := -isystem ${abs_srctree}/usr/include
+ KHDR_DIR := ${abs_srctree}/usr/include
DEFAULT_INSTALL_HDR_PATH := 1
endif
@@ -161,7 +163,7 @@ export KHDR_INCLUDES
# all isn't the first target in the file.
.DEFAULT_GOAL := all
-all:
+all: kernel_header_files
@ret=1; \
for TARGET in $(TARGETS); do \
BUILD_TARGET=$$BUILD/$$TARGET; \
@@ -172,6 +174,23 @@ all:
ret=$$((ret * $$?)); \
done; exit $$ret;
+kernel_header_files:
+ @ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null; \
+ if [ $$? -ne 0 ]; then \
+ RED='\033[1;31m'; \
+ NOCOLOR='\033[0m'; \
+ echo; \
+ echo -e "$${RED}error$${NOCOLOR}: missing kernel header files."; \
+ echo "Please run this and try again:"; \
+ echo; \
+ echo " cd $(top_srcdir)"; \
+ echo " make headers"; \
+ echo; \
+ exit 1; \
+ fi
+
+.PHONY: kernel_header_files
+
run_tests: all
@for TARGET in $(TARGETS); do \
BUILD_TARGET=$$BUILD/$$TARGET; \
thanks,
--
John Hubbard
NVIDIA
[-- Attachment #2: selftests-makefile.patch --]
[-- Type: text/x-patch, Size: 2076 bytes --]
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 90a62cf75008..bdca160063d8 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -144,10 +144,12 @@ ifneq ($(KBUILD_OUTPUT),)
abs_objtree := $(realpath $(abs_objtree))
BUILD := $(abs_objtree)/kselftest
KHDR_INCLUDES := -isystem ${abs_objtree}/usr/include
+ KHDR_DIR := ${abs_objtree}/usr/include
else
BUILD := $(CURDIR)
abs_srctree := $(shell cd $(top_srcdir) && pwd)
KHDR_INCLUDES := -isystem ${abs_srctree}/usr/include
+ KHDR_DIR := ${abs_srctree}/usr/include
DEFAULT_INSTALL_HDR_PATH := 1
endif
@@ -161,7 +163,7 @@ export KHDR_INCLUDES
# all isn't the first target in the file.
.DEFAULT_GOAL := all
-all:
+all: kernel_header_files
@ret=1; \
for TARGET in $(TARGETS); do \
BUILD_TARGET=$$BUILD/$$TARGET; \
@@ -172,6 +174,23 @@ all:
ret=$$((ret * $$?)); \
done; exit $$ret;
+kernel_header_files:
+ @ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null; \
+ if [ $$? -ne 0 ]; then \
+ RED='\033[1;31m'; \
+ NOCOLOR='\033[0m'; \
+ echo; \
+ echo -e "$${RED}error$${NOCOLOR}: missing kernel header files."; \
+ echo "Please run this and try again:"; \
+ echo; \
+ echo " cd $(top_srcdir)"; \
+ echo " make headers"; \
+ echo; \
+ exit 1; \
+ fi
+
+.PHONY: kernel_header_files
+
run_tests: all
@for TARGET in $(TARGETS); do \
BUILD_TARGET=$$BUILD/$$TARGET; \
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-06-06 20:10 ` John Hubbard
@ 2023-06-07 5:37 ` Muhammad Usama Anjum
0 siblings, 0 replies; 39+ messages in thread
From: Muhammad Usama Anjum @ 2023-06-07 5:37 UTC (permalink / raw)
To: John Hubbard
Cc: Muhammad Usama Anjum, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Jonathan Corbet, linux-doc, Andrew Morton
On 6/7/23 1:10 AM, John Hubbard wrote:
> On 6/6/23 00:38, Muhammad Usama Anjum wrote:
> ...
>>> +kernel_header_files:
>>> + @ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null; \
>>> + if [ $$? -ne 0 ]; then \
>>> + RED='\033[1;31m'; \
>>> + NOCOLOR='\033[0m'; \
>>> + echo; \
>>> + echo -e "$${RED}error$${NOCOLOR}: missing kernel header files."; \
>>> + echo "Please run this and try again:"; \
>>> + echo; \
>>> + echo " cd $(top_srcdir)"; \
>>> + echo " make headers"; \
>>> + echo; \
>>> + exit 1; \
>>> + fi
>> Thank you for adding this. This is outputting error for every selftest
>> directory. We should try to make it even better by just aborting the
>> Make-ing process the first time headers aren't detected. We can do this now
>> or later, fine by me.
>>
> OK, I see. Yes, this can be improved by adding the same mechanism to the
> selftests/Makefile, that is in selftests/mm/Makefile.
>
> I'd like to keep both, because as I mentioned earlier, mm folks like to
> run just that one Makefile, sometimes, and selftests/mm/Makefile is not
> invoking the top level Makefile. Rather, it includes lib.mk--which the
> top level Makefile does *not* include.
>
> Arguably, using includes instead of recursive Make, would improve this
> framework: reduce duplication such as the above. But that's a larger
> project and just food for thought at this point.
>
> Anyway, this works nicely on my system, and I'll attach it as a patch
> also in case you want to try it out. What do you think of this:
Nice patch. Thanks. Lets add this patch as well. Please add the tag for
this new patch:
Tested-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
>
> diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
> index 90a62cf75008..bdca160063d8 100644
> --- a/tools/testing/selftests/Makefile
> +++ b/tools/testing/selftests/Makefile
> @@ -144,10 +144,12 @@ ifneq ($(KBUILD_OUTPUT),)
> abs_objtree := $(realpath $(abs_objtree))
> BUILD := $(abs_objtree)/kselftest
> KHDR_INCLUDES := -isystem ${abs_objtree}/usr/include
> + KHDR_DIR := ${abs_objtree}/usr/include
> else
> BUILD := $(CURDIR)
> abs_srctree := $(shell cd $(top_srcdir) && pwd)
> KHDR_INCLUDES := -isystem ${abs_srctree}/usr/include
> + KHDR_DIR := ${abs_srctree}/usr/include
> DEFAULT_INSTALL_HDR_PATH := 1
> endif
>
> @@ -161,7 +163,7 @@ export KHDR_INCLUDES
> # all isn't the first target in the file.
> .DEFAULT_GOAL := all
>
> -all:
> +all: kernel_header_files
> @ret=1; \
> for TARGET in $(TARGETS); do \
> BUILD_TARGET=$$BUILD/$$TARGET; \
> @@ -172,6 +174,23 @@ all:
> ret=$$((ret * $$?)); \
> done; exit $$ret;
>
> +kernel_header_files:
> + @ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null; \
> + if [ $$? -ne 0 ]; then \
> + RED='\033[1;31m'; \
> + NOCOLOR='\033[0m'; \
> + echo; \
> + echo -e "$${RED}error$${NOCOLOR}: missing kernel header files."; \
> + echo "Please run this and try again:"; \
> + echo; \
> + echo " cd $(top_srcdir)"; \
> + echo " make headers"; \
> + echo; \
> + exit 1; \
> + fi
> +
> +.PHONY: kernel_header_files
> +
> run_tests: all
> @for TARGET in $(TARGETS); do \
> BUILD_TARGET=$$BUILD/$$TARGET; \
>
>
>
> thanks,
--
BR,
Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-06-06 7:16 ` [PATCH v3 11/11] selftests: error out if kernel header files are not yet built John Hubbard
2023-06-06 7:38 ` Muhammad Usama Anjum
@ 2023-06-06 7:57 ` Muhammad Usama Anjum
2023-11-03 12:16 ` Peter Zijlstra
2023-12-08 12:44 ` Miroslav Benes
3 siblings, 0 replies; 39+ messages in thread
From: Muhammad Usama Anjum @ 2023-06-06 7:57 UTC (permalink / raw)
To: John Hubbard, Andrew Morton
Cc: Muhammad Usama Anjum, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Jonathan Corbet, linux-doc
On 6/6/23 12:16 PM, John Hubbard wrote:
> As per a discussion with Muhammad Usama Anjum [1], the following is how
> one is supposed to build selftests:
>
> make headers && make -C tools/testing/selftests/mm
>
> Change the selftest build system's lib.mk to fail out with a helpful
> message if that prerequisite "make headers" has not been done yet.
>
> [1] https://lore.kernel.org/all/bf910fa5-0c96-3707-cce4-5bcc656b6274@collabora.com/
>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: Peter Xu <peterx@redhat.com>
> Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: linux-doc@vger.kernel.org
> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
> ---
> tools/testing/selftests/lib.mk | 36 +++++++++++++++++++++++++++++++---
> 1 file changed, 33 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
> index 05400462c779..b8ea03b9a015 100644
> --- a/tools/testing/selftests/lib.mk
> +++ b/tools/testing/selftests/lib.mk
> @@ -44,10 +44,22 @@ endif
> selfdir = $(realpath $(dir $(filter %/lib.mk,$(MAKEFILE_LIST))))
> top_srcdir = $(selfdir)/../../..
>
> -ifeq ($(KHDR_INCLUDES),)
> -KHDR_INCLUDES := -isystem $(top_srcdir)/usr/include
> +ifneq ($(KBUILD_OUTPUT),)
> + # Make's built-in functions such as $(abspath ...), $(realpath ...) cannot
> + # expand a shell special character '~'. We use a somewhat tedious way here.
> + abs_objtree := $(shell cd $(top_srcdir) && mkdir -p $(KBUILD_OUTPUT) && cd $(KBUILD_OUTPUT) && pwd)
> + $(if $(abs_objtree),, \
> + $(error failed to create output directory "$(KBUILD_OUTPUT)"))
> + # $(realpath ...) resolves symlinks
> + abs_objtree := $(realpath $(abs_objtree))
> + KHDR_DIR := ${abs_objtree}/usr/include
> +else
> + abs_srctree := $(shell cd $(top_srcdir) && pwd)
> + KHDR_DIR := ${abs_srctree}/usr/include
> endif
>
> +KHDR_INCLUDES := -isystem $(KHDR_DIR)
> +
> # The following are built by lib.mk common compile rules.
> # TEST_CUSTOM_PROGS should be used by tests that require
> # custom build rule and prevent common build rule use.
> @@ -58,7 +70,25 @@ TEST_GEN_PROGS := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS))
> TEST_GEN_PROGS_EXTENDED := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS_EXTENDED))
> TEST_GEN_FILES := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_FILES))
>
> -all: $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES)
> +all: kernel_header_files $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) \
> + $(TEST_GEN_FILES)
> +
> +kernel_header_files:
> + @ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null; \
> + if [ $$? -ne 0 ]; then \
> + RED='\033[1;31m'; \
> + NOCOLOR='\033[0m'; \
> + echo; \
> + echo -e "$${RED}error$${NOCOLOR}: missing kernel header files."; \
> + echo "Please run this and try again:"; \
> + echo; \
> + echo " cd $(top_srcdir)"; \
> + echo " make headers"; \
> + echo; \
> + exit 1; \
> + fi
> +
> +.PHONY: kernel_header_files
>
> define RUN_TESTS
> BASE_DIR="$(selfdir)"; \
--
BR,
Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-06-06 7:16 ` [PATCH v3 11/11] selftests: error out if kernel header files are not yet built John Hubbard
2023-06-06 7:38 ` Muhammad Usama Anjum
2023-06-06 7:57 ` Muhammad Usama Anjum
@ 2023-11-03 12:16 ` Peter Zijlstra
2023-11-03 12:22 ` David Hildenbrand
2023-12-08 12:44 ` Miroslav Benes
3 siblings, 1 reply; 39+ messages in thread
From: Peter Zijlstra @ 2023-11-03 12:16 UTC (permalink / raw)
To: John Hubbard, Linus Torvalds
Cc: Andrew Morton, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Muhammad Usama Anjum, Jonathan Corbet, linux-doc
On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
> As per a discussion with Muhammad Usama Anjum [1], the following is how
> one is supposed to build selftests:
>
> make headers && make -C tools/testing/selftests/mm
>
> Change the selftest build system's lib.mk to fail out with a helpful
> message if that prerequisite "make headers" has not been done yet.
>
NAK NAK NAK
This now means I can no longer run selftests, I thank you very much! :-/
root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
***
*** The source tree is not clean, please run 'make mrproper'
*** in /usr/src/linux-2.6
I've always done:
cd tools/testing/selftests/x86; make
and that has always worked
Now I can't bloody well build *any* selftest or risk not being able to
do builds.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-11-03 12:16 ` Peter Zijlstra
@ 2023-11-03 12:22 ` David Hildenbrand
2023-11-03 12:46 ` Peter Zijlstra
2023-12-08 15:14 ` Peter Zijlstra
0 siblings, 2 replies; 39+ messages in thread
From: David Hildenbrand @ 2023-11-03 12:22 UTC (permalink / raw)
To: Peter Zijlstra, John Hubbard, Linus Torvalds
Cc: Andrew Morton, Peter Xu, Shuah Khan, Nathan Chancellor, linux-mm,
linux-kselftest, LKML, Muhammad Usama Anjum, Jonathan Corbet,
linux-doc
On 03.11.23 13:16, Peter Zijlstra wrote:
> On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
>> As per a discussion with Muhammad Usama Anjum [1], the following is how
>> one is supposed to build selftests:
>>
>> make headers && make -C tools/testing/selftests/mm
>>
>> Change the selftest build system's lib.mk to fail out with a helpful
>> message if that prerequisite "make headers" has not been done yet.
>>
>
> NAK NAK NAK
>
> This now means I can no longer run selftests, I thank you very much! :-/
>
> root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
> make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
> ***
> *** The source tree is not clean, please run 'make mrproper'
> *** in /usr/src/linux-2.6
>
>
> I've always done:
>
> cd tools/testing/selftests/x86; make
>
> and that has always worked
>
> Now I can't bloody well build *any* selftest or risk not being able to
> do builds.
This change landed in 6.5, no? And 6.6 was just released. Just curious
why you notice that now.
--
Cheers,
David / dhildenb
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-11-03 12:22 ` David Hildenbrand
@ 2023-11-03 12:46 ` Peter Zijlstra
2023-11-03 12:59 ` David Hildenbrand
2023-12-08 15:14 ` Peter Zijlstra
1 sibling, 1 reply; 39+ messages in thread
From: Peter Zijlstra @ 2023-11-03 12:46 UTC (permalink / raw)
To: David Hildenbrand
Cc: John Hubbard, Linus Torvalds, Andrew Morton, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Muhammad Usama Anjum, Jonathan Corbet, linux-doc
On Fri, Nov 03, 2023 at 01:22:54PM +0100, David Hildenbrand wrote:
> On 03.11.23 13:16, Peter Zijlstra wrote:
> > On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
> > > As per a discussion with Muhammad Usama Anjum [1], the following is how
> > > one is supposed to build selftests:
> > >
> > > make headers && make -C tools/testing/selftests/mm
> > >
> > > Change the selftest build system's lib.mk to fail out with a helpful
> > > message if that prerequisite "make headers" has not been done yet.
> > >
> >
> > NAK NAK NAK
> >
> > This now means I can no longer run selftests, I thank you very much! :-/
> >
> > root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
> > make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
> > ***
> > *** The source tree is not clean, please run 'make mrproper'
> > *** in /usr/src/linux-2.6
> >
> >
> > I've always done:
> >
> > cd tools/testing/selftests/x86; make
> >
> > and that has always worked
> >
> > Now I can't bloody well build *any* selftest or risk not being able to
> > do builds.
>
> This change landed in 6.5, no? And 6.6 was just released. Just curious why
> you notice that now.
Dunno, last time I edited the selftests and needed to recompile was a
few weeks ago.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-11-03 12:46 ` Peter Zijlstra
@ 2023-11-03 12:59 ` David Hildenbrand
2023-11-03 13:00 ` David Hildenbrand
2023-11-03 13:08 ` Peter Zijlstra
0 siblings, 2 replies; 39+ messages in thread
From: David Hildenbrand @ 2023-11-03 12:59 UTC (permalink / raw)
To: Peter Zijlstra
Cc: John Hubbard, Linus Torvalds, Andrew Morton, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Muhammad Usama Anjum, Jonathan Corbet, linux-doc
On 03.11.23 13:46, Peter Zijlstra wrote:
> On Fri, Nov 03, 2023 at 01:22:54PM +0100, David Hildenbrand wrote:
>> On 03.11.23 13:16, Peter Zijlstra wrote:
>>> On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
>>>> As per a discussion with Muhammad Usama Anjum [1], the following is how
>>>> one is supposed to build selftests:
>>>>
>>>> make headers && make -C tools/testing/selftests/mm
>>>>
>>>> Change the selftest build system's lib.mk to fail out with a helpful
>>>> message if that prerequisite "make headers" has not been done yet.
>>>>
>>>
>>> NAK NAK NAK
>>>
>>> This now means I can no longer run selftests, I thank you very much! :-/
>>>
>>> root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
>>> make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
>>> ***
>>> *** The source tree is not clean, please run 'make mrproper'
>>> *** in /usr/src/linux-2.6
>>>
>>>
>>> I've always done:
>>>
>>> cd tools/testing/selftests/x86; make
>>>
>>> and that has always worked
>>>
>>> Now I can't bloody well build *any* selftest or risk not being able to
>>> do builds.
>>
>> This change landed in 6.5, no? And 6.6 was just released. Just curious why
>> you notice that now.
>
> Dunno, last time I edited the selftests and needed to recompile was a
> few weeks ago.
Okay. the question is if your workflow can be easily adjusted, or if we
can improve that header handling as a whole.
The problem I had with this recently: just because we did a "make
headers" once in a git tree doesn't mean that it is still up-to-date.
So once some selftest changes showed up that require newer headers,
building the selftests again fails without a hint that another round of
"make headers" would be required.
--
Cheers,
David / dhildenb
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-11-03 12:59 ` David Hildenbrand
@ 2023-11-03 13:00 ` David Hildenbrand
2023-11-03 13:08 ` Peter Zijlstra
1 sibling, 0 replies; 39+ messages in thread
From: David Hildenbrand @ 2023-11-03 13:00 UTC (permalink / raw)
To: Peter Zijlstra
Cc: John Hubbard, Linus Torvalds, Andrew Morton, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Muhammad Usama Anjum, Jonathan Corbet, linux-doc
On 03.11.23 13:59, David Hildenbrand wrote:
> On 03.11.23 13:46, Peter Zijlstra wrote:
>> On Fri, Nov 03, 2023 at 01:22:54PM +0100, David Hildenbrand wrote:
>>> On 03.11.23 13:16, Peter Zijlstra wrote:
>>>> On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
>>>>> As per a discussion with Muhammad Usama Anjum [1], the following is how
>>>>> one is supposed to build selftests:
>>>>>
>>>>> make headers && make -C tools/testing/selftests/mm
>>>>>
>>>>> Change the selftest build system's lib.mk to fail out with a helpful
>>>>> message if that prerequisite "make headers" has not been done yet.
>>>>>
>>>>
>>>> NAK NAK NAK
>>>>
>>>> This now means I can no longer run selftests, I thank you very much! :-/
>>>>
>>>> root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
>>>> make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
>>>> ***
>>>> *** The source tree is not clean, please run 'make mrproper'
>>>> *** in /usr/src/linux-2.6
>>>>
>>>>
>>>> I've always done:
>>>>
>>>> cd tools/testing/selftests/x86; make
>>>>
>>>> and that has always worked
>>>>
>>>> Now I can't bloody well build *any* selftest or risk not being able to
>>>> do builds.
>>>
>>> This change landed in 6.5, no? And 6.6 was just released. Just curious why
>>> you notice that now.
>>
>> Dunno, last time I edited the selftests and needed to recompile was a
>> few weeks ago.
>
> Okay. the question is if your workflow can be easily adjusted, or if we
> can improve that header handling as a whole.
>
> The problem I had with this recently: just because we did a "make
> headers" once in a git tree doesn't mean that it is still up-to-date.
>
> So once some selftest changes showed up that require newer headers,
> building the selftests again fails without a hint that another round of
> "make headers" would be required.
To clarify: maybe some kind of a warning would be better, ideally that
the headers might be outdated and that another "make headers" would be
required in case there are any build errors.
--
Cheers,
David / dhildenb
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-11-03 12:59 ` David Hildenbrand
2023-11-03 13:00 ` David Hildenbrand
@ 2023-11-03 13:08 ` Peter Zijlstra
1 sibling, 0 replies; 39+ messages in thread
From: Peter Zijlstra @ 2023-11-03 13:08 UTC (permalink / raw)
To: David Hildenbrand
Cc: John Hubbard, Linus Torvalds, Andrew Morton, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Muhammad Usama Anjum, Jonathan Corbet, linux-doc
On Fri, Nov 03, 2023 at 01:59:28PM +0100, David Hildenbrand wrote:
> Okay. the question is if your workflow can be easily adjusted, or if we can
> improve that header handling as a whole.
So on IRC the following was suggested:
make O=defconfig-build headers ; make O=defconfig-build -C tools/testing/selftests/x86
But that makes absolutely no sense to me; because the headers and
selftests are not .config dependent. Furthermore I don't want them in a
kernel build dir.
> The problem I had with this recently: just because we did a "make headers"
> once in a git tree doesn't mean that it is still up-to-date.
>
> So once some selftest changes showed up that require newer headers, building
> the selftests again fails without a hint that another round of "make
> headers" would be required.
Yeah, so I've been adding #ifndef guards all over the place for decades
and that just works. You need it in normal userspace too.
This super reliance on the very latestesetst headers is just a total
pain.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-11-03 12:22 ` David Hildenbrand
2023-11-03 12:46 ` Peter Zijlstra
@ 2023-12-08 15:14 ` Peter Zijlstra
2023-12-08 15:21 ` David Hildenbrand
1 sibling, 1 reply; 39+ messages in thread
From: Peter Zijlstra @ 2023-12-08 15:14 UTC (permalink / raw)
To: David Hildenbrand
Cc: John Hubbard, Linus Torvalds, Andrew Morton, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Muhammad Usama Anjum, Jonathan Corbet, linux-doc
On Fri, Nov 03, 2023 at 01:22:54PM +0100, David Hildenbrand wrote:
> On 03.11.23 13:16, Peter Zijlstra wrote:
> > On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
> > > As per a discussion with Muhammad Usama Anjum [1], the following is how
> > > one is supposed to build selftests:
> > >
> > > make headers && make -C tools/testing/selftests/mm
> > >
> > > Change the selftest build system's lib.mk to fail out with a helpful
> > > message if that prerequisite "make headers" has not been done yet.
> > >
> >
> > NAK NAK NAK
> >
> > This now means I can no longer run selftests, I thank you very much! :-/
> >
> > root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
> > make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
> > ***
> > *** The source tree is not clean, please run 'make mrproper'
> > *** in /usr/src/linux-2.6
> >
> >
> > I've always done:
> >
> > cd tools/testing/selftests/x86; make
> >
> > and that has always worked
> >
> > Now I can't bloody well build *any* selftest or risk not being able to
> > do builds.
>
> This change landed in 6.5, no? And 6.6 was just released. Just curious why
> you notice that now.
And I hit it again (different box etc..)
Can we please get this garbage fixed already?
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-12-08 15:14 ` Peter Zijlstra
@ 2023-12-08 15:21 ` David Hildenbrand
2023-12-08 20:29 ` John Hubbard
0 siblings, 1 reply; 39+ messages in thread
From: David Hildenbrand @ 2023-12-08 15:21 UTC (permalink / raw)
To: Peter Zijlstra
Cc: John Hubbard, Linus Torvalds, Andrew Morton, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Muhammad Usama Anjum, Jonathan Corbet, linux-doc
On 08.12.23 16:14, Peter Zijlstra wrote:
> On Fri, Nov 03, 2023 at 01:22:54PM +0100, David Hildenbrand wrote:
>> On 03.11.23 13:16, Peter Zijlstra wrote:
>>> On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
>>>> As per a discussion with Muhammad Usama Anjum [1], the following is how
>>>> one is supposed to build selftests:
>>>>
>>>> make headers && make -C tools/testing/selftests/mm
>>>>
>>>> Change the selftest build system's lib.mk to fail out with a helpful
>>>> message if that prerequisite "make headers" has not been done yet.
>>>>
>>>
>>> NAK NAK NAK
>>>
>>> This now means I can no longer run selftests, I thank you very much! :-/
>>>
>>> root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
>>> make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
>>> ***
>>> *** The source tree is not clean, please run 'make mrproper'
>>> *** in /usr/src/linux-2.6
>>>
>>>
>>> I've always done:
>>>
>>> cd tools/testing/selftests/x86; make
>>>
>>> and that has always worked
>>>
>>> Now I can't bloody well build *any* selftest or risk not being able to
>>> do builds.
>>
>> This change landed in 6.5, no? And 6.6 was just released. Just curious why
>> you notice that now.
>
> And I hit it again (different box etc..)
>
> Can we please get this garbage fixed already?
I'd suggest to either revert or turn into a warning.
@John?
--
Cheers,
David / dhildenb
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-12-08 15:21 ` David Hildenbrand
@ 2023-12-08 20:29 ` John Hubbard
2023-12-08 22:10 ` Peter Zijlstra
0 siblings, 1 reply; 39+ messages in thread
From: John Hubbard @ 2023-12-08 20:29 UTC (permalink / raw)
To: David Hildenbrand, Peter Zijlstra
Cc: Linus Torvalds, Andrew Morton, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Muhammad Usama Anjum, Jonathan Corbet, linux-doc
On 12/8/23 07:21, David Hildenbrand wrote:
> On 08.12.23 16:14, Peter Zijlstra wrote:
>> On Fri, Nov 03, 2023 at 01:22:54PM +0100, David Hildenbrand wrote:
>>> On 03.11.23 13:16, Peter Zijlstra wrote:
>>>> On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
>>>>> As per a discussion with Muhammad Usama Anjum [1], the following is
>>>>> how
>>>>> one is supposed to build selftests:
>>>>>
>>>>> make headers && make -C tools/testing/selftests/mm
>>>>>
>>>>> Change the selftest build system's lib.mk to fail out with a helpful
>>>>> message if that prerequisite "make headers" has not been done yet.
>>>>>
>>>>
>>>> NAK NAK NAK
>>>>
>>>> This now means I can no longer run selftests, I thank you very much!
>>>> :-/
>>>>
>>>> root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
>>>> make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
>>>> ***
>>>> *** The source tree is not clean, please run 'make mrproper'
>>>> *** in /usr/src/linux-2.6
>>>>
>>>>
>>>> I've always done:
>>>>
>>>> cd tools/testing/selftests/x86; make
>>>>
>>>> and that has always worked
>>>>
>>>> Now I can't bloody well build *any* selftest or risk not being able to
>>>> do builds.
>>>
>>> This change landed in 6.5, no? And 6.6 was just released. Just
>>> curious why
>>> you notice that now.
>>
>> And I hit it again (different box etc..)
>>
>> Can we please get this garbage fixed already?
>
> I'd suggest to either revert or turn into a warning.
That would put us back into a half-broken sort of situation, though...
see below.
>
> @John?
>
I don't have a strong opinion about how this should be done, and in
fact I believed at the time that I was bringing the system into
compliance with what everyone wanted here. :)
There seem to be two conflicting visions:
a) The way it was (much) earlier: use ifdefs and defines to get by
without the latest kernel headers, or
b) Requiring recent kernel headers to build the various selftests.
Shuah, Peter, others: can we choose a direction please? Either
way will work, and I personally don't care which one we choose.
thanks,
--
John Hubbard
NVIDIA
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-12-08 20:29 ` John Hubbard
@ 2023-12-08 22:10 ` Peter Zijlstra
2023-12-09 1:39 ` John Hubbard
0 siblings, 1 reply; 39+ messages in thread
From: Peter Zijlstra @ 2023-12-08 22:10 UTC (permalink / raw)
To: John Hubbard
Cc: David Hildenbrand, Linus Torvalds, Andrew Morton, Peter Xu,
Shuah Khan, Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Muhammad Usama Anjum, Jonathan Corbet, linux-doc
On Fri, Dec 08, 2023 at 12:29:37PM -0800, John Hubbard wrote:
> I don't have a strong opinion about how this should be done, and in
> fact I believed at the time that I was bringing the system into
> compliance with what everyone wanted here. :)
>
> There seem to be two conflicting visions:
>
> a) The way it was (much) earlier: use ifdefs and defines to get by
> without the latest kernel headers, or
>
> b) Requiring recent kernel headers to build the various selftests.
>
> Shuah, Peter, others: can we choose a direction please? Either
> way will work, and I personally don't care which one we choose.
So as David already argued, the current thing does not in fact help with
b. You just have to install once and the error goes away, then carry
that tree for a year and you're running old crap again.
My biggest beef with the whole thing is that I simply do not want to use
'make headers', it doesn't work for me.
I have a ton of output directories and I don't care to build tools into
the output dirs, in fact some of them flat out refuse to work that way
(bpf comes to mind).
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-12-08 22:10 ` Peter Zijlstra
@ 2023-12-09 1:39 ` John Hubbard
0 siblings, 0 replies; 39+ messages in thread
From: John Hubbard @ 2023-12-09 1:39 UTC (permalink / raw)
To: Peter Zijlstra
Cc: David Hildenbrand, Linus Torvalds, Andrew Morton, Peter Xu,
Shuah Khan, Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Muhammad Usama Anjum, Jonathan Corbet, linux-doc
On 12/8/23 14:10, Peter Zijlstra wrote:
> So as David already argued, the current thing does not in fact help with
> b. You just have to install once and the error goes away, then carry
> that tree for a year and you're running old crap again.
>
> My biggest beef with the whole thing is that I simply do not want to use
> 'make headers', it doesn't work for me.
>
> I have a ton of output directories and I don't care to build tools into
> the output dirs, in fact some of them flat out refuse to work that way
> (bpf comes to mind).
Going with that, then, I believe it is best to simply revert commit
9fc96c7c19df ("selftests: error out if kernel header files are not
yet built"). And then follow up with a series of (many) changes to
wean the various selftests off of the kernel headers.
I'll post the revert shortly.
thanks,
--
John Hubbard
NVIDIA
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built
2023-06-06 7:16 ` [PATCH v3 11/11] selftests: error out if kernel header files are not yet built John Hubbard
` (2 preceding siblings ...)
2023-11-03 12:16 ` Peter Zijlstra
@ 2023-12-08 12:44 ` Miroslav Benes
3 siblings, 0 replies; 39+ messages in thread
From: Miroslav Benes @ 2023-12-08 12:44 UTC (permalink / raw)
To: John Hubbard
Cc: Andrew Morton, David Hildenbrand, Peter Xu, Shuah Khan,
Nathan Chancellor, linux-mm, linux-kselftest, LKML,
Muhammad Usama Anjum, Jonathan Corbet, linux-doc, live-patching
Hi John, Muhammad,
On Tue, 6 Jun 2023, John Hubbard wrote:
> As per a discussion with Muhammad Usama Anjum [1], the following is how
> one is supposed to build selftests:
>
> make headers && make -C tools/testing/selftests/mm
>
> Change the selftest build system's lib.mk to fail out with a helpful
> message if that prerequisite "make headers" has not been done yet.
>
> [1] https://lore.kernel.org/all/bf910fa5-0c96-3707-cce4-5bcc656b6274@collabora.com/
could you, please, elaborate more on that one is supposed to build
selftests with 'make headers'? Yes, Documentation/dev-tools/kselftest.rst
mentions that because you might need headers but...
The common way how we test the kernel is to build the kernel, install it
somewhere and run selftests on top. The sequence basically being "make
rpm-pkg; rpm -ivh; cd tools/testing/selftest/livepatch/ in source tree;
sudo make run_tests" (or a similar variation of the procedure). The point
is that we want to test the running kernel with its respective environment
installed in /lib/modules/`uname -r`/ (if needed). This way we can run
newer selftests from the current mainline tree on older kernels among
others.
The commit breaks the use case which worked for a long long time.
It also breaks what Marcos proposed for livepatch selftests in
https://lore.kernel.org/all/20231031-send-lp-kselftests-v3-0-2b1655c2605f@suse.com/
I guess we can always work around it by letting subsystem selftests to
override KHDR_DIR but I am not comfortable with the behaviour that your
commit introduced in the first place to be honest.
Thank you,
Miroslav
^ permalink raw reply [flat|nested] 39+ messages in thread