tools/testing/selftests

tools/testing/selftests

[Andrew Morton]

whinges, perrmanits, maybe I'm doing it wrong, etc.

a)

At the top-level, neither `make mrproper' nor `make clean' remove the
executables from tools/testing/selftests/mm/.  Seems wrong?

b)

Shouldn't I be able to run

	make selftests

from the top level?

c)

tools/testing/selftests/mm
make clean
make

does:

  CC       droppable
  CC       guard-regions
  CC       merge
  CC       rmap
  CC       soft-dirty
gcc -Wall -O2 -I /usr/src/25/tools/testing/selftests/../../..  -isystem /usr/src/25/tools/testing/selftests/../../../usr/include -isystem /usr/src/25/tools/testing/selftests/../../../tools/include/uapi -Wunreachable-code -U_FORTIFY_SOURCE -no-pie -D_GNU_SOURCE= -I/usr/src/25/tools/testing/selftests/../../../tools/testing/selftests  -m32 -mxsave  protection_keys.c vm_util.c thp_settings.c pkey_util.c -lrt -lpthread -lm -lrt -ldl -lm -o /usr/src/25/tools/testing/selftests/mm/protection_keys_32
gcc -Wall -O2 -I /usr/src/25/tools/testing/selftests/../../..  -isystem /usr/src/25/tools/testing/selftests/../../../usr/include -isystem /usr/src/25/tools/testing/selftests/../../../tools/include/uapi -Wunreachable-code -U_FORTIFY_SOURCE -no-pie -D_GNU_SOURCE= -I/usr/src/25/tools/testing/selftests/../../../tools/testing/selftests  -m32 -mxsave  pkey_sighandler_tests.c vm_util.c thp_settings.c pkey_util.c -lrt -lpthread -lm -lrt -ldl -lm -o /usr/src/25/tools/testing/selftests/mm/pkey_sighandler_tests_32

why did it start emitting the gcc command lines?

d) within tools/testing/selftests/mm:

	make clean
	make -j100

   compiles 3-4 things then ends.

   A subsequent `make -j1' compiles nothing.

   Whereas

	make clean
	make -j1

   compiles everything.  Slowly.

Thanks.   

Re: tools/testing/selftests

[Li Wang]

On Fri, Mar 27, 2026 at 02:32:34PM -0700, Andrew Morton wrote:
> 
> whinges, perrmanits, maybe I'm doing it wrong, etc.
> 
> a)
> 
> At the top-level, neither `make mrproper' nor `make clean' remove the
> executables from tools/testing/selftests/mm/.  Seems wrong?
> 
> b)
> 
> Shouldn't I be able to run
> 
> 	make selftests
> 
> from the top level?

Should use:

    make kselftest
    make kselftest-clean

> 
> c)
> 
> tools/testing/selftests/mm
> make clean
> make
> 
> does:
> 
>   CC       droppable
>   CC       guard-regions
>   CC       merge
>   CC       rmap
>   CC       soft-dirty
> gcc -Wall -O2 -I /usr/src/25/tools/testing/selftests/../../..  -isystem /usr/src/25/tools/testing/selftests/../../../usr/include -isystem /usr/src/25/tools/testing/selftests/../../../tools/include/uapi -Wunreachable-code -U_FORTIFY_SOURCE -no-pie -D_GNU_SOURCE= -I/usr/src/25/tools/testing/selftests/../../../tools/testing/selftests  -m32 -mxsave  protection_keys.c vm_util.c thp_settings.c pkey_util.c -lrt -lpthread -lm -lrt -ldl -lm -o /usr/src/25/tools/testing/selftests/mm/protection_keys_32
> gcc -Wall -O2 -I /usr/src/25/tools/testing/selftests/../../..  -isystem /usr/src/25/tools/testing/selftests/../../../usr/include -isystem /usr/src/25/tools/testing/selftests/../../../tools/include/uapi -Wunreachable-code -U_FORTIFY_SOURCE -no-pie -D_GNU_SOURCE= -I/usr/src/25/tools/testing/selftests/../../../tools/testing/selftests  -m32 -mxsave  pkey_sighandler_tests.c vm_util.c thp_settings.c pkey_util.c -lrt -lpthread -lm -lrt -ldl -lm -o /usr/src/25/tools/testing/selftests/mm/pkey_sighandler_tests_32
> 
> why did it start emitting the gcc command lines?

Those two binaries (*_32 / *_64) are built by custom rules in mm/Makefile.
(Makefile: line #218, #226).

Other tests use shared rules from ../lib.mk, which are quieter (they hide
full commands and show short CC ... lines).

Maybe we need:

--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -215,7 +215,8 @@ ifeq ($(CAN_BUILD_I386),1)
 $(BINARIES_32): CFLAGS += -m32 -mxsave
 $(BINARIES_32): LDLIBS += -lrt -ldl -lm
 $(BINARIES_32): $(OUTPUT)/%_32: %.c
-       $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@
+       $(call msg,CC,,$@)
+       $(Q)$(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@
 $(foreach t,$(VMTARGETS),$(eval $(call gen-target-rule-32,$(t))))
 endif

@@ -223,7 +224,8 @@ ifeq ($(CAN_BUILD_X86_64),1)
 $(BINARIES_64): CFLAGS += -m64 -mxsave
 $(BINARIES_64): LDLIBS += -lrt -ldl
 $(BINARIES_64): $(OUTPUT)/%_64: %.c
-       $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@
+       $(call msg,CC,,$@)
+       $(Q)$(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@
 $(foreach t,$(VMTARGETS),$(eval $(call gen-target-rule-64,$(t))))
 endif

> d) within tools/testing/selftests/mm:
> 
> 	make clean
> 	make -j100
> 
>    compiles 3-4 things then ends.
> 
>    A subsequent `make -j1' compiles nothing.

Sorry, I wan't able to reproduce it.
Did you mean -j100 only build few source file but not the whole?

>    Whereas
> 
> 	make clean
> 	make -j1
> 
>    compiles everything.  Slowly.

Both -j100 and -j1 works well from my test (vm: rhel-9.7, x86_64, gcc-11.5.0)

-- 
Regards,
Li Wang

Re: tools/testing/selftests

[Andrew Morton]

On Sat, 28 Mar 2026 10:12:06 +0800 Li Wang <liwang@redhat.com> wrote:

> On Fri, Mar 27, 2026 at 02:32:34PM -0700, Andrew Morton wrote:
> > 
> > whinges, perrmanits, maybe I'm doing it wrong, etc.
> > 
> > a)
> > 
> > At the top-level, neither `make mrproper' nor `make clean' remove the
> > executables from tools/testing/selftests/mm/.  Seems wrong?
> > 
> > b)
> > 
> > Shouldn't I be able to run
> > 
> > 	make selftests
> > 
> > from the top level?
> 
> Should use:
> 
>     make kselftest
>     make kselftest-clean

Thanks.

> Those two binaries (*_32 / *_64) are built by custom rules in mm/Makefile.
> (Makefile: line #218, #226).
> 
> Other tests use shared rules from ../lib.mk, which are quieter (they hide
> full commands and show short CC ... lines).
> 
> Maybe we need:
> 
> --- a/tools/testing/selftests/mm/Makefile
> +++ b/tools/testing/selftests/mm/Makefile
> @@ -215,7 +215,8 @@ ifeq ($(CAN_BUILD_I386),1)
>  $(BINARIES_32): CFLAGS += -m32 -mxsave
>  $(BINARIES_32): LDLIBS += -lrt -ldl -lm
>  $(BINARIES_32): $(OUTPUT)/%_32: %.c
> -       $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@
> +       $(call msg,CC,,$@)
> +       $(Q)$(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@
>  $(foreach t,$(VMTARGETS),$(eval $(call gen-target-rule-32,$(t))))
>  endif
> 
> @@ -223,7 +224,8 @@ ifeq ($(CAN_BUILD_X86_64),1)
>  $(BINARIES_64): CFLAGS += -m64 -mxsave
>  $(BINARIES_64): LDLIBS += -lrt -ldl
>  $(BINARIES_64): $(OUTPUT)/%_64: %.c
> -       $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@
> +       $(call msg,CC,,$@)
> +       $(Q)$(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@
>  $(foreach t,$(VMTARGETS),$(eval $(call gen-target-rule-64,$(t))))
>  endif

OK, minor thing of course.

> > d) within tools/testing/selftests/mm:
> > 
> > 	make clean
> > 	make -j100
> > 
> >    compiles 3-4 things then ends.
> > 
> >    A subsequent `make -j1' compiles nothing.
> 
> Sorry, I wan't able to reproduce it.
> Did you mean -j100 only build few source file but not the whole?

Yes.

On my 128 core machine everything up to -j50 works.  -j51 and higher do
this.

Another (32 core) machine runs -j100 successfully.

Weird.

Re: tools/testing/selftests

[Li Wang]

> > > d) within tools/testing/selftests/mm:
> > > 
> > > 	make clean
> > > 	make -j100
> > > 
> > >    compiles 3-4 things then ends.
> > > 
> > >    A subsequent `make -j1' compiles nothing.
> > 
> > Sorry, I wan't able to reproduce it.
> > Did you mean -j100 only build few source file but not the whole?
> 
> Yes.
> 
> On my 128 core machine everything up to -j50 works.  -j51 and higher do
> this.

Could it be caused by line#262 of selftests/mm/Makefile:

  local_config.mk local_config.h: check_config.sh
  	CC="$(CC)" CFLAGS="$(CFLAGS)" ./check_config.sh

check_config.sh generates two files: local_config.mk and local_config.h.
Makefile lets parallel make -j100 hit a timing race around that shared
generation step. local_config.mk is also included, so if it’s being
regenerated at the wrong moment, make may parse incomplete state and
build only part of targets.

Try this patch on your 128 core system:

--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -259,10 +259,13 @@ $(OUTPUT)/migration: LDLIBS += -lnuma

 $(OUTPUT)/rmap: LDLIBS += -lnuma

-local_config.mk local_config.h: check_config.sh
+local_config.stamp: check_config.sh
        CC="$(CC)" CFLAGS="$(CFLAGS)" ./check_config.sh

-EXTRA_CLEAN += local_config.mk local_config.h
+local_config.mk local_config.h: local_config.stamp
+
+EXTRA_CLEAN += local_config.mk local_config.h local_config.stamp

 ifeq ($(IOURING_EXTRA_LIBS),)
 all: warn_missing_liburing


-- 
Regards,
Li Wang

Re: tools/testing/selftests

[Andrew Morton]

On Sat, 28 Mar 2026 13:56:50 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:

> On Sat, 28 Mar 2026 10:12:06 +0800 Li Wang <liwang@redhat.com> wrote:
> 
> > > from the top level?
> > 
> > Should use:
> > 
> >     make kselftest
> 

I'm probably doing something wrong, but `make -j50 kselftest-all'
appears to have scribbled on my top-level Makefile, so now I'm getting

ts:/usr/src/25> make kselftest-all 
/usr/src/25/Makefile:5: *** Too many open files.  Stop.

ts:/usr/src/25> cat Makefile
# Automatically generated by /usr/src/25/Makefile: don't edit
export KBUILD_OUTPUT = .
export KBUILD_EXTMOD = /usr/src/25
export KBUILD_EXTMOD_OUTPUT = /usr/src/25
include /usr/src/25/Makefile

I've done this twice in succession now.  Any suggestions where I should
look?

Re: tools/testing/selftests

[Li Wang]

On Sat, Mar 28, 2026 at 02:03:11PM -0700, Andrew Morton wrote:
> On Sat, 28 Mar 2026 13:56:50 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> 
> > On Sat, 28 Mar 2026 10:12:06 +0800 Li Wang <liwang@redhat.com> wrote:
> > 
> > > > from the top level?
> > > 
> > > Should use:
> > > 
> > >     make kselftest
> > 
> 
> I'm probably doing something wrong, but `make -j50 kselftest-all'
> appears to have scribbled on my top-level Makefile, so now I'm getting
> 
> ts:/usr/src/25> make kselftest-all 
> /usr/src/25/Makefile:5: *** Too many open files.  Stop.
> 
> ts:/usr/src/25> cat Makefile
> # Automatically generated by /usr/src/25/Makefile: don't edit
> export KBUILD_OUTPUT = .
> export KBUILD_EXTMOD = /usr/src/25
> export KBUILD_EXTMOD_OUTPUT = /usr/src/25
> include /usr/src/25/Makefile
> 
> I've done this twice in succession now.  Any suggestions where I should
> look?

When build selftests from the top-level, sub-makes entered via `-C` may
still inherit the caller's PWD from the environment.

Some selftests use $(PWD) in recursive kbuild invocations, which can
then incorrectly resolve to the kernel top directory instead of the
current test directory.

Maybe try export PWD in the ../selftests/lib.mk?

  export PWD := $(CURDIR)

-- 
Regards,
Li Wang

Re: tools/testing/selftests

[Andrew Morton]

On Sun, 29 Mar 2026 18:04:52 +0800 Li Wang <liwang@redhat.com> wrote:

> > ts:/usr/src/25> cat Makefile
> > # Automatically generated by /usr/src/25/Makefile: don't edit
> > export KBUILD_OUTPUT = .
> > export KBUILD_EXTMOD = /usr/src/25
> > export KBUILD_EXTMOD_OUTPUT = /usr/src/25
> > include /usr/src/25/Makefile
> > 
> > I've done this twice in succession now.  Any suggestions where I should
> > look?
> 
> When build selftests from the top-level, sub-makes entered via `-C` may
> still inherit the caller's PWD from the environment.
> 
> Some selftests use $(PWD) in recursive kbuild invocations, which can
> then incorrectly resolve to the kernel top directory instead of the
> current test directory.
> 
> Maybe try export PWD in the ../selftests/lib.mk?
> 
>   export PWD := $(CURDIR)

Thanks, I'll get onto this stuff soon.

Meanwhile, shouldn't this work?

ts:/usr/src/25/tools/testing/selftests/mm> make merge.i
make: *** No rule to make target 'merge.i'.  Stop.

Re: tools/testing/selftests

[Li Wang]

On Mon, Mar 30, 2026 at 05:09:54PM -0700, Andrew Morton wrote:
> On Sun, 29 Mar 2026 18:04:52 +0800 Li Wang <liwang@redhat.com> wrote:
> 
> > > ts:/usr/src/25> cat Makefile
> > > # Automatically generated by /usr/src/25/Makefile: don't edit
> > > export KBUILD_OUTPUT = .
> > > export KBUILD_EXTMOD = /usr/src/25
> > > export KBUILD_EXTMOD_OUTPUT = /usr/src/25
> > > include /usr/src/25/Makefile
> > > 
> > > I've done this twice in succession now.  Any suggestions where I should
> > > look?
> > 
> > When build selftests from the top-level, sub-makes entered via `-C` may
> > still inherit the caller's PWD from the environment.
> > 
> > Some selftests use $(PWD) in recursive kbuild invocations, which can
> > then incorrectly resolve to the kernel top directory instead of the
> > current test directory.
> > 
> > Maybe try export PWD in the ../selftests/lib.mk?
> > 
> >   export PWD := $(CURDIR)
> 
> Thanks, I'll get onto this stuff soon.
 
I have created a patchset for those build tiny issues,
let me send out to you to have a early try?

> Meanwhile, shouldn't this work?
> 
> ts:/usr/src/25/tools/testing/selftests/mm> make merge.i
> make: *** No rule to make target 'merge.i'.  Stop.

That's probably because Makefile explicitly disables the implicit .c to .i rule.

In line#43 of Makefile:

    MAKEFLAGS += --no-builtin-rules

I doubt do we really need to add back the 'gcc -E' approach? 
The selftests build system intentionally kills built-in rules to prevent
half-broken implicit builds.
(the comment line# 33~42 in the Makefile explains why)

-- 
Regards,
Li Wang

Re: tools/testing/selftests

[Andrew Morton]

On Tue, 31 Mar 2026 09:52:35 +0800 Li Wang <liwang@redhat.com> wrote:

> On Mon, Mar 30, 2026 at 05:09:54PM -0700, Andrew Morton wrote:
> > On Sun, 29 Mar 2026 18:04:52 +0800 Li Wang <liwang@redhat.com> wrote:
> > 
> > > > ts:/usr/src/25> cat Makefile
> > > > # Automatically generated by /usr/src/25/Makefile: don't edit
> > > > export KBUILD_OUTPUT = .
> > > > export KBUILD_EXTMOD = /usr/src/25
> > > > export KBUILD_EXTMOD_OUTPUT = /usr/src/25
> > > > include /usr/src/25/Makefile
> > > > 
> > > > I've done this twice in succession now.  Any suggestions where I should
> > > > look?
> > > 
> > > When build selftests from the top-level, sub-makes entered via `-C` may
> > > still inherit the caller's PWD from the environment.
> > > 
> > > Some selftests use $(PWD) in recursive kbuild invocations, which can
> > > then incorrectly resolve to the kernel top directory instead of the
> > > current test directory.
> > > 
> > > Maybe try export PWD in the ../selftests/lib.mk?
> > > 
> > >   export PWD := $(CURDIR)
> > 
> > Thanks, I'll get onto this stuff soon.
>  
> I have created a patchset for those build tiny issues,
> let me send out to you to have a early try?

Sure.

> > Meanwhile, shouldn't this work?
> > 
> > ts:/usr/src/25/tools/testing/selftests/mm> make merge.i
> > make: *** No rule to make target 'merge.i'.  Stop.
> 
> That's probably because Makefile explicitly disables the implicit .c to .i rule.
> 
> In line#43 of Makefile:
> 
>     MAKEFLAGS += --no-builtin-rules
> 
> I doubt do we really need to add back the 'gcc -E' approach? 

I occasionally use it for figuring out inclusion messes.

`make -V' followed by copy-paste-edit still works ;)

> The selftests build system intentionally kills built-in rules to prevent
> half-broken implicit builds.
> (the comment line# 33~42 in the Makefile explains why)

OK, wow, looks like that solution was painfully arrived at.

Re: tools/testing/selftests

[Li Wang]

> > I have created a patchset for those build tiny issues,
> > let me send out to you to have a early try?
> 
> Sure.

Patch posted to:
  https://lore.kernel.org/all/20260331040156.119158-1-liwang@redhat.com/


-- 
Regards,
Li Wang

Re: tools/testing/selftests

[Shuah Khan]

On 3/27/26 15:32, Andrew Morton wrote:
> 
> whinges, perrmanits, maybe I'm doing it wrong, etc.
> 
> a)
> 
> At the top-level, neither `make mrproper' nor `make clean' remove the
> executables from tools/testing/selftests/mm/.  Seems wrong?

make kselftest-all from top level builds the tests
make kselftest builds and runs the default tests.
make kselftest-clean from top level does this.

Also you can run,

make -C tools/testing/selftests/mm clean

As for mrproper support, I just applied a patch that adds kselftest
test to mrproper cleanup to linux kelftest next.

> 
> b)
> 
> Shouldn't I be able to run
> 
> 	make selftests

make kselftest is the one that does that.

Documentation/dev-tools/kselftest.rst documents different ways to
run the tests.

thanks,
-- Shuah