[PATCH RFD] selftests: create test-specific kconfig fragments

[PATCH RFD] selftests: create test-specific kconfig fragments

[Bamvor Jian Zhang]

There is a discussion about improving the usability of kselftest by
creating test-specific kconfig in recent kernel Summit. Furthormore,
there are different approaches to do it:

1) keep test-specific kconfig fragments inside each selftest so that
merge_configs.sh could build up a kernel that can test the specific
or all feature(s).

2) In the main menu, have an additional option/flag for each feature
that should be enabled when ksefltests are wanted.  Similar to the
CONFIG_COMPILE_TEST flag.

This patch try to do option 1. Hope it is a good start for
discussion. User could use these configs with merge_config.sh script:

Enable config for specific testcases:
(export ARCH=xxx #for cross compiling)
./scripts/kconfig/merge_config.sh .config \
		tools/testing/selftests/xxx/config

Enable configs for all testcases:
(export ARCH=xxx #for cross compiling)
./scripts/kconfig/merge_config.sh .config \
		tools/testing/selftests/*/config

Signed-off-by: Bamvor Jian Zhang <bamvor.zhangjian-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
---
 tools/testing/selftests/cpu-hotplug/config    | 2 ++
 tools/testing/selftests/firmware/config       | 1 +
 tools/testing/selftests/ftrace/config         | 1 +
 tools/testing/selftests/ipc/config            | 2 ++
 tools/testing/selftests/memory-hotplug/config | 4 ++++
 tools/testing/selftests/mount/config          | 2 ++
 tools/testing/selftests/net/config            | 3 +++
 tools/testing/selftests/pstore/config         | 4 ++++
 tools/testing/selftests/seccomp/config        | 2 ++
 tools/testing/selftests/static_keys/config    | 1 +
 tools/testing/selftests/user/config           | 1 +
 tools/testing/selftests/vm/config             | 1 +
 tools/testing/selftests/zram/config           | 2 ++
 13 files changed, 26 insertions(+)
 create mode 100644 tools/testing/selftests/cpu-hotplug/config
 create mode 100644 tools/testing/selftests/firmware/config
 create mode 100644 tools/testing/selftests/ftrace/config
 create mode 100644 tools/testing/selftests/ipc/config
 create mode 100644 tools/testing/selftests/memory-hotplug/config
 create mode 100644 tools/testing/selftests/mount/config
 create mode 100644 tools/testing/selftests/net/config
 create mode 100644 tools/testing/selftests/pstore/config
 create mode 100644 tools/testing/selftests/seccomp/config
 create mode 100644 tools/testing/selftests/static_keys/config
 create mode 100644 tools/testing/selftests/user/config
 create mode 100644 tools/testing/selftests/vm/config
 create mode 100644 tools/testing/selftests/zram/config

diff --git a/tools/testing/selftests/cpu-hotplug/config b/tools/testing/selftests/cpu-hotplug/config
new file mode 100644
index 0000000..e6ab090
--- /dev/null
+++ b/tools/testing/selftests/cpu-hotplug/config
@@ -0,0 +1,2 @@
+CONFIG_NOTIFIER_ERROR_INJECTION=y
+CONFIG_CPU_NOTIFIER_ERROR_INJECT=m
diff --git a/tools/testing/selftests/firmware/config b/tools/testing/selftests/firmware/config
new file mode 100644
index 0000000..c8137f7
--- /dev/null
+++ b/tools/testing/selftests/firmware/config
@@ -0,0 +1 @@
+CONFIG_TEST_FIRMWARE=y
diff --git a/tools/testing/selftests/ftrace/config b/tools/testing/selftests/ftrace/config
new file mode 100644
index 0000000..ef82146
--- /dev/null
+++ b/tools/testing/selftests/ftrace/config
@@ -0,0 +1 @@
+CONFIG_FTRACE=y
diff --git a/tools/testing/selftests/ipc/config b/tools/testing/selftests/ipc/config
new file mode 100644
index 0000000..0702447
--- /dev/null
+++ b/tools/testing/selftests/ipc/config
@@ -0,0 +1,2 @@
+CONFIG_EXPERT=y
+CONFIG_CHECKPOINT_RESTORE=y
diff --git a/tools/testing/selftests/memory-hotplug/config b/tools/testing/selftests/memory-hotplug/config
new file mode 100644
index 0000000..2fde301
--- /dev/null
+++ b/tools/testing/selftests/memory-hotplug/config
@@ -0,0 +1,4 @@
+CONFIG_MEMORY_HOTPLUG=y
+CONFIG_MEMORY_HOTPLUG_SPARSE=y
+CONFIG_NOTIFIER_ERROR_INJECTION=y
+CONFIG_MEMORY_NOTIFIER_ERROR_INJECT=m
diff --git a/tools/testing/selftests/mount/config b/tools/testing/selftests/mount/config
new file mode 100644
index 0000000..b5d881e
--- /dev/null
+++ b/tools/testing/selftests/mount/config
@@ -0,0 +1,2 @@
+CONFIG_USER_NS=y
+CONFIG_DEVPTS_MULTIPLE_INSTANCES=y
diff --git a/tools/testing/selftests/net/config b/tools/testing/selftests/net/config
new file mode 100644
index 0000000..e57b4ac
--- /dev/null
+++ b/tools/testing/selftests/net/config
@@ -0,0 +1,3 @@
+CONFIG_USER_NS=y
+CONFIG_BPF_SYSCALL=y
+CONFIG_TEST_BPF=m
diff --git a/tools/testing/selftests/pstore/config b/tools/testing/selftests/pstore/config
new file mode 100644
index 0000000..6a8e5a9
--- /dev/null
+++ b/tools/testing/selftests/pstore/config
@@ -0,0 +1,4 @@
+CONFIG_MISC_FILESYSTEMS=y
+CONFIG_PSTORE=y
+CONFIG_PSTORE_PMSG=y
+CONFIG_PSTORE_CONSOLE=y
diff --git a/tools/testing/selftests/seccomp/config b/tools/testing/selftests/seccomp/config
new file mode 100644
index 0000000..db1e11b
--- /dev/null
+++ b/tools/testing/selftests/seccomp/config
@@ -0,0 +1,2 @@
+CONFIG_SECCOMP=y
+CONFIG_SECCOMP_FILTER=y
diff --git a/tools/testing/selftests/static_keys/config b/tools/testing/selftests/static_keys/config
new file mode 100644
index 0000000..d538fb7
--- /dev/null
+++ b/tools/testing/selftests/static_keys/config
@@ -0,0 +1 @@
+CONFIG_TEST_STATIC_KEYS=m
diff --git a/tools/testing/selftests/user/config b/tools/testing/selftests/user/config
new file mode 100644
index 0000000..784ed84
--- /dev/null
+++ b/tools/testing/selftests/user/config
@@ -0,0 +1 @@
+CONFIG_TEST_USER_COPY=m
diff --git a/tools/testing/selftests/vm/config b/tools/testing/selftests/vm/config
new file mode 100644
index 0000000..698c7ed
--- /dev/null
+++ b/tools/testing/selftests/vm/config
@@ -0,0 +1 @@
+CONFIG_USERFAULTFD=y
diff --git a/tools/testing/selftests/zram/config b/tools/testing/selftests/zram/config
new file mode 100644
index 0000000..e0cc47e
--- /dev/null
+++ b/tools/testing/selftests/zram/config
@@ -0,0 +1,2 @@
+CONFIG_ZSMALLOC=y
+CONFIG_ZRAM=m
-- 
2.1.4

Re: [PATCH RFD] selftests: create test-specific kconfig fragments

[Kevin Hilman]

Hi Bamvor,

Bamvor Jian Zhang <bamvor.zhangjian-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> writes:

> There is a discussion about improving the usability of kselftest by
> creating test-specific kconfig in recent kernel Summit. Furthormore,
> there are different approaches to do it:
>
> 1) keep test-specific kconfig fragments inside each selftest so that
> merge_configs.sh could build up a kernel that can test the specific
> or all feature(s).
>
> 2) In the main menu, have an additional option/flag for each feature
> that should be enabled when ksefltests are wanted.  Similar to the
> CONFIG_COMPILE_TEST flag.
>
> This patch try to do option 1. Hope it is a good start for
> discussion. User could use these configs with merge_config.sh script:

For broader discussion, could you please resend and include LKML?

Thanks,

Kevin

Re: [PATCH RFD] selftests: create test-specific kconfig fragments

[Michael Ellerman]

On Mon, 2015-11-16 at 14:32 -0800, Kevin Hilman wrote:
> Hi Bamvor,
> 
> Bamvor Jian Zhang <bamvor.zhangjian-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> writes:
> 

> > There is a discussion about improving the usability of kselftest by
> > creating test-specific kconfig in recent kernel Summit. Furthormore,
> > there are different approaches to do it:
> > 
> > 1) keep test-specific kconfig fragments inside each selftest so that
> > merge_configs.sh could build up a kernel that can test the specific
> > or all feature(s).
> > 
> > 2) In the main menu, have an additional option/flag for each feature
> > that should be enabled when ksefltests are wanted.  Similar to the
> > CONFIG_COMPILE_TEST flag.
> > 
> > This patch try to do option 1. Hope it is a good start for
> > discussion. User could use these configs with merge_config.sh script:
> 
> For broader discussion, could you please resend and include LKML?

Yeah good idea.

Before you do, do you want to try adding a top-level target that does the
merge, something like:

 $ make kselftest-mergeconfig


Or some other better name.

cheers

Re: [PATCH RFD] selftests: create test-specific kconfig fragments

[Bamvor Zhang Jian]

On 11/17/2015 07:06 AM, Michael Ellerman wrote:
> On Mon, 2015-11-16 at 14:32 -0800, Kevin Hilman wrote:
>> Hi Bamvor,
>>
>> Bamvor Jian Zhang <bamvor.zhangjian-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> writes:
>>
>
>>> There is a discussion about improving the usability of kselftest by
>>> creating test-specific kconfig in recent kernel Summit. Furthormore,
>>> there are different approaches to do it:
>>>
>>> 1) keep test-specific kconfig fragments inside each selftest so that
>>> merge_configs.sh could build up a kernel that can test the specific
>>> or all feature(s).
>>>
>>> 2) In the main menu, have an additional option/flag for each feature
>>> that should be enabled when ksefltests are wanted.  Similar to the
>>> CONFIG_COMPILE_TEST flag.
>>>
>>> This patch try to do option 1. Hope it is a good start for
>>> discussion. User could use these configs with merge_config.sh script:
>>
>> For broader discussion, could you please resend and include LKML?
>
> Yeah good idea.
>
> Before you do, do you want to try adding a top-level target that does the
> merge, something like:
>
>  $ make kselftest-mergeconfig
>
>
> Or some other better name.
Ok, Do you mean merge all the test config?

Regards

Bamvor
>
> cheers
>

Re: [PATCH RFD] selftests: create test-specific kconfig fragments

[Michael Ellerman]

On Tue, 2015-11-17 at 07:30 +0800, Bamvor Zhang Jian wrote:
> On 11/17/2015 07:06 AM, Michael Ellerman wrote:
> > 
> > Before you do, do you want to try adding a top-level target that does the
> > merge, something like:
> > 
> >  $ make kselftest-mergeconfig
> > 
> > 
> > Or some other better name.

> Ok, Do you mean merge all the test config?

Yeah sorry that wasn't very clear. I meant that it would essentialy do your
logic to merge all the config fragments:

./scripts/kconfig/merge_config.sh .config tools/testing/selftests/*/config


You'll probably need to be more careful with $(srctree) vs $(objtree) etc. Have
a look at the merge_into_defconfig rule in arch/powerpc/Makefile for an
example.

cheers

Re: [PATCH RFD] selftests: create test-specific kconfig fragments

[Bamvor Zhang Jian]

Hi, Michael

On 17 November 2015 at 07:42, Michael Ellerman <mpe-Gsx/Oe8HsFggBc27wqDAHg@public.gmane.org> wrote:
> On Tue, 2015-11-17 at 07:30 +0800, Bamvor Zhang Jian wrote:
>> On 11/17/2015 07:06 AM, Michael Ellerman wrote:
>> >
>> > Before you do, do you want to try adding a top-level target that does the
>> > merge, something like:
>> >
>> >  $ make kselftest-mergeconfig
>> >
>> >
>> > Or some other better name.
>
>> Ok, Do you mean merge all the test config?
>
> Yeah sorry that wasn't very clear. I meant that it would essentialy do your
> logic to merge all the config fragments:
>
> ./scripts/kconfig/merge_config.sh .config tools/testing/selftests/*/config
Yes, I agree with you.
>
>
> You'll probably need to be more careful with $(srctree) vs $(objtree) etc. Have
> a look at the merge_into_defconfig rule in arch/powerpc/Makefile for an
> example.
Yes, It is very useful for me.

Thanks

Bamvor
>
> cheers
>

Re: [PATCH RFD] selftests: create test-specific kconfig fragments

[Bamvor Zhang Jian]

On 17 November 2015 at 06:32, Kevin Hilman <khilman-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> wrote:
> Hi Bamvor,
>
> Bamvor Jian Zhang <bamvor.zhangjian-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> writes:
>
>> There is a discussion about improving the usability of kselftest by
>> creating test-specific kconfig in recent kernel Summit. Furthormore,
>> there are different approaches to do it:
>>
>> 1) keep test-specific kconfig fragments inside each selftest so that
>> merge_configs.sh could build up a kernel that can test the specific
>> or all feature(s).
>>
>> 2) In the main menu, have an additional option/flag for each feature
>> that should be enabled when ksefltests are wanted.  Similar to the
>> CONFIG_COMPILE_TEST flag.
>>
>> This patch try to do option 1. Hope it is a good start for
>> discussion. User could use these configs with merge_config.sh script:
>
> For broader discussion, could you please resend and include LKML?
Oh, sorry miss LKML. I will resend it.

Regards

Bamvor
>
> Thanks,
>
> Kevin