[LTP] [PATCH v3 0/2] Add kirk & ltx tools

[LTP] [PATCH v3 0/2] Add kirk & ltx tools

[Andrea Cervesato]

From: Andrea Cervesato <andrea.cervesato@suse.com>

Replaced runltp-ng with kirk and added ltx tool for parallel execution.

Andrea Cervesato (2):
  Replace runltp-ng with kirk framework
  Add Linux Test eXecutor inside tools

 .gitmodules        |  6 ++++++
 tools/kirk         |  1 +
 tools/ltx/Makefile | 31 +++++++++++++++++++++++++++++++
 tools/ltx/ltx-src  |  1 +
 4 files changed, 39 insertions(+)
 create mode 160000 tools/kirk
 create mode 100644 tools/ltx/Makefile
 create mode 160000 tools/ltx/ltx-src

-- 
2.35.3


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v3 1/2] Replace runltp-ng with kirk framework

[Andrea Cervesato]

From: Andrea Cervesato <andrea.cervesato@suse.com>

Kirk application is the runltp-ng successor and it aims to merge
multiple Linux testing frameworks in one tool, providing support
for remote testing via Qemu, SSH, LTX, parallel execution and much
more.

This patch deprecates runltp-ng, which is not replaced by kirk. All
runltp-ng features are present in kirk and even more.

Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
 .gitmodules | 3 +++
 tools/kirk  | 1 +
 2 files changed, 4 insertions(+)
 create mode 160000 tools/kirk

diff --git a/.gitmodules b/.gitmodules
index d1d558b9e..c389186c9 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -7,3 +7,6 @@
 [submodule "tools/runltp-ng"]
 	path = tools/runltp-ng
 	url = https://github.com/linux-test-project/runltp-ng.git
+[submodule "tools/kirk"]
+	path = tools/kirk
+	url = https://github.com/linux-test-project/kirk.git
diff --git a/tools/kirk b/tools/kirk
new file mode 160000
index 000000000..c7419dd6f
--- /dev/null
+++ b/tools/kirk
@@ -0,0 +1 @@
+Subproject commit c7419dd6f73d90f3f02aa17d30270e895e30c88e
-- 
2.35.3


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 1/2] Replace runltp-ng with kirk framework

[Petr Vorel]

Hi Andrea,

...
> +++ b/.gitmodules
> @@ -7,3 +7,6 @@
>  [submodule "tools/runltp-ng"]
>  	path = tools/runltp-ng
>  	url = https://github.com/linux-test-project/runltp-ng.git
> +[submodule "tools/kirk"]
> +	path = tools/kirk
> +	url = https://github.com/linux-test-project/kirk.git

I'm not sure if anybody asked you to keep runltp-ng, but I'd prefer to remove
runltp-ng as you did in v2. I don't think runltp-ng was really adopted a lot

We should have runltp-ng symlink to kirk at least in LTP:

cd tools/kirk && make install
install -m 00644 ../../tools/kirk/libkirk/*.py /opt/ltp/libkirk
install -m 00775 ../../tools/kirk/kirk /opt/ltp/kirk
...
Here I would add
ln -s /opt/ltp/kirk /opt/ltp/runltp-ng

...
> +++ b/tools/kirk
> @@ -0,0 +1 @@
> +Subproject commit c7419dd6f73d90f3f02aa17d30270e895e30c88e
+1 for fixing this.

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 1/2] Replace runltp-ng with kirk framework

[Andrea Cervesato via ltp]

Hi,

On 9/18/23 11:01, Petr Vorel wrote:
> Hi Andrea,
>
> ...
>> +++ b/.gitmodules
>> @@ -7,3 +7,6 @@
>>   [submodule "tools/runltp-ng"]
>>   	path = tools/runltp-ng
>>   	url = https://github.com/linux-test-project/runltp-ng.git
>> +[submodule "tools/kirk"]
>> +	path = tools/kirk
>> +	url = https://github.com/linux-test-project/kirk.git
> I'm not sure if anybody asked you to keep runltp-ng, but I'd prefer to remove
> runltp-ng as you did in v2. I don't think runltp-ng was really adopted a lot
Yes, I forgot to do git rm on that folder.
>
> We should have runltp-ng symlink to kirk at least in LTP:
>
> cd tools/kirk && make install
> install -m 00644 ../../tools/kirk/libkirk/*.py /opt/ltp/libkirk
> install -m 00775 ../../tools/kirk/kirk /opt/ltp/kirk
> ...
> Here I would add
> ln -s /opt/ltp/kirk /opt/ltp/runltp-ng
This has to be done in the kirk repo.
> ...
>> +++ b/tools/kirk
>> @@ -0,0 +1 @@
>> +Subproject commit c7419dd6f73d90f3f02aa17d30270e895e30c88e
> +1 for fixing this.
>
> Kind regards,
> Petr

Andrea


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v3 2/2] Add Linux Test eXecutor inside tools

[Andrea Cervesato]

From: Andrea Cervesato <andrea.cervesato@suse.com>

The ltx program runs on the system under test (SUT). It's primary
purpose is to run test executables in parallel and serialise the
results.

Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
 .gitmodules        |  3 +++
 tools/ltx/Makefile | 31 +++++++++++++++++++++++++++++++
 tools/ltx/ltx-src  |  1 +
 3 files changed, 35 insertions(+)
 create mode 100644 tools/ltx/Makefile
 create mode 160000 tools/ltx/ltx-src

diff --git a/.gitmodules b/.gitmodules
index c389186c9..2a8b7a399 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -10,3 +10,6 @@
 [submodule "tools/kirk"]
 	path = tools/kirk
 	url = https://github.com/linux-test-project/kirk.git
+[submodule "tools/ltx/ltx-src"]
+	path = tools/ltx/ltx-src
+	url = https://github.com/linux-test-project/ltx.git
diff --git a/tools/ltx/Makefile b/tools/ltx/Makefile
new file mode 100644
index 000000000..4810ec8df
--- /dev/null
+++ b/tools/ltx/Makefile
@@ -0,0 +1,31 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2023 Cyril Hrubis <chrubis@suse.cz>
+# Copyright (c) 2023 Andrea Cervesato <andrea.cervesato@suse.com>
+#
+# Install script for Linux Test eXecutor
+
+top_srcdir		?= ../..
+
+include $(top_srcdir)/include/mk/env_pre.mk
+
+ifneq ($(wildcard $(abs_srcdir)/ltx-src/*),)
+
+BINARY=ltx
+
+MAKE_TARGETS := $(BINARY)
+
+CFLAGS+=-I$(abs_srcdir)/ltx-src/ -I$(abs_srcdir)/ltx-src/msgpack/
+
+$(BINARY): $(wildcard $(abs_srcdir)/ltx-src/*.c $(abs_srcdir)/ltx-src/msgpack/*.c)
+ifdef VERBOSE
+	$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
+else
+	@echo CC $@
+	@$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
+endif
+
+INSTALL_DIR := $(prefix)
+
+endif
+
+include $(top_srcdir)/include/mk/generic_leaf_target.mk
diff --git a/tools/ltx/ltx-src b/tools/ltx/ltx-src
new file mode 160000
index 000000000..d6d150947
--- /dev/null
+++ b/tools/ltx/ltx-src
@@ -0,0 +1 @@
+Subproject commit d6d1509479537f4fdfa9b5adcb67eb6312714999
-- 
2.35.3


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/2] Add Linux Test eXecutor inside tools

[Petr Vorel]

> From: Andrea Cervesato <andrea.cervesato@suse.com>

> The ltx program runs on the system under test (SUT). It's primary
> purpose is to run test executables in parallel and serialise the
> results.

> Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
> ---
>  .gitmodules        |  3 +++
>  tools/ltx/Makefile | 31 +++++++++++++++++++++++++++++++
>  tools/ltx/ltx-src  |  1 +
>  3 files changed, 35 insertions(+)
>  create mode 100644 tools/ltx/Makefile
>  create mode 160000 tools/ltx/ltx-src

> diff --git a/.gitmodules b/.gitmodules
> index c389186c9..2a8b7a399 100644
> --- a/.gitmodules
> +++ b/.gitmodules
> @@ -10,3 +10,6 @@
>  [submodule "tools/kirk"]
>  	path = tools/kirk
>  	url = https://github.com/linux-test-project/kirk.git
> +[submodule "tools/ltx/ltx-src"]
> +	path = tools/ltx/ltx-src
> +	url = https://github.com/linux-test-project/ltx.git
> diff --git a/tools/ltx/Makefile b/tools/ltx/Makefile
> new file mode 100644
> index 000000000..4810ec8df
> --- /dev/null
> +++ b/tools/ltx/Makefile
> @@ -0,0 +1,31 @@
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2023 Cyril Hrubis <chrubis@suse.cz>
> +# Copyright (c) 2023 Andrea Cervesato <andrea.cervesato@suse.com>
> +#
> +# Install script for Linux Test eXecutor
> +
> +top_srcdir		?= ../..
> +
> +include $(top_srcdir)/include/mk/env_pre.mk
> +
> +ifneq ($(wildcard $(abs_srcdir)/ltx-src/*),)
> +
> +BINARY=ltx
> +
> +MAKE_TARGETS := $(BINARY)
> +
> +CFLAGS+=-I$(abs_srcdir)/ltx-src/ -I$(abs_srcdir)/ltx-src/msgpack/
> +
> +$(BINARY): $(wildcard $(abs_srcdir)/ltx-src/*.c $(abs_srcdir)/ltx-src/msgpack/*.c)
> +ifdef VERBOSE
> +	$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
> +else
> +	@echo CC $@
> +	@$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
> +endif
This works, but why we don't use approach used for sparc, i.e.

$(MAKE) -C ltx-src

That way we would not have to redefine the default rules.

Also it detects missing git clone and runs
git submodule update --init if needed.

I'm asking for a same approach, not only because later we can unify and reuse
the code in some make helper, but also because it'd be more user friendly if our
new git submodules work the same way.

But if there is not enough time before release I would not be against this.

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/2] Add Linux Test eXecutor inside tools

[Andrea Cervesato via ltp]

Hi,

Sorry but I seen this review when I already sent the patch 1 or 2 
minutes later.

On 9/18/23 11:17, Petr Vorel wrote:
>> From: Andrea Cervesato<andrea.cervesato@suse.com>
>> The ltx program runs on the system under test (SUT). It's primary
>> purpose is to run test executables in parallel and serialise the
>> results.
>> Signed-off-by: Andrea Cervesato<andrea.cervesato@suse.com>
>> ---
>>   .gitmodules        |  3 +++
>>   tools/ltx/Makefile | 31 +++++++++++++++++++++++++++++++
>>   tools/ltx/ltx-src  |  1 +
>>   3 files changed, 35 insertions(+)
>>   create mode 100644 tools/ltx/Makefile
>>   create mode 160000 tools/ltx/ltx-src
>> diff --git a/.gitmodules b/.gitmodules
>> index c389186c9..2a8b7a399 100644
>> --- a/.gitmodules
>> +++ b/.gitmodules
>> @@ -10,3 +10,6 @@
>>   [submodule "tools/kirk"]
>>   	path = tools/kirk
>>   	url =https://github.com/linux-test-project/kirk.git
>> +[submodule "tools/ltx/ltx-src"]
>> +	path = tools/ltx/ltx-src
>> +	url =https://github.com/linux-test-project/ltx.git
>> diff --git a/tools/ltx/Makefile b/tools/ltx/Makefile
>> new file mode 100644
>> index 000000000..4810ec8df
>> --- /dev/null
>> +++ b/tools/ltx/Makefile
>> @@ -0,0 +1,31 @@
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +# Copyright (c) 2023 Cyril Hrubis<chrubis@suse.cz>
>> +# Copyright (c) 2023 Andrea Cervesato<andrea.cervesato@suse.com>
>> +#
>> +# Install script for Linux Test eXecutor
>> +
>> +top_srcdir		?= ../..
>> +
>> +include $(top_srcdir)/include/mk/env_pre.mk
>> +
>> +ifneq ($(wildcard $(abs_srcdir)/ltx-src/*),)
>> +
>> +BINARY=ltx
>> +
>> +MAKE_TARGETS := $(BINARY)
>> +
>> +CFLAGS+=-I$(abs_srcdir)/ltx-src/ -I$(abs_srcdir)/ltx-src/msgpack/
>> +
>> +$(BINARY): $(wildcard $(abs_srcdir)/ltx-src/*.c $(abs_srcdir)/ltx-src/msgpack/*.c)
>> +ifdef VERBOSE
>> +	$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
>> +else
>> +	@echo CC $@
>> +	@$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
>> +endif
> This works, but why we don't use approach used for sparc, i.e.
>
> $(MAKE) -C ltx-src
>
> That way we would not have to redefine the default rules.
>
> Also it detects missing git clone and runs
> git submodule update --init if needed.
>
> I'm asking for a same approach, not only because later we can unify and reuse
> the code in some make helper, but also because it'd be more user friendly if our
> new git submodules work the same way.
>
> But if there is not enough time before release I would not be against this.
>
> Kind regards,
> Petr
>
I don't see reasons to change Makefile if it's using the right LTP 
variables and approach to tools installation within the LTP folder.

Andrea

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/2] Add Linux Test eXecutor inside tools

[Cyril Hrubis]

Hi!
> This works, but why we don't use approach used for sparc, i.e.
> 
> $(MAKE) -C ltx-src
> 
> That way we would not have to redefine the default rules.

Unfortunately this does not work well either because the ltx does not
have install targed and trying to install a file from a subdirectory is
not possible with the LTP build system. I've tried that and it didn't
work.

Let's merge this version that works, we can try to do make things better
later on.

> Also it detects missing git clone and runs
> git submodule update --init if needed.
>
> I'm asking for a same approach, not only because later we can unify and reuse
> the code in some make helper, but also because it'd be more user friendly if our
> new git submodules work the same way.

Now this needs to be discussed first. Currently the submodule is
installed when user writes make check. That means that it's not
triggered in default build and the default build works without network
connection. I do not think that we should attempt to download additional
repositories on default build.

> But if there is not enough time before release I would not be against this.

I'm would say that there is not enough time for a proper discussion, so
I would really keep things as they are in the latest patchset and leave
the discussion for later.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/2] Add Linux Test eXecutor inside tools

[Cyril Hrubis]

Hi!
> Unfortunately this does not work well either because the ltx does not
> have install targed and trying to install a file from a subdirectory is
> not possible with the LTP build system. I've tried that and it didn't
> work.
> 
> Let's merge this version that works, we can try to do make things better
> later on.

Also one more point $(MAKE) -C ltx-src does not support out-of-tree
build and while it's probably fine for 'make check' not to support it,
since that is a developer tooling, it should work fine for ltx that is
going to be packaged/installed on the SUT.

So I suppose that we are stuck with the LTP ltx Makefile unless we:

- add support for out-of-tree build to ltx/Makefile
- drop out-of-tree build support from LTP

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/2] Add Linux Test eXecutor inside tools

[Petr Vorel]

> Hi!
> > Unfortunately this does not work well either because the ltx does not
> > have install targed and trying to install a file from a subdirectory is
> > not possible with the LTP build system. I've tried that and it didn't
> > work.

> > Let's merge this version that works, we can try to do make things better
> > later on.

> Also one more point $(MAKE) -C ltx-src does not support out-of-tree
> build and while it's probably fine for 'make check' not to support it,
> since that is a developer tooling, it should work fine for ltx that is
> going to be packaged/installed on the SUT.
Very good point! It'd be nice to note this reason in the git commit message.

> So I suppose that we are stuck with the LTP ltx Makefile unless we:

> - add support for out-of-tree build to ltx/Makefile
> - drop out-of-tree build support from LTP

I guess even Buildroot and Yocto does not use out-of-tree build.
Maybe we should really remove the support.

Kind regards,
Petr


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/2] Add Linux Test eXecutor inside tools

[Petr Vorel]

> Hi!
> > This works, but why we don't use approach used for sparc, i.e.

> > $(MAKE) -C ltx-src

> > That way we would not have to redefine the default rules.

> Unfortunately this does not work well either because the ltx does not
> have install targed and trying to install a file from a subdirectory is
> not possible with the LTP build system. I've tried that and it didn't
> work.

> Let's merge this version that works, we can try to do make things better
> later on.

Sure. BTW I prefer v4, which replaces runltp-ng instead of keeping it.

> > Also it detects missing git clone and runs
> > git submodule update --init if needed.

> > I'm asking for a same approach, not only because later we can unify and reuse
> > the code in some make helper, but also because it'd be more user friendly if our
> > new git submodules work the same way.

> Now this needs to be discussed first. Currently the submodule is
> installed when user writes make check. That means that it's not
> triggered in default build and the default build works without network
> connection. I do not think that we should attempt to download additional
> repositories on default build.

Make sense.

> > But if there is not enough time before release I would not be against this.

> I'm would say that there is not enough time for a proper discussion, so
> I would really keep things as they are in the latest patchset and leave
> the discussion for later.

Sure.

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp