[LTP] [PATCH v2 0/2] openposix: Run the Open POSIX tests with kirk

[LTP] [PATCH v2 0/2] openposix: Run the Open POSIX tests with kirk

[Anders Roxell]

Hi,

There are no runtest files for the Open POSIX tests, so kirk can not
run them. Patch 1 installs the missing .args files.

Patch 2 generates runtest/openposix-* at install time. Each line calls
a small wrapper that maps the Open POSIX exit codes to the LTP ones.

Changes in v2:
- Take the list of tests from the installed run.sh files. v1 used all
  the files in DESTDIR, so it picked up old tests left from an earlier
  install, like pthread_detach_1-2. (LTP AI Reviewer)

This does not cover a removed test directory: its old run.sh stays in
DESTDIR, so its tests are still listed.

Tested on arm64. I cross built the series and ran make install into a
DESTDIR with an old pthread_detach_1-2.run-test in it. The runtest
files have 1735 tests, and pthread_detach_1-2 is not one of them.

Then I ran them with kirk in QEMU (tuxrun), and 2 of the 1735 tests
fail. pthread_rwlock_unlock_3-1 fails like in v1. fork_7-1 is
unresolved because the rootfs has no gencat.

v1: https://lore.kernel.org/ltp/20261023123436.1274448-1-anders.roxell@linaro.org/

Cheers,
Anders

Anders Roxell (2):
  openposix: Install the .args files
  openposix: Add runtest files for kirk

 testcases/open_posix_testsuite/Makefile       | 11 +++++-
 testcases/open_posix_testsuite/bin/Makefile   |  6 ++-
 .../open_posix_testsuite/bin/openposix_run.sh | 29 ++++++++++++++
 .../include/mk/config.mk.in                   |  3 ++
 .../scripts/generate-makefiles.sh             |  1 +
 .../scripts/generate-runtest.sh               | 38 +++++++++++++++++++
 6 files changed, 86 insertions(+), 2 deletions(-)
 create mode 100755 testcases/open_posix_testsuite/bin/openposix_run.sh
 create mode 100755 testcases/open_posix_testsuite/scripts/generate-runtest.sh

-- 
2.53.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v2 1/2] openposix: Install the .args files

[Anders Roxell]

run-tests.sh reads the arguments for a test from a .args file next to
it, but make install does not copy the .args files. So on an installed
system the two mqueues_multi_send_rev tests run without the number of
threads and fail with a usage message.

Install the .args files together with the tests.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
---
 testcases/open_posix_testsuite/scripts/generate-makefiles.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/testcases/open_posix_testsuite/scripts/generate-makefiles.sh b/testcases/open_posix_testsuite/scripts/generate-makefiles.sh
index 1c12a456cc41..19c781ba8054 100755
--- a/testcases/open_posix_testsuite/scripts/generate-makefiles.sh
+++ b/testcases/open_posix_testsuite/scripts/generate-makefiles.sh
@@ -182,6 +182,7 @@ install: \$(INSTALL_DIR) run.sh
 				\$(INSTALL_DIR)/\$\$file;	\\
 		fi;						\\
 	done
+	\$(if \$(wildcard \$(srcdir)/*.args),install -m 00644 \$(wildcard \$(srcdir)/*.args) \$(INSTALL_DIR)/)
 	@if [ -d speculative ]; then \$(MAKE) -C speculative install; fi
 
 .PHONY: test
-- 
2.53.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] openposix: Install the .args files

[linuxtestproject.agent]

Hi Anders,

On Wed Nov 4 23:50:50 2026 +0100, Anders Roxell <anders.roxell@linaro.org> wrote:
> openposix: Install the .args files

Verdict - Reviewed

---
Note:

The agent can sometimes produce false positives although often its
findings are genuine. If you find issues with the review, please
comment this email or ignore the suggestions.

Regards,
LTP AI Reviewer

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v2 1/2] openposix: Install the .args files

[Petr Vorel]

Hi Anders,

> run-tests.sh reads the arguments for a test from a .args file next to
> it, but make install does not copy the .args files. So on an installed
> system the two mqueues_multi_send_rev tests run without the number of
> threads and fail with a usage message.

> Install the .args files together with the tests.

> Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
> ---
>  testcases/open_posix_testsuite/scripts/generate-makefiles.sh | 1 +
>  1 file changed, 1 insertion(+)

> diff --git a/testcases/open_posix_testsuite/scripts/generate-makefiles.sh b/testcases/open_posix_testsuite/scripts/generate-makefiles.sh
> index 1c12a456cc41..19c781ba8054 100755
> --- a/testcases/open_posix_testsuite/scripts/generate-makefiles.sh
> +++ b/testcases/open_posix_testsuite/scripts/generate-makefiles.sh
> @@ -182,6 +182,7 @@ install: \$(INSTALL_DIR) run.sh
>  				\$(INSTALL_DIR)/\$\$file;	\\
>  		fi;						\\
>  	done
> +	\$(if \$(wildcard \$(srcdir)/*.args),install -m 00644 \$(wildcard \$(srcdir)/*.args) \$(INSTALL_DIR)/)

We talk about these 2 files:

$ git ls-files |grep args$
testcases/open_posix_testsuite/stress/mqueues/mqueues_multi_send_rev_1.args
testcases/open_posix_testsuite/stress/mqueues/mqueues_multi_send_rev_2.args

Looking at the use:

sh -c "$SCRIPT_DIR/t0 $TIMEOUT_VAL ./$1 $(cat ./$(echo "$1" | sed 's,\.[^\.]*,,').args 2>/dev/null)" > $complog 2>&1

Uh, that ugly run-tests.sh should be rewritten.

Ideally, we would get rid of these 2 files, i.e. the in multi_send_rev_[12].c I
would #define some constant for the default value and replace

	if ((2 != argc) || ((num = atoi(argv[1])) <= 0)) {
		fprintf(stderr, "Usage: %s number_of_threads\n", argv[0]);
		return PTS_FAIL;
	}

with using it if argc < 2.
(+ remove <getopt.h>)

And update run-tests.sh to remove that section.

Would you mind to send v2 doing it?

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v2 2/2] openposix: Add runtest files for kirk

[Anders Roxell]

LTP has no runtest files for the Open POSIX tests, so kirk can not run
them. They are only run with run.sh, which prints its own summary.

Generate a runtest file for each suite at install time. The speculative
tests are left out, like in run.sh, and the arguments from a test's
.args file go on its runtest line. Each line calls a small wrapper,
openposix_run.sh, which runs the test from its own directory like
run-tests.sh does.

The list of tests comes from the installed run.sh files, so it is the
same list that run.sh runs. Old tests left in DESTDIR from an earlier
install do not end up in the runtest files.

The wrapper also maps PTS_UNSUPPORTED and PTS_UNTESTED to TCONF.
Without it kirk reports them as TWARN and TFAIL.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
---
 testcases/open_posix_testsuite/Makefile       | 11 +++++-
 testcases/open_posix_testsuite/bin/Makefile   |  6 ++-
 .../open_posix_testsuite/bin/openposix_run.sh | 29 ++++++++++++++
 .../include/mk/config.mk.in                   |  3 ++
 .../scripts/generate-runtest.sh               | 38 +++++++++++++++++++
 5 files changed, 85 insertions(+), 2 deletions(-)
 create mode 100755 testcases/open_posix_testsuite/bin/openposix_run.sh
 create mode 100755 testcases/open_posix_testsuite/scripts/generate-runtest.sh

diff --git a/testcases/open_posix_testsuite/Makefile b/testcases/open_posix_testsuite/Makefile
index 4bead182170d..ce41643686d5 100644
--- a/testcases/open_posix_testsuite/Makefile
+++ b/testcases/open_posix_testsuite/Makefile
@@ -74,7 +74,7 @@ generate-makefiles: distclean-makefiles
 		$(top_srcdir)/scripts/generate-makefiles.sh
 
 .PHONY: install
-install: bin-install conformance-install functional-install stress-install
+install: bin-install conformance-install functional-install stress-install runtest-install
 
 .PHONY: test
 test: conformance-test functional-test stress-test
@@ -119,6 +119,15 @@ stress-test:
 	@rm -f `if echo "$(LOGFILE)" | grep -q '^/'; then echo "$(LOGFILE)"; else echo "\`pwd\`/$(LOGFILE)"; fi`.$@
 	@$(TEST_MAKE) -C stress test
 
+.PHONY: runtest-install
+runtest-install: $(addsuffix -install,$(SUBDIRS))
+	@mkdir -p $(DESTDIR)/$(runtestdir)
+	@set -e; for suite in $(SUBDIRS); do \
+		$(top_srcdir)/scripts/generate-runtest.sh \
+			$(DESTDIR)/$(testdir) "$(testdir_rel)" $$suite \
+			> $(DESTDIR)/$(runtestdir)/openposix-$$suite; \
+	done
+
 # Tools build and install targets.
 .PHONY: bin-install
 bin-install:
diff --git a/testcases/open_posix_testsuite/bin/Makefile b/testcases/open_posix_testsuite/bin/Makefile
index 262a6567c5b6..5dfff99effcb 100644
--- a/testcases/open_posix_testsuite/bin/Makefile
+++ b/testcases/open_posix_testsuite/bin/Makefile
@@ -16,7 +16,7 @@ clean:
 	@rm -f t0.val
 
 .PHONY: install
-install: clean $(DESTDIR)/$(bindir) $(DESTDIR)/$(testdir_bin)
+install: clean $(DESTDIR)/$(bindir) $(DESTDIR)/$(testdir_bin) $(DESTDIR)/$(ltp_bindir)
 	set -e; for file in $(INSTALL_BIN_TARGETS); do           \
 		install -m 00755 $$file $(DESTDIR)/$(bindir)/$$file; \
 	done
@@ -27,9 +27,13 @@ install: clean $(DESTDIR)/$(bindir) $(DESTDIR)/$(testdir_bin)
 		install -m 00755 $$file $(DESTDIR)/$(testdir_bin)/$$file; \
 	done
 
+	install -m 00755 openposix_run.sh $(DESTDIR)/$(ltp_bindir)/
 
 $(DESTDIR)/$(bindir):
 	mkdir -p $@
 
 $(DESTDIR)/$(testdir_bin):
 	mkdir -p $@
+
+$(DESTDIR)/$(ltp_bindir):
+	mkdir -p $@
diff --git a/testcases/open_posix_testsuite/bin/openposix_run.sh b/testcases/open_posix_testsuite/bin/openposix_run.sh
new file mode 100755
index 000000000000..43c52a38d956
--- /dev/null
+++ b/testcases/open_posix_testsuite/bin/openposix_run.sh
@@ -0,0 +1,29 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2026 Linaro Limited
+
+LTPROOT=${LTPROOT:-$(cd "$(dirname "$0")/../.." && pwd)}
+
+if [ $# -lt 1 ]; then
+	echo "Usage: $(basename "$0") <test path relative to LTPROOT> [args]" >&2
+	exit 2
+fi
+
+test_path=$LTPROOT/$1
+test_file=${test_path##*/}
+shift
+
+if [ ! -x "$test_path" ]; then
+	echo "$test_path is missing or not executable" >&2
+	exit 2
+fi
+
+cd "${test_path%/*}" || exit 2
+
+./"$test_file" "$@"
+ret=$?
+
+case $ret in
+4|5) exit 32 ;;
+*) exit $ret ;;
+esac
diff --git a/testcases/open_posix_testsuite/include/mk/config.mk.in b/testcases/open_posix_testsuite/include/mk/config.mk.in
index 20422e0f1047..fae80116bbf7 100644
--- a/testcases/open_posix_testsuite/include/mk/config.mk.in
+++ b/testcases/open_posix_testsuite/include/mk/config.mk.in
@@ -14,4 +14,7 @@ testdir_rel := @testdir@
 testdir     := ${prefix}/${testdir_rel}
 testdir_bin := ${testdir}/bin
 
+ltp_bindir  := ${prefix}/testcases/bin
+runtestdir  := ${prefix}/runtest
+
 CFLAGS  += -std=c99 -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=700 -Wall -Wextra
diff --git a/testcases/open_posix_testsuite/scripts/generate-runtest.sh b/testcases/open_posix_testsuite/scripts/generate-runtest.sh
new file mode 100755
index 000000000000..da743d3c1edb
--- /dev/null
+++ b/testcases/open_posix_testsuite/scripts/generate-runtest.sh
@@ -0,0 +1,38 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2026 Linaro Limited
+
+if [ $# -ne 3 ]; then
+	echo "Usage: $(basename "$0") <installed testdir> <testdir relative to LTPROOT> <suite>" >&2
+	exit 1
+fi
+
+installed_testdir=$1
+testdir_rel=$2
+suite=$3
+
+cd "$installed_testdir" || exit 1
+
+if [ ! -d "$suite" ]; then
+	echo "$installed_testdir/$suite does not exist" >&2
+	exit 1
+fi
+
+run_tests_line='^[^ ]*/run-tests\.sh [^ ]* '
+
+find "$suite" -name speculative -prune -o -name run.sh -print | sort | \
+while read -r run_sh; do
+	if ! grep -q "$run_tests_line" "$run_sh"; then
+		echo "$run_sh has no run-tests.sh line" >&2
+		exit 1
+	fi
+	dir=${run_sh%/run.sh}
+	tests=$(sed -n "s,$run_tests_line,,p" "$run_sh")
+	for test_name in $tests; do
+		[ -f "$dir/$test_name" ] || continue
+		base=${test_name%.*}
+		args=
+		[ -f "$dir/$base.args" ] && read -r args < "$dir/$base.args"
+		echo "$base openposix_run.sh ${testdir_rel:+$testdir_rel/}$dir/$test_name${args:+ $args}"
+	done
+done
-- 
2.53.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH 1/2] openposix: Install the .args files

[Anders Roxell]

run-tests.sh reads the arguments for a test from a .args file next to
it, but make install does not copy the .args files. So on an installed
system the two mqueues_multi_send_rev tests run without the number of
threads and fail with a usage message.

Install the .args files together with the tests.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
---
 testcases/open_posix_testsuite/scripts/generate-makefiles.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/testcases/open_posix_testsuite/scripts/generate-makefiles.sh b/testcases/open_posix_testsuite/scripts/generate-makefiles.sh
index 1c12a456cc41..19c781ba8054 100755
--- a/testcases/open_posix_testsuite/scripts/generate-makefiles.sh
+++ b/testcases/open_posix_testsuite/scripts/generate-makefiles.sh
@@ -182,6 +182,7 @@ install: \$(INSTALL_DIR) run.sh
 				\$(INSTALL_DIR)/\$\$file;	\\
 		fi;						\\
 	done
+	\$(if \$(wildcard \$(srcdir)/*.args),install -m 00644 \$(wildcard \$(srcdir)/*.args) \$(INSTALL_DIR)/)
 	@if [ -d speculative ]; then \$(MAKE) -C speculative install; fi

 .PHONY: test
--
2.53.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] openposix: Install the .args files

[linuxtestproject.agent]

Hi Anders,

On 23 Oct 2026, Anders Roxell wrote:
> openposix: Install the .args files

--- [PATCH 2/2] ---

> +++ b/testcases/open_posix_testsuite/scripts/generate-runtest.sh
> +find "$suite" -name speculative -prune -o -type f \
> +	\( -name '*.run-test' -o -name '*.sh' ! -name run.sh \) -print | sort | \

Medium: On an in-place upgrade, this scan re-enables obsolete binaries such
as pthread_detach_1-2, which was removed for testing undefined behavior and
is absent from the current run.sh. Generate entries from the current build's
install manifests rather than scanning all files left in the destination.

Verdict - Needs revision

---
Note:

The agent can sometimes produce false positives although often its
findings are genuine. If you find issues with the review, please
comment this email or ignore the suggestions.

Regards,
LTP AI Reviewer

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] openposix: Install the .args files

[Anders Roxell]

On 2026-10-23 17:50, linuxtestproject.agent@gmail.com wrote:
> Hi Anders,
> 
> On 23 Oct 2026, Anders Roxell wrote:
> > openposix: Install the .args files
> 
> --- [PATCH 2/2] ---
> 
> > +++ b/testcases/open_posix_testsuite/scripts/generate-runtest.sh
> > +find "$suite" -name speculative -prune -o -type f \
> > +	\( -name '*.run-test' -o -name '*.sh' ! -name run.sh \) -print | sort | \
> 
> Medium: On an in-place upgrade, this scan re-enables obsolete binaries such
> as pthread_detach_1-2, which was removed for testing undefined behavior and
> is absent from the current run.sh. Generate entries from the current build's
> install manifests rather than scanning all files left in the destination.

This is about 2/2.

Yes, this looks like a real bug. make install never removes old files,
so the find picks up anything that is left from an earlier install.

I send a v2 where the list comes from INSTALL_TARGETS in the generated
Makefiles.

Cheers,
Anders

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp