[LTP] [PATCH 0/2] openposix: Run the Open POSIX tests with kirk

[LTP] [PATCH 0/2] openposix: Run the Open POSIX tests with kirk

[Anders Roxell]

Hi,

There are no runtest files for the Open POSIX tests, so kirk can not
run them. Patch 1 installs the missing .args files.

Patch 2 generates runtest/openposix-* at install time. Each line calls
a small wrapper that maps the Open POSIX exit codes to the LTP ones.

Tested with kirk on arm64 in QEMU: 1735 tests, 1 failure
(pthread_rwlock_unlock_3-1).

Cheers,
Anders

Anders Roxell (2):
  openposix: Install the .args files
  openposix: Add runtest files for kirk

 testcases/open_posix_testsuite/Makefile       | 13 ++++++++-
 testcases/open_posix_testsuite/bin/Makefile   |  6 +++-
 .../open_posix_testsuite/bin/openposix_run.sh | 29 +++++++++++++++++++
 .../include/mk/config.mk.in                   |  3 ++
 .../scripts/generate-makefiles.sh             |  1 +
 .../scripts/generate-runtest.sh               | 29 +++++++++++++++++++
 6 files changed, 79 insertions(+), 2 deletions(-)
 create mode 100755 testcases/open_posix_testsuite/bin/openposix_run.sh
 create mode 100755 testcases/open_posix_testsuite/scripts/generate-runtest.sh

--
2.53.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH 1/2] openposix: Install the .args files

[Anders Roxell]

run-tests.sh reads the arguments for a test from a .args file next to
it, but make install does not copy the .args files. So on an installed
system the two mqueues_multi_send_rev tests run without the number of
threads and fail with a usage message.

Install the .args files together with the tests.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
---
 testcases/open_posix_testsuite/scripts/generate-makefiles.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/testcases/open_posix_testsuite/scripts/generate-makefiles.sh b/testcases/open_posix_testsuite/scripts/generate-makefiles.sh
index 1c12a456cc41..19c781ba8054 100755
--- a/testcases/open_posix_testsuite/scripts/generate-makefiles.sh
+++ b/testcases/open_posix_testsuite/scripts/generate-makefiles.sh
@@ -182,6 +182,7 @@ install: \$(INSTALL_DIR) run.sh
 				\$(INSTALL_DIR)/\$\$file;	\\
 		fi;						\\
 	done
+	\$(if \$(wildcard \$(srcdir)/*.args),install -m 00644 \$(wildcard \$(srcdir)/*.args) \$(INSTALL_DIR)/)
 	@if [ -d speculative ]; then \$(MAKE) -C speculative install; fi

 .PHONY: test
--
2.53.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] openposix: Install the .args files

[linuxtestproject.agent]

Hi Anders,

On 23 Oct 2026, Anders Roxell wrote:
> openposix: Install the .args files

--- [PATCH 2/2] ---

> +++ b/testcases/open_posix_testsuite/scripts/generate-runtest.sh
> +find "$suite" -name speculative -prune -o -type f \
> +	\( -name '*.run-test' -o -name '*.sh' ! -name run.sh \) -print | sort | \

Medium: On an in-place upgrade, this scan re-enables obsolete binaries such
as pthread_detach_1-2, which was removed for testing undefined behavior and
is absent from the current run.sh. Generate entries from the current build's
install manifests rather than scanning all files left in the destination.

Verdict - Needs revision

---
Note:

The agent can sometimes produce false positives although often its
findings are genuine. If you find issues with the review, please
comment this email or ignore the suggestions.

Regards,
LTP AI Reviewer

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] openposix: Install the .args files

[Anders Roxell]

On 2026-10-23 17:50, linuxtestproject.agent@gmail.com wrote:
> Hi Anders,
> 
> On 23 Oct 2026, Anders Roxell wrote:
> > openposix: Install the .args files
> 
> --- [PATCH 2/2] ---
> 
> > +++ b/testcases/open_posix_testsuite/scripts/generate-runtest.sh
> > +find "$suite" -name speculative -prune -o -type f \
> > +	\( -name '*.run-test' -o -name '*.sh' ! -name run.sh \) -print | sort | \
> 
> Medium: On an in-place upgrade, this scan re-enables obsolete binaries such
> as pthread_detach_1-2, which was removed for testing undefined behavior and
> is absent from the current run.sh. Generate entries from the current build's
> install manifests rather than scanning all files left in the destination.

This is about 2/2.

Yes, this looks like a real bug. make install never removes old files,
so the find picks up anything that is left from an earlier install.

I send a v2 where the list comes from INSTALL_TARGETS in the generated
Makefiles.

Cheers,
Anders

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH 2/2] openposix: Add runtest files for kirk

[Anders Roxell]

LTP has no runtest files for the Open POSIX tests, so kirk can not run
them. They are only run with run.sh, which prints its own summary.

Generate a runtest file for each suite at install time. The speculative
tests are left out, like in run.sh, and the arguments from a test's
.args file go on its runtest line. Each line calls a small wrapper,
openposix_run.sh, which runs the test from its own directory like
run-tests.sh does.

The wrapper also maps PTS_UNSUPPORTED and PTS_UNTESTED to TCONF.
Without it kirk reports them as TWARN and TFAIL.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
---
 testcases/open_posix_testsuite/Makefile       | 13 ++++++++-
 testcases/open_posix_testsuite/bin/Makefile   |  6 +++-
 .../open_posix_testsuite/bin/openposix_run.sh | 29 +++++++++++++++++++
 .../include/mk/config.mk.in                   |  3 ++
 .../scripts/generate-runtest.sh               | 29 +++++++++++++++++++
 5 files changed, 78 insertions(+), 2 deletions(-)
 create mode 100755 testcases/open_posix_testsuite/bin/openposix_run.sh
 create mode 100755 testcases/open_posix_testsuite/scripts/generate-runtest.sh

diff --git a/testcases/open_posix_testsuite/Makefile b/testcases/open_posix_testsuite/Makefile
index 4bead182170d..db141ba3cd8b 100644
--- a/testcases/open_posix_testsuite/Makefile
+++ b/testcases/open_posix_testsuite/Makefile
@@ -74,7 +74,7 @@ generate-makefiles: distclean-makefiles
 		$(top_srcdir)/scripts/generate-makefiles.sh

 .PHONY: install
-install: bin-install conformance-install functional-install stress-install
+install: bin-install conformance-install functional-install stress-install runtest-install

 .PHONY: test
 test: conformance-test functional-test stress-test
@@ -119,6 +119,17 @@ stress-test:
 	@rm -f `if echo "$(LOGFILE)" | grep -q '^/'; then echo "$(LOGFILE)"; else echo "\`pwd\`/$(LOGFILE)"; fi`.$@
 	@$(TEST_MAKE) -C stress test

+RUNTEST_SUITES=		conformance functional stress
+
+.PHONY: runtest-install
+runtest-install: $(addsuffix -install,$(RUNTEST_SUITES))
+	@mkdir -p $(DESTDIR)/$(runtestdir)
+	@set -e; for suite in $(RUNTEST_SUITES); do \
+		$(top_srcdir)/scripts/generate-runtest.sh \
+			$(DESTDIR)/$(testdir) "$(testdir_rel)" $$suite \
+			> $(DESTDIR)/$(runtestdir)/openposix-$$suite; \
+	done
+
 # Tools build and install targets.
 .PHONY: bin-install
 bin-install:
diff --git a/testcases/open_posix_testsuite/bin/Makefile b/testcases/open_posix_testsuite/bin/Makefile
index 262a6567c5b6..5dfff99effcb 100644
--- a/testcases/open_posix_testsuite/bin/Makefile
+++ b/testcases/open_posix_testsuite/bin/Makefile
@@ -16,7 +16,7 @@ clean:
 	@rm -f t0.val

 .PHONY: install
-install: clean $(DESTDIR)/$(bindir) $(DESTDIR)/$(testdir_bin)
+install: clean $(DESTDIR)/$(bindir) $(DESTDIR)/$(testdir_bin) $(DESTDIR)/$(ltp_bindir)
 	set -e; for file in $(INSTALL_BIN_TARGETS); do           \
 		install -m 00755 $$file $(DESTDIR)/$(bindir)/$$file; \
 	done
@@ -27,9 +27,13 @@ install: clean $(DESTDIR)/$(bindir) $(DESTDIR)/$(testdir_bin)
 		install -m 00755 $$file $(DESTDIR)/$(testdir_bin)/$$file; \
 	done

+	install -m 00755 openposix_run.sh $(DESTDIR)/$(ltp_bindir)/

 $(DESTDIR)/$(bindir):
 	mkdir -p $@

 $(DESTDIR)/$(testdir_bin):
 	mkdir -p $@
+
+$(DESTDIR)/$(ltp_bindir):
+	mkdir -p $@
diff --git a/testcases/open_posix_testsuite/bin/openposix_run.sh b/testcases/open_posix_testsuite/bin/openposix_run.sh
new file mode 100755
index 000000000000..43c52a38d956
--- /dev/null
+++ b/testcases/open_posix_testsuite/bin/openposix_run.sh
@@ -0,0 +1,29 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2026 Linaro Limited
+
+LTPROOT=${LTPROOT:-$(cd "$(dirname "$0")/../.." && pwd)}
+
+if [ $# -lt 1 ]; then
+	echo "Usage: $(basename "$0") <test path relative to LTPROOT> [args]" >&2
+	exit 2
+fi
+
+test_path=$LTPROOT/$1
+test_file=${test_path##*/}
+shift
+
+if [ ! -x "$test_path" ]; then
+	echo "$test_path is missing or not executable" >&2
+	exit 2
+fi
+
+cd "${test_path%/*}" || exit 2
+
+./"$test_file" "$@"
+ret=$?
+
+case $ret in
+4|5) exit 32 ;;
+*) exit $ret ;;
+esac
diff --git a/testcases/open_posix_testsuite/include/mk/config.mk.in b/testcases/open_posix_testsuite/include/mk/config.mk.in
index 20422e0f1047..fae80116bbf7 100644
--- a/testcases/open_posix_testsuite/include/mk/config.mk.in
+++ b/testcases/open_posix_testsuite/include/mk/config.mk.in
@@ -14,4 +14,7 @@ testdir_rel := @testdir@
 testdir     := ${prefix}/${testdir_rel}
 testdir_bin := ${testdir}/bin

+ltp_bindir  := ${prefix}/testcases/bin
+runtestdir  := ${prefix}/runtest
+
 CFLAGS  += -std=c99 -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=700 -Wall -Wextra
diff --git a/testcases/open_posix_testsuite/scripts/generate-runtest.sh b/testcases/open_posix_testsuite/scripts/generate-runtest.sh
new file mode 100755
index 000000000000..e561a7adcf21
--- /dev/null
+++ b/testcases/open_posix_testsuite/scripts/generate-runtest.sh
@@ -0,0 +1,29 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2026 Linaro Limited
+
+if [ $# -ne 3 ]; then
+	echo "Usage: $(basename "$0") <installed testdir> <testdir relative to LTPROOT> <suite>" >&2
+	exit 1
+fi
+
+installed_testdir=$1
+testdir_rel=$2
+suite=$3
+
+cd "$installed_testdir" || exit 1
+
+if [ ! -d "$suite" ]; then
+	echo "$installed_testdir/$suite does not exist" >&2
+	exit 1
+fi
+
+find "$suite" -name speculative -prune -o -type f \
+	\( -name '*.run-test' -o -name '*.sh' ! -name run.sh \) -print | sort | \
+while read -r test_path; do
+	test_name=${test_path##*/}
+	args_file=${test_path%/*}/${test_name%%.*}.args
+	args=
+	[ -f "$args_file" ] && read -r args < "$args_file"
+	echo "${test_name%.*} openposix_run.sh ${testdir_rel:+$testdir_rel/}$test_path${args:+ $args}"
+done
--
2.53.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp