[LTP] [PATCH v7 0/4] Automatically generate syscalls.h

[LTP] [PATCH v7 0/4] Automatically generate syscalls.h

[Andrea Cervesato]

The problem we have at the moment is that all syscalls input files must
be changed by hand each time we need to test a new syscall. The idea is
to simplify this process, providing a script that is able to read from
kernel sources and to generate all syscalls files we need.

This patch set adds a new command inside the syscalls folder and
it refactors the previous regen.sh code.
The new way we have to generate the syscalls.h file is the following:

	include/lapi/syscalls/generate_arch.sh 		path/to/kernel/source
	include/lapi/syscalls/generate_syscalls.sh 	path/to/syscalls.h

Scripts are independent and they can be run separately.
generate_syscalls.sh is the equivalent of regen.sh, but its code has
been cleaned up.

Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
Changes in v7:
- check for errors in generate_syscalls.sh
- add documentation
- validation: https://github.com/acerv/ltp/actions/runs/11611080151
- Link to v6: https://lore.kernel.org/r/20241031-generate_syscalls-v6-0-1ad86a33ce2d@suse.com

Changes in v6:
- aesthetic changes in the syscall generator
- fix fallback definitions in the syscalls.h file generator 
- Link to v5: https://lore.kernel.org/r/20241030-generate_syscalls-v5-0-eed6a87dc787@suse.com

Changes in v5:
- license SPDX inside scripts files
- quote shell variables inside scripts
- some generic code cleanup
- Link to v4: https://lore.kernel.org/r/20241009-generate_syscalls-v4-0-5328a785bbad@suse.com

Changes in v4:
- remove blacklist file
- update syscalls files
- Link to v3: https://lore.kernel.org/r/20240930-generate_syscalls-v3-0-e56ede4bc7f5@suse.com

Changes in v3:
- fix generate_arch.sh to work properly for all architectures
- rename some arch .in files and handle renaming inside
  generate_syscalls.sh using the right gcc directives
- Link to v2: https://lore.kernel.org/r/20240927-generate_syscalls-v2-0-108f5f2ad318@suse.com

Changes in v2:
- refactor regen.sh
- split syscalls.h and syscalls .in files creation
- updated syscalls .in files
- Link to v1: https://lore.kernel.org/r/20240924-generate_syscalls-v1-0-941507a9cdac@suse.com

---
Andrea Cervesato (4):
      Refactor regen.sh script to generate syscalls
      Add script to generate arch(s) dependant syscalls
      Delete obsolete strip_syscall.awk file
      Add documentation about syscalls.h generator

 configure.ac                                       |   2 +-
 doc/developers/test_case_tutorial.rst              |  27 +--
 include/lapi/syscalls/{aarch64.in => arm64.in}     |   0
 include/lapi/syscalls/generate_arch.sh             | 211 +++++++++++++++++++++
 include/lapi/syscalls/generate_syscalls.sh         | 109 +++++++++++
 .../lapi/syscalls/{loongarch.in => loongarch64.in} |   0
 include/lapi/syscalls/{mips_n64.in => mips64.in}   |   0
 .../lapi/syscalls/{mips_n32.in => mips64n32.in}    |   0
 include/lapi/syscalls/{mips_o32.in => mipso32.in}  |   0
 include/lapi/syscalls/{hppa.in => parisc.in}       |   0
 include/lapi/syscalls/regen.sh                     | 129 -------------
 include/lapi/syscalls/strip_syscall.awk            |  19 --
 .../lapi/syscalls/{order => supported-arch.txt}    |  12 +-
 13 files changed, 341 insertions(+), 168 deletions(-)
---
base-commit: 84e77afb3acc4bc69c434c430cc1c1d000427921
change-id: 20240923-generate_syscalls-780dc2227bdb

Best regards,
-- 
Andrea Cervesato <andrea.cervesato@suse.com>


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v7 1/4] Refactor regen.sh script to generate syscalls

[Andrea Cervesato]

From: Andrea Cervesato <andrea.cervesato@suse.com>

Rename regen.sh into a more meaningful generate_syscalls.sh name, rename
order into a more meaningful supported-syscalls.txt name and rewrite
part of the regen.sh script code in order to execute it from anywhere in
the filesystem, without need to be in its own folder. The new code is
also more clear and concise, using native sh features which are
simplifying the code.

Reviewed-by: Li Wang <liwang@redhat.com>
Reviewed-by: Petr Vorel <pvorel@suse.cz>
Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
 configure.ac                                       |   2 +-
 include/lapi/syscalls/generate_syscalls.sh         | 109 +++++++++++++++++
 include/lapi/syscalls/regen.sh                     | 129 ---------------------
 .../lapi/syscalls/{order => supported-arch.txt}    |   0
 4 files changed, 110 insertions(+), 130 deletions(-)

diff --git a/configure.ac b/configure.ac
index d327974efa71f263d7f7f5aec9d2c5831d53dd0e..cd1233d19fad376973fc880d6689859845613fb0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -386,7 +386,7 @@ else
     AC_SUBST([WITH_REALTIME_TESTSUITE],["no"])
 fi
 
-AC_CONFIG_COMMANDS([syscalls.h], [cd ${ac_top_srcdir}/include/lapi/syscalls; ./regen.sh; cd - >/dev/null])
+AC_CONFIG_COMMANDS([syscalls.h], [cd ${ac_top_srcdir}/include/lapi/syscalls; ./generate_syscalls.sh ../syscalls.h; cd - >/dev/null])
 
 # custom functions
 # NOTE: don't create custom functions for simple checks, put them into this file
diff --git a/include/lapi/syscalls/generate_syscalls.sh b/include/lapi/syscalls/generate_syscalls.sh
new file mode 100755
index 0000000000000000000000000000000000000000..e94a6ea3c1f1623f0f846c724c8c5ff2b95450af
--- /dev/null
+++ b/include/lapi/syscalls/generate_syscalls.sh
@@ -0,0 +1,109 @@
+#!/bin/sh -eux
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Generate the syscalls.h file, merging all architectures syscalls input file
+# which are in the current folder and defined inside supported-arch.txt file.
+
+SYSCALLS_FILE="$1"
+
+if [ -z "${SYSCALLS_FILE}" ]; then
+	echo "Please provide the syscalls.h directory:"
+	echo ""
+	echo "$0 path/of/syscalls.h"
+	echo ""
+	exit 1
+fi
+
+SCRIPT_DIR="$(realpath $(dirname "$0"))"
+SUPPORTED_ARCH="${SCRIPT_DIR}/supported-arch.txt"
+
+echo '// SPDX-License-Identifier: GPL-2.0-or-later
+/************************************************
+ * GENERATED FILE: DO NOT EDIT/PATCH THIS FILE  *
+ *  change your arch specific .in file instead  *
+ ************************************************/
+
+/*
+ * Here we stick all the ugly *fallback* logic for linux
+ * system call numbers (those __NR_ thingies).
+ */
+
+#ifndef LAPI_SYSCALLS_H__
+#define LAPI_SYSCALLS_H__
+
+#include <errno.h>
+#include <sys/syscall.h>
+#include <asm/unistd.h>
+
+#ifdef TST_TEST_H__
+#define TST_SYSCALL_BRK__(NR, SNR) ({ \
+tst_brk(TCONF, \
+	"syscall(%d) " SNR " not supported on your arch", NR); \
+})
+#else
+inline static void dummy_cleanup(void) {}
+
+#define TST_SYSCALL_BRK__(NR, SNR) ({ \
+tst_brkm(TCONF, dummy_cleanup, \
+	"syscall(%d) " SNR " not supported on your arch", NR); \
+})
+#endif
+
+#define tst_syscall(NR, ...) ({ \
+intptr_t tst_ret; \
+if (NR == __LTP__NR_INVALID_SYSCALL) { \
+	errno = ENOSYS; \
+	tst_ret = -1; \
+} else { \
+	tst_ret = syscall(NR, ##__VA_ARGS__); \
+} \
+if (tst_ret == -1 && errno == ENOSYS) { \
+	TST_SYSCALL_BRK__(NR, #NR); \
+} \
+tst_ret; \
+})
+
+#define __LTP__NR_INVALID_SYSCALL -1' >${SYSCALLS_FILE}
+
+while IFS= read -r arch; do
+	(
+		echo
+		case ${arch} in
+		sparc64) echo "#if defined(__sparc__) && defined(__arch64__)" ;;
+		sparc) echo "#if defined(__sparc__) && !defined(__arch64__)" ;;
+		s390) echo "#if defined(__s390__) && !defined(__s390x__)" ;;
+		mips64n32) echo "#if defined(__mips__) && defined(_ABIN32)" ;;
+		mips64) echo "#if defined(__mips__) && defined(_ABI64)" ;;
+		mipso32) echo "#if defined(__mips__) && defined(_ABIO32) && _MIPS_SZLONG == 32" ;;
+		parisc) echo "#ifdef __hppa__" ;;
+		loongarch64) echo "#ifdef __loongarch__" ;;
+		arm64) echo "#ifdef __aarch64__" ;;
+		*) echo "#ifdef __${arch}__" ;;
+		esac
+
+		while read -r line; do
+			set -- ${line}
+			syscall_nr="__NR_$1"
+			shift
+
+			echo "# ifndef ${syscall_nr}"
+			echo "#  define ${syscall_nr} $*"
+			echo "# endif"
+		done <"${SCRIPT_DIR}/${arch}.in"
+		echo "#endif"
+		echo
+	) >>${SYSCALLS_FILE}
+done <${SUPPORTED_ARCH}
+
+(
+	echo
+	echo "/* Common stubs */"
+	for num in $(awk '{print $1}' "${SCRIPT_DIR}/"*.in | sort -u); do
+		syscall_nr="__NR_${num}"
+
+		echo "# ifndef ${syscall_nr}"
+		echo "#  define ${syscall_nr} __LTP__NR_INVALID_SYSCALL"
+		echo "# endif"
+	done
+	echo "#endif"
+) >>${SYSCALLS_FILE}
diff --git a/include/lapi/syscalls/regen.sh b/include/lapi/syscalls/regen.sh
deleted file mode 100755
index 663ce4458bbc67c3a9e8073dfe8359164a9da0ee..0000000000000000000000000000000000000000
--- a/include/lapi/syscalls/regen.sh
+++ /dev/null
@@ -1,129 +0,0 @@
-#!/bin/sh
-
-output="syscalls.h"
-rm -f "${output}".[1-9]*
-output_pid="${output}.$$"
-
-max_jobs=$(getconf _NPROCESSORS_ONLN 2>/dev/null)
-: ${max_jobs:=1}
-
-srcdir=${0%/*}
-
-err() {
-	echo "$*" 1>&2
-	exit 1
-}
-
-cat << EOF > "${output_pid}"
-/************************************************
- * GENERATED FILE: DO NOT EDIT/PATCH THIS FILE  *
- *  change your arch specific .in file instead  *
- ************************************************/
-
-/*
- * Here we stick all the ugly *fallback* logic for linux
- * system call numbers (those __NR_ thingies).
- *
- * Licensed under the GPLv2 or later, see the COPYING file.
- */
-
-#ifndef LAPI_SYSCALLS_H__
-#define LAPI_SYSCALLS_H__
-
-#include <errno.h>
-#include <sys/syscall.h>
-#include <asm/unistd.h>
-
-#ifdef TST_TEST_H__
-#define TST_SYSCALL_BRK__(NR, SNR) ({ \\
-	tst_brk(TCONF, \\
-		"syscall(%d) " SNR " not supported on your arch", NR); \\
-})
-#else
-inline static void dummy_cleanup(void) {}
-
-#define TST_SYSCALL_BRK__(NR, SNR) ({ \\
-	tst_brkm(TCONF, dummy_cleanup, \\
-		"syscall(%d) " SNR " not supported on your arch", NR); \\
-})
-#endif
-
-#define tst_syscall(NR, ...) ({ \\
-	intptr_t tst_ret; \\
-	if (NR == __LTP__NR_INVALID_SYSCALL) { \\
-		errno = ENOSYS; \\
-		tst_ret = -1; \\
-	} else { \\
-		tst_ret = syscall(NR, ##__VA_ARGS__); \\
-	} \\
-	if (tst_ret == -1 && errno == ENOSYS) { \\
-		TST_SYSCALL_BRK__(NR, #NR); \\
-	} \\
-	tst_ret; \\
-})
-
-EOF
-
-jobs=0
-for arch in $(cat "${srcdir}/order") ; do
-	(
-	echo "Generating data for arch $arch ... "
-
-	(
-	echo
-	case ${arch} in
-		sparc64) echo "#if defined(__sparc__) && defined(__arch64__)" ;;
-		sparc) echo "#if defined(__sparc__) && !defined(__arch64__)" ;;
-		s390) echo "#if defined(__s390__) && !defined(__s390x__)" ;;
-		mips_n32) echo "#if defined(__mips__) && defined(_ABIN32)" ;;
-		mips_n64) echo "#if defined(__mips__) && defined(_ABI64)" ;;
-		mips_o32) echo "#if defined(__mips__) && defined(_ABIO32) && _MIPS_SZLONG == 32" ;;
-		*) echo "#ifdef __${arch}__" ;;
-	esac
-	while read line ; do
-		set -- ${line}
-		nr="__NR_$1"
-		shift
-		if [ $# -eq 0 ] ; then
-			err "invalid line found: $line"
-		fi
-		echo "# ifndef ${nr}"
-		echo "#  define ${nr} $*"
-		echo "# endif"
-	done < "${srcdir}/${arch}.in"
-	echo "#endif"
-	echo
-	) >> "${output_pid}.${arch}"
-
-	) &
-
-	jobs=$(( jobs + 1 ))
-	if [ ${jobs} -ge ${max_jobs} ] ; then
-		wait || exit 1
-		jobs=0
-	fi
-done
-
-echo "Generating stub list ... "
-(
-echo
-echo "/* Common stubs */"
-echo "#define __LTP__NR_INVALID_SYSCALL -1" >> "${output_pid}"
-for nr in $(awk '{print $1}' "${srcdir}/"*.in | sort -u) ; do
-	nr="__NR_${nr}"
-	echo "# ifndef ${nr}"
-	echo "#  define ${nr} __LTP__NR_INVALID_SYSCALL"
-	echo "# endif"
-done
-echo "#endif"
-) >> "${output_pid}._footer"
-
-wait || exit 1
-
-printf "Combining them all ... "
-for arch in $(cat "${srcdir}/order") _footer ; do
-	cat "${output_pid}.${arch}"
-done >> "${output_pid}"
-mv "${output_pid}" "../${output}"
-rm -f "${output_pid}"*
-echo "OK!"
diff --git a/include/lapi/syscalls/order b/include/lapi/syscalls/supported-arch.txt
similarity index 100%
rename from include/lapi/syscalls/order
rename to include/lapi/syscalls/supported-arch.txt

-- 
2.43.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v7 2/4] Add script to generate arch(s) dependant syscalls

[Andrea Cervesato]

From: Andrea Cervesato <andrea.cervesato@suse.com>

Add generate_arch.sh script which can be used to generate arch(s)
dependant syscalls file. The way it works is pretty simple: for each
architecture defined into supported-arch.txt, compile kernel headers,
extract the list of syscalls and generate a .in file containing all of
them, associated with their own syscall's number.
The way syscalls files are generated, passes through a C application
which is automatically checking the availability of the syscalls in
the user space environment.

Reviewed-by: Li Wang <liwang@redhat.com>
Reviewed-by: Cyril Hrubis <chrubis@suse.cz>
Reviewed-by: Petr Vorel <pvorel@suse.cz>
Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
 include/lapi/syscalls/{aarch64.in => arm64.in}     |   0
 include/lapi/syscalls/generate_arch.sh             | 211 +++++++++++++++++++++
 .../lapi/syscalls/{loongarch.in => loongarch64.in} |   0
 include/lapi/syscalls/{mips_n64.in => mips64.in}   |   0
 .../lapi/syscalls/{mips_n32.in => mips64n32.in}    |   0
 include/lapi/syscalls/{mips_o32.in => mipso32.in}  |   0
 include/lapi/syscalls/{hppa.in => parisc.in}       |   0
 include/lapi/syscalls/supported-arch.txt           |  12 +-
 8 files changed, 217 insertions(+), 6 deletions(-)

diff --git a/include/lapi/syscalls/aarch64.in b/include/lapi/syscalls/arm64.in
similarity index 100%
rename from include/lapi/syscalls/aarch64.in
rename to include/lapi/syscalls/arm64.in
diff --git a/include/lapi/syscalls/generate_arch.sh b/include/lapi/syscalls/generate_arch.sh
new file mode 100755
index 0000000000000000000000000000000000000000..c88ca945127c3c26162a48babcd6b0f8ca552311
--- /dev/null
+++ b/include/lapi/syscalls/generate_arch.sh
@@ -0,0 +1,211 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# This is an adaptation of the update-tables.sh script, included in the
+# syscalls-table project (https://github.com/hrw/syscalls-table) and released
+# under the MIT license.
+#
+# Author: Andrea Cervesato <andrea.cervesato@suse.com>
+
+KERNELSRC="$1"
+
+# to keep sorting in order
+export LC_ALL=C
+
+if [ -z "$KERNELSRC" ]; then
+	echo "Please provide kernel sources:"
+	echo ""
+	echo "$0 path/to/Linux/kernel/sources"
+	echo ""
+	exit 1
+fi
+
+if [ ! -d "$KERNELSRC" ]; then
+	echo "$KERNELSRC is not a directory"
+	exit 1
+fi
+
+if [ ! -e "${KERNELSRC}/Makefile" ]; then
+	echo "No Makefile in $KERNELSRC directory"
+	exit 1
+fi
+
+TEMP="$(mktemp -d)"
+KVER="$(make -C ${KERNELSRC} kernelversion -s)"
+
+SCRIPT_DIR="$(realpath $(dirname "$0"))"
+SUPPORTED_ARCH="${SCRIPT_DIR}/supported-arch.txt"
+LINUX_HEADERS="${TEMP}/headers"
+
+grab_syscall_names_from_tables()
+{
+	for tbl_file in $(find ${KERNELSRC}/arch -name syscall*.tbl); do
+		grep -E -v "(^#|^$|sys_ni_syscall)" $tbl_file |
+			awk '{ print $3 }' >> ${TEMP}/syscall-names.tosort
+	done
+
+	drop_bad_entries
+}
+
+grab_syscall_names_from_unistd_h()
+{
+	grep -E -h "^#define __NR_" \
+		${LINUX_HEADERS}/usr/include/asm/unistd*.h \
+		${LINUX_HEADERS}/usr/include/asm-generic/unistd.h \
+		>${TEMP}/syscall-names.tosort
+
+	drop_bad_entries
+}
+
+drop_bad_entries()
+{
+	grep -E -v "(unistd.h|NR3264|__NR_syscall|__SC_COMP|__NR_.*Linux|__NR_FAST)" \
+		${TEMP}/syscall-names.tosort |
+		grep -E -v "(__SYSCALL|SYSCALL_BASE|SYSCALL_MASK)" |
+		sed -e "s/#define\s*__NR_//g" -e "s/\s.*//g" |
+		sort -u >${TEMP}/syscall-names.txt
+}
+
+generate_table()
+{
+	echo "- $arch"
+
+	if [ $bits == 32 ]; then
+		extraflags="${extraflags} -D__BITS_PER_LONG=32"
+	fi
+
+	local uppercase_arch=$(echo "$arch" | tr '[:lower:]' '[:upper:]')
+
+	gcc ${TEMP}/list-syscalls.c -U__LP64__ -U__ILP32__ -U__i386__ \
+		-D${uppercase_arch} \
+		-D__${arch}__ ${extraflags} \
+		-I ${LINUX_HEADERS}/usr/include/ \
+		-o ${TEMP}/list-syscalls &>/dev/null
+
+	${TEMP}/list-syscalls > "${TEMP}/${arch}.in.tosort"
+
+	sort -k2,2n "${TEMP}/${arch}.in.tosort" > "${TEMP}/${arch}.in"
+}
+
+generate_list_syscalls_c()
+{
+	(
+		echo
+		echo "
+		#include <stdio.h>
+		#include <asm/unistd.h>
+
+		int main(void)
+		{
+		"
+		for syscall in $(cat ${TEMP}/syscall-names.txt); do
+			echo "
+		#ifdef __NR_$syscall
+			printf(\"$syscall %d\\n\", __NR_$syscall);
+		#endif
+		"
+		done
+		echo " return 0;
+		}"
+	) > ${TEMP}/list-syscalls.c
+}
+
+export_headers()
+{
+	make -s -C ${KERNELSRC} ARCH=${arch} O=${LINUX_HEADERS} \
+		headers_install &>/dev/null
+}
+
+do_all_tables()
+{
+	for archdir in ${KERNELSRC}/arch/*; do
+		arch=$(basename $archdir)
+
+		bits=64
+		extraflags=
+
+		case ${arch} in
+		Kconfig)
+			continue
+			;;
+		um)
+			continue
+			;;
+		esac
+
+		export_headers
+		grab_syscall_names_from_unistd_h
+
+		case ${arch} in
+		arm)
+			bits=32
+			arch=armoabi extraflags= generate_table
+			arch=arm extraflags=-D__ARM_EABI__ generate_table
+			;;
+		loongarch)
+			# 32-bit variant of loongarch may appear
+			arch=loongarch64 extraflags=-D_LOONGARCH_SZLONG=64 generate_table
+			;;
+		mips)
+			arch=mips64 extraflags=-D_MIPS_SIM=_MIPS_SIM_ABI64 generate_table
+			bits=32
+			arch=mipso32 extraflags=-D_MIPS_SIM=_MIPS_SIM_ABI32 generate_table
+			arch=mips64n32 extraflags=-D_MIPS_SIM=_MIPS_SIM_NABI32 generate_table
+			;;
+		powerpc)
+			generate_table
+			arch=powerpc64 generate_table
+			;;
+		riscv)
+			arch=riscv64 extraflags=-D__LP64__ generate_table
+			bits=32
+			arch=riscv32 extraflags=-D__SIZEOF_POINTER__=4 generate_table
+			;;
+		s390)
+			bits=32
+			generate_table
+			bits=64
+			arch=s390x generate_table
+			;;
+		sparc)
+			bits=32
+			extraflags=-D__32bit_syscall_numbers__ generate_table
+			bits=64
+			arch=sparc64 extraflags=-D__arch64__ generate_table
+			;;
+		x86)
+			arch=x86_64 extraflags=-D__LP64__ generate_table
+			bits=32
+			arch=i386 generate_table
+			arch=x32 extraflags=-D__ILP32__ generate_table
+			;;
+		arc | csky | hexagon | m68k | microblaze | nios2 | openrisc | sh | xtensa)
+			bits=32 generate_table
+			;;
+		*)
+			generate_table
+			;;
+		esac
+	done
+}
+
+copy_supported_arch()
+{
+	while IFS= read -r arch; do
+		if [ -f "${TEMP}/${arch}.in" ]; then
+			echo "- ${arch}"
+			cp "${TEMP}/${arch}.in" "${SCRIPT_DIR}/${arch}.in"
+		fi
+	done < ${SUPPORTED_ARCH}
+}
+
+echo "Temporary directory ${TEMP}"
+echo "Extracting syscalls"
+
+grab_syscall_names_from_tables
+generate_list_syscalls_c
+
+do_all_tables
+
+echo "Copying supported syscalls"
+copy_supported_arch
diff --git a/include/lapi/syscalls/loongarch.in b/include/lapi/syscalls/loongarch64.in
similarity index 100%
rename from include/lapi/syscalls/loongarch.in
rename to include/lapi/syscalls/loongarch64.in
diff --git a/include/lapi/syscalls/mips_n64.in b/include/lapi/syscalls/mips64.in
similarity index 100%
rename from include/lapi/syscalls/mips_n64.in
rename to include/lapi/syscalls/mips64.in
diff --git a/include/lapi/syscalls/mips_n32.in b/include/lapi/syscalls/mips64n32.in
similarity index 100%
rename from include/lapi/syscalls/mips_n32.in
rename to include/lapi/syscalls/mips64n32.in
diff --git a/include/lapi/syscalls/mips_o32.in b/include/lapi/syscalls/mipso32.in
similarity index 100%
rename from include/lapi/syscalls/mips_o32.in
rename to include/lapi/syscalls/mipso32.in
diff --git a/include/lapi/syscalls/hppa.in b/include/lapi/syscalls/parisc.in
similarity index 100%
rename from include/lapi/syscalls/hppa.in
rename to include/lapi/syscalls/parisc.in
diff --git a/include/lapi/syscalls/supported-arch.txt b/include/lapi/syscalls/supported-arch.txt
index c18aa38cf4546cdf3ac8c89a45bd1b202ffa7711..c5c5191ac08482d89f0a8c39dfae936538f58f7f 100644
--- a/include/lapi/syscalls/supported-arch.txt
+++ b/include/lapi/syscalls/supported-arch.txt
@@ -1,13 +1,13 @@
-aarch64
 arc
+arm64
 arm
-hppa
 i386
 ia64
-loongarch
-mips_n32
-mips_n64
-mips_o32
+loongarch64
+mips64n32
+mips64
+mipso32
+parisc
 powerpc64
 powerpc
 s390x

-- 
2.43.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v7 3/4] Delete obsolete strip_syscall.awk file

[Andrea Cervesato]

From: Andrea Cervesato <andrea.cervesato@suse.com>

Reviewed-by: Li Wang <liwang@redhat.com>
Reviewed-by: Petr Vorel <pvorel@suse.cz>
Reviewed-by: Cyril Hrubis <chrubis@suse.cz>
Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
 include/lapi/syscalls/strip_syscall.awk | 19 -------------------
 1 file changed, 19 deletions(-)

diff --git a/include/lapi/syscalls/strip_syscall.awk b/include/lapi/syscalls/strip_syscall.awk
deleted file mode 100755
index e8dff422e2667745c144ed984a4d66461fcce0c6..0000000000000000000000000000000000000000
--- a/include/lapi/syscalls/strip_syscall.awk
+++ /dev/null
@@ -1,19 +0,0 @@
-#!/usr/bin/awk -f
-#
-# Dumb script that can be used to strip all of the syscall information from
-# the arch-respective unistd*.h.
-#
-# Examples:
-#
-# 1. Grab the i386 32-bit syscalls from unistd_32.h and put them in i386.in
-# strip_syscall.awk arch/x86/include/asm/unistd_32.h > i386.in
-#
-
-/^#define[[:space:]]+__NR_[0-9a-z]+/ {
-
-	sub (/#define[[:space:]]+__NR_/, "", $0);
-	sub (/[[:space:]]*(\/\*.*)/, "", $0);
-	sub (/[[:space:]]+/, " ", $0);
-
-	print
-}

-- 
2.43.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v7 4/4] Add documentation about syscalls.h generator

[Andrea Cervesato]

From: Andrea Cervesato <andrea.cervesato@suse.com>

Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
 doc/developers/test_case_tutorial.rst | 27 ++++++++++++++-------------
 1 file changed, 14 insertions(+), 13 deletions(-)

diff --git a/doc/developers/test_case_tutorial.rst b/doc/developers/test_case_tutorial.rst
index be9a0ea8bac1d690b4ed4879d110f87c3f2a381a..b48ea4a7a3594266466fcca364f5d0ff1e42d235 100644
--- a/doc/developers/test_case_tutorial.rst
+++ b/doc/developers/test_case_tutorial.rst
@@ -302,24 +302,25 @@ for a distribution's C library version to be older than its kernel or it may use
 cut down C library in comparison to the GNU one. So we must call ``statx()``
 using the general ``syscall()`` interface.
 
-The LTP contains a library for dealing with the ``syscall`` interface, which is
+LTP contains a library for dealing with the ``syscall`` interface, which is
 located in :master:`include/lapi`. System call numbers are listed against the relevant
 call in the ``*.in`` files (e.g. ``x86_64.in``) which are used to generate
-``syscalls.h``, which is the header you should include. On rare occasions you
-may find the system call number is missing from the ``*.in`` files and will need
-to add it (see :master:`include/lapi/syscalls/strip_syscall.awk`).
+``syscalls.h``, the header you should include.
 
-System call numbers vary between architectures, hence there are multiple
-``*.in`` files for each architecture. You can find the various values for the
-``statx`` system call across a number of ``unistd.h`` files in the Linux kernel.
+On rare occasions, you may find that system call number is missing from ``*.in``
+files. In these cases, they will need to be updated using
+`include/lapi/syscalls/generate_arch.sh` script as following:
 
-Note that we don't use the system-call-identifier value available in
-``/usr/include/linux/uinstd.h`` because the kernel might be much newer than the
-user land development packages.
+.. code-block:: bash
+
+    $ include/lapi/syscalls/generate_arch.sh /path/of/linux/sources
+
+The script will generate all the needed ``*.in`` files according to the Linux
+source code which has been used. Make sure that your Linux source code has
+been updated to the latest version.
 
-For ``statx`` we had to add ``statx 332`` to :master:`include/lapi/syscalls/x86_64.in`,
-``statx 383`` to :master:`include/lapi/syscalls/powerpc.in`, etc.  Now lets look at
-the code, which I will explain in more detail further down.
+Once the new syscalls files have been updated, to rebuild our ``syscalls.h``
+file, please use ``./configure`` command.
 
 .. code-block:: c
 

-- 
2.43.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v7 1/4] Refactor regen.sh script to generate syscalls

[Petr Vorel]

Hi Andrea,

> --- /dev/null
> +++ b/include/lapi/syscalls/generate_syscalls.sh
> @@ -0,0 +1,109 @@
> +#!/bin/sh -eux

I'm very sorry, I asked for "#!/bin/sh -eux", but better would be without "x".
"x" prints output of the commands which is too verbose on generate_syscalls.sh.
Could you please, when you merge, change it to "#!/bin/sh -eu"?
I'm not sure if it should stay in the other script.
This is on dash, but I suppose this can be reproducible on bash as well.

$ ./configure
...
+ syscall_nr=__NR_writev
+ echo # ifndef __NR_writev
+ echo #  define __NR_writev __LTP__NR_INVALID_SYSCALL
+ echo # endif
+ echo #endif

$ ./configure > /tmp/out 2> /tmp/err; wc -l /tmp/out /tmp/err

on master:
  290 /tmp/out
    3 /tmp/err
  293 total

on v7 (at least what I apply from https://patchwork.ozlabs.org/series/430633/mbox/):
    289 /tmp/out
  48706 /tmp/err
  48995 total

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v7 1/4] Refactor regen.sh script to generate syscalls

[Cyril Hrubis]

Hi!
> $ ./configure
> ...
> + syscall_nr=__NR_writev
> + echo # ifndef __NR_writev
> + echo #  define __NR_writev __LTP__NR_INVALID_SYSCALL
> + echo # endif
> + echo #endif
> 
> $ ./configure > /tmp/out 2> /tmp/err; wc -l /tmp/out /tmp/err
> 
> on master:
>   290 /tmp/out
>     3 /tmp/err
>   293 total
> 
> on v7 (at least what I apply from https://patchwork.ozlabs.org/series/430633/mbox/):
>     289 /tmp/out
>   48706 /tmp/err
>   48995 total

The /tmp/out now looks fine, there is a single line difference, that is
the empty newline at the start of the file I asked to remove.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v7 1/4] Refactor regen.sh script to generate syscalls

[Andrea Cervesato via ltp]

Hi Petr,

On 10/31/24 14:01, Petr Vorel wrote:
> Hi Andrea,
>
>> --- /dev/null
>> +++ b/include/lapi/syscalls/generate_syscalls.sh
>> @@ -0,0 +1,109 @@
>> +#!/bin/sh -eux
> I'm very sorry, I asked for "#!/bin/sh -eux", but better would be without "x".
> "x" prints output of the commands which is too verbose on generate_syscalls.sh.
> Could you please, when you merge, change it to "#!/bin/sh -eu"?
No problem, I will do it before merge.
> I'm not sure if it should stay in the other script.
> This is on dash, but I suppose this can be reproducible on bash as well.
>
> $ ./configure
> ...
> + syscall_nr=__NR_writev
> + echo # ifndef __NR_writev
> + echo #  define __NR_writev __LTP__NR_INVALID_SYSCALL
> + echo # endif
> + echo #endif
>
> $ ./configure > /tmp/out 2> /tmp/err; wc -l /tmp/out /tmp/err

Did you check the CI result I mentioned in the cover letter?

Andrea

>
> on master:
>    290 /tmp/out
>      3 /tmp/err
>    293 total
>
> on v7 (at least what I apply from https://patchwork.ozlabs.org/series/430633/mbox/):
>      289 /tmp/out
>    48706 /tmp/err
>    48995 total
>
> Kind regards,
> Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v7 1/4] Refactor regen.sh script to generate syscalls

[Petr Vorel]

> Hi!
> > $ ./configure
> > ...
> > + syscall_nr=__NR_writev
> > + echo # ifndef __NR_writev
> > + echo #  define __NR_writev __LTP__NR_INVALID_SYSCALL
> > + echo # endif
> > + echo #endif

> > $ ./configure > /tmp/out 2> /tmp/err; wc -l /tmp/out /tmp/err

> > on master:
> >   290 /tmp/out
> >     3 /tmp/err
> >   293 total

> > on v7 (at least what I apply from https://patchwork.ozlabs.org/series/430633/mbox/):
> >     289 /tmp/out
> >   48706 /tmp/err
> >   48995 total

> The /tmp/out now looks fine, there is a single line difference, that is
> the empty newline at the start of the file I asked to remove.

OK, it can be something on the setup of the VM I'm using (some Debian), which
uses dash as /bin/sh. I was not able to reproduce it on openSUSE Tumbleweed.
Feel free to ignore it.

But I have other error for generate_arch.sh which I'll report on the other
patchset.

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v7 2/4] Add script to generate arch(s) dependant syscalls

[Petr Vorel]

Hi Andrea,

> --- /dev/null
> +++ b/include/lapi/syscalls/generate_arch.sh
> @@ -0,0 +1,211 @@
> +#!/bin/sh
Also here would be nice to use -eu (I know it's a script originated from
elsewhere)...

> +# SPDX-License-Identifier: GPL-2.0-or-later
> +#
> +# This is an adaptation of the update-tables.sh script, included in the
> +# syscalls-table project (https://github.com/hrw/syscalls-table) and released
> +# under the MIT license.
> +#
> +# Author: Andrea Cervesato <andrea.cervesato@suse.com>
> +
> +KERNELSRC="$1"

... but that would require instead of this to check like this before using $1:

if [ $# -eq 0 ]
	echo "Please provide kernel sources:"
	echo ""
	echo "$0 path/to/Linux/kernel/sources"
	echo ""
	exit 1
fi

Also running with dash I get failures:

$ ./include/lapi/syscalls/generate_arch.sh ../linux
...
grep: /tmp/tmp.sVfa08a9De/headers/usr/include/asm/unistd*.h: No such file or directory
grep: /tmp/tmp.sVfa08a9De/headers/usr/include/asm-generic/unistd.h: No such file or directory
./include/lapi/syscalls/generate_arch.sh: 73: [: 64: unexpected operator
./include/lapi/syscalls/generate_arch.sh: 85: /tmp/tmp.sVfa08a9De/list-syscalls: not found
grep: /tmp/tmp.sVfa08a9De/headers/usr/include/asm/unistd*.h: No such file or directory
grep: /tmp/tmp.sVfa08a9De/headers/usr/include/asm-generic/unistd.h: No such file or directory
./include/lapi/syscalls/generate_arch.sh: 73: [: 32: unexpected operator
./include/lapi/syscalls/generate_arch.sh: 85: /tmp/tmp.sVfa08a9De/list-syscalls: not found
grep: /tmp/tmp.sVfa08a9De/headers/usr/include/asm/unistd*.h: No such file or directory
grep: /tmp/tmp.sVfa08a9De/headers/usr/include/asm-generic/unistd.h: No such file or directory
./include/lapi/syscalls/generate_arch.sh: 73: [: 32: unexpected operator
/tmp/tmp.sVfa08a9De/list-syscalls.c: In function 'main':
/tmp/tmp.sVfa08a9De/list-syscalls.c:11:32: warning: missing terminating " character
   11 |                         printf("_llseek %d
      |                                ^
/tmp/tmp.sVfa08a9De/list-syscalls.c:12:1: warning: missing terminating " character
   12 | ", __NR__llseek);
      | ^

That fails in generate_table on:
if [ $bits == 32 ]; then

When I use "!/bin/sh -eux" I see:

make -s -C /home/foo/sources/linux ARCH=alpha O=/tmp/tmp.G3HzE6EnFu/headers headers_install
grep: /tmp/tmp.G3HzE6EnFu/headers/usr/include/asm/unistd*.h: No such file or directory

This looks like it requires to have build kernel before running this script,
right? (I have 6.12-rc4 in the kernel directory). As a result of running the
script are empty include/lapi/syscalls/*.in files.

Again, it can be a combination of some problem with my environment (some
variable set or shell setup) + dash as /bin/sh. Changing script to /bin/bash
fixes everything. It would be nice if anybody test it with installed dash and
!#/bin/dash in both scripts.

And successful run changes include/lapi/syscalls/arm.in:

+++ include/lapi/syscalls/arm.in
@@ -1,406 +1,415 @@
-restart_syscall (__NR_SYSCALL_BASE+  0)
-exit (__NR_SYSCALL_BASE+  1)
-fork (__NR_SYSCALL_BASE+  2)
-read (__NR_SYSCALL_BASE+  3)
...
+restart_syscall 0
+exit 1
+fork 2
+read 3
+write 4

Maybe we could update syscalls with generate_arch.sh after we merge this?

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v7 1/4] Refactor regen.sh script to generate syscalls

[Petr Vorel]

Hi Andrea,

> > > --- /dev/null
> > > +++ b/include/lapi/syscalls/generate_syscalls.sh
> > > @@ -0,0 +1,109 @@
> > > +#!/bin/sh -eux
> > I'm very sorry, I asked for "#!/bin/sh -eux", but better would be without "x".
> > "x" prints output of the commands which is too verbose on generate_syscalls.sh.
> > Could you please, when you merge, change it to "#!/bin/sh -eu"?
> No problem, I will do it before merge.

+1

> > I'm not sure if it should stay in the other script.
> > This is on dash, but I suppose this can be reproducible on bash as well.

> > $ ./configure
> > ...
> > + syscall_nr=__NR_writev
> > + echo # ifndef __NR_writev
> > + echo #  define __NR_writev __LTP__NR_INVALID_SYSCALL
> > + echo # endif
> > + echo #endif

> > $ ./configure > /tmp/out 2> /tmp/err; wc -l /tmp/out /tmp/err

> Did you check the CI result I mentioned in the cover letter?

Yes, I did. Sure, it can be a problem with my setup (or it can be a bug in a
script which is visible only on certain setup).

Kind regards,
Petr

> Andrea

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v7 2/4] Add script to generate arch(s) dependant syscalls

[Andrea Cervesato via ltp]

The kernel doesn't need to be compiled. That's done by the script via 
`make headers` command. I don't know why that is happening honestly.

On 10/31/24 14:58, Petr Vorel wrote:
> That fails in generate_table on:
> if [ $bits == 32 ]; then

That's because the right syntax should be (for bash):

if [[ "$bits" == "32" ]]; then

But in other shells sh compatible, probably I should use:

if [ "$bits" -eq "32" ]; then


I'm gonna fix this.

Andrea


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v7 2/4] Add script to generate arch(s) dependant syscalls

[pvorel]

On 2024-10-31 15:52, Andrea Cervesato wrote:
> The kernel doesn't need to be compiled. That's done by the script via
> `make headers` command. I don't know why that is happening honestly.
> 
> On 10/31/24 14:58, Petr Vorel wrote:
>> That fails in generate_table on:
>> if [ $bits == 32 ]; then
> 
> That's because the right syntax should be (for bash):
> 
> if [[ "$bits" == "32" ]]; then

Ah, I should have caught this, I overlooked an obvious bashism. thanks 
for finding the problem.

I actually run checkbashisms before, but only when run with -f finds a 
problems:

$ checkbashisms -f include/lapi/syscalls/generate_arch.sh
possible bashism in include/lapi/syscalls/generate_arch.sh line 73 
(should be 'b = a'):
	if [ $bits == 32 ]; then
possible bashism in include/lapi/syscalls/generate_arch.sh line 83 
(should be >word 2>&1):
	gcc ${TEMP}/list-syscalls.c -U__LP64__ -U__ILP32__ -U__i386__ \
		-D${uppercase_arch} \
		-D__${arch}__ ${extraflags} \
		-I ${LINUX_HEADERS}/usr/include/ \
		-o ${TEMP}/list-syscalls &>/dev/null
possible bashism in include/lapi/syscalls/generate_arch.sh line 116 
(should be >word 2>&1):
	make -s -C ${KERNELSRC} ARCH=${arch} O=${LINUX_HEADERS} \
		headers_install &>/dev/null

"&>" is yet another bashism.

Ideally, not only checking with "checkbashisms -f", but also running 
script with dash (e.g. change shebang to #!/bin/dash) should catch most 
of the problems.


> 
> But in other shells sh compatible, probably I should use:
> 
> if [ "$bits" -eq "32" ]; then

Yes please.

> 
> 
> I'm gonna fix this.

Thanks a lot!

Kind regards,
Petr
> 
> Andrea

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp