* [LTP] [PATCH v8 1/4] Refactor regen.sh script to generate syscalls
2024-10-31 16:01 [LTP] [PATCH v8 0/4] Automatically generate syscalls.h Andrea Cervesato
@ 2024-10-31 16:01 ` Andrea Cervesato
2024-10-31 16:21 ` Cyril Hrubis
2024-10-31 16:01 ` [LTP] [PATCH v8 2/4] Add script to generate arch(s) dependant syscalls Andrea Cervesato
` (2 subsequent siblings)
3 siblings, 1 reply; 11+ messages in thread
From: Andrea Cervesato @ 2024-10-31 16:01 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
Rename regen.sh into a more meaningful generate_syscalls.sh name, rename
order into a more meaningful supported-syscalls.txt name and rewrite
part of the regen.sh script code in order to execute it from anywhere in
the filesystem, without need to be in its own folder. The new code is
also more clear and concise, using native sh features which are
simplifying the code.
Reviewed-by: Li Wang <liwang@redhat.com>
Reviewed-by: Petr Vorel <pvorel@suse.cz>
Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
configure.ac | 2 +-
include/lapi/syscalls/generate_syscalls.sh | 109 +++++++++++++++++
include/lapi/syscalls/regen.sh | 129 ---------------------
.../lapi/syscalls/{order => supported-arch.txt} | 0
4 files changed, 110 insertions(+), 130 deletions(-)
diff --git a/configure.ac b/configure.ac
index d327974efa71f263d7f7f5aec9d2c5831d53dd0e..cd1233d19fad376973fc880d6689859845613fb0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -386,7 +386,7 @@ else
AC_SUBST([WITH_REALTIME_TESTSUITE],["no"])
fi
-AC_CONFIG_COMMANDS([syscalls.h], [cd ${ac_top_srcdir}/include/lapi/syscalls; ./regen.sh; cd - >/dev/null])
+AC_CONFIG_COMMANDS([syscalls.h], [cd ${ac_top_srcdir}/include/lapi/syscalls; ./generate_syscalls.sh ../syscalls.h; cd - >/dev/null])
# custom functions
# NOTE: don't create custom functions for simple checks, put them into this file
diff --git a/include/lapi/syscalls/generate_syscalls.sh b/include/lapi/syscalls/generate_syscalls.sh
new file mode 100755
index 0000000000000000000000000000000000000000..b17c72ddf09cff4f55b7ef80856cd97862ce1317
--- /dev/null
+++ b/include/lapi/syscalls/generate_syscalls.sh
@@ -0,0 +1,109 @@
+#!/bin/sh -eu
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Generate the syscalls.h file, merging all architectures syscalls input file
+# which are in the current folder and defined inside supported-arch.txt file.
+
+SYSCALLS_FILE="$1"
+
+if [ -z "${SYSCALLS_FILE}" ]; then
+ echo "Please provide the syscalls.h directory:"
+ echo ""
+ echo "$0 path/of/syscalls.h"
+ echo ""
+ exit 1
+fi
+
+SCRIPT_DIR="$(realpath $(dirname "$0"))"
+SUPPORTED_ARCH="${SCRIPT_DIR}/supported-arch.txt"
+
+echo '// SPDX-License-Identifier: GPL-2.0-or-later
+/************************************************
+ * GENERATED FILE: DO NOT EDIT/PATCH THIS FILE *
+ * change your arch specific .in file instead *
+ ************************************************/
+
+/*
+ * Here we stick all the ugly *fallback* logic for linux
+ * system call numbers (those __NR_ thingies).
+ */
+
+#ifndef LAPI_SYSCALLS_H__
+#define LAPI_SYSCALLS_H__
+
+#include <errno.h>
+#include <sys/syscall.h>
+#include <asm/unistd.h>
+
+#ifdef TST_TEST_H__
+#define TST_SYSCALL_BRK__(NR, SNR) ({ \
+tst_brk(TCONF, \
+ "syscall(%d) " SNR " not supported on your arch", NR); \
+})
+#else
+inline static void dummy_cleanup(void) {}
+
+#define TST_SYSCALL_BRK__(NR, SNR) ({ \
+tst_brkm(TCONF, dummy_cleanup, \
+ "syscall(%d) " SNR " not supported on your arch", NR); \
+})
+#endif
+
+#define tst_syscall(NR, ...) ({ \
+intptr_t tst_ret; \
+if (NR == __LTP__NR_INVALID_SYSCALL) { \
+ errno = ENOSYS; \
+ tst_ret = -1; \
+} else { \
+ tst_ret = syscall(NR, ##__VA_ARGS__); \
+} \
+if (tst_ret == -1 && errno == ENOSYS) { \
+ TST_SYSCALL_BRK__(NR, #NR); \
+} \
+tst_ret; \
+})
+
+#define __LTP__NR_INVALID_SYSCALL -1' >${SYSCALLS_FILE}
+
+while IFS= read -r arch; do
+ (
+ echo
+ case ${arch} in
+ sparc64) echo "#if defined(__sparc__) && defined(__arch64__)" ;;
+ sparc) echo "#if defined(__sparc__) && !defined(__arch64__)" ;;
+ s390) echo "#if defined(__s390__) && !defined(__s390x__)" ;;
+ mips64n32) echo "#if defined(__mips__) && defined(_ABIN32)" ;;
+ mips64) echo "#if defined(__mips__) && defined(_ABI64)" ;;
+ mipso32) echo "#if defined(__mips__) && defined(_ABIO32) && _MIPS_SZLONG == 32" ;;
+ parisc) echo "#ifdef __hppa__" ;;
+ loongarch64) echo "#ifdef __loongarch__" ;;
+ arm64) echo "#ifdef __aarch64__" ;;
+ *) echo "#ifdef __${arch}__" ;;
+ esac
+
+ while read -r line; do
+ set -- ${line}
+ syscall_nr="__NR_$1"
+ shift
+
+ echo "# ifndef ${syscall_nr}"
+ echo "# define ${syscall_nr} $*"
+ echo "# endif"
+ done <"${SCRIPT_DIR}/${arch}.in"
+ echo "#endif"
+ echo
+ ) >>${SYSCALLS_FILE}
+done <${SUPPORTED_ARCH}
+
+(
+ echo
+ echo "/* Common stubs */"
+ for num in $(awk '{print $1}' "${SCRIPT_DIR}/"*.in | sort -u); do
+ syscall_nr="__NR_${num}"
+
+ echo "# ifndef ${syscall_nr}"
+ echo "# define ${syscall_nr} __LTP__NR_INVALID_SYSCALL"
+ echo "# endif"
+ done
+ echo "#endif"
+) >>${SYSCALLS_FILE}
diff --git a/include/lapi/syscalls/regen.sh b/include/lapi/syscalls/regen.sh
deleted file mode 100755
index 663ce4458bbc67c3a9e8073dfe8359164a9da0ee..0000000000000000000000000000000000000000
--- a/include/lapi/syscalls/regen.sh
+++ /dev/null
@@ -1,129 +0,0 @@
-#!/bin/sh
-
-output="syscalls.h"
-rm -f "${output}".[1-9]*
-output_pid="${output}.$$"
-
-max_jobs=$(getconf _NPROCESSORS_ONLN 2>/dev/null)
-: ${max_jobs:=1}
-
-srcdir=${0%/*}
-
-err() {
- echo "$*" 1>&2
- exit 1
-}
-
-cat << EOF > "${output_pid}"
-/************************************************
- * GENERATED FILE: DO NOT EDIT/PATCH THIS FILE *
- * change your arch specific .in file instead *
- ************************************************/
-
-/*
- * Here we stick all the ugly *fallback* logic for linux
- * system call numbers (those __NR_ thingies).
- *
- * Licensed under the GPLv2 or later, see the COPYING file.
- */
-
-#ifndef LAPI_SYSCALLS_H__
-#define LAPI_SYSCALLS_H__
-
-#include <errno.h>
-#include <sys/syscall.h>
-#include <asm/unistd.h>
-
-#ifdef TST_TEST_H__
-#define TST_SYSCALL_BRK__(NR, SNR) ({ \\
- tst_brk(TCONF, \\
- "syscall(%d) " SNR " not supported on your arch", NR); \\
-})
-#else
-inline static void dummy_cleanup(void) {}
-
-#define TST_SYSCALL_BRK__(NR, SNR) ({ \\
- tst_brkm(TCONF, dummy_cleanup, \\
- "syscall(%d) " SNR " not supported on your arch", NR); \\
-})
-#endif
-
-#define tst_syscall(NR, ...) ({ \\
- intptr_t tst_ret; \\
- if (NR == __LTP__NR_INVALID_SYSCALL) { \\
- errno = ENOSYS; \\
- tst_ret = -1; \\
- } else { \\
- tst_ret = syscall(NR, ##__VA_ARGS__); \\
- } \\
- if (tst_ret == -1 && errno == ENOSYS) { \\
- TST_SYSCALL_BRK__(NR, #NR); \\
- } \\
- tst_ret; \\
-})
-
-EOF
-
-jobs=0
-for arch in $(cat "${srcdir}/order") ; do
- (
- echo "Generating data for arch $arch ... "
-
- (
- echo
- case ${arch} in
- sparc64) echo "#if defined(__sparc__) && defined(__arch64__)" ;;
- sparc) echo "#if defined(__sparc__) && !defined(__arch64__)" ;;
- s390) echo "#if defined(__s390__) && !defined(__s390x__)" ;;
- mips_n32) echo "#if defined(__mips__) && defined(_ABIN32)" ;;
- mips_n64) echo "#if defined(__mips__) && defined(_ABI64)" ;;
- mips_o32) echo "#if defined(__mips__) && defined(_ABIO32) && _MIPS_SZLONG == 32" ;;
- *) echo "#ifdef __${arch}__" ;;
- esac
- while read line ; do
- set -- ${line}
- nr="__NR_$1"
- shift
- if [ $# -eq 0 ] ; then
- err "invalid line found: $line"
- fi
- echo "# ifndef ${nr}"
- echo "# define ${nr} $*"
- echo "# endif"
- done < "${srcdir}/${arch}.in"
- echo "#endif"
- echo
- ) >> "${output_pid}.${arch}"
-
- ) &
-
- jobs=$(( jobs + 1 ))
- if [ ${jobs} -ge ${max_jobs} ] ; then
- wait || exit 1
- jobs=0
- fi
-done
-
-echo "Generating stub list ... "
-(
-echo
-echo "/* Common stubs */"
-echo "#define __LTP__NR_INVALID_SYSCALL -1" >> "${output_pid}"
-for nr in $(awk '{print $1}' "${srcdir}/"*.in | sort -u) ; do
- nr="__NR_${nr}"
- echo "# ifndef ${nr}"
- echo "# define ${nr} __LTP__NR_INVALID_SYSCALL"
- echo "# endif"
-done
-echo "#endif"
-) >> "${output_pid}._footer"
-
-wait || exit 1
-
-printf "Combining them all ... "
-for arch in $(cat "${srcdir}/order") _footer ; do
- cat "${output_pid}.${arch}"
-done >> "${output_pid}"
-mv "${output_pid}" "../${output}"
-rm -f "${output_pid}"*
-echo "OK!"
diff --git a/include/lapi/syscalls/order b/include/lapi/syscalls/supported-arch.txt
similarity index 100%
rename from include/lapi/syscalls/order
rename to include/lapi/syscalls/supported-arch.txt
--
2.43.0
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [LTP] [PATCH v8 1/4] Refactor regen.sh script to generate syscalls
2024-10-31 16:01 ` [LTP] [PATCH v8 1/4] Refactor regen.sh script to generate syscalls Andrea Cervesato
@ 2024-10-31 16:21 ` Cyril Hrubis
2024-11-01 7:39 ` Petr Vorel
0 siblings, 1 reply; 11+ messages in thread
From: Cyril Hrubis @ 2024-10-31 16:21 UTC (permalink / raw)
To: Andrea Cervesato; +Cc: ltp
Hi!
> +while IFS= read -r arch; do
> + (
> + echo
> + case ${arch} in
> + sparc64) echo "#if defined(__sparc__) && defined(__arch64__)" ;;
> + sparc) echo "#if defined(__sparc__) && !defined(__arch64__)" ;;
> + s390) echo "#if defined(__s390__) && !defined(__s390x__)" ;;
> + mips64n32) echo "#if defined(__mips__) && defined(_ABIN32)" ;;
> + mips64) echo "#if defined(__mips__) && defined(_ABI64)" ;;
> + mipso32) echo "#if defined(__mips__) && defined(_ABIO32) && _MIPS_SZLONG == 32" ;;
It would be nice to get a confirmation from someone who knows mips that
these changes in ifdefs are okay, otherwise:
Reviewed-by: Cyril Hrubis <chrubis@suse.cz>
--
Cyril Hrubis
chrubis@suse.cz
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [LTP] [PATCH v8 1/4] Refactor regen.sh script to generate syscalls
2024-10-31 16:21 ` Cyril Hrubis
@ 2024-11-01 7:39 ` Petr Vorel
2024-11-01 8:07 ` Andrea Cervesato via ltp
0 siblings, 1 reply; 11+ messages in thread
From: Petr Vorel @ 2024-11-01 7:39 UTC (permalink / raw)
To: Cyril Hrubis; +Cc: ltp
Hi all,
> Hi!
> > +while IFS= read -r arch; do
> > + (
> > + echo
> > + case ${arch} in
> > + sparc64) echo "#if defined(__sparc__) && defined(__arch64__)" ;;
> > + sparc) echo "#if defined(__sparc__) && !defined(__arch64__)" ;;
> > + s390) echo "#if defined(__s390__) && !defined(__s390x__)" ;;
> > + mips64n32) echo "#if defined(__mips__) && defined(_ABIN32)" ;;
> > + mips64) echo "#if defined(__mips__) && defined(_ABI64)" ;;
> > + mipso32) echo "#if defined(__mips__) && defined(_ABIO32) && _MIPS_SZLONG == 32" ;;
The original code was:
- mips_n32) echo "#if defined(__mips__) && defined(_ABIN32)" ;;
- mips_n64) echo "#if defined(__mips__) && defined(_ABI64)" ;;
- mips_o32) echo "#if defined(__mips__) && defined(_ABIO32) && _MIPS_SZLONG == 32" ;;
ifdefs are the same, the only thing which changed are arch names. And these are
also changed, see below. The only problem I see is that the change in
supported-arch.txt below happens in the following commit. Shouldn't the rename
happen in this commit already? syscalls.h file generated from the first commit
is indeed different from the one generated in 2nd and 3rd (which are the same.
+++ b/include/lapi/syscalls/supported-arch.txt
@@ -1,13 +1,13 @@
-aarch64
arc
+arm64
arm
-hppa
i386
ia64
-loongarch
-mips_n32
-mips_n64
-mips_o32
+loongarch64
+mips64n32
+mips64
+mipso32
+parisc
powerpc64
powerpc
s390x
1st commit have (which is IMHO wrong)
#ifdef __mips_n32__
2nd and 3rd have:
#if defined(__mips__) && defined(_ABIN32)
The reason due postponed changes is, that some arch use
*) echo "#ifdef __${arch}__" ;;
Whoever will do bisecting, cannot use this commit on mips and other affected
archs due broken syscalls.h. This problem appeared in v5. Previously (v3 and v4)
old names were used in generate_syscalls.sh, and in second commit (when the
generate_arch.sh archs was added) were new ones used.
IMHO clearest solution would be to move rename to 1st commit.
> It would be nice to get a confirmation from someone who knows mips that
> these changes in ifdefs are okay, otherwise:
I guess we know the answer thus we don't have to bother mips folks.
But very good catch.
Kind regards,
Petr
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [LTP] [PATCH v8 1/4] Refactor regen.sh script to generate syscalls
2024-11-01 7:39 ` Petr Vorel
@ 2024-11-01 8:07 ` Andrea Cervesato via ltp
0 siblings, 0 replies; 11+ messages in thread
From: Andrea Cervesato via ltp @ 2024-11-01 8:07 UTC (permalink / raw)
To: Petr Vorel, Cyril Hrubis; +Cc: ltp
Hi Petr,
On 11/1/24 08:39, Petr Vorel wrote:
> Hi all,
>
>> Hi!
>>> +while IFS= read -r arch; do
>>> + (
>>> + echo
>>> + case ${arch} in
>>> + sparc64) echo "#if defined(__sparc__) && defined(__arch64__)" ;;
>>> + sparc) echo "#if defined(__sparc__) && !defined(__arch64__)" ;;
>>> + s390) echo "#if defined(__s390__) && !defined(__s390x__)" ;;
>>> + mips64n32) echo "#if defined(__mips__) && defined(_ABIN32)" ;;
>>> + mips64) echo "#if defined(__mips__) && defined(_ABI64)" ;;
>>> + mipso32) echo "#if defined(__mips__) && defined(_ABIO32) && _MIPS_SZLONG == 32" ;;
> The original code was:
>
> - mips_n32) echo "#if defined(__mips__) && defined(_ABIN32)" ;;
> - mips_n64) echo "#if defined(__mips__) && defined(_ABI64)" ;;
> - mips_o32) echo "#if defined(__mips__) && defined(_ABIO32) && _MIPS_SZLONG == 32" ;;
>
> ifdefs are the same, the only thing which changed are arch names. And these are
> also changed, see below. The only problem I see is that the change in
> supported-arch.txt below happens in the following commit. Shouldn't the rename
> happen in this commit already? syscalls.h file generated from the first commit
> is indeed different from the one generated in 2nd and 3rd (which are the same.
>
> +++ b/include/lapi/syscalls/supported-arch.txt
> @@ -1,13 +1,13 @@
> -aarch64
> arc
> +arm64
> arm
> -hppa
> i386
> ia64
> -loongarch
> -mips_n32
> -mips_n64
> -mips_o32
> +loongarch64
> +mips64n32
> +mips64
> +mipso32
> +parisc
> powerpc64
> powerpc
> s390x
>
> 1st commit have (which is IMHO wrong)
> #ifdef __mips_n32__
>
> 2nd and 3rd have:
> #if defined(__mips__) && defined(_ABIN32)
>
> The reason due postponed changes is, that some arch use
>
> *) echo "#ifdef __${arch}__" ;;
>
> Whoever will do bisecting, cannot use this commit on mips and other affected
> archs due broken syscalls.h. This problem appeared in v5. Previously (v3 and v4)
> old names were used in generate_syscalls.sh, and in second commit (when the
> generate_arch.sh archs was added) were new ones used.
>
> IMHO clearest solution would be to move rename to 1st commit.
I agree with it. I'm going to fix it.
>
>> It would be nice to get a confirmation from someone who knows mips that
>> these changes in ifdefs are okay, otherwise:
> I guess we know the answer thus we don't have to bother mips folks.
> But very good catch.
>
> Kind regards,
> Petr
Andrea
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 11+ messages in thread
* [LTP] [PATCH v8 2/4] Add script to generate arch(s) dependant syscalls
2024-10-31 16:01 [LTP] [PATCH v8 0/4] Automatically generate syscalls.h Andrea Cervesato
2024-10-31 16:01 ` [LTP] [PATCH v8 1/4] Refactor regen.sh script to generate syscalls Andrea Cervesato
@ 2024-10-31 16:01 ` Andrea Cervesato
2024-11-01 9:18 ` Petr Vorel
2024-10-31 16:01 ` [LTP] [PATCH v8 3/4] Delete obsolete strip_syscall.awk file Andrea Cervesato
2024-10-31 16:01 ` [LTP] [PATCH v8 4/4] Add documentation about syscalls.h generator Andrea Cervesato
3 siblings, 1 reply; 11+ messages in thread
From: Andrea Cervesato @ 2024-10-31 16:01 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
Add generate_arch.sh script which can be used to generate arch(s)
dependant syscalls file. The way it works is pretty simple: for each
architecture defined into supported-arch.txt, compile kernel headers,
extract the list of syscalls and generate a .in file containing all of
them, associated with their own syscall's number.
The way syscalls files are generated, passes through a C application
which is automatically checking the availability of the syscalls in
the user space environment.
Reviewed-by: Li Wang <liwang@redhat.com>
Reviewed-by: Cyril Hrubis <chrubis@suse.cz>
Reviewed-by: Petr Vorel <pvorel@suse.cz>
Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
include/lapi/syscalls/{aarch64.in => arm64.in} | 0
include/lapi/syscalls/generate_arch.sh | 206 +++++++++++++++++++++
.../lapi/syscalls/{loongarch.in => loongarch64.in} | 0
include/lapi/syscalls/{mips_n64.in => mips64.in} | 0
.../lapi/syscalls/{mips_n32.in => mips64n32.in} | 0
include/lapi/syscalls/{mips_o32.in => mipso32.in} | 0
include/lapi/syscalls/{hppa.in => parisc.in} | 0
include/lapi/syscalls/supported-arch.txt | 12 +-
8 files changed, 212 insertions(+), 6 deletions(-)
diff --git a/include/lapi/syscalls/aarch64.in b/include/lapi/syscalls/arm64.in
similarity index 100%
rename from include/lapi/syscalls/aarch64.in
rename to include/lapi/syscalls/arm64.in
diff --git a/include/lapi/syscalls/generate_arch.sh b/include/lapi/syscalls/generate_arch.sh
new file mode 100755
index 0000000000000000000000000000000000000000..e4daf6b40b6d5fa554f7abd544ed2f4a53e7d283
--- /dev/null
+++ b/include/lapi/syscalls/generate_arch.sh
@@ -0,0 +1,206 @@
+#!/bin/sh -eu
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# This is an adaptation of the update-tables.sh script, included in the
+# syscalls-table project (https://github.com/hrw/syscalls-table) and released
+# under the MIT license.
+#
+# Author: Andrea Cervesato <andrea.cervesato@suse.com>
+
+if [ "$#" -eq "0" ]; then
+ echo "Please provide kernel sources:"
+ echo ""
+ echo "$0 path/to/Linux/kernel/sources"
+ echo ""
+ exit 1
+fi
+
+KERNELSRC="$1"
+
+# to keep sorting in order
+export LC_ALL=C
+
+if [ ! -d "$KERNELSRC" ]; then
+ echo "$KERNELSRC is not a directory"
+ exit 1
+fi
+
+if [ ! -e "${KERNELSRC}/Makefile" ]; then
+ echo "No Makefile in $KERNELSRC directory"
+ exit 1
+fi
+
+TEMP="$(mktemp -d)"
+KVER="$(make -C ${KERNELSRC} kernelversion -s)"
+
+SCRIPT_DIR="$(realpath $(dirname "$0"))"
+SUPPORTED_ARCH="${SCRIPT_DIR}/supported-arch.txt"
+LINUX_HEADERS="${TEMP}/headers"
+
+grab_syscall_names_from_tables() {
+ for tbl_file in $(find ${KERNELSRC}/arch -name syscall*.tbl); do
+ grep -E -v "(^#|^$|sys_ni_syscall)" $tbl_file |
+ awk '{ print $3 }' >>${TEMP}/syscall-names.tosort
+ done
+
+ drop_bad_entries
+}
+
+grab_syscall_names_from_unistd_h() {
+ grep -E -h "^#define __NR_" \
+ ${LINUX_HEADERS}/usr/include/asm/unistd*.h \
+ ${LINUX_HEADERS}/usr/include/asm-generic/unistd.h \
+ >${TEMP}/syscall-names.tosort
+
+ drop_bad_entries
+}
+
+drop_bad_entries() {
+ grep -E -v "(unistd.h|NR3264|__NR_syscall|__SC_COMP|__NR_.*Linux|__NR_FAST)" \
+ ${TEMP}/syscall-names.tosort |
+ grep -E -v "(__SYSCALL|SYSCALL_BASE|SYSCALL_MASK)" |
+ sed -e "s/#define\s*__NR_//g" -e "s/\s.*//g" |
+ sort -u >${TEMP}/syscall-names.txt
+}
+
+generate_table() {
+ echo "- $arch"
+
+ if [ "$bits" -eq "32" ]; then
+ extraflags="${extraflags} -D__BITS_PER_LONG=32"
+ fi
+
+ local uppercase_arch=$(echo "$arch" | tr '[:lower:]' '[:upper:]')
+
+ # ignore any error generated by gcc. We want to obtain all the
+ # available architecture syscalls for the current platform and to handle
+ # only supported architectures later on
+ gcc ${TEMP}/list-syscalls.c -U__LP64__ -U__ILP32__ -U__i386__ \
+ -D${uppercase_arch} \
+ -D__${arch}__ ${extraflags} \
+ -I ${LINUX_HEADERS}/usr/include/ \
+ -o ${TEMP}/list-syscalls || true
+
+ ${TEMP}/list-syscalls >"${TEMP}/${arch}.in.tosort"
+
+ sort -k2,2n "${TEMP}/${arch}.in.tosort" >"${TEMP}/${arch}.in"
+}
+
+generate_list_syscalls_c() {
+ (
+ echo
+ echo "
+ #include <stdio.h>
+ #include <asm/unistd.h>
+
+ int main(void)
+ {
+ "
+ for syscall in $(cat ${TEMP}/syscall-names.txt); do
+ echo "
+ #ifdef __NR_$syscall
+ printf(\"$syscall %d\\\n\", __NR_$syscall);
+ #endif
+ "
+ done
+ echo " return 0;
+ }"
+ ) >${TEMP}/list-syscalls.c
+}
+
+export_headers() {
+ make -s -C ${KERNELSRC} ARCH=${arch} O=${LINUX_HEADERS} \
+ headers_install >/dev/null 2>&1
+}
+
+do_all_tables() {
+ for archdir in ${KERNELSRC}/arch/*; do
+ arch=$(basename $archdir)
+
+ bits=64
+ extraflags=
+
+ case ${arch} in
+ Kconfig)
+ continue
+ ;;
+ um)
+ continue
+ ;;
+ esac
+
+ export_headers
+ grab_syscall_names_from_unistd_h
+
+ case ${arch} in
+ arm)
+ bits=32
+ arch=armoabi extraflags= generate_table
+ arch=arm extraflags=-D__ARM_EABI__ generate_table
+ ;;
+ loongarch)
+ # 32-bit variant of loongarch may appear
+ arch=loongarch64 extraflags=-D_LOONGARCH_SZLONG=64 generate_table
+ ;;
+ mips)
+ arch=mips64 extraflags=-D_MIPS_SIM=_MIPS_SIM_ABI64 generate_table
+ bits=32
+ arch=mipso32 extraflags=-D_MIPS_SIM=_MIPS_SIM_ABI32 generate_table
+ arch=mips64n32 extraflags=-D_MIPS_SIM=_MIPS_SIM_NABI32 generate_table
+ ;;
+ powerpc)
+ generate_table
+ arch=powerpc64 generate_table
+ ;;
+ riscv)
+ arch=riscv64 extraflags=-D__LP64__ generate_table
+ bits=32
+ arch=riscv32 extraflags=-D__SIZEOF_POINTER__=4 generate_table
+ ;;
+ s390)
+ bits=32
+ generate_table
+ bits=64
+ arch=s390x generate_table
+ ;;
+ sparc)
+ bits=32
+ extraflags=-D__32bit_syscall_numbers__ generate_table
+ bits=64
+ arch=sparc64 extraflags=-D__arch64__ generate_table
+ ;;
+ x86)
+ arch=x86_64 extraflags=-D__LP64__ generate_table
+ bits=32
+ arch=i386 generate_table
+ arch=x32 extraflags=-D__ILP32__ generate_table
+ ;;
+ arc | csky | hexagon | m68k | microblaze | nios2 | openrisc | sh | xtensa)
+ bits=32 generate_table
+ ;;
+ *)
+ generate_table
+ ;;
+ esac
+ done
+}
+
+copy_supported_arch() {
+ while IFS= read -r arch; do
+ if [ -f "${TEMP}/${arch}.in" ]; then
+ echo "- ${arch}"
+ cp "${TEMP}/${arch}.in" "${SCRIPT_DIR}/${arch}.in"
+ fi
+ done <${SUPPORTED_ARCH}
+}
+
+echo "Temporary directory ${TEMP}"
+echo "Extracting syscalls"
+
+grab_syscall_names_from_tables
+generate_list_syscalls_c
+
+do_all_tables
+
+echo "Copying supported syscalls"
+copy_supported_arch
diff --git a/include/lapi/syscalls/loongarch.in b/include/lapi/syscalls/loongarch64.in
similarity index 100%
rename from include/lapi/syscalls/loongarch.in
rename to include/lapi/syscalls/loongarch64.in
diff --git a/include/lapi/syscalls/mips_n64.in b/include/lapi/syscalls/mips64.in
similarity index 100%
rename from include/lapi/syscalls/mips_n64.in
rename to include/lapi/syscalls/mips64.in
diff --git a/include/lapi/syscalls/mips_n32.in b/include/lapi/syscalls/mips64n32.in
similarity index 100%
rename from include/lapi/syscalls/mips_n32.in
rename to include/lapi/syscalls/mips64n32.in
diff --git a/include/lapi/syscalls/mips_o32.in b/include/lapi/syscalls/mipso32.in
similarity index 100%
rename from include/lapi/syscalls/mips_o32.in
rename to include/lapi/syscalls/mipso32.in
diff --git a/include/lapi/syscalls/hppa.in b/include/lapi/syscalls/parisc.in
similarity index 100%
rename from include/lapi/syscalls/hppa.in
rename to include/lapi/syscalls/parisc.in
diff --git a/include/lapi/syscalls/supported-arch.txt b/include/lapi/syscalls/supported-arch.txt
index c18aa38cf4546cdf3ac8c89a45bd1b202ffa7711..c5c5191ac08482d89f0a8c39dfae936538f58f7f 100644
--- a/include/lapi/syscalls/supported-arch.txt
+++ b/include/lapi/syscalls/supported-arch.txt
@@ -1,13 +1,13 @@
-aarch64
arc
+arm64
arm
-hppa
i386
ia64
-loongarch
-mips_n32
-mips_n64
-mips_o32
+loongarch64
+mips64n32
+mips64
+mipso32
+parisc
powerpc64
powerpc
s390x
--
2.43.0
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [LTP] [PATCH v8 2/4] Add script to generate arch(s) dependant syscalls
2024-10-31 16:01 ` [LTP] [PATCH v8 2/4] Add script to generate arch(s) dependant syscalls Andrea Cervesato
@ 2024-11-01 9:18 ` Petr Vorel
0 siblings, 0 replies; 11+ messages in thread
From: Petr Vorel @ 2024-11-01 9:18 UTC (permalink / raw)
To: Andrea Cervesato; +Cc: ltp
Hi Andrea,
the script works for me, great!
Few more comments, I promis the last one, feel free to just ignore them.
Reviewed-by: Petr Vorel <pvorel@suse.cz>
> --- /dev/null
> +++ b/include/lapi/syscalls/generate_arch.sh
> @@ -0,0 +1,206 @@
> +#!/bin/sh -eu
> +# SPDX-License-Identifier: GPL-2.0-or-later
I would put here your/SUSE/LTP copyright (2024) and the original author
copyright:
# Copyright (c) Marcin Juszkiewicz, 2023-2024
The author does not have it in the script, but it has in LICENSE:
Copyright (c) Marcin Juszkiewicz
But I'm not a lawyer, maybe I'm wrong.
> +#
> +# This is an adaptation of the update-tables.sh script, included in the
> +# syscalls-table project (https://github.com/hrw/syscalls-table) and released
> +# under the MIT license.
> +#
> +# Author: Andrea Cervesato <andrea.cervesato@suse.com>
> +
> +if [ "$#" -eq "0" ]; then
> + echo "Please provide kernel sources:"
> + echo ""
> + echo "$0 path/to/Linux/kernel/sources"
> + echo ""
> + exit 1
> +fi
nit: use common syntax ($# is always defined, thus not need to be quoted, 0 as
well, "" is empty string, thus can be omitted):
if [ $# -eq 0 ]; then
echo "Please provide kernel sources:"
echo
echo "$0 path/to/Linux/kernel/sources"
echo
exit 1
fi
...
> +generate_list_syscalls_c() {
> + (
very nit: this first echo is probably not needed.
> + echo
> + echo "
> + #include <stdio.h>
> + #include <asm/unistd.h>
> +
> + int main(void)
> + {
> + "
> + for syscall in $(cat ${TEMP}/syscall-names.txt); do
> + echo "
> + #ifdef __NR_$syscall
> + printf(\"$syscall %d\\\n\", __NR_$syscall);
> + #endif
> + "
> + done
> + echo " return 0;
> + }"
> + ) >${TEMP}/list-syscalls.c
> +}
...
> +
> +do_all_tables() {
> + for archdir in ${KERNELSRC}/arch/*; do
> + arch=$(basename $archdir)
...
> + arch=x86_64 extraflags=-D__LP64__ generate_table
> + bits=32
> + arch=i386 generate_table
> + arch=x32 extraflags=-D__ILP32__ generate_table
> + ;;
> + arc | csky | hexagon | m68k | microblaze | nios2 | openrisc | sh | xtensa)
I've never seen spaces between '|' separator in shell case command. It should
not cause anything, but it's just unusual (original code in
https://github.com/hrw/syscalls-table/blob/master/scripts/update-tables.sh
from which we diverged quite a lot now also does not have it).
Kind regards,
Petr
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 11+ messages in thread
* [LTP] [PATCH v8 3/4] Delete obsolete strip_syscall.awk file
2024-10-31 16:01 [LTP] [PATCH v8 0/4] Automatically generate syscalls.h Andrea Cervesato
2024-10-31 16:01 ` [LTP] [PATCH v8 1/4] Refactor regen.sh script to generate syscalls Andrea Cervesato
2024-10-31 16:01 ` [LTP] [PATCH v8 2/4] Add script to generate arch(s) dependant syscalls Andrea Cervesato
@ 2024-10-31 16:01 ` Andrea Cervesato
2024-10-31 16:01 ` [LTP] [PATCH v8 4/4] Add documentation about syscalls.h generator Andrea Cervesato
3 siblings, 0 replies; 11+ messages in thread
From: Andrea Cervesato @ 2024-10-31 16:01 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
Reviewed-by: Li Wang <liwang@redhat.com>
Reviewed-by: Petr Vorel <pvorel@suse.cz>
Reviewed-by: Cyril Hrubis <chrubis@suse.cz>
Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
include/lapi/syscalls/strip_syscall.awk | 19 -------------------
1 file changed, 19 deletions(-)
diff --git a/include/lapi/syscalls/strip_syscall.awk b/include/lapi/syscalls/strip_syscall.awk
deleted file mode 100755
index e8dff422e2667745c144ed984a4d66461fcce0c6..0000000000000000000000000000000000000000
--- a/include/lapi/syscalls/strip_syscall.awk
+++ /dev/null
@@ -1,19 +0,0 @@
-#!/usr/bin/awk -f
-#
-# Dumb script that can be used to strip all of the syscall information from
-# the arch-respective unistd*.h.
-#
-# Examples:
-#
-# 1. Grab the i386 32-bit syscalls from unistd_32.h and put them in i386.in
-# strip_syscall.awk arch/x86/include/asm/unistd_32.h > i386.in
-#
-
-/^#define[[:space:]]+__NR_[0-9a-z]+/ {
-
- sub (/#define[[:space:]]+__NR_/, "", $0);
- sub (/[[:space:]]*(\/\*.*)/, "", $0);
- sub (/[[:space:]]+/, " ", $0);
-
- print
-}
--
2.43.0
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 11+ messages in thread* [LTP] [PATCH v8 4/4] Add documentation about syscalls.h generator
2024-10-31 16:01 [LTP] [PATCH v8 0/4] Automatically generate syscalls.h Andrea Cervesato
` (2 preceding siblings ...)
2024-10-31 16:01 ` [LTP] [PATCH v8 3/4] Delete obsolete strip_syscall.awk file Andrea Cervesato
@ 2024-10-31 16:01 ` Andrea Cervesato
2024-10-31 16:31 ` Cyril Hrubis
3 siblings, 1 reply; 11+ messages in thread
From: Andrea Cervesato @ 2024-10-31 16:01 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
doc/developers/test_case_tutorial.rst | 27 ++++++++++++++-------------
1 file changed, 14 insertions(+), 13 deletions(-)
diff --git a/doc/developers/test_case_tutorial.rst b/doc/developers/test_case_tutorial.rst
index be9a0ea8bac1d690b4ed4879d110f87c3f2a381a..b48ea4a7a3594266466fcca364f5d0ff1e42d235 100644
--- a/doc/developers/test_case_tutorial.rst
+++ b/doc/developers/test_case_tutorial.rst
@@ -302,24 +302,25 @@ for a distribution's C library version to be older than its kernel or it may use
cut down C library in comparison to the GNU one. So we must call ``statx()``
using the general ``syscall()`` interface.
-The LTP contains a library for dealing with the ``syscall`` interface, which is
+LTP contains a library for dealing with the ``syscall`` interface, which is
located in :master:`include/lapi`. System call numbers are listed against the relevant
call in the ``*.in`` files (e.g. ``x86_64.in``) which are used to generate
-``syscalls.h``, which is the header you should include. On rare occasions you
-may find the system call number is missing from the ``*.in`` files and will need
-to add it (see :master:`include/lapi/syscalls/strip_syscall.awk`).
+``syscalls.h``, the header you should include.
-System call numbers vary between architectures, hence there are multiple
-``*.in`` files for each architecture. You can find the various values for the
-``statx`` system call across a number of ``unistd.h`` files in the Linux kernel.
+On rare occasions, you may find that system call number is missing from ``*.in``
+files. In these cases, they will need to be updated using
+`include/lapi/syscalls/generate_arch.sh` script as following:
-Note that we don't use the system-call-identifier value available in
-``/usr/include/linux/uinstd.h`` because the kernel might be much newer than the
-user land development packages.
+.. code-block:: bash
+
+ $ include/lapi/syscalls/generate_arch.sh /path/of/linux/sources
+
+The script will generate all the needed ``*.in`` files according to the Linux
+source code which has been used. Make sure that your Linux source code has
+been updated to the latest version.
-For ``statx`` we had to add ``statx 332`` to :master:`include/lapi/syscalls/x86_64.in`,
-``statx 383`` to :master:`include/lapi/syscalls/powerpc.in`, etc. Now lets look at
-the code, which I will explain in more detail further down.
+Once the new syscalls files have been updated, to rebuild our ``syscalls.h``
+file, please use ``./configure`` command.
.. code-block:: c
--
2.43.0
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [LTP] [PATCH v8 4/4] Add documentation about syscalls.h generator
2024-10-31 16:01 ` [LTP] [PATCH v8 4/4] Add documentation about syscalls.h generator Andrea Cervesato
@ 2024-10-31 16:31 ` Cyril Hrubis
2024-11-01 7:56 ` Petr Vorel
0 siblings, 1 reply; 11+ messages in thread
From: Cyril Hrubis @ 2024-10-31 16:31 UTC (permalink / raw)
To: Andrea Cervesato; +Cc: ltp
Hi!
> Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
> ---
> doc/developers/test_case_tutorial.rst | 27 ++++++++++++++-------------
> 1 file changed, 14 insertions(+), 13 deletions(-)
>
> diff --git a/doc/developers/test_case_tutorial.rst b/doc/developers/test_case_tutorial.rst
> index be9a0ea8bac1d690b4ed4879d110f87c3f2a381a..b48ea4a7a3594266466fcca364f5d0ff1e42d235 100644
> --- a/doc/developers/test_case_tutorial.rst
> +++ b/doc/developers/test_case_tutorial.rst
> @@ -302,24 +302,25 @@ for a distribution's C library version to be older than its kernel or it may use
> cut down C library in comparison to the GNU one. So we must call ``statx()``
> using the general ``syscall()`` interface.
>
> -The LTP contains a library for dealing with the ``syscall`` interface, which is
> +LTP contains a library for dealing with the ``syscall`` interface, which is
> located in :master:`include/lapi`. System call numbers are listed against the relevant
> call in the ``*.in`` files (e.g. ``x86_64.in``) which are used to generate
> -``syscalls.h``, which is the header you should include. On rare occasions you
> -may find the system call number is missing from the ``*.in`` files and will need
> -to add it (see :master:`include/lapi/syscalls/strip_syscall.awk`).
> +``syscalls.h``, the header you should include.
>
> -System call numbers vary between architectures, hence there are multiple
> -``*.in`` files for each architecture.
I would keep this sentence in there, probably as a last sentence in the
upper paragraph.
> - You can find the various values for the
> -``statx`` system call across a number of ``unistd.h`` files in the Linux kernel.
> +On rare occasions, you may find that system call number is missing from ``*.in``
> +files. In these cases, they will need to be updated using
> +`include/lapi/syscalls/generate_arch.sh` script as following:
^
Just `generate_arch.sh` here, we list the full path in the code
block, no need to repeat it here as well.
> -Note that we don't use the system-call-identifier value available in
> -``/usr/include/linux/uinstd.h`` because the kernel might be much newer than the
> -user land development packages.
> +.. code-block:: bash
> +
> + $ include/lapi/syscalls/generate_arch.sh /path/of/linux/sources
^
to
> +The script will generate all the needed ``*.in`` files according to the Linux
^
accordingly
> +source code which has been used. Make sure that your Linux source code has
> +been updated to the latest version.
>
> -For ``statx`` we had to add ``statx 332`` to :master:`include/lapi/syscalls/x86_64.in`,
> -``statx 383`` to :master:`include/lapi/syscalls/powerpc.in`, etc. Now lets look at
> -the code, which I will explain in more detail further down.
> +Once the new syscalls files have been updated, to rebuild our ``syscalls.h``
> +file, please use ``./configure`` command.
^ ^
re-run script
--
Cyril Hrubis
chrubis@suse.cz
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [LTP] [PATCH v8 4/4] Add documentation about syscalls.h generator
2024-10-31 16:31 ` Cyril Hrubis
@ 2024-11-01 7:56 ` Petr Vorel
0 siblings, 0 replies; 11+ messages in thread
From: Petr Vorel @ 2024-11-01 7:56 UTC (permalink / raw)
To: Cyril Hrubis; +Cc: ltp
HI Andrea, Cyril,
> > +++ b/doc/developers/test_case_tutorial.rst
> > @@ -302,24 +302,25 @@ for a distribution's C library version to be older than its kernel or it may use
> > cut down C library in comparison to the GNU one. So we must call ``statx()``
> > using the general ``syscall()`` interface.
> > -The LTP contains a library for dealing with the ``syscall`` interface, which is
> > +LTP contains a library for dealing with the ``syscall`` interface, which is
> > located in :master:`include/lapi`. System call numbers are listed against the relevant
> > call in the ``*.in`` files (e.g. ``x86_64.in``) which are used to generate
> > -``syscalls.h``, which is the header you should include. On rare occasions you
> > -may find the system call number is missing from the ``*.in`` files and will need
> > -to add it (see :master:`include/lapi/syscalls/strip_syscall.awk`).
> > +``syscalls.h``, the header you should include.
> > -System call numbers vary between architectures, hence there are multiple
> > -``*.in`` files for each architecture.
> I would keep this sentence in there, probably as a last sentence in the
> upper paragraph.
+1
> > - You can find the various values for the
> > -``statx`` system call across a number of ``unistd.h`` files in the Linux kernel.
> > +On rare occasions, you may find that system call number is missing from ``*.in``
> > +files. In these cases, they will need to be updated using
> > +`include/lapi/syscalls/generate_arch.sh` script as following:
> ^
> Just `generate_arch.sh` here, we list the full path in the code
> block, no need to repeat it here as well.
+1. BTW it should be either ``generate_arch.sh`` (double `` is used for
other commands, e.g. ``make check`` or ``./configure``) or it could be a link to
a source:
:master:`include/lapi/syscalls/generate_arch.sh` script as following:
I like links, because reader can see the script content. Also it can help to
verify whether the script is still at the repository (was not moved/deleted).
> > -Note that we don't use the system-call-identifier value available in
> > -``/usr/include/linux/uinstd.h`` because the kernel might be much newer than the
> > -user land development packages.
> > +.. code-block:: bash
> > +
> > + $ include/lapi/syscalls/generate_arch.sh /path/of/linux/sources
> ^
> to
very nit: "path/to/Linux/kernel/sources" (upper "L", which you use later, also
it's used in the script).
> > +The script will generate all the needed ``*.in`` files according to the Linux
> ^
> accordingly
+1
> > +source code which has been used. Make sure that your Linux source code has
> > +been updated to the latest version.
> > -For ``statx`` we had to add ``statx 332`` to :master:`include/lapi/syscalls/x86_64.in`,
> > -``statx 383`` to :master:`include/lapi/syscalls/powerpc.in`, etc. Now lets look at
> > -the code, which I will explain in more detail further down.
> > +Once the new syscalls files have been updated, to rebuild our ``syscalls.h``
> > +file, please use ``./configure`` command.
> ^ ^
> re-run script
+1
Otherwise LGTM, thanks a lot for bothering with the docs.
Kind regards,
Petr
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 11+ messages in thread