* [LTP] [PATCH v7 1/4] Refactor regen.sh script to generate syscalls
2024-10-31 11:57 [LTP] [PATCH v7 0/4] Automatically generate syscalls.h Andrea Cervesato
@ 2024-10-31 11:57 ` Andrea Cervesato
2024-10-31 13:01 ` Petr Vorel
2024-10-31 11:57 ` [LTP] [PATCH v7 2/4] Add script to generate arch(s) dependant syscalls Andrea Cervesato
` (2 subsequent siblings)
3 siblings, 1 reply; 13+ messages in thread
From: Andrea Cervesato @ 2024-10-31 11:57 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
Rename regen.sh into a more meaningful generate_syscalls.sh name, rename
order into a more meaningful supported-syscalls.txt name and rewrite
part of the regen.sh script code in order to execute it from anywhere in
the filesystem, without need to be in its own folder. The new code is
also more clear and concise, using native sh features which are
simplifying the code.
Reviewed-by: Li Wang <liwang@redhat.com>
Reviewed-by: Petr Vorel <pvorel@suse.cz>
Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
configure.ac | 2 +-
include/lapi/syscalls/generate_syscalls.sh | 109 +++++++++++++++++
include/lapi/syscalls/regen.sh | 129 ---------------------
.../lapi/syscalls/{order => supported-arch.txt} | 0
4 files changed, 110 insertions(+), 130 deletions(-)
diff --git a/configure.ac b/configure.ac
index d327974efa71f263d7f7f5aec9d2c5831d53dd0e..cd1233d19fad376973fc880d6689859845613fb0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -386,7 +386,7 @@ else
AC_SUBST([WITH_REALTIME_TESTSUITE],["no"])
fi
-AC_CONFIG_COMMANDS([syscalls.h], [cd ${ac_top_srcdir}/include/lapi/syscalls; ./regen.sh; cd - >/dev/null])
+AC_CONFIG_COMMANDS([syscalls.h], [cd ${ac_top_srcdir}/include/lapi/syscalls; ./generate_syscalls.sh ../syscalls.h; cd - >/dev/null])
# custom functions
# NOTE: don't create custom functions for simple checks, put them into this file
diff --git a/include/lapi/syscalls/generate_syscalls.sh b/include/lapi/syscalls/generate_syscalls.sh
new file mode 100755
index 0000000000000000000000000000000000000000..e94a6ea3c1f1623f0f846c724c8c5ff2b95450af
--- /dev/null
+++ b/include/lapi/syscalls/generate_syscalls.sh
@@ -0,0 +1,109 @@
+#!/bin/sh -eux
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Generate the syscalls.h file, merging all architectures syscalls input file
+# which are in the current folder and defined inside supported-arch.txt file.
+
+SYSCALLS_FILE="$1"
+
+if [ -z "${SYSCALLS_FILE}" ]; then
+ echo "Please provide the syscalls.h directory:"
+ echo ""
+ echo "$0 path/of/syscalls.h"
+ echo ""
+ exit 1
+fi
+
+SCRIPT_DIR="$(realpath $(dirname "$0"))"
+SUPPORTED_ARCH="${SCRIPT_DIR}/supported-arch.txt"
+
+echo '// SPDX-License-Identifier: GPL-2.0-or-later
+/************************************************
+ * GENERATED FILE: DO NOT EDIT/PATCH THIS FILE *
+ * change your arch specific .in file instead *
+ ************************************************/
+
+/*
+ * Here we stick all the ugly *fallback* logic for linux
+ * system call numbers (those __NR_ thingies).
+ */
+
+#ifndef LAPI_SYSCALLS_H__
+#define LAPI_SYSCALLS_H__
+
+#include <errno.h>
+#include <sys/syscall.h>
+#include <asm/unistd.h>
+
+#ifdef TST_TEST_H__
+#define TST_SYSCALL_BRK__(NR, SNR) ({ \
+tst_brk(TCONF, \
+ "syscall(%d) " SNR " not supported on your arch", NR); \
+})
+#else
+inline static void dummy_cleanup(void) {}
+
+#define TST_SYSCALL_BRK__(NR, SNR) ({ \
+tst_brkm(TCONF, dummy_cleanup, \
+ "syscall(%d) " SNR " not supported on your arch", NR); \
+})
+#endif
+
+#define tst_syscall(NR, ...) ({ \
+intptr_t tst_ret; \
+if (NR == __LTP__NR_INVALID_SYSCALL) { \
+ errno = ENOSYS; \
+ tst_ret = -1; \
+} else { \
+ tst_ret = syscall(NR, ##__VA_ARGS__); \
+} \
+if (tst_ret == -1 && errno == ENOSYS) { \
+ TST_SYSCALL_BRK__(NR, #NR); \
+} \
+tst_ret; \
+})
+
+#define __LTP__NR_INVALID_SYSCALL -1' >${SYSCALLS_FILE}
+
+while IFS= read -r arch; do
+ (
+ echo
+ case ${arch} in
+ sparc64) echo "#if defined(__sparc__) && defined(__arch64__)" ;;
+ sparc) echo "#if defined(__sparc__) && !defined(__arch64__)" ;;
+ s390) echo "#if defined(__s390__) && !defined(__s390x__)" ;;
+ mips64n32) echo "#if defined(__mips__) && defined(_ABIN32)" ;;
+ mips64) echo "#if defined(__mips__) && defined(_ABI64)" ;;
+ mipso32) echo "#if defined(__mips__) && defined(_ABIO32) && _MIPS_SZLONG == 32" ;;
+ parisc) echo "#ifdef __hppa__" ;;
+ loongarch64) echo "#ifdef __loongarch__" ;;
+ arm64) echo "#ifdef __aarch64__" ;;
+ *) echo "#ifdef __${arch}__" ;;
+ esac
+
+ while read -r line; do
+ set -- ${line}
+ syscall_nr="__NR_$1"
+ shift
+
+ echo "# ifndef ${syscall_nr}"
+ echo "# define ${syscall_nr} $*"
+ echo "# endif"
+ done <"${SCRIPT_DIR}/${arch}.in"
+ echo "#endif"
+ echo
+ ) >>${SYSCALLS_FILE}
+done <${SUPPORTED_ARCH}
+
+(
+ echo
+ echo "/* Common stubs */"
+ for num in $(awk '{print $1}' "${SCRIPT_DIR}/"*.in | sort -u); do
+ syscall_nr="__NR_${num}"
+
+ echo "# ifndef ${syscall_nr}"
+ echo "# define ${syscall_nr} __LTP__NR_INVALID_SYSCALL"
+ echo "# endif"
+ done
+ echo "#endif"
+) >>${SYSCALLS_FILE}
diff --git a/include/lapi/syscalls/regen.sh b/include/lapi/syscalls/regen.sh
deleted file mode 100755
index 663ce4458bbc67c3a9e8073dfe8359164a9da0ee..0000000000000000000000000000000000000000
--- a/include/lapi/syscalls/regen.sh
+++ /dev/null
@@ -1,129 +0,0 @@
-#!/bin/sh
-
-output="syscalls.h"
-rm -f "${output}".[1-9]*
-output_pid="${output}.$$"
-
-max_jobs=$(getconf _NPROCESSORS_ONLN 2>/dev/null)
-: ${max_jobs:=1}
-
-srcdir=${0%/*}
-
-err() {
- echo "$*" 1>&2
- exit 1
-}
-
-cat << EOF > "${output_pid}"
-/************************************************
- * GENERATED FILE: DO NOT EDIT/PATCH THIS FILE *
- * change your arch specific .in file instead *
- ************************************************/
-
-/*
- * Here we stick all the ugly *fallback* logic for linux
- * system call numbers (those __NR_ thingies).
- *
- * Licensed under the GPLv2 or later, see the COPYING file.
- */
-
-#ifndef LAPI_SYSCALLS_H__
-#define LAPI_SYSCALLS_H__
-
-#include <errno.h>
-#include <sys/syscall.h>
-#include <asm/unistd.h>
-
-#ifdef TST_TEST_H__
-#define TST_SYSCALL_BRK__(NR, SNR) ({ \\
- tst_brk(TCONF, \\
- "syscall(%d) " SNR " not supported on your arch", NR); \\
-})
-#else
-inline static void dummy_cleanup(void) {}
-
-#define TST_SYSCALL_BRK__(NR, SNR) ({ \\
- tst_brkm(TCONF, dummy_cleanup, \\
- "syscall(%d) " SNR " not supported on your arch", NR); \\
-})
-#endif
-
-#define tst_syscall(NR, ...) ({ \\
- intptr_t tst_ret; \\
- if (NR == __LTP__NR_INVALID_SYSCALL) { \\
- errno = ENOSYS; \\
- tst_ret = -1; \\
- } else { \\
- tst_ret = syscall(NR, ##__VA_ARGS__); \\
- } \\
- if (tst_ret == -1 && errno == ENOSYS) { \\
- TST_SYSCALL_BRK__(NR, #NR); \\
- } \\
- tst_ret; \\
-})
-
-EOF
-
-jobs=0
-for arch in $(cat "${srcdir}/order") ; do
- (
- echo "Generating data for arch $arch ... "
-
- (
- echo
- case ${arch} in
- sparc64) echo "#if defined(__sparc__) && defined(__arch64__)" ;;
- sparc) echo "#if defined(__sparc__) && !defined(__arch64__)" ;;
- s390) echo "#if defined(__s390__) && !defined(__s390x__)" ;;
- mips_n32) echo "#if defined(__mips__) && defined(_ABIN32)" ;;
- mips_n64) echo "#if defined(__mips__) && defined(_ABI64)" ;;
- mips_o32) echo "#if defined(__mips__) && defined(_ABIO32) && _MIPS_SZLONG == 32" ;;
- *) echo "#ifdef __${arch}__" ;;
- esac
- while read line ; do
- set -- ${line}
- nr="__NR_$1"
- shift
- if [ $# -eq 0 ] ; then
- err "invalid line found: $line"
- fi
- echo "# ifndef ${nr}"
- echo "# define ${nr} $*"
- echo "# endif"
- done < "${srcdir}/${arch}.in"
- echo "#endif"
- echo
- ) >> "${output_pid}.${arch}"
-
- ) &
-
- jobs=$(( jobs + 1 ))
- if [ ${jobs} -ge ${max_jobs} ] ; then
- wait || exit 1
- jobs=0
- fi
-done
-
-echo "Generating stub list ... "
-(
-echo
-echo "/* Common stubs */"
-echo "#define __LTP__NR_INVALID_SYSCALL -1" >> "${output_pid}"
-for nr in $(awk '{print $1}' "${srcdir}/"*.in | sort -u) ; do
- nr="__NR_${nr}"
- echo "# ifndef ${nr}"
- echo "# define ${nr} __LTP__NR_INVALID_SYSCALL"
- echo "# endif"
-done
-echo "#endif"
-) >> "${output_pid}._footer"
-
-wait || exit 1
-
-printf "Combining them all ... "
-for arch in $(cat "${srcdir}/order") _footer ; do
- cat "${output_pid}.${arch}"
-done >> "${output_pid}"
-mv "${output_pid}" "../${output}"
-rm -f "${output_pid}"*
-echo "OK!"
diff --git a/include/lapi/syscalls/order b/include/lapi/syscalls/supported-arch.txt
similarity index 100%
rename from include/lapi/syscalls/order
rename to include/lapi/syscalls/supported-arch.txt
--
2.43.0
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [LTP] [PATCH v7 1/4] Refactor regen.sh script to generate syscalls
2024-10-31 11:57 ` [LTP] [PATCH v7 1/4] Refactor regen.sh script to generate syscalls Andrea Cervesato
@ 2024-10-31 13:01 ` Petr Vorel
2024-10-31 13:18 ` Cyril Hrubis
2024-10-31 13:37 ` Andrea Cervesato via ltp
0 siblings, 2 replies; 13+ messages in thread
From: Petr Vorel @ 2024-10-31 13:01 UTC (permalink / raw)
To: Andrea Cervesato; +Cc: ltp
Hi Andrea,
> --- /dev/null
> +++ b/include/lapi/syscalls/generate_syscalls.sh
> @@ -0,0 +1,109 @@
> +#!/bin/sh -eux
I'm very sorry, I asked for "#!/bin/sh -eux", but better would be without "x".
"x" prints output of the commands which is too verbose on generate_syscalls.sh.
Could you please, when you merge, change it to "#!/bin/sh -eu"?
I'm not sure if it should stay in the other script.
This is on dash, but I suppose this can be reproducible on bash as well.
$ ./configure
...
+ syscall_nr=__NR_writev
+ echo # ifndef __NR_writev
+ echo # define __NR_writev __LTP__NR_INVALID_SYSCALL
+ echo # endif
+ echo #endif
$ ./configure > /tmp/out 2> /tmp/err; wc -l /tmp/out /tmp/err
on master:
290 /tmp/out
3 /tmp/err
293 total
on v7 (at least what I apply from https://patchwork.ozlabs.org/series/430633/mbox/):
289 /tmp/out
48706 /tmp/err
48995 total
Kind regards,
Petr
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [LTP] [PATCH v7 1/4] Refactor regen.sh script to generate syscalls
2024-10-31 13:01 ` Petr Vorel
@ 2024-10-31 13:18 ` Cyril Hrubis
2024-10-31 13:44 ` Petr Vorel
2024-10-31 13:37 ` Andrea Cervesato via ltp
1 sibling, 1 reply; 13+ messages in thread
From: Cyril Hrubis @ 2024-10-31 13:18 UTC (permalink / raw)
To: Petr Vorel; +Cc: ltp
Hi!
> $ ./configure
> ...
> + syscall_nr=__NR_writev
> + echo # ifndef __NR_writev
> + echo # define __NR_writev __LTP__NR_INVALID_SYSCALL
> + echo # endif
> + echo #endif
>
> $ ./configure > /tmp/out 2> /tmp/err; wc -l /tmp/out /tmp/err
>
> on master:
> 290 /tmp/out
> 3 /tmp/err
> 293 total
>
> on v7 (at least what I apply from https://patchwork.ozlabs.org/series/430633/mbox/):
> 289 /tmp/out
> 48706 /tmp/err
> 48995 total
The /tmp/out now looks fine, there is a single line difference, that is
the empty newline at the start of the file I asked to remove.
--
Cyril Hrubis
chrubis@suse.cz
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LTP] [PATCH v7 1/4] Refactor regen.sh script to generate syscalls
2024-10-31 13:18 ` Cyril Hrubis
@ 2024-10-31 13:44 ` Petr Vorel
0 siblings, 0 replies; 13+ messages in thread
From: Petr Vorel @ 2024-10-31 13:44 UTC (permalink / raw)
To: Cyril Hrubis; +Cc: ltp
> Hi!
> > $ ./configure
> > ...
> > + syscall_nr=__NR_writev
> > + echo # ifndef __NR_writev
> > + echo # define __NR_writev __LTP__NR_INVALID_SYSCALL
> > + echo # endif
> > + echo #endif
> > $ ./configure > /tmp/out 2> /tmp/err; wc -l /tmp/out /tmp/err
> > on master:
> > 290 /tmp/out
> > 3 /tmp/err
> > 293 total
> > on v7 (at least what I apply from https://patchwork.ozlabs.org/series/430633/mbox/):
> > 289 /tmp/out
> > 48706 /tmp/err
> > 48995 total
> The /tmp/out now looks fine, there is a single line difference, that is
> the empty newline at the start of the file I asked to remove.
OK, it can be something on the setup of the VM I'm using (some Debian), which
uses dash as /bin/sh. I was not able to reproduce it on openSUSE Tumbleweed.
Feel free to ignore it.
But I have other error for generate_arch.sh which I'll report on the other
patchset.
Kind regards,
Petr
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LTP] [PATCH v7 1/4] Refactor regen.sh script to generate syscalls
2024-10-31 13:01 ` Petr Vorel
2024-10-31 13:18 ` Cyril Hrubis
@ 2024-10-31 13:37 ` Andrea Cervesato via ltp
2024-10-31 14:08 ` Petr Vorel
1 sibling, 1 reply; 13+ messages in thread
From: Andrea Cervesato via ltp @ 2024-10-31 13:37 UTC (permalink / raw)
To: Petr Vorel, Andrea Cervesato; +Cc: ltp
Hi Petr,
On 10/31/24 14:01, Petr Vorel wrote:
> Hi Andrea,
>
>> --- /dev/null
>> +++ b/include/lapi/syscalls/generate_syscalls.sh
>> @@ -0,0 +1,109 @@
>> +#!/bin/sh -eux
> I'm very sorry, I asked for "#!/bin/sh -eux", but better would be without "x".
> "x" prints output of the commands which is too verbose on generate_syscalls.sh.
> Could you please, when you merge, change it to "#!/bin/sh -eu"?
No problem, I will do it before merge.
> I'm not sure if it should stay in the other script.
> This is on dash, but I suppose this can be reproducible on bash as well.
>
> $ ./configure
> ...
> + syscall_nr=__NR_writev
> + echo # ifndef __NR_writev
> + echo # define __NR_writev __LTP__NR_INVALID_SYSCALL
> + echo # endif
> + echo #endif
>
> $ ./configure > /tmp/out 2> /tmp/err; wc -l /tmp/out /tmp/err
Did you check the CI result I mentioned in the cover letter?
Andrea
>
> on master:
> 290 /tmp/out
> 3 /tmp/err
> 293 total
>
> on v7 (at least what I apply from https://patchwork.ozlabs.org/series/430633/mbox/):
> 289 /tmp/out
> 48706 /tmp/err
> 48995 total
>
> Kind regards,
> Petr
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LTP] [PATCH v7 1/4] Refactor regen.sh script to generate syscalls
2024-10-31 13:37 ` Andrea Cervesato via ltp
@ 2024-10-31 14:08 ` Petr Vorel
0 siblings, 0 replies; 13+ messages in thread
From: Petr Vorel @ 2024-10-31 14:08 UTC (permalink / raw)
To: Andrea Cervesato; +Cc: ltp
Hi Andrea,
> > > --- /dev/null
> > > +++ b/include/lapi/syscalls/generate_syscalls.sh
> > > @@ -0,0 +1,109 @@
> > > +#!/bin/sh -eux
> > I'm very sorry, I asked for "#!/bin/sh -eux", but better would be without "x".
> > "x" prints output of the commands which is too verbose on generate_syscalls.sh.
> > Could you please, when you merge, change it to "#!/bin/sh -eu"?
> No problem, I will do it before merge.
+1
> > I'm not sure if it should stay in the other script.
> > This is on dash, but I suppose this can be reproducible on bash as well.
> > $ ./configure
> > ...
> > + syscall_nr=__NR_writev
> > + echo # ifndef __NR_writev
> > + echo # define __NR_writev __LTP__NR_INVALID_SYSCALL
> > + echo # endif
> > + echo #endif
> > $ ./configure > /tmp/out 2> /tmp/err; wc -l /tmp/out /tmp/err
> Did you check the CI result I mentioned in the cover letter?
Yes, I did. Sure, it can be a problem with my setup (or it can be a bug in a
script which is visible only on certain setup).
Kind regards,
Petr
> Andrea
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 13+ messages in thread
* [LTP] [PATCH v7 2/4] Add script to generate arch(s) dependant syscalls
2024-10-31 11:57 [LTP] [PATCH v7 0/4] Automatically generate syscalls.h Andrea Cervesato
2024-10-31 11:57 ` [LTP] [PATCH v7 1/4] Refactor regen.sh script to generate syscalls Andrea Cervesato
@ 2024-10-31 11:57 ` Andrea Cervesato
2024-10-31 13:58 ` Petr Vorel
2024-10-31 11:57 ` [LTP] [PATCH v7 3/4] Delete obsolete strip_syscall.awk file Andrea Cervesato
2024-10-31 11:57 ` [LTP] [PATCH v7 4/4] Add documentation about syscalls.h generator Andrea Cervesato
3 siblings, 1 reply; 13+ messages in thread
From: Andrea Cervesato @ 2024-10-31 11:57 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
Add generate_arch.sh script which can be used to generate arch(s)
dependant syscalls file. The way it works is pretty simple: for each
architecture defined into supported-arch.txt, compile kernel headers,
extract the list of syscalls and generate a .in file containing all of
them, associated with their own syscall's number.
The way syscalls files are generated, passes through a C application
which is automatically checking the availability of the syscalls in
the user space environment.
Reviewed-by: Li Wang <liwang@redhat.com>
Reviewed-by: Cyril Hrubis <chrubis@suse.cz>
Reviewed-by: Petr Vorel <pvorel@suse.cz>
Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
include/lapi/syscalls/{aarch64.in => arm64.in} | 0
include/lapi/syscalls/generate_arch.sh | 211 +++++++++++++++++++++
.../lapi/syscalls/{loongarch.in => loongarch64.in} | 0
include/lapi/syscalls/{mips_n64.in => mips64.in} | 0
.../lapi/syscalls/{mips_n32.in => mips64n32.in} | 0
include/lapi/syscalls/{mips_o32.in => mipso32.in} | 0
include/lapi/syscalls/{hppa.in => parisc.in} | 0
include/lapi/syscalls/supported-arch.txt | 12 +-
8 files changed, 217 insertions(+), 6 deletions(-)
diff --git a/include/lapi/syscalls/aarch64.in b/include/lapi/syscalls/arm64.in
similarity index 100%
rename from include/lapi/syscalls/aarch64.in
rename to include/lapi/syscalls/arm64.in
diff --git a/include/lapi/syscalls/generate_arch.sh b/include/lapi/syscalls/generate_arch.sh
new file mode 100755
index 0000000000000000000000000000000000000000..c88ca945127c3c26162a48babcd6b0f8ca552311
--- /dev/null
+++ b/include/lapi/syscalls/generate_arch.sh
@@ -0,0 +1,211 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# This is an adaptation of the update-tables.sh script, included in the
+# syscalls-table project (https://github.com/hrw/syscalls-table) and released
+# under the MIT license.
+#
+# Author: Andrea Cervesato <andrea.cervesato@suse.com>
+
+KERNELSRC="$1"
+
+# to keep sorting in order
+export LC_ALL=C
+
+if [ -z "$KERNELSRC" ]; then
+ echo "Please provide kernel sources:"
+ echo ""
+ echo "$0 path/to/Linux/kernel/sources"
+ echo ""
+ exit 1
+fi
+
+if [ ! -d "$KERNELSRC" ]; then
+ echo "$KERNELSRC is not a directory"
+ exit 1
+fi
+
+if [ ! -e "${KERNELSRC}/Makefile" ]; then
+ echo "No Makefile in $KERNELSRC directory"
+ exit 1
+fi
+
+TEMP="$(mktemp -d)"
+KVER="$(make -C ${KERNELSRC} kernelversion -s)"
+
+SCRIPT_DIR="$(realpath $(dirname "$0"))"
+SUPPORTED_ARCH="${SCRIPT_DIR}/supported-arch.txt"
+LINUX_HEADERS="${TEMP}/headers"
+
+grab_syscall_names_from_tables()
+{
+ for tbl_file in $(find ${KERNELSRC}/arch -name syscall*.tbl); do
+ grep -E -v "(^#|^$|sys_ni_syscall)" $tbl_file |
+ awk '{ print $3 }' >> ${TEMP}/syscall-names.tosort
+ done
+
+ drop_bad_entries
+}
+
+grab_syscall_names_from_unistd_h()
+{
+ grep -E -h "^#define __NR_" \
+ ${LINUX_HEADERS}/usr/include/asm/unistd*.h \
+ ${LINUX_HEADERS}/usr/include/asm-generic/unistd.h \
+ >${TEMP}/syscall-names.tosort
+
+ drop_bad_entries
+}
+
+drop_bad_entries()
+{
+ grep -E -v "(unistd.h|NR3264|__NR_syscall|__SC_COMP|__NR_.*Linux|__NR_FAST)" \
+ ${TEMP}/syscall-names.tosort |
+ grep -E -v "(__SYSCALL|SYSCALL_BASE|SYSCALL_MASK)" |
+ sed -e "s/#define\s*__NR_//g" -e "s/\s.*//g" |
+ sort -u >${TEMP}/syscall-names.txt
+}
+
+generate_table()
+{
+ echo "- $arch"
+
+ if [ $bits == 32 ]; then
+ extraflags="${extraflags} -D__BITS_PER_LONG=32"
+ fi
+
+ local uppercase_arch=$(echo "$arch" | tr '[:lower:]' '[:upper:]')
+
+ gcc ${TEMP}/list-syscalls.c -U__LP64__ -U__ILP32__ -U__i386__ \
+ -D${uppercase_arch} \
+ -D__${arch}__ ${extraflags} \
+ -I ${LINUX_HEADERS}/usr/include/ \
+ -o ${TEMP}/list-syscalls &>/dev/null
+
+ ${TEMP}/list-syscalls > "${TEMP}/${arch}.in.tosort"
+
+ sort -k2,2n "${TEMP}/${arch}.in.tosort" > "${TEMP}/${arch}.in"
+}
+
+generate_list_syscalls_c()
+{
+ (
+ echo
+ echo "
+ #include <stdio.h>
+ #include <asm/unistd.h>
+
+ int main(void)
+ {
+ "
+ for syscall in $(cat ${TEMP}/syscall-names.txt); do
+ echo "
+ #ifdef __NR_$syscall
+ printf(\"$syscall %d\\n\", __NR_$syscall);
+ #endif
+ "
+ done
+ echo " return 0;
+ }"
+ ) > ${TEMP}/list-syscalls.c
+}
+
+export_headers()
+{
+ make -s -C ${KERNELSRC} ARCH=${arch} O=${LINUX_HEADERS} \
+ headers_install &>/dev/null
+}
+
+do_all_tables()
+{
+ for archdir in ${KERNELSRC}/arch/*; do
+ arch=$(basename $archdir)
+
+ bits=64
+ extraflags=
+
+ case ${arch} in
+ Kconfig)
+ continue
+ ;;
+ um)
+ continue
+ ;;
+ esac
+
+ export_headers
+ grab_syscall_names_from_unistd_h
+
+ case ${arch} in
+ arm)
+ bits=32
+ arch=armoabi extraflags= generate_table
+ arch=arm extraflags=-D__ARM_EABI__ generate_table
+ ;;
+ loongarch)
+ # 32-bit variant of loongarch may appear
+ arch=loongarch64 extraflags=-D_LOONGARCH_SZLONG=64 generate_table
+ ;;
+ mips)
+ arch=mips64 extraflags=-D_MIPS_SIM=_MIPS_SIM_ABI64 generate_table
+ bits=32
+ arch=mipso32 extraflags=-D_MIPS_SIM=_MIPS_SIM_ABI32 generate_table
+ arch=mips64n32 extraflags=-D_MIPS_SIM=_MIPS_SIM_NABI32 generate_table
+ ;;
+ powerpc)
+ generate_table
+ arch=powerpc64 generate_table
+ ;;
+ riscv)
+ arch=riscv64 extraflags=-D__LP64__ generate_table
+ bits=32
+ arch=riscv32 extraflags=-D__SIZEOF_POINTER__=4 generate_table
+ ;;
+ s390)
+ bits=32
+ generate_table
+ bits=64
+ arch=s390x generate_table
+ ;;
+ sparc)
+ bits=32
+ extraflags=-D__32bit_syscall_numbers__ generate_table
+ bits=64
+ arch=sparc64 extraflags=-D__arch64__ generate_table
+ ;;
+ x86)
+ arch=x86_64 extraflags=-D__LP64__ generate_table
+ bits=32
+ arch=i386 generate_table
+ arch=x32 extraflags=-D__ILP32__ generate_table
+ ;;
+ arc | csky | hexagon | m68k | microblaze | nios2 | openrisc | sh | xtensa)
+ bits=32 generate_table
+ ;;
+ *)
+ generate_table
+ ;;
+ esac
+ done
+}
+
+copy_supported_arch()
+{
+ while IFS= read -r arch; do
+ if [ -f "${TEMP}/${arch}.in" ]; then
+ echo "- ${arch}"
+ cp "${TEMP}/${arch}.in" "${SCRIPT_DIR}/${arch}.in"
+ fi
+ done < ${SUPPORTED_ARCH}
+}
+
+echo "Temporary directory ${TEMP}"
+echo "Extracting syscalls"
+
+grab_syscall_names_from_tables
+generate_list_syscalls_c
+
+do_all_tables
+
+echo "Copying supported syscalls"
+copy_supported_arch
diff --git a/include/lapi/syscalls/loongarch.in b/include/lapi/syscalls/loongarch64.in
similarity index 100%
rename from include/lapi/syscalls/loongarch.in
rename to include/lapi/syscalls/loongarch64.in
diff --git a/include/lapi/syscalls/mips_n64.in b/include/lapi/syscalls/mips64.in
similarity index 100%
rename from include/lapi/syscalls/mips_n64.in
rename to include/lapi/syscalls/mips64.in
diff --git a/include/lapi/syscalls/mips_n32.in b/include/lapi/syscalls/mips64n32.in
similarity index 100%
rename from include/lapi/syscalls/mips_n32.in
rename to include/lapi/syscalls/mips64n32.in
diff --git a/include/lapi/syscalls/mips_o32.in b/include/lapi/syscalls/mipso32.in
similarity index 100%
rename from include/lapi/syscalls/mips_o32.in
rename to include/lapi/syscalls/mipso32.in
diff --git a/include/lapi/syscalls/hppa.in b/include/lapi/syscalls/parisc.in
similarity index 100%
rename from include/lapi/syscalls/hppa.in
rename to include/lapi/syscalls/parisc.in
diff --git a/include/lapi/syscalls/supported-arch.txt b/include/lapi/syscalls/supported-arch.txt
index c18aa38cf4546cdf3ac8c89a45bd1b202ffa7711..c5c5191ac08482d89f0a8c39dfae936538f58f7f 100644
--- a/include/lapi/syscalls/supported-arch.txt
+++ b/include/lapi/syscalls/supported-arch.txt
@@ -1,13 +1,13 @@
-aarch64
arc
+arm64
arm
-hppa
i386
ia64
-loongarch
-mips_n32
-mips_n64
-mips_o32
+loongarch64
+mips64n32
+mips64
+mipso32
+parisc
powerpc64
powerpc
s390x
--
2.43.0
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [LTP] [PATCH v7 2/4] Add script to generate arch(s) dependant syscalls
2024-10-31 11:57 ` [LTP] [PATCH v7 2/4] Add script to generate arch(s) dependant syscalls Andrea Cervesato
@ 2024-10-31 13:58 ` Petr Vorel
2024-10-31 14:52 ` Andrea Cervesato via ltp
0 siblings, 1 reply; 13+ messages in thread
From: Petr Vorel @ 2024-10-31 13:58 UTC (permalink / raw)
To: Andrea Cervesato; +Cc: ltp
Hi Andrea,
> --- /dev/null
> +++ b/include/lapi/syscalls/generate_arch.sh
> @@ -0,0 +1,211 @@
> +#!/bin/sh
Also here would be nice to use -eu (I know it's a script originated from
elsewhere)...
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +#
> +# This is an adaptation of the update-tables.sh script, included in the
> +# syscalls-table project (https://github.com/hrw/syscalls-table) and released
> +# under the MIT license.
> +#
> +# Author: Andrea Cervesato <andrea.cervesato@suse.com>
> +
> +KERNELSRC="$1"
... but that would require instead of this to check like this before using $1:
if [ $# -eq 0 ]
echo "Please provide kernel sources:"
echo ""
echo "$0 path/to/Linux/kernel/sources"
echo ""
exit 1
fi
Also running with dash I get failures:
$ ./include/lapi/syscalls/generate_arch.sh ../linux
...
grep: /tmp/tmp.sVfa08a9De/headers/usr/include/asm/unistd*.h: No such file or directory
grep: /tmp/tmp.sVfa08a9De/headers/usr/include/asm-generic/unistd.h: No such file or directory
./include/lapi/syscalls/generate_arch.sh: 73: [: 64: unexpected operator
./include/lapi/syscalls/generate_arch.sh: 85: /tmp/tmp.sVfa08a9De/list-syscalls: not found
grep: /tmp/tmp.sVfa08a9De/headers/usr/include/asm/unistd*.h: No such file or directory
grep: /tmp/tmp.sVfa08a9De/headers/usr/include/asm-generic/unistd.h: No such file or directory
./include/lapi/syscalls/generate_arch.sh: 73: [: 32: unexpected operator
./include/lapi/syscalls/generate_arch.sh: 85: /tmp/tmp.sVfa08a9De/list-syscalls: not found
grep: /tmp/tmp.sVfa08a9De/headers/usr/include/asm/unistd*.h: No such file or directory
grep: /tmp/tmp.sVfa08a9De/headers/usr/include/asm-generic/unistd.h: No such file or directory
./include/lapi/syscalls/generate_arch.sh: 73: [: 32: unexpected operator
/tmp/tmp.sVfa08a9De/list-syscalls.c: In function 'main':
/tmp/tmp.sVfa08a9De/list-syscalls.c:11:32: warning: missing terminating " character
11 | printf("_llseek %d
| ^
/tmp/tmp.sVfa08a9De/list-syscalls.c:12:1: warning: missing terminating " character
12 | ", __NR__llseek);
| ^
That fails in generate_table on:
if [ $bits == 32 ]; then
When I use "!/bin/sh -eux" I see:
make -s -C /home/foo/sources/linux ARCH=alpha O=/tmp/tmp.G3HzE6EnFu/headers headers_install
grep: /tmp/tmp.G3HzE6EnFu/headers/usr/include/asm/unistd*.h: No such file or directory
This looks like it requires to have build kernel before running this script,
right? (I have 6.12-rc4 in the kernel directory). As a result of running the
script are empty include/lapi/syscalls/*.in files.
Again, it can be a combination of some problem with my environment (some
variable set or shell setup) + dash as /bin/sh. Changing script to /bin/bash
fixes everything. It would be nice if anybody test it with installed dash and
!#/bin/dash in both scripts.
And successful run changes include/lapi/syscalls/arm.in:
+++ include/lapi/syscalls/arm.in
@@ -1,406 +1,415 @@
-restart_syscall (__NR_SYSCALL_BASE+ 0)
-exit (__NR_SYSCALL_BASE+ 1)
-fork (__NR_SYSCALL_BASE+ 2)
-read (__NR_SYSCALL_BASE+ 3)
...
+restart_syscall 0
+exit 1
+fork 2
+read 3
+write 4
Maybe we could update syscalls with generate_arch.sh after we merge this?
Kind regards,
Petr
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [LTP] [PATCH v7 2/4] Add script to generate arch(s) dependant syscalls
2024-10-31 13:58 ` Petr Vorel
@ 2024-10-31 14:52 ` Andrea Cervesato via ltp
2024-10-31 15:31 ` pvorel
0 siblings, 1 reply; 13+ messages in thread
From: Andrea Cervesato via ltp @ 2024-10-31 14:52 UTC (permalink / raw)
To: Petr Vorel, Andrea Cervesato; +Cc: ltp
The kernel doesn't need to be compiled. That's done by the script via
`make headers` command. I don't know why that is happening honestly.
On 10/31/24 14:58, Petr Vorel wrote:
> That fails in generate_table on:
> if [ $bits == 32 ]; then
That's because the right syntax should be (for bash):
if [[ "$bits" == "32" ]]; then
But in other shells sh compatible, probably I should use:
if [ "$bits" -eq "32" ]; then
I'm gonna fix this.
Andrea
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LTP] [PATCH v7 2/4] Add script to generate arch(s) dependant syscalls
2024-10-31 14:52 ` Andrea Cervesato via ltp
@ 2024-10-31 15:31 ` pvorel
0 siblings, 0 replies; 13+ messages in thread
From: pvorel @ 2024-10-31 15:31 UTC (permalink / raw)
To: Andrea Cervesato; +Cc: ltp
On 2024-10-31 15:52, Andrea Cervesato wrote:
> The kernel doesn't need to be compiled. That's done by the script via
> `make headers` command. I don't know why that is happening honestly.
>
> On 10/31/24 14:58, Petr Vorel wrote:
>> That fails in generate_table on:
>> if [ $bits == 32 ]; then
>
> That's because the right syntax should be (for bash):
>
> if [[ "$bits" == "32" ]]; then
Ah, I should have caught this, I overlooked an obvious bashism. thanks
for finding the problem.
I actually run checkbashisms before, but only when run with -f finds a
problems:
$ checkbashisms -f include/lapi/syscalls/generate_arch.sh
possible bashism in include/lapi/syscalls/generate_arch.sh line 73
(should be 'b = a'):
if [ $bits == 32 ]; then
possible bashism in include/lapi/syscalls/generate_arch.sh line 83
(should be >word 2>&1):
gcc ${TEMP}/list-syscalls.c -U__LP64__ -U__ILP32__ -U__i386__ \
-D${uppercase_arch} \
-D__${arch}__ ${extraflags} \
-I ${LINUX_HEADERS}/usr/include/ \
-o ${TEMP}/list-syscalls &>/dev/null
possible bashism in include/lapi/syscalls/generate_arch.sh line 116
(should be >word 2>&1):
make -s -C ${KERNELSRC} ARCH=${arch} O=${LINUX_HEADERS} \
headers_install &>/dev/null
"&>" is yet another bashism.
Ideally, not only checking with "checkbashisms -f", but also running
script with dash (e.g. change shebang to #!/bin/dash) should catch most
of the problems.
>
> But in other shells sh compatible, probably I should use:
>
> if [ "$bits" -eq "32" ]; then
Yes please.
>
>
> I'm gonna fix this.
Thanks a lot!
Kind regards,
Petr
>
> Andrea
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 13+ messages in thread
* [LTP] [PATCH v7 3/4] Delete obsolete strip_syscall.awk file
2024-10-31 11:57 [LTP] [PATCH v7 0/4] Automatically generate syscalls.h Andrea Cervesato
2024-10-31 11:57 ` [LTP] [PATCH v7 1/4] Refactor regen.sh script to generate syscalls Andrea Cervesato
2024-10-31 11:57 ` [LTP] [PATCH v7 2/4] Add script to generate arch(s) dependant syscalls Andrea Cervesato
@ 2024-10-31 11:57 ` Andrea Cervesato
2024-10-31 11:57 ` [LTP] [PATCH v7 4/4] Add documentation about syscalls.h generator Andrea Cervesato
3 siblings, 0 replies; 13+ messages in thread
From: Andrea Cervesato @ 2024-10-31 11:57 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
Reviewed-by: Li Wang <liwang@redhat.com>
Reviewed-by: Petr Vorel <pvorel@suse.cz>
Reviewed-by: Cyril Hrubis <chrubis@suse.cz>
Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
include/lapi/syscalls/strip_syscall.awk | 19 -------------------
1 file changed, 19 deletions(-)
diff --git a/include/lapi/syscalls/strip_syscall.awk b/include/lapi/syscalls/strip_syscall.awk
deleted file mode 100755
index e8dff422e2667745c144ed984a4d66461fcce0c6..0000000000000000000000000000000000000000
--- a/include/lapi/syscalls/strip_syscall.awk
+++ /dev/null
@@ -1,19 +0,0 @@
-#!/usr/bin/awk -f
-#
-# Dumb script that can be used to strip all of the syscall information from
-# the arch-respective unistd*.h.
-#
-# Examples:
-#
-# 1. Grab the i386 32-bit syscalls from unistd_32.h and put them in i386.in
-# strip_syscall.awk arch/x86/include/asm/unistd_32.h > i386.in
-#
-
-/^#define[[:space:]]+__NR_[0-9a-z]+/ {
-
- sub (/#define[[:space:]]+__NR_/, "", $0);
- sub (/[[:space:]]*(\/\*.*)/, "", $0);
- sub (/[[:space:]]+/, " ", $0);
-
- print
-}
--
2.43.0
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 13+ messages in thread* [LTP] [PATCH v7 4/4] Add documentation about syscalls.h generator
2024-10-31 11:57 [LTP] [PATCH v7 0/4] Automatically generate syscalls.h Andrea Cervesato
` (2 preceding siblings ...)
2024-10-31 11:57 ` [LTP] [PATCH v7 3/4] Delete obsolete strip_syscall.awk file Andrea Cervesato
@ 2024-10-31 11:57 ` Andrea Cervesato
3 siblings, 0 replies; 13+ messages in thread
From: Andrea Cervesato @ 2024-10-31 11:57 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
doc/developers/test_case_tutorial.rst | 27 ++++++++++++++-------------
1 file changed, 14 insertions(+), 13 deletions(-)
diff --git a/doc/developers/test_case_tutorial.rst b/doc/developers/test_case_tutorial.rst
index be9a0ea8bac1d690b4ed4879d110f87c3f2a381a..b48ea4a7a3594266466fcca364f5d0ff1e42d235 100644
--- a/doc/developers/test_case_tutorial.rst
+++ b/doc/developers/test_case_tutorial.rst
@@ -302,24 +302,25 @@ for a distribution's C library version to be older than its kernel or it may use
cut down C library in comparison to the GNU one. So we must call ``statx()``
using the general ``syscall()`` interface.
-The LTP contains a library for dealing with the ``syscall`` interface, which is
+LTP contains a library for dealing with the ``syscall`` interface, which is
located in :master:`include/lapi`. System call numbers are listed against the relevant
call in the ``*.in`` files (e.g. ``x86_64.in``) which are used to generate
-``syscalls.h``, which is the header you should include. On rare occasions you
-may find the system call number is missing from the ``*.in`` files and will need
-to add it (see :master:`include/lapi/syscalls/strip_syscall.awk`).
+``syscalls.h``, the header you should include.
-System call numbers vary between architectures, hence there are multiple
-``*.in`` files for each architecture. You can find the various values for the
-``statx`` system call across a number of ``unistd.h`` files in the Linux kernel.
+On rare occasions, you may find that system call number is missing from ``*.in``
+files. In these cases, they will need to be updated using
+`include/lapi/syscalls/generate_arch.sh` script as following:
-Note that we don't use the system-call-identifier value available in
-``/usr/include/linux/uinstd.h`` because the kernel might be much newer than the
-user land development packages.
+.. code-block:: bash
+
+ $ include/lapi/syscalls/generate_arch.sh /path/of/linux/sources
+
+The script will generate all the needed ``*.in`` files according to the Linux
+source code which has been used. Make sure that your Linux source code has
+been updated to the latest version.
-For ``statx`` we had to add ``statx 332`` to :master:`include/lapi/syscalls/x86_64.in`,
-``statx 383`` to :master:`include/lapi/syscalls/powerpc.in`, etc. Now lets look at
-the code, which I will explain in more detail further down.
+Once the new syscalls files have been updated, to rebuild our ``syscalls.h``
+file, please use ``./configure`` command.
.. code-block:: c
--
2.43.0
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 13+ messages in thread