* [RFC v3 01/10] docs: Add specification for native library calls
2023-06-25 21:26 [RFC v3 00/10] Native Library Calls Yeqi Fu
@ 2023-06-25 21:26 ` Yeqi Fu
2023-07-03 15:04 ` Alex Bennée
2023-06-25 21:26 ` [RFC v3 02/10] build: Add configure options for native calls Yeqi Fu
` (8 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: Yeqi Fu @ 2023-06-25 21:26 UTC (permalink / raw)
To: alex.bennee; +Cc: richard.henderson, qemu-devel, Yeqi Fu
Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
---
docs/native_calls.txt | 70 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 70 insertions(+)
create mode 100644 docs/native_calls.txt
diff --git a/docs/native_calls.txt b/docs/native_calls.txt
new file mode 100644
index 0000000000..8906566b13
--- /dev/null
+++ b/docs/native_calls.txt
@@ -0,0 +1,70 @@
+Native Library Calls Optimization for QEMU Linux-User
+====================
+
+Description
+===========
+When running under the linux-user mode in QEMU, the entire program,
+including all library calls, is translated. Many well-understood
+library functions are usually optimized for the processor they run
+on. For example, the semantics of memcpy are well-defined and
+optimized. Instead of translating these library functions, we can
+call their native versions, as the runtime of library functions
+is generally biased towards a few core functions. Thus, only a
+small subset of functions (such as mem* and str*) would need to
+be hooked to be useful.
+
+
+Implementation
+==============
+This feature introduces a set of specialized instructions for native
+calls and provides helpers to translate these instructions to
+corresponding native functions. A shared library is also implemented,
+where native functions are rewritten as specialized instructions.
+At runtime, user programs load the shared library, and specialized
+instructions are executed when native functions are called.
+
+The specialized instructions are implemented using architecture-
+specific macros. These macros utilize unused or invalid opcodes or
+instruction fields to embed the necessary information for native
+function calls. This approach ensures that the specialized
+instructions do not conflict with existing instructions.
+
+For x86 and x86_64, the implementation uses an unused opcode.
+For arm and aarch64, the HLT instruction is used, as it is invalid in
+userspace and has 16 bits of spare immediate data.
+For mips and mips64, the implementation takes advantage of unused
+bytes in the syscall instruction.
+
+Supported Architectures
+=======================
+This feature is applicable to user programs with the following
+architectures now:
+- x86
+- x86_64
+- arm
+- aarch64
+- mips
+- mips64
+
+
+Usage
+=====
+1. Install Cross-Compilation Tools
+Cross-compilation tools are required to build the shared libraries
+that can hook the necessary library functions. For example, a viable
+command on Ubuntu is:
+```
+apt install libc6:i386 gcc-arm-linux-gnueabihf \
+gcc-aarch64-linux-gnu gcc-mips-linux-gnu gcc-mips64-linux-gnuabi64
+```
+2. Locate the Compiled libnative.so
+After compilation, the libnative.so file can be found in the
+`./build/common-user/native/<target>-linux-user` directory.
+
+3. Run the Program with the `--native-bypass` Option
+To run your program with native library bypass, use the
+`--native-bypass` option to import libnative.so:
+```
+./build/qemu-<target> --native-bypass \
+./build/common-user/native/<target>-linux-user/libnative.so ./program
+```
--
2.34.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [RFC v3 01/10] docs: Add specification for native library calls
2023-06-25 21:26 ` [RFC v3 01/10] docs: Add specification for native library calls Yeqi Fu
@ 2023-07-03 15:04 ` Alex Bennée
0 siblings, 0 replies; 20+ messages in thread
From: Alex Bennée @ 2023-07-03 15:04 UTC (permalink / raw)
To: Yeqi Fu; +Cc: richard.henderson, qemu-devel
Yeqi Fu <fufuyqqqqqq@gmail.com> writes:
> Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
> ---
> docs/native_calls.txt | 70 +++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 70 insertions(+)
> create mode 100644 docs/native_calls.txt
>
> diff --git a/docs/native_calls.txt b/docs/native_calls.txt
> new file mode 100644
> index 0000000000..8906566b13
> --- /dev/null
> +++ b/docs/native_calls.txt
We try not to add plain text files to docs now. The preferred format is
rst so we can render the docs:
https://qemu.readthedocs.io/en/latest/
I would suggest putting this in docs/user/native_calls.rst and adding a
reference in docs/user/index.rst
> @@ -0,0 +1,70 @@
> +Native Library Calls Optimization for QEMU Linux-User
> +====================
> +
> +Description
> +===========
newline and also for rst a different underline is needed to make
description a sub-heading (e.g. -----------)
> +When running under the linux-user mode in QEMU, the entire program,
> +including all library calls, is translated. Many well-understood
> +library functions are usually optimized for the processor they run
> +on.
Maybe instead of "Many..."
Because many library functions are tuned specifically for the guest
architecture the result of translating them will not be as efficient
as it could be.
When the semantics of a library function are well defined we can take
advantage of that fact by replacing the translated version with a call
to the native equivalent function. As the runtime of library functions....
> For example, the semantics of memcpy are well-defined and
> +optimized. Instead of translating these library functions, we can
> +call their native versions, as the runtime of library functions
> +is generally biased towards a few core functions. Thus, only a
> +small subset of functions (such as mem* and str*) would need to
> +be hooked to be useful.
> +
> +
> +Implementation
> +==============
newline
> +This feature introduces a set of specialized instructions for native
> +calls and provides helpers to translate these instructions to
> +corresponding native functions. A shared library is also implemented,
> +where native functions are rewritten as specialized instructions.
> +At runtime, user programs load the shared library, and specialized
> +instructions are executed when native functions are called.
I would switch it around. Describe the LD_PRELOAD library first and then
describe how the library uses a special sequence of instructions to
encode the ABI data in a way the translator can pick it up during
execution.
> +
> +The specialized instructions are implemented using architecture-
> +specific macros. These macros utilize unused or invalid opcodes or
> +instruction fields to embed the necessary information for native
> +function calls. This approach ensures that the specialized
> +instructions do not conflict with existing instructions.
> +
> +For x86 and x86_64, the implementation uses an unused opcode.
> +For arm and aarch64, the HLT instruction is used, as it is invalid in
> +userspace and has 16 bits of spare immediate data.
> +For mips and mips64, the implementation takes advantage of unused
> +bytes in the syscall instruction.
I think this paragraph can be split into each architecture and listed as
a sub-heading for each. This can replace the "Supported Architectures" heading.
> +
> +Supported Architectures
> +=======================
> +This feature is applicable to user programs with the following
> +architectures now:
> +- x86
> +- x86_64
> +- arm
> +- aarch64
> +- mips
> +- mips64
> +
> +
> +Usage
> +=====
> +1. Install Cross-Compilation Tools
> +Cross-compilation tools are required to build the shared libraries
> +that can hook the necessary library functions. For example, a viable
> +command on Ubuntu is:
> +```
> +apt install libc6:i386 gcc-arm-linux-gnueabihf \
> +gcc-aarch64-linux-gnu gcc-mips-linux-gnu gcc-mips64-linux-gnuabi64
> +```
I guess libc6:i386 is there because the default x86 host compiler
already supports building 32 bit but we just need the lib bits?
> +2. Locate the Compiled libnative.so
> +After compilation, the libnative.so file can be found in the
> +`./build/common-user/native/<target>-linux-user` directory.
> +
> +3. Run the Program with the `--native-bypass` Option
> +To run your program with native library bypass, use the
> +`--native-bypass` option to import libnative.so:
> +```
> +./build/qemu-<target> --native-bypass \
You can drop the ./build here as there should be a qemu-<target> in the
top level build directory.
> +./build/common-user/native/<target>-linux-user/libnative.so ./program
> +```
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 20+ messages in thread
* [RFC v3 02/10] build: Add configure options for native calls
2023-06-25 21:26 [RFC v3 00/10] Native Library Calls Yeqi Fu
2023-06-25 21:26 ` [RFC v3 01/10] docs: Add specification for native library calls Yeqi Fu
@ 2023-06-25 21:26 ` Yeqi Fu
2023-07-03 15:22 ` Alex Bennée
2023-06-25 21:27 ` [RFC v3 03/10] build: Implement libnative library and configure options Yeqi Fu
` (7 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: Yeqi Fu @ 2023-06-25 21:26 UTC (permalink / raw)
To: alex.bennee
Cc: richard.henderson, qemu-devel, Yeqi Fu, Laurent Vivier,
Philippe Mathieu-Daudé, Jiaxun Yang
Add CONFIG_NATIVE_CALL to architectures supporting native calls.
Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
---
configs/targets/aarch64-linux-user.mak | 1 +
configs/targets/arm-linux-user.mak | 1 +
configs/targets/i386-linux-user.mak | 1 +
configs/targets/mips-linux-user.mak | 1 +
configs/targets/mips64-linux-user.mak | 1 +
configs/targets/x86_64-linux-user.mak | 1 +
6 files changed, 6 insertions(+)
diff --git a/configs/targets/aarch64-linux-user.mak b/configs/targets/aarch64-linux-user.mak
index ba8bc5fe3f..5a8fd98cd9 100644
--- a/configs/targets/aarch64-linux-user.mak
+++ b/configs/targets/aarch64-linux-user.mak
@@ -4,3 +4,4 @@ TARGET_XML_FILES= gdb-xml/aarch64-core.xml gdb-xml/aarch64-fpu.xml gdb-xml/aarch
TARGET_HAS_BFLT=y
CONFIG_SEMIHOSTING=y
CONFIG_ARM_COMPATIBLE_SEMIHOSTING=y
+CONFIG_NATIVE_CALL=y
diff --git a/configs/targets/arm-linux-user.mak b/configs/targets/arm-linux-user.mak
index 7f5d65794c..f934fb82da 100644
--- a/configs/targets/arm-linux-user.mak
+++ b/configs/targets/arm-linux-user.mak
@@ -5,3 +5,4 @@ TARGET_XML_FILES= gdb-xml/arm-core.xml gdb-xml/arm-vfp.xml gdb-xml/arm-vfp3.xml
TARGET_HAS_BFLT=y
CONFIG_SEMIHOSTING=y
CONFIG_ARM_COMPATIBLE_SEMIHOSTING=y
+CONFIG_NATIVE_CALL=y
diff --git a/configs/targets/i386-linux-user.mak b/configs/targets/i386-linux-user.mak
index 5b2546a430..2d8bca8f93 100644
--- a/configs/targets/i386-linux-user.mak
+++ b/configs/targets/i386-linux-user.mak
@@ -2,3 +2,4 @@ TARGET_ARCH=i386
TARGET_SYSTBL_ABI=i386
TARGET_SYSTBL=syscall_32.tbl
TARGET_XML_FILES= gdb-xml/i386-32bit.xml
+CONFIG_NATIVE_CALL=y
diff --git a/configs/targets/mips-linux-user.mak b/configs/targets/mips-linux-user.mak
index b4569a9893..fa005d487a 100644
--- a/configs/targets/mips-linux-user.mak
+++ b/configs/targets/mips-linux-user.mak
@@ -3,3 +3,4 @@ TARGET_ABI_MIPSO32=y
TARGET_SYSTBL_ABI=o32
TARGET_SYSTBL=syscall_o32.tbl
TARGET_BIG_ENDIAN=y
+CONFIG_NATIVE_CALL=y
diff --git a/configs/targets/mips64-linux-user.mak b/configs/targets/mips64-linux-user.mak
index d2ff509a11..ecfe6bcf73 100644
--- a/configs/targets/mips64-linux-user.mak
+++ b/configs/targets/mips64-linux-user.mak
@@ -4,3 +4,4 @@ TARGET_BASE_ARCH=mips
TARGET_SYSTBL_ABI=n64
TARGET_SYSTBL=syscall_n64.tbl
TARGET_BIG_ENDIAN=y
+CONFIG_NATIVE_CALL=y
diff --git a/configs/targets/x86_64-linux-user.mak b/configs/targets/x86_64-linux-user.mak
index 9ceefbb615..a53b017454 100644
--- a/configs/targets/x86_64-linux-user.mak
+++ b/configs/targets/x86_64-linux-user.mak
@@ -3,3 +3,4 @@ TARGET_BASE_ARCH=i386
TARGET_SYSTBL_ABI=common,64
TARGET_SYSTBL=syscall_64.tbl
TARGET_XML_FILES= gdb-xml/i386-64bit.xml
+CONFIG_NATIVE_CALL=y
--
2.34.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [RFC v3 02/10] build: Add configure options for native calls
2023-06-25 21:26 ` [RFC v3 02/10] build: Add configure options for native calls Yeqi Fu
@ 2023-07-03 15:22 ` Alex Bennée
0 siblings, 0 replies; 20+ messages in thread
From: Alex Bennée @ 2023-07-03 15:22 UTC (permalink / raw)
To: Yeqi Fu
Cc: richard.henderson, qemu-devel, Laurent Vivier,
Philippe Mathieu-Daudé, Jiaxun Yang
Yeqi Fu <fufuyqqqqqq@gmail.com> writes:
> Add CONFIG_NATIVE_CALL to architectures supporting native calls.
>
> Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
I would expect this patch to be split up and merged with the enabling
patch for each architecture.
> ---
> configs/targets/aarch64-linux-user.mak | 1 +
> configs/targets/arm-linux-user.mak | 1 +
> configs/targets/i386-linux-user.mak | 1 +
> configs/targets/mips-linux-user.mak | 1 +
> configs/targets/mips64-linux-user.mak | 1 +
> configs/targets/x86_64-linux-user.mak | 1 +
> 6 files changed, 6 insertions(+)
>
> diff --git a/configs/targets/aarch64-linux-user.mak b/configs/targets/aarch64-linux-user.mak
> index ba8bc5fe3f..5a8fd98cd9 100644
> --- a/configs/targets/aarch64-linux-user.mak
> +++ b/configs/targets/aarch64-linux-user.mak
> @@ -4,3 +4,4 @@ TARGET_XML_FILES= gdb-xml/aarch64-core.xml gdb-xml/aarch64-fpu.xml gdb-xml/aarch
> TARGET_HAS_BFLT=y
> CONFIG_SEMIHOSTING=y
> CONFIG_ARM_COMPATIBLE_SEMIHOSTING=y
> +CONFIG_NATIVE_CALL=y
> diff --git a/configs/targets/arm-linux-user.mak b/configs/targets/arm-linux-user.mak
> index 7f5d65794c..f934fb82da 100644
> --- a/configs/targets/arm-linux-user.mak
> +++ b/configs/targets/arm-linux-user.mak
> @@ -5,3 +5,4 @@ TARGET_XML_FILES= gdb-xml/arm-core.xml gdb-xml/arm-vfp.xml gdb-xml/arm-vfp3.xml
> TARGET_HAS_BFLT=y
> CONFIG_SEMIHOSTING=y
> CONFIG_ARM_COMPATIBLE_SEMIHOSTING=y
> +CONFIG_NATIVE_CALL=y
> diff --git a/configs/targets/i386-linux-user.mak b/configs/targets/i386-linux-user.mak
> index 5b2546a430..2d8bca8f93 100644
> --- a/configs/targets/i386-linux-user.mak
> +++ b/configs/targets/i386-linux-user.mak
> @@ -2,3 +2,4 @@ TARGET_ARCH=i386
> TARGET_SYSTBL_ABI=i386
> TARGET_SYSTBL=syscall_32.tbl
> TARGET_XML_FILES= gdb-xml/i386-32bit.xml
> +CONFIG_NATIVE_CALL=y
> diff --git a/configs/targets/mips-linux-user.mak b/configs/targets/mips-linux-user.mak
> index b4569a9893..fa005d487a 100644
> --- a/configs/targets/mips-linux-user.mak
> +++ b/configs/targets/mips-linux-user.mak
> @@ -3,3 +3,4 @@ TARGET_ABI_MIPSO32=y
> TARGET_SYSTBL_ABI=o32
> TARGET_SYSTBL=syscall_o32.tbl
> TARGET_BIG_ENDIAN=y
> +CONFIG_NATIVE_CALL=y
> diff --git a/configs/targets/mips64-linux-user.mak b/configs/targets/mips64-linux-user.mak
> index d2ff509a11..ecfe6bcf73 100644
> --- a/configs/targets/mips64-linux-user.mak
> +++ b/configs/targets/mips64-linux-user.mak
> @@ -4,3 +4,4 @@ TARGET_BASE_ARCH=mips
> TARGET_SYSTBL_ABI=n64
> TARGET_SYSTBL=syscall_n64.tbl
> TARGET_BIG_ENDIAN=y
> +CONFIG_NATIVE_CALL=y
> diff --git a/configs/targets/x86_64-linux-user.mak b/configs/targets/x86_64-linux-user.mak
> index 9ceefbb615..a53b017454 100644
> --- a/configs/targets/x86_64-linux-user.mak
> +++ b/configs/targets/x86_64-linux-user.mak
> @@ -3,3 +3,4 @@ TARGET_BASE_ARCH=i386
> TARGET_SYSTBL_ABI=common,64
> TARGET_SYSTBL=syscall_64.tbl
> TARGET_XML_FILES= gdb-xml/i386-64bit.xml
> +CONFIG_NATIVE_CALL=y
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 20+ messages in thread
* [RFC v3 03/10] build: Implement libnative library and configure options
2023-06-25 21:26 [RFC v3 00/10] Native Library Calls Yeqi Fu
2023-06-25 21:26 ` [RFC v3 01/10] docs: Add specification for native library calls Yeqi Fu
2023-06-25 21:26 ` [RFC v3 02/10] build: Add configure options for native calls Yeqi Fu
@ 2023-06-25 21:27 ` Yeqi Fu
2023-06-25 21:47 ` Philippe Mathieu-Daudé
2023-07-03 15:30 ` Alex Bennée
2023-06-25 21:27 ` [RFC v3 04/10] linux-user: Implement envlist_appendenv Yeqi Fu
` (6 subsequent siblings)
9 siblings, 2 replies; 20+ messages in thread
From: Yeqi Fu @ 2023-06-25 21:27 UTC (permalink / raw)
To: alex.bennee
Cc: richard.henderson, qemu-devel, Yeqi Fu, Paolo Bonzini,
Thomas Huth, Riku Voipio
This commit implements a shared library, where native functions are
rewritten as specialized instructions. At runtime, user programs load
the shared library, and specialized instructions are executed when
native functions are called.
Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
---
Makefile | 2 +
common-user/native/Makefile.include | 9 +++
common-user/native/Makefile.target | 26 +++++++
common-user/native/libnative.c | 112 ++++++++++++++++++++++++++++
configure | 84 ++++++++++++++++-----
include/native/libnative.h | 12 +++
include/native/native-defs.h | 65 ++++++++++++++++
7 files changed, 293 insertions(+), 17 deletions(-)
create mode 100644 common-user/native/Makefile.include
create mode 100644 common-user/native/Makefile.target
create mode 100644 common-user/native/libnative.c
create mode 100644 include/native/libnative.h
create mode 100644 include/native/native-defs.h
diff --git a/Makefile b/Makefile
index 3c7d67142f..787b8954a6 100644
--- a/Makefile
+++ b/Makefile
@@ -185,6 +185,8 @@ SUBDIR_MAKEFLAGS=$(if $(V),,--no-print-directory --quiet)
include $(SRC_PATH)/tests/Makefile.include
+include $(SRC_PATH)/common-user/native/Makefile.include
+
all: recurse-all
ROMS_RULES=$(foreach t, all clean distclean, $(addsuffix /$(t), $(ROMS)))
diff --git a/common-user/native/Makefile.include b/common-user/native/Makefile.include
new file mode 100644
index 0000000000..40d20bcd4c
--- /dev/null
+++ b/common-user/native/Makefile.include
@@ -0,0 +1,9 @@
+.PHONY: build-native
+build-native: $(NATIVE_TARGETS:%=build-native-library-%)
+$(NATIVE_TARGETS:%=build-native-library-%): build-native-library-%:
+ $(call quiet-command, \
+ $(MAKE) -C common-user/native/$* $(SUBDIR_MAKEFLAGS), \
+ "BUILD","$* native library")
+# endif
+
+all: build-native
diff --git a/common-user/native/Makefile.target b/common-user/native/Makefile.target
new file mode 100644
index 0000000000..1bb468a2ec
--- /dev/null
+++ b/common-user/native/Makefile.target
@@ -0,0 +1,26 @@
+# -*- Mode: makefile -*-
+#
+# Library for native calls
+#
+
+all:
+-include ../config-host.mak
+-include config-target.mak
+
+CFLAGS+=-O1 -fPIC -shared -fno-stack-protector -I$(SRC_PATH)/include -D$(TARGET_NAME)
+LDFLAGS+=
+
+ifeq ($(TARGET_NAME),arm)
+EXTRA_CFLAGS+=-marm
+endif
+
+SRC = $(SRC_PATH)/common-user/native/libnative.c
+LIBNATIVE = libnative.so
+
+all: $(LIBNATIVE)
+
+$(LIBNATIVE): $(SRC)
+ $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $< -o $@ $(LDFLAGS)
+
+clean:
+ rm -f $(LIBNATIVE)
diff --git a/common-user/native/libnative.c b/common-user/native/libnative.c
new file mode 100644
index 0000000000..26a004e3b4
--- /dev/null
+++ b/common-user/native/libnative.c
@@ -0,0 +1,112 @@
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include "native/libnative.h"
+#include "native/native-defs.h"
+
+#if defined(i386) || defined(x86_64)
+/* unused opcode */
+#define WRAP_NATIVE_CALL(func_id, abi_map) \
+ do { \
+ __asm__ volatile(".byte 0x0f,0xff\n\t" \
+ ".word %c[imm1],%c[imm2]\n\t" \
+ : /* no outputs */ \
+ : [imm1] "i"(func_id), [imm2] "i"(abi_map) \
+ :); \
+ } while (0)
+
+#endif
+
+#if defined(arm) || defined(aarch64)
+/*
+ * HLT is an invalid instruction for userspace and usefully has 16
+ * bits of spare immeadiate data which we can stuff data in.
+ */
+#define WRAP_NATIVE_CALL(func_id, abi_map) \
+ do { \
+ __asm__ volatile("hlt %c0\n\t" \
+ "hlt %c1\n\t" \
+ : /* no outputs */ \
+ : "i"(func_id), "i"(abi_map) \
+ : "memory"); \
+ } while (0)
+
+#endif
+
+#if defined(mips) || defined(mips64)
+/*
+ * There are some unused bytes in the syscall instruction
+ */
+#define WRAP_NATIVE_CALL(func_id, abi_map) \
+ do { \
+ __asm__ volatile("syscall %c0\n\t" \
+ "syscall %c1\n\t" \
+ : /* no outputs */ \
+ : "i"(func_id), "i"(abi_map) \
+ : "memory"); \
+ } while (0)
+
+#endif
+
+static inline const uint32_t encode_1out_3in(int rtype, int arg1, int arg2,
+ int arg3)
+{
+ return (rtype & 0xf) | ((arg1 & 0xf) << 4) | ((arg2 & 0xf) << 8) |
+ ((arg3 & 0xf) << 12);
+}
+
+static inline const uint32_t encode_0out_3in(int arg1, int arg2, int arg3)
+{
+ return encode_1out_3in(TYPE_NO_ARG, arg1, arg2, arg3);
+}
+
+static inline const uint32_t encode_1out_2in(int rtype, int arg1, int arg2)
+{
+ return encode_1out_3in(rtype, arg1, arg2, TYPE_NO_ARG);
+}
+
+void *memcpy(void *dest, const void *src, size_t n)
+{
+ const uint32_t args = encode_1out_3in(TYPE_PTR_ARG, TYPE_PTR_ARG_W,
+ TYPE_PTR_ARG_R, TYPE_IPTR_ARG);
+ WRAP_NATIVE_CALL(NATIVE_MEMCPY, args);
+}
+
+int memcmp(const void *s1, const void *s2, size_t n)
+{
+ const uint32_t args = encode_1out_3in(TYPE_INT_ARG, TYPE_PTR_ARG_R,
+ TYPE_PTR_ARG_R, TYPE_IPTR_ARG);
+ WRAP_NATIVE_CALL(NATIVE_MEMCMP, args);
+}
+
+void *memset(void *s, int c, size_t n)
+{
+ const uint32_t args = encode_1out_3in(TYPE_PTR_ARG, TYPE_PTR_ARG_W,
+ TYPE_INT_ARG, TYPE_IPTR_ARG);
+ WRAP_NATIVE_CALL(NATIVE_MEMSET, args);
+}
+char *strncpy(char *dest, const char *src, size_t n)
+{
+ const uint32_t args = encode_1out_3in(TYPE_PTR_ARG, TYPE_PTR_ARG_W,
+ TYPE_PTR_ARG_R, TYPE_IPTR_ARG);
+ WRAP_NATIVE_CALL(NATIVE_STRNCPY, args);
+}
+int strncmp(const char *s1, const char *s2, size_t n)
+{
+ const uint32_t args = encode_1out_3in(TYPE_INT_ARG, TYPE_PTR_ARG_R,
+ TYPE_PTR_ARG_R, TYPE_IPTR_ARG);
+ WRAP_NATIVE_CALL(NATIVE_STRNCMP, args);
+}
+char *strcpy(char *dest, const char *src)
+{
+ const uint32_t args =
+ encode_1out_2in(TYPE_PTR_ARG, TYPE_PTR_ARG_W, TYPE_PTR_ARG_R);
+ WRAP_NATIVE_CALL(NATIVE_STRCPY, args);
+}
+int strcmp(const char *s1, const char *s2)
+{
+ const uint32_t args =
+ encode_1out_2in(TYPE_INT_ARG, TYPE_PTR_ARG_R, TYPE_PTR_ARG_R);
+ WRAP_NATIVE_CALL(NATIVE_STRCMP, args);
+}
diff --git a/configure b/configure
index 2a556d14c9..64edbda892 100755
--- a/configure
+++ b/configure
@@ -1838,48 +1838,42 @@ if test "$ccache_cpp2" = "yes"; then
echo "export CCACHE_CPP2=y" >> $config_host_mak
fi
-# tests/tcg configuration
-(config_host_mak=tests/tcg/config-host.mak
-mkdir -p tests/tcg
-echo "# Automatically generated by configure - do not modify" > $config_host_mak
-echo "SRC_PATH=$source_path" >> $config_host_mak
-echo "HOST_CC=$host_cc" >> $config_host_mak
+# prepare config files for cross build
+config_corss_build_host_mak=cross-build/config-host.mak
+mkdir -p cross-build
+echo "# Automatically generated by configure - do not modify" > $config_corss_build_host_mak
+echo "SRC_PATH=$source_path" >> $config_corss_build_host_mak
+echo "HOST_CC=$host_cc" >> $config_corss_build_host_mak
# versioned checked in the main config_host.mak above
if test -n "$gdb_bin"; then
- echo "HAVE_GDB_BIN=$gdb_bin" >> $config_host_mak
+ echo "HAVE_GDB_BIN=$gdb_bin" >> $config_corss_build_host_mak
fi
if test "$plugins" = "yes" ; then
- echo "CONFIG_PLUGIN=y" >> $config_host_mak
+ echo "CONFIG_PLUGIN=y" >> $config_corss_build_host_mak
fi
-tcg_tests_targets=
for target in $target_list; do
arch=${target%%-*}
-
case $target in
xtensa*-linux-user)
- # the toolchain is not complete with headers, only build softmmu tests
continue
;;
*-softmmu)
- test -f "$source_path/tests/tcg/$arch/Makefile.softmmu-target" || continue
qemu="qemu-system-$arch"
;;
*-linux-user|*-bsd-user)
qemu="qemu-$arch"
;;
esac
-
if probe_target_compiler $target || test -n "$container_image"; then
test -n "$container_image" && build_static=y
- mkdir -p "tests/tcg/$target"
- config_target_mak=tests/tcg/$target/config-target.mak
- ln -sf "$source_path/tests/tcg/Makefile.target" "tests/tcg/$target/Makefile"
+ mkdir -p "cross-build/$target"
+ config_target_mak=cross-build/$target/config-target.mak
echo "# Automatically generated by configure - do not modify" > "$config_target_mak"
echo "TARGET_NAME=$arch" >> "$config_target_mak"
echo "TARGET=$target" >> "$config_target_mak"
- write_target_makefile "build-tcg-tests-$target" >> "$config_target_mak"
+ write_target_makefile "$target" >> "$config_target_mak"
echo "BUILD_STATIC=$build_static" >> "$config_target_mak"
echo "QEMU=$PWD/$qemu" >> "$config_target_mak"
@@ -1887,7 +1881,34 @@ for target in $target_list; do
if test "${gdb_arches#*$arch}" != "$gdb_arches"; then
echo "HOST_GDB_SUPPORTS_ARCH=y" >> "$config_target_mak"
fi
+ fi
+done
+
+
+# tests/tcg configuration
+(mkdir -p tests/tcg
+ln -srf $config_corss_build_host_mak tests/tcg/config-host.mak
+
+tcg_tests_targets=
+for target in $target_list; do
+ arch=${target%%-*}
+
+ case $target in
+ xtensa*-linux-user)
+ # the toolchain is not complete with headers, only build softmmu tests
+ continue
+ ;;
+ *-softmmu)
+ test -f "$source_path/tests/tcg/$arch/Makefile.softmmu-target" || continue
+ ;;
+ *-linux-user|*-bsd-user)
+ ;;
+ esac
+ if probe_target_compiler $target || test -n "$container_image"; then
+ mkdir -p "tests/tcg/$target"
+ ln -srf cross-build/$target/config-target.mak tests/tcg/$target/config-target.mak
+ ln -sf $source_path/tests/tcg/Makefile.target tests/tcg/$target/Makefile
echo "run-tcg-tests-$target: $qemu\$(EXESUF)" >> Makefile.prereqs
tcg_tests_targets="$tcg_tests_targets $target"
fi
@@ -1898,6 +1919,35 @@ if test "$tcg" = "enabled"; then
fi
)
+# common-user/native configuration
+(mkdir -p common-user/native
+# reuse the same config-host.mak as tests/tcg
+ln -srf $config_corss_build_host_mak common-user/native/config-host.mak
+
+native_targets=
+for target in $target_list; do
+ arch=${target%%-*}
+ # Skip targets that are not supported
+ case "$arch" in
+ i386|x86_64|arm|aarch64|mips|mips64)
+ case $target in
+ *-linux-user|*-bsd-user)
+ if probe_target_compiler $target || test -n "$container_image"; then
+ mkdir -p "common-user/native/$target"
+ # reuse the same config-target.mak as tests/tcg
+ ln -srf cross-build/$target/config-target.mak common-user/native/$target/config-target.mak
+ ln -sf $source_path/common-user/native/Makefile.target common-user/native/$target/Makefile
+ native_targets="$native_targets $target"
+ fi
+ ;;
+ esac
+ ;;
+ esac
+done
+
+echo "NATIVE_TARGETS=$native_targets" >> config-host.mak
+)
+
if test "$skip_meson" = no; then
cross="config-meson.cross.new"
meson_quote() {
diff --git a/include/native/libnative.h b/include/native/libnative.h
new file mode 100644
index 0000000000..25a59833db
--- /dev/null
+++ b/include/native/libnative.h
@@ -0,0 +1,12 @@
+#ifndef __LIBNATIVE_H__
+#define __LIBNATIVE_H__
+
+void *memcpy(void *dest, const void *src, size_t n);
+int memcmp(const void *s1, const void *s2, size_t n);
+void *memset(void *s, int c, size_t n);
+char *strncpy(char *dest, const char *src, size_t n);
+int strncmp(const char *s1, const char *s2, size_t n);
+char *strcpy(char *dest, const char *src);
+int strcmp(const char *s1, const char *s2);
+
+#endif /* __LIBNATIVE_H__ */
diff --git a/include/native/native-defs.h b/include/native/native-defs.h
new file mode 100644
index 0000000000..37ed479408
--- /dev/null
+++ b/include/native/native-defs.h
@@ -0,0 +1,65 @@
+#ifndef __NATIVE_FUNC_H__
+#define __NATIVE_FUNC_H__
+
+/*
+ * Native function IDs. These are used to identify the native function
+ */
+#define NATIVE_MEMCPY 0x1001
+#define NATIVE_MEMCMP 0x1002
+#define NATIVE_MEMSET 0x1003
+#define NATIVE_STRNCPY 0x1004
+#define NATIVE_STRNCMP 0x1005
+#define NATIVE_STRCPY 0x2001
+#define NATIVE_STRCMP 0x2002
+
+/*
+ * Argument encoding. We only really care about 3 types. The two base
+ * register sizes (32 and 64) and if the value is a pointer (in which
+ * case we need to adjust it g2h before passing to the native
+ * function).
+ */
+#define TYPE_NO_ARG 0x0
+#define TYPE_I32_ARG 0x1 /* uint32_t */
+#define TYPE_I64_ARG 0x2 /* uint64_t */
+#define TYPE_INT_ARG 0x3 /* int */
+#define TYPE_PTR_ARG 0x4 /* void* */
+#define TYPE_PTR_ARG_R 0x5
+#define TYPE_PTR_ARG_W 0x6
+
+/*
+ * Add an alias for the natural register size, it might be easier to
+ * pass this in.
+ */
+#if UINTPTR_MAX == 0xFFFFFFFF
+#define TYPE_IPTR_ARG TYPE_I32_ARG
+#elif UINTPTR_MAX == 0xFFFFFFFFFFFFFFFFu
+#define TYPE_IPTR_ARG TYPE_I64_ARG
+#else
+#error TBD pointer size
+#endif
+
+#define GET_RVALUE(types) ((types)&0xf)
+#define GET_ARG1(types) (((types) >> 4) & 0xf)
+#define GET_ARG2(types) (((types) >> 8) & 0xf)
+#define GET_ARG3(types) (((types) >> 12) & 0xf)
+
+#define IS_PTR_RVALUE(types) (GET_RVALUE(types) >= TYPE_PTR_ARG)
+#define IS_PTR_ARG1(types) (GET_ARG1(types) >= TYPE_PTR_ARG)
+#define IS_PTR_ARG2(types) (GET_ARG2(types) >= TYPE_PTR_ARG)
+#define IS_PTR_ARG3(types) (GET_ARG3(types) >= TYPE_PTR_ARG)
+
+#define IS_ARG_PTR(type) (type >= TYPE_PTR_ARG)
+#define IS_ARG_READ(type) (type == TYPE_PTR_ARG_R)
+#define IS_ARG_WRITE(type) (type == TYPE_PTR_ARG_W)
+
+/*
+ * Check if the native bypass feature is enabled.
+ */
+#if defined(CONFIG_USER_ONLY) && defined(CONFIG_NATIVE_CALL)
+extern bool native_bypass_enabled;
+#define native_call_enabled() (native_bypass_enabled)
+#else
+#define native_call_enabled() 0
+#endif
+
+#endif /* __NATIVE_FUNC_H__ */
--
2.34.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [RFC v3 03/10] build: Implement libnative library and configure options
2023-06-25 21:27 ` [RFC v3 03/10] build: Implement libnative library and configure options Yeqi Fu
@ 2023-06-25 21:47 ` Philippe Mathieu-Daudé
2023-07-03 15:30 ` Alex Bennée
1 sibling, 0 replies; 20+ messages in thread
From: Philippe Mathieu-Daudé @ 2023-06-25 21:47 UTC (permalink / raw)
To: Yeqi Fu, alex.bennee
Cc: richard.henderson, qemu-devel, Paolo Bonzini, Thomas Huth,
Riku Voipio
On 25/6/23 23:27, Yeqi Fu wrote:
> This commit implements a shared library, where native functions are
> rewritten as specialized instructions. At runtime, user programs load
> the shared library, and specialized instructions are executed when
> native functions are called.
>
> Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
> ---
> Makefile | 2 +
> common-user/native/Makefile.include | 9 +++
> common-user/native/Makefile.target | 26 +++++++
> common-user/native/libnative.c | 112 ++++++++++++++++++++++++++++
> configure | 84 ++++++++++++++++-----
> include/native/libnative.h | 12 +++
> include/native/native-defs.h | 65 ++++++++++++++++
> 7 files changed, 293 insertions(+), 17 deletions(-)
> create mode 100644 common-user/native/Makefile.include
> create mode 100644 common-user/native/Makefile.target
> create mode 100644 common-user/native/libnative.c
> create mode 100644 include/native/libnative.h
> create mode 100644 include/native/native-defs.h
> diff --git a/configure b/configure
> index 2a556d14c9..64edbda892 100755
> --- a/configure
> +++ b/configure
> @@ -1838,48 +1838,42 @@ if test "$ccache_cpp2" = "yes"; then
> echo "export CCACHE_CPP2=y" >> $config_host_mak
> fi
>
> -# tests/tcg configuration
> -(config_host_mak=tests/tcg/config-host.mak
> -mkdir -p tests/tcg
> -echo "# Automatically generated by configure - do not modify" > $config_host_mak
> -echo "SRC_PATH=$source_path" >> $config_host_mak
> -echo "HOST_CC=$host_cc" >> $config_host_mak
> +# prepare config files for cross build
> +config_corss_build_host_mak=cross-build/config-host.mak
> +mkdir -p cross-build
> +echo "# Automatically generated by configure - do not modify" > $config_corss_build_host_mak
> +echo "SRC_PATH=$source_path" >> $config_corss_build_host_mak
> +echo "HOST_CC=$host_cc" >> $config_corss_build_host_mak
Typo "cross". Possibly part of the 'configure' changes unrelated
to libnative could be split in a preliminary patch.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC v3 03/10] build: Implement libnative library and configure options
2023-06-25 21:27 ` [RFC v3 03/10] build: Implement libnative library and configure options Yeqi Fu
2023-06-25 21:47 ` Philippe Mathieu-Daudé
@ 2023-07-03 15:30 ` Alex Bennée
1 sibling, 0 replies; 20+ messages in thread
From: Alex Bennée @ 2023-07-03 15:30 UTC (permalink / raw)
To: Yeqi Fu
Cc: richard.henderson, qemu-devel, Paolo Bonzini, Thomas Huth,
Riku Voipio
Yeqi Fu <fufuyqqqqqq@gmail.com> writes:
> This commit implements a shared library, where native functions are
> rewritten as specialized instructions. At runtime, user programs load
> the shared library, and specialized instructions are executed when
> native functions are called.
This commit breaks the build:
make -j9 all
config-host.mak is out-of-date, running configure
python determined to be '/usr/bin/python3'
python version: Python 3.11.2
mkvenv: Creating non-isolated virtual environment at 'pyvenv'
mkvenv: checking for meson>=0.63.0
The Meson build system
Version: 1.0.1
Source dir: /home/alex/lsrc/qemu.git
Build dir: /home/alex/lsrc/qemu.git/builds/user
Build type: native build
Project name: qemu
Project version: 8.0.50
C compiler for the host machine: cc -m64 -mcx16 (gcc 12.2.0 "cc (Debian 12.2.0-14) 12.2.0")
C linker for the host machine: cc -m64 -mcx16 ld.bfd 2.40
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program scripts/symlink-install-tree.py found: YES (/home/alex/lsrc/qemu.git/builds/user/pyvenv/bin/python3 /home/alex/lsrc/qemu.git/scripts/symlink-install-tree.py)
Program sh found: YES (/usr/bin/sh)
C++ compiler for the host machine: c++ -m64 -mcx16 (gcc 12.2.0 "c++ (Debian 12.2.0-14) 12.2.0")
C++ linker for the host machine: c++ -m64 -mcx16 ld.bfd 2.40
../../meson.build:68:0: ERROR: Key TARGET_DIRS is not in the dictionary.
A full log can be found at /home/alex/lsrc/qemu.git/builds/user/meson-logs/meson-log.txt
ERROR: meson setup failed
make: *** [Makefile:83: config-host.mak] Error 1
Compilation exited abnormally with code 2 at Mon Jul 3 16:29:43
I would split this into a number of separate commits.
- configure logic to move around and share the config.mk
- add the build machinery for libnative with a default WRAP_NATIVE
which asserts()
- merge the per-arch macros with arch enabling patch
>
> Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
> ---
> Makefile | 2 +
> common-user/native/Makefile.include | 9 +++
> common-user/native/Makefile.target | 26 +++++++
> common-user/native/libnative.c | 112 ++++++++++++++++++++++++++++
> configure | 84 ++++++++++++++++-----
> include/native/libnative.h | 12 +++
> include/native/native-defs.h | 65 ++++++++++++++++
> 7 files changed, 293 insertions(+), 17 deletions(-)
> create mode 100644 common-user/native/Makefile.include
> create mode 100644 common-user/native/Makefile.target
> create mode 100644 common-user/native/libnative.c
> create mode 100644 include/native/libnative.h
> create mode 100644 include/native/native-defs.h
>
> diff --git a/Makefile b/Makefile
> index 3c7d67142f..787b8954a6 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -185,6 +185,8 @@ SUBDIR_MAKEFLAGS=$(if $(V),,--no-print-directory --quiet)
>
> include $(SRC_PATH)/tests/Makefile.include
>
> +include $(SRC_PATH)/common-user/native/Makefile.include
> +
> all: recurse-all
>
> ROMS_RULES=$(foreach t, all clean distclean, $(addsuffix /$(t), $(ROMS)))
> diff --git a/common-user/native/Makefile.include b/common-user/native/Makefile.include
> new file mode 100644
> index 0000000000..40d20bcd4c
> --- /dev/null
> +++ b/common-user/native/Makefile.include
> @@ -0,0 +1,9 @@
> +.PHONY: build-native
> +build-native: $(NATIVE_TARGETS:%=build-native-library-%)
> +$(NATIVE_TARGETS:%=build-native-library-%): build-native-library-%:
> + $(call quiet-command, \
> + $(MAKE) -C common-user/native/$* $(SUBDIR_MAKEFLAGS), \
> + "BUILD","$* native library")
> +# endif
> +
> +all: build-native
> diff --git a/common-user/native/Makefile.target b/common-user/native/Makefile.target
> new file mode 100644
> index 0000000000..1bb468a2ec
> --- /dev/null
> +++ b/common-user/native/Makefile.target
> @@ -0,0 +1,26 @@
> +# -*- Mode: makefile -*-
> +#
> +# Library for native calls
> +#
> +
> +all:
> +-include ../config-host.mak
> +-include config-target.mak
> +
> +CFLAGS+=-O1 -fPIC -shared -fno-stack-protector -I$(SRC_PATH)/include -D$(TARGET_NAME)
> +LDFLAGS+=
> +
> +ifeq ($(TARGET_NAME),arm)
> +EXTRA_CFLAGS+=-marm
> +endif
I think I mentioned before that this is something that the configure
script should add to config-target.mak.
> +
> +SRC = $(SRC_PATH)/common-user/native/libnative.c
> +LIBNATIVE = libnative.so
> +
> +all: $(LIBNATIVE)
> +
> +$(LIBNATIVE): $(SRC)
> + $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $< -o $@ $(LDFLAGS)
> +
> +clean:
> + rm -f $(LIBNATIVE)
> diff --git a/common-user/native/libnative.c b/common-user/native/libnative.c
> new file mode 100644
> index 0000000000..26a004e3b4
> --- /dev/null
> +++ b/common-user/native/libnative.c
> @@ -0,0 +1,112 @@
> +#include <stdint.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +
> +#include "native/libnative.h"
> +#include "native/native-defs.h"
> +
> +#if defined(i386) || defined(x86_64)
> +/* unused opcode */
> +#define WRAP_NATIVE_CALL(func_id, abi_map) \
> + do { \
> + __asm__ volatile(".byte 0x0f,0xff\n\t" \
> + ".word %c[imm1],%c[imm2]\n\t" \
> + : /* no outputs */ \
> + : [imm1] "i"(func_id), [imm2] "i"(abi_map) \
> + :); \
> + } while (0)
> +
> +#endif
> +
> +#if defined(arm) || defined(aarch64)
> +/*
> + * HLT is an invalid instruction for userspace and usefully has 16
> + * bits of spare immeadiate data which we can stuff data in.
> + */
> +#define WRAP_NATIVE_CALL(func_id, abi_map) \
> + do { \
> + __asm__ volatile("hlt %c0\n\t" \
> + "hlt %c1\n\t" \
> + : /* no outputs */ \
> + : "i"(func_id), "i"(abi_map) \
> + : "memory"); \
> + } while (0)
> +
> +#endif
> +
> +#if defined(mips) || defined(mips64)
> +/*
> + * There are some unused bytes in the syscall instruction
> + */
> +#define WRAP_NATIVE_CALL(func_id, abi_map) \
> + do { \
> + __asm__ volatile("syscall %c0\n\t" \
> + "syscall %c1\n\t" \
> + : /* no outputs */ \
> + : "i"(func_id), "i"(abi_map) \
> + : "memory"); \
> + } while (0)
> +
> +#endif
> +
> +static inline const uint32_t encode_1out_3in(int rtype, int arg1, int arg2,
> + int arg3)
> +{
> + return (rtype & 0xf) | ((arg1 & 0xf) << 4) | ((arg2 & 0xf) << 8) |
> + ((arg3 & 0xf) << 12);
> +}
> +
> +static inline const uint32_t encode_0out_3in(int arg1, int arg2, int arg3)
> +{
> + return encode_1out_3in(TYPE_NO_ARG, arg1, arg2, arg3);
> +}
> +
> +static inline const uint32_t encode_1out_2in(int rtype, int arg1, int arg2)
> +{
> + return encode_1out_3in(rtype, arg1, arg2, TYPE_NO_ARG);
> +}
> +
> +void *memcpy(void *dest, const void *src, size_t n)
> +{
> + const uint32_t args = encode_1out_3in(TYPE_PTR_ARG, TYPE_PTR_ARG_W,
> + TYPE_PTR_ARG_R, TYPE_IPTR_ARG);
> + WRAP_NATIVE_CALL(NATIVE_MEMCPY, args);
> +}
> +
> +int memcmp(const void *s1, const void *s2, size_t n)
> +{
> + const uint32_t args = encode_1out_3in(TYPE_INT_ARG, TYPE_PTR_ARG_R,
> + TYPE_PTR_ARG_R, TYPE_IPTR_ARG);
> + WRAP_NATIVE_CALL(NATIVE_MEMCMP, args);
> +}
> +
> +void *memset(void *s, int c, size_t n)
> +{
> + const uint32_t args = encode_1out_3in(TYPE_PTR_ARG, TYPE_PTR_ARG_W,
> + TYPE_INT_ARG, TYPE_IPTR_ARG);
> + WRAP_NATIVE_CALL(NATIVE_MEMSET, args);
> +}
> +char *strncpy(char *dest, const char *src, size_t n)
> +{
> + const uint32_t args = encode_1out_3in(TYPE_PTR_ARG, TYPE_PTR_ARG_W,
> + TYPE_PTR_ARG_R, TYPE_IPTR_ARG);
> + WRAP_NATIVE_CALL(NATIVE_STRNCPY, args);
> +}
> +int strncmp(const char *s1, const char *s2, size_t n)
> +{
> + const uint32_t args = encode_1out_3in(TYPE_INT_ARG, TYPE_PTR_ARG_R,
> + TYPE_PTR_ARG_R, TYPE_IPTR_ARG);
> + WRAP_NATIVE_CALL(NATIVE_STRNCMP, args);
> +}
> +char *strcpy(char *dest, const char *src)
> +{
> + const uint32_t args =
> + encode_1out_2in(TYPE_PTR_ARG, TYPE_PTR_ARG_W, TYPE_PTR_ARG_R);
> + WRAP_NATIVE_CALL(NATIVE_STRCPY, args);
> +}
> +int strcmp(const char *s1, const char *s2)
> +{
> + const uint32_t args =
> + encode_1out_2in(TYPE_INT_ARG, TYPE_PTR_ARG_R, TYPE_PTR_ARG_R);
> + WRAP_NATIVE_CALL(NATIVE_STRCMP, args);
> +}
> diff --git a/configure b/configure
> index 2a556d14c9..64edbda892 100755
> --- a/configure
> +++ b/configure
> @@ -1838,48 +1838,42 @@ if test "$ccache_cpp2" = "yes"; then
> echo "export CCACHE_CPP2=y" >> $config_host_mak
> fi
>
> -# tests/tcg configuration
> -(config_host_mak=tests/tcg/config-host.mak
> -mkdir -p tests/tcg
> -echo "# Automatically generated by configure - do not modify" > $config_host_mak
> -echo "SRC_PATH=$source_path" >> $config_host_mak
> -echo "HOST_CC=$host_cc" >> $config_host_mak
> +# prepare config files for cross build
I think we could expand this comment a little. Something like:
# Prepare the config files for cross building. There are two parts:
# - cross-build/config-host.mak (common host definitions)
# - cross-build/<target>/config-target.mak
#
# These files are then symlinked to the directories that need them which
# including the TCG tests (tests/tcg/<target>) and the libnative library
# for linux-user (common/native/<target>/).
> +config_corss_build_host_mak=cross-build/config-host.mak
I'm not sure renaming the config_host_mak variable helps here. Although
now I see the problem because there are two config-host's here:
✗ diff -y config-host.mak ./tests/tcg/config-host.mak
# Automatically generated by configure - do not modify # Automatically generated by configure - do not modify
<
all: <
CONFIG_POSIX=y <
CONFIG_LINUX=y <
SRC_PATH=/home/alex/lsrc/qemu.git SRC_PATH=/home/alex/lsrc/qemu.git
TARGET_DIRS=aarch64-linux-user aarch64_be-linux-user alpha-li | HOST_CC=cc
CONFIG_PLUGIN=y <
HAVE_GDB_BIN=/usr/bin/gdb-multiarch HAVE_GDB_BIN=/usr/bin/gdb-multiarch
ENGINE=docker | CONFIG_PLUGIN=y
RUNC=docker <
ROMS= <
PYTHON=/home/alex/lsrc/qemu.git/builds/user/pyvenv/bin/python <
GENISOIMAGE=/usr/bin/genisoimage <
MESON=/home/alex/lsrc/qemu.git/builds/user/pyvenv/bin/meson <
NINJA=/usr/bin/ninja <
PKG_CONFIG=pkg-config <
CC=cc <
EXESUF= <
CONFIG_DEFAULT_TARGETS=y <
TCG_TESTS_TARGETS= aarch64-linux-user alpha-linux-user arm-li <
and the result of this patch squashes two together and looses the extra
information the top config-host.mak needs.
> +mkdir -p cross-build
> +echo "# Automatically generated by configure - do not modify" > $config_corss_build_host_mak
> +echo "SRC_PATH=$source_path" >> $config_corss_build_host_mak
> +echo "HOST_CC=$host_cc" >> $config_corss_build_host_mak
Phillipe has already mentioned the s/corss/cross/
>
> # versioned checked in the main config_host.mak above
> if test -n "$gdb_bin"; then
> - echo "HAVE_GDB_BIN=$gdb_bin" >> $config_host_mak
> + echo "HAVE_GDB_BIN=$gdb_bin" >> $config_corss_build_host_mak
> fi
> if test "$plugins" = "yes" ; then
> - echo "CONFIG_PLUGIN=y" >> $config_host_mak
> + echo "CONFIG_PLUGIN=y" >> $config_corss_build_host_mak
> fi
>
> -tcg_tests_targets=
> for target in $target_list; do
> arch=${target%%-*}
> -
> case $target in
> xtensa*-linux-user)
> - # the toolchain is not complete with headers, only build softmmu tests
> continue
> ;;
> *-softmmu)
> - test -f "$source_path/tests/tcg/$arch/Makefile.softmmu-target" || continue
> qemu="qemu-system-$arch"
> ;;
> *-linux-user|*-bsd-user)
> qemu="qemu-$arch"
> ;;
> esac
> -
> if probe_target_compiler $target || test -n "$container_image"; then
> test -n "$container_image" && build_static=y
> - mkdir -p "tests/tcg/$target"
> - config_target_mak=tests/tcg/$target/config-target.mak
> - ln -sf "$source_path/tests/tcg/Makefile.target" "tests/tcg/$target/Makefile"
> + mkdir -p "cross-build/$target"
> + config_target_mak=cross-build/$target/config-target.mak
> echo "# Automatically generated by configure - do not modify" > "$config_target_mak"
> echo "TARGET_NAME=$arch" >> "$config_target_mak"
> echo "TARGET=$target" >> "$config_target_mak"
> - write_target_makefile "build-tcg-tests-$target" >> "$config_target_mak"
> + write_target_makefile "$target" >> "$config_target_mak"
> echo "BUILD_STATIC=$build_static" >> "$config_target_mak"
> echo "QEMU=$PWD/$qemu" >> "$config_target_mak"
>
> @@ -1887,7 +1881,34 @@ for target in $target_list; do
> if test "${gdb_arches#*$arch}" != "$gdb_arches"; then
> echo "HOST_GDB_SUPPORTS_ARCH=y" >> "$config_target_mak"
> fi
> + fi
> +done
> +
> +
> +# tests/tcg configuration
> +(mkdir -p tests/tcg
> +ln -srf $config_corss_build_host_mak tests/tcg/config-host.mak
> +
> +tcg_tests_targets=
> +for target in $target_list; do
> + arch=${target%%-*}
> +
> + case $target in
> + xtensa*-linux-user)
> + # the toolchain is not complete with headers, only build softmmu tests
> + continue
> + ;;
> + *-softmmu)
> + test -f "$source_path/tests/tcg/$arch/Makefile.softmmu-target" || continue
> + ;;
> + *-linux-user|*-bsd-user)
> + ;;
> + esac
>
> + if probe_target_compiler $target || test -n "$container_image"; then
> + mkdir -p "tests/tcg/$target"
> + ln -srf cross-build/$target/config-target.mak tests/tcg/$target/config-target.mak
> + ln -sf $source_path/tests/tcg/Makefile.target tests/tcg/$target/Makefile
> echo "run-tcg-tests-$target: $qemu\$(EXESUF)" >> Makefile.prereqs
> tcg_tests_targets="$tcg_tests_targets $target"
> fi
> @@ -1898,6 +1919,35 @@ if test "$tcg" = "enabled"; then
> fi
> )
>
> +# common-user/native configuration
> +(mkdir -p common-user/native
> +# reuse the same config-host.mak as tests/tcg
> +ln -srf $config_corss_build_host_mak common-user/native/config-host.mak
> +
> +native_targets=
> +for target in $target_list; do
> + arch=${target%%-*}
> + # Skip targets that are not supported
> + case "$arch" in
> + i386|x86_64|arm|aarch64|mips|mips64)
> + case $target in
> + *-linux-user|*-bsd-user)
> + if probe_target_compiler $target || test -n
> "$container_image"; then
We are repeating the compiler probe here. We should only need to do it
once for all targets and then if there is a valid cross-build config
detect that when symlinking the tests/tcg and common-user versions.
> + mkdir -p "common-user/native/$target"
> + # reuse the same config-target.mak as tests/tcg
> + ln -srf cross-build/$target/config-target.mak common-user/native/$target/config-target.mak
> + ln -sf $source_path/common-user/native/Makefile.target common-user/native/$target/Makefile
> + native_targets="$native_targets $target"
> + fi
> + ;;
> + esac
> + ;;
> + esac
> +done
> +
> +echo "NATIVE_TARGETS=$native_targets" >> config-host.mak
> +)
> +
> if test "$skip_meson" = no; then
> cross="config-meson.cross.new"
> meson_quote() {
> diff --git a/include/native/libnative.h b/include/native/libnative.h
> new file mode 100644
> index 0000000000..25a59833db
> --- /dev/null
> +++ b/include/native/libnative.h
> @@ -0,0 +1,12 @@
> +#ifndef __LIBNATIVE_H__
> +#define __LIBNATIVE_H__
> +
> +void *memcpy(void *dest, const void *src, size_t n);
> +int memcmp(const void *s1, const void *s2, size_t n);
> +void *memset(void *s, int c, size_t n);
> +char *strncpy(char *dest, const char *src, size_t n);
> +int strncmp(const char *s1, const char *s2, size_t n);
> +char *strcpy(char *dest, const char *src);
> +int strcmp(const char *s1, const char *s2);
> +
> +#endif /* __LIBNATIVE_H__ */
> diff --git a/include/native/native-defs.h b/include/native/native-defs.h
> new file mode 100644
> index 0000000000..37ed479408
> --- /dev/null
> +++ b/include/native/native-defs.h
> @@ -0,0 +1,65 @@
> +#ifndef __NATIVE_FUNC_H__
> +#define __NATIVE_FUNC_H__
> +
> +/*
> + * Native function IDs. These are used to identify the native function
> + */
> +#define NATIVE_MEMCPY 0x1001
> +#define NATIVE_MEMCMP 0x1002
> +#define NATIVE_MEMSET 0x1003
> +#define NATIVE_STRNCPY 0x1004
> +#define NATIVE_STRNCMP 0x1005
> +#define NATIVE_STRCPY 0x2001
> +#define NATIVE_STRCMP 0x2002
> +
> +/*
> + * Argument encoding. We only really care about 3 types. The two base
> + * register sizes (32 and 64) and if the value is a pointer (in which
> + * case we need to adjust it g2h before passing to the native
> + * function).
> + */
> +#define TYPE_NO_ARG 0x0
> +#define TYPE_I32_ARG 0x1 /* uint32_t */
> +#define TYPE_I64_ARG 0x2 /* uint64_t */
> +#define TYPE_INT_ARG 0x3 /* int */
> +#define TYPE_PTR_ARG 0x4 /* void* */
> +#define TYPE_PTR_ARG_R 0x5
> +#define TYPE_PTR_ARG_W 0x6
> +
> +/*
> + * Add an alias for the natural register size, it might be easier to
> + * pass this in.
> + */
> +#if UINTPTR_MAX == 0xFFFFFFFF
> +#define TYPE_IPTR_ARG TYPE_I32_ARG
> +#elif UINTPTR_MAX == 0xFFFFFFFFFFFFFFFFu
> +#define TYPE_IPTR_ARG TYPE_I64_ARG
> +#else
> +#error TBD pointer size
> +#endif
> +
> +#define GET_RVALUE(types) ((types)&0xf)
> +#define GET_ARG1(types) (((types) >> 4) & 0xf)
> +#define GET_ARG2(types) (((types) >> 8) & 0xf)
> +#define GET_ARG3(types) (((types) >> 12) & 0xf)
> +
> +#define IS_PTR_RVALUE(types) (GET_RVALUE(types) >= TYPE_PTR_ARG)
> +#define IS_PTR_ARG1(types) (GET_ARG1(types) >= TYPE_PTR_ARG)
> +#define IS_PTR_ARG2(types) (GET_ARG2(types) >= TYPE_PTR_ARG)
> +#define IS_PTR_ARG3(types) (GET_ARG3(types) >= TYPE_PTR_ARG)
> +
> +#define IS_ARG_PTR(type) (type >= TYPE_PTR_ARG)
> +#define IS_ARG_READ(type) (type == TYPE_PTR_ARG_R)
> +#define IS_ARG_WRITE(type) (type == TYPE_PTR_ARG_W)
> +
> +/*
> + * Check if the native bypass feature is enabled.
> + */
> +#if defined(CONFIG_USER_ONLY) && defined(CONFIG_NATIVE_CALL)
> +extern bool native_bypass_enabled;
> +#define native_call_enabled() (native_bypass_enabled)
> +#else
> +#define native_call_enabled() 0
> +#endif
> +
> +#endif /* __NATIVE_FUNC_H__ */
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 20+ messages in thread
* [RFC v3 04/10] linux-user: Implement envlist_appendenv
2023-06-25 21:26 [RFC v3 00/10] Native Library Calls Yeqi Fu
` (2 preceding siblings ...)
2023-06-25 21:27 ` [RFC v3 03/10] build: Implement libnative library and configure options Yeqi Fu
@ 2023-06-25 21:27 ` Yeqi Fu
2023-07-03 16:38 ` Alex Bennée
2023-06-25 21:27 ` [RFC v3 05/10] linux-user: Implement native-bypass option support Yeqi Fu
` (5 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: Yeqi Fu @ 2023-06-25 21:27 UTC (permalink / raw)
To: alex.bennee; +Cc: richard.henderson, qemu-devel, Yeqi Fu
Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
---
include/qemu/envlist.h | 1 +
util/envlist.c | 61 ++++++++++++++++++++++++++++++++++++++++++
2 files changed, 62 insertions(+)
diff --git a/include/qemu/envlist.h b/include/qemu/envlist.h
index 6006dfae44..865eb18e17 100644
--- a/include/qemu/envlist.h
+++ b/include/qemu/envlist.h
@@ -7,6 +7,7 @@ envlist_t *envlist_create(void);
void envlist_free(envlist_t *);
int envlist_setenv(envlist_t *, const char *);
int envlist_unsetenv(envlist_t *, const char *);
+int envlist_appendenv(envlist_t *, const char *, const char *);
int envlist_parse_set(envlist_t *, const char *);
int envlist_parse_unset(envlist_t *, const char *);
char **envlist_to_environ(const envlist_t *, size_t *);
diff --git a/util/envlist.c b/util/envlist.c
index db937c0427..635c9c4fab 100644
--- a/util/envlist.c
+++ b/util/envlist.c
@@ -201,6 +201,67 @@ envlist_unsetenv(envlist_t *envlist, const char *env)
return (0);
}
+/*
+ * Appends environment value to envlist. If the environment
+ * variable already exists, the new value is appended to the
+ * existing one.
+ *
+ * Returns 0 in success, errno otherwise.
+ */
+int
+envlist_appendenv(envlist_t *envlist, const char *env, const char *separator)
+{
+ struct envlist_entry *entry = NULL;
+ const char *eq_sign;
+ size_t envname_len;
+
+ if ((envlist == NULL) || (env == NULL) || (separator == NULL)) {
+ return (EINVAL);
+ }
+
+ /* find out first equals sign in given env */
+ eq_sign = strchr(env, '=');
+ if (eq_sign == NULL) {
+ return (EINVAL);
+ }
+
+ if (strchr(eq_sign + 1, '=') != NULL) {
+ return (EINVAL);
+ }
+
+ envname_len = eq_sign - env + 1;
+
+ /*
+ * If there already exists variable with given name,
+ * we append the new value to the existing one.
+ */
+ for (entry = envlist->el_entries.lh_first; entry != NULL;
+ entry = entry->ev_link.le_next) {
+ if (strncmp(entry->ev_var, env, envname_len) == 0) {
+ break;
+ }
+ }
+
+ if (entry != NULL) {
+ char *new_env_value = NULL;
+ size_t new_env_len = strlen(entry->ev_var) + strlen(eq_sign)
+ + strlen(separator) + 1;
+ new_env_value = g_malloc(new_env_len);
+ strcpy(new_env_value, entry->ev_var);
+ strcat(new_env_value, separator);
+ strcat(new_env_value, eq_sign + 1);
+ g_free((char *)entry->ev_var);
+ entry->ev_var = new_env_value;
+ } else {
+ envlist->el_count++;
+ entry = g_malloc(sizeof(*entry));
+ entry->ev_var = g_strdup(env);
+ QLIST_INSERT_HEAD(&envlist->el_entries, entry, ev_link);
+ }
+
+ return (0);
+}
+
/*
* Returns given envlist as array of strings (in same form that
* global variable environ is). Caller must free returned memory
--
2.34.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [RFC v3 04/10] linux-user: Implement envlist_appendenv
2023-06-25 21:27 ` [RFC v3 04/10] linux-user: Implement envlist_appendenv Yeqi Fu
@ 2023-07-03 16:38 ` Alex Bennée
2023-07-03 16:55 ` Warner Losh
0 siblings, 1 reply; 20+ messages in thread
From: Alex Bennée @ 2023-07-03 16:38 UTC (permalink / raw)
To: Yeqi Fu; +Cc: richard.henderson, qemu-devel
Yeqi Fu <fufuyqqqqqq@gmail.com> writes:
> Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
> ---
> include/qemu/envlist.h | 1 +
> util/envlist.c | 61 ++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 62 insertions(+)
>
> diff --git a/include/qemu/envlist.h b/include/qemu/envlist.h
> index 6006dfae44..865eb18e17 100644
> --- a/include/qemu/envlist.h
> +++ b/include/qemu/envlist.h
> @@ -7,6 +7,7 @@ envlist_t *envlist_create(void);
> void envlist_free(envlist_t *);
> int envlist_setenv(envlist_t *, const char *);
> int envlist_unsetenv(envlist_t *, const char *);
> +int envlist_appendenv(envlist_t *, const char *, const char *);
> int envlist_parse_set(envlist_t *, const char *);
> int envlist_parse_unset(envlist_t *, const char *);
> char **envlist_to_environ(const envlist_t *, size_t *);
> diff --git a/util/envlist.c b/util/envlist.c
> index db937c0427..635c9c4fab 100644
> --- a/util/envlist.c
> +++ b/util/envlist.c
> @@ -201,6 +201,67 @@ envlist_unsetenv(envlist_t *envlist, const char *env)
> return (0);
> }
>
> +/*
> + * Appends environment value to envlist. If the environment
> + * variable already exists, the new value is appended to the
> + * existing one.
> + *
> + * Returns 0 in success, errno otherwise.
> + */
> +int
> +envlist_appendenv(envlist_t *envlist, const char *env, const char *separator)
> +{
> + struct envlist_entry *entry = NULL;
> + const char *eq_sign;
> + size_t envname_len;
> +
> + if ((envlist == NULL) || (env == NULL) || (separator == NULL)) {
> + return (EINVAL);
No () around the EINVAL needed here.
> + }
> +
> + /* find out first equals sign in given env */
> + eq_sign = strchr(env, '=');
> + if (eq_sign == NULL) {
> + return (EINVAL);
> + }
> +
> + if (strchr(eq_sign + 1, '=') != NULL) {
> + return (EINVAL);
> + }
> +
> + envname_len = eq_sign - env + 1;
> +
> + /*
> + * If there already exists variable with given name,
> + * we append the new value to the existing one.
> + */
> + for (entry = envlist->el_entries.lh_first; entry != NULL;
> + entry = entry->ev_link.le_next) {
> + if (strncmp(entry->ev_var, env, envname_len) == 0) {
> + break;
> + }
> + }
> +
> + if (entry != NULL) {
> + char *new_env_value = NULL;
> + size_t new_env_len = strlen(entry->ev_var) + strlen(eq_sign)
> + + strlen(separator) + 1;
> + new_env_value = g_malloc(new_env_len);
> + strcpy(new_env_value, entry->ev_var);
> + strcat(new_env_value, separator);
> + strcat(new_env_value, eq_sign + 1);
> + g_free((char *)entry->ev_var);
> + entry->ev_var = new_env_value;
> + } else {
> + envlist->el_count++;
> + entry = g_malloc(sizeof(*entry));
> + entry->ev_var = g_strdup(env);
> + QLIST_INSERT_HEAD(&envlist->el_entries, entry, ev_link);
> + }
> +
> + return (0);
> +}
> +
We really should add something to tests/unit/test-env to check the
various operations work as expected.
> /*
> * Returns given envlist as array of strings (in same form that
> * global variable environ is). Caller must free returned memory
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [RFC v3 04/10] linux-user: Implement envlist_appendenv
2023-07-03 16:38 ` Alex Bennée
@ 2023-07-03 16:55 ` Warner Losh
0 siblings, 0 replies; 20+ messages in thread
From: Warner Losh @ 2023-07-03 16:55 UTC (permalink / raw)
To: Alex Bennée; +Cc: Yeqi Fu, Richard Henderson, QEMU Developers
[-- Attachment #1: Type: text/plain, Size: 3718 bytes --]
On Mon, Jul 3, 2023, 10:40 AM Alex Bennée <alex.bennee@linaro.org> wrote:
>
> Yeqi Fu <fufuyqqqqqq@gmail.com> writes:
>
> > Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
> > ---
> > include/qemu/envlist.h | 1 +
> > util/envlist.c | 61 ++++++++++++++++++++++++++++++++++++++++++
> > 2 files changed, 62 insertions(+)
> >
> > diff --git a/include/qemu/envlist.h b/include/qemu/envlist.h
> > index 6006dfae44..865eb18e17 100644
> > --- a/include/qemu/envlist.h
> > +++ b/include/qemu/envlist.h
> > @@ -7,6 +7,7 @@ envlist_t *envlist_create(void);
> > void envlist_free(envlist_t *);
> > int envlist_setenv(envlist_t *, const char *);
> > int envlist_unsetenv(envlist_t *, const char *);
> > +int envlist_appendenv(envlist_t *, const char *, const char *);
> > int envlist_parse_set(envlist_t *, const char *);
> > int envlist_parse_unset(envlist_t *, const char *);
> > char **envlist_to_environ(const envlist_t *, size_t *);
> > diff --git a/util/envlist.c b/util/envlist.c
> > index db937c0427..635c9c4fab 100644
> > --- a/util/envlist.c
> > +++ b/util/envlist.c
> > @@ -201,6 +201,67 @@ envlist_unsetenv(envlist_t *envlist, const char
> *env)
> > return (0);
>
No parens here
> }
> >
> > +/*
> > + * Appends environment value to envlist. If the environment
> > + * variable already exists, the new value is appended to the
> > + * existing one.
> > + *
> > + * Returns 0 in success, errno otherwise.
> > + */
> > +int
> > +envlist_appendenv(envlist_t *envlist, const char *env, const char
> *separator)
> > +{
> > + struct envlist_entry *entry = NULL;
> > + const char *eq_sign;
> > + size_t envname_len;
> > +
> > + if ((envlist == NULL) || (env == NULL) || (separator == NULL)) {
> > + return (EINVAL);
>
> No () around the EINVAL needed here.
>
And elsewhere...
Warner
> + }
> > +
> > + /* find out first equals sign in given env */
> > + eq_sign = strchr(env, '=');
> > + if (eq_sign == NULL) {
> > + return (EINVAL);
> > + }
> > +
> > + if (strchr(eq_sign + 1, '=') != NULL) {
> > + return (EINVAL);
> > + }
> > +
> > + envname_len = eq_sign - env + 1;
> > +
> > + /*
> > + * If there already exists variable with given name,
> > + * we append the new value to the existing one.
> > + */
> > + for (entry = envlist->el_entries.lh_first; entry != NULL;
> > + entry = entry->ev_link.le_next) {
> > + if (strncmp(entry->ev_var, env, envname_len) == 0) {
> > + break;
> > + }
> > + }
> > +
> > + if (entry != NULL) {
> > + char *new_env_value = NULL;
> > + size_t new_env_len = strlen(entry->ev_var) + strlen(eq_sign)
> > + + strlen(separator) + 1;
> > + new_env_value = g_malloc(new_env_len);
> > + strcpy(new_env_value, entry->ev_var);
> > + strcat(new_env_value, separator);
> > + strcat(new_env_value, eq_sign + 1);
> > + g_free((char *)entry->ev_var);
> > + entry->ev_var = new_env_value;
> > + } else {
> > + envlist->el_count++;
> > + entry = g_malloc(sizeof(*entry));
> > + entry->ev_var = g_strdup(env);
> > + QLIST_INSERT_HEAD(&envlist->el_entries, entry, ev_link);
> > + }
> > +
> > + return (0);
> > +}
> > +
>
> We really should add something to tests/unit/test-env to check the
> various operations work as expected.
>
>
> > /*
> > * Returns given envlist as array of strings (in same form that
> > * global variable environ is). Caller must free returned memory
>
>
> --
> Alex Bennée
> Virtualisation Tech Lead @ Linaro
>
>
[-- Attachment #2: Type: text/html, Size: 5369 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread
* [RFC v3 05/10] linux-user: Implement native-bypass option support
2023-06-25 21:26 [RFC v3 00/10] Native Library Calls Yeqi Fu
` (3 preceding siblings ...)
2023-06-25 21:27 ` [RFC v3 04/10] linux-user: Implement envlist_appendenv Yeqi Fu
@ 2023-06-25 21:27 ` Yeqi Fu
2023-07-03 18:21 ` Alex Bennée
2023-06-25 21:27 ` [RFC v3 06/10] accel/tcg: Add support for native library calls Yeqi Fu
` (4 subsequent siblings)
9 siblings, 1 reply; 20+ messages in thread
From: Yeqi Fu @ 2023-06-25 21:27 UTC (permalink / raw)
To: alex.bennee; +Cc: richard.henderson, qemu-devel, Yeqi Fu, Laurent Vivier
This commit implements the -native-bypass support in linux-user. The
native_calls_enabled() function can be true only when the
'-native-bypass' option is given.
Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
---
linux-user/main.c | 36 ++++++++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)
diff --git a/linux-user/main.c b/linux-user/main.c
index 5e6b2e1714..98e31c77d5 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -60,6 +60,13 @@
#include "semihosting/semihost.h"
#endif
+#if defined(CONFIG_NATIVE_CALL)
+#include "native/native-defs.h"
+
+static const char *native_lib;
+bool native_bypass_enabled;
+#endif
+
#ifndef AT_FLAGS_PRESERVE_ARGV0
#define AT_FLAGS_PRESERVE_ARGV0_BIT 0
#define AT_FLAGS_PRESERVE_ARGV0 (1 << AT_FLAGS_PRESERVE_ARGV0_BIT)
@@ -125,6 +132,7 @@ static void usage(int exitcode);
static const char *interp_prefix = CONFIG_QEMU_INTERP_PREFIX;
const char *qemu_uname_release;
+
#if !defined(TARGET_DEFAULT_STACK_SIZE)
/* XXX: on x86 MAP_GROWSDOWN only works if ESP <= address + 32, so
we allocate a bigger stack. Need a better solution, for example
@@ -293,6 +301,18 @@ static void handle_arg_set_env(const char *arg)
free(r);
}
+#if defined(CONFIG_NATIVE_CALL)
+static void handle_arg_native_bypass(const char *arg)
+{
+ if (access(arg, F_OK) != 0) {
+ fprintf(stderr, "native library %s does not exist\n", arg);
+ exit(EXIT_FAILURE);
+ }
+ native_lib = arg;
+ native_bypass_enabled = true;
+}
+#endif
+
static void handle_arg_unset_env(const char *arg)
{
char *r, *p, *token;
@@ -522,6 +542,10 @@ static const struct qemu_argument arg_table[] = {
"", "Generate a /tmp/perf-${pid}.map file for perf"},
{"jitdump", "QEMU_JITDUMP", false, handle_arg_jitdump,
"", "Generate a jit-${pid}.dump file for perf"},
+#if defined(CONFIG_NATIVE_CALL)
+ {"native-bypass", "QEMU_NATIVE_BYPASS", true, handle_arg_native_bypass,
+ "", "native bypass for library calls in user mode only."},
+#endif
{NULL, NULL, false, NULL, NULL, NULL}
};
@@ -826,6 +850,18 @@ int main(int argc, char **argv, char **envp)
}
}
+#if defined(CONFIG_NATIVE_CALL)
+ /* Set the library for native bypass */
+ if (native_bypass_enabled) {
+ GString *lib = g_string_new(native_lib);
+ lib = g_string_prepend(lib, "LD_PRELOAD=");
+ if (envlist_appendenv(envlist, g_string_free(lib, false), ":") != 0) {
+ fprintf(stderr,
+ "failed to append the native library to environment.\n");
+ exit(EXIT_FAILURE);
+ }
+ }
+#endif
target_environ = envlist_to_environ(envlist, NULL);
envlist_free(envlist);
--
2.34.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [RFC v3 05/10] linux-user: Implement native-bypass option support
2023-06-25 21:27 ` [RFC v3 05/10] linux-user: Implement native-bypass option support Yeqi Fu
@ 2023-07-03 18:21 ` Alex Bennée
0 siblings, 0 replies; 20+ messages in thread
From: Alex Bennée @ 2023-07-03 18:21 UTC (permalink / raw)
To: Yeqi Fu; +Cc: richard.henderson, qemu-devel, Laurent Vivier
Yeqi Fu <fufuyqqqqqq@gmail.com> writes:
> This commit implements the -native-bypass support in linux-user. The
> native_calls_enabled() function can be true only when the
> '-native-bypass' option is given.
>
> Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
> ---
> linux-user/main.c | 36 ++++++++++++++++++++++++++++++++++++
> 1 file changed, 36 insertions(+)
>
> diff --git a/linux-user/main.c b/linux-user/main.c
> index 5e6b2e1714..98e31c77d5 100644
> --- a/linux-user/main.c
> +++ b/linux-user/main.c
> @@ -60,6 +60,13 @@
> #include "semihosting/semihost.h"
> #endif
>
> +#if defined(CONFIG_NATIVE_CALL)
> +#include "native/native-defs.h"
> +
> +static const char *native_lib;
> +bool native_bypass_enabled;
This bool feels redundant if we can check for a non-null native-lib. You
could certainly expose a function though:
bool native_bypass_enabled() {
return native_lib ? true : false;
}
?
> +#endif
> +
> #ifndef AT_FLAGS_PRESERVE_ARGV0
> #define AT_FLAGS_PRESERVE_ARGV0_BIT 0
> #define AT_FLAGS_PRESERVE_ARGV0 (1 << AT_FLAGS_PRESERVE_ARGV0_BIT)
> @@ -125,6 +132,7 @@ static void usage(int exitcode);
> static const char *interp_prefix = CONFIG_QEMU_INTERP_PREFIX;
> const char *qemu_uname_release;
>
> +
rm whitespace
> #if !defined(TARGET_DEFAULT_STACK_SIZE)
> /* XXX: on x86 MAP_GROWSDOWN only works if ESP <= address + 32, so
> we allocate a bigger stack. Need a better solution, for example
> @@ -293,6 +301,18 @@ static void handle_arg_set_env(const char *arg)
> free(r);
> }
>
> +#if defined(CONFIG_NATIVE_CALL)
> +static void handle_arg_native_bypass(const char *arg)
> +{
> + if (access(arg, F_OK) != 0) {
> + fprintf(stderr, "native library %s does not exist\n", arg);
> + exit(EXIT_FAILURE);
> + }
> + native_lib = arg;
> + native_bypass_enabled = true;
> +}
> +#endif
> +
> static void handle_arg_unset_env(const char *arg)
> {
> char *r, *p, *token;
> @@ -522,6 +542,10 @@ static const struct qemu_argument arg_table[] = {
> "", "Generate a /tmp/perf-${pid}.map file for perf"},
> {"jitdump", "QEMU_JITDUMP", false, handle_arg_jitdump,
> "", "Generate a jit-${pid}.dump file for perf"},
> +#if defined(CONFIG_NATIVE_CALL)
> + {"native-bypass", "QEMU_NATIVE_BYPASS", true, handle_arg_native_bypass,
> + "", "native bypass for library calls in user mode only."},
> +#endif
> {NULL, NULL, false, NULL, NULL, NULL}
> };
>
> @@ -826,6 +850,18 @@ int main(int argc, char **argv, char **envp)
> }
> }
>
> +#if defined(CONFIG_NATIVE_CALL)
> + /* Set the library for native bypass */
> + if (native_bypass_enabled) {
Then this could be:
if (native_lib && g_file_test(native_lib, G_FILE_TEST_EXITS)) {
Or maybe better:
if (native_lib) {
if (g_file_test(native_lib, G_FILE_TEST_EXITS)) {
.. setup ..
} else {
fprintf(stderr, "can't open %s\n", native_lib);
exit(EXIT_FAILURE);
}
}
> + GString *lib = g_string_new(native_lib);
> + lib = g_string_prepend(lib, "LD_PRELOAD=");
> + if (envlist_appendenv(envlist, g_string_free(lib, false), ":") != 0) {
> + fprintf(stderr,
> + "failed to append the native library to environment.\n");
> + exit(EXIT_FAILURE);
> + }
> + }
> +#endif
> target_environ = envlist_to_environ(envlist, NULL);
> envlist_free(envlist);
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 20+ messages in thread
* [RFC v3 06/10] accel/tcg: Add support for native library calls
2023-06-25 21:26 [RFC v3 00/10] Native Library Calls Yeqi Fu
` (4 preceding siblings ...)
2023-06-25 21:27 ` [RFC v3 05/10] linux-user: Implement native-bypass option support Yeqi Fu
@ 2023-06-25 21:27 ` Yeqi Fu
2023-06-25 21:27 ` [RFC v3 07/10] target/i386: " Yeqi Fu
` (3 subsequent siblings)
9 siblings, 0 replies; 20+ messages in thread
From: Yeqi Fu @ 2023-06-25 21:27 UTC (permalink / raw)
To: alex.bennee; +Cc: richard.henderson, qemu-devel, Yeqi Fu, Paolo Bonzini
This commit implements several helpers for extracting function id and
argument types, and then using this information to invoke native functions.
Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
---
accel/tcg/tcg-runtime.c | 37 +++++++++++++++++
accel/tcg/tcg-runtime.h | 4 ++
include/native/native-calls.h | 75 +++++++++++++++++++++++++++++++++++
3 files changed, 116 insertions(+)
create mode 100644 include/native/native-calls.h
diff --git a/accel/tcg/tcg-runtime.c b/accel/tcg/tcg-runtime.c
index e4e030043f..3f173f2a53 100644
--- a/accel/tcg/tcg-runtime.c
+++ b/accel/tcg/tcg-runtime.c
@@ -148,3 +148,40 @@ void HELPER(exit_atomic)(CPUArchState *env)
{
cpu_loop_exit_atomic(env_cpu(env), GETPC());
}
+
+target_ulong helper_native_call(CPUArchState *env, target_ulong arg1,
+ target_ulong arg2, target_ulong arg3,
+ uint32_t abi_map, uint32_t func_id, uint32_t mmu_idx)
+{
+ if (GET_ARG3(abi_map)) {
+ return do_native_3in1out(env, arg1, arg2, arg3, abi_map, func_id,
+ mmu_idx);
+ } else {
+ return do_native_2in1out(env, arg1, arg2, abi_map, func_id, mmu_idx);
+ }
+}
+
+
+uint32_t helper_native_call_i32(CPUArchState *env, uint32_t arg1, uint32_t arg2,
+ uint32_t arg3, uint32_t abi_map,
+ uint32_t func_id, uint32_t mmu_idx)
+{
+ if (GET_ARG3(abi_map)) {
+ return do_native_3in1out(env, arg1, arg2, arg3, abi_map, func_id,
+ mmu_idx);
+ } else {
+ return do_native_2in1out(env, arg1, arg2, abi_map, func_id, mmu_idx);
+ }
+}
+
+uint64_t helper_native_call_i64(CPUArchState *env, uint64_t arg1, uint64_t arg2,
+ uint64_t arg3, uint32_t abi_map,
+ uint32_t func_id, uint32_t mmu_idx)
+{
+ if (GET_ARG3(abi_map)) {
+ return do_native_3in1out(env, arg1, arg2, arg3, abi_map, func_id,
+ mmu_idx);
+ } else {
+ return do_native_2in1out(env, arg1, arg2, abi_map, func_id, mmu_idx);
+ }
+}
diff --git a/accel/tcg/tcg-runtime.h b/accel/tcg/tcg-runtime.h
index 6f8c2061d0..3b1fe606e5 100644
--- a/accel/tcg/tcg-runtime.h
+++ b/accel/tcg/tcg-runtime.h
@@ -300,3 +300,7 @@ DEF_HELPER_FLAGS_4(gvec_leu32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
DEF_HELPER_FLAGS_4(gvec_leu64, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
DEF_HELPER_FLAGS_5(gvec_bitsel, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
+
+DEF_HELPER_7(native_call, tl, env, tl, tl, tl, i32, i32, i32)
+DEF_HELPER_7(native_call_i32, i32, env, i32, i32, i32, i32, i32, i32)
+DEF_HELPER_7(native_call_i64, i64, env, i64, i64, i64, i32, i32, i32)
diff --git a/include/native/native-calls.h b/include/native/native-calls.h
new file mode 100644
index 0000000000..5c99344a65
--- /dev/null
+++ b/include/native/native-calls.h
@@ -0,0 +1,75 @@
+#include "exec/cpu_ldst.h"
+#include "cpu.h"
+#include "native-defs.h"
+
+typedef uintptr_t (*nfunc_2in1out)(uintptr_t, uintptr_t);
+typedef uintptr_t (*nfunc_3in1out)(uintptr_t, uintptr_t, uintptr_t);
+
+static inline uintptr_t decode_arg(CPUArchState *env, abi_ptr arg,
+ uintptr_t size, uint8_t type,
+ uint32_t mmu_idx);
+abi_ptr do_native_2in1out(CPUArchState *env, abi_ptr arg1, abi_ptr arg2,
+ uint32_t abi_map, uint32_t func_id, uint32_t mmu_idx);
+abi_ptr do_native_3in1out(CPUArchState *env, abi_ptr arg1, abi_ptr arg2,
+ abi_ptr arg3, uint32_t abi_map, uint32_t func_id,
+ uint32_t mmu_idx);
+
+nfunc_3in1out func3in1out_array[] = {
+ (nfunc_3in1out)NULL, (nfunc_3in1out)memcpy, (nfunc_3in1out)memcmp,
+ (nfunc_3in1out)memset, (nfunc_3in1out)strncpy, (nfunc_3in1out)strncmp,
+};
+
+nfunc_2in1out func2in1out_array[] = {
+ (nfunc_2in1out)NULL,
+ (nfunc_2in1out)strcpy,
+ (nfunc_2in1out)strcmp,
+};
+
+static inline uintptr_t decode_arg(CPUArchState *env, abi_ptr arg,
+ uintptr_t size, uint8_t type,
+ uint32_t mmu_idx)
+{
+ if (IS_ARG_PTR(type)) {
+ uintptr_t ra = GETPC();
+ void *host;
+ if (IS_ARG_WRITE(type)) {
+ host = probe_write(env, arg, size, mmu_idx, ra);
+ return (uintptr_t)host;
+ }
+ if (IS_ARG_READ(type)) {
+ host = probe_read(env, arg, size, mmu_idx, ra);
+ return (uintptr_t)host;
+ }
+ CPUState *cs = env_cpu(env);
+ return (uintptr_t)g2h(cs, arg);
+ }
+ return (uintptr_t)arg;
+}
+
+/* Currently, there is no distinction between int32, int64, and int. Their type
+ * conversions should be implemented. */
+abi_ptr do_native_3in1out(CPUArchState *env, abi_ptr arg1, abi_ptr arg2,
+ abi_ptr arg3, uint32_t abi_map, uint32_t func_id,
+ uint32_t mmu_idx)
+{
+ uintptr_t n = (uintptr_t)arg3;
+ uintptr_t n1 = decode_arg(env, arg1, n, GET_ARG1(abi_map), mmu_idx);
+ uintptr_t n2 = decode_arg(env, arg2, n, GET_ARG2(abi_map), mmu_idx);
+ nfunc_3in1out fn = func3in1out_array[func_id - 0x1000];
+ uintptr_t r = fn(n1, n2, n);
+ return IS_PTR_RVALUE(abi_map) ? h2g(r) : (target_ulong)r;
+}
+
+
+abi_ptr do_native_2in1out(CPUArchState *env, abi_ptr arg1, abi_ptr arg2,
+ uint32_t abi_map, uint32_t func_id, uint32_t mmu_idx)
+{
+ CPUState *cs = env_cpu(env);
+ uintptr_t n1 =
+ IS_PTR_ARG1(abi_map) ? (uintptr_t)g2h(cs, arg1) : (uintptr_t)arg1;
+ uintptr_t n2 =
+ IS_PTR_ARG2(abi_map) ? (uintptr_t)g2h(cs, arg2) : (uintptr_t)arg2;
+ nfunc_2in1out fn = func2in1out_array[func_id - 0x2000];
+ uintptr_t r = fn(n1, n2);
+ return IS_PTR_RVALUE(abi_map) ? h2g(r) : (target_ulong)r;
+}
--
2.34.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* [RFC v3 07/10] target/i386: Add support for native library calls
2023-06-25 21:26 [RFC v3 00/10] Native Library Calls Yeqi Fu
` (5 preceding siblings ...)
2023-06-25 21:27 ` [RFC v3 06/10] accel/tcg: Add support for native library calls Yeqi Fu
@ 2023-06-25 21:27 ` Yeqi Fu
2023-06-25 21:27 ` [RFC v3 08/10] target/mips: " Yeqi Fu
` (2 subsequent siblings)
9 siblings, 0 replies; 20+ messages in thread
From: Yeqi Fu @ 2023-06-25 21:27 UTC (permalink / raw)
To: alex.bennee
Cc: richard.henderson, qemu-devel, Yeqi Fu, Paolo Bonzini,
Eduardo Habkost
Upon encountering specialized instructions reserved for native calls,
store the function id and argument types, then invoke helper.
Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
---
target/i386/tcg/translate.c | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
diff --git a/target/i386/tcg/translate.c b/target/i386/tcg/translate.c
index 91c9c0c478..27d5d2231b 100644
--- a/target/i386/tcg/translate.c
+++ b/target/i386/tcg/translate.c
@@ -33,6 +33,7 @@
#include "helper-tcg.h"
#include "exec/log.h"
+#include "native/native-defs.h"
#define PREFIX_REPZ 0x01
#define PREFIX_REPNZ 0x02
@@ -6806,6 +6807,38 @@ static bool disas_insn(DisasContext *s, CPUState *cpu)
case 0x1d0 ... 0x1fe:
disas_insn_new(s, cpu, b);
break;
+ case 0x1ff:
+ if (native_call_enabled()) {
+ uint16_t sig;
+ sig = x86_lduw_code(env, s);
+ TCGv_i32 func_id = tcg_constant_i32(sig);
+ sig = x86_lduw_code(env, s);
+ TCGv_i32 abi_map = tcg_constant_i32(sig);
+ TCGv arg1 = tcg_temp_new();
+ TCGv arg2 = tcg_temp_new();
+ TCGv arg3 = tcg_temp_new();
+#ifdef TARGET_X86_64
+ tcg_gen_mov_tl(arg1, cpu_regs[R_EDI]);
+ tcg_gen_mov_tl(arg2, cpu_regs[R_ESI]);
+ tcg_gen_mov_tl(arg3, cpu_regs[R_EDX]);
+#else
+ uintptr_t ra = GETPC();
+ uint32_t a1 = cpu_ldl_data_ra(env, env->regs[R_ESP] + 4, ra);
+ uint32_t a2 = cpu_ldl_data_ra(env, env->regs[R_ESP] + 8, ra);
+ uint32_t a3 = cpu_ldl_data_ra(env, env->regs[R_ESP] + 12, ra);
+ tcg_gen_movi_tl(arg1, a1);
+ tcg_gen_movi_tl(arg2, a2);
+ tcg_gen_movi_tl(arg3, a3);
+#endif
+ TCGv res = tcg_temp_new();
+ TCGv_i32 mmu_idx = tcg_constant_i32(MMU_USER_IDX);
+ gen_helper_native_call(res, cpu_env, arg1, arg2, arg3,
+ abi_map, func_id, mmu_idx);
+
+ tcg_gen_mov_tl(cpu_regs[R_EAX], res);
+ break;
+ }
+ break;
default:
goto unknown_op;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* [RFC v3 08/10] target/mips: Add support for native library calls
2023-06-25 21:26 [RFC v3 00/10] Native Library Calls Yeqi Fu
` (6 preceding siblings ...)
2023-06-25 21:27 ` [RFC v3 07/10] target/i386: " Yeqi Fu
@ 2023-06-25 21:27 ` Yeqi Fu
2023-06-25 21:53 ` Philippe Mathieu-Daudé
2023-06-25 21:27 ` [RFC v3 09/10] target/arm: " Yeqi Fu
2023-06-25 21:27 ` [RFC v3 10/10] tests/tcg/multiarch: Add nativecalls.c test Yeqi Fu
9 siblings, 1 reply; 20+ messages in thread
From: Yeqi Fu @ 2023-06-25 21:27 UTC (permalink / raw)
To: alex.bennee
Cc: richard.henderson, qemu-devel, Yeqi Fu,
Philippe Mathieu-Daudé, Aurelien Jarno, Jiaxun Yang,
Aleksandar Rikalo
Upon encountering specialized instructions reserved for native calls,
store the function id and argument types, then invoke helper.
Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
---
target/mips/tcg/translate.c | 26 ++++++++++++++++++++++++++
target/mips/tcg/translate.h | 2 ++
2 files changed, 28 insertions(+)
diff --git a/target/mips/tcg/translate.c b/target/mips/tcg/translate.c
index a6ca2e5a3b..15ab889dca 100644
--- a/target/mips/tcg/translate.c
+++ b/target/mips/tcg/translate.c
@@ -36,6 +36,7 @@
#include "qemu/qemu-print.h"
#include "fpu_helper.h"
#include "translate.h"
+#include "native/native-defs.h"
/*
* Many sysemu-only helpers are not reachable for user-only.
@@ -13592,6 +13593,31 @@ static void decode_opc_special(CPUMIPSState *env, DisasContext *ctx)
#endif
break;
case OPC_SYSCALL:
+ uint32_t sig = (ctx->opcode) >> 6;
+ if (native_call_enabled() && (!ctx->native_call_status) && sig) {
+ ctx->native_call_status = true;
+ ctx->native_call_id = sig;
+ break;
+ } else if (native_call_enabled() && (ctx->native_call_status) && sig) {
+ TCGv arg1 = tcg_temp_new();
+ TCGv arg2 = tcg_temp_new();
+ TCGv arg3 = tcg_temp_new();
+
+ tcg_gen_mov_tl(arg1, cpu_gpr[4]);
+ tcg_gen_mov_tl(arg2, cpu_gpr[5]);
+ tcg_gen_mov_tl(arg3, cpu_gpr[6]);
+
+ TCGv_i32 abi_map = tcg_constant_i32(sig);
+ TCGv_i32 func_id = tcg_constant_i32(ctx->native_call_id);
+ TCGv res = tcg_temp_new();
+ TCGv_i32 mmu_idx = tcg_constant_i32(MMU_USER_IDX);
+ gen_helper_native_call(res, cpu_env, arg1, arg2, arg3,
+ abi_map, func_id, mmu_idx);
+ tcg_gen_mov_tl(cpu_gpr[2], res);
+ ctx->native_call_status = false;
+ ctx->native_call_id = 0;
+ break;
+ }
generate_exception_end(ctx, EXCP_SYSCALL);
break;
case OPC_BREAK:
diff --git a/target/mips/tcg/translate.h b/target/mips/tcg/translate.h
index 69f85841d2..bc603297cc 100644
--- a/target/mips/tcg/translate.h
+++ b/target/mips/tcg/translate.h
@@ -49,6 +49,8 @@ typedef struct DisasContext {
bool saar;
bool mi;
int gi;
+ bool native_call_status;
+ int native_call_id;
} DisasContext;
#define DISAS_STOP DISAS_TARGET_0
--
2.34.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [RFC v3 08/10] target/mips: Add support for native library calls
2023-06-25 21:27 ` [RFC v3 08/10] target/mips: " Yeqi Fu
@ 2023-06-25 21:53 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 20+ messages in thread
From: Philippe Mathieu-Daudé @ 2023-06-25 21:53 UTC (permalink / raw)
To: Yeqi Fu, alex.bennee
Cc: richard.henderson, qemu-devel, Aurelien Jarno, Jiaxun Yang,
Aleksandar Rikalo
On 25/6/23 23:27, Yeqi Fu wrote:
> Upon encountering specialized instructions reserved for native calls,
> store the function id and argument types, then invoke helper.
>
> Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
> ---
> target/mips/tcg/translate.c | 26 ++++++++++++++++++++++++++
> target/mips/tcg/translate.h | 2 ++
> 2 files changed, 28 insertions(+)
>
> diff --git a/target/mips/tcg/translate.c b/target/mips/tcg/translate.c
> index a6ca2e5a3b..15ab889dca 100644
> --- a/target/mips/tcg/translate.c
> +++ b/target/mips/tcg/translate.c
> @@ -36,6 +36,7 @@
> #include "qemu/qemu-print.h"
> #include "fpu_helper.h"
> #include "translate.h"
> +#include "native/native-defs.h"
>
> /*
> * Many sysemu-only helpers are not reachable for user-only.
> @@ -13592,6 +13593,31 @@ static void decode_opc_special(CPUMIPSState *env, DisasContext *ctx)
> #endif
> break;
> case OPC_SYSCALL:
> + uint32_t sig = (ctx->opcode) >> 6;
if (native_call_enabled() && sig) {
if (ctx->native_call_status) {
...
} else {
...
}
break;
}
> + if (native_call_enabled() && (!ctx->native_call_status) && sig) {
> + ctx->native_call_status = true;
> + ctx->native_call_id = sig;
> + break;
> + } else if (native_call_enabled() && (ctx->native_call_status) && sig) {
> + TCGv arg1 = tcg_temp_new();
> + TCGv arg2 = tcg_temp_new();
> + TCGv arg3 = tcg_temp_new();
> +
> + tcg_gen_mov_tl(arg1, cpu_gpr[4]);
> + tcg_gen_mov_tl(arg2, cpu_gpr[5]);
> + tcg_gen_mov_tl(arg3, cpu_gpr[6]);
> +
> + TCGv_i32 abi_map = tcg_constant_i32(sig);
> + TCGv_i32 func_id = tcg_constant_i32(ctx->native_call_id);
> + TCGv res = tcg_temp_new();
> + TCGv_i32 mmu_idx = tcg_constant_i32(MMU_USER_IDX);
> + gen_helper_native_call(res, cpu_env, arg1, arg2, arg3,
> + abi_map, func_id, mmu_idx);
> + tcg_gen_mov_tl(cpu_gpr[2], res);
> + ctx->native_call_status = false;
> + ctx->native_call_id = 0;
> + break;
> + }
> generate_exception_end(ctx, EXCP_SYSCALL);
> break;
> case OPC_BREAK:
^ permalink raw reply [flat|nested] 20+ messages in thread
* [RFC v3 09/10] target/arm: Add support for native library calls
2023-06-25 21:26 [RFC v3 00/10] Native Library Calls Yeqi Fu
` (7 preceding siblings ...)
2023-06-25 21:27 ` [RFC v3 08/10] target/mips: " Yeqi Fu
@ 2023-06-25 21:27 ` Yeqi Fu
2023-07-03 12:13 ` Alex Bennée
2023-06-25 21:27 ` [RFC v3 10/10] tests/tcg/multiarch: Add nativecalls.c test Yeqi Fu
9 siblings, 1 reply; 20+ messages in thread
From: Yeqi Fu @ 2023-06-25 21:27 UTC (permalink / raw)
To: alex.bennee
Cc: richard.henderson, qemu-devel, Yeqi Fu, Peter Maydell,
open list:ARM TCG CPUs
Upon encountering specialized instructions reserved for native calls,
store the function id and argument types, then invoke helper.
Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
---
target/arm/tcg/translate-a64.c | 27 ++++++++++++++++++++++++++-
target/arm/tcg/translate.c | 25 +++++++++++++++++++++++--
target/arm/tcg/translate.h | 6 ++++++
3 files changed, 55 insertions(+), 3 deletions(-)
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 741a608739..24a664b928 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -35,6 +35,7 @@
#include "cpregs.h"
#include "translate-a64.h"
#include "qemu/atomic128.h"
+#include "native/native-defs.h"
static TCGv_i64 cpu_X[32];
static TCGv_i64 cpu_pc;
@@ -2331,11 +2332,35 @@ static void disas_exc(DisasContext *s, uint32_t insn)
/* BRK */
gen_exception_bkpt_insn(s, syn_aa64_bkpt(imm16));
break;
- case 2:
+ case 2: /* HLT */
if (op2_ll != 0) {
unallocated_encoding(s);
break;
}
+ if (native_call_enabled() && (!s->native_call_status)) {
+ s->native_call_status = true;
+ s->native_call_id = imm16;
+ break;
+ } else if (native_call_enabled() && (s->native_call_status)) {
+ TCGv_i64 arg1 = tcg_temp_new_i64();
+ TCGv_i64 arg2 = tcg_temp_new_i64();
+ TCGv_i64 arg3 = tcg_temp_new_i64();
+
+ tcg_gen_mov_i64(arg1, cpu_reg(s, 0));
+ tcg_gen_mov_i64(arg2, cpu_reg(s, 1));
+ tcg_gen_mov_i64(arg3, cpu_reg(s, 2));
+
+ TCGv_i32 abi_map = tcg_constant_i32(imm16);
+ TCGv_i32 func_id = tcg_constant_i32(s->native_call_id);
+ TCGv_i64 res = tcg_temp_new_i64();
+ TCGv_i32 mmu_idx = tcg_constant_i32(MMU_USER_IDX);
+ gen_helper_native_call(res, cpu_env, arg1, arg2, arg3,
+ abi_map, func_id, mmu_idx);
+ tcg_gen_mov_i64(cpu_reg(s, 0), res);
+ s->native_call_status = false;
+ s->native_call_id = 0;
+ break;
+ }
/* HLT. This has two purposes.
* Architecturally, it is an external halting debug instruction.
* Since QEMU doesn't implement external debug, we treat this as
diff --git a/target/arm/tcg/translate.c b/target/arm/tcg/translate.c
index 7468476724..7b90ce50d0 100644
--- a/target/arm/tcg/translate.c
+++ b/target/arm/tcg/translate.c
@@ -34,7 +34,7 @@
#include "exec/helper-gen.h"
#include "exec/log.h"
#include "cpregs.h"
-
+#include "native/native-defs.h"
#define ENABLE_ARCH_4T arm_dc_feature(s, ARM_FEATURE_V4T)
#define ENABLE_ARCH_5 arm_dc_feature(s, ARM_FEATURE_V5)
@@ -58,6 +58,7 @@ TCGv_i32 cpu_CF, cpu_NF, cpu_VF, cpu_ZF;
TCGv_i64 cpu_exclusive_addr;
TCGv_i64 cpu_exclusive_val;
+
#include "exec/gen-icount.h"
static const char * const regnames[] =
@@ -1147,12 +1148,32 @@ static inline void gen_hlt(DisasContext *s, int imm)
* semihosting, to provide some semblance of security
* (and for consistency with our 32-bit semihosting).
*/
+ if (native_call_enabled() && (!s->native_call_status)) {
+ s->native_call_status = true;
+ s->native_call_id = imm;
+ return;
+ } else if (native_call_enabled() && (s->native_call_status)) {
+ TCGv_i32 arg1 = load_reg(s, 0);
+ TCGv_i32 arg2 = load_reg(s, 1);
+ TCGv_i32 arg3 = load_reg(s, 2);
+
+ TCGv_i32 abi_map = tcg_constant_i32(imm);
+ TCGv_i32 func_id = tcg_constant_i32(s->native_call_id);
+ TCGv_i32 res = tcg_temp_new_i32();
+ TCGv_i32 mmu_idx = tcg_constant_i32(MMU_USER_IDX);
+ gen_helper_native_call_i32(res, cpu_env, arg1, arg2, arg3,
+ abi_map, func_id, mmu_idx);
+
+ store_reg(s, 0, res);
+ s->native_call_status = false;
+ s->native_call_id = 0;
+ return;
+ }
if (semihosting_enabled(s->current_el == 0) &&
(imm == (s->thumb ? 0x3c : 0xf000))) {
gen_exception_internal_insn(s, EXCP_SEMIHOST);
return;
}
-
unallocated_encoding(s);
}
diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h
index a9d1f4adc2..280f8ba215 100644
--- a/target/arm/tcg/translate.h
+++ b/target/arm/tcg/translate.h
@@ -149,6 +149,12 @@ typedef struct DisasContext {
int c15_cpar;
/* TCG op of the current insn_start. */
TCGOp *insn_start;
+ /*
+ * Indicate whether the next instruction is a native function call (true)
+ * or not (false).
+ */
+ bool native_call_status;
+ int native_call_id;
} DisasContext;
typedef struct DisasCompare {
--
2.34.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [RFC v3 09/10] target/arm: Add support for native library calls
2023-06-25 21:27 ` [RFC v3 09/10] target/arm: " Yeqi Fu
@ 2023-07-03 12:13 ` Alex Bennée
0 siblings, 0 replies; 20+ messages in thread
From: Alex Bennée @ 2023-07-03 12:13 UTC (permalink / raw)
To: Yeqi Fu
Cc: richard.henderson, qemu-devel, Peter Maydell,
open list:ARM TCG CPUs
Yeqi Fu <fufuyqqqqqq@gmail.com> writes:
> Upon encountering specialized instructions reserved for native calls,
> store the function id and argument types, then invoke helper.
>
> Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
> ---
> target/arm/tcg/translate-a64.c | 27 ++++++++++++++++++++++++++-
> target/arm/tcg/translate.c | 25 +++++++++++++++++++++++--
> target/arm/tcg/translate.h | 6 ++++++
> 3 files changed, 55 insertions(+), 3 deletions(-)
>
> diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
> index 741a608739..24a664b928 100644
> --- a/target/arm/tcg/translate-a64.c
> +++ b/target/arm/tcg/translate-a64.c
<snip>
> --- a/target/arm/tcg/translate.c
> +++ b/target/arm/tcg/translate.c
> @@ -34,7 +34,7 @@
> #include "exec/helper-gen.h"
> #include "exec/log.h"
> #include "cpregs.h"
> -
> +#include "native/native-defs.h"
>
> #define ENABLE_ARCH_4T arm_dc_feature(s, ARM_FEATURE_V4T)
> #define ENABLE_ARCH_5 arm_dc_feature(s, ARM_FEATURE_V5)
> @@ -58,6 +58,7 @@ TCGv_i32 cpu_CF, cpu_NF, cpu_VF, cpu_ZF;
> TCGv_i64 cpu_exclusive_addr;
> TCGv_i64 cpu_exclusive_val;
>
> +
stray whitespace which will often make re-basing patches a pain.
> #include "exec/gen-icount.h"
>
> static const char * const regnames[] =
> @@ -1147,12 +1148,32 @@ static inline void gen_hlt(DisasContext *s, int imm)
> * semihosting, to provide some semblance of security
> * (and for consistency with our 32-bit semihosting).
> */
> + if (native_call_enabled() && (!s->native_call_status)) {
> + s->native_call_status = true;
> + s->native_call_id = imm;
> + return;
> + } else if (native_call_enabled() && (s->native_call_status)) {
> + TCGv_i32 arg1 = load_reg(s, 0);
> + TCGv_i32 arg2 = load_reg(s, 1);
> + TCGv_i32 arg3 = load_reg(s, 2);
> +
> + TCGv_i32 abi_map = tcg_constant_i32(imm);
> + TCGv_i32 func_id = tcg_constant_i32(s->native_call_id);
> + TCGv_i32 res = tcg_temp_new_i32();
> + TCGv_i32 mmu_idx = tcg_constant_i32(MMU_USER_IDX);
> + gen_helper_native_call_i32(res, cpu_env, arg1, arg2, arg3,
> + abi_map, func_id, mmu_idx);
> +
> + store_reg(s, 0, res);
> + s->native_call_status = false;
> + s->native_call_id = 0;
> + return;
> + }
> if (semihosting_enabled(s->current_el == 0) &&
> (imm == (s->thumb ? 0x3c : 0xf000))) {
> gen_exception_internal_insn(s, EXCP_SEMIHOST);
> return;
> }
> -
also here.
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 20+ messages in thread
* [RFC v3 10/10] tests/tcg/multiarch: Add nativecalls.c test
2023-06-25 21:26 [RFC v3 00/10] Native Library Calls Yeqi Fu
` (8 preceding siblings ...)
2023-06-25 21:27 ` [RFC v3 09/10] target/arm: " Yeqi Fu
@ 2023-06-25 21:27 ` Yeqi Fu
9 siblings, 0 replies; 20+ messages in thread
From: Yeqi Fu @ 2023-06-25 21:27 UTC (permalink / raw)
To: alex.bennee; +Cc: richard.henderson, qemu-devel, Yeqi Fu
Add a test for native calls to verify the functionality of native
functions. This requires cross-compiling test cases and building them
as dynamically linked binaries. Also, introduce necessary system
libraries in QEMU.
Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
---
tests/tcg/multiarch/Makefile.target | 9 +-
tests/tcg/multiarch/native/nativecalls.c | 103 +++++++++++++++++++++++
2 files changed, 111 insertions(+), 1 deletion(-)
create mode 100644 tests/tcg/multiarch/native/nativecalls.c
diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target
index 373db69648..c4ea7117c2 100644
--- a/tests/tcg/multiarch/Makefile.target
+++ b/tests/tcg/multiarch/Makefile.target
@@ -128,7 +128,14 @@ run-semiconsole: semiconsole
run-plugin-semiconsole-with-%:
$(call skip-test, $<, "MANUAL ONLY")
-TESTS += semihosting semiconsole
+nativecalls: native/nativecalls.c
+ $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $< -o $@ $(filter-out -static,$(LDFLAGS))
+
+run-nativecalls: nativecalls
+ $(call run-test,$<, $(QEMU) -L /usr/$(subst -gcc,,$(CC)) --native-bypass $(SRC_PATH)/build/common-user/native/$(TARGET)/libnative.so $<, \
+ "native call")
+
+TESTS += semihosting semiconsole nativecalls
endif
# Update TESTS
diff --git a/tests/tcg/multiarch/native/nativecalls.c b/tests/tcg/multiarch/native/nativecalls.c
new file mode 100644
index 0000000000..2d4ddb3a8e
--- /dev/null
+++ b/tests/tcg/multiarch/native/nativecalls.c
@@ -0,0 +1,103 @@
+#include <assert.h>
+#include <stdlib.h>
+#include <string.h>
+
+void compare_memory(const void *a, const void *b, size_t n)
+{
+ const unsigned char *p1 = a;
+ const unsigned char *p2 = b;
+ for (size_t i = 0; i < n; i++) {
+ assert(p1[i] == p2[i]);
+ }
+}
+
+void test_memcpy()
+{
+ char src[] = "Hello, world!";
+ char dest[20];
+ memcpy(dest, src, 13);
+ compare_memory(dest, src, 13);
+}
+
+void test_memcmp()
+{
+ char str1[] = "abc";
+ char str2[] = "abc";
+ char str3[] = "def";
+ assert(memcmp(str1, str2, 3) == 0);
+ assert(memcmp(str1, str3, 3) != 0);
+}
+
+void test_memset()
+{
+ char buffer[10];
+ memset(buffer, 'A', 10);
+ int i;
+ for (i = 0; i < 10; i++) {
+ assert(buffer[i] == 'A');
+ }
+}
+
+void test_strncpy()
+{
+ char src[] = "Hello, world!";
+ char dest[20];
+ strncpy(dest, src, 13);
+ compare_memory(dest, src, 13);
+}
+
+void test_strncmp()
+{
+ char str1[] = "abc";
+ char str2[] = "abc";
+ char str3[] = "def";
+ assert(strncmp(str1, str2, 2) == 0);
+ assert(strncmp(str1, str3, 2) != 0);
+}
+
+
+void test_strcpy()
+{
+ char src[] = "Hello, world!";
+ char dest[20];
+ strcpy(dest, src);
+ compare_memory(dest, src, 13);
+}
+
+void test_strcmp()
+{
+ char str1[] = "abc";
+ char str2[] = "abc";
+ char str3[] = "def";
+ assert(strcmp(str1, str2) == 0);
+ assert(strcmp(str1, str3) != 0);
+}
+
+void test_memcpy_bad_addr()
+{
+ char src[] = "Hello, world!";
+ char *dst = (char *)0x1;
+ memcpy(dst, src, 13);
+}
+
+void test_memset_bad_addr()
+{
+ char *dst = (char *)0x1;
+ memset(dst, 'A', 10);
+}
+
+int main()
+{
+ test_memcpy();
+ test_memcmp();
+ test_memset();
+ test_strncpy();
+ test_strncmp();
+ test_strcpy();
+ test_strcmp();
+
+ test_memcpy_bad_addr();
+ test_memset_bad_addr();
+
+ return EXIT_SUCCESS;
+}
--
2.34.1
^ permalink raw reply related [flat|nested] 20+ messages in thread