* [RFC v5 01/10] build: Implement logic for sharing cross-building config files
2023-08-25 10:19 [RFC v5 00/10] Native Library Calls Yeqi Fu
@ 2023-08-25 10:19 ` Yeqi Fu
2023-08-25 10:20 ` [RFC v5 02/10] build: Implement libnative library and the build machinery for libnative Yeqi Fu
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Yeqi Fu @ 2023-08-25 10:19 UTC (permalink / raw)
To: alex.bennee
Cc: richard.henderson, qemu-devel, Yeqi Fu, Paolo Bonzini,
Thomas Huth
Since both TCG tests and libnative libraries require cross-building,
the config files for cross-building, config_target_mak, are now saved
in the cross-build directory for sharing. This allows TCG tests and
libnative libraries to use these config files through symbolic links
when cross-building configuration is needed.
Since config_host_mak essentially contains all the information from
the original tests/tcg/config-host.mak, the original config-host.mak
has been deleted and replaced with a symbolic link to config_host_mak.
Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
---
configure | 61 ++++++++++++++++++++++++++++++++++---------------------
1 file changed, 38 insertions(+), 23 deletions(-)
diff --git a/configure b/configure
index 2b41c49c0d..7a1e463d9c 100755
--- a/configure
+++ b/configure
@@ -1751,32 +1751,23 @@ if test "$ccache_cpp2" = "yes"; then
echo "export CCACHE_CPP2=y" >> $config_host_mak
fi
-# tests/tcg configuration
-(config_host_mak=tests/tcg/config-host.mak
-mkdir -p tests/tcg
-echo "# Automatically generated by configure - do not modify" > $config_host_mak
-echo "SRC_PATH=$source_path" >> $config_host_mak
-echo "HOST_CC=$host_cc" >> $config_host_mak
+# Prepare the config files for cross building.
+# This process generates 'cross-build/<target>/config-target.mak' files.
+# These files are then symlinked to the directories that need them which
+# including the TCG tests (tests/tcg/<target>) and the libnative library
+# for linux-user (common/native/<target>/).
+mkdir -p cross-build
-# versioned checked in the main config_host.mak above
-if test -n "$gdb_bin"; then
- echo "HAVE_GDB_BIN=$gdb_bin" >> $config_host_mak
-fi
-if test "$plugins" = "yes" ; then
- echo "CONFIG_PLUGIN=y" >> $config_host_mak
-fi
-
-tcg_tests_targets=
for target in $target_list; do
arch=${target%%-*}
-
case $target in
xtensa*-linux-user)
- # the toolchain is not complete with headers, only build softmmu tests
+ # the toolchain for tests/tcg is not complete with headers
continue
;;
*-softmmu)
- test -f "$source_path/tests/tcg/$arch/Makefile.softmmu-target" || continue
+ # skip installing config-target.mak when we have no tests to build
+ test -f "${source_path}/tests/tcg/${arch}/Makefile.softmmu-target" || continue
qemu="qemu-system-$arch"
;;
*-linux-user|*-bsd-user)
@@ -1786,22 +1777,46 @@ for target in $target_list; do
if probe_target_compiler $target || test -n "$container_image"; then
test -n "$container_image" && build_static=y
- mkdir -p "tests/tcg/$target"
- config_target_mak=tests/tcg/$target/config-target.mak
- ln -sf "$source_path/tests/tcg/Makefile.target" "tests/tcg/$target/Makefile"
+ mkdir -p "cross-build/${target}"
+ config_target_mak=cross-build/${target}/config-target.mak
echo "# Automatically generated by configure - do not modify" > "$config_target_mak"
echo "TARGET_NAME=$arch" >> "$config_target_mak"
echo "TARGET=$target" >> "$config_target_mak"
- write_target_makefile "build-tcg-tests-$target" >> "$config_target_mak"
+ write_target_makefile "$target" >> "$config_target_mak"
echo "BUILD_STATIC=$build_static" >> "$config_target_mak"
echo "QEMU=$PWD/$qemu" >> "$config_target_mak"
+ # get the interpreter prefix and the path of libnative required for native call tests
+ if test -n "$target_cc" && [ -d "/usr/$(echo "$target_cc" | sed 's/-gcc//')" ]; then
+ echo "LD_PREFIX=/usr/$(echo "$target_cc" | sed 's/-gcc//')" >> "$config_target_mak"
+ fi
+
# will GDB work with these binaries?
if test "${gdb_arches#*$arch}" != "$gdb_arches"; then
echo "HOST_GDB_SUPPORTS_ARCH=y" >> "$config_target_mak"
fi
+ fi
+done
+
+# tests/tcg configuration
+(mkdir -p tests/tcg
+# create a symlink to the config-host.mak file in the tests/tcg
+ln -srf $config_host_mak tests/tcg/config-host.mak
+echo "HOST_CC=$host_cc" >> $config_host_mak
+
+tcg_tests_targets=
+for target in $target_list; do
+ case $target in
+ *-softmmu)
+ test -f "${source_path}/tests/tcg/${arch}/Makefile.softmmu-target" || continue
+ ;;
+ esac
- echo "run-tcg-tests-$target: $qemu\$(EXESUF)" >> Makefile.prereqs
+ if test -f cross-build/${target}/config-target.mak; then
+ mkdir -p "tests/tcg/${target}"
+ ln -srf cross-build/${target}/config-target.mak tests/tcg/${target}/config-target.mak
+ ln -sf ${source_path}/tests/tcg/Makefile.target tests/tcg/${target}/Makefile
+ echo "run-tcg-tests-${target}: $qemu\$(EXESUF)" >> Makefile.prereqs
tcg_tests_targets="$tcg_tests_targets $target"
fi
done
--
2.34.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* [RFC v5 02/10] build: Implement libnative library and the build machinery for libnative
2023-08-25 10:19 [RFC v5 00/10] Native Library Calls Yeqi Fu
2023-08-25 10:19 ` [RFC v5 01/10] build: Implement logic for sharing cross-building config files Yeqi Fu
@ 2023-08-25 10:20 ` Yeqi Fu
2023-08-26 1:34 ` Richard Henderson
2023-08-25 10:20 ` [RFC v5 03/10] linux-user: Implement envlist_appendenv and add tests for envlist Yeqi Fu
2023-08-25 10:20 ` [RFC v5 04/10] linux-user: Implement native-bypass option support Yeqi Fu
3 siblings, 1 reply; 7+ messages in thread
From: Yeqi Fu @ 2023-08-25 10:20 UTC (permalink / raw)
To: alex.bennee
Cc: richard.henderson, qemu-devel, Yeqi Fu, Paolo Bonzini,
Thomas Huth, Riku Voipio
This commit implements a shared library, where native functions are
rewritten as special instructions. At runtime, user programs load
the shared library, and special instructions are executed when
native functions are called.
Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
---
Makefile | 2 +
common-user/native/Makefile.include | 9 ++++
common-user/native/Makefile.target | 22 +++++++++
common-user/native/libnative.S | 69 +++++++++++++++++++++++++++++
configure | 39 ++++++++++++++++
5 files changed, 141 insertions(+)
create mode 100644 common-user/native/Makefile.include
create mode 100644 common-user/native/Makefile.target
create mode 100644 common-user/native/libnative.S
diff --git a/Makefile b/Makefile
index 5d48dfac18..6f6147b40f 100644
--- a/Makefile
+++ b/Makefile
@@ -182,6 +182,8 @@ SUBDIR_MAKEFLAGS=$(if $(V),,--no-print-directory --quiet)
include $(SRC_PATH)/tests/Makefile.include
+include $(SRC_PATH)/common-user/native/Makefile.include
+
all: recurse-all
ROMS_RULES=$(foreach t, all clean distclean, $(addsuffix /$(t), $(ROMS)))
diff --git a/common-user/native/Makefile.include b/common-user/native/Makefile.include
new file mode 100644
index 0000000000..40d20bcd4c
--- /dev/null
+++ b/common-user/native/Makefile.include
@@ -0,0 +1,9 @@
+.PHONY: build-native
+build-native: $(NATIVE_TARGETS:%=build-native-library-%)
+$(NATIVE_TARGETS:%=build-native-library-%): build-native-library-%:
+ $(call quiet-command, \
+ $(MAKE) -C common-user/native/$* $(SUBDIR_MAKEFLAGS), \
+ "BUILD","$* native library")
+# endif
+
+all: build-native
diff --git a/common-user/native/Makefile.target b/common-user/native/Makefile.target
new file mode 100644
index 0000000000..65d90102e2
--- /dev/null
+++ b/common-user/native/Makefile.target
@@ -0,0 +1,22 @@
+# -*- Mode: makefile -*-
+#
+# Library for native calls
+#
+
+all:
+-include ../../../config-host.mak
+-include config-target.mak
+
+CFLAGS+=-shared -D $(TARGET_NAME)
+LDFLAGS+=
+
+SRC = $(SRC_PATH)/common-user/native/libnative.S
+LIBNATIVE = libnative.so
+
+all: $(LIBNATIVE)
+
+$(LIBNATIVE): $(SRC)
+ $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(EXTRA_NATIVE_CALL_FLAGS) $< -o $@ $(LDFLAGS)
+
+clean:
+ rm -f $(LIBNATIVE)
diff --git a/common-user/native/libnative.S b/common-user/native/libnative.S
new file mode 100644
index 0000000000..3692eaa3cf
--- /dev/null
+++ b/common-user/native/libnative.S
@@ -0,0 +1,69 @@
+#if defined(i386) || defined(x86_64)
+/*
+ * An unused instruction is utilized to mark a native call.
+ */
+#define __SPECIAL_INSTR .byte 0x0f, 0xff;
+#define __RET_INSTR ret;
+#endif
+
+#if defined(arm) || defined(aarch64)
+/*
+ * HLT is an invalid instruction for userspace programs,
+ * and is used to mark a native call.
+ */
+#define __SPECIAL_INSTR hlt 0xffff;
+#if defined(aarch64)
+#define __RET_INSTR ret;
+#else
+#define __RET_INSTR bx lr;
+#endif
+#endif
+
+
+#if defined(mips) || defined(mips64)
+/*
+ * The syscall instruction contains 20 unused bits, which are typically
+ * set to 0. These bits can be used to store non-zero data,
+ * distinguishing them from a regular syscall instruction.
+ */
+#define __SPECIAL_INSTR syscall 0xffff;
+#define __RET_INSTR jr $ra;
+#endif
+
+/* Symbols of native functions */
+.section .data
+sym_memset: .asciz "memset"
+sym_memcpy: .asciz "memcpy"
+sym_strncpy: .asciz "strncpy"
+sym_memcmp: .asciz "memcmp"
+sym_strncmp: .asciz "strncmp"
+sym_strcpy: .asciz "strcpy"
+sym_strcat: .asciz "strcat"
+sym_strcmp: .asciz "strcmp"
+
+.macro define_function name
+\name:
+#if defined(x86_64) || defined(aarch64)
+ __SPECIAL_INSTR
+ .quad sym_\name
+ __RET_INSTR
+#elif defined(mips64)
+.align 4
+ __SPECIAL_INSTR
+ .quad sym_\name
+ __RET_INSTR
+#elif defined(i386) || defined(mips) || defined(arm)
+ __SPECIAL_INSTR
+ .long sym_\name
+ __RET_INSTR
+#endif
+.endm
+
+define_function memcpy
+define_function strncpy
+define_function memset
+define_function memcmp
+define_function strncmp
+define_function strcpy
+define_function strcat
+define_function strcmp
diff --git a/configure b/configure
index 7a1e463d9c..de533b27a2 100755
--- a/configure
+++ b/configure
@@ -1826,6 +1826,45 @@ if test "$tcg" = "enabled"; then
fi
)
+# common-user/native configuration
+(mkdir -p common-user/native
+
+native_targets=
+for target in $target_list; do
+ case $target in
+ *-softmmu)
+ continue
+ ;;
+ esac
+
+ # native call is only supported on these architectures
+ arch=${target%%-*}
+ config_target_mak=common-user/native/${target}/config-target.mak
+ case $arch in
+ i386|x86_64|arm|aarch64|mips|mips64)
+ if test -f cross-build/${target}/config-target.mak; then
+ mkdir -p "common-user/native/${target}"
+ ln -srf cross-build/${target}/config-target.mak "$config_target_mak"
+ if test $arch = arm; then
+ echo "EXTRA_NATIVE_CALL_FLAGS=-marm" >> "$config_target_mak"
+ fi
+ if test $arch = $cpu || \
+ { test $arch = i386 && test $cpu = x86_64; } || \
+ { test $arch = arm && test $cpu = aarch64; } || \
+ { test $arch = mips && test $cpu = mips64; }; then
+ echo "LD_PREFIX=/" >> "$config_target_mak"
+ fi
+ echo "LIBNATIVE=$PWD/common-user/native/${target}/libnative.so" >> "$config_target_mak"
+ ln -sf ${source_path}/common-user/native/Makefile.target common-user/native/${target}/Makefile
+ native_targets="$native_targets ${target}"
+ fi
+ ;;
+ esac
+done
+
+echo "NATIVE_TARGETS=$native_targets" >> config-host.mak
+)
+
if test "$skip_meson" = no; then
cross="config-meson.cross.new"
meson_quote() {
--
2.34.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [RFC v5 02/10] build: Implement libnative library and the build machinery for libnative
2023-08-25 10:20 ` [RFC v5 02/10] build: Implement libnative library and the build machinery for libnative Yeqi Fu
@ 2023-08-26 1:34 ` Richard Henderson
0 siblings, 0 replies; 7+ messages in thread
From: Richard Henderson @ 2023-08-26 1:34 UTC (permalink / raw)
To: Yeqi Fu, alex.bennee; +Cc: qemu-devel, Paolo Bonzini, Thomas Huth, Riku Voipio
On 8/25/23 03:20, Yeqi Fu wrote:
> This commit implements a shared library, where native functions are
> rewritten as special instructions. At runtime, user programs load
> the shared library, and special instructions are executed when
> native functions are called.
>
> Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
...
> diff --git a/common-user/native/libnative.S b/common-user/native/libnative.S
> new file mode 100644
> index 0000000000..3692eaa3cf
> --- /dev/null
> +++ b/common-user/native/libnative.S
> @@ -0,0 +1,69 @@
> +#if defined(i386) || defined(x86_64)
> +/*
> + * An unused instruction is utilized to mark a native call.
> + */
> +#define __SPECIAL_INSTR .byte 0x0f, 0xff;
> +#define __RET_INSTR ret;
> +#endif
> +
> +#if defined(arm) || defined(aarch64)
> +/*
> + * HLT is an invalid instruction for userspace programs,
> + * and is used to mark a native call.
> + */
> +#define __SPECIAL_INSTR hlt 0xffff;
> +#if defined(aarch64)
> +#define __RET_INSTR ret;
> +#else
> +#define __RET_INSTR bx lr;
> +#endif
> +#endif
> +
> +
> +#if defined(mips) || defined(mips64)
> +/*
> + * The syscall instruction contains 20 unused bits, which are typically
> + * set to 0. These bits can be used to store non-zero data,
> + * distinguishing them from a regular syscall instruction.
> + */
> +#define __SPECIAL_INSTR syscall 0xffff;
> +#define __RET_INSTR jr $ra;
> +#endif
> +
> +/* Symbols of native functions */
> +.section .data
> +sym_memset: .asciz "memset"
> +sym_memcpy: .asciz "memcpy"
> +sym_strncpy: .asciz "strncpy"
> +sym_memcmp: .asciz "memcmp"
> +sym_strncmp: .asciz "strncmp"
> +sym_strcpy: .asciz "strcpy"
> +sym_strcat: .asciz "strcat"
> +sym_strcmp: .asciz "strcmp"
> +
> +.macro define_function name
> +\name:
> +#if defined(x86_64) || defined(aarch64)
> + __SPECIAL_INSTR
> + .quad sym_\name
> + __RET_INSTR
> +#elif defined(mips64)
> +.align 4
> + __SPECIAL_INSTR
> + .quad sym_\name
> + __RET_INSTR
> +#elif defined(i386) || defined(mips) || defined(arm)
> + __SPECIAL_INSTR
> + .long sym_\name
> + __RET_INSTR
> +#endif
> +.endm
> +
> +define_function memcpy
> +define_function strncpy
> +define_function memset
> +define_function memcmp
> +define_function strncmp
> +define_function strcpy
> +define_function strcat
> +define_function strcmp
This cannot possibly work, since none of the symbols are marked .globl, and are therefore
not exported from your libnative.so.
Furthermore, you placed your strings in .data, but then failed to change back to .text, so
none of the instructions are in an executable load segment.
I conclude that your testing succeeded only because no library calls were replaced.
This is not sufficient testing.
In review of previous versions, I have mentioned that the x86 UD0 instruction has more
bytes than simply 0x0f 0xff -- at minimum 3 -- and moreover can be used in the assembler
to produce pc-relative values.
We can clean up the assembly as follows.
r~
-----
.macro special_instr sym
#if defined(__i386__)
ud0 \sym-1f, %eax; 1:
#elif defined(__x86_64__)
ud0 \sym(%rip), %eax
#elif defined(__arm__) || defined(__aarch64__)
hlt 0xffff
1: .word \sym - 1b
#elif defined(__mips__)
syscall 0xffff
1: .word \sym - 1b
#else
# error
#endif
.endm
.macro ret_instr
#if defined(__i386__) || defined(__x86_64__) || defined(__aarch64__)
ret
#elif defined(__arm__)
bx lr
#elif defined(__mips__)
jr $ra
#else
# error
#endif
.endm
/* Symbols of native functions */
.macro define_function name
.text
\name:
special_instr 9f
ret_instr
.globl \name
.type \name, %function
.size \name, . - \name
.section .rodata
9: .asciz "\name"
.endm
define_function memcmp
define_function memcpy
define_function memset
define_function strcat
define_function strcmp
define_function strcpy
define_function strncmp
define_function strncpy
^ permalink raw reply [flat|nested] 7+ messages in thread
* [RFC v5 03/10] linux-user: Implement envlist_appendenv and add tests for envlist
2023-08-25 10:19 [RFC v5 00/10] Native Library Calls Yeqi Fu
2023-08-25 10:19 ` [RFC v5 01/10] build: Implement logic for sharing cross-building config files Yeqi Fu
2023-08-25 10:20 ` [RFC v5 02/10] build: Implement libnative library and the build machinery for libnative Yeqi Fu
@ 2023-08-25 10:20 ` Yeqi Fu
2023-08-25 10:20 ` [RFC v5 04/10] linux-user: Implement native-bypass option support Yeqi Fu
3 siblings, 0 replies; 7+ messages in thread
From: Yeqi Fu @ 2023-08-25 10:20 UTC (permalink / raw)
To: alex.bennee; +Cc: richard.henderson, qemu-devel, Yeqi Fu
Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
---
include/qemu/envlist.h | 13 ++++++
tests/unit/meson.build | 1 +
tests/unit/test-envlist.c | 94 +++++++++++++++++++++++++++++++++++++++
util/envlist.c | 67 +++++++++++++++++++++++-----
4 files changed, 165 insertions(+), 10 deletions(-)
create mode 100644 tests/unit/test-envlist.c
diff --git a/include/qemu/envlist.h b/include/qemu/envlist.h
index 6006dfae44..dc0fe4e644 100644
--- a/include/qemu/envlist.h
+++ b/include/qemu/envlist.h
@@ -1,12 +1,25 @@
#ifndef ENVLIST_H
#define ENVLIST_H
+#include "qemu/queue.h"
+
+struct envlist_entry {
+ char *ev_var; /* actual env value */
+ QLIST_ENTRY(envlist_entry) ev_link;
+};
+
+struct envlist {
+ QLIST_HEAD(, envlist_entry) el_entries; /* actual entries */
+ size_t el_count; /* number of entries */
+};
+
typedef struct envlist envlist_t;
envlist_t *envlist_create(void);
void envlist_free(envlist_t *);
int envlist_setenv(envlist_t *, const char *);
int envlist_unsetenv(envlist_t *, const char *);
+int envlist_appendenv(envlist_t *, const char *, const char *);
int envlist_parse_set(envlist_t *, const char *);
int envlist_parse_unset(envlist_t *, const char *);
char **envlist_to_environ(const envlist_t *, size_t *);
diff --git a/tests/unit/meson.build b/tests/unit/meson.build
index 93977cc32d..9b25b45271 100644
--- a/tests/unit/meson.build
+++ b/tests/unit/meson.build
@@ -50,6 +50,7 @@ tests = {
'test-qapi-util': [],
'test-interval-tree': [],
'test-xs-node': [qom],
+ 'test-envlist': [],
}
if have_system or have_tools
diff --git a/tests/unit/test-envlist.c b/tests/unit/test-envlist.c
new file mode 100644
index 0000000000..d583c6ee33
--- /dev/null
+++ b/tests/unit/test-envlist.c
@@ -0,0 +1,94 @@
+/*
+ * testenvlist unit-tests.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/envlist.h"
+
+static void envlist_create_free_test(void)
+{
+ envlist_t *testenvlist;
+
+ testenvlist = envlist_create();
+ g_assert(testenvlist != NULL);
+ g_assert(testenvlist->el_count == 0);
+
+ envlist_free(testenvlist);
+}
+
+static void envlist_set_unset_test(void)
+{
+ envlist_t *testenvlist;
+ const char *env = "TEST=123";
+ struct envlist_entry *entry;
+
+ testenvlist = envlist_create();
+ g_assert(envlist_setenv(testenvlist, env) == 0);
+ g_assert(testenvlist->el_count == 1);
+ entry = testenvlist->el_entries.lh_first;
+ g_assert_cmpstr(entry->ev_var, ==, "TEST=123");
+ g_assert(envlist_unsetenv(testenvlist, "TEST") == 0);
+ g_assert(testenvlist->el_count == 0);
+
+ envlist_free(testenvlist);
+}
+
+static void envlist_length_test(void)
+{
+ envlist_t *testenvlist;
+ const char *env = "TEST1=123,TEST2=456";
+
+ testenvlist = envlist_create();
+ g_assert(envlist_parse_set(testenvlist, env) == 0);
+ g_assert(testenvlist->el_count == 2);
+ g_assert(envlist_parse_unset(testenvlist, "TEST1,TEST2") == 0);
+ g_assert(testenvlist->el_count == 0);
+
+ envlist_free(testenvlist);
+}
+
+static void envlist_appendenv_test(void)
+{
+ envlist_t *testenvlist;
+ const char *env = "TEST=123";
+ struct envlist_entry *entry;
+
+ testenvlist = envlist_create();
+ g_assert(envlist_setenv(testenvlist, env) == 0);
+ g_assert(envlist_appendenv(testenvlist, "TEST=456", ";") == 0);
+ entry = testenvlist->el_entries.lh_first;
+ g_assert_cmpstr(entry->ev_var, ==, "TEST=123;456");
+
+ envlist_free(testenvlist);
+}
+
+static void envlist_to_environ_test(void)
+{
+ envlist_t *testenvlist;
+ const char *env = "TEST1=123,TEST2=456";
+ size_t count;
+ char **environ;
+
+ testenvlist = envlist_create();
+ g_assert(envlist_parse_set(testenvlist, env) == 0);
+ g_assert(testenvlist->el_count == 2);
+ environ = envlist_to_environ(testenvlist, &count);
+ g_assert(count == 2);
+ g_assert_cmpstr(environ[1], ==, "TEST1=123");
+ g_assert_cmpstr(environ[0], ==, "TEST2=456");
+
+ envlist_free(testenvlist);
+}
+
+int main(int argc, char **argv)
+{
+ g_test_init(&argc, &argv, NULL);
+
+ g_test_add_func("/envlist/create_free", envlist_create_free_test);
+ g_test_add_func("/envlist/set_unset", envlist_set_unset_test);
+ g_test_add_func("/envlist/length", envlist_length_test);
+ g_test_add_func("/envlist/appendenv", envlist_appendenv_test);
+ g_test_add_func("/envlist/to_environ", envlist_to_environ_test);
+
+ return g_test_run();
+}
diff --git a/util/envlist.c b/util/envlist.c
index db937c0427..2dfead4a0f 100644
--- a/util/envlist.c
+++ b/util/envlist.c
@@ -2,16 +2,6 @@
#include "qemu/queue.h"
#include "qemu/envlist.h"
-struct envlist_entry {
- const char *ev_var; /* actual env value */
- QLIST_ENTRY(envlist_entry) ev_link;
-};
-
-struct envlist {
- QLIST_HEAD(, envlist_entry) el_entries; /* actual entries */
- size_t el_count; /* number of entries */
-};
-
static int envlist_parse(envlist_t *envlist,
const char *env, int (*)(envlist_t *, const char *));
@@ -201,6 +191,63 @@ envlist_unsetenv(envlist_t *envlist, const char *env)
return (0);
}
+/*
+ * Appends environment value to envlist. If the environment
+ * variable already exists, the new value is appended to the
+ * existing one.
+ *
+ * Returns 0 in success, errno otherwise.
+ */
+int
+envlist_appendenv(envlist_t *envlist, const char *env, const char *separator)
+{
+ struct envlist_entry *entry = NULL;
+ const char *eq_sign;
+ size_t envname_len;
+
+ if ((envlist == NULL) || (env == NULL) || (separator == NULL)) {
+ return -EINVAL;
+ }
+
+ /* find out first equals sign in given env */
+ eq_sign = strchr(env, '=');
+ if (eq_sign == NULL) {
+ return -EINVAL;
+ }
+
+ if (strchr(eq_sign + 1, '=') != NULL) {
+ return -EINVAL;
+ }
+
+ envname_len = eq_sign - env + 1;
+
+ /*
+ * If there already exists variable with given name,
+ * we append the new value to the existing one.
+ */
+ for (entry = envlist->el_entries.lh_first; entry != NULL;
+ entry = entry->ev_link.le_next) {
+ if (strncmp(entry->ev_var, env, envname_len) == 0) {
+ break;
+ }
+ }
+
+ if (entry != NULL) {
+ GString *new_val = g_string_new(entry->ev_var);
+ g_string_append(new_val, separator);
+ g_string_append(new_val, eq_sign + 1);
+ g_free(entry->ev_var);
+ entry->ev_var = g_string_free(new_val, false);
+ } else {
+ envlist->el_count++;
+ entry = g_malloc(sizeof(*entry));
+ entry->ev_var = g_strdup(env);
+ QLIST_INSERT_HEAD(&envlist->el_entries, entry, ev_link);
+ }
+
+ return 0;
+}
+
/*
* Returns given envlist as array of strings (in same form that
* global variable environ is). Caller must free returned memory
--
2.34.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* [RFC v5 04/10] linux-user: Implement native-bypass option support
2023-08-25 10:19 [RFC v5 00/10] Native Library Calls Yeqi Fu
` (2 preceding siblings ...)
2023-08-25 10:20 ` [RFC v5 03/10] linux-user: Implement envlist_appendenv and add tests for envlist Yeqi Fu
@ 2023-08-25 10:20 ` Yeqi Fu
2023-08-26 1:50 ` Richard Henderson
3 siblings, 1 reply; 7+ messages in thread
From: Yeqi Fu @ 2023-08-25 10:20 UTC (permalink / raw)
To: alex.bennee; +Cc: richard.henderson, qemu-devel, Yeqi Fu, Laurent Vivier
This commit implements the -native-bypass support in linux-user. The
native_calls_enabled() function can be true only when the
'-native-bypass' option is given.
Signed-off-by: Yeqi Fu <fufuyqqqqqq@gmail.com>
---
include/native/native.h | 9 +++++++++
linux-user/main.c | 38 ++++++++++++++++++++++++++++++++++++++
linux-user/syscall.c | 21 +++++++++++++++++++++
3 files changed, 68 insertions(+)
create mode 100644 include/native/native.h
diff --git a/include/native/native.h b/include/native/native.h
new file mode 100644
index 0000000000..7d1baadfcf
--- /dev/null
+++ b/include/native/native.h
@@ -0,0 +1,9 @@
+/*
+ * Check if the native bypass feature is enabled.
+ */
+#if defined(CONFIG_USER_ONLY) && defined(CONFIG_NATIVE_CALL)
+extern char *native_lib_path;
+#define native_bypass_enabled() (native_lib_path != NULL)
+#else
+#define native_bypass_enabled() false
+#endif
diff --git a/linux-user/main.c b/linux-user/main.c
index dba67ffa36..5cf02c071b 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -60,6 +60,11 @@
#include "semihosting/semihost.h"
#endif
+#if defined(CONFIG_NATIVE_CALL)
+#include "native/native.h"
+char *native_lib_path;
+#endif
+
#ifndef AT_FLAGS_PRESERVE_ARGV0
#define AT_FLAGS_PRESERVE_ARGV0_BIT 0
#define AT_FLAGS_PRESERVE_ARGV0 (1 << AT_FLAGS_PRESERVE_ARGV0_BIT)
@@ -293,6 +298,17 @@ static void handle_arg_set_env(const char *arg)
free(r);
}
+#if defined(CONFIG_NATIVE_CALL)
+static void handle_arg_native_bypass(const char *arg)
+{
+ if (access(arg, F_OK) != 0) {
+ fprintf(stderr, "native library %s does not exist\n", arg);
+ exit(EXIT_FAILURE);
+ }
+ native_lib_path = g_strdup(arg);
+}
+#endif
+
static void handle_arg_unset_env(const char *arg)
{
char *r, *p, *token;
@@ -522,6 +538,10 @@ static const struct qemu_argument arg_table[] = {
"", "Generate a /tmp/perf-${pid}.map file for perf"},
{"jitdump", "QEMU_JITDUMP", false, handle_arg_jitdump,
"", "Generate a jit-${pid}.dump file for perf"},
+#if defined(CONFIG_NATIVE_CALL)
+ {"native-bypass", "QEMU_NATIVE_BYPASS", true, handle_arg_native_bypass,
+ "", "native bypass for library calls"},
+#endif
{NULL, NULL, false, NULL, NULL, NULL}
};
@@ -834,6 +854,24 @@ int main(int argc, char **argv, char **envp)
}
}
+#if defined(CONFIG_NATIVE_CALL)
+ /* Set the library for native bypass */
+ if (native_lib_path) {
+ if (g_file_test(native_lib_path, G_FILE_TEST_IS_REGULAR)) {
+ GString *lib = g_string_new(native_lib_path);
+ lib = g_string_prepend(lib, "LD_PRELOAD=");
+ if (envlist_appendenv(envlist, g_string_free(lib, false), ":")) {
+ fprintf(stderr,
+ "failed to append the native library to environment.\n");
+ exit(EXIT_FAILURE);
+ }
+ } else {
+ fprintf(stderr, "native library %s does not exist.\n",
+ native_lib_path);
+ exit(EXIT_FAILURE);
+ }
+ }
+#endif
target_environ = envlist_to_environ(envlist, NULL);
envlist_free(envlist);
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 08162cc966..bd4c3045ff 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -143,6 +143,7 @@
#include "fd-trans.h"
#include "tcg/tcg.h"
#include "cpu_loop-common.h"
+#include "native/native.h"
#ifndef CLONE_IO
#define CLONE_IO 0x80000000 /* Clone io context */
@@ -8626,6 +8627,7 @@ static int do_execveat(CPUArchState *cpu_env, int dirfd,
abi_ulong addr;
char **q;
void *p;
+ unsigned int i;
argc = 0;
@@ -8696,6 +8698,25 @@ static int do_execveat(CPUArchState *cpu_env, int dirfd,
goto execve_efault;
}
+ /*
+ * An error may occur when executing execv, stating that the
+ * shared library from LD_PRELOAD cannot be preloaded on a
+ * different arch. So, we find LD_PRELOAD and remove it from
+ * envp before executing the execv.
+ */
+ if (native_bypass_enabled()) {
+ i = 0;
+ while (envp[i] != NULL) {
+ if (strncmp(envp[i], "LD_PRELOAD=", 11) == 0) {
+ for (int j = i; envp[j] != NULL; j++) {
+ envp[j] = envp[j + 1];
+ }
+ } else {
+ i++;
+ }
+ }
+ }
+
if (is_proc_myself(p, "exe")) {
ret = get_errno(safe_execveat(dirfd, exec_path, argp, envp, flags));
} else {
--
2.34.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [RFC v5 04/10] linux-user: Implement native-bypass option support
2023-08-25 10:20 ` [RFC v5 04/10] linux-user: Implement native-bypass option support Yeqi Fu
@ 2023-08-26 1:50 ` Richard Henderson
0 siblings, 0 replies; 7+ messages in thread
From: Richard Henderson @ 2023-08-26 1:50 UTC (permalink / raw)
To: Yeqi Fu, alex.bennee; +Cc: qemu-devel, Laurent Vivier
On 8/25/23 03:20, Yeqi Fu wrote:
> +#if defined(CONFIG_NATIVE_CALL)
> + /* Set the library for native bypass */
> + if (native_lib_path) {
> + if (g_file_test(native_lib_path, G_FILE_TEST_IS_REGULAR)) {
> + GString *lib = g_string_new(native_lib_path);
> + lib = g_string_prepend(lib, "LD_PRELOAD=");
> + if (envlist_appendenv(envlist, g_string_free(lib, false), ":")) {
> + fprintf(stderr,
> + "failed to append the native library to environment.\n");
> + exit(EXIT_FAILURE);
> + }
> + } else {
> + fprintf(stderr, "native library %s does not exist.\n",
> + native_lib_path);
> + exit(EXIT_FAILURE);
> + }
> + }
> +#endif
Here you append to the existing LD_PRELOAD.
> + /*
> + * An error may occur when executing execv, stating that the
> + * shared library from LD_PRELOAD cannot be preloaded on a
> + * different arch. So, we find LD_PRELOAD and remove it from
> + * envp before executing the execv.
> + */
> + if (native_bypass_enabled()) {
> + i = 0;
> + while (envp[i] != NULL) {
> + if (strncmp(envp[i], "LD_PRELOAD=", 11) == 0) {
> + for (int j = i; envp[j] != NULL; j++) {
> + envp[j] = envp[j + 1];
> + }
> + } else {
> + i++;
> + }
> + }
> + }
Here you simply remove LD_PRELOAD entirely.
At most you should only remove libnative.so.
I'm not at all sure that you should be modifying the target environment at all. It's ok
for simple testing, but it is definitely error prone. There are a couple of different
solutions:
(1) Dynamically modify /etc/ld.so.preload, similar to how we handle various /proc files.
(2) Merge libnative.so with vdso.so (and select one of two images depending on bypass
enabled).
r~
^ permalink raw reply [flat|nested] 7+ messages in thread