[Qemu-devel] linux-user: add option to intercept execve() syscalls

[Qemu-devel] linux-user: add option to intercept execve() syscalls

[Joel Holdsworth]

This patch-set includes Peter Angelatos's previous patch-set [1] and
adds code to pass arguments for setting the environment variables,
passing the interpeter prefix, and passing the strace option.

[1] https://patchwork.ozlabs.org/patch/582756/

[Qemu-devel] [PATCH 1/4] linux-user: add option to intercept execve() syscalls

[Joel Holdsworth]

From: Petros Angelatos <petrosagg@resin.io>

In order for one to use QEMU user mode emulation under a chroot, it is
required to use binfmt_misc. This can be avoided by QEMU never doing a
raw execve() to the host system.

Introduce a new option, -execve, that uses the current QEMU interpreter
to intercept execve().

qemu_execve() will prepend the interpreter path , similar to what
binfmt_misc would do, and then pass the modified execve() to the host.

It is necessary to parse hashbang scripts in that function otherwise
the kernel will try to run the interpreter of a script without QEMU and
get an invalid exec format error.

Signed-off-by: Petros Angelatos <petrosagg@resin.io>
Tested-by: Laurent Vivier <laurent@vivier.eu>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/main.c    |  36 ++++++++++++++++
 linux-user/qemu.h    |   1 +
 linux-user/syscall.c | 117 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 153 insertions(+), 1 deletion(-)

diff --git a/linux-user/main.c b/linux-user/main.c
index 95ed11d..093f858 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -17,6 +17,7 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 #include "qemu/osdep.h"
+#include <sys/auxv.h>
 #include <sys/mman.h>
 #include <sys/syscall.h>
 #include <sys/resource.h>
@@ -78,6 +79,7 @@ static void usage(int exitcode);
 
 static const char *interp_prefix = CONFIG_QEMU_INTERP_PREFIX;
 const char *qemu_uname_release;
+const char *qemu_execve_path;
 
 /* XXX: on x86 MAP_GROWSDOWN only works if ESP <= address + 32, so
    we allocate a bigger stack. Need a better solution, for example
@@ -3868,6 +3870,38 @@ static void handle_arg_guest_base(const char *arg)
     have_guest_base = 1;
 }
 
+static void handle_arg_execve(const char *arg)
+{
+    const char *execfn;
+    char buf[PATH_MAX];
+    char *ret;
+    int len;
+
+    /* try getauxval() */
+    execfn = (const char *) getauxval(AT_EXECFN);
+
+    if (execfn != 0) {
+        ret = realpath(execfn, buf);
+
+        if (ret != NULL) {
+            qemu_execve_path = strdup(buf);
+            return;
+        }
+    }
+
+    /* try /proc/self/exe */
+    len = readlink("/proc/self/exe", buf, sizeof(buf) - 1);
+
+    if (len != -1) {
+        buf[len] = '\0';
+        qemu_execve_path = strdup(buf);
+        return;
+    }
+
+    fprintf(stderr, "qemu_execve: unable to determine intepreter's path\n");
+    exit(EXIT_FAILURE);
+}
+
 static void handle_arg_reserved_va(const char *arg)
 {
     char *p;
@@ -3953,6 +3987,8 @@ static const struct qemu_argument arg_table[] = {
      "uname",      "set qemu uname release string to 'uname'"},
     {"B",          "QEMU_GUEST_BASE",  true,  handle_arg_guest_base,
      "address",    "set guest_base address to 'address'"},
+    {"execve",     "QEMU_EXECVE",      false, handle_arg_execve,
+     "",           "use this interpreter when a process calls execve()"},
     {"R",          "QEMU_RESERVED_VA", true,  handle_arg_reserved_va,
      "size",       "reserve 'size' bytes for guest virtual address space"},
     {"d",          "QEMU_LOG",         true,  handle_arg_log,
diff --git a/linux-user/qemu.h b/linux-user/qemu.h
index 208c63e..50fe716 100644
--- a/linux-user/qemu.h
+++ b/linux-user/qemu.h
@@ -138,6 +138,7 @@ void init_task_state(TaskState *ts);
 void task_settid(TaskState *);
 void stop_all_tasks(void);
 extern const char *qemu_uname_release;
+extern const char *qemu_execve_path;
 extern unsigned long mmap_min_addr;
 
 /* ??? See if we can avoid exposing so much of the loader internals.  */
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 032d338..750e381 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -101,6 +101,7 @@ int __clone2(int (*fn)(void *), void *child_stack_base,
 #include <linux/route.h>
 #include <linux/filter.h>
 #include <linux/blkpg.h>
+#include <linux/binfmts.h>
 #include "linux_loop.h"
 #include "uname.h"
 
@@ -5847,6 +5848,118 @@ static target_timer_t get_timer_id(abi_long arg)
     return timerid;
 }
 
+/* qemu_execve() Must return target values and target errnos. */
+static abi_long qemu_execve(char *filename, char *argv[],
+                  char *envp[])
+{
+    char *i_arg = NULL, *i_name = NULL;
+    char **new_argp;
+    int argc, fd, ret, i, offset = 3;
+    char *cp;
+    char buf[BINPRM_BUF_SIZE];
+
+    /* normal execve case */
+    if (qemu_execve_path == NULL || *qemu_execve_path == 0) {
+        return get_errno(execve(filename, argv, envp));
+    }
+
+    for (argc = 0; argv[argc] != NULL; argc++) {
+        /* nothing */ ;
+    }
+
+    fd = open(filename, O_RDONLY);
+    if (fd == -1) {
+        return get_errno(fd);
+    }
+
+    ret = read(fd, buf, BINPRM_BUF_SIZE);
+    if (ret == -1) {
+        close(fd);
+        return get_errno(ret);
+    }
+
+    /* if we have less than 2 bytes, we can guess it is not executable */
+    if (ret < 2) {
+        close(fd);
+        return -host_to_target_errno(ENOEXEC);
+    }
+
+    close(fd);
+
+    /* adapted from the kernel
+     * https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/fs/binfmt_script.c
+     */
+    if ((buf[0] == '#') && (buf[1] == '!')) {
+        /*
+         * This section does the #! interpretation.
+         * Sorta complicated, but hopefully it will work.  -TYT
+         */
+
+        buf[BINPRM_BUF_SIZE - 1] = '\0';
+        cp = strchr(buf, '\n');
+        if (cp == NULL) {
+            cp = buf + BINPRM_BUF_SIZE - 1;
+        }
+        *cp = '\0';
+        while (cp > buf) {
+            cp--;
+            if ((*cp == ' ') || (*cp == '\t')) {
+                *cp = '\0';
+            } else {
+                break;
+            }
+        }
+        for (cp = buf + 2; (*cp == ' ') || (*cp == '\t'); cp++) {
+            /* nothing */ ;
+        }
+        if (*cp == '\0') {
+            return -ENOEXEC; /* No interpreter name found */
+        }
+        i_name = cp;
+        i_arg = NULL;
+        for ( ; *cp && (*cp != ' ') && (*cp != '\t'); cp++) {
+            /* nothing */ ;
+        }
+        while ((*cp == ' ') || (*cp == '\t')) {
+            *cp++ = '\0';
+        }
+        if (*cp) {
+            i_arg = cp;
+        }
+
+        if (i_arg) {
+            offset = 5;
+        } else {
+            offset = 4;
+        }
+    }
+
+    new_argp = alloca((argc + offset + 1) * sizeof(void *));
+
+    /* Copy the original arguments with offset */
+    for (i = 0; i < argc; i++) {
+        new_argp[i + offset] = argv[i];
+    }
+
+    new_argp[0] = strdup(qemu_execve_path);
+    new_argp[1] = strdup("-0");
+    new_argp[offset] = filename;
+    new_argp[argc + offset] = NULL;
+
+    if (i_name) {
+        new_argp[2] = i_name;
+        new_argp[3] = i_name;
+
+        if (i_arg) {
+            new_argp[4] = i_arg;
+        }
+    } else {
+        new_argp[2] = argv[0];
+    }
+
+    return get_errno(execve(qemu_execve_path, new_argp, envp));
+}
+
 /* do_syscall() should always have a single exit point at the end so
    that actions, such as logging of syscall results, can be performed.
    All errnos that do_syscall() returns must be -TARGET_<errcode>. */
@@ -6106,7 +6219,9 @@ abi_long do_syscall(void *cpu_env, int num, abi_long arg1,
 
             if (!(p = lock_user_string(arg1)))
                 goto execve_efault;
-            ret = get_errno(execve(p, argp, envp));
+
+            ret = qemu_execve(p, argp, envp);
+
             unlock_user(p, arg1, 0);
 
             goto execve_end;
-- 
1.9.1

[Qemu-devel] [PATCH 2/4] linux-user: pass environment arguments in execve

[Joel Holdsworth]

Previously, when emulating execve(2), qemu would execute a child
instance of the emulator with the environment variables provided by
the parent process. This caused problems with qemu if any of the
variables affected the child emulator's behaviour e.g.
LD_LIBRARY_PATH.

This patch solves this issue by passing the environment variables
with '-E' arguments to the child qemu instance. The call to
execve(2) is replaced by a call to execv(2) so that the parent
emulator's environment variable state is propagated into the child.

Any variables from the host environment that are not in the in the
execve() call are removed with a '-U' argument.
---
 linux-user/syscall.c | 96 ++++++++++++++++++++++++++++++++++++++--------------
 1 file changed, 70 insertions(+), 26 deletions(-)

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 750e381..b95f75a 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -5853,9 +5853,13 @@ static abi_long qemu_execve(char *filename, char *argv[],
                   char *envp[])
 {
     char *i_arg = NULL, *i_name = NULL;
-    char **new_argp;
-    int argc, fd, ret, i, offset = 3;
+    char **qemu_argp, **argp;
+    int i, j;
+    size_t qemu_argc = 3, argc, host_envc, envpc;
+    int fd, ret;
     char *cp;
+    size_t def_envc = 0, undef_envc = 0;
+    char **def_env, **undef_env;
     char buf[BINPRM_BUF_SIZE];
 
     /* normal execve case */
@@ -5863,10 +5867,12 @@ static abi_long qemu_execve(char *filename, char *argv[],
         return get_errno(execve(filename, argv, envp));
     }
 
-    for (argc = 0; argv[argc] != NULL; argc++) {
-        /* nothing */ ;
-    }
+    /* count the number of arguments and environment variables */
+    for (argc = 0; argv[argc]; argc++);
+    for (host_envc = 0; environ[host_envc]; host_envc++);
+    for (envpc = 0; envp[envpc]; envpc++);
 
+    /* read the file header so we can check the shebang */
     fd = open(filename, O_RDONLY);
     if (fd == -1) {
         return get_errno(fd);
@@ -5927,37 +5933,75 @@ static abi_long qemu_execve(char *filename, char *argv[],
             i_arg = cp;
         }
 
-        if (i_arg) {
-            offset = 5;
-        } else {
-            offset = 4;
-        }
+        if (i_arg)
+            qemu_argc += 2;
+        else
+            qemu_argc += 1;
+    }
+
+    /* list environment variables to define */
+    def_env = alloca((envpc + 1) * sizeof(envp[0]));
+    for (i = 0; i != envpc; i++) {
+        for (j = 0; j != host_envc; j++)
+            if (!strcmp(envp[i], environ[j]))
+                break;
+        if (j == host_envc)
+            def_env[def_envc++] = envp[i];
     }
 
-    new_argp = alloca((argc + offset + 1) * sizeof(void *));
+    argc += def_envc * 2;
 
-    /* Copy the original arguments with offset */
-    for (i = 0; i < argc; i++) {
-        new_argp[i + offset] = argv[i];
+    /* list environment variables to undefine */
+    undef_env = alloca((host_envc + 1) * sizeof(envp[0]));
+    for (i = 0; i != host_envc; i++) {
+        const char *const host_env = environ[i];
+	const size_t key_len = strchr(host_env, '=') - host_env;
+        for (j = 0; j != envpc; j++)
+            if (!strncmp(host_env, envp[j], key_len))
+                break;
+        if (j == envpc)
+            undef_env[undef_envc++] = strndup(environ[i], key_len);
     }
 
-    new_argp[0] = strdup(qemu_execve_path);
-    new_argp[1] = strdup("-0");
-    new_argp[offset] = filename;
-    new_argp[argc + offset] = NULL;
+    argc += undef_envc * 2;
 
-    if (i_name) {
-        new_argp[2] = i_name;
-        new_argp[3] = i_name;
+    /* allocate the argument list */
+    argp = qemu_argp = alloca((qemu_argc + 1) * sizeof(void *));
 
-        if (i_arg) {
-            new_argp[4] = i_arg;
-        }
+    /* set up the qemu arguments */
+    *argp++ = strdup(qemu_execve_path);
+
+    /* add arguments for the enironment variables */
+    for (i = 0; i < def_envc; i++) {
+        *argp++ = strdup("-E");
+        *argp++ = def_env[i];
+    }
+
+    for (i = 0; i < undef_envc; i++) {
+        *argp++ = strdup("-U");
+        *argp++ = undef_env[i];
+    }
+
+    /* add the path to the executable */
+    *argp++ = strdup("-0");
+    if (i_name) {
+        *argp++ = i_name;
+        *argp++ = i_name;
+        if (i_arg)
+            *argp++ = i_arg;
     } else {
-        new_argp[2] = argv[0];
+        *argp++ = argv[0];
     }
 
-    return get_errno(execve(qemu_execve_path, new_argp, envp));
+    *argp++ = filename;
+
+    /* copy the original arguments with offset */
+    for (i = 1; i < argc; i++)
+        *argp++ = argv[i];
+
+    *argp++ = NULL;
+
+    return get_errno(execv(qemu_execve_path, qemu_argp));
 }
 
 /* do_syscall() should always have a single exit point at the end so
-- 
1.9.1

[Qemu-devel] [PATCH 3/4] linux-user: pass elf interpreter prefix in execve

[Joel Holdsworth]

---
 linux-user/syscall.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index b95f75a..fb75c09 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -5855,7 +5855,7 @@ static abi_long qemu_execve(char *filename, char *argv[],
     char *i_arg = NULL, *i_name = NULL;
     char **qemu_argp, **argp;
     int i, j;
-    size_t qemu_argc = 3, argc, host_envc, envpc;
+    size_t qemu_argc = 5, argc, host_envc, envpc;
     int fd, ret;
     char *cp;
     size_t def_envc = 0, undef_envc = 0;
@@ -5970,6 +5970,8 @@ static abi_long qemu_execve(char *filename, char *argv[],
 
     /* set up the qemu arguments */
     *argp++ = strdup(qemu_execve_path);
+    *argp++ = strdup("-L");
+    *argp++ = strdup(path("/"));
 
     /* add arguments for the enironment variables */
     for (i = 0; i < def_envc; i++) {
-- 
1.9.1

[Qemu-devel] [PATCH 4/4] linux-user: pass strace argument in execve

[Joel Holdsworth]

---
 linux-user/syscall.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index fb75c09..314a890 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -5966,6 +5966,8 @@ static abi_long qemu_execve(char *filename, char *argv[],
     argc += undef_envc * 2;
 
     /* allocate the argument list */
+    if (do_strace)
+        qemu_argc++;
     argp = qemu_argp = alloca((qemu_argc + 1) * sizeof(void *));
 
     /* set up the qemu arguments */
@@ -5973,6 +5975,9 @@ static abi_long qemu_execve(char *filename, char *argv[],
     *argp++ = strdup("-L");
     *argp++ = strdup(path("/"));
 
+    if (do_strace)
+        *argp++ = strdup("-strace");
+
     /* add arguments for the enironment variables */
     for (i = 0; i < def_envc; i++) {
         *argp++ = strdup("-E");
-- 
1.9.1

Re: [Qemu-devel] linux-user: add option to intercept execve() syscalls

[Riku Voipio]

On Wed, May 25, 2016 at 05:07:48PM +0100, Joel Holdsworth wrote:
> This patch-set includes Peter Angelatos's previous patch-set [1] and
> adds code to pass arguments for setting the environment variables,
> passing the interpeter prefix, and passing the strace option.

Considering the messiness this serieas adds to QEMU, I do wonder how
much of win this avoidance really is. If you have permissions to chroot,
you generally have permissions to set binfmt_misc too. Alternatively
these kind of exec manipulations are already done by external tools like
proot and scratchbox.

However if you are ready to stay around to maintain it, and nobody else
objects the code, I can merge it.


> [1] https://patchwork.ozlabs.org/patch/582756/
> 

Re: [Qemu-devel] linux-user: add option to intercept execve() syscalls

[Joel Holdsworth]

On 30/05/16 08:52, Riku Voipio wrote:
> On Wed, May 25, 2016 at 05:07:48PM +0100, Joel Holdsworth wrote:
> Considering the messiness this serieas adds to QEMU, I do wonder how
> much of win this avoidance really is.
Suggestions on how to make it less messy would be welcome.
> If you have permissions to chroot,
> you generally have permissions to set binfmt_misc too. Alternatively
> these kind of exec manipulations are already done by external tools like
> proot and scratchbox.
True. Though I would say that's quite a convoluted solution compared to 
this patch.

The issue I encountered came when I was trying to run ARM gstreamer unit 
tests. libgstreamer runs the gst-plugin-scanner helper executable at 
startup to scan for plugins. However it took me a long time to figure 
this issue out because in the current behavior linux-user doesn't make 
it clear what qemu will run all child processes on the host even if they 
are a different arch.

Adding these patches offers the choice, and makes the situation clearer 
to the user.

> However if you are ready to stay around to maintain it, and nobody else
> objects the code, I can merge it.
I'll be around for the time being certainly.

Best Regards
Joel Holdsworth