[Qemu-devel] qemu-i386 thread support and TLS

[Qemu-devel] qemu-i386 thread support and TLS

[G Portokalidis]

Hello all,
I've trying to implement thread support for user-space Qemu, similarly
to what David has posted in a previous e-mail.
I understand that it is probably not possible to fully support threads
between very different architectures, but i am only interested on
getting x86 on x86 linux emulation working.
I am using Qemu to dynamically instrument binaries to provide
additional security.

I have coded set_thread_area() and clone() like in the previous
NPTLish patch posted by providing a separate GDT for each thread, but
used pthreads instead of clone() to start the new thread. That makes
it easier to support emulator TLS variables.

I have also declared global_env and cpu_single_env as thread local variables.

The new thread starts, but unfortunately it's generates a SIGSEGV
within cpu_loop(). I understand that the translator is not thread
safe, but i also tried pausing the calling thread to see whether the
child would be able to run by itself getting the same result.

I think that make qemu-i386 thread safe without TLS would be too hard,
so i haven't looked into that.


Do you have any hints on why would qemu crash even after setting up GDT?
I am interested in making the translator thread safe, but it seems
quite complicated. Any hints on which parts need to be made thread
local, or need locks?

I am also sending part of the code, sorry it's not a diff i have
changed the qemu source too much and it would just complicate things.

struct clone_arg_struct {
	CPUX86State *env;
	pthread_cond_t pid_ready;
	pid_t pid;
	struct user_desc *tls;
	int *ptid, *ctid;
};

static void *clone_func(void *opaque)
{
	struct clone_arg_struct *clone_args = opaque;
	CPUX86State *env;

	env = clone_args->env;
	thread_env = env;
	clone_args->pid = gettid();
	if (clone_args->ptid)
		*clone_args->ptid = clone_args->pid;
	if (clone_args->ctid)
		*clone_args->ctid = clone_args->pid;
	if (clone_args->tls) {
			do_set_thread_area(env, clone_args->tls);
	}

	pthread_cond_signal(&clone_args->pid_ready);
	// clone_args is not valid anymore

	thread_env = env;

	printf("new thread!!!\n");
	cpu_loop(env);
	// Never get here?
	return NULL;
}

static long do_clone(CPUX86State *env, int flags, void *cstack, int *ptid,
		struct user_desc *newtls, int *ctid)
{
	pthread_t thread;
	CPUX86State *new_env;
	struct clone_arg_struct clone_args;
	pthread_mutex_t ca_mutex;
	long ret;

	if (!(flags & CLONE_VM)) {
		// No CLONE_VM results to behaviour similar to fork()
		printf("do_clone() no CLONE_VM!\n");
		if ((flags & ~CSIGNAL) != 0)
			return -EINVAL;
		syscall2(TARGET_NR_clone, flags, cstack, ret);
		return ret;
	}

	new_env = cpu_init();
	if (!new_env)
		return -ENOMEM;
	memcpy(new_env, env, sizeof(CPUX86State));
	new_env->gdt.base = (unsigned long)malloc(env->gdt.limit + 1);
	if (!new_env->gdt.base) {
		cpu_close(new_env);
		return -ENOMEM;
	}
	memcpy((void *)new_env->gdt.base, (void *)env->gdt.base,
			env->gdt.limit + 1);
	if (!cstack)
		printf("do_clone() no childstack provided!!!\n");
	// Activate user provided stack
	new_env->regs[R_ESP] = (unsigned long)cstack;
	// This will indicate success
	new_env->regs[R_EAX] = 0;

	// XXX: Is there any better way to retrieve the child's thread id ???
	clone_args.env = new_env;
	clone_args.ptid = (flags & CLONE_PARENT_SETTID)? ptid : NULL;
	clone_args.ctid = (flags & CLONE_CHILD_SETTID)? ctid : NULL;
	clone_args.tls = (flags & CLONE_SETTLS)? newtls : NULL;
	pthread_cond_init(&clone_args.pid_ready, NULL);
	if (pthread_create(&thread, NULL, clone_func, &clone_args) != 0)
		goto error;
	pthread_mutex_init(&ca_mutex, NULL);
	if (pthread_cond_wait(&clone_args.pid_ready, &ca_mutex) != 0)
		goto error;
	if (flags & CLONE_PARENT_SETTID)
		*ptid = clone_args.pid;

	pause();
	printf("resumed\n");
	return clone_args.pid;

error:
	return -errno;
}


static long do_set_thread_area(CPUX86State *env, struct user_desc *uinfo)
{
	struct desc_struct *desc, *tls_array;

       uinfo->entry_number = 6; // This is the one used by glibc, just
an ugly hack for now
	if (uinfo->entry_number < TARGET_GDT_ENTRY_TLS_MIN ||
			uinfo->entry_number > TARGET_GDT_ENTRY_TLS_MAX)
		return -EINVAL;

	tls_array = (struct desc_struct *)env->gdt.base +
		TARGET_GDT_ENTRY_TLS_MIN;
	desc = tls_array + (uinfo->entry_number - TARGET_GDT_ENTRY_TLS_MIN);

	if (LDT_empty(uinfo)) {
		desc->a = 0;
		desc->b = 0;
	} else {
		desc->a = LDT_entry_a(uinfo);
		desc->b = LDT_entry_b(uinfo);
	}
	return 0;
}