From mboxrd@z Thu Jan  1 00:00:00 1970
From: Marcelo Tosatti <mtosatti@redhat.com>
Subject: Re: qemu-kvm-1.1.0 crashing with kernel 3.5.0-rc6
Date: Tue, 31 Jul 2012 13:29:40 -0300
Message-ID: <20120731162940.GA7243@amt.cnet>
References: <5015662A.2000006@redhat.com>
 <501577D1.7030205@googlemail.com>
 <20120729175453.GA32360@redhat.com>
 <50158A97.3050909@googlemail.com>
 <50169389.1020607@googlemail.com>
 <50169421.6060406@redhat.com>
 <50169515.7050900@googlemail.com>
 <5016B8C3.7040103@redhat.com>
 <20120730233616.GA24530@amt.cnet>
 <5017A131.8000609@redhat.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: Chris Clayton <chris2553@googlemail.com>,
	Gleb Natapov <gleb@redhat.com>,
	Eric Northup <digitaleric@google.com>, kvm@vger.kernel.org,
	Jan Kiszka <jan.kiszka@siemens.com>
To: Avi Kivity <avi@redhat.com>
Return-path: <kvm-owner@vger.kernel.org>
Received: from mx1.redhat.com ([209.132.183.28]:60364 "EHLO mx1.redhat.com"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1755777Ab2GaSWK (ORCPT <rfc822;kvm@vger.kernel.org>);
	Tue, 31 Jul 2012 14:22:10 -0400
Content-Disposition: inline
In-Reply-To: <5017A131.8000609@redhat.com>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

On Tue, Jul 31, 2012 at 12:11:13PM +0300, Avi Kivity wrote:
> On 07/31/2012 02:36 AM, Marcelo Tosatti wrote:
> > On Mon, Jul 30, 2012 at 07:39:31PM +0300, Avi Kivity wrote:
> >> On 07/30/2012 05:07 PM, Chris Clayton wrote:
> >> >>
> >> >>>> With kernel 3.5.0 with b2da15ac26a0c00 reverted, I have just had 15
> >> >>>> clean invocations of vanilla qemu-kvm-1.1.1. So that commit would seem
> >> >>>> to be the problem.
> >> >>>
> >> >>> Just to be sure, I've run some more tests today. No crashes occurred in
> >> >>> 20 runs of vanilla qemu-kvm-1.1.1 on kernel 3.5.0 with b2da15ac26a0c00
> >> >>> reverted.
> >> >>
> >> >> Ok.  I'm trying to reproduce it here on a nested-virt setup, since the
> >> >> code looks correct.
> >> >>
> >> >> What's your preemption settings?
> >> >>
> >> >>
> >> > [chris:~/kernel/linux-3.5.0]$ grep PREEMPT .config
> >> > CONFIG_TREE_PREEMPT_RCU=y
> >> > CONFIG_PREEMPT_RCU=y
> >> > CONFIG_PREEMPT_NOTIFIERS=y
> >> > # CONFIG_PREEMPT_NONE is not set
> >> > # CONFIG_PREEMPT_VOLUNTARY is not set
> >> > CONFIG_PREEMPT=y
> >> > CONFIG_PREEMPT_COUNT=y
> >> 
> >> Here's what I think that is happening
> >> 
> >>   vcpu_load
> >>   ...
> >>   vmx_save_host_state
> >>   vmx_vcpu_run
> >>   (ds.cpl, es.cpl cleared by hardware)
> >> 
> >>   interrupt
> >>     push ds, es  # pushes bad ds, es
> >>     schedule
> >>       vmx_vcpu_put
> >>         vmx_load_host_state
> >>           reload ds, es
> >>     pop ds, es  # of other thread's stack
> >>     iret
> >>   # other thread runs
> >>   interrupt
> >>     schedule  # back in vcpu thread
> >>     interrupt return: pop ds, es  # <-- problem
> >>     iret
> >> 
> >>    ...
> >>    vcpu_put
> >> 
> >>    # bad ds, es, but !vmx->host_state.loaded
> >> 
> >> Marcelo, did I miss something here?
> > 
> > Don't think so.
> 
> So the same problem should happen with %fs and %gs, no?

AFAICS: 

depends on CONFIG_X86_32_LAZY_GS for GS, unconditional for FS.

> x86_64 is safe, since it entry_64.S never saves/restores segment registers.

Is the comment 

        /*
         * The sysexit path does not restore ds/es, so we must set them
         * to
         * a reasonable value ourselves.
         */

Correct?

syscall_exit -> syscall_exit_work -> resume_userspace ->
restore_all -> RESTORE_REGS