From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: cr3 OOS optimisation breaks 32-bit GNU/kFreeBSD guest Date: Mon, 23 Feb 2009 17:16:12 +0200 Message-ID: <49A2BDBC.7070804@redhat.com> References: <20090223003305.GW12976@hall.aurel32.net> <20090223014713.GA11438@amt.cnet> <20090223140115.GB5946@hall.aurel32.net> <20090223145208.GA5782@amt.cnet> <49A2B9D9.3030103@redhat.com> <20090223150638.GA5798@amt.cnet> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Aurelien Jarno , kvm@vger.kernel.org To: Marcelo Tosatti Return-path: Received: from mx2.redhat.com ([66.187.237.31]:36462 "EHLO mx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754177AbZBWPQX (ORCPT ); Mon, 23 Feb 2009 10:16:23 -0500 In-Reply-To: <20090223150638.GA5798@amt.cnet> Sender: kvm-owner@vger.kernel.org List-ID: Marcelo Tosatti wrote: > On Mon, Feb 23, 2009 at 04:59:37PM +0200, Avi Kivity wrote: > =20 >> Marcelo Tosatti wrote: >> =20 >>>> Thanks for your fast answer and for your help for debugging. >>>> =20 >>>> =20 >>> If you confirm that FreeBSD is indeed relying on cr3 to sync global >>> pages, it might be better to disable the optimization. Lets hope th= at is >>> not the case. >>> =20 >>> =20 >> cr3 writes explicitly do not flush global pages; otherwise what woul= d be =20 >> the point of global pages at all? >> =20 > > From the Intel TLB doc: > > The processor is always free to invalidate additional entries in the = TLBs > and paging-structure caches. The following are some examples: > > =E2=80=A2 MOV to CR3 may invalidate TLB entries for global pages. > > The reasoning was if an optimization breaks an important guest which > contains a bug that happens to not trigger on real HW due to position= ing > of the stars, it is reasonable to disable that optimization. > =20 This means the OS may not rely on the TLB retaining its contents. For=20 example, you can't do 1. set pte to global+present 2. access through pte to load tlb entry 3. clear pte 4. switch cr3 5. access through same pte again, relying on tlb entry to service the= =20 access So the processor may choose to ignore the global bit on some or all tlb= =20 entries, but software cannot assume that it does. Typically it will=20 honor the global bit since otherwise it's useless. I don't think this is what is happening with FreeBSD. It may be that=20 spte population on invlpg is confusing the guest (though that is allowe= d=20 as a speculative read?). For example, the sequence: 1. invlpg 2. set pte to A (present+accessed) 3. set pte to B (present+accessed) kvm behaves as if a speculative read always happens between 2 and 3,=20 which would be very rare on real hardware. --=20 error compiling committee.c: too many arguments to function