From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: cr3 OOS optimisation breaks 32-bit GNU/kFreeBSD guest Date: Sun, 22 Mar 2009 11:35:00 +0200 Message-ID: <49C60644.2090904@redhat.com> References: <20090223003305.GW12976@hall.aurel32.net> <20090320231405.GA26415@amt.cnet> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Aurelien Jarno , kvm@vger.kernel.org To: Marcelo Tosatti Return-path: Received: from mx2.redhat.com ([66.187.237.31]:34145 "EHLO mx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750924AbZCVJfO (ORCPT ); Sun, 22 Mar 2009 05:35:14 -0400 In-Reply-To: <20090320231405.GA26415@amt.cnet> Sender: kvm-owner@vger.kernel.org List-ID: Marcelo Tosatti wrote: > On Mon, Feb 23, 2009 at 01:33:05AM +0100, Aurelien Jarno wrote: > =20 >> Hi, >> >> Since kvm-81, I have noticed that GNU/kFreeBSD 32-bit guest are cras= hing >> under high load (during a compilation for example) with the followin= g=20 >> error message: >> >> | Fatal trap 12: page fault while in kernel mode >> | fault virtual address =3D 0x4 >> | fault code =3D supervisor read, page not present >> | instruction pointer =3D 0x20:0xc0a4fc00 >> | stack pointer =3D 0x28:0xe66d7a70 >> | frame pointer =3D 0x28:0xe66d7a80 >> | code segment =3D base 0x0, limit 0xfffff, type 0x1b >> | =3D DPL 0, pres 1, def32 1, gran 1 >> | processor eflags =3D interrupt enabled, resume, IOPL =3D 0 >> | current process =3D 24037 (bash) >> | trap number =3D 12 >> | panic: page fault >> | Uptime: 4m7s >> | Cannot dump. No dump device defined. >> | Automatic reboot in 15 seconds - press a key on the console to abo= rt >> =20 > > Aurelien, > > Can you try this patch please. > > ------- > > KVM: MMU: do not allow unsync global pages to become unreachable > > Section 5.1 of the Intel TLB doc: > > "=E2=80=A2 INVLPG. This instruction takes a single operand, which is = a linear > address. The instruction invalidates any TLB entries for the page num= ber > of that linear address including those for global pages (see Section > 7). INVLPG also invalidates all entries in all paging-structure cache= s > regardless of the linear addresses to which they correspond." > > Section 7: > > "The following items detail the invalidation of global TLB entries: > > =E2=80=A2 An execution of INVLPG invalidates any TLB entries for the = page > number of the linear address that is its operand, even if the entries > are global." > > If an unsync global page becomes unreachable via the shadow tree, whi= ch > can happen if one its parent pages is zapped, invlpg will fail to > invalidate translations for gvas contained in such unreachable pages. > > So invalidate all unsync global entries when zapping a page. Another > possibility would be to cover global pages in the unsync bitmaps, but > that would increase overhead for mmu_sync_roots. > > Index: kvm/arch/x86/kvm/mmu.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- kvm.orig/arch/x86/kvm/mmu.c > +++ kvm/arch/x86/kvm/mmu.c > @@ -1362,6 +1362,16 @@ static void kvm_mmu_unlink_parents(struc > } > } > =20 > +static void mmu_invalidate_unsync_global(struct kvm *kvm) > +{ > + struct kvm_mmu_page *sp, *n; > + > + list_for_each_entry_safe(sp, n, &kvm->arch.oos_global_pages, oos_li= nk) { > + kvm_mmu_page_unlink_children(kvm, sp); > + kvm_unlink_unsync_page(kvm, sp); > + } > +} > + > static int mmu_zap_unsync_children(struct kvm *kvm, > struct kvm_mmu_page *parent) > { > @@ -1384,6 +1394,8 @@ static int mmu_zap_unsync_children(struc > kvm_mmu_pages_init(parent, &parents, &pages); > } > =20 > + mmu_invalidate_unsync_global(kvm); > + > return zapped; > } > =20 > =20 Good catch, indeed. But is it sufficient? We could unlink a page=20 through other means, for example by the guest zapping a page directory=20 entry. Maybe it's best to resync when relinking a global page? --=20 error compiling committee.c: too many arguments to function