From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gleb Natapov Subject: Re: Partial huge page backing with KVM/qemu Date: Mon, 26 Aug 2013 11:02:09 +0300 Message-ID: <20130826080209.GX8218@redhat.com> References: <7F858974825DC945874D45551C161B941156CCC4@eusaamb105.ericsson.se> <20130825085210.GI8218@redhat.com> <7F858974825DC945874D45551C161B941156D883@eusaamb105.ericsson.se> Mime-Version: 1.0 Content-Type: text/plain; charset=cp1255 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: "kvm@vger.kernel.org" To: Chris Leduc Return-path: Received: from mx1.redhat.com ([209.132.183.28]:31322 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755240Ab3HZICM convert rfc822-to-8bit (ORCPT ); Mon, 26 Aug 2013 04:02:12 -0400 Content-Disposition: inline In-Reply-To: <7F858974825DC945874D45551C161B941156D883@eusaamb105.ericsson.se> Sender: kvm-owner@vger.kernel.org List-ID: On Mon, Aug 26, 2013 at 02:09:57AM +0000, Chris Leduc wrote: >=20 >=20 > > -----Original Message----- > > From: Gleb Natapov [mailto:gleb@redhat.com] > > Sent: Sunday, August 25, 2013 1:52 AM > > To: Chris Leduc > > Cc: kvm@vger.kernel.org > > Subject: Re: Partial huge page backing with KVM/qemu > >=20 > > On Sat, Aug 24, 2013 at 12:32:07AM +0000, Chris Leduc wrote: > > > Hi - In a KVM/qemu environment is it possible for the host to bac= k only a > > portion of the guests memory with huge pages?=A0 In some situations= it may > > not be desirable to back the entirety of a guest's memory with huge= pages > > (as can be done via libvirt memoryBacking option). > > What are those situations? > For example to limit a guest with 64GB of total memory to use 4GB of = huge pages for fast lookup memory. This takes advantage of the 4 TLB e= ntries for 1G pages on a Sandy/Ivy Bridge processor to ensure a page wa= lk is never necessary for this fast memory. An example is a high perfo= rmance data plane application. The remainder of the less frequently ac= cessed memory can be in normal pages. >=20 When two level paging (EPT) is in use combined mappings are stored in TLB, not linear mappings (see 28.3.1). I am not sure those will ever use 1G TLB. Not with KVM anyway since KVM does not use 1G pages for EP= T tables since the chance to get as much of contiguous memory on a runnin= g system is close to zero. > > > What would be very useful is to request huge pages in the guest, = either at > > boot time or dynamically, and have the host back them with physical= huge > > pages, but not back the rest of the normal page guest memory with h= uge > > pages from the host. > > > > > > The equivalent in Xen is setting allowsuperpage=3D1 on the hyperv= isor boot > > line. > > > > > As far as I can tell this disables/enables use of huge pages by XEN= vm, not > > something you say you want. >=20 > The Xen documentation is not clear on this, but in practice this flag= allows the host to back up guest huge page requests with physical huge= pages. So the guest could for example add hugepages=3DN to its boot l= ine and these pages would be backed in the host with corresponding phys= ical huge pages. Allow me to be sceptical on this :) With shadow paging sure, same is tr= ue for KVM: if guest maps memory with huge page and memory is contiguous o= n a host too KVM will create huge shadow page, but with two level paging hypervisor has no idea how guest's page tables look. The best it can do is to map entire guest physical memory using huge pages. >=20 > From experimentation with KVM, requesting hugepages at guest boot tim= e (without memory backing enabled) will result in guest hugepages backe= d by host normal pages. What do you mean by "requesting hugepages at guest boot time" and how have you checked that guest hugepages backed by host normal pages? Do you have THP enabled? Without THP you need to back guest's memory with huge pages using "=96mem-path /hugepagesfs". But again only 2MB pages a= re supported. -- Gleb.