Re: Partial huge page backing with KVM/qemu

[Gleb Natapov <gleb@redhat.com>]

On Mon, Aug 26, 2013 at 02:09:57AM +0000, Chris Leduc wrote:
> 
> 
> > -----Original Message-----
> > From: Gleb Natapov [mailto:gleb@redhat.com]
> > Sent: Sunday, August 25, 2013 1:52 AM
> > To: Chris Leduc
> > Cc: kvm@vger.kernel.org
> > Subject: Re: Partial huge page backing with KVM/qemu
> > 
> > On Sat, Aug 24, 2013 at 12:32:07AM +0000, Chris Leduc wrote:
> > > Hi - In a KVM/qemu environment is it possible for the host to back only a
> > portion of the guests memory with huge pages?  In some situations it may
> > not be desirable to back the entirety of a guest's memory with huge pages
> > (as can be done via libvirt memoryBacking option).
> > What are those situations?
> For example to limit a guest with 64GB of total memory to use 4GB of huge pages for fast lookup memory.  This takes advantage of the 4 TLB entries for 1G pages on a Sandy/Ivy Bridge processor to ensure a page walk is never necessary for this fast memory.  An example is a high performance data plane application.  The remainder of the less frequently accessed memory can be in normal pages.
> 
When two level paging (EPT) is in use combined mappings are stored in
TLB, not linear mappings (see 28.3.1). I am not sure those will ever
use 1G TLB.  Not with KVM anyway since KVM does not use 1G pages for EPT
tables since the chance to get as much of contiguous memory on a running
system is close to zero.

> > > What would be very useful is to request huge pages in the guest, either at
> > boot time or dynamically, and have the host back them with physical huge
> > pages, but not back the rest of the normal page guest memory with huge
> > pages from the host.
> > >
> > > The equivalent in Xen is setting allowsuperpage=1 on the hypervisor boot
> > line.
> > >
> > As far as I can tell this disables/enables use of huge pages by XEN vm, not
> > something you say you want.
> 
> The Xen documentation is not clear on this, but in practice this flag allows the host to back up guest huge page requests with physical huge pages.  So the guest could for example add hugepages=N to its boot line and these pages would be backed in the host with corresponding physical huge pages.
Allow me to be sceptical on this :) With shadow paging sure, same is true
for KVM: if guest maps memory with huge page and memory is contiguous on
a host too KVM will create huge shadow page, but with two level paging
hypervisor has no idea how guest's page tables look. The best it can do
is to map entire guest physical memory using huge pages.

> 
> From experimentation with KVM, requesting hugepages at guest boot time (without memory backing enabled) will result in guest hugepages backed by host normal pages.
What do you mean by "requesting hugepages at guest boot time" and how
have you checked that guest hugepages backed by host normal pages? Do
you have THP enabled? Without THP you need to back guest's memory with
huge pages using "–mem-path /hugepagesfs". But again only 2MB pages are
supported.

--
			Gleb.