Re: Can't boot guest with more than 3585MB when using large pages

[Alex Williamson <alex.williamson@hp.com>]

On Wed, 2009-03-25 at 13:10 -0300, Marcelo Tosatti wrote:
> On Tue, Mar 24, 2009 at 04:57:46PM -0500, Ryan Harper wrote:
> > * Alex Williamson <alex.williamson@hp.com> [2009-03-24 16:07]:
> > > 
> > > On a 2.6.29, x86_64 host/guest, what's special about specifying a guest
> > > size of -m 3586 when using -mem-path backed by hugetlbfs?  3585 works,
> > > 3586 hangs here:
> > > 
> > > ...
> > > PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
> > > Placing 64MB software IO TLB between ffff880020000000 - ffff880024000000
> > > software IO TLB at phys 0x20000000 - 0x24000000
> > > Memory: 3504832k/4196352k available (2926k kernel code, 524740k absent, 166780k reserved, 1260k data, 496k init)
> > 
> > I've seen this as well, haven't had a chance to dig into the issue yet
> > either.  Certainly can test patches if anyone has an idea of what's
> > wrong here.
> 
> Can you strace and see if the mmap on hugetlbfs is correctly sized?

Seems reasonable with some 2MB rounding.

Failing case, -m 3586:

open("/hugepages//kvm.5fuuH5", O_RDWR|O_CREAT|O_EXCL, 0600) = 9
unlink("/hugepages//kvm.5fuuH5")        = 0
ftruncate(9, 3783262208)                = 0
mmap(NULL, 3783262208, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_POPULATE, 9, 0) = 0x7f37a5e00000

Working case, -m 3585:

open("/hugepages//kvm.Mv6Zgd", O_RDWR|O_CREAT|O_EXCL, 0600) = 9
unlink("/hugepages//kvm.Mv6Zgd")        = 0
ftruncate(9, 3781165056)                = 0
mmap(NULL, 3781165056, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_POPULATE, 9, 0) = 0x7fd44b800000

Working case using disk backing: -mem-path /tmp -mem-prealloc -m 3586:

open("/tmp/kvm.nPlxl1", O_RDWR|O_CREAT|O_EXCL, 0600) = 9
unlink("/tmp/kvm.nPlxl1")               = 0
ftruncate(9, 3783262208)                = 0
mmap(NULL, 3783262208, PROT_READ|PROT_WRITE, MAP_PRIVATE, 9, 0) = 0x7f432e055000