(XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016

(XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016

[Konrad Rzeszutek Wilk]

I am not sure how this is happening, but I can only reproduce this
is I use the "max" parameter in the 'dom0_mem=*:max:*' combination:

sh-4.1# xl info | grep dom0_mem
xen_commandline        : cpuinfo dom0_mem=1G,max:2711M cpufreq=verbose com1=115200,8n1 console=com1,vga loglvl=all guest_loglvl=


02:32:22 # 19 :/sys/devices/system/xen_memory/xen_memory0/ 
> cat /proc/meminfo |grep Direct
DirectMap4k:     2776516 kB
DirectMap2M:           0 kB

02:33:11 # 23 :/sys/devices/system/xen_memory/xen_memory0/
> echo 2776516 > target_kb

(XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016
(XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 (239 of 256)
(XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016
(XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 (0 of 17)
(XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016
(XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 (0 of 17)

>
The value that it decided it was Ok with was: 2776510kB which is 6 pages short
of the goal and that makes the messages disappear.

Any ideas of what that might be? Could it be the shared_info, hypercall page,
start_info, xenconsole and some other ones are the magic 6 pages which
inhibit how much we can balloon up to?

Re: (XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016

[Jan Beulich]

>>> On 27.04.12 at 04:34, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> I am not sure how this is happening, but I can only reproduce this
> is I use the "max" parameter in the 'dom0_mem=*:max:*' combination:

Of course - without the ",max:" there just isn't any limit for Dom0.

> sh-4.1# xl info | grep dom0_mem
> xen_commandline        : cpuinfo dom0_mem=1G,max:2711M cpufreq=verbose 
> com1=115200,8n1 console=com1,vga loglvl=all guest_loglvl=
> 
> 
> 02:32:22 # 19 :/sys/devices/system/xen_memory/xen_memory0/ 
>> cat /proc/meminfo |grep Direct
> DirectMap4k:     2776516 kB
> DirectMap2M:           0 kB
> 
> 02:33:11 # 23 :/sys/devices/system/xen_memory/xen_memory0/
>> echo 2776516 > target_kb
> 
> (XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016
> (XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 
> (239 of 256)
> (XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016
> (XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 (0 
> of 17)
> (XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016
> (XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 (0 
> of 17)
> 
>>
> The value that it decided it was Ok with was: 2776510kB which is 6 pages 
> short of the goal and that makes the messages disappear.

How would that be? 2711MiB = 2776064kiB, which 446k off the value
above. And apart from that, the value above isn't even divisible by 4
(i.e. not an even number of pages).

> Any ideas of what that might be? Could it be the shared_info, hypercall page,
> start_info, xenconsole and some other ones are the magic 6 pages which
> inhibit how much we can balloon up to?

Not likely: The hypercall page is in kernel (image) memory, and there's
no console page at all fro Dom0.

Jan

Re: (XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016

[Konrad Rzeszutek Wilk]

> How would that be? 2711MiB = 2776064kiB, which 446k off the value
> above. And apart from that, the value above isn't even divisible by 4

I messed up on that. Redid the numbers and I was off.

> (i.e. not an even number of pages).

To make this a bit easier I used 'dom0_max=max:3G', which means
(with this swiss-cheese type E820 on this Intel box):

[    0.000000] Released 75745 pages of unused memory

so I should have 75745 pages left to play with. But what I found is that
I can only go up to 786415 which is 17 pages short of the 786432 goal.

Here are the steps:

$cat `find /sys -name current_kb`
2842816
$echo $((3*1024*1024))
3145728
$echo "3145728" > `find /sys -name target_kb`
$cat `find /sys -name current_kb`
3145660
$xl dmesg | tail
(XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 786433 (786432) > 786432
(XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 (0 of 17)

> > Any ideas of what that might be? Could it be the shared_info, hypercall page,
> > start_info, xenconsole and some other ones are the magic 6 pages which
> > inhibit how much we can balloon up to?
> 
> Not likely: The hypercall page is in kernel (image) memory, and there's
> no console page at all fro Dom0.

17 pages.. Hmm
> 
> Jan

Re: (XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016

[Konrad Rzeszutek Wilk]

On Fri, Apr 27, 2012 at 10:31:22AM -0400, Konrad Rzeszutek Wilk wrote:
> > How would that be? 2711MiB = 2776064kiB, which 446k off the value
> > above. And apart from that, the value above isn't even divisible by 4
> 
> I messed up on that. Redid the numbers and I was off.
> 
> > (i.e. not an even number of pages).
> 
> To make this a bit easier I used 'dom0_max=max:3G', which means
> (with this swiss-cheese type E820 on this Intel box):
> 
> [    0.000000] Released 75745 pages of unused memory
> 
> so I should have 75745 pages left to play with. But what I found is that
> I can only go up to 786415 which is 17 pages short of the 786432 goal.
> 
> Here are the steps:
> 
> $cat `find /sys -name current_kb`
> 2842816
> $echo $((3*1024*1024))
> 3145728
> $echo "3145728" > `find /sys -name target_kb`
> $cat `find /sys -name current_kb`
> 3145660
> $xl dmesg | tail
> (XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 786433 (786432) > 786432
> (XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 (0 of 17)
> 
> > > Any ideas of what that might be? Could it be the shared_info, hypercall page,
> > > start_info, xenconsole and some other ones are the magic 6 pages which
> > > inhibit how much we can balloon up to?
> > 
> > Not likely: The hypercall page is in kernel (image) memory, and there's
> > no console page at all fro Dom0.
> 
> 17 pages.. Hmm

I am still not exactly sure what the problem is, but by running this on
various machines I found that I can be off by 1,2,3, 4 or 17 pages. It
looked to vary based on the amount of ACPI tables that showed up in MADT.

So I think what is happening is that the initial domain gets the ACPI
regions (which are shared) accounted in its d->tot_pages. I can't
pinpoint the exact piece of code in the hypervisor for this.

But what I did do on the Linux side was using the current_reservation
hypercall (so d->tot_pages) and based on that would populate exactly
up to start_info->nr_pages - d->tot_pages pages and did not get any of
those errors.

> > 
> > Jan
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel