Strange memory balloning problem even if balloning is disabled

Strange memory balloning problem even if balloning is disabled

[Fabio Fantoni]

On systems with wheezy as dom0 and kernel 3.2 it seems there are no 
visible problems.
After updating the kernel to 3.16 (from backports repository), after 
starting the domUs, I get many of these errors in syslog and kern.log 
(infinite loop):
xen:balloon: Cannot add additional memory (-17)

The strange thing is that it should not use balloning because both dom0 
and domUs have fixed memory set and balloning disabled.
dom0 from grub.cfg: dom0_mem=2G,max:2G
vi /etc/xen/xl.conf: autoballoon=0
domUs xl cfg: memory parameter only, for example memory=2048

Why it seems to try to use balloning even if everything is set with a 
fixed amount of memory?
Could it be a regression in recent kernels or a problem already present 
for some time but only now showing in logs?

I've been suggested by one developer in the mailing list to use this 
workaround for now to avoid at least the problem of oversized logs:
dom0 from grub.cfg: dom0_mem=2G,max:3G

After latest kernel update (3.16.7-ckt4-3~bpo70+1) with this addition:
[xen] cancel ballooning if adding new memory failed (Closes: #776448)
I removed the workaround and retried.
With xen-unstable (4.6 - actual master) it reports only one balloning 
error for each domU after start, whereas with xen 4.5.0 and actual 
stable-4.5 I still have continuous errors about failed balloning. E.g. 
after starting one W7 domU syslog grew by 20 mb in only 3-4 minutes.
I think there is some important fix about memory that still have to be 
backported to 4.5-stable (probably in libxl about domU's create).

Even though I can't observe anything wrong besides errors in logs, if 
there is something wrong in memory calculation and/or use of balloning I 
think it should be found and solved.

Someone can take a look at these please?

Thanks for any reply and sorry for my bad english.

Re: Strange memory balloning problem even if balloning is disabled

[Jan Beulich]

>>> On 02.03.15 at 16:52, <fabio.fantoni@m2r.biz> wrote:
> On systems with wheezy as dom0 and kernel 3.2 it seems there are no 
> visible problems.
> After updating the kernel to 3.16 (from backports repository), after 
> starting the domUs, I get many of these errors in syslog and kern.log 
> (infinite loop):
> xen:balloon: Cannot add additional memory (-17)

I have a vague recollection of some fix from David aiming at
avoiding to issue this message, but you would clearly have better
luck getting a qualified response if you Cc-ed kernel people when
reporting kernel problems (assuming you need explicit Cc-s at all,
as most relevant people read the list anyway afaik) - I certainly
can't figure by what criteria you put together the Cc list.

Jan

Re: Strange memory balloning problem even if balloning is disabled

[David Vrabel]

On 02/03/15 16:23, Jan Beulich wrote:
>>>> On 02.03.15 at 16:52, <fabio.fantoni@m2r.biz> wrote:
>> On systems with wheezy as dom0 and kernel 3.2 it seems there are no 
>> visible problems.
>> After updating the kernel to 3.16 (from backports repository), after 
>> starting the domUs, I get many of these errors in syslog and kern.log 
>> (infinite loop):
>> xen:balloon: Cannot add additional memory (-17)
> 
> I have a vague recollection of some fix from David aiming at
> avoiding to issue this message, but you would clearly have better
> luck getting a qualified response if you Cc-ed kernel people when
> reporting kernel problems (assuming you need explicit Cc-s at all,
> as most relevant people read the list anyway afaik) - I certainly
> can't figure by what criteria you put together the Cc list.

You need

3dcf63677d4eb7fdfc13290c8558c301d2588fe8 (xen/balloon: cancel ballooning
if adding new memory failed)
fd8b79511349efd1f0decea920f61b93acb34a75 (xen/balloon: Don't continue
ballooning when BP_ECANCELED is encountered)

Or you can disable CONFIG_XEN_BALLOON_MEMORY_HOTPLUG.

David

Re: Strange memory balloning problem even if balloning is disabled

[Fabio Fantoni]

[-- Attachment #1.1: Type: text/plain, Size: 2817 bytes --]

Il 02/03/2015 17:31, David Vrabel ha scritto:
> On 02/03/15 16:23, Jan Beulich wrote:
>>>>> On 02.03.15 at 16:52, <fabio.fantoni@m2r.biz> wrote:
>>> On systems with wheezy as dom0 and kernel 3.2 it seems there are no
>>> visible problems.
>>> After updating the kernel to 3.16 (from backports repository), after
>>> starting the domUs, I get many of these errors in syslog and kern.log
>>> (infinite loop):
>>> xen:balloon: Cannot add additional memory (-17)
>> I have a vague recollection of some fix from David aiming at
>> avoiding to issue this message, but you would clearly have better
>> luck getting a qualified response if you Cc-ed kernel people when
>> reporting kernel problems (assuming you need explicit Cc-s at all,
>> as most relevant people read the list anyway afaik) - I certainly
>> can't figure by what criteria you put together the Cc list.
> You need
>
> 3dcf63677d4eb7fdfc13290c8558c301d2588fe8 (xen/balloon: cancel ballooning
> if adding new memory failed)
> fd8b79511349efd1f0decea920f61b93acb34a75 (xen/balloon: Don't continue
> ballooning when BP_ECANCELED is encountered)
>
> Or you can disable CONFIG_XEN_BALLOON_MEMORY_HOTPLUG.
>
> David

Readded the other part of my initial mail that explain better the 
problem (also with the patch above applied)...

The strange thing is that it should not use balloning because both dom0 
and domUs have fixed memory set and balloning disabled.
dom0 from grub.cfg: dom0_mem=2G,max:2G
vi /etc/xen/xl.conf: autoballoon=0
domUs xl cfg: memory parameter only, for example memory=2048

Why it seems to try to use balloning even if everything is set with a 
fixed amount of memory?
Could it be a regression in recent kernels or a problem already present 
for some time but only now showing in logs?

I've been suggested by one developer in the mailing list to use this 
workaround for now to avoid at least the problem of oversized logs:
dom0 from grub.cfg: dom0_mem=2G,max:3G

After latest kernel update (3.16.7-ckt4-3~bpo70+1) with this addition:
[xen] cancel ballooning if adding new memory failed (Closes: #776448)
I removed the workaround and retried.
With xen-unstable (4.6 - actual master) it reports only one balloning 
error for each domU after start, *whereas with xen 4.5.0 and actual 
stable-4.5 I still have continuous errors about failed balloning.* E.g. 
after starting one W7 domU syslog grew by 20 mb in only 3-4 minutes.
I think there is some important fix about memory that still have to be 
backported to 4.5-stable (probably in libxl about domU's create).

Even though I can't observe anything wrong besides errors in logs, if 
there is something wrong in memory calculation and/or use of balloning I 
think it should be found and solved.

Someone can take a look at these please?

Thanks for any reply and sorry for my bad english.

[-- Attachment #1.2: Type: text/html, Size: 3772 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Strange memory balloning problem even if balloning is disabled

[David Vrabel]

On 03/03/15 09:05, Fabio Fantoni wrote:
> Il 02/03/2015 17:31, David Vrabel ha scritto:
>> On 02/03/15 16:23, Jan Beulich wrote:
>>>>>> On 02.03.15 at 16:52, <fabio.fantoni@m2r.biz> wrote:
>>>> On systems with wheezy as dom0 and kernel 3.2 it seems there are no 
>>>> visible problems.
>>>> After updating the kernel to 3.16 (from backports repository), after 
>>>> starting the domUs, I get many of these errors in syslog and kern.log 
>>>> (infinite loop):
>>>> xen:balloon: Cannot add additional memory (-17)
>>> I have a vague recollection of some fix from David aiming at
>>> avoiding to issue this message, but you would clearly have better
>>> luck getting a qualified response if you Cc-ed kernel people when
>>> reporting kernel problems (assuming you need explicit Cc-s at all,
>>> as most relevant people read the list anyway afaik) - I certainly
>>> can't figure by what criteria you put together the Cc list.
>> You need
>>
>> 3dcf63677d4eb7fdfc13290c8558c301d2588fe8 (xen/balloon: cancel ballooning
>> if adding new memory failed)
>> fd8b79511349efd1f0decea920f61b93acb34a75 (xen/balloon: Don't continue
>> ballooning when BP_ECANCELED is encountered)
>>
>> Or you can disable CONFIG_XEN_BALLOON_MEMORY_HOTPLUG.
>>
>> David
> 
> Readded the other part of my initial mail that explain better the
> problem (also with the patch above applied)...
> 
> The strange thing is that it should not use balloning because both dom0
> and domUs have fixed memory set and balloning disabled.

Ballooned pages are required to back grant mapped pages in blkback and
netback.

David

Re: Strange memory balloning problem even if balloning is disabled

[Fabio Fantoni]

Il 03/03/2015 12:09, David Vrabel ha scritto:
> On 03/03/15 09:05, Fabio Fantoni wrote:
>> Il 02/03/2015 17:31, David Vrabel ha scritto:
>>> On 02/03/15 16:23, Jan Beulich wrote:
>>>>>>> On 02.03.15 at 16:52, <fabio.fantoni@m2r.biz> wrote:
>>>>> On systems with wheezy as dom0 and kernel 3.2 it seems there are no
>>>>> visible problems.
>>>>> After updating the kernel to 3.16 (from backports repository), after
>>>>> starting the domUs, I get many of these errors in syslog and kern.log
>>>>> (infinite loop):
>>>>> xen:balloon: Cannot add additional memory (-17)
>>>> I have a vague recollection of some fix from David aiming at
>>>> avoiding to issue this message, but you would clearly have better
>>>> luck getting a qualified response if you Cc-ed kernel people when
>>>> reporting kernel problems (assuming you need explicit Cc-s at all,
>>>> as most relevant people read the list anyway afaik) - I certainly
>>>> can't figure by what criteria you put together the Cc list.
>>> You need
>>>
>>> 3dcf63677d4eb7fdfc13290c8558c301d2588fe8 (xen/balloon: cancel ballooning
>>> if adding new memory failed)
>>> fd8b79511349efd1f0decea920f61b93acb34a75 (xen/balloon: Don't continue
>>> ballooning when BP_ECANCELED is encountered)
>>>
>>> Or you can disable CONFIG_XEN_BALLOON_MEMORY_HOTPLUG.
>>>
>>> David
>> Readded the other part of my initial mail that explain better the
>> problem (also with the patch above applied)...
>>
>> The strange thing is that it should not use balloning because both dom0
>> and domUs have fixed memory set and balloning disabled.
> Ballooned pages are required to back grant mapped pages in blkback and
> netback.
>
> David

Thanks for your reply.
Based on your reply seems I must enable balloning in dom0 (for example 
dom0_mem=2G,max:3G) and/or xl.conf (autoballoon=1) to have it full 
working (for grant mapped pages), is it correct?

Re: Strange memory balloning problem even if balloning is disabled

[Fabio Fantoni]

Il 03/03/2015 12:47, Fabio Fantoni ha scritto:
> Il 03/03/2015 12:09, David Vrabel ha scritto:
>> On 03/03/15 09:05, Fabio Fantoni wrote:
>>> Il 02/03/2015 17:31, David Vrabel ha scritto:
>>>> On 02/03/15 16:23, Jan Beulich wrote:
>>>>>>>> On 02.03.15 at 16:52, <fabio.fantoni@m2r.biz> wrote:
>>>>>> On systems with wheezy as dom0 and kernel 3.2 it seems there are no
>>>>>> visible problems.
>>>>>> After updating the kernel to 3.16 (from backports repository), after
>>>>>> starting the domUs, I get many of these errors in syslog and 
>>>>>> kern.log
>>>>>> (infinite loop):
>>>>>> xen:balloon: Cannot add additional memory (-17)
>>>>> I have a vague recollection of some fix from David aiming at
>>>>> avoiding to issue this message, but you would clearly have better
>>>>> luck getting a qualified response if you Cc-ed kernel people when
>>>>> reporting kernel problems (assuming you need explicit Cc-s at all,
>>>>> as most relevant people read the list anyway afaik) - I certainly
>>>>> can't figure by what criteria you put together the Cc list.
>>>> You need
>>>>
>>>> 3dcf63677d4eb7fdfc13290c8558c301d2588fe8 (xen/balloon: cancel 
>>>> ballooning
>>>> if adding new memory failed)
>>>> fd8b79511349efd1f0decea920f61b93acb34a75 (xen/balloon: Don't continue
>>>> ballooning when BP_ECANCELED is encountered)
>>>>
>>>> Or you can disable CONFIG_XEN_BALLOON_MEMORY_HOTPLUG.
>>>>
>>>> David
>>> Readded the other part of my initial mail that explain better the
>>> problem (also with the patch above applied)...
>>>
>>> The strange thing is that it should not use balloning because both dom0
>>> and domUs have fixed memory set and balloning disabled.
>> Ballooned pages are required to back grant mapped pages in blkback and
>> netback.
>>
>> David
>
> Thanks for your reply.
> Based on your reply seems I must enable balloning in dom0 (for example 
> dom0_mem=2G,max:3G) and/or xl.conf (autoballoon=1) to have it full 
> working (for grant mapped pages), is it correct?

With same dom0 I tried on different server (Dell PE T620 instead Dell PE 
T310) and after xl create of a W7 domU...
Some seconds after dom0 becomes unusable, unreachable over the network 
and not respond to commandsconnecting from drac.
I did forced powerdown and looking the logs I saw only 36mb of this 
error in 2 minutes:
xen:balloon: Cannot add additional memory (-17)

I have kernel with patch that should avoiding this 
(3.16.7-ckt4-3~bpo70+1) and in grub.cfg dom0_mem=2G,max:3G

I not found anything else strange on logs except this in dom0's logs.

If you need more informations or tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.