Re: PROBLEM: Xen ballon driver seems to be broken in 2.6.26

Re: PROBLEM: Xen ballon driver seems to be broken in 2.6.26

[Jeremy Fitzhardinge]

Németh Tamás wrote:
> I use openSUSE 10.3 both for dom0 and domU, but the kernels and the entire Xen 
> system (hypervisor, tools and dom0 kernel) is replaced by the newest version 
> downloadable from xen.org. Every component is compiled from source. The 
> hypervisor is 64 bit, and the dom0 and domU systems are 32 bit, PAE. The domU 
> kernel is the vanilla 2.6.26. It's OK, that migration, save, restore, etc. 
> don't work, as you plan the support for these features to be included in 
> 2.6.27 AFAIK. (BTW, when will you include the domU support for x86_64 and CPU 
> hotplug?) But:
>
> The balloon driver also seems to be broken. If i decrease the amount of memory 
> in domU, it becomes immediately almost unresponsive. I can ping the domU, and 
> even Xvnc continues to operate (the "screen" of domU changes, the KDE clock 
> continues to count the seconds), but i cannot start new programs. Sadly it 
> includes the fact that i can't even run dmesg. Neither the output of 'xm 
> console'  gives any error message. When i destroy and restart the domU then, 
> i cannot find anything about the crash in /var/log/messages. When i try to 
> start a new program, then the domU totally hangs finally, the KDE clock 
> stops, and it cannot be shut down (only destroyed).
>   

So that's consistent with the balloon driver over-aggressively shrinking 
the domain, until there's no memory left to do anything.  The balloon 
driver has no safety checks to prevent that from happening.

What value are you setting the target size to?  Are you using the right 
units?

    J

RE: Re: PROBLEM: Xen ballon driver seems to be broken in 2.6.26

[Dan Magenheimer]

[-- Attachment #1: Type: text/plain, Size: 2631 bytes --]

Jeremy --

A patch was made in Xen to the balloon driver a few months ago
to avoid this problem.  The patch should probably be applied
to the balloon driver in upstream Linux.

Nemeth --

What is your "newest" Xen version?  If 3.2, the balloon driver fix
isn't there yet, but I have a bash-script workaround.  See
tools/xenballoond in xen-unstable.

Dan

> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com]On Behalf Of Jeremy
> Fitzhardinge
> Sent: Tuesday, July 22, 2008 12:03 PM
> To: Németh Tamás
> Cc: xen-devel@lists.xensource.com; jeremy@xensource.com
> Subject: [Xen-devel] Re: PROBLEM: Xen ballon driver seems to be broken
> in 2.6.26
> 
> 
> Németh Tamás wrote:
> > I use openSUSE 10.3 both for dom0 and domU, but the kernels 
> and the entire Xen 
> > system (hypervisor, tools and dom0 kernel) is replaced by 
> the newest version 
> > downloadable from xen.org. Every component is compiled from 
> source. The 
> > hypervisor is 64 bit, and the dom0 and domU systems are 32 
> bit, PAE. The domU 
> > kernel is the vanilla 2.6.26. It's OK, that migration, 
> save, restore, etc. 
> > don't work, as you plan the support for these features to 
> be included in 
> > 2.6.27 AFAIK. (BTW, when will you include the domU support 
> for x86_64 and CPU 
> > hotplug?) But:
> >
> > The balloon driver also seems to be broken. If i decrease 
> the amount of memory 
> > in domU, it becomes immediately almost unresponsive. I can 
> ping the domU, and 
> > even Xvnc continues to operate (the "screen" of domU 
> changes, the KDE clock 
> > continues to count the seconds), but i cannot start new 
> programs. Sadly it 
> > includes the fact that i can't even run dmesg. Neither the 
> output of 'xm 
> > console'  gives any error message. When i destroy and 
> restart the domU then, 
> > i cannot find anything about the crash in 
> /var/log/messages. When i try to 
> > start a new program, then the domU totally hangs finally, 
> the KDE clock 
> > stops, and it cannot be shut down (only destroyed).
> >   
> 
> So that's consistent with the balloon driver 
> over-aggressively shrinking 
> the domain, until there's no memory left to do anything.  The balloon 
> driver has no safety checks to prevent that from happening.
> 
> What value are you setting the target size to?  Are you using 
> the right 
> units?
> 
>     J
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: PROBLEM: Xen ballon driver seems to be broken in 2.6.26

[Jeremy Fitzhardinge]

Dan Magenheimer wrote:
> A patch was made in Xen to the balloon driver a few months ago
> to avoid this problem.  The patch should probably be applied
> to the balloon driver in upstream Linux.
>   

I should check that out again, but my memory is that I didn't really 
agree with the approach being taken.

    J

RE: Re: PROBLEM: Xen ballon driver seems to be broken in 2.6.26

[Dan Magenheimer]

[-- Attachment #1: Type: text/plain, Size: 426 bytes --]

> Dan Magenheimer wrote:
> > A patch was made in Xen to the balloon driver a few months ago
> > to avoid this problem.  The patch should probably be applied
> > to the balloon driver in upstream Linux.
> >
> 
> I should check that out again, but my memory is that I didn't really
> agree with the approach being taken.

For the record, I don't agree with the approach either.  But
the results do agree with me. :-)

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: PROBLEM: Xen ballon driver seems to be broken in 2.6.26

[Jeremy Fitzhardinge]

Németh Tamás wrote:
>> What is your "newest" Xen version?  If 3.2, the balloon driver fix
>> isn't there yet, but I have a bash-script workaround.  See
>> tools/xenballoond in xen-unstable.
>>     
>
>  The source codes of the hyperhisor and the tools were downloaded from
> xen.org a few days ago, so Xen is 3.2.1 and the dom0 kernel is
> 2.6.18.8-xen-3.2.0 (based on 2.6.18.8).
>
>  The domU kernel is 2.6.26 of course.
>
>
>
>   
>>> So that's consistent with the balloon driver
>>> over-aggressively shrinking
>>> the domain, until there's no memory left to do anything.  The balloon
>>> driver has no safety checks to prevent that from happening.
>>>
>>> What value are you setting the target size to?  Are you using
>>> the right
>>> units?
>>>       
>
> Please take a look at this:
>
> carrier5:/usr/local/LOCALHOST/src/linux-2.6.26 # xm list
> Name                                        ID   Mem VCPUs      State   Time(s)
> Domain-0                                     0  2048     1     r-----    370.3
> u-96                                         5  1024     1     -b----     14.5
> carrier5:/usr/local/LOCALHOST/src/linux-2.6.26 # xm mem-set u-96 768M
> carrier5:/usr/local/LOCALHOST/src/linux-2.6.26 # xm list
> Name                                        ID   Mem VCPUs      State   Time(s)
> Domain-0                                     0  2048     1     r-----    371.4
> u-96                                         5   768     1     -b----     14.6
> carrier5:/usr/local/LOCALHOST/src/linux-2.6.26 #
>
> DomU - although being seemngly frozen - still responses for icmp echo
> requests.

Thanks, I'll have a look at it.

    J