[PATCH] Fix memory leak in xend

[PATCH] Fix memory leak in xend

[Jim Fehlig]

[-- Attachment #1: Type: text/plain, Size: 252 bytes --]

We observed OOM kills of xend after ~20 days in openSUSE build service
workers.  xend reached around 2.5 G virt, 1.5G res, and 1.8G shared
memory usage. Attached patch provides a plug.

Cheers,
Jim


    Signed-off-by: Jim Fehlig <jfehlig@novell.com>


[-- Attachment #2: xend-memleak.patch --]
[-- Type: text/x-patch, Size: 433 bytes --]

diff -r bd97e45e073a tools/python/xen/xend/XendDomainInfo.py
--- a/tools/python/xen/xend/XendDomainInfo.py	Tue Jul 08 09:28:50 2008 +0100
+++ b/tools/python/xen/xend/XendDomainInfo.py	Thu Jul 10 15:16:57 2008 -0600
@@ -2607,6 +2607,7 @@ class XendDomainInfo:
     def _cleanupVm(self):
         """Cleanup VM resources.  Idempotent.  Nothrow guarantee."""
 
+        self.metrics.destroy()
         self._unwatchVm()
 
         try:

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH] Fix memory leak in xend

[Stefan de Konink]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Jim Fehlig schreef:
> We observed OOM kills of xend after ~20 days in openSUSE build service
> workers.  xend reached around 2.5 G virt, 1.5G res, and 1.8G shared
> memory usage. Attached patch provides a plug.

Did you see problems after this patch? I have applied it on 3.2.1 which
borks about missing vm_metrics.


Stefan
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEAREKAAYFAki3HhkACgkQYH1+F2Rqwn049gCeIYOm5MSf/NQfgdb9TC009wjS
9l8An0PeV5xp/jYamySsSACJO1OI2Jp7
=bJwS
-----END PGP SIGNATURE-----

Re: [PATCH] Fix memory leak in xend

[Zhigang Wang]

[-- Attachment #1: Type: text/plain, Size: 947 bytes --]

I also encountered this many times. I come up with a patch.

Jim, please review.

thanks,

zhigang

Stefan de Konink wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
> 
> Jim Fehlig schreef:
>> We observed OOM kills of xend after ~20 days in openSUSE build service
>> workers.  xend reached around 2.5 G virt, 1.5G res, and 1.8G shared
>> memory usage. Attached patch provides a plug.
> 
> Did you see problems after this patch? I have applied it on 3.2.1 which
> borks about missing vm_metrics.
> 
> 
> Stefan
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.9 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iEYEAREKAAYFAki3HhkACgkQYH1+F2Rqwn049gCeIYOm5MSf/NQfgdb9TC009wjS
> 9l8An0PeV5xp/jYamySsSACJO1OI2Jp7
> =bJwS
> -----END PGP SIGNATURE-----
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


[-- Attachment #2: block-vm-metrics-destroy-exception.patch --]
[-- Type: text/x-patch, Size: 855 bytes --]

This patch prevent VM metrics destroy from raising exceptions.

Everytime an error accured during VM start, VM metrics destroy will raise an 
exception and override the right exception which should be raised to the upper
invocation.

Signed-off-by: Zhigang Wang <zhigang.x.wang@oracle.com>

--- xen-unstable/tools/python/xen/xend/XendDomainInfo.py	2008-08-28 10:17:49.000000000 +0800
+++ xen-unstable.local/tools/python/xen/xend/XendDomainInfo.py	2008-08-29 09:04:11.000000000 +0800
@@ -2656,7 +2656,11 @@ class XendDomainInfo:
 
         from xen.xend import XendDomain
         if not XendDomain.instance().is_domain_managed(self):
-            self.metrics.destroy()
+            try:
+                self.metrics.destroy()
+            except:
+                log.exception("Destroy VM metrics failed.")
+
         self._unwatchVm()
 
         try:

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH] Fix memory leak in xend

[Stefan de Konink]

On Fri, 29 Aug 2008, Zhigang Wang wrote:

> I also encountered this many times. I come up with a patch.
>
> Jim, please review.
>
> thanks,
>
> zhigang
>
> Stefan de Konink wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA512
> >
> > Jim Fehlig schreef:
> >> We observed OOM kills of xend after ~20 days in openSUSE build service
> >> workers.  xend reached around 2.5 G virt, 1.5G res, and 1.8G shared
> >> memory usage. Attached patch provides a plug.
> >
> > Did you see problems after this patch? I have applied it on 3.2.1 which
> > borks about missing vm_metrics.
> >
> >
> > Stefan
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v2.0.9 (GNU/Linux)
> > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> >
> > iEYEAREKAAYFAki3HhkACgkQYH1+F2Rqwn049gCeIYOm5MSf/NQfgdb9TC009wjS
> > 9l8An0PeV5xp/jYamySsSACJO1OI2Jp7
> > =bJwS
> > -----END PGP SIGNATURE-----
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
>

The patch you provide seems to be for the 3.3 version right? The
is_managed attribute/function, is it available for 3.2? Can anyone
backport this?

(I think giving me a reference to a kernel that is older than 2.6.18 and
supports x86_64 + Xen 3.3 is more work ;)


Stefan

Re: [PATCH] Fix memory leak in xend

[Zhigang Wang]

Stefan de Konink wrote:
> On Fri, 29 Aug 2008, Zhigang Wang wrote:
> 
>> I also encountered this many times. I come up with a patch.
>>
>> Jim, please review.
>>
>> thanks,
>>
>> zhigang
>>
>> Stefan de Konink wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA512
>>>
>>> Jim Fehlig schreef:
>>>> We observed OOM kills of xend after ~20 days in openSUSE build service
>>>> workers.  xend reached around 2.5 G virt, 1.5G res, and 1.8G shared
>>>> memory usage. Attached patch provides a plug.
>>> Did you see problems after this patch? I have applied it on 3.2.1 which
>>> borks about missing vm_metrics.
>>>
>>>
>>> Stefan
>>> -----BEGIN PGP SIGNATURE-----
>>> Version: GnuPG v2.0.9 (GNU/Linux)
>>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>>>
>>> iEYEAREKAAYFAki3HhkACgkQYH1+F2Rqwn049gCeIYOm5MSf/NQfgdb9TC009wjS
>>> 9l8An0PeV5xp/jYamySsSACJO1OI2Jp7
>>> =bJwS
>>> -----END PGP SIGNATURE-----
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
> 
> The patch you provide seems to be for the 3.3 version right? The
> is_managed attribute/function, is it available for 3.2? Can anyone
> backport this?
> 
I don't think this patch can solve your VM start problem. It only gives your
more accurate description of the problem you are encountering.
you should check /var/log/xen/xend-debug.log to see which is the right causing
of this problem.
> (I think giving me a reference to a kernel that is older than 2.6.18 and
> supports x86_64 + Xen 3.3 is more work ;)
Xen 3.3 has the same ABI of 3.0+. you can try the latest fedora kernels.

zhigang
> 
> 
> Stefan
> 

Re: [PATCH] Fix memory leak in xend

[Stefan de Konink]

On Fri, 29 Aug 2008, Zhigang Wang wrote:

> > The patch you provide seems to be for the 3.3 version right? The
> > is_managed attribute/function, is it available for 3.2? Can anyone
> > backport this?
> >
> I don't think this patch can solve your VM start problem. It only gives your
> more accurate description of the problem you are encountering.
> you should check /var/log/xen/xend-debug.log to see which is the right causing
> of this problem.

The problem that I encounter like Jim are massive python processes... that
will eventually OOM. So only in that perspective I want a solution.

> > (I think giving me a reference to a kernel that is older than 2.6.18 and
> > supports x86_64 + Xen 3.3 is more work ;)
> Xen 3.3 has the same ABI of 3.0+. you can try the latest fedora kernels.

I'm running on a Gentoo 2.6.21 Xen patched kernel now, so if I can use
that within 3.3 I'll give it a shot. (And apply your patch)


Stefan

Re: [PATCH] Fix memory leak in xend

[Jim Fehlig]

Stefan de Konink wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> Jim Fehlig schreef:memory leak
>   
>> We observed OOM kills of xend after ~20 days in openSUSE build service
>> workers.  xend reached around 2.5 G virt, 1.5G res, and 1.8G shared
>> memory usage. Attached patch provides a plug.
>>     
>
> Did you see problems after this patch? I have applied it on 3.2.1 which
> borks about missing vm_metrics.
>   

Yeah :-(.  The patch got committed as c/s 18030.  You'll also want 18064 
- and maybe even 18140.

See http://lists.xensource.com/archives/html/xen-devel/2008-07/msg01230.html

Jim

[PATCH] Fix memory leak in xend

[Kouya Shimura]

[-- Attachment #1: message body text --]
[-- Type: text/plain, Size: 79 bytes --]

Fix memory leak in xend.

Signed-off-by: Kouya Shimura <kouya@jp.fujitsu.com>


[-- Attachment #2: leak.patch --]
[-- Type: text/plain, Size: 355 bytes --]

diff -r a00cc97b392a tools/python/xen/lowlevel/xs/xs.c
--- a/tools/python/xen/lowlevel/xs/xs.c	Wed Sep 12 09:43:33 2007 +0100
+++ b/tools/python/xen/lowlevel/xs/xs.c	Thu Sep 13 13:43:04 2007 +0900
@@ -365,6 +365,7 @@ static PyObject *xspy_set_permissions(Xs
         goto exit;
     }
 
+    free(xsperms);
     Py_INCREF(Py_None);
     return Py_None;
 

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

[PATCH] Fix 'memory leak' in xend

[Jim Fehlig]

[-- Attachment #1: Type: text/plain, Size: 456 bytes --]

I found that xend's memory usage grows considerably when running a
script such as

while true; do xm new foo ; sleep 2s; xm delete foo ; sleep 2s; done

XendAPIStore maintains a list of class instances and in the case of
new/create operation, an associated VMMetrics object is stashed in the
list but never removed on delete/shutdown.  This patch removes the
instance by invoking XendBase.destroy() method.

Signed-off-by: Jim Fehlig <jfehlig@novell.com>


[-- Attachment #2: xend_mem_leak.diff --]
[-- Type: text/x-patch, Size: 546 bytes --]

diff -r 9071521d4864 tools/python/xen/xend/XendDomain.py
--- a/tools/python/xen/xend/XendDomain.py	Fri Sep 07 11:39:10 2007 +0100
+++ b/tools/python/xen/xend/XendDomain.py	Mon Sep 10 11:35:59 2007 -0600
@@ -1088,6 +1088,7 @@ class XendDomain:
         log.info("Domain %s (%s) deleted." %
                  (dominfo.getName(), dominfo.info.get('uuid')))
                 
+        dominfo.metrics.destroy()
         self._managed_domain_unregister(dominfo)
         self._remove_domain(dominfo)
         XendDevices.destroy_device_state(dominfo)

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel