From mboxrd@z Thu Jan  1 00:00:00 1970
From: Dan Magenheimer <dan.magenheimer@oracle.com>
Subject: RE: tmem and construct_dom0 memory allocation race
Date: Tue, 22 Jun 2010 10:23:20 -0700 (PDT)
Message-ID: <2f4192b4-cdb8-495f-a469-b1dc2b4a9044@default>
References: <AANLkTilpcK7fxh10G8WaPTCQvamAjl5crV_24L9-W-An@mail.gmail.com
	C84626FD.1827E%keir.fraser@eu.citrix.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Return-path: <xen-devel-bounces@lists.xensource.com>
In-Reply-To: <C84626FD.1827E%keir.fraser@eu.citrix.com>
List-Unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xensource.com>
List-Help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-Subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
Sender: xen-devel-bounces@lists.xensource.com
Errors-To: xen-devel-bounces@lists.xensource.com
To: Keir Fraser <keir.fraser@eu.citrix.com>, Dulloor <dulloor@gmail.com>, Jan Beulich <JBeulich@novell.com>
Cc: xen-devel@lists.xensource.com
List-Id: xen-devel@lists.xenproject.org

> From: Keir Fraser [mailto:keir.fraser@eu.citrix.com]
> Subject: Re: [Xen-devel] tmem and construct_dom0 memory allocation race
>=20
> On 22/06/2010 08:17, "Dulloor" <dulloor@gmail.com> wrote:
>=20
> > Hi Keir, You are right .. there is no race. I spent some time
> > debugging this. The problem is that a zero-order allocation (from
> > alloc_chunk, for the last dom0 page) fails with tmem on (in
> > alloc_heap_pages), even though there are pages available in the heap.
> > I don't think tmem really intends to get triggered so early. What do
> > you think ?
>=20
> That's one for Dan to comment on.

Hmmm... the special casing in alloc_heap_pages to avoid fragmentation
need not be invoked if tmem doesn't hold any pages (as is the
case at dom0 boot)...

Does this patch fix the problem?  If so...

Signed-off-by: Dan Magenheimer

diff -r a24dbfcbdf69 xen/common/page_alloc.c
--- a/xen/common/page_alloc.c=09Tue Jun 22 07:19:38 2010 +0100
+++ b/xen/common/page_alloc.c=09Tue Jun 22 11:17:44 2010 -0600
@@ -316,11 +316,14 @@ static struct page_info *alloc_heap_page
     spin_lock(&heap_lock);
=20
     /*
-     * TMEM: When available memory is scarce, allow only mid-size allocati=
ons
-     * to avoid worst of fragmentation issues. Others try TMEM pools then =
fail.
+     * TMEM: When available memory is scarce due to tmem absorbing it, all=
ow
+     * only mid-size allocations to avoid worst of fragmentation issues.
+     * Others try tmem pools then fail.  This is a workaround until all
+     * post-dom0-creation-multi-page allocations can be eliminated.
      */
     if ( opt_tmem && ((order =3D=3D 0) || (order >=3D 9)) &&
-         (total_avail_pages <=3D midsize_alloc_zone_pages) )
+         (total_avail_pages <=3D midsize_alloc_zone_pages) &&
+         tmem_freeable_pages() )
         goto try_tmem;
=20
     /*
diff -r a24dbfcbdf69 xen/common/tmem.c
--- a/xen/common/tmem.c=09Tue Jun 22 07:19:38 2010 +0100
+++ b/xen/common/tmem.c=09Tue Jun 22 11:17:44 2010 -0600
@@ -2850,6 +2850,11 @@ EXPORT void *tmem_relinquish_pages(unsig
     return pfp;
 }
=20
+EXPORT unsigned long tmem_freeable_pages(void)
+{
+    return tmh_freeable_pages();
+}
+
 /* called at hypervisor startup */
 static int __init init_tmem(void)
 {
diff -r a24dbfcbdf69 xen/include/xen/tmem.h
--- a/xen/include/xen/tmem.h=09Tue Jun 22 07:19:38 2010 +0100
+++ b/xen/include/xen/tmem.h=09Tue Jun 22 11:17:44 2010 -0600
@@ -11,6 +11,7 @@
=20
 extern void tmem_destroy(void *);
 extern void *tmem_relinquish_pages(unsigned int, unsigned int);
+extern unsigned long tmem_freeable_pages(void);
 extern int  opt_tmem;
=20
 #endif /* __XEN_TMEM_H__ */