[PATCH][RFC] sub-page anonymous domain memory allocator

[PATCH][RFC] sub-page anonymous domain memory allocator

[Daniel Magenheimer]

[-- Attachment #1: Type: text/plain, Size: 661 bytes --]

This patch uses some #define-and-Makefile trickery to completely
leverage common/xmalloc.c to provide a sub-page memory
allocator that takes memory from anonymous domain heap instead
of xen heap.  Note that it is useful only on a 64-bit hypervisor.
(Some other #define trickery can be used so that 32-bit
hypervisor uses xenheap and 64-bit hypervisor uses adheap.)

Keir says that a goal for 3.4 is to combine xen heap and
domain heap on 64-bit hypervisors.  This is a step in that
direction that I am using today in some work that requires
more memory than xenheap can provide (which I'll hopefully
post sometime in the next few weeks).

Comments?

[-- Attachment #2: admalloc.patch --]
[-- Type: application/octet-stream, Size: 2933 bytes --]

diff -r 566ee473fc27 xen/common/Makefile
--- a/xen/common/Makefile	Fri Aug 22 10:45:30 2008 +0100
+++ b/xen/common/Makefile	Fri Aug 29 11:00:31 2008 -0600
@@ -27,6 +27,7 @@ obj-y += vsprintf.o
 obj-y += vsprintf.o
 obj-y += xmalloc.o
 obj-y += rcupdate.o
+obj-$(x86_64) += admalloc.o
 
 obj-$(perfc)       += perfc.o
 obj-$(crash_debug) += gdbstub.o
@@ -42,6 +43,9 @@ subdir-$(ia64) += hvm
 
 subdir-y += libelf
 
+admalloc.o: xmalloc.c $(HDRS) Makefile
+	$(CC) -DANON_DOMAIN $(CFLAGS) -c $< -o $@
+
 # Object file contains changeset and compiler information.
 version.o: $(BASEDIR)/include/xen/compile.h
 
diff -r 566ee473fc27 xen/common/xmalloc.c
--- a/xen/common/xmalloc.c	Fri Aug 22 10:45:30 2008 +0100
+++ b/xen/common/xmalloc.c	Fri Aug 29 11:00:31 2008 -0600
@@ -35,6 +35,21 @@
 #include <xen/prefetch.h>
 #include <xen/irq.h>
 #include <xen/smp.h>
+
+/*
+ * combined with Makefile trickery, creates xmalloc-equivalent routines
+ * (e.g. admalloc/adfree) that use anonymous domain heap instead
+ * of xen heap... useful only on x86_64
+ */
+#ifdef ANON_DOMAIN
+#include <xen/admalloc.h>
+#define alloc_xenheap_pages(order) alloc_adheap_pages(order)
+#undef alloc_xenheap_page
+#define alloc_xenheap_page() alloc_adheap_pages(0)
+#define xfree adfree
+#define _xmalloc _admalloc
+#define free_xenheap_pages free_adheap_pages
+#endif
 
 /*
  * XMALLOC_DEBUG:
diff -r 566ee473fc27 xen/include/xen/admalloc.h
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/xen/include/xen/admalloc.h	Fri Aug 29 11:00:31 2008 -0600
@@ -0,0 +1,48 @@
+
+#ifndef __ADMALLOC_H__
+#define __ADMALLOC_H__
+
+/* Allocate space for typed object. */
+#define admalloc(_type) ((_type *)_admalloc(sizeof(_type), __alignof__(_type)))
+
+/* Allocate space for array of typed objects. */
+#define admalloc_array(_type, _num) ((_type *)_admalloc_array(sizeof(_type), __alignof__(_type), _num))
+
+/* Allocate untyped storage. */
+#define admalloc_bytes(_bytes) (_admalloc(_bytes, SMP_CACHE_BYTES))
+
+/* Free any of the above. */
+extern void adfree(void *);
+
+/* Underlying functions */
+extern void *_admalloc(size_t size, size_t align);
+static inline void *_admalloc_array(size_t size, size_t align, size_t num)
+{
+	/* Check for overflow. */
+	if (size && num > UINT_MAX / size)
+		return NULL;
+ 	return _admalloc(size * num, align);
+}
+
+static inline void free_adheap_pages(void *v, unsigned int order)
+{
+    struct page_info *pg = virt_to_page(v);
+
+    page_set_owner(pg, NULL);
+    free_domheap_pages(virt_to_page(v), order);
+}
+
+static inline struct xmalloc_hdr *alloc_adheap_pages(unsigned int order)
+{
+    struct page_info *pg = alloc_domheap_pages(NULL, order, 0);
+
+    if (pg == NULL)
+        return NULL;
+    /* FIXME is this right? */
+    return (struct xmalloc_hdr *)page_to_virt(pg);
+}
+
+#define free_adheap_page(pg) free_adheap_pages(pg,0)
+#define alloc_adheap_page() alloc_adheap_pages(0)
+
+#endif /* __ADMALLOC_H__ */

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH][RFC] sub-page anonymous domain memory allocator

[Keir Fraser]

You can more simply just have the existing xmalloc interface go at
alloc_domheap_pages(NULL) on x86/64. That would just need some #ifdef
abstraction within xmalloc.c.

 -- Keir

On 29/8/08 18:12, "Daniel Magenheimer" <dan.magenheimer@oracle.com> wrote:

> This patch uses some #define-and-Makefile trickery to completely
> leverage common/xmalloc.c to provide a sub-page memory
> allocator that takes memory from anonymous domain heap instead
> of xen heap.  Note that it is useful only on a 64-bit hypervisor.
> (Some other #define trickery can be used so that 32-bit
> hypervisor uses xenheap and 64-bit hypervisor uses adheap.)
> 
> Keir says that a goal for 3.4 is to combine xen heap and
> domain heap on 64-bit hypervisors.  This is a step in that
> direction that I am using today in some work that requires
> more memory than xenheap can provide (which I'll hopefully
> post sometime in the next few weeks).
> 
> Comments?
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: [PATCH][RFC] sub-page anonymous domain memory allocator

[Daniel Magenheimer]

True, but this way the new anonymous domain sub-page
allocator co-exists with the existing xmalloc mechanism,
which I didn't want to mess with right now.  In the future,
when they are both eating from the same pool of memory,
it won't matter.

Related, any suggestions on how to implement a "low-water mark"
(for any or all of the heaps/nodes)?  I'd like to be
know when memory is running low (and maybe "very low"),
rather than just have it run out.  Especially since
alloc_xenheap_pages gets "chatty".

Thanks,
Dan

> -----Original Message-----
> From: Keir Fraser [mailto:keir.fraser@eu.citrix.com]
> Sent: Friday, August 29, 2008 12:19 PM
> To: Daniel Magenheimer; xen-devel mailing list
> Subject: Re: [Xen-devel] [PATCH][RFC] sub-page anonymous domain memory
> allocator
> 
> 
> You can more simply just have the existing xmalloc interface go at
> alloc_domheap_pages(NULL) on x86/64. That would just need some #ifdef
> abstraction within xmalloc.c.
> 
>  -- Keir
> 
> On 29/8/08 18:12, "Daniel Magenheimer" 
> <dan.magenheimer@oracle.com> wrote:
> 
> > This patch uses some #define-and-Makefile trickery to completely
> > leverage common/xmalloc.c to provide a sub-page memory
> > allocator that takes memory from anonymous domain heap instead
> > of xen heap.  Note that it is useful only on a 64-bit hypervisor.
> > (Some other #define trickery can be used so that 32-bit
> > hypervisor uses xenheap and 64-bit hypervisor uses adheap.)
> > 
> > Keir says that a goal for 3.4 is to combine xen heap and
> > domain heap on 64-bit hypervisors.  This is a step in that
> > direction that I am using today in some work that requires
> > more memory than xenheap can provide (which I'll hopefully
> > post sometime in the next few weeks).
> > 
> > Comments?
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>