Re: Memory allocation in NUMA system

["Yang, Xiaowei" <xiaowei.yang@intel.com>]

Keir Fraser wrote:
> On 25/7/08 08:51, "Yang, Xiaowei" <xiaowei.yang@intel.com> wrote:
> 
>  >> How about by default we guarantee no more than 25% of a node's memory is
>  >> classed as 'DMA memory', and we reduce the DMA address width 
> variable in Xen
>  >> to ensure that?
>  >>
>  >> So, in your example, we would reduce dma_bitsize to 30.
>  >>
>  >>  -- Keir
>  >>
>  >>
>  > Yes, a good suggestion!
> 
> Indeed the only reason we still have dma_bitsize is to break the
> select-NUMA-node-first memory allocation search strategy. So tweaking the
> dma_bitsize approach further to strike the correct NUMA-vs-DMA balance does
> seem the right thing to do. Feel free to work up a patch.
> 
>  -- Keir
> 
> 
> 
How about this one?

diff -r 63317b6c3eab xen/common/page_alloc.c
--- a/xen/common/page_alloc.c	Mon Jul 14 15:21:03 2008 +0100
+++ b/xen/common/page_alloc.c	Fri Jul 25 18:24:16 2008 +0800
@@ -55,7 +55,7 @@
  /*
   * Bit width of the DMA heap.
   */
-static unsigned int dma_bitsize = CONFIG_DMA_BITSIZE;
+static unsigned int dma_bitsize;
  static void __init parse_dma_bits(char *s)
  {
      unsigned int v = simple_strtol(s, NULL, 0);
@@ -583,6 +583,16 @@
              init_heap_pages(pfn_dom_zone_type(i), mfn_to_page(i), 1);
      }

+    /* Reserve up to 25% of node0's memory for DMA */
+    if ( dma_bitsize == 0 )
+    {
+        dma_bitsize = 
pfn_dom_zone_type(NODE_DATA(0)->node_spanned_pages / 4)
+                      + PAGE_SHIFT;
+
+        ASSERT(dma_bitsize <= BITS_PER_LONG + PAGE_SHIFT);
+        ASSERT(dma_bitsize > PAGE_SHIFT + 1);
+    }
+
      printk("Domain heap initialised: DMA width %u bits\n", dma_bitsize);
  }
  #undef avail_for_domheap
diff -r 63317b6c3eab xen/include/asm-x86/config.h
--- a/xen/include/asm-x86/config.h	Mon Jul 14 15:21:03 2008 +0100
+++ b/xen/include/asm-x86/config.h	Fri Jul 25 18:24:16 2008 +0800
@@ -96,8 +96,6 @@

  /* Primary stack is restricted to 8kB by guard pages. */
  #define PRIMARY_STACK_SIZE 8192
-
-#define CONFIG_DMA_BITSIZE 32

  #define BOOT_TRAMPOLINE 0x8c000
  #define bootsym_phys(sym)                                 \


Thanks,
Xiaowei