Re: Re: [PATCH 1/6][RESEND] xen: Add NUMA support to Xen

[Ryan Harper <ryanh@us.ibm.com>]

[-- Attachment #1: Type: text/plain, Size: 2518 bytes --]

* Keir Fraser <Keir.Fraser@cl.cam.ac.uk> [2006-05-16 08:03]:
> 
> On 16 May 2006, at 13:57, Andi Kleen wrote:
> 
> >>Yes, my gut feeling looking at x86_64's numa.c is that it's going to
> >>need some heavier surgery than srat.c. I wouldn't worry so much about
> >>keeping that one close to the Linux original: if we end up pulling 
> >>down
> >>more Linux memory bookkeeping code later then we can always go back 
> >>and
> >>sync the file more closely. Keep it as clean as possible though,
> >>obviously (e.g., replacing whole functions is nicer than functions 
> >>that
> >>are a hacky halfway house between Linux and Xen, etc).
> >
> >If it helps I can split these functions into smaller ones in the 
> >mainline
> >sources. That could isolate the pglists in only very small functions.
> 
> Thanks: I guess we'll see how the new patch turns out. It would 

I've got the ACPI numa.c parser function using
linux/arch/x86_64/mm/srat.c almost entirely unmodified.
linux/arch/x86_64/mm/numa.c was gutted retainly only the calls between
itself and srat.c.  I kept the memnodemap which has a nice phys_to_nid()
function.  There is still some cleanup to do, but I'd like some feedback
on what I used and didn't.

I didn't bring in mmzone.h but I did pull a few ideas and defines from
there.  We have a simple node_data structure with start_pfn,
spanned_pages and node_id.

I attempted to follow x86_64 numa.c which stashes the structure on
node-local memory.  This was problematic for 32-bit NUMA support.  On
the opteron  2-way that I had, the starting physical address for the
second node is 80000000, and when turned into a virtual address
(PAGE_OFFSET is FF000000), the resulting va is 17F000000, which
overflows unsigned long in 32-bit.  I started to use u64's, but several
functions (like map_pages_to_xen()) only take unsigned longs.  Rather
than go through getting that function working, I think it is perfectly
fine to just have a static array.  The structure we store out there
isn't accessed in any fast path.  If the structure becomes more
complicated in the future, or someone not having the structure on
node-local memory becomes a performance issue we can revisit.

The patch is function on 32-bit and 64-bit boxes and parse the SRAT
table and fills out the node_data array.  I installed a simple
keyhandler 'u' to dump the info to check that it was function after
booting up.


-- 
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
ryanh@us.ibm.com

[-- Attachment #2: acpi_numa.tgz --]
[-- Type: application/x-gtar, Size: 9693 bytes --]

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel