NUMA support on Xen ?

NUMA support on Xen ?

[Deepti Chheda]

Hi,

I am a Masters student from Carnegie Mellon University. I am looking for a research topics for an Advanced OS & DS course we have.

I wanted to know what is the current support for NUMA on Xen ? 
Does it support the IBM x440 and AMD64 Opteron ? 
Also, does the Xen scheduler do NUMA aware scheduling so it does not degrade the VM performace ?

My group is currently looking into Scheduling policies for NUMA systems for multiple VMs.

Let me know if its already done in Xen.

Thanks,
Deepti Chheda

RE: NUMA support on Xen ?

[Ian Pratt]

> I wanted to know what is the current support for NUMA on Xen ? 
> Does it support the IBM x440 and AMD64 Opteron ? 
> Also, does the Xen scheduler do NUMA aware scheduling so it 
> does not degrade the VM performace ?

There are NUMA memory allocator patches, but the scheduler is currently
unaware of NUMA topology (though you can set things up manually using
vcpu-pin).

I suspect that for most users of small NUMA systems just spraying the VM
data across nodes is the best way forward unless the user sets things up
explicitly. 

The key scheduler extension we need at the moment is something that
deals with the case where there are fewer runnable VCPUs than physical
CPUS, and tries to ensure that they are spread appart across sockets and
cores. i.e. don't run them on two hyperthreads of the same core unless
you have to.

Ian 
 
> My group is currently looking into Scheduling policies for 
> NUMA systems for multiple VMs.
> 
> Let me know if its already done in Xen.
> 
> Thanks,
> Deepti Chheda
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 

Re: NUMA support on Xen ?

[Ryan Harper]

* Deepti Chheda <dchheda@andrew.cmu.edu> [2006-10-04 03:19]:
> Hi,
> 
> I am a Masters student from Carnegie Mellon University. I am looking
> for a research topics for an Advanced OS & DS course we have.
> 
> I wanted to know what is the current support for NUMA on Xen ? 

Xen doesn't have any NUMA support currently.  I've posted [1]patches to add
NUMA support into Xen.  This support covers systems that provide ACPI
SRAT tables describing the topology of the processors and memory of the
system.  It works for 32-bit and 64-bit system.  The patches give Xen an
NUMA-aware page allocator, update memory hypercalls to allocate
node-local memory when creating VMs, and exports NUMA topology to
user-space via the management tools.

> Does it support the IBM x440 and AMD64 Opteron ? 

I've not tested on an x440, but the patches have been tested on an x445,
x460, AMD opteron, Unisys ES7000, and IA64.  I believe that the patches
would work fine on an x440.

> Also, does the Xen scheduler do NUMA aware scheduling so it does not
> degrade the VM performace ?

The current patchset does not address scheduling.


1. http://lists.xensource.com/archives/html/xen-devel/2006-09/msg00958.html

-- 
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
ryanh@us.ibm.com