From mboxrd@z Thu Jan 1 00:00:00 1970 From: George Dunlap Subject: Re: NUMA TODO-list for xen-devel Date: Fri, 3 Aug 2012 12:00:46 +0100 Message-ID: <501BAF5E.6080805@eu.citrix.com> References: <1343837796.4958.32.camel@Solace> <501A67C502000078000921FF@nat28.tlf.novell.com> <1343914490.4873.18.camel@Solace> <501BB4C202000078000926DB@nat28.tlf.novell.com> <501B9E81.1020302@amd.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <501B9E81.1020302@amd.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Andre Przywara Cc: Anil Madhavapeddy , Andrew Cooper , xen-devel , Dario Faggioli , Jan Beulich , Yang Z Zhang List-Id: xen-devel@lists.xenproject.org On 03/08/12 10:48, Andre Przywara wrote: >>> I think we could use cpu hot-plug to change the "virtual topology" of >>> VMs, couldn't we? We could probably even do that on a running guest >>> if we really needed to. >> Hmm, not sure - using hotplug behind the back of the guest might >> be possible, but you'd first need to hot-unplug the vCPU. That's >> something that I don't think you can do on HVM guests (and for >> PV guests, guest visible NUMA support makes even less sense >> than for HVM ones). > I don't think that hotplug would really work. I have checked this some > times ago, at least the Linux NUMA code cannot be really fooled by this. > The SRAT table is firmware defined and static by nature, so there is no > code in Linux to change the NUMA topology at runtime. This is especially > true for the memory layout. I was more thinking of giving a VM the biggest topology you would want at boot, and then asking Linux to online or offline vcpus; for example, giving it a 4x2 topology (4 vcores x 2 vnodes). When running on a system with 2 cores per node, you offline 2 vcpus per vnode, giving it an effective layout of 2x2. When running on a system with 4 cores per node, you could offline all of the cores on one node, giving it an effective topology of 4x1. Unfortunately, I just realized that you could change the number of vcpus in a given node, but you couldn't move the memory around very easily. Unless you have memory hotplug? Hmm..... :-) -George