From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andre Przywara Subject: Re: [PATCH 1 of 3 v5/leftover] libxl: enable automatic placement of guests on NUMA nodes [and 1 more messages] Date: Thu, 19 Jul 2012 20:37:02 +0200 Message-ID: <500853CE.8090500@amd.com> References: <0411b2cebd725b193465.1341932614@Solace> <20485.35590.105351.434937@mariner.uk.xensource.com> <5fa66c8b9093399e5bc3.1342458792@Solace> <20485.43293.833036.352186@mariner.uk.xensource.com> <1342570947.11794.92.camel@Abyss> <1342602839.26734.6.camel@zakaz.uk.xensource.com> <1342604616.19530.123.camel@Solace> <1342605184.26734.19.camel@zakaz.uk.xensource.com> <20486.38718.474388.724787@mariner.uk.xensource.com> <5006BCC1.7050001@amd.com> <20488.7456.990576.233871@mariner.uk.xensource.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20488.7456.990576.233871@mariner.uk.xensource.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Ian Jackson Cc: Ian Campbell , Stefano Stabellini , George Dunlap , Juergen Gross , xen-devel , Dario Faggioli List-Id: xen-devel@lists.xenproject.org On 07/19/2012 04:43 PM, Ian Jackson wrote: > Andre Przywara writes ("Re: [PATCH 1 of 3 v5/leftover] libxl: enable automatic placement of guests on NUMA nodes [and 1 more messages]"): >> On 07/18/2012 01:00 PM, Ian Jackson wrote: >>> What is the maximum number of NUMA nodes we might expect to see on a >>> single system in the next five years? I would argue that 32 is too >>> optimistic. 128 or 256 seem like more reasonable upper bounds. >> >> Wow, what are you talking about? >> To calm this down from the AMD side: >> The current Opteron NUMA architecture is limited to exactly 8 nodes. >> This has ever been the case since the release of Opteron and changing >> this is not trivial and will not happen in any near future. In general I >> don't think we will see much bigger NUMA systems, but more cluster like >> architectures. > > One of these `more cluster like architectures', if it has shared > memory at all, may well end up being most easily represented in the > model supported by Xen as a NUMA host with a large number of nodes. > > We need to plan for the software that we are writing today to work for > at least the next 5 years /even if we intend to replace the algorithm > in the next release/. It is very difficult to foresee what might > happen to hardware in that time. > > But it's OK, we don't need to panic. We just need a safety catch > which stops this algorithm running in situations where it won't work. > Following a discussion with Dario, AIUI he plans to implement a rule > that it will bail on systems with more than 8 NUMA nodes. > > That is releasable in 4.2 and ought to be satisfactory for you ? Absolutely, that's fine. I just wanted to make sure that we don't drop the patches at all because of the theoretical fear of a large NUMA box lingering somewhere around the corner... Thanks, Andre. -- Andre Przywara AMD-Operating System Research Center (OSRC), Dresden, Germany