From mboxrd@z Thu Jan 1 00:00:00 1970 From: George Dunlap Subject: Re: [PATCH RESEND 05/12] xen: numa-sched: make space for per-vcpu node-affinity Date: Wed, 6 Nov 2013 14:56:16 +0000 Message-ID: <527A5890.90709@eu.citrix.com> References: <20131105142844.30446.78671.stgit@Solace> <20131105143500.30446.9976.stgit@Solace> <5279143702000078000FFB15@nat28.tlf.novell.com> <527908B2.5090208@eu.citrix.com> <52790A93.4020903@eu.citrix.com> <52791B8702000078000FFBC4@nat28.tlf.novell.com> <5279114B.9080405@eu.citrix.com> <52792326.4050206@eu.citrix.com> <527927E3.3000004@eu.citrix.com> <1383730770.9207.93.camel@Solace> <527A1DFC0200007800100033@nat28.tlf.novell.com> <1383732000.9207.99.camel@Solace> <527A2BA8.9060601@eu.citrix.com> <1383747963.9207.134.camel@Solace> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1Ve4Wt-0005Pt-TI for xen-devel@lists.xenproject.org; Wed, 06 Nov 2013 14:56:24 +0000 In-Reply-To: <1383747963.9207.134.camel@Solace> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Dario Faggioli Cc: MarcusGranado , Justin Weaver , IanCampbell , Li Yechen , Andrew Cooper , JuergenGross , Ian Jackson , Jan Beulich , xen-devel , Daniel De Graaf , KeirFraser , Matt Wilson , Elena Ufimtseva List-Id: xen-devel@lists.xenproject.org On 06/11/13 14:26, Dario Faggioli wrote: > On mer, 2013-11-06 at 11:44 +0000, George Dunlap wrote: >> On 06/11/13 10:00, Dario Faggioli wrote: >>> I see, and that sounds sensible to me... It's mostly a matter a matter >>> of deciding whether o not we want something like that, and, if yes, >>> whether we want it based on hard of soft. >>> >>> Personally, I think I agree with you on having it based on hard >>> affinities by default. >>> >>> Let's see if George get to say something before I get to that part of >>> the (re)implementation. :-) >> I would probably have it based on soft affinities, since that's where we >> expect to have the domain's vcpus actually running most if the time; >> > True. However, doing that would rule out cpupool and vcpu-pinning. I guess I was assuming that a vcpu's soft affinity would always be considered a subset of its hard affinity and the cpus in its cpupool. >> I think for now I might advise putting off doing a NUMA interface at the >> libxl level, and do a full vNUMA interface in another series (perhaps >> for 4.5, depending on the timing). >> > Well, I agree that all this is of very little use without vNUMA, but at > the same time, it's not necessarily only useful for it. Also, whatever > it is vcpu-node-affinity or soft-affinity, if it is not wired up > properly up to the higher layers, there's very few point of having the > HV part only... So my idea was to redo and resend everything, including > libxl and xl bits. > > Of course, that doesn't mean we must necessarily have this for 4.4 > (although I think it would still be feasible), just that we either > check-in or wait for both the implementation and the interface. Again, > how's the updated release schedule? Well we definitely should plumb through access to the soft affinity directly through libxl. The question is whether it would be useful to have a libxl per-vcpu *numa* affinity that is not yet a full vNUMA implementation. If long-term we want to have vcpu -> vnode, and then vnode->pnode, the having a direct vcpu->pnode interface in the interim is probably not particularly useful, I don't think. Re the release schedule: we agreed to move the code freeze to November 18, so we've got just under two weeks. I think keeping the libxl side simple will allow us to at least get the "soft affinity" stuff in for 4.4, whether the vNUMA stuff makes it or not. -George