From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [PATCH v2 16/16] libxl: automatic NUMA placement affects soft affinity Date: Thu, 14 Nov 2013 17:11:38 +0100 Message-ID: <1384445498.29902.155.camel@Abyss> References: <20131113190852.18086.5437.stgit@Solace> <20131113191334.18086.72275.stgit@Solace> <21124.59769.120811.671985@mariner.uk.xensource.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7986289073428363741==" Return-path: In-Reply-To: <21124.59769.120811.671985@mariner.uk.xensource.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Ian Jackson Cc: Marcus Granado , Keir Fraser , Ian Campbell , Li Yechen , George Dunlap , Andrew Cooper , Juergen Gross , xen-devel@lists.xen.org, Jan Beulich , Justin Weaver , Matt Wilson , Elena Ufimtseva List-Id: xen-devel@lists.xenproject.org --===============7986289073428363741== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-SwEuTjgOa8wcMSbbk6xp" --=-SwEuTjgOa8wcMSbbk6xp Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On gio, 2013-11-14 at 15:17 +0000, Ian Jackson wrote: > Dario Faggioli writes ("[PATCH v2 16/16] libxl: automatic NUMA placement = affects soft affinity"): > > diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c > > index a1c16b0..d241efc 100644 > > --- a/tools/libxl/libxl_dom.c > > +++ b/tools/libxl/libxl_dom.c > > @@ -222,21 +222,39 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domi= d, > > * some weird error manifests) the subsequent call to > > * libxl_domain_set_nodeaffinity() will do the actual placement, > > * whatever that turns out to be. > > + * > > + * As far as scheduling is concerned, we achieve NUMA-aware schedu= ling > > + * by having the results of placement affect the soft affinity of = all > > + * the vcpus of the domain. Of course, we want that iff placement = is > > + * enabled and actually happens, so we only change info->cpumap_so= ft to > > + * reflect the placement result if that is the case > > */ > > if (libxl_defbool_val(info->numa_placement)) { >=20 > But isn't the default for this true ? >=20 It is. > > - if (!libxl_bitmap_is_full(&info->cpumap)) { > > + /* We require both hard and soft affinity not to be set */ > > + if (!libxl_bitmap_is_full(&info->cpumap) || > > + !libxl_bitmap_is_full(&info->cpumap_soft)) { > > LOG(ERROR, "Can run NUMA placement only if no vcpu " > > - "affinity is specified"); > > + "(hard or soft) affinity is specified"); > > return ERROR_INVAL; >=20 > So the result will be that any attempt to set cpumaps in an > otherwise-naive config file will cause errors, rather than just > disabling the numa placement ? >=20 Nope, because, as it discussed already not more than a couple of days back, what xl does when finding a "cpus=3D" option (and from this patch on, a "cpus_soft=3D" option) is: 1. set numa_placement to false 2. set cpumap (or cpumap_soft) as requested. :-) So, as far as xl is concerned, this is fine. It is true that every other consumer of libxl needs to do the same (i.e., setting numa_placement to false), or it will hit the error, and that may or may not be obvious. For sure, they'll figure out if the check out xl (which is meant to serve as a 'reference implementation' too, right?), and, it is all documented, at least in docs/misc (I can double check whether it is also stressed enough in libxl.h somewhere, and put it there if not). For one, I probably should improve the comment above the if(), to avoid this sort of confusion to happen again. Anyway, like it or not, this is the kind of interface we came up with when designing, refining, and checking in that bit: http://lists.xen.org/archives/html/xen-devel/2012-07/msg01077.html If we don't like it, I think there is some room for amending, since, despite libxl API being stable, this looks enough as an implementation detail to me.... It's just a matter of agreeing on what we actually want. :-) Thanks and Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-SwEuTjgOa8wcMSbbk6xp Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (GNU/Linux) iEYEABECAAYFAlKE9joACgkQk4XaBE3IOsSZSgCfXZly9cD69HklalmWPawN+Fpw NjQAn2H4yRw5WaP9cGlWfqEfRoWPuHbC =1ueZ -----END PGP SIGNATURE----- --=-SwEuTjgOa8wcMSbbk6xp-- --===============7986289073428363741== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============7986289073428363741==--