* [PATCH 1/5] xl: update the manpage about "cpus=" and NUMA node-affinity
2013-09-16 18:40 [PATCH 0/5] xl: allow for node-wise specification of vcpu pinning Dario Faggioli
@ 2013-09-16 18:40 ` Dario Faggioli
2013-09-16 18:40 ` [PATCH 2/5] libxl: introduce libxl_node_to_cpumap Dario Faggioli
` (4 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Dario Faggioli @ 2013-09-16 18:40 UTC (permalink / raw)
To: xen-devel; +Cc: George Dunlap, Ian Jackson, Ian Campbell
Since d06b1bf169a01a9c7b0947d7825e58cb455a0ba5 ('libxl: automatic placement
deals with node-affinity') it is no longer true that, if no "cpus=" option
is specified, xl picks up some pCPUs by default and pin the domain there.
In fact, it is the NUMA node-affinity that is affected by automatic
placement, not vCPU to pCPU pinning.
Update the xl config file documenation accordingly, as it seems to have
been forgotten at that time.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
docs/man/xl.cfg.pod.5 | 22 +++++++++++++++-------
1 file changed, 15 insertions(+), 7 deletions(-)
diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index 769767b..fd3979a 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -104,8 +104,8 @@ created online and the remainder will be offline.
=item B<cpus="CPU-LIST">
-List of which cpus the guest is allowed to use. By default xl will pick
-some cpus on its own (see below). A C<CPU-LIST> may be specified as follows:
+List of which cpus the guest is allowed to use. Default is no pinning at
+all (more on this below). A C<CPU-LIST> may be specified as follows:
=over 4
@@ -125,11 +125,19 @@ run on cpu #3 of the host.
=back
-If this option is not specified, libxl automatically tries to place the new
-domain on the host's NUMA nodes (provided the host has more than one NUMA
-node) by pinning it to the cpus of those nodes. A heuristic approach is
-utilized with the goals of maximizing performance for the domain and, at
-the same time, achieving efficient utilization of the host's CPUs and RAM.
+If this option is not specified, no vcpu to cpu pinning is established,
+and the vcpus of the guest can run on all the cpus of the host.
+
+If we are on a NUMA machine (i.e., if the host has more than one NUMA
+node) and this option is not specified, libxl automatically tries to
+place the guest on the least possible number of nodes. That, however,
+will not affect vcpu pinning, so the guest will still be able to run on
+all the cpus, it will just prefer the ones from the node it has been
+placed on. A heuristic approach is used for choosing the best node (or
+set of nodes), with the goals of maximizing performance for the guest
+and, at the same time, achieving efficient utilization of host cpus
+and memory. See F<docs/misc/xl-numa-placement.markdown> for more
+details.
=back
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH 2/5] libxl: introduce libxl_node_to_cpumap
2013-09-16 18:40 [PATCH 0/5] xl: allow for node-wise specification of vcpu pinning Dario Faggioli
2013-09-16 18:40 ` [PATCH 1/5] xl: update the manpage about "cpus=" and NUMA node-affinity Dario Faggioli
@ 2013-09-16 18:40 ` Dario Faggioli
2013-09-16 18:41 ` [PATCH 3/5] xl: allow for node-wise specification of vcpu pinning Dario Faggioli
` (3 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Dario Faggioli @ 2013-09-16 18:40 UTC (permalink / raw)
To: xen-devel; +Cc: George Dunlap, Ian Jackson, Ian Campbell
As an helper for the special case (of libxl_nodemap_to_cpumap) when
one wants the cpumap for just one node.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
tools/libxl/libxl_utils.c | 22 ++++++++++++++++++++++
tools/libxl/libxl_utils.h | 3 +++
2 files changed, 25 insertions(+)
diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
index 4309e5e..5c125e6 100644
--- a/tools/libxl/libxl_utils.c
+++ b/tools/libxl/libxl_utils.c
@@ -668,6 +668,28 @@ int libxl_nodemap_to_cpumap(libxl_ctx *ctx,
return rc;
}
+int libxl_node_to_cpumap(libxl_ctx *ctx, int node,
+ libxl_bitmap *cpumap)
+{
+ libxl_bitmap nodemap;
+ int rc = 0;
+
+ libxl_bitmap_init(&nodemap);
+
+ rc = libxl_node_bitmap_alloc(ctx, &nodemap, 0);
+ if (rc)
+ goto out;
+
+ libxl_bitmap_set_none(&nodemap);
+ libxl_bitmap_set(&nodemap, node);
+
+ rc = libxl_nodemap_to_cpumap(ctx, &nodemap, cpumap);
+
+ out:
+ libxl_bitmap_dispose(&nodemap);
+ return rc;
+}
+
int libxl_cpumap_to_nodemap(libxl_ctx *ctx,
const libxl_bitmap *cpumap,
libxl_bitmap *nodemap)
diff --git a/tools/libxl/libxl_utils.h b/tools/libxl/libxl_utils.h
index a430362..7b84e6a 100644
--- a/tools/libxl/libxl_utils.h
+++ b/tools/libxl/libxl_utils.h
@@ -129,6 +129,9 @@ static inline int libxl_node_bitmap_alloc(libxl_ctx *ctx,
int libxl_nodemap_to_cpumap(libxl_ctx *ctx,
const libxl_bitmap *nodemap,
libxl_bitmap *cpumap);
+/* Populate cpumap with the cpus spanned by node */
+int libxl_node_to_cpumap(libxl_ctx *ctx, int node,
+ libxl_bitmap *cpumap);
/* Populate nodemap with the nodes of the cpus in cpumap */
int libxl_cpumap_to_nodemap(libxl_ctx *ctx,
const libxl_bitmap *cpuemap,
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH 3/5] xl: allow for node-wise specification of vcpu pinning
2013-09-16 18:40 [PATCH 0/5] xl: allow for node-wise specification of vcpu pinning Dario Faggioli
2013-09-16 18:40 ` [PATCH 1/5] xl: update the manpage about "cpus=" and NUMA node-affinity Dario Faggioli
2013-09-16 18:40 ` [PATCH 2/5] libxl: introduce libxl_node_to_cpumap Dario Faggioli
@ 2013-09-16 18:41 ` Dario Faggioli
2013-09-16 18:41 ` [PATCH 4/5] xl: implement and enable dryrun mode for `xl vcpu-pin' Dario Faggioli
` (2 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Dario Faggioli @ 2013-09-16 18:41 UTC (permalink / raw)
To: xen-devel; +Cc: George Dunlap, Ian Jackson, Ian Campbell
Making it possible to use something like the following:
* "nodes:0-3": all pCPUs of nodes 0,1,2,3;
* "nodes:0-3,^node:2": all pCPUS of nodes 0,1,3;
* "1,nodes:1-2,^6": pCPU 1 plus all pCPUs of nodes 1,2
but not pCPU 6;
* ...
In both domain config file and `xl vcpu-pin'.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
---
Changes from v2:
* turned a 'return' into 'goto out', consistently with the
most of exit patterns;
* harmonized error handling: now parse_range() return a
libxl error code, as requested during review;
* dealing with "all" moved inside update_cpumap_range().
It's tricky to move it in parse_range() (as requested
during review), since we need the cpumap being modified
handy when dealing with it. However, having it in
update_cpumap_range() simplifies the code just as much
as that;
* explicitly checking for junk after a valid value or range
in parse_range(), as requested during review;
* xl exits on parsing failing, so no need to reset the
cpumap to something sensible in vcpupin_parse(), as
suggested during review;
Changes from v1:
* code rearranged in order to look more simple to follow
and understand, as requested during review;
* improved docs in xl.cfg.pod.5, as requested during
review;
* strtoul() now returns into unsigned long, and the
case where it returns ULONG_MAX is now taken into
account, as requested during review;
* stuff like "all,^7" now works, as requested during
review. Specifying just "^7" does not work either
before or after this change
* killed some magic (i.e., `ptr += 5 + (ptr[4] == 's'`)
by introducing STR_SKIP_PREFIX() macro, as requested
during review.
---
docs/man/xl.cfg.pod.5 | 20 ++++++
tools/libxl/xl_cmdimpl.c | 145 ++++++++++++++++++++++++++++++++--------------
2 files changed, 121 insertions(+), 44 deletions(-)
diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index fd3979a..cbe57d6 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -115,7 +115,25 @@ To allow all the vcpus of the guest to run on all the cpus on the host.
=item "0-3,5,^1"
-To allow all the vcpus of the guest to run on cpus 0,2,3,5.
+To allow all the vcpus of the guest to run on cpus 0,2,3,5. Combining
+this with "all" is possible, meaning "all,^7" results in all the vcpus
+of the guest running on all the cpus on the host except cpu 7.
+
+=item "nodes:0-3,node:^2"
+
+To allow all the vcpus of the guest to run on the cpus from NUMA nodes
+0,1,3 of the host. So, if cpus 0-3 belongs to node 0, cpus 4-7 belongs
+to node 1 and cpus 8-11 to node 3, the above would mean all the vcpus
+of the guest will run on cpus 0-3,8-11.
+
+Combining this notation with the one above is possible. For instance,
+"1,node:2,^6", means all the vcpus of the guest will run on cpu 1 and
+on all the cpus of NUMA node 2, but not on cpu 6. Following the same
+example as above, that would be cpus 1,4,5,7.
+
+Combining this with "all" is also possible, meaning "all,^nodes:1"
+results in all the vcpus of the guest running on all the cpus on the
+host, except for the cpus belonging to the host NUMA node 1.
=item ["2", "3"] (or [2, 3])
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 3d7eaad..e2d4a85 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -59,6 +59,11 @@
} \
})
+#define STR_HAS_PREFIX( a, b ) \
+ ( strncmp(a, b, strlen(b)) == 0 )
+#define STR_SKIP_PREFIX( a, b ) \
+ ( STR_HAS_PREFIX(a, b) ? (a) += strlen(b) : NULL )
+
int logfile = 2;
@@ -481,61 +486,115 @@ static void split_string_into_string_list(const char *str,
free(s);
}
-static int vcpupin_parse(char *cpu, libxl_bitmap *cpumap)
+static int parse_range(const char *str, unsigned long *a, unsigned long *b)
+{
+ char *nstr, *endptr;
+
+ *a = *b = strtoul(str, &endptr, 10);
+ if (endptr == str || *a == ULONG_MAX)
+ return ERROR_INVAL;
+
+ if (*endptr == '-') {
+ nstr = endptr + 1;
+
+ *b = strtoul(nstr, &endptr, 10);
+ if (endptr == nstr || *b == ULONG_MAX || *b < *a)
+ return ERROR_INVAL;
+ }
+
+ /* Valid value or range so far, but we also don't want junk after that */
+ if (*endptr != '\0')
+ return ERROR_INVAL;
+
+ return 0;
+}
+
+/*
+ * Add or removes a specific set of cpus (specified in str, either as
+ * single cpus or as entire NUMA nodes) to/from cpumap.
+ */
+static int update_cpumap_range(const char *str, libxl_bitmap *cpumap)
{
- libxl_bitmap exclude_cpumap;
- uint32_t cpuida, cpuidb;
- char *endptr, *toka, *tokb, *saveptr = NULL;
- int i, rc = 0, rmcpu;
+ unsigned long ida, idb;
+ libxl_bitmap node_cpumap;
+ bool is_not = false, is_nodes = false;
+ int rc = 0;
+
+ libxl_bitmap_init(&node_cpumap);
- if (!strcmp(cpu, "all")) {
+ rc = libxl_node_bitmap_alloc(ctx, &node_cpumap, 0);
+ if (rc) {
+ fprintf(stderr, "libxl_node_bitmap_alloc failed.\n");
+ goto out;
+ }
+
+ /* Are we adding or removing cpus/nodes? */
+ if (STR_SKIP_PREFIX(str, "^")) {
+ is_not = true;
+ }
+
+ /* Are we dealing with cpus or full nodes? */
+ if (STR_SKIP_PREFIX(str, "nodes:")) {
+ is_nodes = true;
+ }
+
+ if (STR_HAS_PREFIX(str, "all")) {
libxl_bitmap_set_any(cpumap);
- return 0;
+ goto out;
}
- if (libxl_cpu_bitmap_alloc(ctx, &exclude_cpumap, 0)) {
- fprintf(stderr, "Error: Failed to allocate cpumap.\n");
- return ENOMEM;
+ rc = parse_range(str, &ida, &idb);
+ if (rc) {
+ fprintf(stderr, "Invalid pcpu range: %s.\n", str);
+ goto out;
}
- for (toka = strtok_r(cpu, ",", &saveptr); toka;
- toka = strtok_r(NULL, ",", &saveptr)) {
- rmcpu = 0;
- if (*toka == '^') {
- /* This (These) Cpu(s) will be removed from the map */
- toka++;
- rmcpu = 1;
- }
- /* Extract a valid (range of) cpu(s) */
- cpuida = cpuidb = strtoul(toka, &endptr, 10);
- if (endptr == toka) {
- fprintf(stderr, "Error: Invalid argument.\n");
- rc = EINVAL;
- goto vcpp_out;
- }
- if (*endptr == '-') {
- tokb = endptr + 1;
- cpuidb = strtoul(tokb, &endptr, 10);
- if (endptr == tokb || cpuida > cpuidb) {
- fprintf(stderr, "Error: Invalid argument.\n");
- rc = EINVAL;
- goto vcpp_out;
+ /* Add or remove the specified cpus in the range */
+ while (ida <= idb) {
+ if (is_nodes) {
+ /* Add/Remove all the cpus of a NUMA node */
+ int i;
+
+ rc = libxl_node_to_cpumap(ctx, ida, &node_cpumap);
+ if (rc) {
+ fprintf(stderr, "libxl_node_to_cpumap failed.\n");
+ goto out;
}
+
+ /* Add/Remove all the cpus in the node cpumap */
+ libxl_for_each_set_bit(i, node_cpumap) {
+ is_not ? libxl_bitmap_reset(cpumap, i) :
+ libxl_bitmap_set(cpumap, i);
+ }
+ } else {
+ /* Add/Remove this cpu */
+ is_not ? libxl_bitmap_reset(cpumap, ida) :
+ libxl_bitmap_set(cpumap, ida);
}
- while (cpuida <= cpuidb) {
- rmcpu == 0 ? libxl_bitmap_set(cpumap, cpuida) :
- libxl_bitmap_set(&exclude_cpumap, cpuida);
- cpuida++;
- }
+ ida++;
}
- /* Clear all the cpus from the removal list */
- libxl_for_each_set_bit(i, exclude_cpumap) {
- libxl_bitmap_reset(cpumap, i);
- }
+ out:
+ libxl_bitmap_dispose(&node_cpumap);
+ return rc;
+}
+
+/*
+ * Takes a string representing a set of cpus (specified either as
+ * single cpus or as eintire NUMA nodes) and turns it into the
+ * corresponding libxl_bitmap (in cpumap).
+ */
+static int vcpupin_parse(char *cpu, libxl_bitmap *cpumap)
+{
+ char *ptr, *saveptr = NULL;
+ int rc = 0;
-vcpp_out:
- libxl_bitmap_dispose(&exclude_cpumap);
+ for (ptr = strtok_r(cpu, ",", &saveptr); ptr;
+ ptr = strtok_r(NULL, ",", &saveptr)) {
+ rc = update_cpumap_range(ptr, cpumap);
+ if (rc)
+ break;
+ }
return rc;
}
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH 4/5] xl: implement and enable dryrun mode for `xl vcpu-pin'
2013-09-16 18:40 [PATCH 0/5] xl: allow for node-wise specification of vcpu pinning Dario Faggioli
` (2 preceding siblings ...)
2013-09-16 18:41 ` [PATCH 3/5] xl: allow for node-wise specification of vcpu pinning Dario Faggioli
@ 2013-09-16 18:41 ` Dario Faggioli
2013-09-16 18:41 ` [PATCH 5/5] xl: test script for the cpumap parser (for vCPU pinning) Dario Faggioli
2013-09-16 18:47 ` [PATCH 0/5] xl: allow for node-wise specification of vcpu pinning Dario Faggioli
5 siblings, 0 replies; 10+ messages in thread
From: Dario Faggioli @ 2013-09-16 18:41 UTC (permalink / raw)
To: xen-devel; +Cc: George Dunlap, Ian Jackson, Ian Campbell
As it can be useful to see if the outcome of some complex vCPU
pinning bitmap specification looks as expected.
This also allow for the introduction of some automatic testing
and verification for the bitmap parsing code, as it happens
already in check-xl-disk-parse and check-xl-vif-parse.
In particular, to make the above possible, this commit also
changes the implementation of the vcpu-pin command so that,
instead of always returning 0, it returns an error if the
parsing fails.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v2:
* fixed a typo in the changelog
---
tools/libxl/xl_cmdimpl.c | 46 ++++++++++++++++++++++++++++++++++-----------
tools/libxl/xl_cmdtable.c | 2 +-
2 files changed, 36 insertions(+), 12 deletions(-)
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index e2d4a85..823c6d8 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -4533,30 +4533,54 @@ int main_vcpulist(int argc, char **argv)
return 0;
}
-static void vcpupin(uint32_t domid, const char *vcpu, char *cpu)
+static int vcpupin(uint32_t domid, const char *vcpu, char *cpu)
{
libxl_vcpuinfo *vcpuinfo;
libxl_bitmap cpumap;
uint32_t vcpuid;
char *endptr;
- int i, nb_vcpu;
+ int i, nb_vcpu, rc = -1;
+
+ libxl_bitmap_init(&cpumap);
vcpuid = strtoul(vcpu, &endptr, 10);
if (vcpu == endptr) {
if (strcmp(vcpu, "all")) {
fprintf(stderr, "Error: Invalid argument.\n");
- return;
+ goto out;
}
vcpuid = -1;
}
if (libxl_cpu_bitmap_alloc(ctx, &cpumap, 0)) {
- goto vcpupin_out;
+ goto out;
}
if (vcpupin_parse(cpu, &cpumap))
- goto vcpupin_out1;
+ goto out;
+
+ if (dryrun_only) {
+ libxl_cputopology *info = libxl_get_cpu_topology(ctx, &i);
+
+ if (!info) {
+ fprintf(stderr, "libxl_list_vcpu failed.\n");
+ goto out;
+ }
+ libxl_cputopology_list_free(info, i);
+
+ fprintf(stdout, "cpumap: ");
+ print_cpumap(cpumap.map, i, stdout);
+ fprintf(stdout, "\n");
+
+ if (ferror(stdout) || fflush(stdout)) {
+ perror("stdout");
+ exit(-1);
+ }
+
+ rc = 0;
+ goto out;
+ }
if (vcpuid != -1) {
if (libxl_set_vcpuaffinity(ctx, domid, vcpuid, &cpumap) == -1) {
@@ -4566,7 +4590,7 @@ static void vcpupin(uint32_t domid, const char *vcpu, char *cpu)
else {
if (!(vcpuinfo = libxl_list_vcpu(ctx, domid, &nb_vcpu, &i))) {
fprintf(stderr, "libxl_list_vcpu failed.\n");
- goto vcpupin_out1;
+ goto out;
}
for (i = 0; i < nb_vcpu; i++) {
if (libxl_set_vcpuaffinity(ctx, domid, vcpuinfo[i].vcpuid,
@@ -4577,10 +4601,11 @@ static void vcpupin(uint32_t domid, const char *vcpu, char *cpu)
}
libxl_vcpuinfo_list_free(vcpuinfo, nb_vcpu);
}
- vcpupin_out1:
+
+ rc = 0;
+ out:
libxl_bitmap_dispose(&cpumap);
- vcpupin_out:
- ;
+ return rc;
}
int main_vcpupin(int argc, char **argv)
@@ -4591,8 +4616,7 @@ int main_vcpupin(int argc, char **argv)
/* No options */
}
- vcpupin(find_domain(argv[optind]), argv[optind+1] , argv[optind+2]);
- return 0;
+ return vcpupin(find_domain(argv[optind]), argv[optind+1] , argv[optind+2]);
}
static void vcpuset(uint32_t domid, const char* nr_vcpus, int check_host)
diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
index 326a660..d3dcbf0 100644
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -211,7 +211,7 @@ struct cmd_spec cmd_table[] = {
"[Domain, ...]",
},
{ "vcpu-pin",
- &main_vcpupin, 0, 1,
+ &main_vcpupin, 1, 1,
"Set which CPUs a VCPU can use",
"<Domain> <VCPU|all> <CPUs|all>",
},
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH 5/5] xl: test script for the cpumap parser (for vCPU pinning)
2013-09-16 18:40 [PATCH 0/5] xl: allow for node-wise specification of vcpu pinning Dario Faggioli
` (3 preceding siblings ...)
2013-09-16 18:41 ` [PATCH 4/5] xl: implement and enable dryrun mode for `xl vcpu-pin' Dario Faggioli
@ 2013-09-16 18:41 ` Dario Faggioli
2013-09-16 18:47 ` [PATCH 0/5] xl: allow for node-wise specification of vcpu pinning Dario Faggioli
5 siblings, 0 replies; 10+ messages in thread
From: Dario Faggioli @ 2013-09-16 18:41 UTC (permalink / raw)
To: xen-devel; +Cc: George Dunlap, Ian Jackson, Ian Campbell
This commit introduces "check-xl-vcpupin-parse" for helping
verifying and debugging the (v)CPU bitmap parsing code in xl.
The script runs "xl -N vcpu-pin 0 all <some strings>"
repeatedly, with various input strings, and checks that the
output is as expected.
This is what the script can do:
# ./check-xl-vcpupin-parse -h
usage: ./check-xl-vcpupin-parse [options]
Tests various vcpu-pinning strings. If run without arguments acts
as follows:
- generates some test data and saves them in
check-xl-vcpupin-parse.data;
- tests all the generated configurations (reading them back from
check-xl-vcpupin-parse.data).
An example of a test vector file is provided in
check-xl-vcpupin-parse.data-example.
Options:
-h prints this message
-r seed uses seed for initializing the rundom number generator
(default: the script PID)
-s string tries using string as a vcpu pinning configuration and
reports whether that succeeds or not
-o ofile save the test data in ofile
(default: check-xl-vcpupin-parse.data)
-i ifile read test data from ifile
An example test data file (generated on a 2 NUMA nodes, 16 CPUs
host) is being provided in check-xl-vcpupin-parse.data-example.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
---
Changes from v2:
* killed the `sleep 1', as requested during review;
* allow for passing a custom randon seed, and report what is
the actual random seed used, as requested during review;
* allow for testing for specific pinning configuration strings,
as suggested during review;
* stores the test data in a file, after generating them, and
read them back from there for actual testing, as suggested
during review;
* allow for reading the test data from an existing test file
instead than always generating new ones.
Changes from v1:
* this was not there in v1, and adding it has been requested
during review.
---
tools/libxl/check-xl-vcpupin-parse | 294 +++++++++++++++++++++++
tools/libxl/check-xl-vcpupin-parse.data-example | 53 ++++
2 files changed, 347 insertions(+)
create mode 100755 tools/libxl/check-xl-vcpupin-parse
create mode 100644 tools/libxl/check-xl-vcpupin-parse.data-example
diff --git a/tools/libxl/check-xl-vcpupin-parse b/tools/libxl/check-xl-vcpupin-parse
new file mode 100755
index 0000000..fb4689c
--- /dev/null
+++ b/tools/libxl/check-xl-vcpupin-parse
@@ -0,0 +1,294 @@
+#!/bin/bash
+
+set -e
+
+if [ -x ./xl ] ; then
+ export LD_LIBRARY_PATH=.:../libxc:../xenstore:
+ XL=./xl
+else
+ XL=xl
+fi
+
+fprefix=tmp.check-xl-vcpupin-parse
+outfile=check-xl-vcpupin-parse.data
+
+usage () {
+cat <<END
+usage: $0 [options]
+
+Tests various vcpu-pinning strings. If run without arguments acts
+as follows:
+ - generates some test data and saves them in $outfile;
+ - tests all the generated configurations (reading them back from
+ $outfile).
+
+An example of a test vector file is provided in ${outfile}-example.
+
+Options:
+ -h prints this message
+ -r seed uses seed for initializing the rundom number generator
+ (default: the script PID)
+ -s string tries using string as a vcpu pinning configuration and
+ reports whether that succeeds or not
+ -o ofile save the test data in ofile (default: $outfile)
+ -i ifile read test data from ifile
+END
+}
+
+expected () {
+ cat >$fprefix.expected
+}
+
+# by default, re-seed with our PID
+seed=$$
+failures=0
+
+# Execute one test and check the result against the provided
+# rc value and output
+one () {
+ expected_rc=$1; shift
+ printf "test case %s...\n" "$*"
+ set +e
+ ${XL} -N vcpu-pin 0 all "$@" </dev/null >$fprefix.actual 2>/dev/null
+ actual_rc=$?
+ if [ $actual_rc != $expected_rc ]; then
+ diff -u $fprefix.expected $fprefix.actual
+ echo >&2 "test case \`$*' failed ($actual_rc $diff_rc)"
+ failures=$(( $failures + 1 ))
+ fi
+ set -e
+}
+
+# Write an entry in the test vector file. Format is as follows:
+# test-string*expected-rc*expected-output
+write () {
+ printf "$1*$2*$3\n" >> $outfile
+}
+
+complete () {
+ if [ "$failures" = 0 ]; then
+ echo all ok.; exit 0
+ else
+ echo "$failures tests failed."; exit 1
+ fi
+}
+
+# Test a specific pinning string
+string () {
+ expected_rc=$1; shift
+ printf "test case %s...\n" "$*"
+ set +e
+ ${XL} -N vcpu-pin 0 all "$@" &> /dev/null
+ actual_rc=$?
+ set -e
+
+ if [ $actual_rc != $expected_rc ]; then
+ echo >&2 "test case \`$*' failed ($actual_rc)"
+ else
+ echo >&2 "test case \`$*' succeeded"
+ fi
+
+ exit 0
+}
+
+# Read a test vector file (provided as $1) line by line and
+# test all the entries it contains
+run ()
+{
+ while read line
+ do
+ if [ ${line:0:1} != '#' ]; then
+ test_string="`echo $line | cut -f1 -d'*'`"
+ exp_rc="`echo $line | cut -f2 -d'*'`"
+ exp_output="`echo $line | cut -f3 -d'*'`"
+
+ expected <<END
+$exp_output
+END
+ one $exp_rc "$test_string"
+ fi
+ done < $1
+
+ complete
+
+ exit 0
+}
+
+while getopts "hr:s:o:i:" option
+do
+ case $option in
+ h)
+ usage
+ exit 0
+ ;;
+ r)
+ seed=$OPTARG
+ ;;
+ s)
+ string 0 "$OPTARG"
+ ;;
+ o)
+ outfile=$OPTARG
+ ;;
+ i)
+ run $OPTARG
+ ;;
+ esac
+done
+
+#---------- test data ----------
+#
+nr_cpus=`xl info | grep nr_cpus | cut -f2 -d':'`
+nr_nodes=`xl info | grep nr_nodes | cut -f2 -d':'`
+nr_cpus_per_node=`xl info -n | sed '/cpu:/,/numa_info/!d' | head -n -1 | \
+ awk '{print $4}' | uniq -c | tail -1 | awk '{print $1}'`
+cat >$outfile <<END
+# WARNING: some of these tests are topology based tests.
+# Expect failures if the topology is not detected correctly
+# detected topology: $nr_cpus CPUs, $nr_nodes nodes, $nr_cpus_per_node CPUs per node.
+#
+# seed used for random number generation: seed=${seed}.
+#
+# Format is as follows:
+# test-string*expected-return-code*expected-output
+#
+END
+
+# Re-seed the random number generator
+RANDOM=$seed
+
+echo "# Testing a wring configuration" >> $outfile
+write foo 255 ""
+
+echo "# Testing the 'all' syntax" >> $outfile
+write "all" 0 "cpumap: any cpu"
+write "nodes:all" 0 "cpumap: any cpu"
+write "all,nodes:all" 0 "cpumap: any cpu"
+write "all,^nodes:0,all" 0 "cpumap: any cpu"
+
+echo "# Testing the empty cpumap case" >> $outfile
+write "^0" 0 "cpumap: none"
+
+echo "# A few attempts of pinning to just one random cpu" >> $outfile
+if [ $nr_cpus -gt 1 ]; then
+ for i in `seq 0 3`; do
+ cpu=$(($RANDOM % nr_cpus))
+ write "$cpu" 0 "cpumap: $cpu"
+ done
+fi
+
+echo "# A few attempts of pinning to all but one random cpu" >> $outfile
+if [ $nr_cpus -gt 2 ]; then
+ for i in `seq 0 3`; do
+ cpu=$(($RANDOM % nr_cpus))
+ if [ $cpu -eq 0 ]; then
+ expected_range="1-$((nr_cpus - 1))"
+ elif [ $cpu -eq 1 ]; then
+ expected_range="0,2-$((nr_cpus - 1))"
+ elif [ $cpu -eq $((nr_cpus - 2)) ]; then
+ expected_range="0-$((cpu - 1)),$((nr_cpus - 1))"
+ elif [ $cpu -eq $((nr_cpus - 1)) ]; then
+ expected_range="0-$((nr_cpus - 2))"
+ else
+ expected_range="0-$((cpu - 1)),$((cpu + 1))-$((nr_cpus - 1))"
+ fi
+ write "all,^$cpu" 0 "cpumap: $expected_range"
+ done
+fi
+
+echo "# A few attempts of pinning to a random range of cpus" >> $outfile
+if [ $nr_cpus -gt 2 ]; then
+ for i in `seq 0 3`; do
+ cpua=$(($RANDOM % nr_cpus))
+ range=$((nr_cpus - cpua))
+ cpub=$(($RANDOM % range))
+ cpubb=$((cpua + cpub))
+ if [ $cpua -eq $cpubb ]; then
+ expected_range="$cpua"
+ else
+ expected_range="$cpua-$cpubb"
+ fi
+ write "$expected_range" 0 "cpumap: $expected_range"
+ done
+fi
+
+echo "# A few attempts of pinning to just one random node" >> $outfile
+if [ $nr_nodes -gt 1 ]; then
+ for i in `seq 0 3`; do
+ node=$(($RANDOM % nr_nodes))
+ # this assumes that the first $nr_cpus_per_node (from cpu
+ # 0 to cpu $nr_cpus_per_node-1) are assigned to the first node
+ # (node 0), the second $nr_cpus_per_node (from $nr_cpus_per_node
+ # to 2*$nr_cpus_per_node-1) are assigned to the second node (node
+ # 1), etc. Expect failures if that is not the case.
+ write "nodes:$node" 0 "cpumap: $((nr_cpus_per_node*node))-$((nr_cpus_per_node*(node+1)-1))"
+ done
+fi
+
+echo "# A few attempts of pinning to all but one random node" >> $outfile
+if [ $nr_nodes -gt 1 ]; then
+ for i in `seq 0 3`; do
+ node=$(($RANDOM % nr_nodes))
+ # this assumes that the first $nr_cpus_per_node (from cpu
+ # 0 to cpu $nr_cpus_per_node-1) are assigned to the first node
+ # (node 0), the second $nr_cpus_per_node (from $nr_cpus_per_node
+ # to 2*$nr_cpus_per_node-1) are assigned to the second node (node
+ # 1), etc. Expect failures if that is not the case.
+ if [ $node -eq 0 ]; then
+ expected_range="$nr_cpus_per_node-$((nr_cpus - 1))"
+ elif [ $node -eq $((nr_nodes - 1)) ]; then
+ expected_range="0-$((nr_cpus - nr_cpus_per_node - 1))"
+ else
+ expected_range="0-$((nr_cpus_per_node*node-1)),$((nr_cpus_per_node*(node+1)))-$nr_cpus"
+ fi
+ write "all,^nodes:$node" 0 "cpumap: $expected_range"
+ done
+fi
+
+echo "# A few attempts of pinning to a random range of nodes" >> $outfile
+if [ $nr_nodes -gt 1 ]; then
+ for i in `seq 0 3`; do
+ nodea=$(($RANDOM % nr_nodes))
+ range=$((nr_nodes - nodea))
+ nodeb=$(($RANDOM % range))
+ nodebb=$((nodea + nodeb))
+ # this assumes that the first $nr_cpus_per_node (from cpu
+ # 0 to cpu $nr_cpus_per_node-1) are assigned to the first node
+ # (node 0), the second $nr_cpus_per_node (from $nr_cpus_per_node
+ # to 2*$nr_cpus_per_node-1) are assigned to the second node (node
+ # 1), etc. Expect failures if that is not the case.
+ if [ $nodea -eq 0 ] && [ $nodebb -eq $((nr_nodes - 1)) ]; then
+ expected_range="any cpu"
+ else
+ expected_range="$((nr_cpus_per_node*nodea))-$((nr_cpus_per_node*(nodebb+1) - 1))"
+ fi
+ write "nodes:$nodea-$nodebb" 0 "cpumap: $expected_range"
+ done
+fi
+
+echo "# A few attempts of pinning to a node but excluding one random cpu" >> $outfile
+if [ $nr_nodes -gt 1 ]; then
+ for i in `seq 0 3`; do
+ node=$(($RANDOM % nr_nodes))
+ # this assumes that the first $nr_cpus_per_node (from cpu
+ # 0 to cpu $nr_cpus_per_node-1) are assigned to the first node
+ # (node 0), the second $nr_cpus_per_node (from $nr_cpus_per_node
+ # to 2*$nr_cpus_per_node-1) are assigned to the second node (node
+ # 1), etc. Expect failures if that is not the case.
+ cpu=$(($RANDOM % nr_cpus_per_node + nr_cpus_per_node*node))
+ if [ $cpu -eq $((nr_cpus_per_node*node)) ]; then
+ expected_range="$((nr_cpus_per_node*node + 1))-$((nr_cpus_per_node*(node+1) - 1))"
+ elif [ $cpu -eq $((nr_cpus_per_node*node + 1)) ]; then
+ expected_range="$((nr_cpus_per_node*node)),$((nr_cpus_per_node*node + 2))-$((nr_cpus_per_node*(node+1) - 1))"
+ elif [ $cpu -eq $((nr_cpus_per_node*(node+1) - 2)) ]; then
+ expected_range="$((nr_cpus_per_node*node))-$((nr_cpus_per_node*(node+1) - 3)),$((nr_cpus_per_node*(node+1) - 1))"
+ elif [ $cpu -eq $((nr_cpus_per_node*(node+1) - 1)) ]; then
+ expected_range="$((nr_cpus_per_node*node))-$((nr_cpus_per_node*(node+1) - 2))"
+ else
+ expected_range="$((nr_cpus_per_node*node))-$((cpu - 1)),$((cpu + 1))-$((nr_cpus_per_node*(node+1) - 1))"
+ fi
+ write "nodes:$node,^$cpu" 0 "cpumap: $expected_range"
+ done
+fi
+
+run $outfile
diff --git a/tools/libxl/check-xl-vcpupin-parse.data-example b/tools/libxl/check-xl-vcpupin-parse.data-example
new file mode 100644
index 0000000..c9cf660
--- /dev/null
+++ b/tools/libxl/check-xl-vcpupin-parse.data-example
@@ -0,0 +1,53 @@
+# WARNING: some of these tests are topology based tests.
+# Expect failures if the topology is not detected correctly
+# detected topology: 16 CPUs, 2 nodes, 8 CPUs per node.
+#
+# seed used for random number generation: seed=13328.
+#
+# Format is as follows:
+# test-string*expected-return-code*expected-output
+#
+# Testing a wring configuration
+foo*255*
+# Testing the 'all' syntax
+all*0*cpumap: any cpu
+nodes:all*0*cpumap: any cpu
+all,nodes:all*0*cpumap: any cpu
+all,^nodes:0,all*0*cpumap: any cpu
+# Testing the empty cpumap case
+^0*0*cpumap: none
+# A few attempts of pinning to just one random cpu
+0*0*cpumap: 0
+9*0*cpumap: 9
+6*0*cpumap: 6
+0*0*cpumap: 0
+# A few attempts of pinning to all but one random cpu
+all,^12*0*cpumap: 0-11,13-15
+all,^6*0*cpumap: 0-5,7-15
+all,^3*0*cpumap: 0-2,4-15
+all,^7*0*cpumap: 0-6,8-15
+# A few attempts of pinning to a random range of cpus
+13-15*0*cpumap: 13-15
+7*0*cpumap: 7
+3-5*0*cpumap: 3-5
+8-11*0*cpumap: 8-11
+# A few attempts of pinning to just one random node
+nodes:1*0*cpumap: 8-15
+nodes:0*0*cpumap: 0-7
+nodes:0*0*cpumap: 0-7
+nodes:0*0*cpumap: 0-7
+# A few attempts of pinning to all but one random node
+all,^nodes:0*0*cpumap: 8-15
+all,^nodes:1*0*cpumap: 0-7
+all,^nodes:1*0*cpumap: 0-7
+all,^nodes:0*0*cpumap: 8-15
+# A few attempts of pinning to a random range of nodes
+nodes:1-1*0*cpumap: 8-15
+nodes:1-1*0*cpumap: 8-15
+nodes:0-1*0*cpumap: any cpu
+nodes:0-0*0*cpumap: 0-7
+# A few attempts of pinning to a node but excluding one random cpu
+nodes:1,^8*0*cpumap: 9-15
+nodes:0,^6*0*cpumap: 0-5,7
+nodes:1,^9*0*cpumap: 8,10-15
+nodes:0,^5*0*cpumap: 0-4,6-7
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH 0/5] xl: allow for node-wise specification of vcpu pinning
2013-09-16 18:40 [PATCH 0/5] xl: allow for node-wise specification of vcpu pinning Dario Faggioli
` (4 preceding siblings ...)
2013-09-16 18:41 ` [PATCH 5/5] xl: test script for the cpumap parser (for vCPU pinning) Dario Faggioli
@ 2013-09-16 18:47 ` Dario Faggioli
2013-09-25 11:03 ` Dario Faggioli
5 siblings, 1 reply; 10+ messages in thread
From: Dario Faggioli @ 2013-09-16 18:47 UTC (permalink / raw)
To: xen-devel; +Cc: George Dunlap, Ian Jackson, Ian Campbell
[-- Attachment #1.1: Type: text/plain, Size: 1180 bytes --]
On lun, 2013-09-16 at 20:40 +0200, Dario Faggioli wrote:
> Hi all,
>
> Take 3 of this series. It allows for specifying vcpu pinning taking NUMA nodes
> into account. IOW, something like this:
>
> * "nodes:0-3": all pCPUs of nodes 0,1,2,3;
> * "nodes:0-3,^node:2": all pCPUS of nodes 0,1,3;
> * "1,nodes:1-2,^6": pCPU 1 plus all pCPUs of nodes 1,2
> but not pCPU 6;
>
> All comments that v2 got have been addressed. In particular, the test script
> has been rearranged and enhanced to match the requested behavior (generate a
> test vector file and then read it).
>
And of course I forgot to put a "v3" tag in the subject line.
Also, here's the git tree for this version of the series:
git://xenbits.xen.org/people/dariof/xen.git numa/xl-vcpupin-nodewise-v3
http://xenbits.xen.org/gitweb/?p=people/dariof/xen.git;a=shortlog;h=refs/heads/numa/xl-vcpupin-nodewise-v3
Sorry,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [PATCH 0/5] xl: allow for node-wise specification of vcpu pinning
2013-09-16 18:47 ` [PATCH 0/5] xl: allow for node-wise specification of vcpu pinning Dario Faggioli
@ 2013-09-25 11:03 ` Dario Faggioli
2013-11-05 17:24 ` George Dunlap
0 siblings, 1 reply; 10+ messages in thread
From: Dario Faggioli @ 2013-09-25 11:03 UTC (permalink / raw)
To: xen-devel; +Cc: George Dunlap, Ian Jackson, Ian Campbell
[-- Attachment #1.1: Type: text/plain, Size: 1160 bytes --]
On lun, 2013-09-16 at 20:47 +0200, Dario Faggioli wrote:
> On lun, 2013-09-16 at 20:40 +0200, Dario Faggioli wrote:
> > Hi all,
> >
> > Take 3 of this series. It allows for specifying vcpu pinning taking NUMA nodes
> > into account. IOW, something like this:
> >
> > * "nodes:0-3": all pCPUs of nodes 0,1,2,3;
> > * "nodes:0-3,^node:2": all pCPUS of nodes 0,1,3;
> > * "1,nodes:1-2,^6": pCPU 1 plus all pCPUs of nodes 1,2
> > but not pCPU 6;
> >
> > All comments that v2 got have been addressed. In particular, the test script
> > has been rearranged and enhanced to match the requested behavior (generate a
> > test vector file and then read it).
> >
> And of course I forgot to put a "v3" tag in the subject line.
>
Ping?
I know IanJ is away, but perhaps I at least can get the Ack-s or the
comments from the others (well, from IanC, I suppose :-) ).
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 0/5] xl: allow for node-wise specification of vcpu pinning
2013-09-25 11:03 ` Dario Faggioli
@ 2013-11-05 17:24 ` George Dunlap
2013-11-05 22:33 ` Dario Faggioli
0 siblings, 1 reply; 10+ messages in thread
From: George Dunlap @ 2013-11-05 17:24 UTC (permalink / raw)
To: Dario Faggioli; +Cc: Ian Jackson, Ian Campbell, xen-devel
On 09/25/2013 12:03 PM, Dario Faggioli wrote:
> On lun, 2013-09-16 at 20:47 +0200, Dario Faggioli wrote:
>> On lun, 2013-09-16 at 20:40 +0200, Dario Faggioli wrote:
>>> Hi all,
>>>
>>> Take 3 of this series. It allows for specifying vcpu pinning taking NUMA nodes
>>> into account. IOW, something like this:
>>>
>>> * "nodes:0-3": all pCPUs of nodes 0,1,2,3;
>>> * "nodes:0-3,^node:2": all pCPUS of nodes 0,1,3;
>>> * "1,nodes:1-2,^6": pCPU 1 plus all pCPUs of nodes 1,2
>>> but not pCPU 6;
>>>
>>> All comments that v2 got have been addressed. In particular, the test script
>>> has been rearranged and enhanced to match the requested behavior (generate a
>>> test vector file and then read it).
>>>
>> And of course I forgot to put a "v3" tag in the subject line.
>>
> Ping?
>
> I know IanJ is away, but perhaps I at least can get the Ack-s or the
> comments from the others (well, from IanC, I suppose :-) ).
Dario,
All of these were folded into the series you just re-sent, right? I can
delete these from my queue of things to look at? :-) (Yes, it's been
there this whole time...)
-George
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 0/5] xl: allow for node-wise specification of vcpu pinning
2013-11-05 17:24 ` George Dunlap
@ 2013-11-05 22:33 ` Dario Faggioli
0 siblings, 0 replies; 10+ messages in thread
From: Dario Faggioli @ 2013-11-05 22:33 UTC (permalink / raw)
To: George Dunlap; +Cc: Ian Jackson, Ian Campbell, xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 921 bytes --]
On mar, 2013-11-05 at 17:24 +0000, George Dunlap wrote:
> On 09/25/2013 12:03 PM, Dario Faggioli wrote:
> > On lun, 2013-09-16 at 20:47 +0200, Dario Faggioli wrote:
> > Ping?
> >
> > I know IanJ is away, but perhaps I at least can get the Ack-s or the
> > comments from the others (well, from IanC, I suppose :-) ).
>
> Dario,
>
> All of these were folded into the series you just re-sent, right?
>
Right indeed.
> I can
> delete these from my queue of things to look at? :-)
>
Go ahead. :-)
> (Yes, it's been there this whole time...)
>
Right... Sure... Oh, look, a pink elephant crossing the road! :-P :-P
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread