[PATCH] xl cpupool-numa-split: reduce number of Dom0 vcpus

[PATCH] xl cpupool-numa-split: reduce number of Dom0 vcpus

[Juergen Gross]

[-- Attachment #1: Type: text/plain, Size: 284 bytes --]

When reducing the number of physical cpus available for Domain-0 by xl
cpupool-numa-split, reduce the number of vcpus accordingly.

Signed-off-by: juergen.gross@ts.fujitsu.com


1 file changed, 20 insertions(+), 2 deletions(-)
tools/libxl/xl_cmdimpl.c |   22 ++++++++++++++++++++--



[-- Attachment #2: xen-work.patch --]
[-- Type: text/x-patch, Size: 1830 bytes --]

# HG changeset patch
# User Juergen Gross <juergen.gross@ts.fujitsu.com>
# Date 1297680409 -3600
# Node ID fdcb4e8eb4bb02f2c14b5333b9839dbd7b8786df
# Parent  f3579aa973c24a4ecee809a184b08c1855cd07ba
xl cpupool-numa-split: reduce number of Dom0 vcpus

When reducing the number of physical cpus available for Domain-0 by xl
cpupool-numa-split, reduce the number of vcpus accordingly.

Signed-off-by: juergen.gross@ts.fujitsu.com

diff -r f3579aa973c2 -r fdcb4e8eb4bb tools/libxl/xl_cmdimpl.c
--- a/tools/libxl/xl_cmdimpl.c	Mon Feb 14 11:02:50 2011 +0100
+++ b/tools/libxl/xl_cmdimpl.c	Mon Feb 14 11:46:49 2011 +0100
@@ -5762,6 +5762,7 @@ int main_cpupoolnumasplit(int argc, char
     libxl_cpumap cpumap;
     libxl_cpupoolinfo *poolinfo;
     libxl_topologyinfo topology;
+    libxl_dominfo info;
 
     while ((opt = getopt(argc, argv, "h")) != -1) {
         switch (opt) {
@@ -5817,11 +5818,28 @@ int main_cpupoolnumasplit(int argc, char
         goto out;
     }
 
+    n = 0;
     for (c = 0; c < topology.nodemap.entries; c++) {
         if (topology.nodemap.array[c] == node) {
             topology.nodemap.array[c] = LIBXL_CPUARRAY_INVALID_ENTRY;
-        }
-    }
+            libxl_cpumap_set(&cpumap, n);
+            n++;
+        }
+    }
+    if (libxl_set_vcpuonline(&ctx, 0, &cpumap)) {
+        fprintf(stderr, "error on removing vcpus for Domain-0\n");
+        goto out;
+    }
+    for (c = 0; c < 10; c++) {
+        if (libxl_domain_info(&ctx, &info, 0)) {
+            fprintf(stderr, "error on getting info for Domain-0\n");
+            goto out;
+        }
+        if (info.vcpu_online == n) {
+            break;
+        }
+    }
+    memset(cpumap.map, 0, cpumap.size);
 
     for (c = 0; c < topology.nodemap.entries; c++) {
         if (topology.nodemap.array[c] == LIBXL_CPUARRAY_INVALID_ENTRY) {

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH] xl cpupool-numa-split: reduce number of Dom0 vcpus

[George Dunlap]

Is looping 10 times really the best option?

 -George

On Mon, Feb 14, 2011 at 10:47 AM, Juergen Gross
<juergen.gross@ts.fujitsu.com> wrote:
> When reducing the number of physical cpus available for Domain-0 by xl
> cpupool-numa-split, reduce the number of vcpus accordingly.
>
> Signed-off-by: juergen.gross@ts.fujitsu.com
>
>
> 1 file changed, 20 insertions(+), 2 deletions(-)
> tools/libxl/xl_cmdimpl.c |   22 ++++++++++++++++++++--
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>

Re: [PATCH] xl cpupool-numa-split: reduce number of Dom0 vcpus

[Juergen Gross]

[-- Attachment #1: Type: text/plain, Size: 505 bytes --]

On 02/14/11 13:36, George Dunlap wrote:
> Is looping 10 times really the best option?

Oops! Forgot the sleep(1) :-(

What about this one?

-- 
Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@ts.fujitsu.com
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

[-- Attachment #2: xen-work.patch --]
[-- Type: text/x-patch, Size: 1830 bytes --]

# HG changeset patch
# User Juergen Gross <juergen.gross@ts.fujitsu.com>
# Date 1297680409 -3600
# Node ID fdcb4e8eb4bb02f2c14b5333b9839dbd7b8786df
# Parent  f3579aa973c24a4ecee809a184b08c1855cd07ba
xl cpupool-numa-split: reduce number of Dom0 vcpus

When reducing the number of physical cpus available for Domain-0 by xl
cpupool-numa-split, reduce the number of vcpus accordingly.

Signed-off-by: juergen.gross@ts.fujitsu.com

diff -r f3579aa973c2 -r fdcb4e8eb4bb tools/libxl/xl_cmdimpl.c
--- a/tools/libxl/xl_cmdimpl.c	Mon Feb 14 11:02:50 2011 +0100
+++ b/tools/libxl/xl_cmdimpl.c	Mon Feb 14 11:46:49 2011 +0100
@@ -5762,6 +5762,7 @@ int main_cpupoolnumasplit(int argc, char
     libxl_cpumap cpumap;
     libxl_cpupoolinfo *poolinfo;
     libxl_topologyinfo topology;
+    libxl_dominfo info;
 
     while ((opt = getopt(argc, argv, "h")) != -1) {
         switch (opt) {
@@ -5817,11 +5818,28 @@ int main_cpupoolnumasplit(int argc, char
         goto out;
     }
 
+    n = 0;
     for (c = 0; c < topology.nodemap.entries; c++) {
         if (topology.nodemap.array[c] == node) {
             topology.nodemap.array[c] = LIBXL_CPUARRAY_INVALID_ENTRY;
-        }
-    }
+            libxl_cpumap_set(&cpumap, n);
+            n++;
+        }
+    }
+    if (libxl_set_vcpuonline(&ctx, 0, &cpumap)) {
+        fprintf(stderr, "error on removing vcpus for Domain-0\n");
+        goto out;
+    }
+    for (c = 0; c < 10; c++) {
+        if (libxl_domain_info(&ctx, &info, 0)) {
+            fprintf(stderr, "error on getting info for Domain-0\n");
+            goto out;
+        }
+        if (info.vcpu_online == n) {
+            break;
+        }
+    }
+    memset(cpumap.map, 0, cpumap.size);
 
     for (c = 0; c < topology.nodemap.entries; c++) {
         if (topology.nodemap.array[c] == LIBXL_CPUARRAY_INVALID_ENTRY) {

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH] xl cpupool-numa-split: reduce number of Dom0 vcpus

[Ian Jackson]

Juergen Gross writes ("Re: [Xen-devel] [PATCH] xl cpupool-numa-split: reduce number of Dom0 vcpus"):
> On 02/14/11 13:36, George Dunlap wrote:
> > Is looping 10 times really the best option?
> 
> Oops! Forgot the sleep(1) :-(
> What about this one?

Um, are you sure ?  It still doesn't seem to have a sleep and you
haven't answered George's question ...

Ian.

Re: [PATCH] xl cpupool-numa-split: reduce number of Dom0 vcpus

[Juergen Gross]

[-- Attachment #1: Type: text/plain, Size: 1213 bytes --]

On 02/14/11 16:45, Ian Jackson wrote:
> Juergen Gross writes ("Re: [Xen-devel] [PATCH] xl cpupool-numa-split: reduce number of Dom0 vcpus"):
>> On 02/14/11 13:36, George Dunlap wrote:
>>> Is looping 10 times really the best option?
>>
>> Oops! Forgot the sleep(1) :-(
>> What about this one?
>
> Um, are you sure ?  It still doesn't seem to have a sleep and you
> haven't answered George's question ...

Sigh. Modifying the patch isn't enough, have to send the modified version...

Regarding the question: libxl_set_vcpuonline() is asynchronous. It will
trigger the offlining (or onlining) of vcpus, but the result is not checked
and will happen sometimes in the future, if ever. So some kind of timeout
seems to be a good idea. 10 seconds should be enough to do the work.

I added a failure exit if not all required vcpus are offlined.


Juergen

-- 
Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@ts.fujitsu.com
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

[-- Attachment #2: xen-work.patch --]
[-- Type: text/x-patch, Size: 1963 bytes --]

# HG changeset patch
# User Juergen Gross <juergen.gross@ts.fujitsu.com>
# Date 1297680409 -3600
# Node ID fdcb4e8eb4bb02f2c14b5333b9839dbd7b8786df
# Parent  f3579aa973c24a4ecee809a184b08c1855cd07ba
xl cpupool-numa-split: reduce number of Dom0 vcpus

When reducing the number of physical cpus available for Domain-0 by xl
cpupool-numa-split, reduce the number of vcpus accordingly.

Signed-off-by: juergen.gross@ts.fujitsu.com

diff -r f3579aa973c2 -r fdcb4e8eb4bb tools/libxl/xl_cmdimpl.c
--- a/tools/libxl/xl_cmdimpl.c	Mon Feb 14 11:02:50 2011 +0100
+++ b/tools/libxl/xl_cmdimpl.c	Mon Feb 14 11:46:49 2011 +0100
@@ -5762,6 +5762,7 @@ int main_cpupoolnumasplit(int argc, char
     libxl_cpumap cpumap;
     libxl_cpupoolinfo *poolinfo;
     libxl_topologyinfo topology;
+    libxl_dominfo info;
 
     while ((opt = getopt(argc, argv, "h")) != -1) {
         switch (opt) {
@@ -5817,11 +5818,33 @@ int main_cpupoolnumasplit(int argc, char
         goto out;
     }
 
+    n = 0;
     for (c = 0; c < topology.nodemap.entries; c++) {
         if (topology.nodemap.array[c] == node) {
             topology.nodemap.array[c] = LIBXL_CPUARRAY_INVALID_ENTRY;
-        }
-    }
+            libxl_cpumap_set(&cpumap, n);
+            n++;
+        }
+    }
+    if (libxl_set_vcpuonline(&ctx, 0, &cpumap)) {
+        fprintf(stderr, "error on removing vcpus for Domain-0\n");
+        goto out;
+    }
+    for (c = 0; c < 10; c++) {
+        if (libxl_domain_info(&ctx, &info, 0)) {
+            fprintf(stderr, "error on getting info for Domain-0\n");
+            goto out;
+        }
+        if (info.vcpu_online == n) {
+            break;
+        }
+        sleep(1);
+    }
+    if (info.vcpu_online > n) {
+        fprintf(stderr, "failed to offline vcpus\n");
+        goto out;
+    }
+    memset(cpumap.map, 0, cpumap.size);
 
     for (c = 0; c < topology.nodemap.entries; c++) {
         if (topology.nodemap.array[c] == LIBXL_CPUARRAY_INVALID_ENTRY) {

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH] xl cpupool-numa-split: reduce number of Dom0 vcpus

[Ian Jackson]

Juergen Gross writes ("Re: [Xen-devel] [PATCH] xl cpupool-numa-split: reduce number of Dom0 vcpus"):
> I added a failure exit if not all required vcpus are offlined.

Thanks.  But, is this test correct ?

> +        if (info.vcpu_online == n) {
> +            break;
> +        }

Is it possible for the online map to have n cpus in it, but for them
to be the wrong n cpus ?

What if two copies of this code run concurrently ?  (Perhaps in 4.2 we
will need to add a lock?)

Ian.

Re: [PATCH] xl cpupool-numa-split: reduce number of Dom0 vcpus

[Juergen Gross]

On 02/15/11 20:27, Ian Jackson wrote:
> Juergen Gross writes ("Re: [Xen-devel] [PATCH] xl cpupool-numa-split: reduce number of Dom0 vcpus"):
>> I added a failure exit if not all required vcpus are offlined.
>
> Thanks.  But, is this test correct ?
>
>> +        if (info.vcpu_online == n) {
>> +            break;
>> +        }
>
> Is it possible for the online map to have n cpus in it, but for them
> to be the wrong n cpus ?

The number of online vcpus is important, not which vcpus.
In a strange case it could lead to a wrong number of online vcpus, if the
number was correct in the beginning and e.g. vcpu 0 was offline and
offlining the last vcpu fails. But I don't think this would be a huge
problem, as offlining the vcpus is just a optimization measure.

>
> What if two copies of this code run concurrently ?  (Perhaps in 4.2 we
> will need to add a lock?)

The result of both copies should be the same, as it is depending on the
machine configuration only.
And we are speaking of a global machine configuration action. This would mean
two administrators are configuring the same machine without knowing what the
other is doing. Seems to be a very strange scenario. A possible failure
(meaning: not reaching the desired configuration, but no crashing) should be
regarded as acceptable in this case.


Juergen

-- 
Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@ts.fujitsu.com
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

Re: [PATCH] xl cpupool-numa-split: reduce number of Dom0 vcpus

[Ian Jackson]

Juergen Gross writes ("Re: [Xen-devel] [PATCH] xl cpupool-numa-split: reduce number of Dom0 vcpus"):
> Regarding the question: libxl_set_vcpuonline() is asynchronous. It will
> trigger the offlining (or onlining) of vcpus, but the result is not checked
> and will happen sometimes in the future, if ever. So some kind of timeout
> seems to be a good idea. 10 seconds should be enough to do the work.
> 
> I added a failure exit if not all required vcpus are offlined.

Thanks, and for your replies to my questions.  I have applied your
patch.

Ian.