x86/dom0: limit dom0_max_vcpus value

x86/dom0: limit dom0_max_vcpus value

[Andrew Cooper]

[-- Attachment #1: Type: text/plain, Size: 272 bytes --]

This caused particularly poor performance when booting a server in
uniprocessor mode for debugging reasons, and had 4 dom0 vcpus competing
for 1pcpus worth of time.

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com


[-- Attachment #2: limit-dom_max_vcpus.patch --]
[-- Type: text/x-patch, Size: 1413 bytes --]

# HG changeset patch
# Parent 5d20d2f6ffed0a49f030f04a8870f1926babbcbf
x86/dom0: limit dom0_max_vcpus value

The command line parameter "dom0_max_vcpus" is useful on large servers where
dom0 does not want many vcpus, as an alternative to booting and hot-unplugging
down to a sensible number (which is poor memory-wize as many drivers allocate
large per-cpu buffers which will never be used after the hot-unplug).

However, as the code currently stands, the value for "dom0_max_vcpus" is used as
an absolute value rather than a maximum, resulting in dom0 getting more vcpus
than pcpus.  This can lead to unexpected poor performance.

Therefore, augment the "opt_dom0_max_vcpus == 0" test to also check whether the
user specified parameter is greater than the number of available pcpus.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

diff -r 5d20d2f6ffed xen/arch/x86/domain_build.c
--- a/xen/arch/x86/domain_build.c
+++ b/xen/arch/x86/domain_build.c
@@ -87,8 +87,9 @@ integer_param("dom0_max_vcpus", opt_dom0
 
 struct vcpu *__init alloc_dom0_vcpu0(void)
 {
-    if ( opt_dom0_max_vcpus == 0 )
-        opt_dom0_max_vcpus = num_cpupool_cpus(cpupool0);
+    int nr_pcpus = num_cpupool_cpus(cpupool0);
+    if ( opt_dom0_max_vcpus == 0 || opt_dom0_max_vcpus > nr_pcpus)
+        opt_dom0_max_vcpus = nr_pcpus;
     if ( opt_dom0_max_vcpus > MAX_VIRT_CPUS )
         opt_dom0_max_vcpus = MAX_VIRT_CPUS;
 

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: x86/dom0: limit dom0_max_vcpus value

[Jan Beulich]

>>> On 12.03.12 at 16:56, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> This caused particularly poor performance when booting a server in
> uniprocessor mode for debugging reasons, and had 4 dom0 vcpus competing
> for 1pcpus worth of time.

NAK. This was intentionally removed in an earlier c/s (and I'm in fact
making use of this for certain types of stress tests). No-one forces
you or anyone else to boot with dom0_max_vcpus=4 when there's
just a single pCPU.

Jan

Re: x86/dom0: limit dom0_max_vcpus value

[Andrew Cooper]

On 12/03/12 16:12, Jan Beulich wrote:
>>>> On 12.03.12 at 16:56, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> This caused particularly poor performance when booting a server in
>> uniprocessor mode for debugging reasons, and had 4 dom0 vcpus competing
>> for 1pcpus worth of time.
> NAK. This was intentionally removed in an earlier c/s (and I'm in fact
> making use of this for certain types of stress tests). No-one forces
> you or anyone else to boot with dom0_max_vcpus=4 when there's
> just a single pCPU.
>
> Jan

What is the justification for removing it? c/s 18266:d31546a3883e has no
explanation.

This is now resulting in a command line parameter with "max" in its name
acting unlike all other "max" parameters.  There is certainly an
argument for introducing a "dom0_cpus" parameter for setting an exact
number, but I feel that this behavior is wrong for a parameter with
"max" in its name.

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com

Re: x86/dom0: limit dom0_max_vcpus value

[Jan Beulich]

>>> On 12.03.12 at 17:32, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> On 12/03/12 16:12, Jan Beulich wrote:
>>>>> On 12.03.12 at 16:56, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>>> This caused particularly poor performance when booting a server in
>>> uniprocessor mode for debugging reasons, and had 4 dom0 vcpus competing
>>> for 1pcpus worth of time.
>> NAK. This was intentionally removed in an earlier c/s (and I'm in fact
>> making use of this for certain types of stress tests). No-one forces
>> you or anyone else to boot with dom0_max_vcpus=4 when there's
>> just a single pCPU.
>>
>> Jan
> 
> What is the justification for removing it? c/s 18266:d31546a3883e has no
> explanation.

The restriction was artificial, enforcing policy where none should be
enforced.

> This is now resulting in a command line parameter with "max" in its name
> acting unlike all other "max" parameters.  There is certainly an
> argument for introducing a "dom0_cpus" parameter for setting an exact
> number, but I feel that this behavior is wrong for a parameter with
> "max" in its name.

No, you misunderstand the 'max' here: Dom0 can't ever grow beyond
that number (see the implementation of XEN_DOMCTL_max_vcpus),
and hence what you specify here _is_ the maximum. If you want less
active ones, you need to bring them _down_ once the system is up
(via the tool stack or kernel interfaces).

Jan