[PATCH] All domains (including dom0) should be best effort upon creation.

[PATCH] All domains (including dom0) should be best effort upon creation.

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 2626 bytes --]

All domains (including dom0) should be best effort upon creation.

In the sedf scheduler, while trying to guarantee to dom0 the proper
amount of CPU time for being able to do its job, we end up allocating
75% CPU-share to each and every of its VCPUs. This, combined with the
fact that such a scheduler has no load balancing logic at all (and
thus *all* dom0's VCPUs just rise and stay on physical CPU #0), means
that for example on a 12-cores system we're trying to exploit
75x12=900% of what we have on a PCPU, which is definitely too much!

Moreover, even if a cleverer mechanism for distributing VCPUs among
the available PCPUs (if not a proper load balancer) will be put in
place some day, it will still be difficult to decide how much guaranteed
CPU bandwidth each of the dom0's VCPUs should be provided with, without
posing the system at risk of livelock or starvation, even during
boot time.

Therefore, since sedf is capable of some form of best-effort
scheduling, the best thing we can do is ask for this behaviour for
dom0, as it is for all other domains, right upon creation. It will
then be the sysadmin's job to modify dom0's scheduling parameters
to better fit his particular needs, maybe after spreading the load
a bit by pinning VCPUs on PCPUs, or using cpupools, etc.

Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>


diff -r 01c8b27e3d7d xen/common/sched_sedf.c
--- a/xen/common/sched_sedf.c	Thu Dec 15 16:56:21 2011 +0000
+++ b/xen/common/sched_sedf.c	Fri Dec 16 16:05:37 2011 +0100
@@ -359,19 +359,9 @@ static void *sedf_alloc_vdata(const stru
     inf->latency     = 0;
     inf->status      = EXTRA_AWARE | SEDF_ASLEEP;
     inf->extraweight = 1;
-
-    if ( v->domain->domain_id == 0 )
-    {
-        /* Domain0 gets 75% guaranteed (15ms every 20ms). */
-        inf->period    = MILLISECS(20);
-        inf->slice     = MILLISECS(15);
-    }
-    else
-    {
-        /* Best-effort extratime only. */
-        inf->period    = WEIGHT_PERIOD;
-        inf->slice     = 0;
-    }
+    /* Upon creation all domain are best-effort. */
+    inf->period      = WEIGHT_PERIOD;
+    inf->slice       = 0;
 
     inf->period_orig = inf->period; inf->slice_orig = inf->slice;
     INIT_LIST_HEAD(&(inf->list));

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-------------------------------------------------------------------
Dario Faggioli, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
PhD Candidate, ReTiS Lab, Scuola Superiore Sant'Anna, Pisa (Italy)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH] All domains (including dom0) should be best effort upon creation.

[George Dunlap]

On Fri, Dec 16, 2011 at 4:29 PM, Dario Faggioli
<dario.faggioli@citrix.com> wrote:
> All domains (including dom0) should be best effort upon creation.
>
> In the sedf scheduler, while trying to guarantee to dom0 the proper
> amount of CPU time for being able to do its job, we end up allocating
> 75% CPU-share to each and every of its VCPUs. This, combined with the
> fact that such a scheduler has no load balancing logic at all (and
> thus *all* dom0's VCPUs just rise and stay on physical CPU #0), means
> that for example on a 12-cores system we're trying to exploit
> 75x12=900% of what we have on a PCPU, which is definitely too much!
>
> Moreover, even if a cleverer mechanism for distributing VCPUs among
> the available PCPUs (if not a proper load balancer) will be put in
> place some day, it will still be difficult to decide how much guaranteed
> CPU bandwidth each of the dom0's VCPUs should be provided with, without
> posing the system at risk of livelock or starvation, even during
> boot time.
>
> Therefore, since sedf is capable of some form of best-effort
> scheduling, the best thing we can do is ask for this behaviour for
> dom0, as it is for all other domains, right upon creation. It will
> then be the sysadmin's job to modify dom0's scheduling parameters
> to better fit his particular needs, maybe after spreading the load
> a bit by pinning VCPUs on PCPUs, or using cpupools, etc.
>
> Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>

Looks reasonable to me.

Acked-by: George Dunlap <george.dunlap@eu.citrix.com>


>
>
> diff -r 01c8b27e3d7d xen/common/sched_sedf.c
> --- a/xen/common/sched_sedf.c   Thu Dec 15 16:56:21 2011 +0000
> +++ b/xen/common/sched_sedf.c   Fri Dec 16 16:05:37 2011 +0100
> @@ -359,19 +359,9 @@ static void *sedf_alloc_vdata(const stru
>     inf->latency     = 0;
>     inf->status      = EXTRA_AWARE | SEDF_ASLEEP;
>     inf->extraweight = 1;
> -
> -    if ( v->domain->domain_id == 0 )
> -    {
> -        /* Domain0 gets 75% guaranteed (15ms every 20ms). */
> -        inf->period    = MILLISECS(20);
> -        inf->slice     = MILLISECS(15);
> -    }
> -    else
> -    {
> -        /* Best-effort extratime only. */
> -        inf->period    = WEIGHT_PERIOD;
> -        inf->slice     = 0;
> -    }
> +    /* Upon creation all domain are best-effort. */
> +    inf->period      = WEIGHT_PERIOD;
> +    inf->slice       = 0;
>
>     inf->period_orig = inf->period; inf->slice_orig = inf->slice;
>     INIT_LIST_HEAD(&(inf->list));
>
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -------------------------------------------------------------------
> Dario Faggioli, http://retis.sssup.it/people/faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
> PhD Candidate, ReTiS Lab, Scuola Superiore Sant'Anna, Pisa (Italy)
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

Re: [PATCH] All domains (including dom0) should be best effort upon creation.

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 653 bytes --]

On Fri, 2011-12-16 at 18:06 +0000, George Dunlap wrote:
> > Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
> 
> Looks reasonable to me.
> 
> Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
> 
> 
Again... Should I do something else or can this be checked-in? :-)

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-------------------------------------------------------------------
Dario Faggioli, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
PhD Candidate, ReTiS Lab, Scuola Superiore Sant'Anna, Pisa (Italy)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel