Plans for the SEDF scheduler

Plans for the SEDF scheduler

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 5289 bytes --]

Hi everyone,

I've copied the committers, Lars as community manager, Wei as the
release manager, George as the scheduler maintainer, RT-Xen and
DornerWorks people, which have some interest in the field, I believe.
Please, widen the crowd if you think it's necessary/best.

Straight to the point: the SEDF scheduler code is in a rather bad state:

 * it is buggy (tests are failing since... well, at least since when I
   joined the community, 3 years ago!). Basically, it was not designed
   with host and guest SMP support in mind, and has never been updated;

 * it is not used. Well, that's hard to tell, of course, but I really
   really expect that to be the case.

 * it is useless as:
    - it tries to be a general purpose scheduler, but we have Credit
      (and, soon enough, hopefully, Credit2) for that
    - for real-time we have ARINC653 (super-hard real-time), maintained
      and supported, and RTDS (hard/soft real-time), actively
      developed.

For these reasons, I think we should get rid of it. Actually, I've
discovered that this was the plan in 2006 already:  :-)

 git show db51cd09d37ea44b126bb259f9392248afd768e6
 ...
 diff --git a/docs/src/interface.tex b/docs/src/interface.tex
 index c9017c7..9a59840 100644
 --- a/docs/src/interface.tex
 +++ b/docs/src/interface.tex
 @@ -209,8 +209,8 @@ implement timeout values when they block.
 
  Xen offers a uniform API for CPU schedulers.  It is possible to choose
  from a number of schedulers at boot and it should be easy to add more.
 -The SEDF, BVT, and Credit schedulers are part of the normal Xen
 -distribution.  BVT and SEDF will be going away and their use should be
 +The SEDF and Credit schedulers are part of the normal Xen
 +distribution.  SEDF will be going away and its use should be
  avoided once the credit scheduler has stabilized and become the default.
  The Credit scheduler provides proportional fair shares of the
  host's CPUs to the running domains. It does this while transparently

I'm up for sending a series to that effect, but I'd be interested in
your opinion on what's the best way to do so.

Looking at the commit referred above, it seems like a scheduler can just
disappear which, as far as Xen and libxc are concerned, does seem ok to
me. Is that so?

I'm less sure of libxl, because of the API stability claims. It looks
like this is all about (as there are not scheduler specific API
functions, which is good) this:

 # Consistent with values defined in domctl.h
 # Except unknown which we have made up
 libxl_scheduler = Enumeration("scheduler", [
     (0, "unknown"),
     (4, "sedf"),
     (5, "credit"),
     (6, "credit2"),
     (7, "arinc653"),
     (8, "rtds"),
     ])

The solutions I see are:
 1. get rid of the field. This does not sound ideal from an API
    stability point of view;
 2. keep the field 'physically', but return warnings and errors when it 
    is used. This is a change in behavior, so also not really stable
    (but, hey, the scheduler is going away, the behavior will change,
    one way or another!)
 3. RTDS has a really similar interface and, to a certain extent, a 
    similar behavior too. Would it make sense to, upon suitable
    warnings, of course, convert the requests within libxl, i.e.,
    when someone asks for SEDF, we give him RTDS?

Honestly, I'd prefer 2, and I don't much like 3. However, it really
seems that 3 gives us a way to keep most of our promises about API
stability... Thoughts?

About domain_sched_params:

 libxl_domain_sched_params = Struct("domain_sched_params",[
     ("sched",        libxl_scheduler),
     ("weight",       integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_WEIGHT_DEFAULT'}),
     ("cap",          integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_CAP_DEFAULT'}),
     ("period",       integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_PERIOD_DEFAULT'}),
     ("slice",        integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_SLICE_DEFAULT'}),
     ("latency",      integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_LATENCY_DEFAULT'}),
     ("extratime",    integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT'}),
     ("budget",       integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT'}),
     ])

SEDF now uses 'weight', 'period', 'slice', 'latency' and 'extratime'. If
considering option 3, RTDS:
 - does not use 'weight' right now, but we can use it as an alternative
   way to set 'budget', which is not far from the way SEDF uses it;
 - uses 'period';
 - does not use 'slice', but that is just a synonym for 'budget';
 - does not use 'latency' right now; we can use it as an alternative way
   to set 'period', but that would be a bit different from what SEDF
   does with it;
 - does not use 'extratime' for now, but will, at some point, in a very
   similar way to how SEDF uses it.

Oh and, whatever route we decide to go for, is it perhaps necessary that
we do something like deprecate it for one release (4.6), e.g., by
printing a warning saying that <<SEDF is still here now, but is
deprecated and you won't find it in the next release>>, and then do the
actual removal/conversion in 4.7?

Let me know what you think.

Thanks and Regards,
Dario

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Plans for the SEDF scheduler

[George Dunlap]

On 05/21/2015 01:39 PM, Dario Faggioli wrote:
> Hi everyone,
> 
> I've copied the committers, Lars as community manager, Wei as the
> release manager, George as the scheduler maintainer, RT-Xen and
> DornerWorks people, which have some interest in the field, I believe.
> Please, widen the crowd if you think it's necessary/best.
> 
> Straight to the point: the SEDF scheduler code is in a rather bad state:
> 
>  * it is buggy (tests are failing since... well, at least since when I
>    joined the community, 3 years ago!). Basically, it was not designed
>    with host and guest SMP support in mind, and has never been updated;
> 
>  * it is not used. Well, that's hard to tell, of course, but I really
>    really expect that to be the case.
> 
>  * it is useless as:
>     - it tries to be a general purpose scheduler, but we have Credit
>       (and, soon enough, hopefully, Credit2) for that
>     - for real-time we have ARINC653 (super-hard real-time), maintained
>       and supported, and RTDS (hard/soft real-time), actively
>       developed.
> 
> For these reasons, I think we should get rid of it. Actually, I've
> discovered that this was the plan in 2006 already:  :-)
> 
>  git show db51cd09d37ea44b126bb259f9392248afd768e6
>  ...
>  diff --git a/docs/src/interface.tex b/docs/src/interface.tex
>  index c9017c7..9a59840 100644
>  --- a/docs/src/interface.tex
>  +++ b/docs/src/interface.tex
>  @@ -209,8 +209,8 @@ implement timeout values when they block.
>  
>   Xen offers a uniform API for CPU schedulers.  It is possible to choose
>   from a number of schedulers at boot and it should be easy to add more.
>  -The SEDF, BVT, and Credit schedulers are part of the normal Xen
>  -distribution.  BVT and SEDF will be going away and their use should be
>  +The SEDF and Credit schedulers are part of the normal Xen
>  +distribution.  SEDF will be going away and its use should be
>   avoided once the credit scheduler has stabilized and become the default.
>   The Credit scheduler provides proportional fair shares of the
>   host's CPUs to the running domains. It does this while transparently
> 
> I'm up for sending a series to that effect, but I'd be interested in
> your opinion on what's the best way to do so.
> 
> Looking at the commit referred above, it seems like a scheduler can just
> disappear which, as far as Xen and libxc are concerned, does seem ok to
> me. Is that so?
> 
> I'm less sure of libxl, because of the API stability claims. It looks
> like this is all about (as there are not scheduler specific API
> functions, which is good) this:
> 
>  # Consistent with values defined in domctl.h
>  # Except unknown which we have made up
>  libxl_scheduler = Enumeration("scheduler", [
>      (0, "unknown"),
>      (4, "sedf"),
>      (5, "credit"),
>      (6, "credit2"),
>      (7, "arinc653"),
>      (8, "rtds"),
>      ])
> 
> The solutions I see are:
>  1. get rid of the field. This does not sound ideal from an API
>     stability point of view;
>  2. keep the field 'physically', but return warnings and errors when it 
>     is used. This is a change in behavior, so also not really stable
>     (but, hey, the scheduler is going away, the behavior will change,
>     one way or another!)

I think this is probably the best option.

>  3. RTDS has a really similar interface and, to a certain extent, a 
>     similar behavior too. Would it make sense to, upon suitable
>     warnings, of course, convert the requests within libxl, i.e.,
>     when someone asks for SEDF, we give him RTDS?
> 
> Honestly, I'd prefer 2, and I don't much like 3. However, it really
> seems that 3 gives us a way to keep most of our promises about API
> stability... Thoughts?
> 
> About domain_sched_params:
> 
>  libxl_domain_sched_params = Struct("domain_sched_params",[
>      ("sched",        libxl_scheduler),
>      ("weight",       integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_WEIGHT_DEFAULT'}),
>      ("cap",          integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_CAP_DEFAULT'}),
>      ("period",       integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_PERIOD_DEFAULT'}),
>      ("slice",        integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_SLICE_DEFAULT'}),
>      ("latency",      integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_LATENCY_DEFAULT'}),
>      ("extratime",    integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT'}),
>      ("budget",       integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT'}),
>      ])
> 
> SEDF now uses 'weight', 'period', 'slice', 'latency' and 'extratime'. If
> considering option 3, RTDS:
>  - does not use 'weight' right now, but we can use it as an alternative
>    way to set 'budget', which is not far from the way SEDF uses it;
>  - uses 'period';
>  - does not use 'slice', but that is just a synonym for 'budget';
>  - does not use 'latency' right now; we can use it as an alternative way
>    to set 'period', but that would be a bit different from what SEDF
>    does with it;
>  - does not use 'extratime' for now, but will, at some point, in a very
>    similar way to how SEDF uses it.
> 
> Oh and, whatever route we decide to go for, is it perhaps necessary that
> we do something like deprecate it for one release (4.6), e.g., by
> printing a warning saying that <<SEDF is still here now, but is
> deprecated and you won't find it in the next release>>, and then do the
> actual removal/conversion in 4.7?

I think given that it's been "going away" for 9 years, we should
probably just remove it and put it back if people complain.  (And if
they really really can't use RTDS instead.)

 -George

Re: Plans for the SEDF scheduler

[Wei Liu]

On Thu, May 21, 2015 at 02:39:37PM +0200, Dario Faggioli wrote:
> Hi everyone,
> 
> I've copied the committers, Lars as community manager, Wei as the
> release manager, George as the scheduler maintainer, RT-Xen and
> DornerWorks people, which have some interest in the field, I believe.
> Please, widen the crowd if you think it's necessary/best.
> 
> Straight to the point: the SEDF scheduler code is in a rather bad state:
> 
>  * it is buggy (tests are failing since... well, at least since when I
>    joined the community, 3 years ago!). Basically, it was not designed
>    with host and guest SMP support in mind, and has never been updated;
> 
>  * it is not used. Well, that's hard to tell, of course, but I really
>    really expect that to be the case.
> 
>  * it is useless as:
>     - it tries to be a general purpose scheduler, but we have Credit
>       (and, soon enough, hopefully, Credit2) for that
>     - for real-time we have ARINC653 (super-hard real-time), maintained
>       and supported, and RTDS (hard/soft real-time), actively
>       developed.
> 
> For these reasons, I think we should get rid of it. Actually, I've
> discovered that this was the plan in 2006 already:  :-)
> 
>  git show db51cd09d37ea44b126bb259f9392248afd768e6
>  ...
>  diff --git a/docs/src/interface.tex b/docs/src/interface.tex
>  index c9017c7..9a59840 100644
>  --- a/docs/src/interface.tex
>  +++ b/docs/src/interface.tex
>  @@ -209,8 +209,8 @@ implement timeout values when they block.
>  
>   Xen offers a uniform API for CPU schedulers.  It is possible to choose
>   from a number of schedulers at boot and it should be easy to add more.
>  -The SEDF, BVT, and Credit schedulers are part of the normal Xen
>  -distribution.  BVT and SEDF will be going away and their use should be
>  +The SEDF and Credit schedulers are part of the normal Xen
>  +distribution.  SEDF will be going away and its use should be
>   avoided once the credit scheduler has stabilized and become the default.
>   The Credit scheduler provides proportional fair shares of the
>   host's CPUs to the running domains. It does this while transparently
> 
> I'm up for sending a series to that effect, but I'd be interested in
> your opinion on what's the best way to do so.
> 
> Looking at the commit referred above, it seems like a scheduler can just
> disappear which, as far as Xen and libxc are concerned, does seem ok to
> me. Is that so?
> 
> I'm less sure of libxl, because of the API stability claims. It looks
> like this is all about (as there are not scheduler specific API
> functions, which is good) this:
> 
>  # Consistent with values defined in domctl.h
>  # Except unknown which we have made up
>  libxl_scheduler = Enumeration("scheduler", [
>      (0, "unknown"),
>      (4, "sedf"),
>      (5, "credit"),
>      (6, "credit2"),
>      (7, "arinc653"),
>      (8, "rtds"),
>      ])
> 
> The solutions I see are:
>  1. get rid of the field. This does not sound ideal from an API
>     stability point of view;

Don't get rid of that field, because this breaks compilation.

>  2. keep the field 'physically', but return warnings and errors when it 
>     is used. This is a change in behavior, so also not really stable
>     (but, hey, the scheduler is going away, the behavior will change,
>     one way or another!)

This is OK. It's just like deprecating any other fields.

>  3. RTDS has a really similar interface and, to a certain extent, a 
>     similar behavior too. Would it make sense to, upon suitable
>     warnings, of course, convert the requests within libxl, i.e.,
>     when someone asks for SEDF, we give him RTDS?
> 

No. I don't think this is OK.

> Honestly, I'd prefer 2, and I don't much like 3. However, it really
> seems that 3 gives us a way to keep most of our promises about API
> stability... Thoughts?
> 

The API is still stable. We don't carry the burden of maintaining
out dated technologies forever.

> About domain_sched_params:
> 
>  libxl_domain_sched_params = Struct("domain_sched_params",[
>      ("sched",        libxl_scheduler),
>      ("weight",       integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_WEIGHT_DEFAULT'}),
>      ("cap",          integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_CAP_DEFAULT'}),
>      ("period",       integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_PERIOD_DEFAULT'}),
>      ("slice",        integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_SLICE_DEFAULT'}),
>      ("latency",      integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_LATENCY_DEFAULT'}),
>      ("extratime",    integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT'}),
>      ("budget",       integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT'}),
>      ])
> 
> SEDF now uses 'weight', 'period', 'slice', 'latency' and 'extratime'. If
> considering option 3, RTDS:
>  - does not use 'weight' right now, but we can use it as an alternative
>    way to set 'budget', which is not far from the way SEDF uses it;
>  - uses 'period';
>  - does not use 'slice', but that is just a synonym for 'budget';
>  - does not use 'latency' right now; we can use it as an alternative way
>    to set 'period', but that would be a bit different from what SEDF
>    does with it;
>  - does not use 'extratime' for now, but will, at some point, in a very
>    similar way to how SEDF uses it.
> 
> Oh and, whatever route we decide to go for, is it perhaps necessary that
> we do something like deprecate it for one release (4.6), e.g., by
> printing a warning saying that <<SEDF is still here now, but is
> deprecated and you won't find it in the next release>>, and then do the
> actual removal/conversion in 4.7?
> 

That would also help. I think this is better than just remove it in this
release.

One thing to consider is that we have a test case for it, we don't want
the removal to cause regression, hence blocking the pushgate.

Wei.

> Let me know what you think.
> 
> Thanks and Regards,
> Dario

Re: Plans for the SEDF scheduler

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 865 bytes --]

On Thu, 2015-05-21 at 13:53 +0100, Wei Liu wrote:
> On Thu, May 21, 2015 at 02:39:37PM +0200, Dario Faggioli wrote:

> > Oh and, whatever route we decide to go for, is it perhaps necessary that
> > we do something like deprecate it for one release (4.6), e.g., by
> > printing a warning saying that <<SEDF is still here now, but is
> > deprecated and you won't find it in the next release>>, and then do the
> > actual removal/conversion in 4.7?
> > 
> 
> That would also help. I think this is better than just remove it in this
> release.
> 
I'm fine with both... let's see what others think.

> One thing to consider is that we have a test case for it, we don't want
> the removal to cause regression, hence blocking the pushgate.
> 
I'm writing a patch to remove the testcase (actually, to switch it to
RTDs) right now :-)

Regards,
Dario

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Plans for the SEDF scheduler

[Ian Campbell]

On Thu, 2015-05-21 at 13:45 +0100, George Dunlap wrote:
[...]
> >  2. keep the field 'physically', but return warnings and errors when it 
> >     is used. This is a change in behavior, so also not really stable
> >     (but, hey, the scheduler is going away, the behavior will change,
> >     one way or another!)
> 
> I think this is probably the best option.

Agreed.

[...]
> I think given that it's been "going away" for 9 years, we should
> probably just remove it

Agreed.

>  and put it back if people complain.  (And if
> they really really can't use RTDS instead.)

Nah ;-)

Ian.

Re: Plans for the SEDF scheduler

[Meng Xu]

[Add Chong and Dagaen who is working on improving RTDS scheduler now.]
> I'm less sure of libxl, because of the API stability claims. It looks
> like this is all about (as there are not scheduler specific API
> functions, which is good) this:
>
>  # Consistent with values defined in domctl.h
>  # Except unknown which we have made up
>  libxl_scheduler = Enumeration("scheduler", [
>      (0, "unknown"),
>      (4, "sedf"),
>      (5, "credit"),
>      (6, "credit2"),
>      (7, "arinc653"),
>      (8, "rtds"),
>      ])
>
> The solutions I see are:
>  1. get rid of the field. This does not sound ideal from an API
>     stability point of view;
>  2. keep the field 'physically', but return warnings and errors when it
>     is used. This is a change in behavior, so also not really stable
>     (but, hey, the scheduler is going away, the behavior will change,
>     one way or another!)
>  3. RTDS has a really similar interface and, to a certain extent, a
>     similar behavior too. Would it make sense to, upon suitable
>     warnings, of course, convert the requests within libxl, i.e.,
>     when someone asks for SEDF, we give him RTDS?
>
> Honestly, I'd prefer 2, and I don't much like 3. However, it really
> seems that 3 gives us a way to keep most of our promises about API
> stability... Thoughts?

I think option 2) is better than the other two choices. The SEDF does
not use a strictly EDF scheduling policy as RTDS does, so the
scheduling sequence of VCPUs won't be same even when we tuned RTDS to
single-core EDF scheduler. This implicit transformation may give users
more difficulty to figure out how to use the real time scheduler in
Xen.

Thanks,

Meng


-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania
http://www.cis.upenn.edu/~mengxu/