[BUG] XEN-4.8-rc2 sched-rtds does not accept settings

[BUG] XEN-4.8-rc2 sched-rtds does not accept settings

[Juergen Schinker]

Hey Meng XU 


xl -vf sched-rtds -v all -d 0 -p 20000 -b 8000

whatever I set - the changes are not accepted and stay like you see further down

and the system feels very lame

Cpupool Pool-0: sched=RTDS
Name                                ID    Period    Budget
Domain-0                             0     10000      4000
zimbra                               1     10000      4000
ethereum-os                          2     10000      4000
mirot                                3     10000      4000
wily                                 4     10000      4000
 
host                   : xen
release                : 4.7.0-1-amd64
version                : #1 SMP Debian 4.7.6-1 (2016-10-07)
machine                : x86_64
nr_cpus                : 4
max_cpu_id             : 3
nr_nodes               : 1
cores_per_socket       : 2
threads_per_core       : 2
cpu_mhz                : 2312
hw_caps                : b7ebfbff:77bae3ff:28100800:00000001:00000001:00000281:00000000:00000100
virt_caps              : hvm hvm_directio
total_memory           : 32711
free_memory            : 0
sharing_freed_memory   : 0
sharing_used_memory    : 0
outstanding_claims     : 0
free_cpus              : 0
xen_major              : 4
xen_minor              : 8
xen_extra              : .0-rc
xen_version            : 4.8.0-rc
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 
xen_scheduler          : rtds
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : Mon Oct 10 11:10:56 2016 -0700 git:68dc718
xen_commandline        : placeholder sched=rtds iommu=1
cc_compiler            : gcc (Debian 6.2.0-6) 6.2.0 20161010
cc_compile_by          : root
cc_compile_domain      : 
cc_compile_date        : Sun Oct 16 23:56:56 BST 2016
build_id               : 847a9dd70ab5f581818fa9cc09ed609ce2b420d1
xend_config_format     : 4

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [BUG] XEN-4.8-rc2 sched-rtds does not accept settings

[Wei Liu]

CC Dario and Meng

On Mon, Oct 17, 2016 at 10:55:21PM +0100, Juergen Schinker wrote:
> Hey Meng XU 
> 
> 
> xl -vf sched-rtds -v all -d 0 -p 20000 -b 8000
> 
> whatever I set - the changes are not accepted and stay like you see further down
> 
> and the system feels very lame
> 
> Cpupool Pool-0: sched=RTDS
> Name                                ID    Period    Budget
> Domain-0                             0     10000      4000
> zimbra                               1     10000      4000
> ethereum-os                          2     10000      4000
> mirot                                3     10000      4000
> wily                                 4     10000      4000
>  
> host                   : xen
> release                : 4.7.0-1-amd64
> version                : #1 SMP Debian 4.7.6-1 (2016-10-07)
> machine                : x86_64
> nr_cpus                : 4
> max_cpu_id             : 3
> nr_nodes               : 1
> cores_per_socket       : 2
> threads_per_core       : 2
> cpu_mhz                : 2312
> hw_caps                : b7ebfbff:77bae3ff:28100800:00000001:00000001:00000281:00000000:00000100
> virt_caps              : hvm hvm_directio
> total_memory           : 32711
> free_memory            : 0
> sharing_freed_memory   : 0
> sharing_used_memory    : 0
> outstanding_claims     : 0
> free_cpus              : 0
> xen_major              : 4
> xen_minor              : 8
> xen_extra              : .0-rc
> xen_version            : 4.8.0-rc
> xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 
> xen_scheduler          : rtds
> xen_pagesize           : 4096
> platform_params        : virt_start=0xffff800000000000
> xen_changeset          : Mon Oct 10 11:10:56 2016 -0700 git:68dc718
> xen_commandline        : placeholder sched=rtds iommu=1
> cc_compiler            : gcc (Debian 6.2.0-6) 6.2.0 20161010
> cc_compile_by          : root
> cc_compile_domain      : 
> cc_compile_date        : Sun Oct 16 23:56:56 BST 2016
> build_id               : 847a9dd70ab5f581818fa9cc09ed609ce2b420d1
> xend_config_format     : 4
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [BUG] XEN-4.8-rc2 sched-rtds does not accept settings

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 2472 bytes --]

On Tue, 2016-10-18 at 11:51 +0100, Wei Liu wrote:
> CC Dario and Meng
> 
Hi,

Thanks Wei for the Cc...

> On Mon, Oct 17, 2016 at 10:55:21PM +0100, Juergen Schinker wrote:
> > 
> > xl -vf sched-rtds -v all -d 0 -p 20000 -b 8000
> > 
> > whatever I set - the changes are not accepted and stay like you see
> > further down
> > 
> > and the system feels very lame
> > 
> > Cpupool Pool-0: sched=RTDS
> > Name                                ID    Period    Budget
> > Domain-0                             0     10000      4000
> > zimbra                               1     10000      4000
> > ethereum-os                          2     10000      4000
> > mirot                                3     10000      4000
> > wily                                 4     10000      4000
> > 
Right, but what does the following shows:

 xl sched-rtds -d 0 -v all

?

On my system, just using `xl sched-rtds` as above also still shows the
4000/10000 budget over period pairs, but adding '-v all' shows the
correct (i.e., the new) parameters.

In fact, the two commands are basically showing different things. I
agree it's not really clear, and I'll think to ways of improving the
situation (e.g., by improving the documentation, but also by making the
`xl' interface more consistent).

About the 'system feels lame', what does that exactly mean? Actually,
what is it that you are trying to achieve by using RTDS?

RTDS is a special purpose scheduler, and should be chosen for the
proper reasons and configured accurately. Typical use case is need for
deterministic/real-time scheduling of VMs.

Oh, and also, default parameters are 4000/10000=40% utilization. By
setting them to 8000/20000=40%, you're not changing much, at least
nothing that will be noticeable as a 'general system feeling', because
all domains (well, each vCPU of each domain), including dom0 (dom0's
vCPUs), are still limited to use at most 40% of pCPU time --although
with different periodicity.

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [BUG] XEN-4.8-rc2 sched-rtds does not accept settings

[Meng Xu]

On Tue, Oct 18, 2016 at 6:51 AM, Wei Liu <wei.liu2@citrix.com> wrote:
> CC Dario and Meng

Thank you very much for cc.ing me, Wei! :-)

>
> On Mon, Oct 17, 2016 at 10:55:21PM +0100, Juergen Schinker wrote:
>> Hey Meng XU

Hi,

Thanks for your interest in RTDS scheduler! :-)

>>
>>
>> xl -vf sched-rtds -v all -d 0 -p 20000 -b 8000
>>
>> whatever I set - the changes are not accepted and stay like you see further down
>>
>> and the system feels very lame
>>
>> Cpupool Pool-0: sched=RTDS
>> Name                                ID    Period    Budget
>> Domain-0                             0     10000      4000
>> zimbra                               1     10000      4000
>> ethereum-os                          2     10000      4000
>> mirot                                3     10000      4000
>> wily                                 4     10000      4000

I guess you used xl sched-rtds to show the paramters.
The "xl sched-rtds" is a legacy command. It is supposed to show the
period and budget of *all* VCPUs of the same domain.  In Xen 4.5, RTDS
scheduler only allows setting same period and budget for *all* VCPUs
of the same domain.
In Xen 4.7, RTDS scheduler allows setting individual period and budget
for each VCPU of the same domain. Then "xl sched-rtds" won't make
sense because VCPUs have different periods and budgets.

The short story is:
If you use "xl sched-rtds -v all", you will see the parameters are changed.

Another thing is that the current RTDS scheduler will only allocate
the amount of resource to the VCPU you set.  Since you set period and
budget to be 20000 and 8000, you only get 40% CPU resource. (We won't
get more resource than you set under the current RTDS.)

We are working on another improved version of the RTDS, which could
let a VCPU get more resource that specified when there is idle
resource left in the system.
If you could tell us what you want to use RTDS to achieve, we could
take that into consideration when we improve the RTDS further.

Thanks,

Meng

-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania
http://www.cis.upenn.edu/~mengxu/

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [BUG] XEN-4.8-rc2 sched-rtds does not accept settings

[Juergen Schinker]

----- On 18 Oct, 2016, at 17:41, Meng Xu xumengpanda@gmail.com wrote:

> On Tue, Oct 18, 2016 at 6:51 AM, Wei Liu <wei.liu2@citrix.com> wrote:
>> CC Dario and Meng
> 
> Thank you very much for cc.ing me, Wei! :-)
> 

why are you not on the list?

 
> The short story is:
> If you use "xl sched-rtds -v all", you will see the parameters are changed.
> 
> Another thing is that the current RTDS scheduler will only allocate
> the amount of resource to the VCPU you set.  Since you set period and
> budget to be 20000 and 8000, you only get 40% CPU resource. (We won't
> get more resource than you set under the current RTDS.)
> 
what is the min max?

> We are working on another improved version of the RTDS, which could
> let a VCPU get more resource that specified when there is idle
> resource left in the system.
> If you could tell us what you want to use RTDS to achieve, we could
> take that into consideration when we improve the RTDS further.
> 
 
well I just wanted to test it and see how it performs and don't have a special need for this scheduler

except shutting up loudmouthes who say you can not assure min  resources to a domain :-)

btw i switched to oxenstored

Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [BUG] XEN-4.8-rc2 sched-rtds does not accept settings

[Wei Liu]

On Tue, Oct 18, 2016 at 07:52:29PM +0100, Juergen Schinker wrote:
> 
> 
> ----- On 18 Oct, 2016, at 17:41, Meng Xu xumengpanda@gmail.com wrote:
> 
> > On Tue, Oct 18, 2016 at 6:51 AM, Wei Liu <wei.liu2@citrix.com> wrote:
> >> CC Dario and Meng
> > 
> > Thank you very much for cc.ing me, Wei! :-)
> > 
> 
> why are you not on the list?
> 

You only sent to xen-devel@lists.xen.org, which is a public list.  Most
if not all developers subscribe to that list and have specific filter
for it to avoid getting overwhelmed by the traffic.

When you CC people directly, maybe they will have another filter to
deliver the email to their inbox. Your email will have a better chance
to be seen by relevant developers.

But keep in mind that when CC'ing people please keep the list of people
as relevant as possible.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [BUG] XEN-4.8-rc2 sched-rtds does not accept settings

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 2157 bytes --]

On Tue, 2016-10-18 at 19:52 +0100, Juergen Schinker wrote:
> > On Tue, Oct 18, 2016 at 6:51 AM, Wei Liu <wei.liu2@citrix.com>
> > wrote:
> > > 
> > Another thing is that the current RTDS scheduler will only allocate
> > the amount of resource to the VCPU you set.  Since you set period
> > and
> > budget to be 20000 and 8000, you only get 40% CPU resource. (We
> > won't
> > get more resource than you set under the current RTDS.)
> > 
> what is the min max?
> 
Not sure what you mean, but I think the answer is that it's a max.

I.e., a vcpu will never get more than 8ms every 20ms, which means it
will never get more than 40% CPU time. But if it asks less than that,
it will (of course!) get less).

It can also be seen as a min, but that depends on the system
configuration. So far, there is nothing that prevents you to assign 40%
of CPU time to 10 vCPUs. This means you've allocated 400% CPUs. If you
have, say, only 2 CPUs, you're overbooking, and there's no way everyone
will get all of what they ask for.

How graceful the performance degradation is, I'll leave it to Meng (I
recall EDF does that in a very nice manner, but I can't remember
whether that is the case in the actual algorithm RTDS is based on).

But again, I'm not sure whether this is what you were asking.
 
> well I just wanted to test it and see how it performs and don't have
> a special need for this scheduler
> 
Ah, ok. When doing such experiments, consider using cpupools. They come
in very handy, as you can dynamically assign a scheduler to a set of
CPUs, move domains there, and test and check whatever you want, without
having to reboot with different parameters, etc.

> except shutting up loudmouthes who say you can not assure
> min  resources to a domain :-)
> 
Right, this is a very good purpose... Let us know if you need help on
it! :-P

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel