Scheduling

Scheduling

[David_Wolinsky]

[-- Attachment #1.1: Type: text/plain, Size: 465 bytes --]

Is scheduling broken in xen-unstable?

Here's what I did...

Added sched=rrobin to my kernel and started xen
Ran xm rrobin, unknown command
Ran xm help rrobin, unkown command

So round robin is throw out

So I tried the default bvt...
Without appending it to my kernel, I ran...
xm bvt_ctxallow 1 - Error:  Internal server error

With it appended to my kernel, I run..
xm bvt_ctxallow 1 - Computer hard crashes

Any suggestions?

Thanks,
David

[-- Attachment #1.2: Type: text/html, Size: 1370 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: Scheduling

[Ian Pratt]

 
> Added sched=rrobin to my kernel and started xen Ran xm 
> rrobin, unknown command Ran xm help rrobin, unkown command 

sched=rrobin is not in unstable anymore. 

The default is the SEDF scheduler, but you can still set sched=bvt

As I recall, there is some documentation on SEDF in tools/misc

We should file a bug that using a scheduler op on a non existent
scheduler does bad things.
 
Ian

> So round robin is throw out 
> 
> So I tried the default bvt...
> Without appending it to my kernel, I ran... xm bvt_ctxallow 1 - 
> Error:  Internal server error 
> 
> With it appended to my kernel, I run.. 
> xm bvt_ctxallow 1 - Computer hard crashes 
> 
> Any suggestions? 
> 
> Thanks,
> David 
> 
> 

RE: Scheduling

[David_Wolinsky]

Submitted bug to bugzilla.

I am now changing my scheduling and have noticed some different results,
perhaps you could help me in my studies....

I am running CPU intensive VMs and am trying to find out at what
scheduling they'll run the best (since the simulation being run over a
long duration, short periods and slices are not important).  I have 8
VMs so my first few tests were running in periods of 1, 2, and 10
seconds, divided among the 8 domUs... 

So I typed
Xm sedf (1,8) 1,2,10e9 125,250,1250e6 0 0 0
(ie, xm sedf 1 1000000000 125000000 0 0 0)

In this case, I excluded scheduling for dom0...

Could you please help me refine my scheduling.

Thanks,
David

-----Original Message-----
From: Ian Pratt [mailto:m+Ian.Pratt@cl.cam.ac.uk] 
Sent: Monday, July 25, 2005 11:50 AM
To: Wolinsky, David; xen-devel@lists.xensource.com
Cc: ian.pratt@cl.cam.ac.uk
Subject: RE: [Xen-devel] Scheduling

 
> Added sched=rrobin to my kernel and started xen Ran xm rrobin, unknown

> command Ran xm help rrobin, unkown command

sched=rrobin is not in unstable anymore. 

The default is the SEDF scheduler, but you can still set sched=bvt

As I recall, there is some documentation on SEDF in tools/misc

We should file a bug that using a scheduler op on a non existent
scheduler does bad things.
 
Ian

> So round robin is throw out
> 
> So I tried the default bvt...
> Without appending it to my kernel, I ran... xm bvt_ctxallow 1 -
> Error:  Internal server error
> 
> With it appended to my kernel, I run.. 
> xm bvt_ctxallow 1 - Computer hard crashes
> 
> Any suggestions? 
> 
> Thanks,
> David
> 
> 

Re: Scheduling

[Stephan Diestelhorst]

>I am now changing my scheduling and have noticed some different results,

>perhaps you could help me in my studies....
>
>I am running CPU intensive VMs and am trying to find out at what
>scheduling they'll run the best (since the simulation being run over a
>long duration, short periods and slices are not important).  I have 8
>VMs so my first few tests were running in periods of 1, 2, and 10
>seconds, divided among the 8 domUs... 
>  
>
>So I typed
>Xm sedf (1,8) 1,2,10e9 125,250,1250e6 0 0 0
>(ie, xm sedf 1 1000000000 125000000 0 0 0)
>
>In this case, I excluded scheduling for dom0...
>
>Could you please help me refine my scheduling.
>
>  
>
Sure. Actually I have not thought of such long periods and slices and in
fact there are parts in the code that limit the slice and period lengths
to roughly 4 seconds (due to arithmetic overflow), as I guessed that
those long periods would be quite exotic. In fact I think you might be
better of to use the scheduler in extra-time mode, that means you don't
guarantee time to the domains, but rather split the remaining (i.e. when
all realtime domains have finished) time into pieces of various sizes.
I guess this might be more appropriate to you, I assume that your long
running simulation is not a real-time application?

So you might just try to do
xm sedf (1,8) 0 0 0 1 w
With w specifying a weight for the domain. This works intuitively, so a
domain with weight 4 gets twice the amount of CPU time as one with
weight 2, which gets 2/5 of one with weight 5.

I hope that this suits your needs, if not, let me know and I'll change
some of the arithmetic code.
BTW: What did actually happen when you used the above command?

Stephan

>-----Original Message-----
>From: Ian Pratt [mailto:m+Ian.Pratt@cl.cam.ac.uk] 
>Sent: Monday, July 25, 2005 11:50 AM
>To: Wolinsky, David; xen-devel@lists.xensource.com
>Cc: ian.pratt@cl.cam.ac.uk
>Subject: RE: [Xen-devel] Scheduling
>
> 
>  
>
>>Added sched=rrobin to my kernel and started xen Ran xm rrobin, unknown
>>    
>>
>
>  
>
>>command Ran xm help rrobin, unkown command
>>    
>>
>
>sched=rrobin is not in unstable anymore. 
>
>The default is the SEDF scheduler, but you can still set sched=bvt
>
>As I recall, there is some documentation on SEDF in tools/misc
>
>We should file a bug that using a scheduler op on a non existent
>scheduler does bad things.
> 
>Ian
>
>  
>
>>So round robin is throw out
>>
>>So I tried the default bvt...
>>Without appending it to my kernel, I ran... xm bvt_ctxallow 1 -
>>Error:  Internal server error
>>
>>With it appended to my kernel, I run.. 
>>xm bvt_ctxallow 1 - Computer hard crashes
>>
>>Any suggestions? 
>>
>>Thanks,
>>David
>>
>>
>>    
>>
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel
>  
>

RE: Scheduling

[S. Diestelhorst]

On Jul 28 2005, David_Wolinsky@Dell.com wrote:
David,
  I'm away over the long weekend, will look into that in more detail when 
I'll be back on Tuesday.

>Yeah its not a real time application.  And while weighting is a nice,
>how does it deal with the period?  If you could point to where in the
>code the scheduler is, I would love to look through it an get a better
>idea of how it works. 

The weighting doesn't deal with periods, its parameter is the size of an 
EXTRA_QUANTUM. The interesting code is in the deschedule_extra_domain and 
do_extra_schedule (function names from my memory...). Have a look there. 
The basic principle is that each acquisition of a extra-quantum 'costs' a 
price (called score in the code) and the domain with the lowest due payment 
gets the next quantum. This price is inverse to the weight. To avoid 
overflows, rather than incrementing the own due amount of 'payment', the 
amount of other domains can be decreased. I hope this makes some sense. It 
should become more clear, when you have a look at the code. Please be aware 
that there are two extraqueues, both operate by the same principle, but the 
computation of the costs is different. You will want to have a look at the 
EXTRA_UTIL_Q, (or L1 extraq).

> My short term goal was to see if scheduling could
>create an overhead...  Here were my results...
>			Period	Slice		BOPs
>		8 VM	1 ms		125 us	6858
>..

What does BOP stand for? I use it for begin of period, but how does that 
fit into your measurement?

>With the base numbers being this....
>
>Host	32403.5
>1 VM	32057
>...

What does the second number mean?

>But I figure these numbers might help you in your task, please
>let me know if you do anything drastic to the scheduler between now and
>next Wednesday... If so I'll re-run my tests, and hold off publishing my
>analysis.

Well not to the scheduler itself, but there will be a load balancer in the 
near future, which runs outside the scheduler though.

>Also any what happens with a number > 4 sec?

Not sure, there could be some arithmetic overflows in some of the score 
estimation functions, because I multiply two time values. As they are in 
nanoseconds, 4 secs ~ 2^32 ns -> if you multiply them you will get an 
arithmetic overflow (max. result size is 64 bits). Pre right-shifting will 
improve the range, but of course will loose some precision (but who needs 
nanoseconds in this context anyways?) Result of this will be that the L0 
extraq might do some interesting stuff, although only to a limited extent, 
so it's def. not critical and hard to notice!

Cheers,
  Stephan
>-----Original Message-----
>From: Stephan Diestelhorst [mailto:sd386@cl.cam.ac.uk] 
>Sent: Tuesday, July 26, 2005 5:44 PM
>To: Wolinsky, David
>Cc: m+Ian.Pratt@cl.cam.ac.uk; xen-devel@lists.xensource.com
>Subject: Re: [Xen-devel] Scheduling
>
>>I am now changing my scheduling and have noticed some different 
>>results,
>
>>perhaps you could help me in my studies....
>>
>>I am running CPU intensive VMs and am trying to find out at what 
>>scheduling they'll run the best (since the simulation being run over a 
>>long duration, short periods and slices are not important).  I have 8 
>>VMs so my first few tests were running in periods of 1, 2, and 10 
>>seconds, divided among the 8 domUs...
>>  
>>
>>So I typed
>>Xm sedf (1,8) 1,2,10e9 125,250,1250e6 0 0 0 (ie, xm sedf 1 1000000000 
>>125000000 0 0 0)
>>
>>In this case, I excluded scheduling for dom0...
>>
>>Could you please help me refine my scheduling.
>>
>>  
>>
>Sure. Actually I have not thought of such long periods and slices and in
>fact there are parts in the code that limit the slice and period lengths
>to roughly 4 seconds (due to arithmetic overflow), as I guessed that
>those long periods would be quite exotic. In fact I think you might be
>better of to use the scheduler in extra-time mode, that means you don't
>guarantee time to the domains, but rather split the remaining (i.e. when
>all realtime domains have finished) time into pieces of various sizes.
>I guess this might be more appropriate to you, I assume that your long
>running simulation is not a real-time application?
>
>So you might just try to do
>xm sedf (1,8) 0 0 0 1 w
>With w specifying a weight for the domain. This works intuitively, so a
>domain with weight 4 gets twice the amount of CPU time as one with
>weight 2, which gets 2/5 of one with weight 5.
>
>I hope that this suits your needs, if not, let me know and I'll change
>some of the arithmetic code.
>BTW: What did actually happen when you used the above command?
>
>Stephan
>
>>-----Original Message-----
>>From: Ian Pratt [mailto:m+Ian.Pratt@cl.cam.ac.uk]
>>Sent: Monday, July 25, 2005 11:50 AM
>>To: Wolinsky, David; xen-devel@lists.xensource.com
>>Cc: ian.pratt@cl.cam.ac.uk
>>Subject: RE: [Xen-devel] Scheduling
>>
>> 
>>  
>>
>>>Added sched=rrobin to my kernel and started xen Ran xm rrobin, unknown
>>>    
>>>
>>
>>  
>>
>>>command Ran xm help rrobin, unkown command
>>>    
>>>
>>
>>sched=rrobin is not in unstable anymore. 
>>
>>The default is the SEDF scheduler, but you can still set sched=bvt
>>
>>As I recall, there is some documentation on SEDF in tools/misc
>>
>>We should file a bug that using a scheduler op on a non existent 
>>scheduler does bad things.
>> 
>>Ian
>>
>>  
>>
>>>So round robin is throw out
>>>
>>>So I tried the default bvt...
>>>Without appending it to my kernel, I ran... xm bvt_ctxallow 1 -
>>>Error:  Internal server error
>>>
>>>With it appended to my kernel, I run.. 
>>>xm bvt_ctxallow 1 - Computer hard crashes
>>>
>>>Any suggestions? 
>>>
>>>Thanks,
>>>David
>>>
>>>
>>>    
>>>
>>
>>_______________________________________________
>>Xen-devel mailing list
>>Xen-devel@lists.xensource.com
>>http://lists.xensource.com/xen-devel
>>  
>>
>

RE: Scheduling

[David_Wolinsky]

Sorry for the late response, I too myself just got in today... Just to
answer a few questions, 

BOPs stands for business operations per second
Host - real system, ie, linux 2.6.12.x running natively

Also note, I do not include dom0 as a "VM" since we're not benchmarking
its performance...
	that is, 1 VM = 1 domU + 1 dom0, ..., 8 VM = 8 domU + 1 dom0

As I am still catching up on other tasks, I will respond / look into the
code later.

Thanks,
David