Re: Credit scheduler

[Mats Petersson <mats@planetcatfish.com>]

At 21:46 12/07/2007, pak333@comcast.net wrote:
>Hi Atsushi
>
>I am still confused, so let me explain what i think should happen 
>and understand from you why it may not happen.
>
>I have 2 cpu intensive VMs and 1 IO intensive VM. My system has 4 
>physical cpus and 8 virtual cpus.
>
>Now using default parameters for the credit scheduler, the cpu 
>intersive vcpu should run for 30 millsec. It does not, it runs for 
>less (few microsecs).
>
>You said this could be because of softirq which are raised after a 
>system call. Can i disable the softirqs. What happens if i do? If i 
>cannot disable is there a way to see what VM is raising the softirq. 
>If the cpu-VM raises the softirq it gets preempted or does it 
>continue to run. How can I montior the softirqs raised by this VM. 
>Can I tell if the premption is from a softirq or from something else.

I don't believe you can "disable" a softirq - you could of course 
avoid doing the call to "do_softirq", but that is most likely just 
going to completely remove any scheduling capability at all in the 
system, making the system "single-tasking". Not a recommended way, in 
my opinion.

>
>Also, if the IO VM requests an IO, it will block and dom0 wakes up 
>and gets scheduled to run to service the IO. Will it preempt the cpu 
>intensive VM. If so why? Shouldn't the cpuintensiveV M get its 
>quantum of time.
>Or does the IOVM get higher priority to preempt the cpu intensive 
>VM. How does the scheduler  pick which cpu to run dom0 on ( if all 
>the vcpus running are cpu intensive)? If there is a mix of cpu-vcpus 
>and Io-vcpus, which will be the victim


The whole idea of having separate queues for IO and CPU intensive VMs 
(or processes in a normal kernel scenario) is to allow the 
IO-intensive tasks to avoid waiting for the next time-slot when a 
CPU-hogging VM/process is running  [1]. The normal behaviour for the 
scheduler is to determine dynamically if the current VM/process is IO 
or CPU intensive.

Unless Dom0 is used for doing some silly CPU-intensive task 
(calculating PI with a million decimal points or compiling Xen + 
Linux kernel, for example), it will most likely look to the scheduler 
like a IO-intensive VM. So it will have the same priority as any 
other IO-intensive VM (assuming Dom0 has equal scheduler parameters 
as other guests, which I'm pretty sure is the default behaviour).

If, like in your example, there are a number of processors busy with 
CPU-intensive tasks, and others with IO-intensive tasks, the most 
likely scenario is that any new IO-request will go be run on a 
"previously CPU-intensive" CPU - but on the other hand, if you have a 
lot of IO-intensive processing, there should be some processor(s) 
that are "asleep" - unless all CPU's are busy running CPU-intensive tasks...

Note that scheduling is a complex subject, and there have been dozens 
of PhD dissertations written on the subject.

[1] The idea being that if we process the IO-request that just 
completed, we can set up another IO-request, and this will get better 
IO-throughput than waiting a long time (relatively speaking 30ms is 
an eternity to the processor).

--
Mats
>
>As you see i have a lot of questions, so please bear with me and 
>help me understand the scheduling behavior of Xen
>
>Thanks
>Prabha
>
>
>
>
>-------------- Original message --------------
>From: Atsushi SAKAI <sakaia@jp.fujitsu.com>
>
> > Hi Prabha
> >
> > Please check do_softirq in assembler.
> > for x86
> > call do_softirq
> > is in.
> >
> > And usually do_softirq is executed at the end of system call.
> >
> > Thanks
> > Atsushi SAKAI
> >
> >
> >
> > pak333@comcast.net wrote:
> >
> > > Thanks for the explanation, i am not sure i understood it correctly.
> > >
> > >
> > > when does softirq get set. So when softirq is set a processor enters the
> > scheduler every 3 micro secs? where in the source code is this 
> handled. please
> > could you send me some pointers
> > >
> > > thanks
> > > -Prabha
> > >
> > >
> > >
> > >
> > > -------------- Or iginal message --------------
> > > From: Atsushi SAKAI
> > >
> > > > Hi,$B!!(BPrabha
> > > >
> > > > $B!!(B30msec is the maximum time slice.
> > > > And I guess your 3 microsecond is the response after
> > > > SCHEDULE_SOFTIRQ is set.
> > > > As you know,
> > > > If SCHEDULE_SOFTIRQ is set,
> > > > it waits softirq calls schedule(). (this time interval is 3microsec)
> > > >
> > > > And I/O intensive has higher priority than CPU intensive.
> > > > So I/O intensive job is first dispatched domain in runq.
> > > > This is because latency improvement for I/O intensive guest.
> > > >
> > > > So your behavior is not strange.
> > > >
> > > > Thanks
> > > > Atsushi SAKAI
> > > >
> > > >
> > > > pak333@comcast.net wrote:
> > &g
>t; >
> > > > > Hi
> > > > >
> > > > > I have noticed that VMs are typically spending 3 microsecs 
> or less before
> > they
> > > > are being prempted. I thought that the credit schduler time 
> slice was 30 ms.
> > I
> > > > have 4 VMs running and they are all cpu intensive except for 
> 1 (which is IO
> > > > intensive) but having a VM spend max 3 micro secs before 
> being kicked out
> > seems
> > > > strange.
> > > > >
> > > > > Is there something else going on that i am not aware of. Is 
> the time slice
> > > > really 30 millisecs? I am using default parameters of the 
> credit scheduler.
> > > > >
> > > > > Thanks
> > > > > -Prabha
> > > >
> > > >
> >
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen source.com
> > http://lists.xensource.com/xen-devel
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel