patch for virtual machine oriented scheduling(1)

patch for virtual machine oriented scheduling(1)

[alex]

the following patchs provide an extra control(besides the control of
Linux scheduler) over the execution of vcpu threads.

In this patch, Xen's credit
scheduler(http://wiki.xensource.com/xenwiki/CreditScheduler) is used.
User can use  "cat" and
"echo" command to view and control a guest os' credit.
e.g.,
[root@localhost ~]#  echo "weight=500" > /proc/kvm/12345
will change the credit of guest whose qemu process has the pid 12345 to be 500.

The patch consists of 3 parts:
1. modification to the standard KVM
2. modification to the Xen scheduler
3. helper functions

However, some are unnecessary in the latest Linux kernel.

The difficulties in the ports lie in:
1. Linux does not provide timer mechanism that the timer function is
bind to a dedicate CPU:
    in case of one cpu receives another cpu's schedule timer
expiration, IPI is used to relay it.
2. before linux 2.6.27, the smp_call_function_xxx()  can not be re-entered.
   if kvm is sending ipi at the time of relaying timer expiration
information, deadlock would occur in kernel versions below 2.6.27

In my implementation, tasklets are used to run the function of
scheduling, and  kernel thread is used to send IPI(in kernels above
2.6.27, this is unnecessary)

Originally, this code is developed at the release version of KVM-83.
In order to post it, I ported to the latest .git tree.  As a result,
modifications to files like external-module-compat-comm.h are omited.

NOTE:
1. Because my not having an AMD machine, only intel platforms are tested.
2. Because sched_setaffinity() is used (while Linux does not export
this symbol), the way of loading kvm modules are changed to be
./myins <directory that holds the kvm.ko and kvm-intel.ko>

Re: patch for virtual machine oriented scheduling(1)

[Andrew Theurer]

alex wrote:
> the following patchs provide an extra control(besides the control of
> Linux scheduler) over the execution of vcpu threads.
>
> In this patch, Xen's credit
> scheduler(http://wiki.xensource.com/xenwiki/CreditScheduler) is used.
> User can use  "cat" and
> "echo" command to view and control a guest os' credit.
> e.g.,
> [root@localhost ~]#  echo "weight=500" > /proc/kvm/12345
> will change the credit of guest whose qemu process has the pid 12345 to be 500.
>
> The patch consists of 3 parts:
> 1. modification to the standard KVM
> 2. modification to the Xen scheduler
> 3. helper functions
>   
Just wondering, was it not possible to introduce a new scheduling class 
in the current scheduler?  My impression was that the current scheduler 
was fairly modular and should allow this.

-Andrew


> However, some are unnecessary in the latest Linux kernel.
>
> The difficulties in the ports lie in:
> 1. Linux does not provide timer mechanism that the timer function is
> bind to a dedicate CPU:
>     in case of one cpu receives another cpu's schedule timer
> expiration, IPI is used to relay it.
> 2. before linux 2.6.27, the smp_call_function_xxx()  can not be re-entered.
>    if kvm is sending ipi at the time of relaying timer expiration
> information, deadlock would occur in kernel versions below 2.6.27
>
> In my implementation, tasklets are used to run the function of
> scheduling, and  kernel thread is used to send IPI(in kernels above
> 2.6.27, this is unnecessary)
>
> Originally, this code is developed at the release version of KVM-83.
> In order to post it, I ported to the latest .git tree.  As a result,
> modifications to files like external-module-compat-comm.h are omited.
>
> NOTE:
> 1. Because my not having an AMD machine, only intel platforms are tested.
> 2. Because sched_setaffinity() is used (while Linux does not export
> this symbol), the way of loading kvm modules are changed to be
> ./myins <directory that holds the kvm.ko and kvm-intel.ko>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>   

Re: patch for virtual machine oriented scheduling(1)

[alex]

On Fri, Apr 24, 2009 at 12:18 AM, Andrew Theurer
<habanero@linux.vnet.ibm.com> wrote:
>
> Just wondering, was it not possible to introduce a new scheduling class in
> the current scheduler?  My impression was that the current scheduler was
> fairly modular and should allow this.
>
> -Andrew
The reasons are:
1. I am afraid that modifications to the Linux scheduler might cause
unexpected effects, because it is a global one.
2. an extra execution control solution like mine is the simplest one
to implement.
3. As far as I can see, the benefit of replacing Linux scheduler only
lies that as soon as a scheduling decision is made, it can be put into
practice immediately. I think this benefit is neglectable.

Re: patch for virtual machine oriented scheduling(1)

[Avi Kivity]

alex wrote:
>> Just wondering, was it not possible to introduce a new scheduling class in
>> the current scheduler?  My impression was that the current scheduler was
>> fairly modular and should allow this.
>>
>> -Andrew
>>     
> The reasons are:
> 1. I am afraid that modifications to the Linux scheduler might cause
> unexpected effects, because it is a global one.
>   

True, extra care is needed.

> 2. an extra execution control solution like mine is the simplest one
> to implement.
>   

But it results in an overall more complex system.

> 3. As far as I can see, the benefit of replacing Linux scheduler only
> lies that as soon as a scheduling decision is made, it can be put into
> practice immediately. I think this benefit is neglectable.
>   

It also make maintenance easier, and brings the benefits to ordinary 
(non-kvm) processes.

-- 
error compiling committee.c: too many arguments to function

Re: patch for virtual machine oriented scheduling(1)

[alex]

Hi,

Sorry for that something is wrong with my code posted earlier.

Just now, I found a bug in it, which was introduced by my porting my
work(which is developed under KVM83 and using svn depository) to KVM's
git depository by hand.  The bug is : you can not shutdown SMP guest
by killing the related qemu process.

I am examing my patch code to see the reason.

BTW:  counting the host mode time is quite easy. Hooking the virtual
machine runstate change to the time sched_in/sched_out is called will
satify Avi Kivity's requirement.