Re: Scheduler problem in XEN 3.4.0

[George Dunlap <george.dunlap@eu.citrix.com>]

Pankaj Parakh wrote:
> Hi George,
>
> Thanks for showing interest, First of all I'll try to setup debugging
> environment for Xen, I also checked the mailing list of Ananth also I
> know him personally, he has left it some where in middle.
>
> I tried printk's and locking method to find out which function is
> being called in schedule.c, I found the following sequence:-
>
> schedule_init()
> schedule_domain_init()
> schedule_vcpu_init()
> schedule_domain_init()
> schedule_vcpu_init()
> (from here no function is called from schedule.c, but system hangs)
>
> I cant say what the problem is, I am only changing the sched_priv data
> of vcpu struct, also my rr sched has only three functions viz
> init_vcpu, destroy_vcpu, do_schedule.
>
> I have attached the code as well.
>
> Following are my doubts:
>   
[Language point: In many English dialects, such as US and UK, "doubt" 
implies something negative.  To avoid being misunderstood by speakers of 
those dialects, "questions" might be a better word to use.]
> 1. Is the above function sequence is right ? Why two times init_domain
> is called during the boot.
>   
I suspect that the idle domain (32727) and domain 0 are the two domains 
being initialized.  You could easily find out by adding the domain id to 
the printk.
> 2. Do a scheduler policy need to manipulate other part of vcpu struct
> (other than sched_priv)
>   
I don't believe so.  You could look through other schedulers (like the 
credit scheduler) to see if this is so.
> 3. Is it necessary to maintain domain information inside scheduler policy
>   
I don't know.

A quick scan through the attached file turn up a couple of points:
+ It appears that the algorithm adds *all* vcpus to a single global 
runqueue, and scans through them looking for "runnable" vcpus on 
schedule.  This seems pretty pointless: why not add them to the global 
runqueue on wake?
+ Furthermore, I'm not sure your linked-list implementation is sound; 
for example, the initial v->sched_priv is not initialized to NULL. (Not 
going to spend a lot of time trying to figure out if that's OK or not.)
+ There is some missing logic regarding the v->processor field and 
sync_vcpu_execstate().  Xen is designed expecting a per-cpu runqueue 
with explicit migrations of vcpus between pcpus.  One of the reasons for 
this is so that when switching between a vcpu and the idle domain, it 
doesn't actually need to do a full context switch.  As a result, if you 
run a vcpu on one cpu, and then run it on another without an explicit 
sync, you may have stale data in the vcpu struct (and Xen will throw a BUG).
+ You don't implement a .wake() method.  I think that Xen will wake() 
dom0 when it's ready, expecting the .wake() method to raise a SCHEDULE 
softirq on the appropriate pcpu (from which the .do_schedule() method is 
called).  So no wake method means no schedule(), and no schedule means 
it just runs the idle domain.

It might be best to start with just one pcpu, and adding multiple cpus 
once you get things running.  Try adding a wake() method that will check 
to see if cpu 0 is idle; if it is, raise SCHEDULE_SOFTIRQ on that cpu, 
and see what you get.  After that, try adding some logic to figure out 
which other cpu to wake instead; but be advised that you'll probably hit 
the BUG() relating to sync_vcpu_execstate() not being properly called.

 -George
> On Tue, Oct 13, 2009 at 5:17 PM, George Dunlap
> <George.Dunlap@eu.citrix.com> wrote:
>   
>> Pankaj,
>>
>> I haven't used the round-robin scheduler code in that book, but
>> another guy named Ananth tried to use it unsuccessfully as well.  You
>> can see some of that thread here (not sure why I can't find the
>> original post):
>>  http://lists.xensource.com/archives/html/xen-devel/2009-05/msg00004.html
>>
>> Most of us are not so interested in finding the bug in the books'
>> code, but we are interested in helping *you* find it.  If you continue
>> to do hypervisor work (and especially if you do anything with the
>> scheduler), you're going to have to learn how to debug a hypervisor,
>> which is often rather a pain in the neck.
>>
>> There is some advice in the thread linked to above about setting up a
>> serial console.  You can add printk()'s around to find out what the
>> sched_rr code is doing and where it's going wrong, and ask more
>> questions on the list if you get stuck.  (Feel free to cc me to bring
>> it to my attention, but always send it to the whole list.)
>>
>> Good luck,
>>  -George
>>
>>
>> On Mon, Oct 12, 2009 at 6:15 PM, Pankaj Parakh
>> <me.pankajparakh@gmail.com> wrote:
>>     
>>> Hi All,
>>>
>>> I am trying to learn about schedulers of XEN, so for a start I am
>>> using XEN 3.4.0 and using book - The Definitive Guide to the Xen
>>> Hypervisor - by David Chisnall, I have followed its steps to add
>>> scheduler which is there in Chap. 12
>>> But I dont know what is the problem, I am unable to boot with that
>>> scheduler selected, I have been trying to debug this problem, but
>>> kinda stuck in it.
>>> The scheduler given in that book is a trivial round robin scheduler.
>>>
>>> Is the problem is with the code or with the procedure, I dont know that also.
>>>
>>> Plzz help me out of this.
>>>
>>> Thanks
>>> Pankaj Parakh
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>>>
>>>       
>
>
>
> --
> Pankaj Parakh
>