[PATCH V2 0/1] Improved RTDS scheduler

[Tianyang Chen <tiche@seas.upenn.edu>]

Changes since V1:
    Removed redundant cover letter
    Removed redundant patch file that was added with the last commit
    Please dis-regard V1 because it was in wrong format. Sorry about that.

Current RTDS scheduler is time driven and is called every 1ms. During each scheduler call, the repl_update() scans both runq and depeletedq, which might not be necessary every 1ms.

Since each vcpu is implemented as a deferable server, budget is preserved during its period and refilled in the next. It is not necessary to check every 1ms as the current design does. The replenishment is needed at the nearest next period(nearest current_deadline) of all runnable vcpus.

This improved design tries to reduce scheduler invocation by using an event driven approach;rt_schedule() will return a value when the scheduler needs to be called next time. In addition, the sched_rt will have one dedicated timer to handle replenishment when necessary. In other words, the budget replenishment and scheduler decision(rt_schedule) are separated.

Based on previous decision between Dario, Dagaen and Meng, the improved design can be implemented/modified as follows:

rt_schedule(): picks the highest runnable vcpu based on cpu affinity and ret.time will be passed to schedule().

rt_vcpu_wake(): when a vcpu is awake, it tickles instead of picking one from runq.

rt_context_saved(): when context switching is finished, the preempted vcpu will be put back into the runq. Picking from runq and tickling are removed.

repl_handler(): a timer handler which is reprogrammed to fire at the nearest vcpu deadline to replenish vcpus on runq and depeletedq. When the replenishment is done, each replenished vcpu in the runq should tickle a pcpu to see if it needs to preempt any running vcpus.


schedule.c SCHEDULE_SOFTIRQ:
    rt_schedule():
        [spin_lock]
        burn_budget(scurr)
        snext = runq_pick()
        [spin_unlock]


sched_rt.c TIMER_SOFTIRQ
    replenishment_timer_handler()
        [spin_lock]
        <for_each_depleted_vcpu(i, i.repl_time < NOW()) {
            replenish(i)
            runq_tickle(i)
        }>
        program_timer()
        [spin_lock]

The transient behavior should be noted. It happens between a vcpu tickles and a pcpu actually picks it. As previous discussions, this is unavoidable.

Previous discussions:
http://lists.xenproject.org/archives/html/xen-devel/2015-06/msg02629.html

Signed-off-by: Tianyang Chen <tiche@seas.upenn.edu>
Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>
Signed-off-by: Dagaen Golomb <dgolomb@seas.upenn.edu>


Tianyang Chen (1):
  Improved RTDS scheduler

 xen/common/sched_rt.c |  159 +++++++++++++++++++++++++++++--------------------
 1 file changed, 95 insertions(+), 64 deletions(-)

-- 
1.7.9.5