[PATCH v3 0/5] Towards work-conserving RTDS

[PATCH v3 0/5] Towards work-conserving RTDS

[Meng Xu]

This series of patches make RTDS scheduler work-conserving
without breaking real-time guarantees.
VCPUs with extratime flag set can get extra time
from the unreserved system resource.
System administrators can decide which VCPUs have extratime flag set.

Example:
Set the extratime bit of all VCPUs of domain 1:
# xl sched-rtds -d 1 -v all -p 10000 -b 2000 -e 1
Each VCPU of domain 1 will be guaranteed to have 2000ms every 10000ms
(if the system is schedulable).
If there is a CPU having no work to do,
domain 1's VCPUs will be scheduled onto the CPU,
even though the VCPUs have got 2000ms in 10000ms.

Clear the extra bit of all VCPUs of domain 1:
# xl sched-rtds -d 1 -v all -p 10000 -b 2000 -e 0

Set/Clear the extratime bit of one specific VCPU of domain 1:
# xl sched-rtds -d 1 -v 1 -p 10000 -b 2000 -e 1
# xl sched-rtds -d 1 -v 1 -p 10000 -b 2000 -e 0


The original design of the work-conserving RTDS was discussed at
https://www.mail-archive.com/xen-devel@lists.xen.org/msg77150.html

The first version was discussed at
https://www.mail-archive.com/xen-devel@lists.xen.org/msg117361.html

The second version was discussed at
https://www.mail-archive.com/xen-devel@lists.xen.org/msg120618.html

The series of patch can be found at github:
https://github.com/PennPanda/RT-Xen
under the branch:
xenbits/rtds/work-conserving-v3.1

Changes from v2
Sanity check the input of -e option which can only be 0 or 1
Set -e to 1 by default if 3rd party library does not set -e option
Set vcpu extratime in sched_rtds_vcpu_get function function, which
fixes a bug in previous version.
Change EXTRATIME to Extratime in the xl output

Changes from v1
Change XEN_DOMCTL_SCHED_RTDS_extratime to XEN_DOMCTL_SCHEDRT_extra
Revise xentrace, xenalyze, and docs
Add LIBXL_HAVE_SCHED_RTDS_VCPU_EXTRA symbol in libxl.h

Changes from RFC v1
Merge changes in sched_rt.c into one patch;
Minor change in variable name and comments.

Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>

[PATCH v3 1/5] xen:rtds: towards work conserving RTDS
[PATCH v3 2/5] libxl: enable per-VCPU extratime flag for RTDS
[PATCH v3 3/5] xl: enable per-VCPU extratime flag for RTDS
[PATCH v3 4/5] xentrace: enable per-VCPU extratime flag for RTDS
[PATCH v3 5/5] docs: enable per-VCPU extratime flag for RTDS

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 1/5] xen:rtds: towards work conserving RTDS

[Meng Xu]

Make RTDS scheduler work conserving without breaking the real-time guarantees.

VCPU model:
Each real-time VCPU is extended to have an extratime flag
and a priority_level field.
When a VCPU's budget is depleted in the current period,
if it has extratime flag set,
its priority_level will increase by 1 and its budget will be refilled;
othewrise, the VCPU will be moved to the depletedq.

Scheduling policy is modified global EDF:
A VCPU v1 has higher priority than another VCPU v2 if
(i) v1 has smaller priority_leve; or
(ii) v1 has the same priority_level but has a smaller deadline

Queue management:
Run queue holds VCPUs with extratime flag set and VCPUs with
remaining budget. Run queue is sorted in increasing order of VCPUs priorities.
Depleted queue holds VCPUs which have extratime flag cleared and depleted budget.
Replenished queue is not modified.

Distribution of spare bandwidth
Spare bandwidth is distributed among all VCPUs with extratime flag set,
proportional to these VCPUs utilizations

Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>

---
Changes from v2
Explain how to distribute spare bandwidth in commit log
Minor change in has_extratime function without functionality change.

Changes from v1
Change XEN_DOMCTL_SCHED_RTDS_extratime to XEN_DOMCTL_SCHEDRT_extra as
suggested by Dario

Changes from RFC v1
Rewording comments and commit message
Remove is_work_conserving field from rt_vcpu structure
Use one bit in VCPU's flag to indicate if a VCPU will have extra time
Correct comments style
---
 xen/common/sched_rt.c       | 90 ++++++++++++++++++++++++++++++++++++++-------
 xen/include/public/domctl.h |  4 ++
 2 files changed, 80 insertions(+), 14 deletions(-)

diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
index 5c51cd9..b770287 100644
--- a/xen/common/sched_rt.c
+++ b/xen/common/sched_rt.c
@@ -49,13 +49,15 @@
  * A PCPU is feasible if the VCPU can run on this PCPU and (the PCPU is idle or
  * has a lower-priority VCPU running on it.)
  *
- * Each VCPU has a dedicated period and budget.
+ * Each VCPU has a dedicated period, budget and a extratime flag
  * The deadline of a VCPU is at the end of each period;
  * A VCPU has its budget replenished at the beginning of each period;
  * While scheduled, a VCPU burns its budget.
  * The VCPU needs to finish its budget before its deadline in each period;
  * The VCPU discards its unused budget at the end of each period.
- * If a VCPU runs out of budget in a period, it has to wait until next period.
+ * When a VCPU runs out of budget in a period, if its extratime flag is set,
+ * the VCPU increases its priority_level by 1 and refills its budget; otherwise,
+ * it has to wait until next period.
  *
  * Each VCPU is implemented as a deferable server.
  * When a VCPU has a task running on it, its budget is continuously burned;
@@ -63,7 +65,8 @@
  *
  * Queue scheme:
  * A global runqueue and a global depletedqueue for each CPU pool.
- * The runqueue holds all runnable VCPUs with budget, sorted by deadline;
+ * The runqueue holds all runnable VCPUs with budget,
+ * sorted by priority_level and deadline;
  * The depletedqueue holds all VCPUs without budget, unsorted;
  *
  * Note: cpumask and cpupool is supported.
@@ -151,6 +154,14 @@
 #define RTDS_depleted (1<<__RTDS_depleted)
 
 /*
+ * RTDS_extratime: Can the vcpu run in the time that is
+ * not part of any real-time reservation, and would therefore
+ * be otherwise left idle?
+ */
+#define __RTDS_extratime    4
+#define RTDS_extratime (1<<__RTDS_extratime)
+
+/*
  * rt tracing events ("only" 512 available!). Check
  * include/public/trace.h for more details.
  */
@@ -201,6 +212,8 @@ struct rt_vcpu {
     struct rt_dom *sdom;
     struct vcpu *vcpu;
 
+    unsigned priority_level;
+
     unsigned flags;              /* mark __RTDS_scheduled, etc.. */
 };
 
@@ -245,6 +258,11 @@ static inline struct list_head *rt_replq(const struct scheduler *ops)
     return &rt_priv(ops)->replq;
 }
 
+static inline bool has_extratime(const struct rt_vcpu *svc)
+{
+    return svc->flags & RTDS_extratime;
+}
+
 /*
  * Helper functions for manipulating the runqueue, the depleted queue,
  * and the replenishment events queue.
@@ -274,6 +292,21 @@ vcpu_on_replq(const struct rt_vcpu *svc)
 }
 
 /*
+ * If v1 priority >= v2 priority, return value > 0
+ * Otherwise, return value < 0
+ */
+static s_time_t
+compare_vcpu_priority(const struct rt_vcpu *v1, const struct rt_vcpu *v2)
+{
+    int prio = v2->priority_level - v1->priority_level;
+
+    if ( prio == 0 )
+        return v2->cur_deadline - v1->cur_deadline;
+
+    return prio;
+}
+
+/*
  * Debug related code, dump vcpu/cpu information
  */
 static void
@@ -303,6 +336,7 @@ rt_dump_vcpu(const struct scheduler *ops, const struct rt_vcpu *svc)
     cpulist_scnprintf(keyhandler_scratch, sizeof(keyhandler_scratch), mask);
     printk("[%5d.%-2u] cpu %u, (%"PRI_stime", %"PRI_stime"),"
            " cur_b=%"PRI_stime" cur_d=%"PRI_stime" last_start=%"PRI_stime"\n"
+           " \t\t priority_level=%d has_extratime=%d\n"
            " \t\t onQ=%d runnable=%d flags=%x effective hard_affinity=%s\n",
             svc->vcpu->domain->domain_id,
             svc->vcpu->vcpu_id,
@@ -312,6 +346,8 @@ rt_dump_vcpu(const struct scheduler *ops, const struct rt_vcpu *svc)
             svc->cur_budget,
             svc->cur_deadline,
             svc->last_start,
+            svc->priority_level,
+            has_extratime(svc),
             vcpu_on_q(svc),
             vcpu_runnable(svc->vcpu),
             svc->flags,
@@ -423,15 +459,18 @@ rt_update_deadline(s_time_t now, struct rt_vcpu *svc)
      */
     svc->last_start = now;
     svc->cur_budget = svc->budget;
+    svc->priority_level = 0;
 
     /* TRACE */
     {
         struct __packed {
             unsigned vcpu:16, dom:16;
+            unsigned priority_level;
             uint64_t cur_deadline, cur_budget;
         } d;
         d.dom = svc->vcpu->domain->domain_id;
         d.vcpu = svc->vcpu->vcpu_id;
+        d.priority_level = svc->priority_level;
         d.cur_deadline = (uint64_t) svc->cur_deadline;
         d.cur_budget = (uint64_t) svc->cur_budget;
         trace_var(TRC_RTDS_BUDGET_REPLENISH, 1,
@@ -454,7 +493,7 @@ rt_update_deadline(s_time_t now, struct rt_vcpu *svc)
  * cases, if the vcpu with the earliest deadline is what we
  * are dealing with).
  */
-static inline bool_t
+static inline bool
 deadline_queue_remove(struct list_head *queue, struct list_head *elem)
 {
     int pos = 0;
@@ -466,7 +505,7 @@ deadline_queue_remove(struct list_head *queue, struct list_head *elem)
     return !pos;
 }
 
-static inline bool_t
+static inline bool
 deadline_queue_insert(struct rt_vcpu * (*qelem)(struct list_head *),
                       struct rt_vcpu *svc, struct list_head *elem,
                       struct list_head *queue)
@@ -477,7 +516,7 @@ deadline_queue_insert(struct rt_vcpu * (*qelem)(struct list_head *),
     list_for_each ( iter, queue )
     {
         struct rt_vcpu * iter_svc = (*qelem)(iter);
-        if ( svc->cur_deadline <= iter_svc->cur_deadline )
+        if ( compare_vcpu_priority(svc, iter_svc) > 0 )
             break;
         pos++;
     }
@@ -537,8 +576,9 @@ runq_insert(const struct scheduler *ops, struct rt_vcpu *svc)
     ASSERT( !vcpu_on_q(svc) );
     ASSERT( vcpu_on_replq(svc) );
 
-    /* add svc to runq if svc still has budget */
-    if ( svc->cur_budget > 0 )
+    /* add svc to runq if svc still has budget or its extratime is set */
+    if ( svc->cur_budget > 0 ||
+         has_extratime(svc) )
         deadline_runq_insert(svc, &svc->q_elem, runq);
     else
         list_add(&svc->q_elem, &prv->depletedq);
@@ -857,6 +897,8 @@ rt_alloc_vdata(const struct scheduler *ops, struct vcpu *vc, void *dd)
     svc->vcpu = vc;
     svc->last_start = 0;
 
+    __set_bit(__RTDS_extratime, &svc->flags);
+    svc->priority_level = 0;
     svc->period = RTDS_DEFAULT_PERIOD;
     if ( !is_idle_vcpu(vc) )
         svc->budget = RTDS_DEFAULT_BUDGET;
@@ -966,8 +1008,16 @@ burn_budget(const struct scheduler *ops, struct rt_vcpu *svc, s_time_t now)
 
     if ( svc->cur_budget <= 0 )
     {
-        svc->cur_budget = 0;
-        __set_bit(__RTDS_depleted, &svc->flags);
+        if ( has_extratime(svc) )
+        {
+            svc->priority_level++;
+            svc->cur_budget = svc->budget;
+        }
+        else
+        {
+            svc->cur_budget = 0;
+            __set_bit(__RTDS_depleted, &svc->flags);
+        }
     }
 
     /* TRACE */
@@ -976,11 +1026,15 @@ burn_budget(const struct scheduler *ops, struct rt_vcpu *svc, s_time_t now)
             unsigned vcpu:16, dom:16;
             uint64_t cur_budget;
             int delta;
+            unsigned priority_level;
+            bool has_extratime;
         } d;
         d.dom = svc->vcpu->domain->domain_id;
         d.vcpu = svc->vcpu->vcpu_id;
         d.cur_budget = (uint64_t) svc->cur_budget;
         d.delta = delta;
+        d.priority_level = svc->priority_level;
+        d.has_extratime = svc->flags & RTDS_extratime;
         trace_var(TRC_RTDS_BUDGET_BURN, 1,
                   sizeof(d),
                   (unsigned char *) &d);
@@ -1088,7 +1142,7 @@ rt_schedule(const struct scheduler *ops, s_time_t now, bool_t tasklet_work_sched
              vcpu_runnable(current) &&
              scurr->cur_budget > 0 &&
              ( is_idle_vcpu(snext->vcpu) ||
-               scurr->cur_deadline <= snext->cur_deadline ) )
+               compare_vcpu_priority(scurr, snext) > 0 ) )
             snext = scurr;
     }
 
@@ -1194,7 +1248,7 @@ runq_tickle(const struct scheduler *ops, struct rt_vcpu *new)
         }
         iter_svc = rt_vcpu(iter_vc);
         if ( latest_deadline_vcpu == NULL ||
-             iter_svc->cur_deadline > latest_deadline_vcpu->cur_deadline )
+             compare_vcpu_priority(iter_svc, latest_deadline_vcpu) < 0 )
             latest_deadline_vcpu = iter_svc;
 
         cpumask_clear_cpu(cpu, &not_tickled);
@@ -1203,7 +1257,7 @@ runq_tickle(const struct scheduler *ops, struct rt_vcpu *new)
 
     /* 2) candicate has higher priority, kick out lowest priority vcpu */
     if ( latest_deadline_vcpu != NULL &&
-         new->cur_deadline < latest_deadline_vcpu->cur_deadline )
+         compare_vcpu_priority(latest_deadline_vcpu, new) < 0 )
     {
         SCHED_STAT_CRANK(tickled_busy_cpu);
         cpu_to_tickle = latest_deadline_vcpu->vcpu->processor;
@@ -1394,6 +1448,10 @@ rt_dom_cntl(
                 svc = rt_vcpu(d->vcpu[local_sched.vcpuid]);
                 local_sched.u.rtds.budget = svc->budget / MICROSECS(1);
                 local_sched.u.rtds.period = svc->period / MICROSECS(1);
+                if ( has_extratime(svc) )
+                    local_sched.u.rtds.flags |= XEN_DOMCTL_SCHEDRT_extra;
+                else
+                    local_sched.u.rtds.flags &= ~XEN_DOMCTL_SCHEDRT_extra;
                 spin_unlock_irqrestore(&prv->lock, flags);
 
                 if ( copy_to_guest_offset(op->u.v.vcpus, index,
@@ -1418,6 +1476,10 @@ rt_dom_cntl(
                 svc = rt_vcpu(d->vcpu[local_sched.vcpuid]);
                 svc->period = period;
                 svc->budget = budget;
+                if ( local_sched.u.rtds.flags & XEN_DOMCTL_SCHEDRT_extra )
+                    __set_bit(__RTDS_extratime, &svc->flags);
+                else
+                    __clear_bit(__RTDS_extratime, &svc->flags);
                 spin_unlock_irqrestore(&prv->lock, flags);
             }
             /* Process a most 64 vCPUs without checking for preemptions. */
@@ -1492,7 +1554,7 @@ static void repl_timer_handler(void *data){
         {
             struct rt_vcpu *next_on_runq = q_elem(runq->next);
 
-            if ( svc->cur_deadline > next_on_runq->cur_deadline )
+            if ( compare_vcpu_priority(svc, next_on_runq) < 0 )
                 runq_tickle(ops, next_on_runq);
         }
         else if ( __test_and_clear_bit(__RTDS_depleted, &svc->flags) &&
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 8853445..70027ab 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -342,6 +342,10 @@ struct xen_domctl_sched_credit2 {
 struct xen_domctl_sched_rtds {
     uint32_t period;
     uint32_t budget;
+/* Can this vCPU execute beyond its reserved amount of time? */
+#define _XEN_DOMCTL_SCHEDRT_extra   0
+#define XEN_DOMCTL_SCHEDRT_extra    (1U<<_XEN_DOMCTL_SCHEDRT_extra)
+    uint32_t flags;
 };
 
 typedef struct xen_domctl_schedparam_vcpu {
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 2/5] libxl: enable per-VCPU extratime flag for RTDS

[Meng Xu]

Modify libxl_vcpu_sched_params_get/set and sched_rtds_vcpu_get/set
functions to support per-VCPU extratime flag

Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>

---
Changes from v2
1) Move extratime out of the section
   that is marked as depreciated in libxl_domain_sched_params.
2) Set vcpu extratime in sched_rtds_vcpu_get function function;
   This fix a bug in previous version
   when run command "xl sched-rtds -d 0 -v 1" which
   outputs vcpu extratime value incorrectly.

Changes from v1
1) Add LIBXL_HAVE_SCHED_RTDS_VCPU_EXTRA to indicate if extratime flag is
supported
2) Change flag name in domctl.h from XEN_DOMCTL_SCHED_RTDS_extratime to
XEN_DOMCTL_SCHEDRT_extra

Changes from RFC v1
Change work_conserving flag to extratime flag
---
 tools/libxl/libxl.h         |  6 ++++++
 tools/libxl/libxl_sched.c   | 17 +++++++++++++++++
 tools/libxl/libxl_types.idl |  8 ++++----
 3 files changed, 27 insertions(+), 4 deletions(-)

diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index f82b91e..5e9aed7 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -257,6 +257,12 @@
 #define LIBXL_HAVE_SCHED_RTDS_VCPU_PARAMS 1
 
 /*
+ * LIBXL_HAVE_SCHED_RTDS_VCPU_EXTRA indicates RTDS scheduler
+ * now supports per-vcpu extratime settings.
+ */
+#define LIBXL_HAVE_SCHED_RTDS_VCPU_EXTRA 1
+
+/*
  * libxl_domain_build_info has the arm.gic_version field.
  */
 #define LIBXL_HAVE_BUILDINFO_ARM_GIC_VERSION 1
diff --git a/tools/libxl/libxl_sched.c b/tools/libxl/libxl_sched.c
index 7d144d0..512788f 100644
--- a/tools/libxl/libxl_sched.c
+++ b/tools/libxl/libxl_sched.c
@@ -532,6 +532,8 @@ static int sched_rtds_vcpu_get(libxl__gc *gc, uint32_t domid,
     for (i = 0; i < num_vcpus; i++) {
         scinfo->vcpus[i].period = vcpus[i].u.rtds.period;
         scinfo->vcpus[i].budget = vcpus[i].u.rtds.budget;
+        scinfo->vcpus[i].extratime =
+                !!(vcpus[i].u.rtds.flags & XEN_DOMCTL_SCHEDRT_extra);
         scinfo->vcpus[i].vcpuid = vcpus[i].vcpuid;
     }
     rc = 0;
@@ -579,6 +581,8 @@ static int sched_rtds_vcpu_get_all(libxl__gc *gc, uint32_t domid,
     for (i = 0; i < num_vcpus; i++) {
         scinfo->vcpus[i].period = vcpus[i].u.rtds.period;
         scinfo->vcpus[i].budget = vcpus[i].u.rtds.budget;
+        scinfo->vcpus[i].extratime =
+                !!(vcpus[i].u.rtds.flags & XEN_DOMCTL_SCHEDRT_extra);
         scinfo->vcpus[i].vcpuid = vcpus[i].vcpuid;
     }
     rc = 0;
@@ -628,6 +632,10 @@ static int sched_rtds_vcpu_set(libxl__gc *gc, uint32_t domid,
         vcpus[i].vcpuid = scinfo->vcpus[i].vcpuid;
         vcpus[i].u.rtds.period = scinfo->vcpus[i].period;
         vcpus[i].u.rtds.budget = scinfo->vcpus[i].budget;
+        if (scinfo->vcpus[i].extratime)
+            vcpus[i].u.rtds.flags |= XEN_DOMCTL_SCHEDRT_extra;
+        else
+            vcpus[i].u.rtds.flags &= ~XEN_DOMCTL_SCHEDRT_extra;
     }
 
     r = xc_sched_rtds_vcpu_set(CTX->xch, domid,
@@ -676,6 +684,10 @@ static int sched_rtds_vcpu_set_all(libxl__gc *gc, uint32_t domid,
         vcpus[i].vcpuid = i;
         vcpus[i].u.rtds.period = scinfo->vcpus[0].period;
         vcpus[i].u.rtds.budget = scinfo->vcpus[0].budget;
+        if (scinfo->vcpus[0].extratime)
+            vcpus[i].u.rtds.flags |= XEN_DOMCTL_SCHEDRT_extra;
+        else
+            vcpus[i].u.rtds.flags &= ~XEN_DOMCTL_SCHEDRT_extra;
     }
 
     r = xc_sched_rtds_vcpu_set(CTX->xch, domid,
@@ -726,6 +738,11 @@ static int sched_rtds_domain_set(libxl__gc *gc, uint32_t domid,
         sdom.period = scinfo->period;
     if (scinfo->budget != LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT)
         sdom.budget = scinfo->budget;
+    /* Set extratime by default */
+    if (scinfo->extratime)
+        sdom.flags |= XEN_DOMCTL_SCHEDRT_extra;
+    else
+        sdom.flags &= ~XEN_DOMCTL_SCHEDRT_extra;
     if (sched_rtds_validate_params(gc, sdom.period, sdom.budget))
         return ERROR_INVAL;
 
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 2d0bb8a..dd7d364 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -421,14 +421,14 @@ libxl_domain_sched_params = Struct("domain_sched_params",[
     ("cap",          integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_CAP_DEFAULT'}),
     ("period",       integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_PERIOD_DEFAULT'}),
     ("budget",       integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT'}),
+    ("extratime",    integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT'}),
 
-    # The following three parameters ('slice', 'latency' and 'extratime') are deprecated,
+    # The following three parameters ('slice' and 'latency') are deprecated,
     # and will have no effect if used, since the SEDF scheduler has been removed.
-    # Note that 'period' was an SDF parameter too, but it is still effective as it is
-    # now used (together with 'budget') by the RTDS scheduler.
+    # Note that 'period' and 'extratime' was an SDF parameter too, but it is still effective
+    # as they are now used (together with 'budget') by the RTDS scheduler.
     ("slice",        integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_SLICE_DEFAULT'}),
     ("latency",      integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_LATENCY_DEFAULT'}),
-    ("extratime",    integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT'}),
     ])
 
 libxl_vnode_info = Struct("vnode_info", [
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 3/5] xl: enable per-VCPU extratime flag for RTDS

[Meng Xu]

Change main_sched_rtds and related output functions to support
per-VCPU extratime flag.

Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>

---
Changes from v2
Validate the -e option input that can only be 0 or 1
Update docs/man/xl.pod.1.in
Change EXTRATIME to Extratime

Changes from v1
No change because we agree on using -e 0/1 option to
set if a vcpu will get extra time or not

Changes from RFC v1
Changes work_conserving flag to extratime flag
---
 docs/man/xl.pod.1.in   | 59 +++++++++++++++++++++++++----------------------
 tools/xl/xl_cmdtable.c |  3 ++-
 tools/xl/xl_sched.c    | 62 +++++++++++++++++++++++++++++++++++---------------
 3 files changed, 78 insertions(+), 46 deletions(-)

diff --git a/docs/man/xl.pod.1.in b/docs/man/xl.pod.1.in
index cd8bb1c..486a24f 100644
--- a/docs/man/xl.pod.1.in
+++ b/docs/man/xl.pod.1.in
@@ -1117,11 +1117,11 @@ as B<--ratelimit_us> in B<sched-credit>
 Set or get rtds (Real Time Deferrable Server) scheduler parameters.
 This rt scheduler applies Preemptive Global Earliest Deadline First
 real-time scheduling algorithm to schedule VCPUs in the system.
-Each VCPU has a dedicated period and budget.
-VCPUs in the same domain have the same period and budget.
+Each VCPU has a dedicated period, budget and extratime.
 While scheduled, a VCPU burns its budget.
 A VCPU has its budget replenished at the beginning of each period;
 Unused budget is discarded at the end of each period.
+A VCPU with extratime set gets extra time from the unreserved system resource.
 
 B<OPTIONS>
 
@@ -1145,6 +1145,11 @@ Period of time, in microseconds, over which to replenish the budget.
 Amount of time, in microseconds, that the VCPU will be allowed
 to run every period.
 
+=item B<-e Extratime>, B<--extratime=Extratime>
+
+Binary flag to decide if the VCPU will be allowed to get extra time from
+the unreserved system resource.
+
 =item B<-c CPUPOOL>, B<--cpupool=CPUPOOL>
 
 Restrict output to domains in the specified cpupool.
@@ -1160,57 +1165,57 @@ all the domains:
 
     xl sched-rtds -v all
     Cpupool Pool-0: sched=RTDS
-    Name                        ID VCPU    Period    Budget
-    Domain-0                     0    0     10000      4000
-    vm1                          1    0       300       150
-    vm1                          1    1       400       200
-    vm1                          1    2     10000      4000
-    vm1                          1    3      1000       500
-    vm2                          2    0     10000      4000
-    vm2                          2    1     10000      4000
+    Name                        ID VCPU    Period    Budget  Extratime
+    Domain-0                     0    0     10000      4000        yes
+    vm1                          2    0       300       150        yes
+    vm1                          2    1       400       200        yes
+    vm1                          2    2     10000      4000        yes
+    vm1                          2    3      1000       500        yes
+    vm2                          4    0     10000      4000        yes
+    vm2                          4    1     10000      4000        yes
 
 Without any arguments, it will output the default scheduling
 parameters for each domain:
 
     xl sched-rtds
     Cpupool Pool-0: sched=RTDS
-    Name                        ID    Period    Budget
-    Domain-0                     0     10000      4000
-    vm1                          1     10000      4000
-    vm2                          2     10000      4000
+    Name                        ID    Period    Budget  Extratime
+    Domain-0                     0     10000      4000        yes
+    vm1                          2     10000      4000        yes
+    vm2                          4     10000      4000        yes
 
 
-2) Use, for instancei, B<-d vm1, -v all> to see the budget and
+2) Use, for instance, B<-d vm1, -v all> to see the budget and
 period of all VCPUs of a specific domain (B<vm1>):
 
     xl sched-rtds -d vm1 -v all
-    Name                        ID VCPU    Period    Budget
-    vm1                          1    0       300       150
-    vm1                          1    1       400       200
-    vm1                          1    2     10000      4000
-    vm1                          1    3      1000       500
+    Name                        ID VCPU    Period    Budget  Extratime
+    vm1                          2    0       300       150        yes
+    vm1                          2    1       400       200        yes
+    vm1                          2    2     10000      4000        yes
+    vm1                          2    3      1000       500        yes
 
 To see the parameters of a subset of the VCPUs of a domain, use:
 
     xl sched-rtds -d vm1 -v 0 -v 3
-    Name                        ID VCPU    Period    Budget
-    vm1                          1    0       300       150
-    vm1                          1    3      1000       500
+    Name                        ID VCPU    Period    Budget  Extratime
+    vm1                          2    0       300       150        yes
+    vm1                          2    3      1000       500        yes
 
 If no B<-v> is specified, the default scheduling parameters for the
 domain are shown:
 
     xl sched-rtds -d vm1
-    Name                        ID    Period    Budget
-    vm1                          1     10000      4000
+    Name                        ID    Period    Budget  Extratime
+    vm1                          2     10000      4000        yes
 
 
 3) Users can set the budget and period of multiple VCPUs of a
 specific domain with only one command,
-e.g., "xl sched-rtds -d vm1 -v 0 -p 100 -b 50 -v 3 -p 300 -b 150".
+e.g., "xl sched-rtds -d vm1 -v 0 -p 100 -b 50 -e 1 -v 3 -p 300 -b 150 -e 0".
 
 To change the parameters of all the VCPUs of a domain, use B<-v all>,
-e.g., "xl sched-rtds -d vm1 -v all -p 500 -b 250".
+e.g., "xl sched-rtds -d vm1 -v all -p 500 -b 250 -e 1".
 
 =back
 
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 68a8a72..5546cf6 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -273,12 +273,13 @@ struct cmd_spec cmd_table[] = {
     { "sched-rtds",
       &main_sched_rtds, 0, 1,
       "Get/set rtds scheduler parameters",
-      "[-d <Domain> [-v[=VCPUID/all]] [-p[=PERIOD]] [-b[=BUDGET]]]",
+      "[-d <Domain> [-v[=VCPUID/all]] [-p[=PERIOD]] [-b[=BUDGET]] [-e[=Extratime]]]",
       "-d DOMAIN, --domain=DOMAIN     Domain to modify\n"
       "-v VCPUID/all, --vcpuid=VCPUID/all    VCPU to modify or output;\n"
       "               Using '-v all' to modify/output all vcpus\n"
       "-p PERIOD, --period=PERIOD     Period (us)\n"
       "-b BUDGET, --budget=BUDGET     Budget (us)\n"
+      "-e Extratime, --extratime=Extratime Extratime (1=yes, 0=no)\n"
     },
     { "domid",
       &main_domid, 0, 0,
diff --git a/tools/xl/xl_sched.c b/tools/xl/xl_sched.c
index 7fabce3..7965ccb 100644
--- a/tools/xl/xl_sched.c
+++ b/tools/xl/xl_sched.c
@@ -252,7 +252,7 @@ static int sched_rtds_domain_output(
     libxl_domain_sched_params scinfo;
 
     if (domid < 0) {
-        printf("%-33s %4s %9s %9s\n", "Name", "ID", "Period", "Budget");
+        printf("%-33s %4s %9s %9s %10s\n", "Name", "ID", "Period", "Budget", "Extratime");
         return 0;
     }
 
@@ -263,11 +263,12 @@ static int sched_rtds_domain_output(
     }
 
     domname = libxl_domid_to_name(ctx, domid);
-    printf("%-33s %4d %9d %9d\n",
+    printf("%-33s %4d %9d %9d %10s\n",
         domname,
         domid,
         scinfo.period,
-        scinfo.budget);
+        scinfo.budget,
+        scinfo.extratime ? "yes" : "no");
     free(domname);
     libxl_domain_sched_params_dispose(&scinfo);
     return 0;
@@ -280,8 +281,8 @@ static int sched_rtds_vcpu_output(int domid, libxl_vcpu_sched_params *scinfo)
     int i;
 
     if (domid < 0) {
-        printf("%-33s %4s %4s %9s %9s\n", "Name", "ID",
-               "VCPU", "Period", "Budget");
+        printf("%-33s %4s %4s %9s %9s %10s\n", "Name", "ID",
+               "VCPU", "Period", "Budget", "Extratime");
         return 0;
     }
 
@@ -291,12 +292,13 @@ static int sched_rtds_vcpu_output(int domid, libxl_vcpu_sched_params *scinfo)
 
     domname = libxl_domid_to_name(ctx, domid);
     for ( i = 0; i < scinfo->num_vcpus; i++ ) {
-        printf("%-33s %4d %4d %9"PRIu32" %9"PRIu32"\n",
+        printf("%-33s %4d %4d %9"PRIu32" %9"PRIu32" %10s\n",
                domname,
                domid,
                scinfo->vcpus[i].vcpuid,
                scinfo->vcpus[i].period,
-               scinfo->vcpus[i].budget);
+               scinfo->vcpus[i].budget,
+               scinfo->vcpus[i].extratime ? "yes" : "no");
     }
     free(domname);
     return 0;
@@ -310,8 +312,8 @@ static int sched_rtds_vcpu_output_all(int domid,
     int i;
 
     if (domid < 0) {
-        printf("%-33s %4s %4s %9s %9s\n", "Name", "ID",
-               "VCPU", "Period", "Budget");
+        printf("%-33s %4s %4s %9s %9s %10s\n", "Name", "ID",
+               "VCPU", "Period", "Budget", "Extratime");
         return 0;
     }
 
@@ -322,12 +324,13 @@ static int sched_rtds_vcpu_output_all(int domid,
 
     domname = libxl_domid_to_name(ctx, domid);
     for ( i = 0; i < scinfo->num_vcpus; i++ ) {
-        printf("%-33s %4d %4d %9"PRIu32" %9"PRIu32"\n",
+        printf("%-33s %4d %4d %9"PRIu32" %9"PRIu32" %10s\n",
                domname,
                domid,
                scinfo->vcpus[i].vcpuid,
                scinfo->vcpus[i].period,
-               scinfo->vcpus[i].budget);
+               scinfo->vcpus[i].budget,
+               scinfo->vcpus[i].extratime ? "yes" : "no");
     }
     free(domname);
     return 0;
@@ -711,14 +714,18 @@ int main_sched_rtds(int argc, char **argv)
     int *vcpus = (int *)xmalloc(sizeof(int)); /* IDs of VCPUs that change */
     int *periods = (int *)xmalloc(sizeof(int)); /* period is in microsecond */
     int *budgets = (int *)xmalloc(sizeof(int)); /* budget is in microsecond */
+    bool *extratimes = (bool *)xmalloc(sizeof(bool)); /* extratime is bool */
     int v_size = 1; /* size of vcpus array */
     int p_size = 1; /* size of periods array */
     int b_size = 1; /* size of budgets array */
+    int e_size = 1; /* size of extratimes array */
     int v_index = 0; /* index in vcpus array */
     int p_index =0; /* index in periods array */
     int b_index =0; /* index for in budgets array */
+    int e_index = 0; /* index in extratimes array */
     bool opt_p = false;
     bool opt_b = false;
+    bool opt_e = false;
     bool opt_v = false;
     bool opt_all = false; /* output per-dom parameters */
     int opt, i, rc, r;
@@ -726,12 +733,13 @@ int main_sched_rtds(int argc, char **argv)
         {"domain", 1, 0, 'd'},
         {"period", 1, 0, 'p'},
         {"budget", 1, 0, 'b'},
+        {"extratime", 1, 0, 'e'},
         {"vcpuid",1, 0, 'v'},
         {"cpupool", 1, 0, 'c'},
         COMMON_LONG_OPTS
     };
 
-    SWITCH_FOREACH_OPT(opt, "d:p:b:v:c", opts, "sched-rtds", 0) {
+    SWITCH_FOREACH_OPT(opt, "d:p:b:e:v:c", opts, "sched-rtds", 0) {
     case 'd':
         dom = optarg;
         break;
@@ -755,6 +763,20 @@ int main_sched_rtds(int argc, char **argv)
         budgets[b_index++] = strtol(optarg, NULL, 10);
         opt_b = 1;
         break;
+    case 'e':
+        if (e_index >= e_size) { /* extratime array is full */
+            e_size *= 2;
+            extratimes = xrealloc(extratimes, e_size);
+        }
+        if (strcmp(optarg, "0") && strcmp(optarg, "1"))
+        {
+            fprintf(stderr, "Invalid extratime.\n");
+            r = EXIT_FAILURE;
+            goto out;
+        }
+        extratimes[e_index++] = strtol(optarg, NULL, 10);
+        opt_e = 1;
+        break;
     case 'v':
         if (!strcmp(optarg, "all")) { /* get or set all vcpus of a domain */
             opt_all = 1;
@@ -772,18 +794,18 @@ int main_sched_rtds(int argc, char **argv)
         break;
     }
 
-    if (cpupool && (dom || opt_p || opt_b || opt_v || opt_all)) {
+    if (cpupool && (dom || opt_p || opt_b || opt_e || opt_v || opt_all)) {
         fprintf(stderr, "Specifying a cpupool is not allowed with "
                 "other options.\n");
         r = EXIT_FAILURE;
         goto out;
     }
-    if (!dom && (opt_p || opt_b || opt_v)) {
+    if (!dom && (opt_p || opt_b || opt_e || opt_v)) {
         fprintf(stderr, "Missing parameters.\n");
         r = EXIT_FAILURE;
         goto out;
     }
-    if (dom && !opt_v && !opt_all && (opt_p || opt_b)) {
+    if (dom && !opt_v && !opt_all && (opt_p || opt_b || opt_e)) {
         fprintf(stderr, "Must specify VCPU.\n");
         r = EXIT_FAILURE;
         goto out;
@@ -794,8 +816,9 @@ int main_sched_rtds(int argc, char **argv)
         goto out;
     }
     if (((v_index > b_index) && opt_b) || ((v_index > p_index) && opt_p)
-        || p_index != b_index) {
-        fprintf(stderr, "Incorrect number of period and budget\n");
+         || ((v_index > e_index) && opt_e) || p_index != b_index
+         || p_index != e_index ) {
+        fprintf(stderr, "Incorrect number of period, budget and extratime\n");
         r = EXIT_FAILURE;
         goto out;
     }
@@ -829,7 +852,7 @@ int main_sched_rtds(int argc, char **argv)
                 r = EXIT_FAILURE;
                 goto out;
             }
-        } else if (!opt_p && !opt_b) {
+        } else if (!opt_p && !opt_b && !opt_e) {
             /* get per-vcpu rtds scheduling parameters */
             libxl_vcpu_sched_params scinfo;
             libxl_vcpu_sched_params_init(&scinfo);
@@ -861,6 +884,7 @@ int main_sched_rtds(int argc, char **argv)
                     scinfo.vcpus[i].vcpuid = vcpus[i];
                     scinfo.vcpus[i].period = periods[i];
                     scinfo.vcpus[i].budget = budgets[i];
+                    scinfo.vcpus[i].extratime = extratimes[i] ? 1 : 0;
                 }
                 rc = sched_vcpu_set(domid, &scinfo);
             } else { /* set params for all vcpus */
@@ -869,6 +893,7 @@ int main_sched_rtds(int argc, char **argv)
                                xmalloc(sizeof(libxl_sched_params));
                 scinfo.vcpus[0].period = periods[0];
                 scinfo.vcpus[0].budget = budgets[0];
+                scinfo.vcpus[0].extratime = extratimes[0] ? 1 : 0;
                 rc = sched_vcpu_set_all(domid, &scinfo);
             }
 
@@ -885,6 +910,7 @@ out:
     free(vcpus);
     free(periods);
     free(budgets);
+    free(extratimes);
     return r;
 }
 
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 4/5] xentrace: enable per-VCPU extratime flag for RTDS

[Meng Xu]

Change repl_budget event output for xentrace formats and xenalyze

Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>

---
No changes from v2

Changes from v1
Add this changes from v1
---
 tools/xentrace/formats    | 2 +-
 tools/xentrace/xenalyze.c | 8 +++++---
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/tools/xentrace/formats b/tools/xentrace/formats
index d6e7e3f..7d3a209 100644
--- a/tools/xentrace/formats
+++ b/tools/xentrace/formats
@@ -75,7 +75,7 @@
 0x00022801  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:tickle        [ cpu = %(1)d ]
 0x00022802  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:runq_pick     [ dom:vcpu = 0x%(1)08x, cur_deadline = 0x%(3)08x%(2)08x, cur_budget = 0x%(5)08x%(4)08x ]
 0x00022803  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:burn_budget   [ dom:vcpu = 0x%(1)08x, cur_budget = 0x%(3)08x%(2)08x, delta = %(4)d ]
-0x00022804  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:repl_budget   [ dom:vcpu = 0x%(1)08x, cur_deadline = 0x%(3)08x%(2)08x, cur_budget = 0x%(5)08x%(4)08x ]
+0x00022804  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:repl_budget   [ dom:vcpu = 0x%(1)08x, priority_level = 0x%(2)08d cur_deadline = 0x%(4)08x%(3)08x, cur_budget = 0x%(6)08x%(5)08x ]
 0x00022805  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:sched_tasklet
 0x00022806  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:schedule      [ cpu[16]:tasklet[8]:idle[4]:tickled[4] = %(1)08x ]
 
diff --git a/tools/xentrace/xenalyze.c b/tools/xentrace/xenalyze.c
index 79bdba7..2783204 100644
--- a/tools/xentrace/xenalyze.c
+++ b/tools/xentrace/xenalyze.c
@@ -7946,12 +7946,14 @@ void sched_process(struct pcpu_info *p)
             if(opt.dump_all) {
                 struct {
                     unsigned int vcpuid:16, domid:16;
+                    unsigned int priority_level;
                     uint64_t cur_dl, cur_bg;
                 } __attribute__((packed)) *r = (typeof(r))ri->d;
 
-                printf(" %s rtds:repl_budget d%uv%u, deadline = %"PRIu64", "
-                       "budget = %"PRIu64"\n", ri->dump_header,
-                       r->domid, r->vcpuid, r->cur_dl, r->cur_bg);
+                printf(" %s rtds:repl_budget d%uv%u, priority_level = %u,"
+                       "deadline = %"PRIu64", budget = %"PRIu64"\n",
+                       ri->dump_header, r->domid, r->vcpuid,
+                       r->priority_level, r->cur_dl, r->cur_bg);
             }
             break;
         case TRC_SCHED_CLASS_EVT(RTDS, 5): /* SCHED_TASKLET    */
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 5/5] docs: enable per-VCPU extratime flag for RTDS

[Meng Xu]

Revise xl tool use case by adding -e option
Remove work-conserving from TODO list

Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>

---
No change from v2

Changes from v1
Revise rtds docs
---
 docs/features/sched_rtds.pandoc | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/docs/features/sched_rtds.pandoc b/docs/features/sched_rtds.pandoc
index 354097b..d51b499 100644
--- a/docs/features/sched_rtds.pandoc
+++ b/docs/features/sched_rtds.pandoc
@@ -40,7 +40,7 @@ as follows:
 
 It is possible, for a multiple vCPUs VM, to change the parameters of
 each vCPU individually:
-    * `xl sched-rtds -d vm-rt -v 0 -p 20000 -b 10000 -v 1 -p 45000 -b 12000`
+    * `xl sched-rtds -d vm-rt -v 0 -p 20000 -b 10000 -e 1 -v 1 -p 45000 -b 12000 -e 0`
 
 # Technical details
 
@@ -53,7 +53,8 @@ the presence of the LIBXL\_HAVE\_SCHED\_RTDS symbol. The ability of
 specifying different scheduling parameters for each vcpu has been
 introduced later, and is available if the following symbols are defined:
     * `LIBXL\_HAVE\_VCPU\_SCHED\_PARAMS`,
-    * `LIBXL\_HAVE\_SCHED\_RTDS\_VCPU\_PARAMS`.
+    * `LIBXL\_HAVE\_SCHED\_RTDS\_VCPU\_PARAMS`,
+    * `LIBXL\_HAVE\_SCHED\_RTDS\_VCPU\_EXTRA`.
 
 # Limitations
 
@@ -95,7 +96,6 @@ at a macroscopic level), the following should be done:
 
 # Areas for improvement
 
-* Work-conserving mode to be added;
 * performance assessment, especially focusing on what level of real-time
   behavior the scheduler enables.
 
@@ -118,4 +118,5 @@ at a macroscopic level), the following should be done:
 Date       Revision Version  Notes
 ---------- -------- -------- -------------------------------------------
 2016-10-14 1        Xen 4.8  Document written
+2017-08-31 2        Xen 4.10 Revise for work conserving feature
 ---------- -------- -------- -------------------------------------------
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 5/5] docs: enable per-VCPU extratime flag for RTDS

[Wei Liu]

On Tue, Oct 10, 2017 at 07:17:45PM -0400, Meng Xu wrote:
> Revise xl tool use case by adding -e option
> Remove work-conserving from TODO list
> 
> Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>

Acked-by: Wei Liu <wei.liu2@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 2/5] libxl: enable per-VCPU extratime flag for RTDS

[Wei Liu]

On Tue, Oct 10, 2017 at 07:17:42PM -0400, Meng Xu wrote:
> Modify libxl_vcpu_sched_params_get/set and sched_rtds_vcpu_get/set
> functions to support per-VCPU extratime flag
> 
> Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>

Acked-by: Wei Liu <wei.liu2@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 3/5] xl: enable per-VCPU extratime flag for RTDS

[Wei Liu]

On Tue, Oct 10, 2017 at 07:17:43PM -0400, Meng Xu wrote:
> Change main_sched_rtds and related output functions to support
> per-VCPU extratime flag.
> 
> Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>

Acked-by: Wei Liu <wei.liu2@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 5/5] docs: enable per-VCPU extratime flag for RTDS

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 455 bytes --]

On Tue, 2017-10-10 at 19:17 -0400, Meng Xu wrote:
> Revise xl tool use case by adding -e option
> Remove work-conserving from TODO list
> 
> Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>
> 
Reviewed-by: Dario Faggioli <raistlin@linux.it>

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 3/5] xl: enable per-VCPU extratime flag for RTDS

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 461 bytes --]

On Tue, 2017-10-10 at 19:17 -0400, Meng Xu wrote:
> Change main_sched_rtds and related output functions to support
> per-VCPU extratime flag.
> 
> Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>
> 
Reviewed-by: Dario Faggioli <raistlin@linux.it>

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 2/5] libxl: enable per-VCPU extratime flag for RTDS

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 485 bytes --]

On Tue, 2017-10-10 at 19:17 -0400, Meng Xu wrote:
> Modify libxl_vcpu_sched_params_get/set and sched_rtds_vcpu_get/set
> functions to support per-VCPU extratime flag
> 
> Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>
> 
Reviewed-by: Dario Faggioli <raistlin@linux.it>

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 1/5] xen:rtds: towards work conserving RTDS

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1460 bytes --]

On Tue, 2017-10-10 at 19:17 -0400, Meng Xu wrote:
> Make RTDS scheduler work conserving without breaking the real-time
> guarantees.
> 
> VCPU model:
> Each real-time VCPU is extended to have an extratime flag
> and a priority_level field.
> When a VCPU's budget is depleted in the current period,
> if it has extratime flag set,
> its priority_level will increase by 1 and its budget will be
> refilled;
> othewrise, the VCPU will be moved to the depletedq.
> 
> Scheduling policy is modified global EDF:
> A VCPU v1 has higher priority than another VCPU v2 if
> (i) v1 has smaller priority_leve; or
> (ii) v1 has the same priority_level but has a smaller deadline
> 
> Queue management:
> Run queue holds VCPUs with extratime flag set and VCPUs with
> remaining budget. Run queue is sorted in increasing order of VCPUs
> priorities.
> Depleted queue holds VCPUs which have extratime flag cleared and
> depleted budget.
> Replenished queue is not modified.
> 
> Distribution of spare bandwidth
> Spare bandwidth is distributed among all VCPUs with extratime flag
> set,
> proportional to these VCPUs utilizations
> 
> Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>
> 
Reviewed-by: Dario Faggioli <raistlin@linux.it>

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 4/5] xentrace: enable per-VCPU extratime flag for RTDS

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 2707 bytes --]

On Tue, 2017-10-10 at 19:17 -0400, Meng Xu wrote:
> --- a/tools/xentrace/formats
> +++ b/tools/xentrace/formats
> @@ -75,7 +75,7 @@
>  0x00022801  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:tickle        [
> cpu = %(1)d ]
>  0x00022802  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:runq_pick     [
> dom:vcpu = 0x%(1)08x, cur_deadline = 0x%(3)08x%(2)08x, cur_budget =
> 0x%(5)08x%(4)08x ]
>  0x00022803  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:burn_budget   [
> dom:vcpu = 0x%(1)08x, cur_budget = 0x%(3)08x%(2)08x, delta = %(4)d ]
> -0x00022804  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:repl_budget   [
> dom:vcpu = 0x%(1)08x, cur_deadline = 0x%(3)08x%(2)08x, cur_budget =
> 0x%(5)08x%(4)08x ]
> +0x00022804  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:repl_budget   [
> dom:vcpu = 0x%(1)08x, priority_level = 0x%(2)08d cur_deadline =
> 0x%(4)08x%(3)08x, cur_budget = 0x%(6)08x%(5)08x ]
>  0x00022805  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:sched_tasklet
>  0x00022806  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:schedule      [
> cpu[16]:tasklet[8]:idle[4]:tickled[4] = %(1)08x ]
>  
But, both in case of this file and below in xenalyze.c, you update 1
record (the one of REPL_BUDGET). However, in patch 1, you added the
priority_level field to two records: REPL_BUDGET and BURN_BUDGET.

Or am I missing something?

Regards,
Dario

> diff --git a/tools/xentrace/xenalyze.c b/tools/xentrace/xenalyze.c
> index 79bdba7..2783204 100644
> --- a/tools/xentrace/xenalyze.c
> +++ b/tools/xentrace/xenalyze.c
> @@ -7946,12 +7946,14 @@ void sched_process(struct pcpu_info *p)
>              if(opt.dump_all) {
>                  struct {
>                      unsigned int vcpuid:16, domid:16;
> +                    unsigned int priority_level;
>                      uint64_t cur_dl, cur_bg;
>                  } __attribute__((packed)) *r = (typeof(r))ri->d;
>  
> -                printf(" %s rtds:repl_budget d%uv%u, deadline =
> %"PRIu64", "
> -                       "budget = %"PRIu64"\n", ri->dump_header,
> -                       r->domid, r->vcpuid, r->cur_dl, r->cur_bg);
> +                printf(" %s rtds:repl_budget d%uv%u, priority_level
> = %u,"
> +                       "deadline = %"PRIu64", budget = %"PRIu64"\n",
> +                       ri->dump_header, r->domid, r->vcpuid,
> +                       r->priority_level, r->cur_dl, r->cur_bg);
>              }
>              break;
>          case TRC_SCHED_CLASS_EVT(RTDS, 5): /* SCHED_TASKLET    */
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 4/5] xentrace: enable per-VCPU extratime flag for RTDS

[Meng Xu]

On Wed, Oct 11, 2017 at 6:57 AM, Dario Faggioli <raistlin@linux.it> wrote:
> On Tue, 2017-10-10 at 19:17 -0400, Meng Xu wrote:
>> --- a/tools/xentrace/formats
>> +++ b/tools/xentrace/formats
>> @@ -75,7 +75,7 @@
>>  0x00022801  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:tickle        [
>> cpu = %(1)d ]
>>  0x00022802  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:runq_pick     [
>> dom:vcpu = 0x%(1)08x, cur_deadline = 0x%(3)08x%(2)08x, cur_budget =
>> 0x%(5)08x%(4)08x ]
>>  0x00022803  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:burn_budget   [
>> dom:vcpu = 0x%(1)08x, cur_budget = 0x%(3)08x%(2)08x, delta = %(4)d ]
>> -0x00022804  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:repl_budget   [
>> dom:vcpu = 0x%(1)08x, cur_deadline = 0x%(3)08x%(2)08x, cur_budget =
>> 0x%(5)08x%(4)08x ]
>> +0x00022804  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:repl_budget   [
>> dom:vcpu = 0x%(1)08x, priority_level = 0x%(2)08d cur_deadline =
>> 0x%(4)08x%(3)08x, cur_budget = 0x%(6)08x%(5)08x ]
>>  0x00022805  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:sched_tasklet
>>  0x00022806  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:schedule      [
>> cpu[16]:tasklet[8]:idle[4]:tickled[4] = %(1)08x ]
>>
> But, both in case of this file and below in xenalyze.c, you update 1
> record (the one of REPL_BUDGET). However, in patch 1, you added the
> priority_level field to two records: REPL_BUDGET and BURN_BUDGET.
>
> Or am I missing something?

OMG, my fault. I forgot to check this. I will add this and double
check it by running some tests.

Best,

Meng
-- 
Meng Xu
Ph.D. Candidate in Computer and Information Science
University of Pennsylvania
http://www.cis.upenn.edu/~mengxu/

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel