From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [PATCH v6][RFC]xen: sched: convert RTDS from time to event driven model Date: Fri, 26 Feb 2016 00:31:18 +0100 Message-ID: <1456443078.2959.85.camel@citrix.com> References: <1456430736-4606-1-git-send-email-tiche@seas.upenn.edu> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1405397958707458069==" Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.84) (envelope-from ) id 1aZ5NW-0008Mi-Mm for xen-devel@lists.xenproject.org; Thu, 25 Feb 2016 23:31:26 +0000 In-Reply-To: <1456430736-4606-1-git-send-email-tiche@seas.upenn.edu> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Tianyang Chen , xen-devel@lists.xenproject.org Cc: george.dunlap@citrix.com, Dagaen Golomb , Meng Xu List-Id: xen-devel@lists.xenproject.org --===============1405397958707458069== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-5ApKvZN7GblqLodGcrkr" --=-5ApKvZN7GblqLodGcrkr Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hey again, Thanks for turning up so quickly. We are getting closer and closer, although (of course :-)) I have some more comments. However, is there a particular reason why you are keeping the RFC tag? Until you do that, it's like saying that you are chasing feedback, but you do not think yourself the patch should be considered for being upstreamed. As far as my opinion goes, this patch is not ready to go in right now (as I said, I've got questions and comments), but its status is way past RFC. On Thu, 2016-02-25 at 15:05 -0500, Tianyang Chen wrote: > changes since v5: > =C2=A0=C2=A0=C2=A0=C2=A0removed unnecessary vcpu_on_replq() checks > =C2=A0=C2=A0=C2=A0=C2=A0deadline_queue_insert() returns a flag to indicat= e if it's > =C2=A0=C2=A0=C2=A0=C2=A0needed to re-program the timer > =C2=A0=C2=A0=C2=A0=C2=A0removed unnecessary timer checks > =C2=A0=C2=A0=C2=A0=C2=A0added helper functions to remove vcpus from queue= s > =C2=A0=C2=A0=C2=A0=C2=A0coding style >=20 > Changes since v4: > =C2=A0=C2=A0=C2=A0=C2=A0removed unnecessary replenishment queue checks in= vcpu_wake() > =C2=A0=C2=A0=C2=A0=C2=A0extended replq_remove() to all cases in vcpu_slee= p() > =C2=A0=C2=A0=C2=A0=C2=A0used _deadline_queue_insert() helper function for= both queues > =C2=A0=C2=A0=C2=A0=C2=A0_replq_insert() and _replq_remove() program timer= internally >=20 > Changes since v3: > =C2=A0=C2=A0=C2=A0=C2=A0removed running queue. > =C2=A0=C2=A0=C2=A0=C2=A0added repl queue to keep track of repl events. > =C2=A0=C2=A0=C2=A0=C2=A0timer is now per scheduler. > =C2=A0=C2=A0=C2=A0=C2=A0timer is init on a valid cpu in a cpupool. >=20 So, this does not belong here. It is ok to have it in this part of the email, but it should not land in the actual commit changelog, once the patch will be committed into Xen's git tree. The way to achieve the above is to put this summary of changes below the actual changelog, and below the Signed-of-by lines, after a marker that looks like this "---". > Budget replenishment and enforcement are separated by adding > a replenishment timer, which fires at the next most imminent > release time of all runnable vcpus. >=20 > A new runningq has been added to keep track of all vcpus that > are on pcpus. >=20 Mmm.. Is this the proper changelog? runningq is something we discussed, and that appeared in v2, but is certainly no longer here... :-O > diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c > index 2e5430f..16f77f9 100644 > --- a/xen/common/sched_rt.c > +++ b/xen/common/sched_rt.c >=C2=A0 > @@ -213,8 +220,14 @@ static inline struct list_head > *rt_depletedq(const struct scheduler *ops) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0return &rt_priv(ops)->depletedq; > =C2=A0} > =C2=A0 > +static inline struct list_head *rt_replq(const struct scheduler > *ops) > +{ > +=C2=A0=C2=A0=C2=A0=C2=A0return &rt_priv(ops)->replq; > +} > + > =C2=A0/* > - * Queue helper functions for runq and depletedq > + * Queue helper functions for runq, depletedq > + * and replenishment event queue > Full stop. :-) In any case, I'd change this in something like: "Helper functions for manipulating the runqueue, the depleted queue, and the replenishment events queue." > @@ -228,6 +241,18 @@ __q_elem(struct list_head *elem) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0return list_entry(elem, struct rt_vcpu, q_e= lem); > =C2=A0} > =C2=A0 > +static struct rt_vcpu * > +__replq_elem(struct list_head *elem) > +{ > +=C2=A0=C2=A0=C2=A0=C2=A0return list_entry(elem, struct rt_vcpu, replq_el= em); > +} > + > +static int > +__vcpu_on_replq(const struct rt_vcpu *svc) > +{ > +=C2=A0=C2=A0=C2=A0return !list_empty(&svc->replq_elem); > +} > + > Ok, sorry for changing my mind again, but I really can't stand seeing these underscores. Please, rename these as replq_elem() and vcpu_on_replq(). There is nor sensible reason why we should prefix these with '__'. I know, that will create some amount of inconsistency, but: =C2=A0- there is inconsistency already (here and in other sched_* file) =C2=A0- not introducing more __ prefixed function is a step in the right=C2= =A0 =C2=A0 =C2=A0direction; we'll convert the one that are already there with t= ime. > + * Removing a vcpu from the replenishment queue could > + * re-program the timer for the next replenishment event > + * if there is any on the list >=20 > + */ > +static inline void > +__replq_remove(const struct scheduler *ops, struct rt_vcpu *svc) > +{ > +=C2=A0=C2=A0=C2=A0=C2=A0struct rt_private *prv =3D rt_priv(ops); > +=C2=A0=C2=A0=C2=A0=C2=A0struct list_head *replq =3D rt_replq(ops); > +=C2=A0=C2=A0=C2=A0=C2=A0struct timer* repl_timer =3D prv->repl_timer; > + > +=C2=A0=C2=A0=C2=A0=C2=A0/* > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* Disarm the timer before re-programming i= t. > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* It doesn't matter if the vcpu to be remo= ved > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* is on top of the list or not because the= timer > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* is stopped and needs to be re-programmed= anyways > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0*/ > +=C2=A0=C2=A0=C2=A0=C2=A0stop_timer(repl_timer); > + > +=C2=A0=C2=A0=C2=A0=C2=A0deadline_queue_remove(&svc->replq_elem); > + > +=C2=A0=C2=A0=C2=A0=C2=A0/* re-arm the timer for the next replenishment e= vent */ > +=C2=A0=C2=A0=C2=A0=C2=A0if( !list_empty(replq) ) > +=C2=A0=C2=A0=C2=A0=C2=A0{ > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0struct rt_vcpu *svc_next= =3D __replq_elem(replq->next); > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0set_timer(repl_timer, sv= c_next->cur_deadline); > +=C2=A0=C2=A0=C2=A0=C2=A0} > Wait, maybe you misunderstood and/or I did not make myself clear enough (in which case, sorry). I never meant to say "always stop the timer". Atually, in one of my last email I said the opposite, and I think that would be the best thing to do. Do you think there is any specific reason why we need to always stop and restart it? If not, I think we can: =C2=A0- have deadline_queue_remove() also return whether the element=C2=A0 =C2=A0 =C2=A0removed was the first one (i.e., the one the timer was program= med=C2=A0 =C2=A0 =C2=A0after); =C2=A0- if it was, re-program the timer after the new front of the queue; =C2=A0- if it wasn't, nothing to do. Thoughts? > +/* > + * An utility function that inserts a vcpu to a > + * queue based on certain order (EDF). The returned > + * value is 1 if a vcpu has been inserted to the > + * front of a list > + */ > +static int > +deadline_queue_insert(struct rt_vcpu * (*_get_q_elem)(struct > list_head *elem), > +=C2=A0=C2=A0=C2=A0=C2=A0struct rt_vcpu *svc, struct list_head *elem, str= uct list_head > *queue) > +{ > +=C2=A0=C2=A0=C2=A0=C2=A0struct list_head *iter; > +=C2=A0=C2=A0=C2=A0=C2=A0int pos =3D 0; > + > +=C2=A0=C2=A0=C2=A0=C2=A0list_for_each(iter, queue) > +=C2=A0=C2=A0=C2=A0=C2=A0{ > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0struct rt_vcpu * iter_sv= c =3D (*_get_q_elem)(iter); > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if ( svc->cur_deadline <= =3D iter_svc->cur_deadline ) > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= break; > + > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0pos++; > +=C2=A0=C2=A0=C2=A0=C2=A0} > + > +=C2=A0=C2=A0=C2=A0=C2=A0list_add_tail(elem, iter); > +=C2=A0=C2=A0=C2=A0=C2=A0return !pos; > =C2=A0} >=20 Ok. > @@ -405,22 +500,38 @@ __runq_insert(const struct scheduler *ops, > struct rt_vcpu *svc) > =C2=A0 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0/* add svc to runq if svc still has budget = */ > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if ( svc->cur_budget > 0 ) > -=C2=A0=C2=A0=C2=A0=C2=A0{ > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0list_for_each(iter, runq= ) > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0{ > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= struct rt_vcpu * iter_svc =3D __q_elem(iter); > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= if ( svc->cur_deadline <=3D iter_svc->cur_deadline ) > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0break; > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0} > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0list_add_tail(&svc->q_el= em, iter); > -=C2=A0=C2=A0=C2=A0=C2=A0} > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0deadline_queue_insert(&_= _q_elem, svc, &svc->q_elem, runq); > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0else > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0{ > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0list_add(&svc->q_el= em, &prv->depletedq); > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0ASSERT( __vcpu_on_replq(= svc) ); > So, by doing this, you're telling me that, if the vcpu is added to the depleted queue, there must be a replenishment planned for it (or the ASSERT() would fail). But if we are adding it to the runq, there may or may not be a replenishment planned for it. Is this what we want? Why there must not be a replenishment planned already for a vcpu going into the runq (to happen at its next deadline)? > =C2=A0/* > + * Insert svc into the replenishment event list > + * in replenishment time order. > + * vcpus that need to be replished earlier go first. > + * The timer may be re-programmed if svc is inserted > + * at the front of the event list. > + */ > +static void > +__replq_insert(const struct scheduler *ops, struct rt_vcpu *svc) > +{ > +=C2=A0=C2=A0=C2=A0=C2=A0struct list_head *replq =3D rt_replq(ops); > +=C2=A0=C2=A0=C2=A0=C2=A0struct rt_private *prv =3D rt_priv(ops); > +=C2=A0=C2=A0=C2=A0=C2=A0struct timer *repl_timer =3D prv->repl_timer; > +=C2=A0=C2=A0=C2=A0=C2=A0int set; > + > +=C2=A0=C2=A0=C2=A0=C2=A0ASSERT( !__vcpu_on_replq(svc) ); > + > +=C2=A0=C2=A0=C2=A0=C2=A0set =3D deadline_queue_insert(&__replq_elem, svc= , &svc- > >replq_elem, replq); > +=C2=A0 > +=C2=A0=C2=A0=C2=A0=C2=A0if( set ) > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0set_timer(repl_timer,svc= ->cur_deadline); > +} A matter of taste, mostly, but I'd avoid the local variable (if this function will at some point become more complex, then we'll see, but for now, I think it's ok to just use the return value of deadline_queue_insert() inside the if(). > @@ -868,6 +968,8 @@ rt_schedule(const struct scheduler *ops, s_time_t > now, bool_t tasklet_work_sched > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0set_bit(__RTDS_dela= yed_runq_add, &scurr->flags); > =C2=A0 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0snext->last_start =3D now; > + > You don't need to add neither this blank line... > +=C2=A0=C2=A0=C2=A0=C2=A0ret.time =3D=C2=A0=C2=A0-1; /* if an idle vcpu i= s picked */=C2=A0 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if ( !is_idle_vcpu(snext->vcpu) ) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0{ > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if ( snext !=3D scu= rr ) > @@ -880,9 +982,11 @@ rt_schedule(const struct scheduler *ops, > s_time_t now, bool_t tasklet_work_sched > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0snext->vcpu->processor =3D cpu; > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0ret.migrated =3D 1; > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0} > + > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0ret.time =3D snext->budg= et; /* invoke the scheduler next time > */ > + ...nor this one. > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0} > @@ -924,6 +1028,8 @@ rt_vcpu_sleep(const struct scheduler *ops, > struct vcpu *vc) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0__q_remove(svc); > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0else if ( svc->flags & RTDS_delayed_runq_ad= d ) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0clear_bit(__RTDS_de= layed_runq_add, &svc->flags); > + > +=C2=A0=C2=A0=C2=A0=C2=A0__replq_remove(ops, svc); > What I said in my last email is that you probably can get rid of this (see below, whe commenting on context_saved()). =C2=A0 > @@ -1051,6 +1153,22 @@ rt_vcpu_wake(const struct scheduler *ops, > struct vcpu *vc) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0else > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0SCHED_STAT_CRANK(vc= pu_wake_not_runnable); > =C2=A0 > +=C2=A0=C2=A0=C2=A0=C2=A0/* > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* If a deadline passed while svc was aslee= p/blocked, we need > new > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* scheduling parameters ( a new deadline a= nd full budget), and > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* also a new replenishment event > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0*/ > +=C2=A0=C2=A0=C2=A0=C2=A0if ( now >=3D svc->cur_deadline) > +=C2=A0=C2=A0=C2=A0=C2=A0{=C2=A0=C2=A0=C2=A0 > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0rt_update_deadline(now, = svc); > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0__replq_remove(ops, svc)= ; > +=C2=A0=C2=A0=C2=A0=C2=A0} > + > +=C2=A0=C2=A0=C2=A0=C2=A0if( !__vcpu_on_replq(svc) ) > +=C2=A0=C2=A0=C2=A0=C2=A0{ > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0__replq_insert(ops, svc)= ; > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0ASSERT( vcpu_runnable(vc= ) ); > Mmm... What's this assert about? > @@ -1087,10 +1193,6 @@ static void > =C2=A0rt_context_saved(const struct scheduler *ops, struct vcpu *vc) > =C2=A0{ > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0struct rt_vcpu *svc =3D rt_vcpu(vc); > -=C2=A0=C2=A0=C2=A0=C2=A0struct rt_vcpu *snext =3D NULL; > -=C2=A0=C2=A0=C2=A0=C2=A0struct rt_dom *sdom =3D NULL; > -=C2=A0=C2=A0=C2=A0=C2=A0struct rt_private *prv =3D rt_priv(ops); > -=C2=A0=C2=A0=C2=A0=C2=A0cpumask_t *online; > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0spinlock_t *lock =3D vcpu_schedule_lock_irq= (vc); > =C2=A0 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0clear_bit(__RTDS_scheduled, &svc->flags); > @@ -1102,14 +1204,7 @@ rt_context_saved(const struct scheduler *ops, > struct vcpu *vc) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0likely(vcpu_r= unnable(vc)) ) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0{ > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0__runq_insert(ops, = svc); > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0__repl_update(ops, NOW()= ); > - > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0ASSERT(!list_empty(&prv-= >sdom)); > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0sdom =3D list_entry(prv-= >sdom.next, struct rt_dom, sdom_elem); > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0online =3D cpupool_domai= n_cpumask(sdom->dom); > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0snext =3D __runq_pick(op= s, online); /* pick snext from ALL > cpus */ > - > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0runq_tickle(ops, snext); > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0runq_tickle(ops, svc); > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0} > So, here we are. What I meant was to make this function look more or less like this: static void rt_context_saved(const struct scheduler *ops, struct vcpu *vc) { =C2=A0=C2=A0=C2=A0=C2=A0struct rt_vcpu *svc =3D rt_vcpu(vc); =C2=A0=C2=A0=C2=A0=C2=A0spinlock_t *lock =3D vcpu_schedule_lock_irq(vc); =C2=A0=C2=A0=C2=A0=C2=A0clear_bit(__RTDS_scheduled, &svc->flags); =C2=A0=C2=A0=C2=A0=C2=A0/* not insert idle vcpu to runq */ =C2=A0=C2=A0=C2=A0=C2=A0if ( is_idle_vcpu(vc) ) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0goto out; =C2=A0=C2=A0=C2=A0=C2=A0if ( test_and_clear_bit(__RTDS_delayed_runq_add, &s= vc->flags) && =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0likely(vcpu_runnable(= vc)) ) =C2=A0=C2=A0=C2=A0=C2=A0{ =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0__runq_insert(ops, svc); =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0runq_tickle(ops, snext); =C2=A0=C2=A0=C2=A0=C2=A0} else __replq_remove(ops, svc); out: =C2=A0=C2=A0=C2=A0=C2=A0vcpu_schedule_unlock_irq(lock, vc); } And, as said above, if you do this, try also removing the __replq_remove() call from rt_vcpu_sleep(), this one should cover for that (and, actually, more than just that!). After all this, check whether you still have the assert in __replq_insert() triggering and let me know Thanks and Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-5ApKvZN7GblqLodGcrkr Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEABECAAYFAlbPjscACgkQk4XaBE3IOsS2jwCdF4dnM75CXQ3z2+RYekxzvnfZ ikgAoIz1gCS+JVrL3lgT9B4F3tJph2qJ =Jbvu -----END PGP SIGNATURE----- --=-5ApKvZN7GblqLodGcrkr-- --===============1405397958707458069== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwOi8vbGlzdHMueGVuLm9y Zy94ZW4tZGV2ZWwK --===============1405397958707458069==--