From mboxrd@z Thu Jan 1 00:00:00 1970 From: Keir Fraser Subject: Re: Foreign VCPU register change? Date: Thu, 23 Aug 2012 19:55:34 +0100 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8265482884604195495==" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Cutter 409 , xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org > This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --===============8265482884604195495== Content-type: multipart/alternative; boundary="B_3428596536_124287228" > This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --B_3428596536_124287228 Content-type: text/plain; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable If you have your own flag in v->pause_flags then indeed you do not need to vcpu_pause_nosync() in the vmexit handler. The best sequence would be: - vmexit handler: set flag in v->pause_flags, then vcpu_sleep_nosync() - domctl entry: vcpu_sleep_sync() - domctl exit: clear flag in v->pause_flags, then vcpu_wake() So that=B9s pretty much what you had in the first place, except for the extra vcpu_sleep_sync() on domctl entry. That=B9s absolutely critical, and why your pause_nosync on vmexit, unpause after domctl doesn=B9t work =8B something is needed on domctl entry to be sure that the vcpu is descheduled and its stat= e is synchronised. Of course the extra machinery of vcpu_pause/unpause is harmless enough, but it=B9s not actually necessary here. -- Keir On 23/08/2012 19:11, "Cutter 409" wrote: > Thanks, Keir! >=20 > I've spent so much time trying to track down this problem, even before I > realized the registers weren't actually changing. You have no idea how he= lpful > that was. >=20 > Before I tried your example, I just wrapped the code to change the regist= er in > vcpu_pause() and vcpu_unpause(), which worked. >=20 > Everything seems fine at the moment, is there any reason I should still c= hange > the vcpu_sleep_nosync() to vcpu_pause_nosync()? It seems to actually work= as > is; I'm setting a bit in v->pause_flags before I call it, then clear the = bit > before I wake it. I also tried pause_nosync on vmexit, unpause after domc= tl, > but that didn't work. >=20 > Thanks again! >=20 >=20 > On Thu, Aug 23, 2012 at 1:54 PM, Keir Fraser wrote: >> So, for example, one possibly-valid scheme would be: >>=20 >> =A0- vcpu_pause_nosync() from the vmexit handler >>=20 >> =A0- vcpu_sleep_sync() at the start of the domctl >> =A0- vcpu_unpause() at the end of the domctl >>=20 >> HTH, >> =A0Keir >>=20 >>=20 >>=20 >> On 23/08/2012 18:49, "Keir Fraser" > > wrote: >>=20 >>> FWIW I would expect your approach to basically work. >>>=20 >>> Except... Does your domctl do a vcpu_pause()/vcpu_unpause() on the vcpu= ? >>> This will ensure that the vcpu is both fully de-scheduled, and all of i= ts >>> register state is synced back into its vcpu structure. >>>=20 >>> Otherwise you race the vcpu_sleep_nosync() -- and that=B9s assuming you a= lso >>> have a reason for that vcpu to sleep (e.g., non-zero pause counter), el= se >>> vcpu_sleep_*() operations do nothing! >>>=20 >>> In short, your problems are almost certainly something to do with the >>> subtleties of actually putting a vcpu properly to sleep. >>>=20 >>> =A0-- Keir >>>=20 >>>=20 >>> On 23/08/2012 18:37, "Cutter 409" >> > wrote: >>>=20 >>>> I'm making the register change directly from the hypervisor, inside of= the >>>> domctl code.=20 >>>>=20 >>>> It's a custom domctl that I've added. I'll look into what setcontext d= oes >>>> after it modifies the register values, though. >>>>=20 >>>> Thank you! >>>>=20 >>>> On Thu, Aug 23, 2012 at 1:34 PM, Keir Fraser >>> > wrote: >>>>> On 23/08/2012 18:25, "Cutter 409" >>>> > wrote: >>>>>=20 >>>>>> > With Xen-4.1.2: >>>>>> > >>>>>> > I'm trying to change a register value in a paused vmx vcpu. The ge= neral >>>>>> > process looks like this: >>>>>> > >>>>>> > 1. Some vmexit calls vcpu_sleep_nosync(v) on the vcpu >>>>>> > 2. From dom0, I issue a domctl to change a register via >>>>>> > v->arch.guest_context.user_reg, then vcpu_wake(v) >>>>>=20 >>>>> Which domctl? From dom0 userspace you can use the libxc functions >>>>> xc_vcpu_getcontext() and xc_vcpu_setcontext() to read/modify register >>>>> state. >>>>>=20 >>>>> You can read the libxc sources to see what hypercall these map to, if= you >>>>> don't want to use libxc for any reason. >>>>>=20 >>>>> =A0-- Keir >>>>>=20 >>>>>> > However, the guest register does not seem to be changed when I do = it >>>>>> this way. >>>>>> > Is there something I need to do to mark the registers as "dirty" ?= Is >>>>>> there a >>>>>> > way to force the foreign vcpu to update the changed registers? Or = maybe >>>>>> I just >>>>>> > have to change the registers somewhere else? >>>>>> > >>>>>> > I've tried directly using vmcs_enter(v), __vmwrite(), vmcs_exit(v) >>>>>> also, but >>>>>> > that doesn't seem to make a change either. >>>>>> > >>>>>> > Thanks! >>>>>> > >>>>>> > >>>>>> > >>>>>> > _______________________________________________ >>>>>> > Xen-devel mailing list >>>>>> > Xen-devel@lists.xen.org >>>>>> > http://lists.xen.org/xen-devel >>>>>=20 >>>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> _______________________________________________ >>>> Xen-devel mailing list >>>> Xen-devel@lists.xen.org >>>> http://lists.xen.org/xen-devel >>>=20 >=20 >=20 >=20 > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel --B_3428596536_124287228 Content-type: text/html; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable Re: [Xen-devel] Foreign VCPU register change? If you have your own flag in v->pause_flags then indeed you do not need= to vcpu_pause_nosync()  in the vmexit handler.

The best sequence would be:
 - vmexit handler: set flag in v->pause_flags, then vcpu_sleep_nosy= nc()
 - domctl entry: vcpu_sleep_sync()
 - domctl exit: clear flag in v->pause_flags, then vcpu_wake()

So that’s pretty much what you had in the first place, except for the= extra vcpu_sleep_sync() on domctl entry. That’s absolutely critical, = and why your pause_nosync on vmexit, unpause after domctl doesn’t work= — something is needed on domctl entry to be sure that the vcpu= is descheduled and its state is synchronised. Of course the extra machinery= of vcpu_pause/unpause is harmless enough, but it’s not actually neces= sary here.

 -- Keir


On 23/08/2012 19:11, "Cutter 409" <cutter409@gmail.com> wrote:

<= SPAN STYLE=3D'font-size:11pt'>Thanks, Keir!

I've spent so much time trying to track down this problem, even before I re= alized the registers weren't actually changing. You have no idea how helpful= that was.

Before I tried your example, I just wrapped the code to change the register= in vcpu_pause() and vcpu_unpause(), which worked.

Everything seems fine at the moment, is there any reason I should still cha= nge the vcpu_sleep_nosync() to vcpu_pause_nosync()? It seems to actually wor= k as is; I'm setting a bit in v->pause_flags before I call it, then clear= the bit before I wake it. I also tried pause_nosync on vmexit, unpause afte= r domctl, but that didn't work.

Thanks again!


On Thu, Aug 23, 2012 at 1:54 PM, Keir Fraser <keir.xen@gmail.com> wrote:
<= SPAN STYLE=3D'font-size:11pt'>So, for example, one possibly-valid scheme would= be:

=A0- vcpu_pause_nosync() from the vmexit handler

=A0- vcpu_sleep_sync() at the start of the domctl
=A0- vcpu_unpause() at the end of the domctl

HTH,
=A0Keir



On 23/08/2012 18:49, "Keir Fraser" <keir.xen@gmail.com <http://kei= r.xen@gmail.com> > wrote:

<= SPAN STYLE=3D'font-size:11pt'>FWIW I would expect your approach to basically w= ork.

Except... Does your domctl do a vcpu_pause()/vcpu_unpause() on the vcpu? Th= is will ensure that the vcpu is both fully de-scheduled, and all of its regi= ster state is synced back into its vcpu structure.

Otherwise you race the vcpu_sleep_nosync() -- and that’s assuming you= also have a reason for that vcpu to sleep (e.g., non-zero pause counter), e= lse vcpu_sleep_*() operations do nothing!

In short, your problems are almost certainly something to do with the subtl= eties of actually putting a vcpu properly to sleep.

=A0-- Keir


On 23/08/2012 18:37, "Cutter 409" <cutter409@gmail.com <http://c= utter409@gmail.com> > wrote:

<= SPAN STYLE=3D'font-size:11pt'>I'm making the register change directly from the= hypervisor, inside of the domctl code.

It's a custom domctl that I've added. I'll look into what setcontext does a= fter it modifies the register values, though.

Thank you!

On Thu, Aug 23, 2012 at 1:34 PM, Keir Fraser <keir.xen@gmail.com <http://kei= r.xen@gmail.com> > wrote:
<= SPAN STYLE=3D'font-size:11pt'>On 23/08/2012 18:25, "Cutter 409" <= cutter409@gmail.com <http://cutter409@gmail.com> > wrote:

> With Xen-4.1.2:
>
> I'm trying to change a register value in a paused vmx vcpu. The genera= l
> process looks like this:
>
> 1. Some vmexit calls vcpu_sleep_nosync(v) on the vcpu
> 2. From dom0, I issue a domctl to change a register via
> v->arch.guest_context.user_reg, then vcpu_wake(v)

Which domctl? From dom0 userspace you can use the libxc functions
xc_vcpu_getcontext() and xc_vcpu_setcontext() to read/modify register state= .

You can read the libxc sources to see what hypercall these map to, if you don't want to use libxc for any reason.

=A0-- Keir

> However, the guest register does not seem to be changed when I do it t= his way.
> Is there something I need to do to mark the registers as "dirty&q= uot; ? Is there a
> way to force the foreign vcpu to update the changed registers? Or mayb= e I just
> have to change the registers somewhere else?
>
> I've tried directly using vmcs_enter(v), __vmwrite(), vmcs_exit(v) als= o, but
> that doesn't seem to make a change either.
>
> Thanks!
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org <http://Xen-devel@lists.xen.org> =
> http://lists.xen.org/xen-deve= l


=

___________= ____________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org <http://Xen-devel@lists.xen.org>
http://lists.xen.org/xen-devel=



___________= ____________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel=
--B_3428596536_124287228-- --===============8265482884604195495== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============8265482884604195495==--