From mboxrd@z Thu Jan 1 00:00:00 1970 From: Abhinav Srivastava Subject: Re: device-mmio emulation in Xen Date: Tue, 5 Jan 2010 22:02:01 +0530 (IST) Message-ID: <694563.92055.qm@web7905.mail.in.yahoo.com> References: <20100105160015.GB30054@whitby.uk.xensource.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <20100105160015.GB30054@whitby.uk.xensource.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Tim Deegan Cc: "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org =0AHi Tim,=0A=0AThanks for the reply. I understood what you had described. = However, I have one question:=0A=0A1. How does Xen know which code path to = execute? What I am noticing is that sometimes it goes to all the way till t= he end of the page fault handler code to call handle_mmio function, and som= etimes it goes into the fast path to call this function, bypassing all the = page fault handlers code? Why it does not go into the fast path all the tim= e? OR Xen does something at the first time so that second time, the executi= on can directly go into the fast path? And, does this needs to be done for = all physical pages given to MMIO?=0A=0AThanks,=0AAbhinav=0A=0A=0A=0A--- On = Tue, 5/1/10, Tim Deegan wrote:=0A=0A> From: Tim Dee= gan =0A> Subject: Re: [Xen-devel] device-mmio emulat= ion in Xen=0A> To: "Abhinav Srivastava" =0A> C= c: "xen-devel@lists.xensource.com" =0A> Date= : Tuesday, 5 January, 2010, 9:30 PM=0A> Hi, =0A> =0A> At 15:32 +0000 on 05 = Jan (1262705557), Abhinav Srivastava=0A> wrote:=0A> > I noticed that during= a network copy operation Xen=0A> page faults a lot and control goes to sh_= page_fault=0A> function. When I printed some debugging info, it showed me= =0A> gmfn =3D -1. Then the execution goes through the regular path=0A> of t= he page fault handler code, which means it creates an=0A> entry using shado= w_get_and_create_l1e, propagates it using=0A> l1e_propagate_from_guest, and= finally updates the entry=0A> using shadow_set_l1e. It finally goes into t= he device-model=0A> mmio condition. In this condition, it extracts a guest= =0A> physical address and calls "goto mmio", which in turn calls=0A> handle= _mmio function that emulates the instruction.=0A> > =0A> > However with the= gmfn =3D -1 condition, the execution=0A> sometimes directly goto=0A> > to = handle_mmio function using the fast_fault_path with=0A> going through the r= egular path. It seems like there are two=0A> possible execution paths, and = I did not understand which one=0A> is chosen when?=0A> > =0A> =0A> /*******= ***********************************************************************=0A>= * We implement a "fast path" for two special cases: faults=0A> that requi= re=0A> * MMIO emulation, and faults where the guest PTE is not=0A> present= .=A0 We=0A> * record these as shadow l1 entries that have reserved=0A> bit= s set in=0A> * them, so we can spot them immediately in the fault=0A> hand= ler and handle=0A> * them without needing to hold the shadow lock or walk = the=0A> guest=0A> * pagetables.=0A> *=0A> * This is only feasible for PA= E and 64bit Xen: 32-bit=0A> non-PAE PTEs don't=0A> * have reserved bits th= at we can use for this.=0A> */=0A> =0A> > I have some questions related to= this behavior:=0A> > =0A> > 1. Why are there so many faults duing network = copy=0A> operation?=0A> =0A> The faults are a HVM guest doing MMIO operatio= ns to control=0A> its emulated=0A> network card.=0A> =0A> > 2. What does gm= fn =3D -1 signify? Is it reserved for=0A> mmio addresses?=0A> =0A> It's INV= ALID_MFN.=A0 It means there is no memory mapped=0A> at the address the=0A> = guest accessed.=A0 Xen treats that as MMIO, though=0A> there's a plan to ha= ve=0A> MMIO regions explicitly registered instead.=0A> =0A> > 3. How does X= en handle this gmfn =3D -1? It seems like=0A> on the regular path=0A> > it = still creates, propagates, and updates entries for=0A> gmfn =3D -1. How doe= s Xen handles this at the shadow page=0A> table level?=0A> =0A> Yes, it cre= ates a deliberately invalid shadow PTE; then in=0A> the pagefault=0A> handl= er it can use the pagefault error code and the=0A> contents of the PTE =0A>= to skip the bulk of the shadow fault handling and go=0A> straight to handl= e_mmio.=0A> =0A> > 4. What are these two code execution paths, and when=0A>= does Xen decide which=0A> > path to choose?=0A> > =0A> > 5. Finally, is th= ere anyway these faults can be=0A> reduced?=0A> =0A> Yes, use PV drivers in= the guest.=A0 That eliminates the=0A> whole=0A> emulated-MMIO system, and = all the pagefaults associated=0A> with it. =0A> =0A> Cheers,=0A> =0A> Tim.= =0A> =0A> > I would very appreciate any help in this regard.=0A> > =0A> > T= hanks,=0A> > Abhinav=0A> > =0A> > =0A> > =0A> >=A0 =A0 =A0=A0=A0The INTERNE= T now has a=0A> personality. YOURS! See your Yahoo! Homepage. http://in.yah= oo.com/=0A> > =0A> > _______________________________________________=0A> > = Xen-devel mailing list=0A> > Xen-devel@lists.xensource.com=0A> > http://lis= ts.xensource.com/xen-devel=0A> =0A> -- =0A> Tim Deegan =0A> Principal Software Engineer, Citrix Systems (R&D) Ltd.=0A> [Company= #02300071, SL9 0DZ, UK.]=0A> =0A=0A=0A The INTERNET now has a persona= lity. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/