From mboxrd@z Thu Jan 1 00:00:00 1970 From: Keir Fraser Subject: Re: Walking an HVM's shadow page tables and other memory management questions. Date: Tue, 03 Jul 2007 10:04:29 +0100 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0751229137==" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Mime-version: 1.0 Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Roger Cruz , xen-devel@lists.xensource.com Cc: Tim Deegan List-Id: xen-devel@lists.xenproject.org > This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --===============0751229137== Content-type: multipart/alternative; boundary="B_3266301870_230165" > This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --B_3266301870_230165 Content-type: text/plain; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable You are barking up the wrong tree by attempting to poke the mapping into a guest pte. After all, what would you poke? Guest PTEs address the guest-pseudo-physical space, in which the foreign page is not present. You actually want to follow ia64=B9s lead here. When running in =8Cauto-translate=B9 mode (i.e., on shadow page tables) then the guest address for a host mapping should not be interpreted as a virtual address but instead as a pseudo-physical address. So you will be mapping a grant reference into the pseudo-physical space and then a guest PTE can map the appropriate pseudo-physical frame number in th= e usual way. The slightly tricky bit is working out how to encode a grant-mapping in the p2m table. My advice would be to use a page-not-presen= t encoding (p2m table entries are the same format as page-table entries) as this then lets you define special encodings of your choice with most of the remaining bits. Tim Deegan may be able to give more advice. -- Keir On 2/7/07 21:25, "Roger Cruz" wrote: > Hello, > =20 > I=B9m new to Xen and especially to the hypervisor code. I=B9m working off a > 3.0.4.1 base and have the following questions regarding the memory manage= ment > code for an x86, 32-bit platform (capable of supporting PAE). I=B9m doing = some > research into providing grant table hypercall support from a Windows 2003= HVM. > I have made all the necessary changes to allow the hypercall to make it i= nto > the hypervisor and execute the correct grant table ops. > =20 > I=B9m now testing the GNTTABOP_map_grant_ref with the GNTMAP_host_map and i= t > correctly obtains the MFN from the grantor domain. It then attempts to t= ake > the HVM host VA address (a windows kernel VA from the non-paged pool) and= walk > the guest=B9s page table to obtain the PFN. I am building the hypervisor b= y > simply typing =B3make xen=B2 without any other configuration changes from a > default source installation. > =20 > The first problem I encountered is that it appears the code assumes the g= uest > to be in PAE mode. In particular, guest_walk_tables() in > xen/arch/x86/mm/shadow/multi.c, line 252 has this code snippet: > =20 > #else /* PAE only... */ > /* Get l3e from the cache of the guest's top level table */ > gw->l3e =3D (guest_l3e_t *)&v->arch.shadow.gl3e[guest_l3_table_offset(v= a)]; > #endif /* PAE or 64... */ > =20 > Which accesses the L3 entries fro the shadow page tables. When I instrum= ent > this code, I get l3e to be 0 as shown below (the line #s won=B9t match beca= use > of the instrumentation). > =20 > (XEN) multi.c:236:d1 guest_walk_tables: va: 0x81699000. > (XEN) multi.c:257:d1 guest_walk_tables: get l3e from cache: 0xff1a6ed0. > (XEN) multi.c:263:d1 guest_walk_tables: l3e not present: 0x0. > (XEN) multi.c:574:d1 sh_guest_map_l1e: va:81699000 > =20 > If I add the /PAE switch to the boot.ini file, then I can get past this > problem. Hence my statement that it appears the hypervisor is assuming g= uests > are running with at least PAE mode enable, which may not be the case. Co= uld > someone please guide me here? > =20 > The 2nd problem I encountered also has to do with walking the shadow page > tables to obtain the MFN of the underlying Windows VA address. > sh_guest_map_l1e(), Line 520 in the same file, has this code executed aft= er it > walks the guest page tables to obtain the walk_t gw variable. > =20 > if ( gw.l2e && > (guest_l2e_get_flags(*gw.l2e) & _PAGE_PRESENT) && > !(guest_supports_superpages(v) && (guest_l2e_get_flags(*gw.l2e) & > _PAGE_PSE)) ) > =20 > =20 > (XEN) mm.c:2573:d1 grant host mapping: va:81696000 frame:0x15f140 > (XEN) mm.c:2507:d1 grant va mapping: va:81696000 > (XEN) multi.c:236:d1 guest_walk_tables: va: 0x81696000. > (XEN) multi.c:257:d1 guest_walk_tables: get l3e from cache: 0xff1a6ed0. > (XEN) multi.c:270:d1 guest_walk_tables: l3e flags: 0x1, pfn:0xe9a > (XEN) , mfn:0x9e13d<1>multi.c:285:d1 hypervisor l2e mapped address > 0xfec8b058 > (XEN) multi.c:315:d1 large pages. 0x1e3 > (XEN) multi.c:574:d1 sh_guest_map_l1e: va:81696000 > (XEN) multi.c:579:d1 sh_guest_map_l1e: gw.l2e flags:0x1e3, supports large= 1 > (XEN) multi.c:596:d1 pl1e :0x0, > (XEN) mm.c:2512:d1 Could not find L1 PTE for address 81696000 > =20 > =20 > It looks like it specifically avoids mapping a superpage found in Windows= PDE > into the hypervisor=B9s virtual space, which I assume are 4KB-pages. What > puzzles me is that for a hypercall to read the arguments from the caller=B9= s > guest space, it uses __hvm_copy which calls shadow_gva_to_gfn() to walk t= he > guest=B9s shadow page tables to get to the underlying MFN. Couldn=B9t this co= de > here also do the same? > =20 > Thanks in advance for any insight into this area. > =20 > =20 > Roger Cruz > Principal SW Engineer > Marathon Technologies Corp. > 978-489-1153 > =20 >=20 >=20 > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel --B_3266301870_230165 Content-type: text/html; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable Re: [Xen-devel] Walking an HVM's shadow page tables and other memory= management questions. You a= re barking up the wrong tree by attempting to poke the mapping into a guest = pte. After all, what would you poke? Guest PTEs address the guest-pseudo-phy= sical space, in which the foreign page is not present.

You actually want to follow ia64’s lead here. When running in ‘= auto-translate’ mode (i.e., on shadow page tables) then the guest addr= ess for a host mapping should not be interpreted as a virtual address but in= stead as a pseudo-physical address.

So you will be mapping a grant reference into the pseudo-physical space and= then a guest PTE can map the appropriate pseudo-physical frame number in th= e usual way. The slightly tricky bit is working out how to encode a grant-ma= pping in the p2m table. My advice would be to use a page-not-present encodin= g (p2m table entries are the same format as page-table entries) as this then= lets you define special encodings of your choice with most of the remaining= bits.

Tim Deegan may be able to give more advice.

 -- Keir

On 2/7/07 21:25, "Roger Cruz" <rcruz@marathontechnologies.com&= gt; wrote:

Hello,
 
I’m new to Xen and especially to the hypervisor code.  I’m= working off a 3.0.4.1 base and have the following questions regarding the m= emory management code for an x86, 32-bit platform (capable of supporting PAE= ).  I’m doing some research into providing grant table hypercall = support from a Windows 2003 HVM.  I have made all the necessary changes= to allow the hypercall to make it into the hypervisor and execute the corre= ct grant table ops.
 
I’m now testing the GNTTABOP_map_grant_ref with the GNTMAP_host_map a= nd it correctly obtains the MFN from the grantor domain.  It then attem= pts to take the HVM host VA address (a windows kernel VA from the non-paged = pool) and walk the guest’s page table to obtain the PFN.  I am bu= ilding the hypervisor by simply typing “make xen” without any ot= her configuration changes from a default source installation.
 
The first problem I encountered is that it appears the code assumes the gue= st to be in PAE mode.  In particular, guest_walk_tables() in xen/arch/x= 86/mm/shadow/multi.c, line 252 has this code snippet:
 
#else /* PAE only... */
    /* Get l3e from the cache of the guest's top level = table */
    gw->l3e =3D (guest_l3e_t *)&v->arch.shadow.= gl3e[guest_l3_table_offset(va)];
#endif /* PAE or 64... */
 
Which accesses the L3 entries fro the shadow page tables.  When I inst= rument this code, I get l3e to be 0 as shown below (the line #s won’t = match because of the instrumentation).
 
(XEN) multi.c:236:d1 guest_walk_tables: va: 0x81699000.
(XEN) multi.c:257:d1 guest_walk_tables: get l3e from cache: 0xff1a6ed0.
(XEN) multi.c:263:d1 guest_walk_tables: l3e not present: 0x0.
(XEN) multi.c:574:d1 sh_guest_map_l1e: va:81699000
 
If I add the /PAE switch to the boot.ini file, then I can get past this pro= blem.  Hence my statement that it appears the hypervisor is assuming gu= ests are running with at least PAE mode enable, which may not be the case. &= nbsp;Could someone please guide me here?
 
The 2nd problem I encountered also has to do with walking the shadow page t= ables to obtain the MFN of the underlying Windows VA address. sh_guest_map_l= 1e(), Line 520 in the same file, has this code executed after it walks the g= uest page tables to obtain the walk_t gw variable.
 
    if ( gw.l2e &&
        (guest_l2e_get_flags(*gw.l2= e) & _PAGE_PRESENT) &&
        !(guest_supports_superpages= (v) && (guest_l2e_get_flags(*gw.l2e) & _PAGE_PSE)) )
 
 
(XEN) mm.c:2573:d1 grant host mapping: va:81696000 frame:0x15f140
(XEN) mm.c:2507:d1 grant va mapping: va:81696000
(XEN) multi.c:236:d1 guest_walk_tables: va: 0x81696000.
(XEN) multi.c:257:d1 guest_walk_tables: get l3e from cache: 0xff1a6ed0.
(XEN) multi.c:270:d1 guest_walk_tables: l3e flags: 0x1, pfn:0xe9a
(XEN) , mfn:0x9e13d<G><1>multi.c:285:d1 hypervisor l2e mapped a= ddress 0xfec8b058
(XEN) multi.c:315:d1 large pages. 0x1e3
(XEN) multi.c:574:d1 sh_guest_map_l1e: va:81696000
(XEN) multi.c:579:d1 sh_guest_map_l1e: gw.l2e flags:0x1e3, supports large 1=
(XEN) multi.c:596:d1 pl1e :0x0,
(XEN) mm.c:2512:d1 Could not find L1 PTE for address 81696000
 
 
It looks like it specifically avoids mapping a superpage found in Windows P= DE into the hypervisor’s virtual space, which I assume are 4KB-pages. =  What puzzles me is that for a hypercall to read the arguments from the= caller’s guest space, it uses __hvm_copy which calls shadow_gva_to_gf= n() to walk the guest’s shadow page tables to get to the underlying MF= N. Couldn’t this code here also do the same?
 
Thanks in advance for any insight into this area.
 
 
Roger Cruz
Principal SW Engineer
Marathon Technologies Corp.
978-489-1153

____________________= ___________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/x= en-devel

 --B_3266301870_230165-- --===============0751229137== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============0751229137==--