From mboxrd@z Thu Jan 1 00:00:00 1970 From: John Byrne Subject: Re: Live migration leaves page tables read-only? Date: Fri, 08 Dec 2006 21:40:56 -0800 Message-ID: <457A4C68.6050800@hp.com> References: <456CD0A5.1060701@hp.com> <456CD2DC.2020201@hp.com> <8A87A9A84C201449A0C56B728ACF491E01FA12@liverpoolst.ad.cl.cam.ac.uk> <456CF5F9.7070009@hp.com> <456F6AF5.2090005@hp.com> <8A87A9A84C201449A0C56B728ACF491E01FA69@liverpoolst.ad.cl.cam.ac.uk> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------060302030604080308020300" Return-path: In-Reply-To: <8A87A9A84C201449A0C56B728ACF491E01FA69@liverpoolst.ad.cl.cam.ac.uk> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Ian Pratt Cc: xen-devel List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --------------060302030604080308020300 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Ian Pratt wrote: >>>> What happens if you use non-live relo? >>> I thought I had tested that way back at the beginning without seeing > the >>> problem, but I must not have, because I just retested it to be sure > and >>> it died the same way. (Now I am truly confused and I need to go back > and >>> re-examine some of my earlier experiments.) >>> >> After redoing some of my tests and understanding more about how Xen >> handles page tables, I started looking at ptwr_do_page_fault() and put >> debugging code into it. (On Xen 3.0.3 x86-64.) The fixup is failing > in >> x86_emulate_memop(). Building a debug version of Xen provided some >> additional information (the final line is from my debugging, after the >> ":" is domid, addr, pte, pte flags, type_info, page owner, domain): > > You say you can repro the problem using non-live relo. In that case, you > should also be able to repro it using save/restore, which has almost > identical code paths. > > Please try and isolate whether the crash happens on save or restore, and > further whether a given saved images crashes every time in the same way > when you try and restore it (mfns will be different, but pfns may be the > same). > > > Ian > > I finally ran down the problem. SAP is protecting the pages PROT_NONE, so the page-present bit in the pte is not set and canonicalize/uncanonicalize code in save/restore ignore the pte. I've attached a patch. It is possible that this change should be made to the l1e tests in xc_ptrace.c; I'm not sure. John Byrne Signed-off-by: John Byrne --------------060302030604080308020300 Content-Type: text/x-patch; name="migprotnone.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="migprotnone.patch" diff -r 1ad7dff99968 tools/libxc/xc_linux_restore.c --- a/tools/libxc/xc_linux_restore.c Fri Dec 08 18:37:19 2006 +0000 +++ b/tools/libxc/xc_linux_restore.c Fri Dec 08 21:37:27 2006 -0600 @@ -73,7 +73,7 @@ static int uncanonicalize_pagetable(unsi else pte = ((uint64_t *)page)[i]; - if(pte & _PAGE_PRESENT) { + if(pte_present(pte)) { pfn = (pte >> PAGE_SHIFT) & 0xffffffff; diff -r 1ad7dff99968 tools/libxc/xc_linux_save.c --- a/tools/libxc/xc_linux_save.c Fri Dec 08 18:37:19 2006 +0000 +++ b/tools/libxc/xc_linux_save.c Fri Dec 08 21:36:59 2006 -0600 @@ -471,7 +471,7 @@ static int canonicalize_pagetable(unsign if (i >= xen_start && i < xen_end) pte = 0; - if (pte & _PAGE_PRESENT) { + if (pte_present(pte)) { mfn = (pte >> PAGE_SHIFT) & 0xfffffff; if (!MFN_IS_IN_PSEUDOPHYS_MAP(mfn)) { diff -r 1ad7dff99968 tools/libxc/xg_private.h --- a/tools/libxc/xg_private.h Fri Dec 08 18:37:19 2006 +0000 +++ b/tools/libxc/xg_private.h Fri Dec 08 17:48:49 2006 -0600 @@ -46,6 +46,10 @@ unsigned long csum_page (void * page); #define _PAGE_PSE 0x080 #define _PAGE_GLOBAL 0x100 +#define _PAGE_PROTNONE 0x080 /* If not present */ + +#define pte_present(_pteval) ((_pteval) & (_PAGE_PRESENT|_PAGE_PROTNONE)) + #define L1_PAGETABLE_SHIFT_PAE 12 #define L2_PAGETABLE_SHIFT_PAE 21 #define L3_PAGETABLE_SHIFT_PAE 30 --------------060302030604080308020300 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --------------060302030604080308020300--