x86_64 live migration problems: help needed with shadow page table

x86_64 live migration problems: help needed with shadow page table

[John Byrne]

[-- Attachment #1: Type: text/plain, Size: 1283 bytes --]


Hi,

I've been doing most of my testing with Novell's SLES10 3.0.2 9742c; 
however, I did verify the problem existed in xen-unstable as of last week.

x86_64 live migration is unreliable, if the domain is under stress (I 
used a kernel make), the domU frequently OOPses or the compile gets a 
segfault afterwards.

The primary issue seems to be that L1 and L2 page table pages are not 
getting marked dirty it the shadow_dirty_bitmap. (It is running 4-level 
page tables and I have yet to see a verify problem with the L3 and L4 
page tables.)

I have shown this to my satisfaction by adding code in xc_linux_save.c 
to mark all L1 and L2 pages for fixups on the last iteration. I've 
managed to migrate 10 times without obvious problems, so far. (I also 
found the the clear_bit() and set_bit() routines were broken on x86_64; 
a patch against the latest xen-unstable is attached.)

I cannot call what I've done a fix since it causing the transfer of 2000 
extra, and mostly unnecessary, pages on the last iteration. So what I'd 
like help with is where to fix the shadow page table code so the L1 and 
L2 pages get marked dirty properly. It is not immediately obvious to me 
where this needs to be done and I'm hoping someone can save me a lot of 
time.

Thanks,

John Byrne






[-- Attachment #2: xc_linux_save_64bit.patch --]
[-- Type: text/x-patch, Size: 689 bytes --]

diff -r 411a3c01bb40 tools/libxc/xc_linux_save.c
--- a/tools/libxc/xc_linux_save.c	Tue Jun 20 18:51:46 2006 +0100
+++ b/tools/libxc/xc_linux_save.c	Thu Jun 22 11:45:56 2006 -0700
@@ -91,12 +91,12 @@ static inline int test_bit (int nr, vola
 
 static inline void clear_bit (int nr, volatile void * addr)
 {
-    BITMAP_ENTRY(nr, addr) &= ~(1 << BITMAP_SHIFT(nr));
+    BITMAP_ENTRY(nr, addr) &= ~(1UL << BITMAP_SHIFT(nr));
 }
 
 static inline void set_bit ( int nr, volatile void * addr)
 {
-    BITMAP_ENTRY(nr, addr) |= (1 << BITMAP_SHIFT(nr));
+    BITMAP_ENTRY(nr, addr) |= (1UL << BITMAP_SHIFT(nr));
 }
 
 /* Returns the hamming weight (i.e. the number of bits set) in a N-bit word */

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel