[PATCH for 4.6 v2 0/5] Migration v2 fix

[PATCH for 4.6 v2 0/5] Migration v2 fix

[Wei Liu]

Wei Liu (5):
  libxc: clearer migration v2 debug message
  libxc: migration v2 prefix Memory -> Frames
  libxc: fix indentation
  libxc: don't populate same pfn more than once in populate_pfns
  libxc: add assertion to avoid setting same bit more than once

 tools/libxc/xc_sr_restore.c        | 7 ++++---
 tools/libxc/xc_sr_restore_x86_pv.c | 4 ++--
 tools/libxc/xc_sr_save.c           | 2 +-
 3 files changed, 7 insertions(+), 6 deletions(-)

-- 
2.1.4

[PATCH for 4.6 v2 1/5] libxc: clearer migration v2 debug message

[Wei Liu]

Previous the message was like:

SAVE:
xc: detail: 32 bits, 3 levels
xc: detail: max_pfn 0xfffff, p2m_frames 1024
xc: detail: max_mfn 0x130000

RESTORE:
xc: detail: max_mfn 0x130000
xc: detail: 32 bits, 3 levels
xc: detail: Expanded p2m from 0 to 0xfffff

It's not immediately clear that the last line in restore messages was
referring to max_pfn. Change the debug message a bit to make that
clearer.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 tools/libxc/xc_sr_restore_x86_pv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libxc/xc_sr_restore_x86_pv.c b/tools/libxc/xc_sr_restore_x86_pv.c
index ef26c64..c65a2f1 100644
--- a/tools/libxc/xc_sr_restore_x86_pv.c
+++ b/tools/libxc/xc_sr_restore_x86_pv.c
@@ -66,7 +66,7 @@ static int expand_p2m(struct xc_sr_context *ctx, unsigned long max_pfn)
     for ( i = (old_end_frame ? old_end_frame + 1 : 0); i <= end_frame; ++i )
         ctx->x86_pv.p2m_pfns[i] = INVALID_MFN;
 
-    DPRINTF("Expanded p2m from %#lx to %#lx", old_max, max_pfn);
+    DPRINTF("Changed max_pfn from %#lx to %#lx", old_max, max_pfn);
     return 0;
 }
 
-- 
2.1.4

[PATCH for 4.6 v2 2/5] libxc: migration v2 prefix Memory -> Frames

[Wei Liu]

The prefix "Memory" is confusing because the numbers shown after that
are referring to frames. They have no bearing on how many pages a domain
actually owns or how many actual pages are processed.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxc/xc_sr_save.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
index 58667af..924c425 100644
--- a/tools/libxc/xc_sr_save.c
+++ b/tools/libxc/xc_sr_save.c
@@ -659,7 +659,7 @@ static int send_domain_memory_nonlive(struct xc_sr_context *ctx)
     if ( rc )
         goto err;
 
-    xc_set_progress_prefix(xch, "Memory");
+    xc_set_progress_prefix(xch, "Frames");
 
     rc = send_all_pages(ctx);
     if ( rc )
-- 
2.1.4

[PATCH for 4.6 v2 3/5] libxc: fix indentation

[Wei Liu]

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 tools/libxc/xc_sr_restore_x86_pv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libxc/xc_sr_restore_x86_pv.c b/tools/libxc/xc_sr_restore_x86_pv.c
index c65a2f1..bc604b3 100644
--- a/tools/libxc/xc_sr_restore_x86_pv.c
+++ b/tools/libxc/xc_sr_restore_x86_pv.c
@@ -970,7 +970,7 @@ static int x86_pv_localise_page(struct xc_sr_context *ctx,
     }
 
     if ( to_populate && populate_pfns(ctx, to_populate, pfns, NULL) )
-            return -1;
+        return -1;
 
     for ( i = 0; i < (PAGE_SIZE / sizeof(uint64_t)); ++i )
     {
-- 
2.1.4

[PATCH for 4.6 v2 4/5] libxc: don't populate same pfn more than once in populate_pfns

[Wei Liu]

The original implementation of populate_pfns didn't consider the same
pfn can be present multiple times in the array. The mechanism to prevent
populating the same pfn multiple times only worked if the recurring pfn
appeared in different batches.

This bug is discovered by Linux 4.1 32 bit kernel save / restore test,
which has several ptes pointing to same pfn, which results in an array
containing recurring pfn.  When libxc called x86_pv_localise_page, the
original implementation would populate the same pfn more than once.

The fix is to set bit in populated bitmap as we generate list of pfns to
be populated.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxc/xc_sr_restore.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c
index df885b6..924dd55 100644
--- a/tools/libxc/xc_sr_restore.c
+++ b/tools/libxc/xc_sr_restore.c
@@ -214,6 +214,9 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned count,
                           types[i] != XEN_DOMCTL_PFINFO_BROKEN))) &&
              !pfn_is_populated(ctx, original_pfns[i]) )
         {
+            rc = pfn_set_populated(ctx, original_pfns[i]);
+            if ( rc )
+                goto err;
             pfns[nr_pfns] = mfns[nr_pfns] = original_pfns[i];
             ++nr_pfns;
         }
@@ -238,9 +241,6 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned count,
                 goto err;
             }
 
-            rc = pfn_set_populated(ctx, pfns[i]);
-            if ( rc )
-                goto err;
             ctx->restore.ops.set_gfn(ctx, pfns[i], mfns[i]);
         }
     }
-- 
2.1.4

[PATCH for 4.6 v2 5/5] libxc: add assertion to avoid setting same bit more than once

[Wei Liu]

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 tools/libxc/xc_sr_restore.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c
index 924dd55..f48e7fc 100644
--- a/tools/libxc/xc_sr_restore.c
+++ b/tools/libxc/xc_sr_restore.c
@@ -181,6 +181,7 @@ static int pfn_set_populated(struct xc_sr_context *ctx, xen_pfn_t pfn)
         ctx->restore.max_populated_pfn = new_max;
     }
 
+    assert(!test_bit(pfn, ctx->restore.populated_pfns));
     set_bit(pfn, ctx->restore.populated_pfns);
 
     return 0;
-- 
2.1.4

Re: [PATCH for 4.6 v2 2/5] libxc: migration v2 prefix Memory -> Frames

[Andrew Cooper]

On 05/09/15 00:59, Wei Liu wrote:
> The prefix "Memory" is confusing because the numbers shown after that
> are referring to frames. They have no bearing on how many pages a domain
> actually owns or how many actual pages are processed.

There is a direct bearing on how many pages are processed, by virtue of 
the "$X of $Y" later in the line.  I think your final phrase is 
misleading and should just be dropped.

Also, there are two further places which need "Memory" changing in a 
similar way; send_memory_verify() (which itself should probably be 
renamed to verify_frames()), and update_progress_string() which is the 
companion to the hunk below, but for live migration.

~Andrew

>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
>   tools/libxc/xc_sr_save.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
> index 58667af..924c425 100644
> --- a/tools/libxc/xc_sr_save.c
> +++ b/tools/libxc/xc_sr_save.c
> @@ -659,7 +659,7 @@ static int send_domain_memory_nonlive(struct xc_sr_context *ctx)
>       if ( rc )
>           goto err;
>   
> -    xc_set_progress_prefix(xch, "Memory");
> +    xc_set_progress_prefix(xch, "Frames");
>   
>       rc = send_all_pages(ctx);
>       if ( rc )

Re: [PATCH for 4.6 v2 4/5] libxc: don't populate same pfn more than once in populate_pfns

[Andrew Cooper]

On 05/09/15 00:59, Wei Liu wrote:
> The original implementation of populate_pfns didn't consider the same
> pfn can be present multiple times in the array. The mechanism to prevent
> populating the same pfn multiple times only worked if the recurring pfn
> appeared in different batches.
>
> This bug is discovered by Linux 4.1 32 bit kernel save / restore test,
> which has several ptes pointing to same pfn, which results in an array
> containing recurring pfn.  When libxc called x86_pv_localise_page, the
> original implementation would populate the same pfn more than once.
>
> The fix is to set bit in populated bitmap as we generate list of pfns to
> be populated.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

Re: [PATCH for 4.6 v2 2/5] libxc: migration v2 prefix Memory -> Frames

[Wei Liu]

On Sun, Sep 06, 2015 at 05:23:57PM +0100, Andrew Cooper wrote:
> On 05/09/15 00:59, Wei Liu wrote:
> >The prefix "Memory" is confusing because the numbers shown after that
> >are referring to frames. They have no bearing on how many pages a domain
> >actually owns or how many actual pages are processed.
> 
> There is a direct bearing on how many pages are processed, by virtue of the
> "$X of $Y" later in the line.  I think your final phrase is misleading and
> should just be dropped.
> 

OK.

> Also, there are two further places which need "Memory" changing in a similar
> way; send_memory_verify() (which itself should probably be renamed to
> verify_frames()), and update_progress_string() which is the companion to the
> hunk below, but for live migration.
> 

I will fix those, too.

Wei.

> ~Andrew
> 
> >
> >Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> >---
> >  tools/libxc/xc_sr_save.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> >diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
> >index 58667af..924c425 100644
> >--- a/tools/libxc/xc_sr_save.c
> >+++ b/tools/libxc/xc_sr_save.c
> >@@ -659,7 +659,7 @@ static int send_domain_memory_nonlive(struct xc_sr_context *ctx)
> >      if ( rc )
> >          goto err;
> >-    xc_set_progress_prefix(xch, "Memory");
> >+    xc_set_progress_prefix(xch, "Frames");
> >      rc = send_all_pages(ctx);
> >      if ( rc )