All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
To: Yang Hongyang <yanghy@cn.fujitsu.com>
Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org, eddie.dong@intel.com,
	GuiJianfeng@cn.fujitsu.com, mrhines@linux.vnet.ibm.com,
	wency@cn.fujitsu.com
Subject: Re: [RFC PATCH 07/17] COLO buffer: implement colo buffer as well as QEMUFileOps based on it
Date: Fri, 1 Aug 2014 15:52:25 +0100	[thread overview]
Message-ID: <20140801145225.GC2430@work-vm> (raw)
In-Reply-To: <1406125538-27992-8-git-send-email-yanghy@cn.fujitsu.com>

* Yang Hongyang (yanghy@cn.fujitsu.com) wrote:
> We need a buffer to store migration data.
> 
> On save side:
>   all saved data was write into colo buffer first, so that we can know
> the total size of the migration data. this can also separate the data
> transmission from colo control data, we use colo control data over
> socket fd to synchronous both side's stat.
> 
> On restore side:
>   all migration data was read into colo buffer first, then load data
> from the buffer: If network error happens while data transmission,
> the slaver can still functinal because the migration data are not yet
> loaded.

This is very similar to the QEMUSizedBuffer based QEMUFile's that Stefan Berger
wrote and that I use in both my postcopy and BER patchsets:

 http://lists.gnu.org/archive/html/qemu-devel/2014-07/msg00846.html

 (and to the similar code from Isaku Yamahata).

I think we should be able to use a shared version even if we need some changes.

> 
> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
> ---
>  migration-colo.c | 112 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 112 insertions(+)
> 
> diff --git a/migration-colo.c b/migration-colo.c
> index d566b9d..b90d9b6 100644
> --- a/migration-colo.c
> +++ b/migration-colo.c
> @@ -11,6 +11,7 @@
>  #include "qemu/main-loop.h"
>  #include "qemu/thread.h"
>  #include "block/coroutine.h"
> +#include "qemu/error-report.h"
>  #include "migration/migration-colo.h"
>  
>  static QEMUBH *colo_bh;
> @@ -20,14 +21,122 @@ bool colo_supported(void)
>      return true;
>  }
>  
> +/* colo buffer */
> +
> +#define COLO_BUFFER_BASE_SIZE (1000*1000*4ULL)
> +#define COLO_BUFFER_MAX_SIZE (1000*1000*1000*10ULL)

Powers of 2 are nicer!

Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

WARNING: multiple messages have this Message-ID (diff)
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
To: Yang Hongyang <yanghy@cn.fujitsu.com>
Cc: kvm@vger.kernel.org, GuiJianfeng@cn.fujitsu.com,
	eddie.dong@intel.com, qemu-devel@nongnu.org,
	mrhines@linux.vnet.ibm.com
Subject: Re: [Qemu-devel] [RFC PATCH 07/17] COLO buffer: implement colo buffer as well as QEMUFileOps based on it
Date: Fri, 1 Aug 2014 15:52:25 +0100	[thread overview]
Message-ID: <20140801145225.GC2430@work-vm> (raw)
In-Reply-To: <1406125538-27992-8-git-send-email-yanghy@cn.fujitsu.com>

* Yang Hongyang (yanghy@cn.fujitsu.com) wrote:
> We need a buffer to store migration data.
> 
> On save side:
>   all saved data was write into colo buffer first, so that we can know
> the total size of the migration data. this can also separate the data
> transmission from colo control data, we use colo control data over
> socket fd to synchronous both side's stat.
> 
> On restore side:
>   all migration data was read into colo buffer first, then load data
> from the buffer: If network error happens while data transmission,
> the slaver can still functinal because the migration data are not yet
> loaded.

This is very similar to the QEMUSizedBuffer based QEMUFile's that Stefan Berger
wrote and that I use in both my postcopy and BER patchsets:

 http://lists.gnu.org/archive/html/qemu-devel/2014-07/msg00846.html

 (and to the similar code from Isaku Yamahata).

I think we should be able to use a shared version even if we need some changes.

> 
> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
> ---
>  migration-colo.c | 112 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 112 insertions(+)
> 
> diff --git a/migration-colo.c b/migration-colo.c
> index d566b9d..b90d9b6 100644
> --- a/migration-colo.c
> +++ b/migration-colo.c
> @@ -11,6 +11,7 @@
>  #include "qemu/main-loop.h"
>  #include "qemu/thread.h"
>  #include "block/coroutine.h"
> +#include "qemu/error-report.h"
>  #include "migration/migration-colo.h"
>  
>  static QEMUBH *colo_bh;
> @@ -20,14 +21,122 @@ bool colo_supported(void)
>      return true;
>  }
>  
> +/* colo buffer */
> +
> +#define COLO_BUFFER_BASE_SIZE (1000*1000*4ULL)
> +#define COLO_BUFFER_MAX_SIZE (1000*1000*1000*10ULL)

Powers of 2 are nicer!

Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

  parent reply	other threads:[~2014-08-01 14:52 UTC|newest]

Thread overview: 80+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-07-23 14:25 [RFC PATCH 00/17] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service Yang Hongyang
2014-07-23 14:25 ` [Qemu-devel] " Yang Hongyang
2014-07-23 14:25 ` [RFC PATCH 01/17] configure: add CONFIG_COLO to switch COLO support Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-07-23 14:25 ` [RFC PATCH 02/17] COLO: introduce an api colo_supported() to indicate " Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-07-23 15:47   ` Eric Blake
2014-07-23 15:47     ` Eric Blake
2014-07-23 14:25 ` [RFC PATCH 03/17] COLO migration: add a migration capability 'colo' Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-07-23 14:41   ` Eric Blake
2014-07-23 14:41     ` Eric Blake
2014-07-23 14:25 ` [RFC PATCH 04/17] COLO info: use colo info to tell migration target colo is enabled Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-08-01 14:43   ` Dr. David Alan Gilbert
2014-08-01 14:43     ` [Qemu-devel] " Dr. David Alan Gilbert
2014-09-12  6:36     ` Hongyang Yang
2014-09-12  6:36       ` [Qemu-devel] " Hongyang Yang
2014-07-23 14:25 ` [RFC PATCH 05/17] COLO save: integrate COLO checkpointed save into qemu migration Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-08-01 14:46   ` Dr. David Alan Gilbert
2014-08-01 14:46     ` [Qemu-devel] " Dr. David Alan Gilbert
2014-07-23 14:25 ` [RFC PATCH 06/17] COLO restore: integrate COLO checkpointed restore into qemu restore Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-07-23 14:25 ` [RFC PATCH 07/17] COLO buffer: implement colo buffer as well as QEMUFileOps based on it Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-07-23 18:24   ` Eric Blake
2014-07-23 18:24     ` Eric Blake
2014-08-01 14:52   ` Dr. David Alan Gilbert [this message]
2014-08-01 14:52     ` Dr. David Alan Gilbert
2014-09-17  1:43     ` Hongyang Yang
2014-09-17  1:43       ` [Qemu-devel] " Hongyang Yang
2014-07-23 14:25 ` [RFC PATCH 08/17] COLO: disable qdev hotplug Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-07-23 14:25 ` [RFC PATCH 09/17] COLO ctl: implement API's that communicate with colo agent Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-07-23 14:25 ` [RFC PATCH 10/17] COLO ctl: introduce is_slave() and is_master() Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-08-01 14:55   ` Dr. David Alan Gilbert
2014-08-01 14:55     ` [Qemu-devel] " Dr. David Alan Gilbert
2014-07-23 14:25 ` [RFC PATCH 11/17] COLO ctl: implement colo checkpoint protocol Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-08-01 15:03   ` Dr. David Alan Gilbert
2014-08-01 15:03     ` [Qemu-devel] " Dr. David Alan Gilbert
2014-09-12  6:20     ` Hongyang Yang
2014-09-12  6:20       ` [Qemu-devel] " Hongyang Yang
2014-09-12 11:17       ` Dr. David Alan Gilbert
2014-09-12 11:17         ` [Qemu-devel] " Dr. David Alan Gilbert
2014-09-12 11:40         ` Hongyang Yang
2014-09-12 11:40           ` [Qemu-devel] " Hongyang Yang
2014-09-12 11:57           ` Dr. David Alan Gilbert
2014-09-12 11:57             ` [Qemu-devel] " Dr. David Alan Gilbert
2014-07-23 14:25 ` [RFC PATCH 12/17] COLO ctl: add a RunState RUN_STATE_COLO Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-07-23 15:48   ` Eric Blake
2014-07-23 15:48     ` Eric Blake
2014-07-23 14:25 ` [RFC PATCH 13/17] COLO ctl: implement colo save Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-08-01 15:07   ` Dr. David Alan Gilbert
2014-08-01 15:07     ` [Qemu-devel] " Dr. David Alan Gilbert
2014-07-23 14:25 ` [RFC PATCH 14/17] COLO ctl: implement colo restore Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-07-23 14:25 ` [RFC PATCH 15/17] COLO save: reuse migration bitmap under colo checkpoint Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-08-01 15:09   ` Dr. David Alan Gilbert
2014-08-01 15:09     ` [Qemu-devel] " Dr. David Alan Gilbert
2014-07-23 14:25 ` [RFC PATCH 16/17] COLO ram cache: implement colo ram cache on slaver Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-08-01 15:10   ` Dr. David Alan Gilbert
2014-08-01 15:10     ` [Qemu-devel] " Dr. David Alan Gilbert
2014-09-12  6:30     ` Hongyang Yang
2014-09-12  6:30       ` [Qemu-devel] " Hongyang Yang
2014-07-23 14:25 ` [RFC PATCH 17/17] HACK: trigger checkpoint every 500ms Yang Hongyang
2014-07-23 14:25   ` [Qemu-devel] " Yang Hongyang
2014-07-23 15:44 ` [Qemu-devel] [RFC PATCH 00/17] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service Eric Blake
2014-07-23 15:44   ` Eric Blake
2014-07-24  2:24   ` Hongyang Yang
2014-07-24  2:24     ` [Qemu-devel] " Hongyang Yang
2014-08-01 16:02 ` Dr. David Alan Gilbert
2014-08-01 16:02   ` [Qemu-devel] " Dr. David Alan Gilbert

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20140801145225.GC2430@work-vm \
    --to=dgilbert@redhat.com \
    --cc=GuiJianfeng@cn.fujitsu.com \
    --cc=eddie.dong@intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=mrhines@linux.vnet.ibm.com \
    --cc=qemu-devel@nongnu.org \
    --cc=wency@cn.fujitsu.com \
    --cc=yanghy@cn.fujitsu.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.