From mboxrd@z Thu Jan  1 00:00:00 1970
From: Hongyang Yang <yanghy@cn.fujitsu.com>
Subject: Re: [RFC PATCH 07/17] COLO buffer: implement colo buffer as well
 as QEMUFileOps based on it
Date: Wed, 17 Sep 2014 09:43:42 +0800
Message-ID: <5418E74E.20806@cn.fujitsu.com>
References: <1406125538-27992-1-git-send-email-yanghy@cn.fujitsu.com> <1406125538-27992-8-git-send-email-yanghy@cn.fujitsu.com> <20140801145225.GC2430@work-vm>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8;
	format=flowed
Content-Transfer-Encoding: QUOTED-PRINTABLE
Cc: <qemu-devel@nongnu.org>, <kvm@vger.kernel.org>,
	<eddie.dong@intel.com>, <GuiJianfeng@cn.fujitsu.com>,
	<mrhines@linux.vnet.ibm.com>, <wency@cn.fujitsu.com>
To: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Return-path: <kvm-owner@vger.kernel.org>
Received: from cn.fujitsu.com ([59.151.112.132]:6480 "EHLO
	heian.cn.fujitsu.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org
	with ESMTP id S1753777AbaIQBpB convert rfc822-to-8bit (ORCPT
	<rfc822;kvm@vger.kernel.org>); Tue, 16 Sep 2014 21:45:01 -0400
In-Reply-To: <20140801145225.GC2430@work-vm>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

Hi

=E5=9C=A8 08/01/2014 10:52 PM, Dr. David Alan Gilbert =E5=86=99=E9=81=93=
:
> * Yang Hongyang (yanghy@cn.fujitsu.com) wrote:
>> We need a buffer to store migration data.
>>
>> On save side:
>>    all saved data was write into colo buffer first, so that we can k=
now
>> the total size of the migration data. this can also separate the dat=
a
>> transmission from colo control data, we use colo control data over
>> socket fd to synchronous both side's stat.
>>
>> On restore side:
>>    all migration data was read into colo buffer first, then load dat=
a
>> from the buffer: If network error happens while data transmission,
>> the slaver can still functinal because the migration data are not ye=
t
>> loaded.
>
> This is very similar to the QEMUSizedBuffer based QEMUFile's that Ste=
fan Berger
> wrote and that I use in both my postcopy and BER patchsets:
>
>   http://lists.gnu.org/archive/html/qemu-devel/2014-07/msg00846.html
>
>   (and to the similar code from Isaku Yamahata).
>
> I think we should be able to use a shared version even if we need som=
e changes.

I've being using this patch as COLO buffer, see my latest branch:
https://github.com/macrosheep/qemu/tree/colo_v0.4

But this QEMUSizedBuffer does not exactly match our needs, although it =
can work.
We need a static buffer that lives through COLO process so that we don'=
t need
to alloc/free buffers every checkpoint. Currently open the buffer for w=
rite
always alloc a new buffer, I think I need the following modification:

1. ability to open an existing QEMUSizedBuffer for write
2. a reset_buf() api, that will clear buffered data, or just rewind QEM=
UFile
    related to the buffer is enough.

>
>>
>> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
>> ---
>>   migration-colo.c | 112 +++++++++++++++++++++++++++++++++++++++++++=
++++++++++++
>>   1 file changed, 112 insertions(+)
>>
>> diff --git a/migration-colo.c b/migration-colo.c
>> index d566b9d..b90d9b6 100644
>> --- a/migration-colo.c
>> +++ b/migration-colo.c
>> @@ -11,6 +11,7 @@
>>   #include "qemu/main-loop.h"
>>   #include "qemu/thread.h"
>>   #include "block/coroutine.h"
>> +#include "qemu/error-report.h"
>>   #include "migration/migration-colo.h"
>>
>>   static QEMUBH *colo_bh;
>> @@ -20,14 +21,122 @@ bool colo_supported(void)
>>       return true;
>>   }
>>
>> +/* colo buffer */
>> +
>> +#define COLO_BUFFER_BASE_SIZE (1000*1000*4ULL)
>> +#define COLO_BUFFER_MAX_SIZE (1000*1000*1000*10ULL)
>
> Powers of 2 are nicer!
>
> Dave
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> .
>

--=20
Thanks,
Yang.