From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35362) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XU4J0-0003Ui-6U for qemu-devel@nongnu.org; Tue, 16 Sep 2014 21:45:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XU4Iw-0004lF-6D for qemu-devel@nongnu.org; Tue, 16 Sep 2014 21:45:14 -0400 Received: from [59.151.112.132] (port=58203 helo=heian.cn.fujitsu.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XU4Iv-0004Og-Jl for qemu-devel@nongnu.org; Tue, 16 Sep 2014 21:45:09 -0400 Message-ID: <5418E74E.20806@cn.fujitsu.com> Date: Wed, 17 Sep 2014 09:43:42 +0800 From: Hongyang Yang MIME-Version: 1.0 References: <1406125538-27992-1-git-send-email-yanghy@cn.fujitsu.com> <1406125538-27992-8-git-send-email-yanghy@cn.fujitsu.com> <20140801145225.GC2430@work-vm> In-Reply-To: <20140801145225.GC2430@work-vm> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [RFC PATCH 07/17] COLO buffer: implement colo buffer as well as QEMUFileOps based on it List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Dr. David Alan Gilbert" Cc: kvm@vger.kernel.org, GuiJianfeng@cn.fujitsu.com, eddie.dong@intel.com, qemu-devel@nongnu.org, mrhines@linux.vnet.ibm.com Hi =E5=9C=A8 08/01/2014 10:52 PM, Dr. David Alan Gilbert =E5=86=99=E9=81=93: > * Yang Hongyang (yanghy@cn.fujitsu.com) wrote: >> We need a buffer to store migration data. >> >> On save side: >> all saved data was write into colo buffer first, so that we can know >> the total size of the migration data. this can also separate the data >> transmission from colo control data, we use colo control data over >> socket fd to synchronous both side's stat. >> >> On restore side: >> all migration data was read into colo buffer first, then load data >> from the buffer: If network error happens while data transmission, >> the slaver can still functinal because the migration data are not yet >> loaded. > > This is very similar to the QEMUSizedBuffer based QEMUFile's that Stefan = Berger > wrote and that I use in both my postcopy and BER patchsets: > > http://lists.gnu.org/archive/html/qemu-devel/2014-07/msg00846.html > > (and to the similar code from Isaku Yamahata). > > I think we should be able to use a shared version even if we need some ch= anges. I've being using this patch as COLO buffer, see my latest branch: https://github.com/macrosheep/qemu/tree/colo_v0.4 But this QEMUSizedBuffer does not exactly match our needs, although it can = work. We need a static buffer that lives through COLO process so that we don't ne= ed to alloc/free buffers every checkpoint. Currently open the buffer for write always alloc a new buffer, I think I need the following modification: 1. ability to open an existing QEMUSizedBuffer for write 2. a reset_buf() api, that will clear buffered data, or just rewind QEMUFil= e related to the buffer is enough. > >> >> Signed-off-by: Yang Hongyang >> --- >> migration-colo.c | 112 +++++++++++++++++++++++++++++++++++++++++++++++= ++++++++ >> 1 file changed, 112 insertions(+) >> >> diff --git a/migration-colo.c b/migration-colo.c >> index d566b9d..b90d9b6 100644 >> --- a/migration-colo.c >> +++ b/migration-colo.c >> @@ -11,6 +11,7 @@ >> #include "qemu/main-loop.h" >> #include "qemu/thread.h" >> #include "block/coroutine.h" >> +#include "qemu/error-report.h" >> #include "migration/migration-colo.h" >> >> static QEMUBH *colo_bh; >> @@ -20,14 +21,122 @@ bool colo_supported(void) >> return true; >> } >> >> +/* colo buffer */ >> + >> +#define COLO_BUFFER_BASE_SIZE (1000*1000*4ULL) >> +#define COLO_BUFFER_MAX_SIZE (1000*1000*1000*10ULL) > > Powers of 2 are nicer! > > Dave > -- > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > . > --=20 Thanks, Yang.