From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48247) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZjpZ7-0001VD-BY for qemu-devel@nongnu.org; Wed, 07 Oct 2015 10:19:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZjpZ4-0003Gq-71 for qemu-devel@nongnu.org; Wed, 07 Oct 2015 10:19:33 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53684) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZjpZ4-0003GR-02 for qemu-devel@nongnu.org; Wed, 07 Oct 2015 10:19:30 -0400 Date: Wed, 7 Oct 2015 15:19:24 +0100 From: "Dr. David Alan Gilbert" Message-ID: <20151007141923.GH2710@work-vm> References: <5614531B.5080107@redhat.com> <1444198846-5383-1-git-send-email-den@openvz.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <1444198846-5383-1-git-send-email-den@openvz.org> Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [RFC 0/8] QEMUFile-way to gather VM's memory statistics List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Denis V. Lunev" Cc: Igor Redko , quintela@redhat.com, qemu-devel@nongnu.org, annam@virtuozzo.com, amit.shah@redhat.com, jsnow@redhat.com (ccing Juan and Amit in) * Denis V. Lunev (den@openvz.org) wrote: > This code is not ready to be merged. It is sent in the hope that it cou= ld > be useful and we could invent better architectural approach for the > feature if possible. >=20 > +tch set summary: > - we added a feature that allows to gather VM's memory usage statistics > with minimum interference with VM=E2=80=99s work (no transferring dat= a thru > network, probing time ~ downtime, etc.) > - this feature is implemented as a new capability for the migrate comma= nd. > Migration started with this capability turned on will only gather dir= ty > memory statistics regardless of given URI. >=20 > More details on measurement procedure: > To gather statistics, mechanisms created for live migration and snapsho= ts > are used. This is the easiest approach for getting dirty bitmap of memo= ry > pages. Hi Denis, Thank you; it's something I know we had wondered about doing, so we cou= ld get some idea of how bad a particular application was. We'd also been thinking of doing it as a dummy transport. > Main points: > 1. A function for bitmap resetting was added. During migration the proc= ess > is bit-wise and so very time-consuming. What is more, it involves me= mory > pages copying to the buffer. > 2. A new hook hook_ram_sync() for QEMUFile was added. According to the > previous point, we should pass the number of dirty pages before the > resetting. As a result of passing control using hook_ram_sync() we c= an > add a delay and let the VM to 'dirty' memory > 3. New transport was created (QEMUFile implementation). As we want to u= se > migration_thread() function, all logic that processes and outputs th= e > gathered information should be hidden here. >=20 > Algorithm for dirty rate estimation: > 1. Start memory logging (ram_save_setup()) > 2. The first reset of dirty bitmap, pass size of VM memory > 3. Sleep =D0=BD=D0=B0 =D0=B2=D1=80=D0=B5=D0=BC=D1=8F =D1=80=D0=B0=D0=B2= =D0=BD=D0=BE=D0=B5 downtime'=D1=83 | translate > 'for a time equal to' > 4. Synchronization of bitmap > 5. The second reset of dirty bitmap, pass size of dirty VM memory. > 6. Compute dirty byte rate, assuming that time between two synchronizat= ion > equals downtime. > 7. Fill corresponding fields in MigrationState. > 8. After finishing of hook_ram_sync return error code "-42" > 9. In migration_thread() error code "-42" is interpreted as successful > completion and MigrationStatus is changed Yeh I'm not quite sure why you don't just let it fall through the end of = migration and dump these into the normal output of info migrate in the 'completed' = output. Dave >=20 > Signed-off-by: Denis V. Lunev > Signed-off-by: Igor Redko > Reviewed-by: Anna Melekhova >=20 > Igor Redko (7): > qemu-file: new hook in qemu-file > migration: add new capability test-only > migration: add function for reseting migration bitmap > migration: add draft of new transport > migration: implementation of hook_ram_sync > migration: new migration test mode > migration: add output of gathered statistics >=20 > include/migration/migration.h | 5 +++ > include/migration/qemu-file.h | 1 + > migration/Makefile.objs | 2 +- > migration/migration.c | 53 +++++++++++++++++++++-- > migration/qemu-file.c | 12 ++++++ > migration/ram.c | 30 ++++++++++++- > migration/test.c | 99 +++++++++++++++++++++++++++++++++++= ++++++++ > qapi-schema.json | 9 +++- > qmp-commands.hx | 1 + > 9 files changed, 204 insertions(+), 8 deletions(-) > create mode 100644 migration/test.c >=20 -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK