From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48882) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zji5v-0002W7-7A for qemu-devel@nongnu.org; Wed, 07 Oct 2015 02:20:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Zji5r-0000N2-9M for qemu-devel@nongnu.org; Wed, 07 Oct 2015 02:20:55 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:10075 helo=relay.sw.ru) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zji5q-0000Mc-UV for qemu-devel@nongnu.org; Wed, 07 Oct 2015 02:20:51 -0400 From: "Denis V. Lunev" Date: Wed, 7 Oct 2015 09:20:38 +0300 Message-Id: <1444198846-5383-1-git-send-email-den@openvz.org> In-Reply-To: <5614531B.5080107@redhat.com> References: <5614531B.5080107@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: [Qemu-devel] [RFC 0/8] QEMUFile-way to gather VM's memory statistics List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Denis V. Lunev" , jsnow@redhat.com, qemu-devel@nongnu.org, Igor Redko , annam@virtuozzo.com This code is not ready to be merged. It is sent in the hope that it could be useful and we could invent better architectural approach for the feature if possible. +tch set summary: - we added a feature that allows to gather VM's memory usage statistics with minimum interference with VM=E2=80=99s work (no transferring data = thru network, probing time ~ downtime, etc.) - this feature is implemented as a new capability for the migrate command. Migration started with this capability turned on will only gather dirty memory statistics regardless of given URI. More details on measurement procedure: To gather statistics, mechanisms created for live migration and snapshots are used. This is the easiest approach for getting dirty bitmap of memory pages. Main points: 1. A function for bitmap resetting was added. During migration the proces= s is bit-wise and so very time-consuming. What is more, it involves memo= ry pages copying to the buffer. 2. A new hook hook_ram_sync() for QEMUFile was added. According to the previous point, we should pass the number of dirty pages before the resetting. As a result of passing control using hook_ram_sync() we can add a delay and let the VM to 'dirty' memory 3. New transport was created (QEMUFile implementation). As we want to use migration_thread() function, all logic that processes and outputs the gathered information should be hidden here. Algorithm for dirty rate estimation: 1. Start memory logging (ram_save_setup()) 2. The first reset of dirty bitmap, pass size of VM memory 3. Sleep =D0=BD=D0=B0 =D0=B2=D1=80=D0=B5=D0=BC=D1=8F =D1=80=D0=B0=D0=B2=D0= =BD=D0=BE=D0=B5 downtime'=D1=83 4. Synchronization of bitmap 5. The second reset of dirty bitmap, pass size of dirty VM memory. 6. Compute dirty byte rate, assuming that time between two synchronizatio= n equals downtime. 7. Fill corresponding fields in MigrationState. 8. After finishing of hook_ram_sync return error code "-42" 9. In migration_thread() error code "-42" is interpreted as successful completion and MigrationStatus is changed Signed-off-by: Denis V. Lunev Signed-off-by: Igor Redko Reviewed-by: Anna Melekhova Igor Redko (7): qemu-file: new hook in qemu-file migration: add new capability test-only migration: add function for reseting migration bitmap migration: add draft of new transport migration: implementation of hook_ram_sync migration: new migration test mode migration: add output of gathered statistics include/migration/migration.h | 5 +++ include/migration/qemu-file.h | 1 + migration/Makefile.objs | 2 +- migration/migration.c | 53 +++++++++++++++++++++-- migration/qemu-file.c | 12 ++++++ migration/ram.c | 30 ++++++++++++- migration/test.c | 99 +++++++++++++++++++++++++++++++++++++= ++++++ qapi-schema.json | 9 +++- qmp-commands.hx | 1 + 9 files changed, 204 insertions(+), 8 deletions(-) create mode 100644 migration/test.c