[Qemu-devel] [RFC 0/8] QEMUFile-way to gather VM's memory statistics

["Denis V. Lunev" <den@openvz.org>]

This code is not ready to be merged. It is sent in the hope that it could
be useful and we could invent better architectural approach for the
feature if possible.

+tch set summary:
- we added a feature that allows to gather VM's memory usage statistics
  with minimum interference with VM’s work (no transferring data thru
  network, probing time ~ downtime, etc.)
- this feature is implemented as a new capability for the migrate command.
  Migration started with this capability turned on will only gather dirty
  memory statistics regardless of given URI.

More details on measurement procedure:
To gather statistics, mechanisms created for live migration and snapshots
are used. This is the easiest approach for getting dirty bitmap of memory
pages.

Main points:
1. A function for bitmap resetting was added. During migration the process
   is bit-wise and so very time-consuming. What is more, it involves memory
   pages copying to the buffer.
2. A new hook hook_ram_sync() for QEMUFile was added. According to the
   previous point, we should pass the number of dirty pages before the
   resetting. As a result of passing control using hook_ram_sync() we can
   add a delay and let the VM to 'dirty' memory
3. New transport was created (QEMUFile implementation). As we want to use
   migration_thread() function, all logic that processes and outputs the
   gathered information should be hidden here.

Algorithm for dirty rate estimation:
1. Start memory logging (ram_save_setup())
2. The first reset of dirty bitmap, pass size of VM memory
3. Sleep на время равное downtime'у
4. Synchronization of bitmap
5. The second reset of dirty bitmap, pass size of dirty VM memory.
6. Compute dirty byte rate, assuming that time between two synchronization
   equals downtime.
7. Fill corresponding fields in MigrationState.
8. After finishing of hook_ram_sync return error code "-42"
9. In migration_thread() error code "-42" is interpreted as successful
   completion and MigrationStatus is changed

Signed-off-by: Denis V. Lunev <den@openvz.org>
Signed-off-by: Igor Redko <redkoi@virtuozzo.com>
Reviewed-by: Anna Melekhova <annam@virtuozzo.com>

Igor Redko (7):
  qemu-file: new hook in qemu-file
  migration: add new capability test-only
  migration: add function for reseting migration bitmap
  migration: add draft of new transport
  migration: implementation of hook_ram_sync
  migration: new migration test mode
  migration: add output of gathered statistics

 include/migration/migration.h |  5 +++
 include/migration/qemu-file.h |  1 +
 migration/Makefile.objs       |  2 +-
 migration/migration.c         | 53 +++++++++++++++++++++--
 migration/qemu-file.c         | 12 ++++++
 migration/ram.c               | 30 ++++++++++++-
 migration/test.c              | 99 +++++++++++++++++++++++++++++++++++++++++++
 qapi-schema.json              |  9 +++-
 qmp-commands.hx               |  1 +
 9 files changed, 204 insertions(+), 8 deletions(-)
 create mode 100644 migration/test.c