Re: [Qemu-devel] [PATCH QEMU 0/5] spee up live migration by skipping free pages

[Alexander Graf <agraf@suse.de>]

On 04/19/2016 04:20 PM, Liang Li wrote:
> Current QEMU live migration implementation marks all guest's RAM pages
> as dirtied in the ram bulk stage, all these pages will be processed
> and it consumes quite a lot of CPU cycles and network bandwidth.
>
>  From guest's point of view, it doesn't care about the content in free
> page. We can make use of this fact and skip processing the free
> pages, it can save a lot CPU cycles and reduce the network traffic
> significantly while speed up the live migration process obviously.
>
> This patch set is the QEMU side implementation.
>
> The virtio-balloon is extended so that QEMU can get the free pages
> information from the guest.
>
> After getting the free page bitmap, QEMU can use it to filter out
> guest's free pages. This make the live migration process much more
> efficient.
>
> In order to skip more free pages, we add an interface to let the user
> decide whether dropping the cache in guest during live migration.

Wouldn't that potentially break things like CoLo? Which side is the slow 
one? We already do condense zero pages into small metadata in the 
migration stream. Is writing them out the slow part? If so, just don't 
write them in you're on a clean state.


Alex

>
> Performance data
> ================
>
> Test environment:
>
> CPU: Intel (R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz
> Host RAM: 64GB
> Host Linux Kernel:  4.4.0           Host OS: CentOS 7.1
> Guest Linux Kernel:  4.5.rc6        Guest OS: CentOS 6.6
> Network:  Intel X710 with 10 Gigabit connection
> Guest RAM: 8GB
>
> Case 1: Idle guest just boots:
> ==================================================
>                      | original  |  skip free page
> --------------------------------------------------
> total time(ms)      |    1505   |   573
> --------------------------------------------------
> transferred ram(KB) |   399792  |  379057
> ==================================================
>
> Case 2: The guest has ever run some memory consuming workload, the
> workload is terminated before live migration.
> ==================================================
>                      | original  |   skip free page
> --------------------------------------------------
> total time(ms)      |   10641    |   597
> --------------------------------------------------
> transferred ram(KB) |  8350829  |  389900
> ==================================================
>
> Case 3: The guest has ever built the linux kernel, the building is
> completed before live migration. The page cache is dropped
> during live migration in this case.
> ==================================================
>                      | original  |   skip free page
> --------------------------------------------------
> total time(ms)      |   2636    |   914
> --------------------------------------------------
> transferred ram(KB) |  1316747  |  421980
> ==================================================
>
> Liang Li (5):
>    bitmap: Add a new bitmap_move function
>    kvm: Add two new arch specific functions
>    virtio-balloon: Add a new feature to balloon device
>    migration: filter out free pages during live migration
>    migration: Add the interface for cache drop control
>
>   balloon.c                                       | 29 +++++++-
>   hmp.c                                           |  8 +++
>   hw/virtio/virtio-balloon.c                      | 92 +++++++++++++++++++++++-
>   include/hw/virtio/virtio-balloon.h              | 30 +++++++-
>   include/migration/migration.h                   |  1 +
>   include/qemu/bitmap.h                           | 13 ++++
>   include/standard-headers/linux/virtio_balloon.h |  1 +
>   include/sysemu/balloon.h                        | 12 +++-
>   include/sysemu/kvm.h                            |  2 +
>   migration/migration.c                           | 31 +++++++-
>   migration/ram.c                                 | 95 +++++++++++++++++++++++++
>   qapi-schema.json                                | 25 ++++++-
>   target-arm/kvm.c                                | 14 ++++
>   target-i386/kvm.c                               | 35 +++++++++
>   target-mips/kvm.c                               | 14 ++++
>   target-ppc/kvm.c                                | 14 ++++
>   target-s390x/kvm.c                              | 14 ++++
>   17 files changed, 421 insertions(+), 9 deletions(-)
>