From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48521) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1asWmQ-0007nE-1z for qemu-devel@nongnu.org; Tue, 19 Apr 2016 10:37:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1asWmM-0005X8-0L for qemu-devel@nongnu.org; Tue, 19 Apr 2016 10:37:30 -0400 Received: from mx2.suse.de ([195.135.220.15]:38337) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1asWmL-0005X1-Lq for qemu-devel@nongnu.org; Tue, 19 Apr 2016 10:37:25 -0400 References: <1461075643-3668-1-git-send-email-liang.z.li@intel.com> From: Alexander Graf Message-ID: <571642A2.4060003@suse.de> Date: Tue, 19 Apr 2016 16:37:22 +0200 MIME-Version: 1.0 In-Reply-To: <1461075643-3668-1-git-send-email-liang.z.li@intel.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH QEMU 0/5] spee up live migration by skipping free pages List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Liang Li , mst@redhat.com, quintela@redhat.com, amit.shah@redhat.com, pbonzini@redhat.com, lcapitulino@redhat.com Cc: armbru@redhat.com, peter.maydell@linaro.org, rth@twiddle.net, ehabkost@redhat.com, james.hogan@imgtec.com, aurelien@aurel32.net, leon.alrae@imgtec.com, borntraeger@de.ibm.com, cornelia.huck@de.ibm.com, qemu-devel@nongnu.org, kvm@vger.kernel.org On 04/19/2016 04:20 PM, Liang Li wrote: > Current QEMU live migration implementation marks all guest's RAM pages > as dirtied in the ram bulk stage, all these pages will be processed > and it consumes quite a lot of CPU cycles and network bandwidth. > > From guest's point of view, it doesn't care about the content in free > page. We can make use of this fact and skip processing the free > pages, it can save a lot CPU cycles and reduce the network traffic > significantly while speed up the live migration process obviously. > > This patch set is the QEMU side implementation. > > The virtio-balloon is extended so that QEMU can get the free pages > information from the guest. > > After getting the free page bitmap, QEMU can use it to filter out > guest's free pages. This make the live migration process much more > efficient. > > In order to skip more free pages, we add an interface to let the user > decide whether dropping the cache in guest during live migration. Wouldn't that potentially break things like CoLo? Which side is the slow one? We already do condense zero pages into small metadata in the migration stream. Is writing them out the slow part? If so, just don't write them in you're on a clean state. Alex > > Performance data > ================ > > Test environment: > > CPU: Intel (R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz > Host RAM: 64GB > Host Linux Kernel: 4.4.0 Host OS: CentOS 7.1 > Guest Linux Kernel: 4.5.rc6 Guest OS: CentOS 6.6 > Network: Intel X710 with 10 Gigabit connection > Guest RAM: 8GB > > Case 1: Idle guest just boots: > ================================================== > | original | skip free page > -------------------------------------------------- > total time(ms) | 1505 | 573 > -------------------------------------------------- > transferred ram(KB) | 399792 | 379057 > ================================================== > > Case 2: The guest has ever run some memory consuming workload, the > workload is terminated before live migration. > ================================================== > | original | skip free page > -------------------------------------------------- > total time(ms) | 10641 | 597 > -------------------------------------------------- > transferred ram(KB) | 8350829 | 389900 > ================================================== > > Case 3: The guest has ever built the linux kernel, the building is > completed before live migration. The page cache is dropped > during live migration in this case. > ================================================== > | original | skip free page > -------------------------------------------------- > total time(ms) | 2636 | 914 > -------------------------------------------------- > transferred ram(KB) | 1316747 | 421980 > ================================================== > > Liang Li (5): > bitmap: Add a new bitmap_move function > kvm: Add two new arch specific functions > virtio-balloon: Add a new feature to balloon device > migration: filter out free pages during live migration > migration: Add the interface for cache drop control > > balloon.c | 29 +++++++- > hmp.c | 8 +++ > hw/virtio/virtio-balloon.c | 92 +++++++++++++++++++++++- > include/hw/virtio/virtio-balloon.h | 30 +++++++- > include/migration/migration.h | 1 + > include/qemu/bitmap.h | 13 ++++ > include/standard-headers/linux/virtio_balloon.h | 1 + > include/sysemu/balloon.h | 12 +++- > include/sysemu/kvm.h | 2 + > migration/migration.c | 31 +++++++- > migration/ram.c | 95 +++++++++++++++++++++++++ > qapi-schema.json | 25 ++++++- > target-arm/kvm.c | 14 ++++ > target-i386/kvm.c | 35 +++++++++ > target-mips/kvm.c | 14 ++++ > target-ppc/kvm.c | 14 ++++ > target-s390x/kvm.c | 14 ++++ > 17 files changed, 421 insertions(+), 9 deletions(-) >