From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-f199.google.com (mail-qt0-f199.google.com [209.85.216.199]) by kanga.kvack.org (Postfix) with ESMTP id 5FD556B0038 for ; Wed, 7 Dec 2016 10:42:12 -0500 (EST) Received: by mail-qt0-f199.google.com with SMTP id w39so265499912qtw.0 for ; Wed, 07 Dec 2016 07:42:12 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id z47si14741391qtb.204.2016.12.07.07.42.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 07 Dec 2016 07:42:11 -0800 (PST) From: David Hildenbrand Subject: Re: [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration References: <1480495397-23225-1-git-send-email-liang.z.li@intel.com> Message-ID: <0b18c636-ee67-cbb4-1ba3-81a06150db76@redhat.com> Date: Wed, 7 Dec 2016 16:42:05 +0100 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: "Li, Liang Z" , "kvm@vger.kernel.org" Cc: "virtio-dev@lists.oasis-open.org" , "mhocko@suse.com" , "mst@redhat.com" , "Hansen, Dave" , "qemu-devel@nongnu.org" , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , "kirill.shutemov@linux.intel.com" , "pbonzini@redhat.com" , "akpm@linux-foundation.org" , "virtualization@lists.linux-foundation.org" , "dgilbert@redhat.com" , Andrea Arcangeli Am 07.12.2016 um 14:35 schrieb Li, Liang Z: >> Am 30.11.2016 um 09:43 schrieb Liang Li: >>> This patch set contains two parts of changes to the virtio-balloon. >>> >>> One is the change for speeding up the inflating & deflating process, >>> the main idea of this optimization is to use bitmap to send the page >>> information to host instead of the PFNs, to reduce the overhead of >>> virtio data transmission, address translation and madvise(). This can >>> help to improve the performance by about 85%. >> >> Do you have some statistics/some rough feeling how many consecutive bits are >> usually set in the bitmaps? Is it really just purely random or is there some >> granularity that is usually consecutive? >> > > I did something similar. Filled the balloon with 15GB for a 16GB idle guest, by > using bitmap, the madvise count was reduced to 605. when using the PFNs, the madvise count > was 3932160. It means there are quite a lot consecutive bits in the bitmap. > I didn't test for a guest with heavy memory workload. Would it then even make sense to go one step further and report {pfn, length} combinations? So simply send over an array of {pfn, length}? This idea came up when talking to Andrea Arcangeli (put him on cc). And it makes sense if you think about: a) hugetlb backing: The host may only be able to free huge pages (we might want to communicate that to the guest later, that's another story). Still we would have to send bitmaps full of 4k frames (512 bits for 2mb frames). Of course, we could add a way to communicate that we are using a different bitmap-granularity. b) if we really inflate huge memory regions (and it sounds like that according to your measurements), we can minimize the communication to the hypervisor and therefore the madvice calls. c) we don't want to optimize for inflating guests with almost full memory (and therefore little consecutive memory areas) - my opinion :) Thanks for the explanation! -- David -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org