From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56147) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZK2s1-0002ot-38 for qemu-devel@nongnu.org; Tue, 28 Jul 2015 07:16:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZK2rw-000530-Rb for qemu-devel@nongnu.org; Tue, 28 Jul 2015 07:16:29 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50451) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZK2rw-00052h-Jo for qemu-devel@nongnu.org; Tue, 28 Jul 2015 07:16:24 -0400 Date: Tue, 28 Jul 2015 12:16:17 +0100 From: "Dr. David Alan Gilbert" Message-ID: <20150728111617.GD2247@work-vm> References: <1434450415-11339-1-git-send-email-dgilbert@redhat.com> <1434450415-11339-43-git-send-email-dgilbert@redhat.com> <20150728061547.GL12267@grmbl.mre> <20150728090815.GC2247@work-vm> <20150728100140.GN12267@grmbl.mre> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150728100140.GN12267@grmbl.mre> Subject: Re: [Qemu-devel] [PATCH v7 42/42] Inhibit ballooning during postcopy List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Amit Shah Cc: aarcange@redhat.com, yamahata@private.email.ne.jp, quintela@redhat.com, liang.z.li@intel.com, qemu-devel@nongnu.org, luis@cs.umu.se, pbonzini@redhat.com, david@gibson.dropbear.id.au * Amit Shah (amit.shah@redhat.com) wrote: > On (Tue) 28 Jul 2015 [10:08:15], Dr. David Alan Gilbert wrote: > > * Amit Shah (amit.shah@redhat.com) wrote: > > > On (Tue) 16 Jun 2015 [11:26:55], Dr. David Alan Gilbert (git) wrote: > > > > From: "Dr. David Alan Gilbert" > > > > > > > > Postcopy detects accesses to pages that haven't been transferred yet > > > > using userfaultfd, and it causes exceptions on pages that are 'not > > > > present'. > > > > Ballooning also causes pages to be marked as 'not present' when the > > > > guest inflates the balloon. > > > > Potentially a balloon could be inflated to discard pages that are > > > > currently inflight during postcopy and that may be arriving at about > > > > the same time. > > > > > > > > To avoid this confusion, disable ballooning during postcopy. > > > > > > > > When disabled we drop balloon requests from the guest. Since ballooning > > > > is generally initiated by the host, the management system should avoid > > > > initiating any balloon instructions to the guest during migration, > > > > although it's not possible to know how long it would take a guest to > > > > process a request made prior to the start of migration. > > > > > > > > Queueing the requests until after migration would be nice, but is > > > > non-trivial, since the set of inflate/deflate requests have to > > > > be compared with the state of the page to know what the final > > > > outcome is allowed to be. > > > > > > I didn't track the previous discussion, but there were plans to have > > > guest-initiated balloon requests for cases where the guest wants to > > > co-operate with hosts and return any free mem available We don't > > > currently have guests that do this, but we also don't want to have a > > > dependency between the host and guest -- they should be independent. > > > > > > This approach here seems the simplest possible, short of maintaining > > > another bitmap for the duration of postcopy which indicates > > > guest-freed memory pages which postcopy should not populate, after > > > receiving them at the dest (this sounds better to me than queuing up > > > guest requests). > > > > > > The downside here is that the guest offered some memory back, and we > > > don't use it. The guest also doesn't use it -- so it's a double loss, > > > of sorts. > > > > > > Thoughts? I don't have a problem with this current approach, but if > > > we could get something better, that'll be good too. > > > > It needs something like that bitmap, but it would take quite a bit > > of care to manage the interaction between: > > a) The guest emitting balloon notifications > > b) Pages being received from the source > > c) Destination use of that page > > > > we also have to think what to do with a page that's been ballooned > > after reception of the source page; the madvise(dontneed) that's used > > normally would cause userfault to fire again, and we can't allow that. > > (We could make it the same as receiving a zero page). But then we would > > also have to cope with the source sending us a page after the destination > > has ballooned it and make sure to discard that (I suspect there are further > > ordering examples that have to also be considered). > > Yeah. I'm fine with the current approach, with the downsides > mentioned. Maybe in the commit message, make it explicit that the > guest may think it's given up ownership, but the host won't honour > this till postcopy isn't finished. OK, I've added the text: 'Guest initiated ballooning will not know if it's really freed a page of host memory or not.' > Anyway: > > Reviewed-by: Amit Shah Thanks. Dave > > > Amit -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK