From mboxrd@z Thu Jan 1 00:00:00 1970 From: Juergen Gross Subject: Re: Issue with PV superpage handling Date: Mon, 09 Jul 2012 08:02:24 +0200 Message-ID: <4FFA73F0.3000805@ts.fujitsu.com> References: <201206250938.21418.dcm@mccr.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <201206250938.21418.dcm@mccr.org> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Dave McCracken Cc: George Dunlap , Ian Campbell , Xen Developers List-Id: xen-devel@lists.xenproject.org Am 25.06.2012 16:38, schrieb Dave McCracken: > > Awhile back I added the domain config flag "superpages" to support Linux > hugepages in PV domains. When the flag is set, the PV domain is populated > entirely with superpages. If not enough superpage-sized chunks can be found, > the domain creation fails. > > At some time after my patch was accepted, the code I added to domain restore > was removed because I broke page allocation batching. I put it on my TODO > list to reimplement it, then it got lost, for which I apologize. > > Now I have gotten back to reimplementing PV superpage support in restore, I > find that recently other code was added to restore that, while triggered by > the superpage flag, only allocates superpages opportunistically and falls back > to small pages if it fails. This breaks the original semantics of the flag > and could cause any OS that depends on the semantics to fail catastrophically. > > I have a patch that implements the original semantics of the superpage flag > while preserving the batch allocation behavior. I can remove the competing > code and submit mine, but I have a question. What value is there in > implementing opportunistic allocation of superpages for a PV (or an HVM) > domain in restore? It clearly can't be based on the superpages flag. > Opportunistic superpage allocation is already the default behavior for HVM > domain creation. Should it also be a default on HVM restore? What about for > PV domains? Is there any real benefit? There is a real benefit. We are seeing severe performance penalties after migrating a HVM domain. Performance is going down by 10% or more! Our OS (BS2000) is trying to use superpages where possible. Before live migration I can see that the complete memory for the domain is allocated in at least 2MB chunks, after the migration not a single superpage is left. With EPT this not only makes each TLB-miss more expensive, but there are much more TLB-misses, as no 2MB TLB-entries are possible at all! Juergen -- Juergen Gross Principal Developer Operating Systems PDG ES&S SWE OS6 Telephone: +49 (0) 89 3222 2967 Fujitsu Technology Solutions e-mail: juergen.gross@ts.fujitsu.com Domagkstr. 28 Internet: ts.fujitsu.com D-80807 Muenchen Company details: ts.fujitsu.com/imprint.html