From mboxrd@z Thu Jan 1 00:00:00 1970 From: Matthew Rushton Subject: Re: [RFC PATCH] page_alloc: use first half of higher order chunks when halving Date: Tue, 25 Mar 2014 13:18:31 -0700 Message-ID: <5331E497.20408@gmail.com> References: <1395746524-9670-1-git-send-email-msw@linux.com> <53316C13.4090006@citrix.com> <20140325132044.GA11708@u109add4315675089e695.ant.amazon.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WSXny-0004Ry-02 for xen-devel@lists.xenproject.org; Tue, 25 Mar 2014 20:18:38 +0000 Received: by mail-pa0-f52.google.com with SMTP id rd3so946918pab.11 for ; Tue, 25 Mar 2014 13:18:34 -0700 (PDT) In-Reply-To: <20140325132044.GA11708@u109add4315675089e695.ant.amazon.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Matt Wilson , Andrew Cooper Cc: Keir Fraser , Jan Beulich , Tim Deegan , Matt Wilson , xen-devel@lists.xenproject.org List-Id: xen-devel@lists.xenproject.org On 03/25/14 06:20, Matt Wilson wrote: > On Tue, Mar 25, 2014 at 11:44:19AM +0000, Andrew Cooper wrote: >> On 25/03/14 11:22, Matt Wilson wrote: >>> From: Matt Rushton >>> >>> This patch makes the Xen heap allocator use the first half of higher >>> order chunks instead of the second half when breaking them down for >>> smaller order allocations. >>> >>> Linux currently remaps the memory overlapping PCI space one page at a >>> time. Before this change this resulted in the mfns being allocated in >>> reverse order and led to discontiguous dom0 memory. This forced dom0 >>> to use bounce buffers for doing DMA and resulted in poor performance. >>> >>> This change more gracefully handles the dom0 use case and returns >>> contiguous memory for subsequent allocations. >>> >>> Cc: xen-devel@lists.xenproject.org >>> Cc: Keir Fraser >>> Cc: Jan Beulich >>> Cc: Konrad Rzeszutek Wilk >>> Cc: Tim Deegan >>> Cc: Andrew Cooper >>> Signed-off-by: Matt Rushton >>> Signed-off-by: Matt Wilson >> How does dom0 work out that it is safe to join multiple pfns into a dma >> buffer without the swiotlb? > I'm not familiar enough with how this works to say. Perhaps Matt R. > can chime in during his day. My guess is that xen_swiotlb_alloc_coherent() > avoids allocating a contiguous region if the pages allocated already > happen to be physically contiguous. > > Konrad, can you enlighten us? The setup code in question that does the > remapping one page at a time is in arch/x86/xen/setup.c. > The swiotlb code will check if the underlying mfns are contiguous and use a bounce buffer if and only if they are not. Everything goes through the swiotlb via the normal Linux dma apis it's just a matter of if it uses a bounce buffer or not. >>> --- >>> xen/common/page_alloc.c | 5 +++-- >>> 1 file changed, 3 insertions(+), 2 deletions(-) >>> >>> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c >>> index 601319c..27e7f18 100644 >>> --- a/xen/common/page_alloc.c >>> +++ b/xen/common/page_alloc.c >>> @@ -677,9 +677,10 @@ static struct page_info *alloc_heap_pages( >>> /* We may have to halve the chunk a number of times. */ >>> while ( j != order ) >>> { >>> - PFN_ORDER(pg) = --j; >>> + struct page_info *pg2; >> At the very least, Xen style mandates a blank line after this variable >> declaration. > Ack. > >> ~Andrew >> >>> + pg2 = pg + (1 << --j); >>> + PFN_ORDER(pg) = j; >>> page_list_add_tail(pg, &heap(node, zone, j)); >>> - pg += 1 << j; >>> } >>> >>> ASSERT(avail[node][zone] >= request);