From mboxrd@z Thu Jan 1 00:00:00 1970 From: Matthew Rushton Subject: Re: [RFC PATCH] page_alloc: use first half of higher order chunks when halving Date: Wed, 26 Mar 2014 10:47:44 -0700 Message-ID: <533312C0.1050507@gmail.com> References: <1395746524-9670-1-git-send-email-msw@linux.com> <20140325121922.GB12931@deinos.phlegethon.org> <20140325132740.GB11708@u109add4315675089e695.ant.amazon.com> <5331E269.9090708@gmail.com> <20140326095533.GA7885@deinos.phlegethon.org> <20140326101746.GA14195@u109add4315675089e695.ant.amazon.com> <20140326150801.GD18387@phenom.dumpdata.com> <20140326151507.GF14195@u109add4315675089e695.ant.amazon.com> <5332F948.1020909@gmail.com> <20140326163609.GD21368@phenom.dumpdata.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WSrvZ-0001dy-3b for xen-devel@lists.xenproject.org; Wed, 26 Mar 2014 17:47:49 +0000 Received: by mail-pa0-f54.google.com with SMTP id lf10so2266686pab.27 for ; Wed, 26 Mar 2014 10:47:45 -0700 (PDT) In-Reply-To: <20140326163609.GD21368@phenom.dumpdata.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Konrad Rzeszutek Wilk Cc: Keir Fraser , Matt Wilson , Matt Wilson , Tim Deegan , Jan Beulich , Andrew Cooper , xen-devel@lists.xenproject.org List-Id: xen-devel@lists.xenproject.org On 03/26/14 09:36, Konrad Rzeszutek Wilk wrote: > On Wed, Mar 26, 2014 at 08:59:04AM -0700, Matthew Rushton wrote: >> On 03/26/14 08:15, Matt Wilson wrote: >>> On Wed, Mar 26, 2014 at 11:08:01AM -0400, Konrad Rzeszutek Wilk wrote: >>>> Could you elaborate a bit more on the use-case please? >>>> My understanding is that most drivers use a scatter gather list - in which >>>> case it does not matter if the underlaying MFNs in the PFNs spare are >>>> not contingous. >>>> >>>> But I presume the issue you are hitting is with drivers doing dma_map_page >>>> and the page is not 4KB but rather large (compound page). Is that the >>>> problem you have observed? >>> Drivers are using very large size arguments to dma_alloc_coherent() >>> for things like RX and TX descriptor rings. > Large size like larger than 512kB? That would also cause problems > on baremetal then when swiotlb is activated I believe. I was looking at network IO performance so the buffers would not have been that large. I think large in this context is relative to the 4k page size and the odds of the buffer spanning a page boundary. For context I saw ~5-10% performance increase with guest network throughput by avoiding bounce buffers and also saw dom0 tcp streaming performance go from ~6Gb/s to over 9Gb/s on my test setup with a 10Gb NIC. > >>> --msw >> It's the dma streaming api I've noticed the problem with, so >> dma_map_single(). Applicable swiotlb code would be >> xen_swiotlb_map_page() and range_straddles_page_boundary(). So yes >> for larger buffers it can cause bouncing.