From mboxrd@z Thu Jan  1 00:00:00 1970
From: Konrad Rzeszutek Wilk <konrad@darnok.org>
Subject: Re: Load increase after memory upgrade (part2)
Date: Wed, 14 Dec 2011 16:23:51 -0400
Message-ID: <20111214202351.GA25896@andromeda.dapyr.net>
References: <zarafa.4ece387c.6e85.6aa0c9ca21e63bd9@uhura.space.zz>
	<zarafa.4edb62f8.6417.1a54a5c25b8f0fd7@uhura.space.zz>
	<20111206032621.GA6568@phenom.dumpdata.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xensource.com>
Content-Disposition: inline
In-Reply-To: <20111206032621.GA6568@phenom.dumpdata.com>
List-Unsubscribe: <http://lists.xensource.com/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xensource.com>
List-Help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-Subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
Sender: xen-devel-bounces@lists.xensource.com
Errors-To: xen-devel-bounces@lists.xensource.com
To: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: "lersek@redhat.com" <lersek@redhat.com>, xen-devel <xen-devel@lists.xensource.com>, Carsten Schiers <carsten@schiers.de>, Ian Campbell <Ian.Campbell@citrix.com>, "zhenzhong.duan@oracle.com" <zhenzhong.duan@oracle.com>
List-Id: xen-devel@lists.xenproject.org

On Mon, Dec 05, 2011 at 10:26:21PM -0500, Konrad Rzeszutek Wilk wrote:
> On Sun, Dec 04, 2011 at 01:09:28PM +0100, Carsten Schiers wrote:
> > Here with two cards enabled and creating a bit "work" by watching TV with one oft hem:
> > 
> > [   23.842720] Starting SWIOTLB debug thread.
> > [   23.842750] swiotlb_start_thread: Go!
> > [   23.842838] xen_swiotlb_start_thread: Go!
> > [   28.841451] 0 [budget_av 0000:00:01.0] bounce: from:435596(slow:0)to:0 map:658 unmap:0 sync:435596
> > [   28.841592] SWIOTLB is 4% full
> > [   33.840147] 0 [budget_av 0000:00:01.0] bounce: from:127652(slow:0)to:0 map:0 unmap:0 sync:127652
> > [   33.840283] SWIOTLB is 4% full
> > [   33.844222] 0 budget_av 0000:00:01.0 alloc coherent: 8, free: 0
> > [   38.840227] 0 [budget_av 0000:00:01.0] bounce: from:128310(slow:0)to:0 map:0 unmap:0 sync:128310
> 
> Whoa. Yes. You are definitly using the bounce buffer :-)
> 
> Now it is time to look at why the drive is not using those coherent ones - it
> looks to allocate just eight of them but does not use them.. Unless it is
> using them _and_ bouncing them (which would be odd).
> 
> And BTW, you can lower your 'swiotlb=XX' value.  The 4% is how much you
> are using of the default size.

So I able to see this with an atl1c ethernet driver on my SandyBridge i3
box. It looks as if the card is truly 32-bit so on a box with 8GB it
bounces the data. If I booted the Xen hypervisor with 'mem=4GB' I get no
bounces (no surprise there).

In other words - I see the same behavior you are seeing. Now off to:
> 
> I should find out_why_ the old Xen kernels do not use the bounce buffer
> so much...

which will require some fiddling around.