Re: Load increase after memory upgrade (part2)

[Laszlo Ersek <lersek@redhat.com>]

On 11/28/11 16:40, Ian Campbell wrote:
> On Mon, 2011-11-28 at 15:28 +0000, Konrad Rzeszutek Wilk wrote:
>> On Fri, Nov 25, 2011 at 11:11:55PM +0100, Carsten Schiers wrote:
>
>>> I looked through my old mails from you and you explained already the necessity of double
>>> bounce buffering (PCI->below 4GB->above 4GB). What I don't understand is: why does the
>>> Xenified kernel not have this kind of issue?
>>
>> That is a puzzle. It should not. The code is very much the same - both
>> use the generic SWIOTLB which has not changed for years.
>
> The swiotlb-xen used by classic-xen kernels (which I assume is what
> Carsten means by "Xenified") isn't exactly the same as the stuff in
> mainline Linux, it's been heavily refactored for one thing. It's not
> impossible that mainline is bouncing something it doesn't really need
> to.

Please excuse me if I'm completely mistaken; my only point of reference 
is that we recently had to backport 
<http://xenbits.xensource.com/hg/linux-2.6.18-xen.hg/rev/940>.

> It's also possible that the dma mask of the device is different/wrong in
> mainline leading to such additional bouncing.

dma_alloc_coherent() -- which I guess is the precursor of 
pci_alloc_consistent() -- asks xen_create_contiguous_region() to back 
the vaddr range with frames machine-addressible inside the device's dma 
mask. xen_create_contiguous_region() seems to land in a XENMEM_exchange 
hypercall (among others). Perhaps this extra layer of indirection allows 
the driver to use low pages directly, without bounce buffers.

> I guess it's also possible that the classic-Xen kernels are playing fast
> and loose by not bouncing something they should (although if so they
> appear to be getting away with it...) or that there is some difference
> which really means mainline needs to bounce while classic-Xen doesn't.

I'm sorry if what I just posted is painfully stupid. I'm taking the risk 
for the 1% chance that it could be helpful.

Wrt. the idle time accounting problem, after Niall's two pings, I'm also 
waiting for a verdict, and/or for myself finding the time and fishing 
out the current patches.

Laszlo