Re: [PATCH] intel-iommu: Manage iommu_coherency globally

[David Woodhouse <dwmw2@infradead.org>]

[-- Attachment #1: Type: text/plain, Size: 1389 bytes --]

On Sat, 2011-11-19 at 11:17 -0800, Chris Wright wrote:
> > Hm, it seems I lied to you about this. The non-coherent mode isn't just
> > a historical mistake; it's configurable by the BIOS, and we actually
> > encourage people to use the non-coherent mode because it makes the
> > hardware page-walk faster — so reduces the latency for IOTLB misses.
> 
> Interesting because for the workloads I've tested it's the exact opposite.
> Tested w/ BIOS enabling and disabling coherency, and w/ non-coherent
> access and streaming DMA (i.e. bare metal NIC bw testing)...the IOMMU
> added smth like 10% when non-coherent vs. coherent. 

Right. I specifically said "latency for IOTLB misses". That is the main
reason we apparently advise BIOS authors to use non-coherent mode.

(Of course, the fact that we're letting the BIOS control this at all
means that we obviously haven't learned *any* lessons from the last few
years of pain with VT-d enabling and BIOS bugs. It should have been
under OS control. But that's a separate, stunningly depressing, issue.)

Of course, The *overall* performance sucks rocks if we are in
non-coherent mode. The bare-metal NIC bandwidth testing is the obvious
test case for it, and thanks for providing ballpark numbers for that.
They perfectly back up the qualitative remarks I made to Rajesh
yesterday on exactly this topic.

-- 
dwmw2

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5818 bytes --]