Re: Infiniband 40GB

[Yann Dupont <Yann.Dupont@univ-nantes.fr>]

Le 04/06/2012 11:35, Alexandre DERUMIER a écrit :
> Hi,
> about this:
>>> Turning off Virtualisation extension in BIOS. Don't know why, but it
>>> gaves us crappy performance. We usually put it on, because we use KVM a
>>> lot. In our case, OSD are in bare metal and disabling virtualisation
>>> extension gives us a very big boost.
>>> It may be a BIOS bug in our machines (DELL M610).
>
> It could be related to iommu, if you pass intel_iommu=on in grub.
> I have already had this kind of problem.
>
> When intel_iommu=on, Linux (completely unrelated to KVM) adds a new level
> of protection which didn't exist without an IOMMU - the network card, which
> without an IOMMU could write (via DMA) to any memory location, now is
> not allowed - the card can only write to memory locates which the OS
> wanted it to write. Theoretically, this can protect the OS against
> various kinds of attacks. But what happens now is that every time that
> Linux passes a new buffer to the card, it needs to change the IOMMU
> mappings. This noticably slows down I/O, unfortunately.
>
>

Infortunately, this is not the case. The intel card supports it, but 
DELL M160 don't.
And I just checked, ou linux command line don't include intel_iommu=on.

BTW, it seems that turning on virtualization on bios kills performance 
on integrated ixgbe driver. Sourceforge one seems less affected. Our 
tests were circa kernel 3.2 , it may have changed since.

Cheers,



-- 
Yann Dupont - Service IRTS, DSI Université de Nantes
Tel : 02.53.48.49.20 - Mail/Jabber : Yann.Dupont@univ-nantes.fr


--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html