From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: decent performance drop for SCSI LLD / SAN initiator when iommu is turned on Date: Thu, 2 May 2013 04:56:03 +0300 Message-ID: <20130502015603.GC26105@redhat.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Or Gerlitz Cc: Roland Dreier , iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, Yan Burman , linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-Id: linux-rdma@vger.kernel.org On Thu, May 02, 2013 at 02:11:15AM +0300, Or Gerlitz wrote: > Hi Roland, IOMMU folks, > > So we've noted that when configuring the kernel && booting with intel > iommu set to on on a physical node (non VM, and without enabling SRIOV > by the HW device driver) raw performance of the iSER (iSCSI RDMA) SAN > initiator is reduced notably, e.g in the testbed we looked today we > had ~260K 1KB random IOPS and 5.5GBs BW for 128KB IOs with iommu > turned off for single LUN, and ~150K IOPS and 4GBs BW with iommu > turned on. No change on the target node between runs. That's why we have iommu=pt. See definition of iommu_pass_through in arch/x86/kernel/pci-dma.c. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html