From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Cooper Subject: Re: [PATCH] iommu/quirk: disable shared EPT for Sandybridge and earlier processors. Date: Thu, 3 Dec 2015 11:24:14 +0000 Message-ID: <5660265E.2010102@citrix.com> References: <1448385479-17614-1-git-send-email-anshul.makkar@citrix.com> <5654AF4C02000078000B8A11@prv-mh.provo.novell.com> <56558D35.2040800@citrix.com> <56571B3202000078000B9652@prv-mh.provo.novell.com> <56570DB1.7020800@citrix.com> <56570F6D.7010907@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "Tian, Kevin" , Jan Beulich Cc: "Zhang, Yang Z" , Malcolm Crossley , Anshul Makkar , "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org On 03/12/15 01:19, Tian, Kevin wrote: >> From: Andrew Cooper [mailto:andrew.cooper3@citrix.com] >> Sent: Thursday, November 26, 2015 9:56 PM >> >> On 26/11/15 13:48, Malcolm Crossley wrote: >>> On 26/11/15 13:46, Jan Beulich wrote: >>>>>>> On 25.11.15 at 11:28, wrote: >>>>> The problem is that SandyBridge IOMMUs advertise 2M support and do >>>>> function with it, but cannot cache 2MB translations in the IOTLBs. >>>>> >>>>> As a result, attempting to use 2M translations causes substantially >>>>> worse performance than 4K translations. >>>> Btw - how does this get explained? At a first glance, even if 2Mb >>>> translations don't get entered into the TLB, it should still be one >>>> less page table level to walk for the IOMMU, and should hence >>>> nevertheless be a benefit. Yet you even say _substantially_ >>>> worse performance results. >>> There is a IOTLB for the 4K translation so if you only use 4K >>> translations then you get to take advantage of the IOTLB. >>> >>> If you use the 2Mb translation then a page table walk has to be >>> performed every time there's a DMA access to that region of the BFN >>> address space. >> Also remember that a high level dma access (from the point of view of a >> driver) will be fragmented at the PCIe max packet size, which is >> typically 256 bytes. >> >> So by not caching the 2Mb translation, a dma access of 4k may undergo 16 >> pagetable walks, one for each PCIe packet. >> >> We observed that using 2Mb mappings results in a 40% overhead, compared >> to using 4k mappings, from the point of view of a sample network workload. >> >> ~Andrew > One confusion here. The original patch just disables shared_ept, w/o > changing IOMMU to not use 2MB mapping. Is there something missing > or other tricks behind? Disabling shared_ept works because the Xen IOMMU interface doesn't support superpages. However, I subsequently changed my mind in this thread about the approach taken, and quirking the IOMMUs not to report superpage capabilities is the correct solution to the problem. ~Andrew