From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeremy Fitzhardinge Subject: Re: pvops: AHCI problems with SB600 Date: Wed, 23 Sep 2009 14:56:16 -0700 Message-ID: <4ABA9980.1020007@goop.org> References: <4AB431AD.1030205@goop.org> <4AB4EEB8.7050107@web.de> <20090921150634.GD20933@phenom.dumpdata.com> <4AB89227.8050302@web.de> <20090922140825.GB21736@phenom.dumpdata.com> <20090923120646.GA3199@phenom.dumpdata.com> <4ABA7578.8030201@goop.org> <20090923193245.GA5682@phenom.dumpdata.com> <4ABA8088.1080807@goop.org> <4ABA8574.2060501@goop.org> <20090923212457.GA5816@phenom.dumpdata.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20090923212457.GA5816@phenom.dumpdata.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Konrad Rzeszutek Wilk Cc: Patrick Scharrenberg , Ian Campbell , Xen-devel List-Id: xen-devel@lists.xenproject.org On 09/23/09 14:24, Konrad Rzeszutek Wilk wrote: > The weird part is that the function you copied-n-pasted (gart_iommu_hole_init) > only detectes and allocates a buffer. It does not set the dma_ops at all. > Setting of the dma_ops is done via the gart_iommu_init() call which is done > much later. But with Xen-SWIOTLB already initialized, the gart_iommu_init() > quits right away. > Perhaps the fix is to make gart_iommu_hole_init() quit if iommu_detected too? Though it is called after xen_swiotlb_init()... > So the kernel sets the dma_ops to the Xen SWIOTLB, and it > allocates an extra 64MB chunk of memory for the GART, which is not > used, and ... somehow all of the ioremap_nocache functions stop working > correctly. Maybe the ioremap_nocache does use some of that memory that > the gart_iommu_hole_init allocated? > Can't see how it would affect it. ioremap allocates a new virtual space for the mapping and then just plugs in the pfns for the pages you want to map. They end up getting _PAGE_IOMAP set in the pte flags, which causes the xen/mmu.c backend to use the address as-is (ie, as an mfn), so the mapping will be constructed properly. Well, that's the theory; but I'd expect we'd be seeing a lot more havok of ioremap is either mapping the wrong pages or using the wrong caching. > With this patch, the GART is forcefully disabled, and the kernel boots fine > (with 6GB, 8GB, etc). > OK, I'll put it in for now. Will we have issues with other forms of iommu? Another thought, could we actually use the gart iommu instead of swiotlb if it is available? I think it leads to exactly the same set of issues as extending normal swiotlb for Xen's use (ie, inserting pfn->mfn conversion into the correct places, and perhaps allocating the memory properly). Worth thinking about; it may shine light on better ways to fix up swiotlb. Thanks, J