From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: pvops: AHCI problems with SB600 Date: Wed, 23 Sep 2009 17:24:57 -0400 Message-ID: <20090923212457.GA5816@phenom.dumpdata.com> References: <4AB431AD.1030205@goop.org> <4AB4EEB8.7050107@web.de> <20090921150634.GD20933@phenom.dumpdata.com> <4AB89227.8050302@web.de> <20090922140825.GB21736@phenom.dumpdata.com> <20090923120646.GA3199@phenom.dumpdata.com> <4ABA7578.8030201@goop.org> <20090923193245.GA5682@phenom.dumpdata.com> <4ABA8088.1080807@goop.org> <4ABA8574.2060501@goop.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <4ABA8574.2060501@goop.org> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Jeremy Fitzhardinge Cc: Patrick Scharrenberg , Xen-devel List-Id: xen-devel@lists.xenproject.org On Wed, Sep 23, 2009 at 01:30:44PM -0700, Jeremy Fitzhardinge wrote: > On 09/23/09 13:09, Jeremy Fitzhardinge wrote: > > if (!no_iommu && > > max_pfn > MAX_DMA32_PFN && > > !printed_gart_size_msg) { > > printk(KERN_ERR "you are using iommu with agp, but GART size is less than 64M\n"); > > printk(KERN_ERR "please increase GART size in your BIOS setup\n"); > > printk(KERN_ERR "if BIOS doesn't have that option, contact your HW vendor!\n"); > > printed_gart_size_msg = 1; > > } > > > > Oh, that's the wrong error message, but the other one has similar > predicates. Hm, but it also skips the test if (swiotlb && !valid_agp)... Right. We don't set the swiotlb. The reason being if you do set it then the original SWIOTLB kicks in. The weird part is that the function you copied-n-pasted (gart_iommu_hole_init) only detectes and allocates a buffer. It does not set the dma_ops at all. Setting of the dma_ops is done via the gart_iommu_init() call which is done much later. But with Xen-SWIOTLB already initialized, the gart_iommu_init() quits right away. So the kernel sets the dma_ops to the Xen SWIOTLB, and it allocates an extra 64MB chunk of memory for the GART, which is not used, and ... somehow all of the ioremap_nocache functions stop working correctly. Maybe the ioremap_nocache does use some of that memory that the gart_iommu_hole_init allocated? With this patch, the GART is forcefully disabled, and the kernel boots fine (with 6GB, 8GB, etc). diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c index 6b76948..1101a9f 100644 --- a/arch/x86/kernel/pci-dma.c +++ b/arch/x86/kernel/pci-dma.c @@ -122,6 +122,8 @@ void __init pci_iommu_alloc(void) * The order of these functions is important for * fall-back/fail-over reasons */ + xen_swiotlb_init(); + gart_iommu_hole_init(); detect_calgary(); @@ -130,8 +132,6 @@ void __init pci_iommu_alloc(void) amd_iommu_detect(); - xen_swiotlb_init(); - pci_swiotlb_init(); } diff --git a/arch/x86/xen/pci-swiotlb.c b/arch/x86/xen/pci-swiotlb.c index 5e2c856..00f2260 100644 --- a/arch/x86/xen/pci-swiotlb.c +++ b/arch/x86/xen/pci-swiotlb.c @@ -43,6 +43,10 @@ #include #include + +#include +#include + #define OFFSET(val,align) ((unsigned long) \ ( (val) & ( (align) - 1))) @@ -985,5 +989,9 @@ void __init xen_swiotlb_init(void) xen_swiotlb_init_with_default_size(64 * (1<<20)); /* default to 64MB */ dma_ops = &xen_swiotlb_dma_ops; iommu_detected = 1; +#ifdef CONFIG_GART_IOMMU + gart_iommu_aperture_disabled = 1; +#endif + } }