Re: pvops: AHCI problems with SB600

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Thu, Sep 24, 2009 at 11:23:16AM -0700, Jeremy Fitzhardinge wrote:
> On 09/24/09 05:44, Konrad Rzeszutek Wilk wrote:
> > There was a lot of havoc - all of the PCI BARs were useless. Is the MFN
> > (from the pfn_to_mfn on this address) suppose to have a specific value?
> >   
> 
> Not sure.  pfn_to_mfn is never supposed to happen on ioremap phys addrs,
> because of _PAGE_IOMAP in the pte.  Its probably worth checking that
> _PAGE_IOMAP is actually getting set.

<sigh> I found the problem. I did not power off the machine after doing
a non-Xen boot where the GART was used. I am not entirely sure what
the GART was doing, but pretty much all writew/readw were not doing/going
where they were suppose to.

> 
> > For all of those setting, no_iommu=1 should do the trick. But in reality
> > I need to double-check that:
> >
> >
> > diff --git a/arch/x86/xen/pci-swiotlb.c b/arch/x86/xen/pci-swiotlb.c
> > index 00f2260..390f698 100644
> > --- a/arch/x86/xen/pci-swiotlb.c
> > +++ b/arch/x86/xen/pci-swiotlb.c
> > @@ -989,6 +989,8 @@ void __init xen_swiotlb_init(void)
> >  		xen_swiotlb_init_with_default_size(64 * (1<<20));	/* default to 64MB */
> >  		dma_ops = &xen_swiotlb_dma_ops;
> >  		iommu_detected = 1;
> > +		no_iommu = 1; /* Forces the other IOMMU (if they are detected) to
> > +				to quit, rather than initialize. */
> >  #ifdef CONFIG_GART_IOMMU
> >  		gart_iommu_aperture_disabled = 1;
> >  #endif
> >
> > <sigh>I think I need to rethink this swiotlb-Xen part. This is starting
> > to look like a hack.
> >   
> 
> It isn't great.  We need a way to either layer or arbitrate between
> these different address translation mechanisms.

I am sending another patch that I think is more nicer, and it takes care of the other IOMMUs.
> 
> >> Another thought, could we actually use the gart iommu instead of swiotlb
> >> if it is available?  I think it leads to exactly the same set of issues
> >> as extending normal swiotlb for Xen's use (ie, inserting pfn->mfn
> >> conversion into the correct places, and perhaps allocating the memory
> >> properly).  Worth thinking about; it may shine light on better ways to
> >> fix up swiotlb.
> >>     
> > Yes! That was my next step - see if it is possible to use it and if so
> > extend it for that purpose (and without any ghastly #ifdef).
> >   
> 
> Good.

Still thinking about this ..