Re: Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Mon, Oct 05, 2009 at 10:01:52AM -0400, Konrad Rzeszutek Wilk wrote:
> On Mon, Oct 05, 2009 at 11:32:31AM +0100, Jan Beulich wrote:
> > >>> Jeremy Fitzhardinge <jeremy@goop.org> 02.10.09 20:42 >>>
> > >On 10/02/09 10:23, Boris Derzhavets wrote:
> > >> Jeremy,
> > >> Please,  be aware of bugzilla.xensource.com [1519]  the most recent
> > >> entries :-
> > >>
> > >> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1519 
> > >>
> > >
> > >Ah, OK.  I pushed a variant of Konrad's patches.  Could you try them out?
> > 
> > Are you really convinced fixing this in DRM is the right thing to do? To
> > me, the use of vmalloc_32() in drivers/ieee1394/ seems to make similar
> > assumptions (pci_map_sg() not modifying the buffer addresses), and
> > who knows where else such assumptions exist. After all, vmalloc_32()
> > is *defined* to have the property assumed by both of the users, and
> > other than for most kmalloc() cases using GFP_DMA{,32} we're talking
> > about double buffering generally large amounts of data here even in
> > the cases where the DMA API is being used properly.
> 
> I was chatting with Jeremy about this, and I realized that in some
> sense the radeon driver assumes that physical != bus addresses. Which is
> OK on x86, but if this card was ever moved to a Sun box it would fail.
> 
> The patch here thwarts this by allocating memory from the IOMMU, so that
> even if bus != physical address from pci_map_page, that is OK.

That is, if the IOMMU, allocates memory from the coherent pool to be same
type of memory that it allocates when calling pci_map_[sg|[page].

> 
> Another way to make this work right is to modify the drivers
> that use vmalloc_32, with pci_map_[sg|page], is for them to check if the physical
> != bus address, and if so, remap the virtual address (page_to_vmalloc) to
> point to the new bus addresses (and free the "old" page). Obviously this
> requires some book-keeping, but it does fix those drivers if they were to move to
> non-x86 architecture. Or on machines where the IOMMU hands out non-physical
> addresses.

Jeremy suggested another one. That is for the Xen IOMMU to fix the virtual
mappings. Meaning we would swipe out the virtual address of the page to point
to a virtual address (along with returning an 32-bit bus address) under the 4GB mark.
That involves also changing the mappings of the old virtual address to the new one
where ever it may be.

Jeremy, did I get that right?