Re: [PATCH]: PCI: GART iommu alignment fixes [v2]

[Prarit Bhargava <prarit@redhat.com>]

>> Interesting. Have you experienced any problems because of that
>> misbehavior in the GART code? AMD IOMMU currently also violates this
>> requirement. I will send a patch to fix that there too.
>>     
>
>   

Joerg,  yes I can see misbehavior caused by this code.  O/w I wouldn't 
be spending my time fixing it :) :)

See below ....

> IIRC, only PARISC and POWER IOMMUs follow the above rule. So I also
> wondered what problem he hit.
>   

I wonder if IBM's Calgary IOMMU needs this fix? ... I've added Ed 
Pollard to find out.

On big memory footprint (16G or above) systems it is possible that the 
e820 map reserves most of the lower 4G of memory for system use*.  So 
it's possible that the 4G region is almost completely reserved at boot 
time and so the kernel starts using the IOMMU for DMA (see 
dma_alloc_coherent()).  The addresses returned are not properly aligned, 
and this causes serious problems for some drivers that require a 
physical aligned address for the device.

P.

* I have one large system with 64G of memory on which I can reproduce 
this issue very quickly.  Even booting the system with mem=16G seems to 
cause the problem, although I did have to load a module that reserved a 
few M of DMA addresses before I started alloc'ing from the IOMMU.

I also reproduced this on a smaller system by loading one module that 
reserved as much DMA-able region as possible, and then loaded another 
module that reserved from the IOMMU.  While this situation is a bit 
contrived the bug still hit -- the returned addresses are not properly 
aligned.