On 02/12/2015 01:25 PM, Mark Hounschell wrote:
> On 02/12/2015 01:08 PM, Joerg Roedel wrote:
>> On Thu, Feb 12, 2015 at 12:53:42PM -0500, Mark Hounschell wrote:
>>> This happens immediately after unloading one of our out of kernel GPL
>>> drivers.
>>> The driver has done NOTHING other than load at bootup.  I'm running a
>>> 3.18.7
>>> kernel (x86_64) on an AMD platform. I can't see anything obviously
>>> wrong in our
>>> driver. It works fine when the iommu is disabled. This particular
>>> machine has 7 of
>>> our cards in it. Four in one expansion rack and 3 in another. The 2
>>> PCI expansion
>>> racks use pci-e interface cards installed in the MB.
>>>
>>> Feb 12 10:47:15 harley kernel: AMD-Vi: Event logged [IO_PAGE_FAULT
>>> device=0f:00.0 domain=0x0000 address=0x00000000000ae640 flags=0x0070]
>>> Feb 12 10:47:15 harley kernel: AMD-Vi: Event logged [IO_PAGE_FAULT
>>> device=0f:00.0 domain=0x0000 address=0x00000000000ae660 flags=0x0070]
>>> Feb 12 10:47:15 harley kernel: AMD-Vi: Event logged [IO_PAGE_FAULT
>>> device=0f:00.0 domain=0x0000 address=0x00000000000ae670 flags=0x0070]
>>> Feb 12 10:47:27 harley kernel: ------------[ cut here ]------------
>>> Feb 12 10:47:27 harley kernel: WARNING: CPU: 3 PID: 0 at
>>> drivers/iommu/amd_iommu.c:2637 dma_ops_domain_unmap.part.13+0x65/0x70()
>>
>> This warning indicates that some driver is unmapping a dma range that
>> was not mapped previously (meaning that a pte in the io-page-tables is
>> zeroed out).
>> The reason for this (and the IO_PAGE_FAULTs) you see are almost
>> certainly because some driver does not use the DMA-API correctly.
>>
>>
>
> I wonder what driver that could be. It certainly isn't the one that I
> just unloaded
> as it for sure has not done anything dma realated. I'm pretty sure I
> uninstalled all
> our other drivers but will go back and verify.
>

I've cleaned the machine of all our drivers and also the nvidia driver. 
If the problem is as you say, it is an in-kernel driver. I've attached a 
dmesg taken after it started. I used the dgap driver from the staging 
directory to unload and trigger this as I have one. That driver does NO 
dma. I know because I'm one of the maintainers and have done lots of 
work on it.

Thanks
Mark