Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Don Dutile <ddutile@redhat.com>]

On 05/07/2015 09:21 PM, Dave Young wrote:
> On 05/07/15 at 10:25am, Don Dutile wrote:
>> On 05/07/2015 10:00 AM, Dave Young wrote:
>>> On 04/07/15 at 10:12am, Don Dutile wrote:
>>>> On 04/06/2015 11:46 PM, Dave Young wrote:
>>>>> On 04/05/15 at 09:54am, Baoquan He wrote:
>>>>>> On 04/03/15 at 05:21pm, Dave Young wrote:
>>>>>>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>>>>>>>> Hi Dave,
>>>>>>>>
>>>>>>>> There may be some possibilities that the old iommu data is corrupted by
>>>>>>>> some other modules. Currently we do not have a better solution for the
>>>>>>>> dmar faults.
>>>>>>>>
>>>>>>>> But I think when this happens, we need to fix the module that corrupted
>>>>>>>> the old iommu data. I once met a similar problem in normal kernel, the
>>>>>>>> queue used by the qi_* functions was written again by another module.
>>>>>>>> The fix was in that module, not in iommu module.
>>>>>>>
>>>>>>> It is too late, there will be no chance to save vmcore then.
>>>>>>>
>>>>>>> Also if it is possible to continue corrupt other area of oldmem because
>>>>>>> of using old iommu tables then it will cause more problems.
>>>>>>>
>>>>>>> So I think the tables at least need some verifycation before being used.
>>>>>>>
>>>>>>
>>>>>> Yes, it's a good thinking anout this and verification is also an
>>>>>> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
>>>>>> and then verify this again when panic happens in purgatory. This checks
>>>>>> whether any code stomps into region reserved for kexec/kernel and corrupt
>>>>>> the loaded kernel.
>>>>>>
>>>>>> If this is decided to do it should be an enhancement to current
>>>>>> patchset but not a approach change. Since this patchset is going very
>>>>>> close to point as maintainers expected maybe this can be merged firstly,
>>>>>> then think about enhancement. After all without this patchset vt-d often
>>>>>> raised error message, hung.
>>>>>
>>>>> It does not convince me, we should do it right at the beginning instead of
>>>>> introduce something wrong.
>>>>>
>>>>> I wonder why the old dma can not be remap to a specific page in kdump kernel
>>>>> so that it will not corrupt more memory. But I may missed something, I will
>>>>> looking for old threads and catch up.
>>>>>
>>>>> Thanks
>>>>> Dave
>>>>>
>>>> The (only) issue is not corruption, but once the iommu is re-configured, the old,
>>>> not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
>>>> will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
>>>> in the kexec kernel.
>>>>
>>>
>>> Don, so if iommu is not reconfigured then these faults will not happen?
>>>
>> Well, if iommu is not reconfigured, then if the crash isn't caused by
>> an IOMMU fault (some systems have firmware-first catch the IOMMU fault & convert
>> them into NMI_IOCK), then the DMA's will continue into the old kernel memory space.
>
> So NMI_IOCK is one reason to cause kernel hang, I think I'm still not clear about
> what does re-configured means though. DMAR faults will happen originally this is the old
> behavior but we are removing the faults by alowing DMA continuing into old memory
> space.
>
A flood of faults occur when the 2nd kernel (re-)configures the IOMMU because
the second kernel effectively clears/disable all DMA except RMRRs, so any DMA from 1st kernel will flood
the system with faults.  Its the flood of dmar faults that eventually wedges &/or crashes the system
while trying to take a kdump.

>>
>>> Baoquan and me has a confusion below today about iommu=off/intel_iommu=off:
>>>
>>> intel_iommu_init()
>>> {
>>> ...
>>>
>>>         dmar_table_init();
>>>
>>>         disable active iommu translations;
>>>
>>>         if (no_iommu || dmar_disabled)
>>>                  goto out_free_dmar;
>>>
>>> ...
>>> }
>>>
>>> Any reason not move no_iommu check to the begining of intel_iommu_init function?
>>>
>> What does that do/help?
>
> Just do not know why the previous handling is necessary with iommu=off, shouldn't
> we do noting and return earlier?
>
> Also there is a guess, dmar faults appears after iommu_init, so not sure if the codes
> before dmar_disabled checking have some effect about enabling the faults messages.
>
> Thanks
> Dave
>