Re: DMAR-IR: IRQ remapping was enabled on dmar6 but we are not in kdump mode

[Baolu Lu <baolu.lu@linux.intel.com>]

On 6/8/24 7:07 PM, Paul Menzel wrote:
> Dear Linux folks,
> 
> 
> Am 15.05.24 um 08:02 schrieb Paul Menzel:
> 
>> Am 15.05.24 um 04:13 schrieb Baolu Lu:
>>> On 5/15/24 3:46 AM, Paul Menzel wrote:
>>>> Am 23.01.24 um 01:55 schrieb Baolu Lu:
>>>>> On 2024/1/22 22:53, Paul Menzel wrote:
>>>>>> Am 22.01.24 um 13:38 schrieb Baolu Lu:
>>>>>>> On 2024/1/19 22:45, Paul Menzel wrote:
>>>>>>>>
>>>>>>>> On a Dell PowerEdge T640, Linux 5.9 and 6.6.12 warn about kdump:
>>>>>>>>
>>>>>>>>      [    2.728445] DMAR-IR: IRQ remapping was enabled on dmar6 
>>>>>>>> but we are not in kdump mode
>>>>>>>>      [    2.736544] DMAR-IR: IRQ remapping was enabled on dmar5 
>>>>>>>> but we are not in kdump mode
>>>>>>>>      [    2.744620] DMAR-IR: IRQ remapping was enabled on dmar4 
>>>>>>>> but we are not in kdump mode
>>>>>>>>      [    2.752695] DMAR-IR: IRQ remapping was enabled on dmar3 
>>>>>>>> but we are not in kdump mode
>>>>>>>>      [    2.760774] DMAR-IR: IRQ remapping was enabled on dmar2 
>>>>>>>> but we are not in kdump mode
>>>>>>>>      [    2.768847] DMAR-IR: IRQ remapping was enabled on dmar1 
>>>>>>>> but we are not in kdump mode
>>>>>>>>      [    2.776922] DMAR-IR: IRQ remapping was enabled on dmar0 
>>>>>>>> but we are not in kdump mode
>>>>>>>>      [    2.784999] DMAR-IR: IRQ remapping was enabled on dmar7 
>>>>>>>> but we are not in kdump mode
>>>>>>>>
>>>>>>>> Looking through the logs, this only happens when using kexec to 
>>>>>>>> restart the system.
>>>>>>>
>>>>>>> The code that warned this is,
>>>>>>>
>>>>>>>   599         if (ir_pre_enabled(iommu)) {
>>>>>>>   600                 if (!is_kdump_kernel()) {
>>>>>>>   601                         pr_warn("IRQ remapping was enabled 
>>>>>>> on %s but we are not in kdump mode\n",
>>>>>>>   602                                 iommu->name);
>>>>>>>   603                         clear_ir_pre_enabled(iommu);
>>>>>>>   604                         iommu_disable_irq_remapping(iommu);
>>>>>>>   605                 }
>>>>>>>
>>>>>>> The VT-d interrupt remapping is enabled during boot, but this is 
>>>>>>> not a
>>>>>>> kdump kernel.
>>>>>>>
>>>>>>> Do you mind checking whether the disable interrupt remapping 
>>>>>>> callback
>>>>>>> was called during kexec reboot?
>>>>>>>
>>>>>>> 1121 struct irq_remap_ops intel_irq_remap_ops = {
>>>>>>> 1122         .prepare                = intel_prepare_irq_remapping,
>>>>>>> 1123         .enable                 = intel_enable_irq_remapping,
>>>>>>> 1124         .disable                = disable_irq_remapping,
>>>>>>> 1125         .reenable               = reenable_irq_remapping,
>>>>>>> 1126         .enable_faulting        = enable_drhd_fault_handling,
>>>>>>> 1127 };
>>>>>>
>>>>>> Is there a way to check this without rebuilding the Linux kernel?
>>>>>
>>>>> I am not sure, but you can check whether any messages are dumped in 
>>>>> the
>>>>> path of .disable callback? or try to use ftrace?
>>>>
>>>> With
>>>>
>>>> ```
>>>> diff --git a/drivers/iommu/intel/irq_remapping.c 
>>>> b/drivers/iommu/intel/irq_remapping.c
>>>> index 712ebfc9870c6..146f19ae5b5f1 100644
>>>> --- a/drivers/iommu/intel/irq_remapping.c
>>>> +++ b/drivers/iommu/intel/irq_remapping.c
>>>> @@ -1030,6 +1030,7 @@ static void disable_irq_remapping(void)
>>>>       struct dmar_drhd_unit *drhd;
>>>>       struct intel_iommu *iommu = NULL;
>>>>
>>>> +     pr_warn("XXX: Called %s\n", __func__);
>>>>       /*
>>>>        * Disable Interrupt-remapping for all the DRHD's now.
>>>>        */
>>>> ```
>>>>
>>>> I can’t see anything in the logs, so it does not seem to be called.
>>>>
>>>> Can you reproduce the issue?
>>>
>>> How did you reproduce this?
>>
>> On a “server” (with Intel Xeon?), in my case Dell PowerEdge T640 and 
>> Dell PowerEdge R930 (Intel E7-8891 v3), run
>>
>>      kexec /boot/bzImage --initrd=/boot/grub/initramfs.igz 
>> --reuse-cmdline
> 
> Were you able to fit some cycles into reproducing/analyzing this issue?

Yeah! I can reproduce this issue with my local development machine. But
I haven't had time to analyze it yet. Perhaps we can remove this
message, or make sure the .disable callback should be called?

Best regards,
baolu