Re: [PATCH V2] iommu/vt-d: Fix a IOMMU perfmon warning when CPU hotplug

[Baolu Lu <baolu.lu@linux.intel.com>]

On 2023/3/29 21:12, Liang, Kan wrote:
> 
> On 2023-03-28 2:20 p.m.,kan.liang@linux.intel.com  wrote:
>> From: Kan Liang<kan.liang@linux.intel.com>
>>
>> A warning can be triggered when hotplug CPU 0.
>> $ echo 0 > /sys/devices/system/cpu/cpu0/online
>>
>> [11958.737635] ------------[ cut here ]------------
>> [11958.742882] Voluntary context switch within RCU read-side critical
>> section!
>> [11958.742891] WARNING: CPU: 0 PID: 19 at kernel/rcu/tree_plugin.h:318
>> rcu_note_context_switch+0x4f4/0x580
>> [11958.860095] RIP: 0010:rcu_note_context_switch+0x4f4/0x580
>> [11958.960360] Call Trace:
>> [11958.963161]  <TASK>
>> [11958.965565]  ? perf_event_update_userpage+0x104/0x150
>> [11958.971293]  __schedule+0x8d/0x960
>> [11958.975165]  ? perf_event_set_state.part.82+0x11/0x50
>> [11958.980891]  schedule+0x44/0xb0
>> [11958.984464]  schedule_timeout+0x226/0x310
>> [11958.989017]  ? __perf_event_disable+0x64/0x1a0
>> [11958.994054]  ? _raw_spin_unlock+0x14/0x30
>> [11958.998605]  wait_for_completion+0x94/0x130
>> [11959.003352]  __wait_rcu_gp+0x108/0x130
>> [11959.007616]  synchronize_rcu+0x67/0x70
>> [11959.011876]  ? invoke_rcu_core+0xb0/0xb0
>> [11959.016333]  ? __bpf_trace_rcu_stall_warning+0x10/0x10
>> [11959.022147]  perf_pmu_migrate_context+0x121/0x370
>> [11959.027478]  iommu_pmu_cpu_offline+0x6a/0xa0
>> [11959.032325]  ? iommu_pmu_del+0x1e0/0x1e0
>> [11959.036782]  cpuhp_invoke_callback+0x129/0x510
>> [11959.041825]  cpuhp_thread_fun+0x94/0x150
>> [11959.046283]  smpboot_thread_fn+0x183/0x220
>> [11959.050933]  ? sort_range+0x20/0x20
>> [11959.054902]  kthread+0xe6/0x110
>> [11959.058479]  ? kthread_complete_and_exit+0x20/0x20
>> [11959.063911]  ret_from_fork+0x1f/0x30
>> [11959.067982]  </TASK>
>> [11959.070489] ---[ end trace 0000000000000000 ]---
>>
>> The synchronize_rcu() will be invoked in the perf_pmu_migrate_context(),
>> when migrating a PMU to a new CPU. However, the current for_each_iommu()
>> is within RCU read-side critical section.
>>
>> Two methods were considered to fix the issue.
>> - Use the dmar_global_lock to replace the RCU read lock when going
>>    through the drhd list. But it triggers a lockdep warning.
>> - Use the cpuhp_setup_state_multi() to set up a dedicated state for each
>>    IOMMU PMU. The lock can be avoided.
>>
>> The latter method is implemented in this patch. Since each IOMMU PMU has
>> a dedicated state, add cpuhp_node and cpu in struct iommu_pmu to track
>> the state. The state can be dynamically allocated now. Remove the
>> CPUHP_AP_PERF_X86_IOMMU_PERF_ONLINE.
>>
>> Fixes: 46284c6ceb5e ("iommu/vt-d: Support cpumask for IOMMU perfmon")
> Forgot to add a tag. If the patch looks good, please add the below tag
> when you pull the patch.
> 
> Reported-by: Ammy Yi<ammy.yi@intel.com>

Sure. Patch has been queued after my test.

https://lore.kernel.org/linux-iommu/20230329134721.469447-1-baolu.lu@linux.intel.com/

Best regards,
baolu