Re: [PATCH 1/3] wifi: ath11k: Set IRQ affinity hint after requesting all shared IRQs

[Baochen Qiang <quic_bqiang@quicinc.com>]

On 8/26/2024 3:01 PM, Manivannan Sadhasivam wrote:
> On Mon, Aug 26, 2024 at 11:04:41AM +0800, Baochen Qiang wrote:
>>
>>
>> On 8/23/2024 11:55 PM, Manivannan Sadhasivam wrote:
> 
> [...]
> 
>>> The warning is due to not clearing the affinity hint before freeing the
>>> IRQ.
>>>
>>> So to fix this, let's set the IRQ affinity hint after requesting all the
>>> shared IRQ. This will make sure that the affinity hint gets cleared in the
>>> error path before freeing the IRQ.
>> if you check 39564b475ac5 ("wifi: ath11k: fix boot failure with one MSI vector") you would see that the hint is set before requesting any IRQ for a purpose.
>>
> 
> Ok, thanks for sharing the history. However, commit 39564b475ac5 looks confusing
> to me. It asserts that changing the IRQ affinity changes the MSI vector
> programmed to the device, but I've never heard of that behavior. IRQ affinity
> change is supposed to only change the CPU mask for the IRQ.
vector has to be changed, or how does kernel change the target CPU of a certain IRQ? On x86 platform, this is done by apic_set_affinity().

> 
> For confirming my suspicion, I added the debug print in pci_write_msg_msi() and
> I can see that it is only getting called once during pci_alloc_irq_vectors().
> 
> Moreover with my series, WLAN is working fine on QCA6390 with a shared vector:
> 
> 213:       6766          0          0          0          0          0          0          0   PCI-MSI 524288 Edge      bhi, mhi, mhi, ce0, ce1, ce2, ce3, ce5, ce7, ce8, DP_EXT_IRQ, DP_EXT_IRQ, DP_EXT_IRQ, DP_EXT_IRQ, DP_EXT_IRQ, DP_EXT_IRQ, DP_EX
> T_IRQ, DP_EXT_IRQ, DP_EXT_IRQ, DP_EXT_IRQ
> 
That is because kernel allocates a vector targeting CPU0 at the very fist time, which is exactly what we want by setting IRQ affinity. So there is no need to change vector any more, and therefore you saw only one print of pci_write_msg_msi(). above interrupt counter is a direct evidence to such guess: all interrupts received on CPU0.

Actually the issue mentioned in commit 39564b475ac5 happens randomly. But whenever it happens, you could see interrupts received on CPUs other than 0.

> So I think the issue fixed by 39564b475ac5 should be reinvestigated.
> 
> - Mani
> 
>>>
>>> Tested-on: QCA6390 hw2.0 PCI WLAN.HST.1.0.1-05266-QCAHSTSWPLZ_V2_TO_X86-1
>>>
>>> Cc: Baochen Qiang <quic_bqiang@quicinc.com>
>>> Fixes: e94b07493da3 ("ath11k: Set IRQ affinity to CPU0 in case of one MSI vector")
>>> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
>>> ---
>>>  drivers/net/wireless/ath/ath11k/pci.c | 24 ++++++++++++------------
>>>  1 file changed, 12 insertions(+), 12 deletions(-)
>>>
>>> diff --git a/drivers/net/wireless/ath/ath11k/pci.c b/drivers/net/wireless/ath/ath11k/pci.c
>>> index 8d63b84d1261..0c22e18e65c7 100644
>>> --- a/drivers/net/wireless/ath/ath11k/pci.c
>>> +++ b/drivers/net/wireless/ath/ath11k/pci.c
>>> @@ -886,16 +886,10 @@ static int ath11k_pci_probe(struct pci_dev *pdev,
>>>  	if (ret)
>>>  		goto err_pci_disable_msi;
>>>  
>>> -	ret = ath11k_pci_set_irq_affinity_hint(ab_pci, cpumask_of(0));
>>> -	if (ret) {
>>> -		ath11k_err(ab, "failed to set irq affinity %d\n", ret);
>>> -		goto err_pci_disable_msi;
>>> -	}
>>> -
>>>  	ret = ath11k_mhi_register(ab_pci);
>>>  	if (ret) {
>>>  		ath11k_err(ab, "failed to register mhi: %d\n", ret);
>>> -		goto err_irq_affinity_cleanup;
>>> +		goto err_pci_disable_msi;
>>>  	}
>>>  
>>>  	ret = ath11k_hal_srng_init(ab);
>>> @@ -916,6 +910,12 @@ static int ath11k_pci_probe(struct pci_dev *pdev,
>>>  		goto err_ce_free;
>>>  	}
>>>  
>>> +	ret = ath11k_pci_set_irq_affinity_hint(ab_pci, cpumask_of(0));
>>> +	if (ret) {
>>> +		ath11k_err(ab, "failed to set irq affinity %d\n", ret);
>>> +		goto err_free_irq;
>>> +	}
>>> +
>>>  	/* kernel may allocate a dummy vector before request_irq and
>>>  	 * then allocate a real vector when request_irq is called.
>>>  	 * So get msi_data here again to avoid spurious interrupt
>>> @@ -924,17 +924,20 @@ static int ath11k_pci_probe(struct pci_dev *pdev,
>>>  	ret = ath11k_pci_config_msi_data(ab_pci);
>>>  	if (ret) {
>>>  		ath11k_err(ab, "failed to config msi_data: %d\n", ret);
>>> -		goto err_free_irq;
>>> +		goto err_irq_affinity_cleanup;
>>>  	}
>>>  
>>>  	ret = ath11k_core_init(ab);
>>>  	if (ret) {
>>>  		ath11k_err(ab, "failed to init core: %d\n", ret);
>>> -		goto err_free_irq;
>>> +		goto err_irq_affinity_cleanup;
>>>  	}
>>>  	ath11k_qmi_fwreset_from_cold_boot(ab);
>>>  	return 0;
>>>  
>>> +err_irq_affinity_cleanup:
>>> +	ath11k_pci_set_irq_affinity_hint(ab_pci, NULL);
>>> +
>>>  err_free_irq:
>>>  	ath11k_pcic_free_irq(ab);
>>>  
>>> @@ -947,9 +950,6 @@ static int ath11k_pci_probe(struct pci_dev *pdev,
>>>  err_mhi_unregister:
>>>  	ath11k_mhi_unregister(ab_pci);
>>>  
>>> -err_irq_affinity_cleanup:
>>> -	ath11k_pci_set_irq_affinity_hint(ab_pci, NULL);
>>> -
>>>  err_pci_disable_msi:
>>>  	ath11k_pci_free_msi(ab_pci);
>>>  
>