Re: ath11k failed to enqueue rx buf: -28

Re: ath11k failed to enqueue rx buf: -28

[James Prestwood]

+ CC ath11k

On 3/27/24 9:25 AM, James Prestwood wrote:
> Hi,
>
> This error was brought to my attention in the kernel logs and I'm 
> wondering if it is of any concern:
>
> kernel: ath11k_pci 0000:03:00.0: failed to enqueue rx buf: -28
>
> It seems to happen every few minutes or so. I don't notice any bad 
> behavior associated with it per-se, but maybe its an issue of some 
> buffer needing to be increased in size? Does this mean a frame is 
> being dropped due to no room to receive it?
>
> Hardware we are running is:
>
> [    4.610399] ath11k_pci 0000:03:00.0: wcn6855 hw2.1
> [    5.777030] ath11k_pci 0000:03:00.0: chip_id 0x12 chip_family 0xb 
> board_id 0xff soc_id 0x400c1211
> [    5.777039] ath11k_pci 0000:03:00.0: fw_version 0x1109996e 
> fw_build_timestamp 2023-12-19 11:11 fw_build_id 
> WLAN.HSP.1.1-03125-QCAHSPSWPL_V1_V2_SILICONZ_LITE-3.6510.36
>
> Thanks,
>
> James
>

Re: ath11k failed to enqueue rx buf: -28

[Jeff Johnson]

On 3/27/2024 9:25 AM, James Prestwood wrote:
> Hi,
> 
> This error was brought to my attention in the kernel logs and I'm 
> wondering if it is of any concern:
> 
> kernel: ath11k_pci 0000:03:00.0: failed to enqueue rx buf: -28
> 
> It seems to happen every few minutes or so. I don't notice any bad 
> behavior associated with it per-se, but maybe its an issue of some 
> buffer needing to be increased in size? Does this mean a frame is being 
> dropped due to no room to receive it?
> 
> Hardware we are running is:
> 
> [    4.610399] ath11k_pci 0000:03:00.0: wcn6855 hw2.1
> [    5.777030] ath11k_pci 0000:03:00.0: chip_id 0x12 chip_family 0xb 
> board_id 0xff soc_id 0x400c1211
> [    5.777039] ath11k_pci 0000:03:00.0: fw_version 0x1109996e 
> fw_build_timestamp 2023-12-19 11:11 fw_build_id 
> WLAN.HSP.1.1-03125-QCAHSPSWPL_V1_V2_SILICONZ_LITE-3.6510.36
> 
> Thanks,
> 
> James
> 
> 

+ ath11k list to get more specific eyes on this issue
+ bcc to internal list as well

Re: ath11k failed to enqueue rx buf: -28

[Baochen Qiang]

On 3/30/2024 2:39 AM, Jeff Johnson wrote:
> On 3/27/2024 9:25 AM, James Prestwood wrote:
>> Hi,
>>
>> This error was brought to my attention in the kernel logs and I'm
>> wondering if it is of any concern:
>>
>> kernel: ath11k_pci 0000:03:00.0: failed to enqueue rx buf: -28
>>
>> It seems to happen every few minutes or so. I don't notice any bad
>> behavior associated with it per-se, but maybe its an issue of some
>> buffer needing to be increased in size? Does this mean a frame is being
>> dropped due to no room to receive it?
>>
>> Hardware we are running is:
>>
>> [    4.610399] ath11k_pci 0000:03:00.0: wcn6855 hw2.1
>> [    5.777030] ath11k_pci 0000:03:00.0: chip_id 0x12 chip_family 0xb
>> board_id 0xff soc_id 0x400c1211
>> [    5.777039] ath11k_pci 0000:03:00.0: fw_version 0x1109996e
>> fw_build_timestamp 2023-12-19 11:11 fw_build_id
>> WLAN.HSP.1.1-03125-QCAHSPSWPL_V1_V2_SILICONZ_LITE-3.6510.36
>>
>> Thanks,
>>
>> James
>>
>>
> 
> + ath11k list to get more specific eyes on this issue
> + bcc to internal list as well
> 
I will look into this.
> 

Re: ath11k failed to enqueue rx buf: -28

[Baochen Qiang]

On 4/1/2024 11:30 AM, Baochen Qiang wrote:
> 
> 
> On 3/30/2024 2:39 AM, Jeff Johnson wrote:
>> On 3/27/2024 9:25 AM, James Prestwood wrote:
>>> Hi,
>>>
>>> This error was brought to my attention in the kernel logs and I'm
>>> wondering if it is of any concern:
>>>
>>> kernel: ath11k_pci 0000:03:00.0: failed to enqueue rx buf: -28
>>>
>>> It seems to happen every few minutes or so. I don't notice any bad
>>> behavior associated with it per-se, but maybe its an issue of some
>>> buffer needing to be increased in size? Does this mean a frame is being
>>> dropped due to no room to receive it?
>>>
>>> Hardware we are running is:
>>>
>>> [    4.610399] ath11k_pci 0000:03:00.0: wcn6855 hw2.1
>>> [    5.777030] ath11k_pci 0000:03:00.0: chip_id 0x12 chip_family 0xb
>>> board_id 0xff soc_id 0x400c1211
>>> [    5.777039] ath11k_pci 0000:03:00.0: fw_version 0x1109996e
>>> fw_build_timestamp 2023-12-19 11:11 fw_build_id
>>> WLAN.HSP.1.1-03125-QCAHSPSWPL_V1_V2_SILICONZ_LITE-3.6510.36
>>>
>>> Thanks,
>>>
>>> James
Hi James, I was not able to reproduce this issue so wondering if you 
could help reproduce and collect logs for debug. If OK please first 
merge the debug patch which is available in patchwork:

https://patchwork.kernel.org/project/linux-wireless/patch/20240411074812.86700-1-quic_bqiang@quicinc.com/

Please also enable full ath11k log:
	modprobe ath11k debug_mask=0xffffffff
	modprobe ath11k_pci

Once enabled you should see lots of ath11k logs.

>>>
>>>
>>
>> + ath11k list to get more specific eyes on this issue
>> + bcc to internal list as well
>>
> I will look into this.
>>

Re: ath11k failed to enqueue rx buf: -28

[James Prestwood]

On 4/11/24 1:00 AM, Baochen Qiang wrote:
>
>
> On 4/1/2024 11:30 AM, Baochen Qiang wrote:
>>
>>
>> On 3/30/2024 2:39 AM, Jeff Johnson wrote:
>>> On 3/27/2024 9:25 AM, James Prestwood wrote:
>>>> Hi,
>>>>
>>>> This error was brought to my attention in the kernel logs and I'm
>>>> wondering if it is of any concern:
>>>>
>>>> kernel: ath11k_pci 0000:03:00.0: failed to enqueue rx buf: -28
>>>>
>>>> It seems to happen every few minutes or so. I don't notice any bad
>>>> behavior associated with it per-se, but maybe its an issue of some
>>>> buffer needing to be increased in size? Does this mean a frame is 
>>>> being
>>>> dropped due to no room to receive it?
>>>>
>>>> Hardware we are running is:
>>>>
>>>> [    4.610399] ath11k_pci 0000:03:00.0: wcn6855 hw2.1
>>>> [    5.777030] ath11k_pci 0000:03:00.0: chip_id 0x12 chip_family 0xb
>>>> board_id 0xff soc_id 0x400c1211
>>>> [    5.777039] ath11k_pci 0000:03:00.0: fw_version 0x1109996e
>>>> fw_build_timestamp 2023-12-19 11:11 fw_build_id
>>>> WLAN.HSP.1.1-03125-QCAHSPSWPL_V1_V2_SILICONZ_LITE-3.6510.36
>>>>
>>>> Thanks,
>>>>
>>>> James
> Hi James, I was not able to reproduce this issue so wondering if you 
> could help reproduce and collect logs for debug. If OK please first 
> merge the debug patch which is available in patchwork:
>
> https://patchwork.kernel.org/project/linux-wireless/patch/20240411074812.86700-1-quic_bqiang@quicinc.com/ 
>
>
> Please also enable full ath11k log:
>     modprobe ath11k debug_mask=0xffffffff
>     modprobe ath11k_pci
>
> Once enabled you should see lots of ath11k logs.
Thank you for looking at this. I'll get these changes in to test but it 
may take me some time.
>
>>>>
>>>>
>>>
>>> + ath11k list to get more specific eyes on this issue
>>> + bcc to internal list as well
>>>
>> I will look into this.
>>>

Re: ath11k failed to enqueue rx buf: -28

[James Prestwood]

Hi Baochen,

On 4/16/24 5:50 AM, James Prestwood wrote:
>
> On 4/11/24 1:00 AM, Baochen Qiang wrote:
>>
>>
>> On 4/1/2024 11:30 AM, Baochen Qiang wrote:
>>>
>>>
>>> On 3/30/2024 2:39 AM, Jeff Johnson wrote:
>>>> On 3/27/2024 9:25 AM, James Prestwood wrote:
>>>>> Hi,
>>>>>
>>>>> This error was brought to my attention in the kernel logs and I'm
>>>>> wondering if it is of any concern:
>>>>>
>>>>> kernel: ath11k_pci 0000:03:00.0: failed to enqueue rx buf: -28
>>>>>
>>>>> It seems to happen every few minutes or so. I don't notice any bad
>>>>> behavior associated with it per-se, but maybe its an issue of some
>>>>> buffer needing to be increased in size? Does this mean a frame is 
>>>>> being
>>>>> dropped due to no room to receive it?
>>>>>
>>>>> Hardware we are running is:
>>>>>
>>>>> [    4.610399] ath11k_pci 0000:03:00.0: wcn6855 hw2.1
>>>>> [    5.777030] ath11k_pci 0000:03:00.0: chip_id 0x12 chip_family 0xb
>>>>> board_id 0xff soc_id 0x400c1211
>>>>> [    5.777039] ath11k_pci 0000:03:00.0: fw_version 0x1109996e
>>>>> fw_build_timestamp 2023-12-19 11:11 fw_build_id
>>>>> WLAN.HSP.1.1-03125-QCAHSPSWPL_V1_V2_SILICONZ_LITE-3.6510.36
>>>>>
>>>>> Thanks,
>>>>>
>>>>> James
>> Hi James, I was not able to reproduce this issue so wondering if you 
>> could help reproduce and collect logs for debug. If OK please first 
>> merge the debug patch which is available in patchwork:
>>
>> https://patchwork.kernel.org/project/linux-wireless/patch/20240411074812.86700-1-quic_bqiang@quicinc.com/ 
>>
>>
>> Please also enable full ath11k log:
>>     modprobe ath11k debug_mask=0xffffffff
>>     modprobe ath11k_pci
>>
>> Once enabled you should see lots of ath11k logs.
> Thank you for looking at this. I'll get these changes in to test but 
> it may take me some time.

I did get these debugging changes onto a client to test but had to 
modify them twice now due to a massive amount of logs associated with a 
few of the prints. The first iteration was using an info print in quite 
a hot path resulting in the logs filling up with:

kernel: ath11k_pci 0000:02:00.0: ath11k_ce_completed_recv_next: pipe 2 
rx_buf_needed 1
kernel: ath11k_pci 0000:02:00.0: ath11k_ce_rx_buf_enqueue_pipe: pipe 2 
rx_buf_needed 0

I then downgraded that to a dbg print, rebuilt, and now I'm seeing tons 
of these:

kernel: ath11k_pci 0000:03:00.0: dst get next entry failed
kernel: ath11k_pci 0000:03:00.0: dst get next entry failed
kernel: ath11k_pci 0000:03:00.0: dst get next entry failed
kernel: ath11k_warn: 367 callbacks suppressed

I have downgraded the "dst get next entry failed" to a debug print as 
well. Just to get a baseline and see if the "failed to enqueue rx buf" 
message is even happening anymore. I may need some more guidance here 
because I don't think I'll be able to enable any kind of debugging 
without the kernel logs filling up immediately. It may be difficult to 
even see the "failed to enqueue rx buf" failure with a debug mask of 
0xffffffff and the changes you suggested.

Is "dst get next entry failed" anything to worry about?

Thanks,

James

>>
>>>>>
>>>>>
>>>>
>>>> + ath11k list to get more specific eyes on this issue
>>>> + bcc to internal list as well
>>>>
>>> I will look into this.
>>>>