Re: Frames acknowledged and silently discarded in firmware

[Javier Cardona <jcardona@fb.com>]

Hi Adrian,

We do not have access to firmware source, nor to any other QCA proprietary tools at this time.
The logs were collected by enabling ATH10K_DBG_DATA in debug_mask.

AFICT, those messages are generated inside ath10k_htt_rx_h_deliver(), and the only place where the frames could be dropped before that is in ath10k_htt_rx_h_filter(), for conditions that do not apply (unconfigured channel or CAC in progress).  But if there is any other way to rule that frames are being dropped before that point, I’ll definitely try it out.

On the same logs, we do receive some fcs-err=1 frames, but they are not the “missing” frames: mac address and sequence number do not match.  This is to say that invalid FCS frames that are later dropped by the driver still make it to the debug logs.
My current working assumption is the missing frames fail integrity checks on reception but they are still acknowledged and then dropped in firmware.  This is what the observations so far seem to indicate… is that at all possible?

Thanks for your help!

Javier

On 2/8/18, 3:23 PM, "adrian.chadd@gmail.com on behalf of Adrian Chadd" <adrian.chadd@gmail.com on behalf of adrian@freebsd.org> wrote:

    hi,
    
    do you have firmware source at the current gig? or access to the pktlog tools?
    
    Have you verified that the frames weren't at all indicated up via HTT?
    Like are they absolutely NOT coming up via HTT, or are they coming up
    via HTT but being thrown out before the driver is ready to pass them
    up.
    
    
    -a
    
    
    On 7 February 2018 at 14:15, Javier Cardona <jcardona@fb.com> wrote:
    > Hi,
    >
    > We have observed a problem where, under certain conditions, the ath10k firmware will acknowledge frames but not send them up to the driver.
    > Frames are sent by a mesh access point (MAP1) to a second mesh AP (MAP2) at MCS 9/NSS-3, which at that distance is probably marginal.  Since frames get acknlowledged by MAP2, MAP1 will not try a lower rate.  But the driver at MAP2 does not receive the frames.
    >
    > We have captures of this exchange for both the unsuccessful as well as the successful case, which happens when we move MAP2 closer to MAP1.   They can be found here:
    > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.dropbox.com_sh_0or86c8vxotygdc_AABI7RmQ2nztcOBF3UDXbPUma&d=DwIFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=CH8h6v1aoPH3YUC48S5HeA&m=TgRrCL-K0Y96MsSLmn_dbarFMV0xfgmY7Vx0lNlVx-4&s=51bnOIdtEMQSDHf7LPZ-CYjl-73l0--y6s8sOBDhDk0&e=
    >
    > In both scenarios, the frames from MAP1 to MAP2 are acknowledged, as observed in sniffer captures.
    >
    > In the successful scenario, the driver logs show the frames being received by the driver.  Sequence numbers in the debug logs match those in the sniffer captures.
    >
    > root@nbg-3ed9da:~# journalctl -kf | grep "peer 60:31:97:3e:82:e6" | grep ucast | grep 'len 374'
    > Feb 05 05:43:31 nbg-3ed9da kernel: ath10k_pci 0000:01:00.0: rx skb d734f6c0 len 374 peer 60:31:97:3e:82:e6 tid 0 (BE) ucast sn 1027 vht sgi rate_idx 9 vht_nss 3 freq 5825 band 1 flag 0x600800 fcs-err 0 mic-err 0 amsdu-more 0
    > Feb 05 05:43:42 nbg-3ed9da kernel: ath10k_pci 0000:01:00.0: rx skb d6aaa480 len 374 peer 60:31:97:3e:82:e6 tid 0 (BE) ucast sn 1031 vht sgi rate_idx 9 vht_nss 3 freq 5825 band 1 flag 0x600800 fcs-err 0 mic-err 0 amsdu-more 0
    > Feb 05 05:43:53 nbg-3ed9da kernel: ath10k_pci 0000:01:00.0: rx skb d53a8d80 len 374 peer 60:31:97:3e:82:e6 tid 0 (BE) ucast sn 1037 vht sgi rate_idx 9 vht_nss 3 freq 5825 band 1 flag 0x600800 fcs-err 0 mic-err 0 amsdu-more 0
    >
    > In the failure scenario, the driver logs show no frames, even if the capture shows that the frames are acknowledged.
    >
    > root@nbg-3ed9da:~# journalctl -kf | grep "peer 60:31:97:3e:82:e6" | grep ucast | grep 'len 374'
    > <nothing here>
    >
    > If we force MAP1 to use a single stream, the frames are received successfully.
    >
    >   # iw mesh0 set bitrates legacy-5 ht-mcs-5 vht-mcs-5 1:0-9
    >
    > It seems as if the firmware is acknowledging but silently discarding frames… is that possible?
    > Can anyone provide some pointers on how to troubleshoot this?
    >
    > We are using this firmware: https://github.com/kvalo/ath10k-firmware/blob/master/QCA9984/hw1.0/3.4/firmware-5.bin_10.4-3.4-00104 and kernel 4.9.31 with a few cherry-picked patches from the ath10k branch.
    > The hardware is QCA994.
    >
    > Best,
    >
    > Javier
    >
    >
    >
    >
    > _______________________________________________
    > ath10k mailing list
    > ath10k@lists.infradead.org
    > http://lists.infradead.org/mailman/listinfo/ath10k
    

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k