Re: ath10k/QCA9980 - Issues introduced in wireless testing 2016-05

Re: ath10k/QCA9980 - Issues introduced in wireless testing 2016-05

[Kalle Valo]

Felix Fietkau <nbd@nbd.name> writes:

> On 2016-06-10 10:50, Michal Kazior wrote:
>> On 9 June 2016 at 09:46, A. Benz <ash.benz@bk.ru> wrote:
>>> Dear All,
>>>
>>> I am using LEDE on my IPQ806x (QCA9980) system (Archer C2600).
>>> With compat-wireless-2016-05-12, I observed traces attached below.
>>> The router is unstable and eventually reboots by itself (randomly).
>>>
>>> Upon reverting to compat-wireless-2016-01, the issue disappears. Nothing
>>> else is changed (software-wise or hardware).
>>> This was confirmed with other users.
>>>
>>> A new compile with the fixes below:
>>> https://git.lede-project.org/?p=lede/nbd/staging.git;a=commit;h=858e26f3c0fc11231f25497cbb2ddca1e5f101e0
>>>
>>> Did not solve the problem.
>>>
>>> Please let me know if I need to provide any further information.
>>>
>>> ------------[ cut here ]------------
>>> WARNING: CPU: 0 PID: 558 at
>>> compat-wireless-2016-05-12/net/mac80211/rx.c:4068
>>> ieee80211_rx_napi+0x8c/0x8a4 [mac80211]()
>> 
>> Can you post what is at rx.c line 4068 (and +/- 3 lines), please?
> It's early in ieee80211_rx_napi:
>
>     sband = local->hw.wiphy->bands[status->band];
>     if (WARN_ON(!sband))
>         goto drop;
>
> I could not easily find a scenario under which status->band would not be
> set properly by the driver, so my guess is there is some nasty memory
> corruption going on.
>
> FWIW, I've received several reports like this from different people on
> different devices. They're also confirming that reverting to the
> snapshot from January makes things stable again.

Adding ath10k list to the loop.

-- 
Kalle Valo

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: ath10k/QCA9980 - Issues introduced in wireless testing 2016-05

[Ben Greear]

On 06/10/2016 05:22 AM, Kalle Valo wrote:
> Felix Fietkau <nbd@nbd.name> writes:
>
>> On 2016-06-10 10:50, Michal Kazior wrote:
>>> On 9 June 2016 at 09:46, A. Benz <ash.benz@bk.ru> wrote:
>>>> Dear All,
>>>>
>>>> I am using LEDE on my IPQ806x (QCA9980) system (Archer C2600).
>>>> With compat-wireless-2016-05-12, I observed traces attached below.
>>>> The router is unstable and eventually reboots by itself (randomly).
>>>>
>>>> Upon reverting to compat-wireless-2016-01, the issue disappears. Nothing
>>>> else is changed (software-wise or hardware).
>>>> This was confirmed with other users.
>>>>
>>>> A new compile with the fixes below:
>>>> https://git.lede-project.org/?p=lede/nbd/staging.git;a=commit;h=858e26f3c0fc11231f25497cbb2ddca1e5f101e0
>>>>
>>>> Did not solve the problem.
>>>>
>>>> Please let me know if I need to provide any further information.
>>>>
>>>> ------------[ cut here ]------------
>>>> WARNING: CPU: 0 PID: 558 at
>>>> compat-wireless-2016-05-12/net/mac80211/rx.c:4068
>>>> ieee80211_rx_napi+0x8c/0x8a4 [mac80211]()
>>>
>>> Can you post what is at rx.c line 4068 (and +/- 3 lines), please?
>> It's early in ieee80211_rx_napi:
>>
>>      sband = local->hw.wiphy->bands[status->band];
>>      if (WARN_ON(!sband))
>>          goto drop;
>>
>> I could not easily find a scenario under which status->band would not be
>> set properly by the driver, so my guess is there is some nasty memory
>> corruption going on.
>>
>> FWIW, I've received several reports like this from different people on
>> different devices. They're also confirming that reverting to the
>> snapshot from January makes things stable again.
>
> Adding ath10k list to the loop.

This looks a lot like the problems I was having.

Two of these 5 patches recently made it upstream (but may not be in LEDE yet),
but the other patches also were related to memory corruption.

See my patches posted on 4/1/16:

https://patchwork.kernel.org/project/ath10k/list/

I don't know where the 5/5 patch ended up.

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: ath10k/QCA9980 - Issues introduced in wireless testing 2016-05

[Felix Fietkau]

On 2016-06-10 14:52, Ben Greear wrote:
> This looks a lot like the problems I was having.
> 
> Two of these 5 patches recently made it upstream (but may not be in LEDE yet),
> but the other patches also were related to memory corruption.
> 
> See my patches posted on 4/1/16:
> 
> https://patchwork.kernel.org/project/ath10k/list/
> 
> I don't know where the 5/5 patch ended up.
I had already asked affected users to test with those patches (I have a
commit that adds them in my staging tree), but it did not resolve the issue.

- Felix

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: ath10k/QCA9980 - Issues introduced in wireless testing 2016-05

[Ben Greear]

On 06/10/2016 05:57 AM, Felix Fietkau wrote:
> On 2016-06-10 14:52, Ben Greear wrote:
>> This looks a lot like the problems I was having.
>>
>> Two of these 5 patches recently made it upstream (but may not be in LEDE yet),
>> but the other patches also were related to memory corruption.
>>
>> See my patches posted on 4/1/16:
>>
>> https://patchwork.kernel.org/project/ath10k/list/
>>
>> I don't know where the 5/5 patch ended up.
> I had already asked affected users to test with those patches (I have a
> commit that adds them in my staging tree), but it did not resolve the issue.

Ok, must be something else then.

If you can run on x86 under KASAN it may provide some clues..that is how I eventually
made progress on the issues I was seeing.  My rebase onto 3.7 has been slow and painful,
but I should be ready to start testing that sometime soon, maybe I can reproduce something
there.

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: ath10k/QCA9980 - Issues introduced in wireless testing 2016-05

[Nagarajan, Ashok Raj]

> On 06/10/2016 05:57 AM, Felix Fietkau wrote:
>>> On 2016-06-10 14:52, Ben Greear wrote:
>>> This looks a lot like the problems I was having.
>>>
>>> Two of these 5 patches recently made it upstream (but may not be in LEDE yet),
>>> but the other patches also were related to memory corruption.
>>>
>>> See my patches posted on 4/1/16:
>>>
>>> https://patchwork.kernel.org/project/ath10k/list/
>>>
>>> I don't know where the 5/5 patch ended up.
>> I had already asked affected users to test with those patches (I have a
>> commit that adds them in my staging tree), but it did not resolve the issue.
>
>Ok, must be something else then.
>
>If you can run on x86 under KASAN it may provide some clues..that is how I eventually
>made progress on the issues I was seeing.  My rebase onto 3.7 has been slow and painful,
>but I should be ready to start testing that sometime soon, maybe I can reproduce something
>there.

Hi Benz,

Could you please check with the following diff if it is solving your issue?

diff --git a/drivers/net/wireless/ath/ath10k/htt_rx.c b/drivers/net/wireless/ath/ath10k/htt_rx.c
index 6f19fca..c192a41 100644
--- a/drivers/net/wireless/ath/ath10k/htt_rx.c
+++ b/drivers/net/wireless/ath/ath10k/htt_rx.c
@@ -1528,7 +1528,7 @@ static void ath10k_htt_rx_h_filter(struct ath10k *ar,
static int ath10k_htt_rx_handle_amsdu(struct ath10k_htt *htt)
{
        struct ath10k *ar = htt->ar;
-       static struct ieee80211_rx_status rx_status;
+       struct ieee80211_rx_status *rx_status = &htt->rx_status;
        struct sk_buff_head amsdu;
        int ret;
 
@@ -1553,11 +1553,11 @@ static int ath10k_htt_rx_handle_amsdu(struct ath10k_htt *htt)
        }
 
        ath10k_pktlog_rx(ar, &amsdu);
-       ath10k_htt_rx_h_ppdu(ar, &amsdu, &rx_status, 0xffff);
+       ath10k_htt_rx_h_ppdu(ar, &amsdu, rx_status, 0xffff);
        ath10k_htt_rx_h_unchain(ar, &amsdu, ret > 0);
-       ath10k_htt_rx_h_filter(ar, &amsdu, &rx_status);
-       ath10k_htt_rx_h_mpdu(ar, &amsdu, &rx_status);
-       ath10k_htt_rx_h_deliver(ar, &amsdu, &rx_status);
+       ath10k_htt_rx_h_filter(ar, &amsdu, rx_status);
+       ath10k_htt_rx_h_mpdu(ar, &amsdu, rx_status);
+       ath10k_htt_rx_h_deliver(ar, &amsdu, rx_status);

Thanks,
Ashok

>Thanks,
>Ben

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k