A-MSDU reception not working?

A-MSDU reception not working?

[Denton Gentry]

In iperf tests using a MacBook STA bridging through an ath10k AP to an
Ethernet server, I'm noticing very selective packet loss. The second
and subsequent frames in an A-MSDU packet appear to be dropped.

The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
sends A-MSDU packets containing two TCP frames. So far as I can tell,
the first TCP frame from an A-MSDU aggregate is delivered and the
second is consistently lost. The MacBook generally retransmits the
lost frame as a singleton with no aggregation, and the retransmitted
frame makes it through.

This became more noticeable after the reordering fixes in
http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html

I see this A-MSDU packet loss behavior both with and without the
reordering fixes, the first packet in an A-MSDU is delivered while the
second is dropped. However, *without* the reordering fixes (and
therefore with packets delivered out of order) the MacBook sends
relatively few A-MSDU frames. *With* the reordering fixes, so all
packets are delivered in order, the MacBook keeps sending A-MSDU and
therefore has to deal with more packet loss. I suspect it is an
interaction with the MacOS TCP congestion window which I'm likely
never going to fully understand, its stuck in a region of the
congestion window where the Wifi driver keeps choosing to using
A-MSDU.

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Ben Greear]

On 06/30/2014 01:15 PM, Denton Gentry wrote:
> In iperf tests using a MacBook STA bridging through an ath10k AP to an
> Ethernet server, I'm noticing very selective packet loss. The second
> and subsequent frames in an A-MSDU packet appear to be dropped.
>
> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
> sends A-MSDU packets containing two TCP frames. So far as I can tell,
> the first TCP frame from an A-MSDU aggregate is delivered and the
> second is consistently lost. The MacBook generally retransmits the
> lost frame as a singleton with no aggregation, and the retransmitted
> frame makes it through.
>
> This became more noticeable after the reordering fixes in
> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>
> I see this A-MSDU packet loss behavior both with and without the
> reordering fixes, the first packet in an A-MSDU is delivered while the
> second is dropped. However, *without* the reordering fixes (and
> therefore with packets delivered out of order) the MacBook sends
> relatively few A-MSDU frames. *With* the reordering fixes, so all
> packets are delivered in order, the MacBook keeps sending A-MSDU and
> therefore has to deal with more packet loss. I suspect it is an
> interaction with the MacOS TCP congestion window which I'm likely
> never going to fully understand, its stuck in a region of the
> congestion window where the Wifi driver keeps choosing to using
> A-MSDU.

We saw a case where ath10k AP would drop all UDP PDUs over about 3800 bytes,
I would guess it might be the same problem.

We have not looked into the problem yet.  Stations work fine
(which is our main concern at present)...

Thanks,
Ben


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com


_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Michal Kazior]

On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
> In iperf tests using a MacBook STA bridging through an ath10k AP to an
> Ethernet server, I'm noticing very selective packet loss. The second
> and subsequent frames in an A-MSDU packet appear to be dropped.
>
> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
> sends A-MSDU packets containing two TCP frames. So far as I can tell,
> the first TCP frame from an A-MSDU aggregate is delivered and the
> second is consistently lost. The MacBook generally retransmits the
> lost frame as a singleton with no aggregation, and the retransmitted
> frame makes it through.
>
> This became more noticeable after the reordering fixes in
> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>
> I see this A-MSDU packet loss behavior both with and without the
> reordering fixes, the first packet in an A-MSDU is delivered while the
> second is dropped. However, *without* the reordering fixes (and
> therefore with packets delivered out of order) the MacBook sends
> relatively few A-MSDU frames. *With* the reordering fixes, so all
> packets are delivered in order, the MacBook keeps sending A-MSDU and
> therefore has to deal with more packet loss. I suspect it is an
> interaction with the MacOS TCP congestion window which I'm likely
> never going to fully understand, its stuck in a region of the
> congestion window where the Wifi driver keeps choosing to using
> A-MSDU.

I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.

ath10k fw reports A-MSDU subframes separately. To avoid memory
copying/allocation overhead each subframe is reported as a singly
A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
A-MPDU reordering intereferes with A-MSDU now?


Michał

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Denton Gentry]

Yes, after some more testing it does look like an unfortunate
interaction between the reorder buffer and A-MSDU. The disaggregated
subframes all carry the same sequence number. The first subframe is
released from the reorder buffer and increments the head_seq_num.
Subsequent subframes are all discarded as being out of date.

[  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
head=0xb06
[  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
head=0xb0b
[  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
head=0xb10
[  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
head=0xb15
[  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
head=0xb1a
[  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
head=0xb1f
[  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
head=0xb24


On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>> Ethernet server, I'm noticing very selective packet loss. The second
>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>
>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>> the first TCP frame from an A-MSDU aggregate is delivered and the
>> second is consistently lost. The MacBook generally retransmits the
>> lost frame as a singleton with no aggregation, and the retransmitted
>> frame makes it through.
>>
>> This became more noticeable after the reordering fixes in
>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>
>> I see this A-MSDU packet loss behavior both with and without the
>> reordering fixes, the first packet in an A-MSDU is delivered while the
>> second is dropped. However, *without* the reordering fixes (and
>> therefore with packets delivered out of order) the MacBook sends
>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>> therefore has to deal with more packet loss. I suspect it is an
>> interaction with the MacOS TCP congestion window which I'm likely
>> never going to fully understand, its stuck in a region of the
>> congestion window where the Wifi driver keeps choosing to using
>> A-MSDU.
>
> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>
> ath10k fw reports A-MSDU subframes separately. To avoid memory
> copying/allocation overhead each subframe is reported as a singly
> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
> A-MPDU reordering intereferes with A-MSDU now?
>
>
> Michał

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Denton Gentry]

There are two issues in handling the dis-aggregated A-MSDU subframes
in ieee80211_sta_manage_reorder_buf:

1. The out-of-date check:

        /* frame with out of date sequence number */
        if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
                dev_kfree_skb(skb);
                goto out;
        }

As all of the subframes carry the same sequence number, the first
subframe will be delivered and increment head_seq_num and then all
subsequent subframes will be discarded.


2. The duplicates check a bit later in the routine:

        /* check if we already stored this frame */
        if (tid_agg_rx->reorder_buf[index]) {
                dev_kfree_skb(skb);
                goto out;
        }

If there is enough packet loss that packets are queued in the reorder
buffer and not immediately released, then only the first subframe will
be stored. Subsequent subframes will be discarded as duplicates.


An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
the reordering buffer changes, and dropped to about 8 Mbps with the
reorder buffer. Hacking around the out-of-date sequence number check
to allow all subframes to egress restores it to 170 Mbps.

In the area where I'm testing there isn't enough 5 GHz noise to make
the duplicates-check issue happen very often. By adding a printk I can
see that it does happen, but it doesn't impact the throughput and I
can't report the impact of fixing it.

----

I do wonder if both of these are symptoms of dis-aggregating the
A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
at a time, and the ath10k A-MSDU case is sending it subframes instead.
Trying to make the ath10k code send up all of the subframes as a chain
of skbs didn't immediately work, but I do wonder if that would better
match the mac80211 expectations.



On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
> Yes, after some more testing it does look like an unfortunate
> interaction between the reorder buffer and A-MSDU. The disaggregated
> subframes all carry the same sequence number. The first subframe is
> released from the reorder buffer and increments the head_seq_num.
> Subsequent subframes are all discarded as being out of date.
>
> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
> head=0xb06
> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
> head=0xb0b
> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
> head=0xb10
> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
> head=0xb15
> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
> head=0xb1a
> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
> head=0xb1f
> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
> head=0xb24
>
>
> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>> Ethernet server, I'm noticing very selective packet loss. The second
>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>
>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>> second is consistently lost. The MacBook generally retransmits the
>>> lost frame as a singleton with no aggregation, and the retransmitted
>>> frame makes it through.
>>>
>>> This became more noticeable after the reordering fixes in
>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>
>>> I see this A-MSDU packet loss behavior both with and without the
>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>> second is dropped. However, *without* the reordering fixes (and
>>> therefore with packets delivered out of order) the MacBook sends
>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>> therefore has to deal with more packet loss. I suspect it is an
>>> interaction with the MacOS TCP congestion window which I'm likely
>>> never going to fully understand, its stuck in a region of the
>>> congestion window where the Wifi driver keeps choosing to using
>>> A-MSDU.
>>
>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>
>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>> copying/allocation overhead each subframe is reported as a singly
>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>> A-MPDU reordering intereferes with A-MSDU now?
>>
>>
>> Michał

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Adrian Chadd]

I think you may have to tell mac80211 that it's okay and not to drop the frames.

The earlier atheros chips would just give you the AMSDU frames as
deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
net80211 and it'd ull it apart. But if the driver is doing it (or,
well, the chip is doing it) then mac80211 needs to know not to drop
those sub-frames.

I wonder if you'll ever get notified before the complete A-MPDU has
been received. That happens on previous chips. eg, you have an A-MPDU
of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
half of one MPDU before you hit EOL, the next notification you process
will be the next MSDU in the same MPDU - and then you'll hit the
reordering hilarity again.

So hm, i will face the same issue in FreeBSD at some point, so I'd
likely do what you're thinking of doing - pass up a chain of mbufs
representing the current MPDU and treat the whole lot as the frame(s)
to care about. Honestly though, I'm kind of wondering whether I should
mostly just do what the Atheros reference driver does and treat it as
ethernet payload frames (ie, it's already de-encapsulated and the
reordering is already done) and just tack on the wifi header bit via
another of the DMA rings.

Ugh, I really should sit down and write the FreeBSD version of this.



-a

(I'm still having flashbacks from working on this firmware at QCA. Aiee.)


On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
> There are two issues in handling the dis-aggregated A-MSDU subframes
> in ieee80211_sta_manage_reorder_buf:
>
> 1. The out-of-date check:
>
>         /* frame with out of date sequence number */
>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>                 dev_kfree_skb(skb);
>                 goto out;
>         }
>
> As all of the subframes carry the same sequence number, the first
> subframe will be delivered and increment head_seq_num and then all
> subsequent subframes will be discarded.
>
>
> 2. The duplicates check a bit later in the routine:
>
>         /* check if we already stored this frame */
>         if (tid_agg_rx->reorder_buf[index]) {
>                 dev_kfree_skb(skb);
>                 goto out;
>         }
>
> If there is enough packet loss that packets are queued in the reorder
> buffer and not immediately released, then only the first subframe will
> be stored. Subsequent subframes will be discarded as duplicates.
>
>
> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
> the reordering buffer changes, and dropped to about 8 Mbps with the
> reorder buffer. Hacking around the out-of-date sequence number check
> to allow all subframes to egress restores it to 170 Mbps.
>
> In the area where I'm testing there isn't enough 5 GHz noise to make
> the duplicates-check issue happen very often. By adding a printk I can
> see that it does happen, but it doesn't impact the throughput and I
> can't report the impact of fixing it.
>
> ----
>
> I do wonder if both of these are symptoms of dis-aggregating the
> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
> at a time, and the ath10k A-MSDU case is sending it subframes instead.
> Trying to make the ath10k code send up all of the subframes as a chain
> of skbs didn't immediately work, but I do wonder if that would better
> match the mac80211 expectations.
>
>
>
> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>> Yes, after some more testing it does look like an unfortunate
>> interaction between the reorder buffer and A-MSDU. The disaggregated
>> subframes all carry the same sequence number. The first subframe is
>> released from the reorder buffer and increments the head_seq_num.
>> Subsequent subframes are all discarded as being out of date.
>>
>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>> head=0xb06
>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>> head=0xb0b
>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>> head=0xb10
>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>> head=0xb15
>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>> head=0xb1a
>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>> head=0xb1f
>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>> head=0xb24
>>
>>
>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>
>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>> second is consistently lost. The MacBook generally retransmits the
>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>> frame makes it through.
>>>>
>>>> This became more noticeable after the reordering fixes in
>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>
>>>> I see this A-MSDU packet loss behavior both with and without the
>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>> second is dropped. However, *without* the reordering fixes (and
>>>> therefore with packets delivered out of order) the MacBook sends
>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>> therefore has to deal with more packet loss. I suspect it is an
>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>> never going to fully understand, its stuck in a region of the
>>>> congestion window where the Wifi driver keeps choosing to using
>>>> A-MSDU.
>>>
>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>
>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>> copying/allocation overhead each subframe is reported as a singly
>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>> A-MPDU reordering intereferes with A-MSDU now?
>>>
>>>
>>> Michał
>
> _______________________________________________
> ath10k mailing list
> ath10k@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/ath10k

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Janusz Dziedzic]

[-- Attachment #1: Type: text/plain, Size: 6965 bytes --]

On 6 July 2014 04:27, Adrian Chadd <adrian@freebsd.org> wrote:
> I think you may have to tell mac80211 that it's okay and not to drop the frames.
>
> The earlier atheros chips would just give you the AMSDU frames as
> deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
> net80211 and it'd ull it apart. But if the driver is doing it (or,
> well, the chip is doing it) then mac80211 needs to know not to drop
> those sub-frames.
>
> I wonder if you'll ever get notified before the complete A-MPDU has
> been received. That happens on previous chips. eg, you have an A-MPDU
> of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
> half of one MPDU before you hit EOL, the next notification you process
> will be the next MSDU in the same MPDU - and then you'll hit the
> reordering hilarity again.
>
> So hm, i will face the same issue in FreeBSD at some point, so I'd
> likely do what you're thinking of doing - pass up a chain of mbufs
> representing the current MPDU and treat the whole lot as the frame(s)
> to care about. Honestly though, I'm kind of wondering whether I should
> mostly just do what the Atheros reference driver does and treat it as
> ethernet payload frames (ie, it's already de-encapsulated and the
> reordering is already done) and just tack on the wifi header bit via
> another of the DMA rings.
>
> Ugh, I really should sit down and write the FreeBSD version of this.
>
>
>
> -a
>
> (I'm still having flashbacks from working on this firmware at QCA. Aiee.)
>
>
> On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
>> There are two issues in handling the dis-aggregated A-MSDU subframes
>> in ieee80211_sta_manage_reorder_buf:
>>
>> 1. The out-of-date check:
>>
>>         /* frame with out of date sequence number */
>>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>>                 dev_kfree_skb(skb);
>>                 goto out;
>>         }
>>
>> As all of the subframes carry the same sequence number, the first
>> subframe will be delivered and increment head_seq_num and then all
>> subsequent subframes will be discarded.
>>
>>
>> 2. The duplicates check a bit later in the routine:
>>
>>         /* check if we already stored this frame */
>>         if (tid_agg_rx->reorder_buf[index]) {
>>                 dev_kfree_skb(skb);
>>                 goto out;
>>         }
>>
>> If there is enough packet loss that packets are queued in the reorder
>> buffer and not immediately released, then only the first subframe will
>> be stored. Subsequent subframes will be discarded as duplicates.
>>
>>
>> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
>> the reordering buffer changes, and dropped to about 8 Mbps with the
>> reorder buffer. Hacking around the out-of-date sequence number check
>> to allow all subframes to egress restores it to 170 Mbps.
>>
>> In the area where I'm testing there isn't enough 5 GHz noise to make
>> the duplicates-check issue happen very often. By adding a printk I can
>> see that it does happen, but it doesn't impact the throughput and I
>> can't report the impact of fixing it.
>>
>> ----
>>
>> I do wonder if both of these are symptoms of dis-aggregating the
>> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
>> at a time, and the ath10k A-MSDU case is sending it subframes instead.
>> Trying to make the ath10k code send up all of the subframes as a chain
>> of skbs didn't immediately work, but I do wonder if that would better
>> match the mac80211 expectations.
>>
>>
>>
>> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>> Yes, after some more testing it does look like an unfortunate
>>> interaction between the reorder buffer and A-MSDU. The disaggregated
>>> subframes all carry the same sequence number. The first subframe is
>>> released from the reorder buffer and increments the head_seq_num.
>>> Subsequent subframes are all discarded as being out of date.
>>>
>>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>>> head=0xb06
>>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>>> head=0xb0b
>>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>>> head=0xb10
>>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>>> head=0xb15
>>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>>> head=0xb1a
>>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>>> head=0xb1f
>>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>>> head=0xb24
>>>
>>>
>>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>>
>>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>>> second is consistently lost. The MacBook generally retransmits the
>>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>>> frame makes it through.
>>>>>
>>>>> This became more noticeable after the reordering fixes in
>>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>>
>>>>> I see this A-MSDU packet loss behavior both with and without the
>>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>>> second is dropped. However, *without* the reordering fixes (and
>>>>> therefore with packets delivered out of order) the MacBook sends
>>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>>> therefore has to deal with more packet loss. I suspect it is an
>>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>>> never going to fully understand, its stuck in a region of the
>>>>> congestion window where the Wifi driver keeps choosing to using
>>>>> A-MSDU.
>>>>
>>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>>
>>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>>> copying/allocation overhead each subframe is reported as a singly
>>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>>> A-MPDU reordering intereferes with A-MSDU now?
>>>>
Denton could you try attached patch: report amsdu in one big frame.
If help, we can add amsdu skb list support to mac80211/cfg80211 - to
improve performance and reduce memcpy, while currently we have skb
frames, put them in one big skb and next cfg80211 split them again
into msdus and report to stack.

BR
Janusz

[-- Attachment #2: 0001-ath10k-amsdu-rx-buid-one-big-frame.patch --]
[-- Type: text/x-diff, Size: 4209 bytes --]

From 5a1761615c5bc819639710044e5dd72889fbbce6 Mon Sep 17 00:00:00 2001
From: Janusz Dziedzic <janusz.dziedzic@tieto.com>
Date: Sun, 6 Jul 2014 10:18:06 +0200
Subject: [RFC/RFT] ath10k: amsdu rx, buid one big frame

Temporary patch, to check if will solve ampdu reordering issue.

Signed-off-by: Janusz Dziedzic <janusz.dziedzic@tieto.com>
---
 drivers/net/wireless/ath/ath10k/htt_rx.c |   72 +++++++++++++++++++++---------
 1 file changed, 51 insertions(+), 21 deletions(-)

diff --git a/drivers/net/wireless/ath/ath10k/htt_rx.c b/drivers/net/wireless/ath/ath10k/htt_rx.c
index 57d5a97..2553f52 100644
--- a/drivers/net/wireless/ath/ath10k/htt_rx.c
+++ b/drivers/net/wireless/ath/ath10k/htt_rx.c
@@ -874,12 +874,15 @@ static void ath10k_htt_rx_amsdu(struct ath10k_htt *htt,
 {
 	struct htt_rx_desc *rxd;
 	struct sk_buff *skb = skb_in;
-	struct sk_buff *first;
+	struct sk_buff *first, *frame = NULL, *tmp;
 	enum rx_msdu_decap_format fmt;
 	enum htt_rx_mpdu_encrypt_type enctype;
 	struct ieee80211_hdr *hdr;
-	u8 hdr_buf[64], addr[ETH_ALEN], *qos;
+	u8 hdr_buf[64], addr[ETH_ALEN];
 	unsigned int hdr_len;
+	struct amsdu_subframe_hdr subframe_hdr;
+	unsigned int size = 0;
+	u8 padding;
 
 	rxd = (void *)skb->data - sizeof(*rxd);
 	enctype = MS(__le32_to_cpu(rxd->mpdu_start.info0),
@@ -890,6 +893,22 @@ static void ath10k_htt_rx_amsdu(struct ath10k_htt *htt,
 	memcpy(hdr_buf, hdr, hdr_len);
 	hdr = (struct ieee80211_hdr *)hdr_buf;
 
+	/* Check size we will need */
+	tmp = skb_in;
+	while (tmp) {
+		size = size + tmp->len;
+		tmp = tmp->next;
+	}
+
+	frame = alloc_skb(size + skb_headroom(skb_in), GFP_KERNEL);
+	if (!frame) {
+		dev_kfree_skb_any(skb_in);
+		return;
+	}
+
+	skb_reserve(frame, skb_headroom(skb_in));
+	frame->dev = skb_in->dev;
+
 	first = skb;
 	while (skb) {
 		void *decap_hdr;
@@ -922,19 +941,28 @@ static void ath10k_htt_rx_amsdu(struct ath10k_htt *htt,
 			memcpy(addr, ieee80211_get_DA(hdr), ETH_ALEN);
 			skb_pull(skb, hdr_len);
 
-			/* push original 802.11 header */
-			hdr = (struct ieee80211_hdr *)hdr_buf;
-			hdr_len = ieee80211_hdrlen(hdr->frame_control);
-			memcpy(skb_push(skb, hdr_len), hdr, hdr_len);
+			/* cfg80211 expect this padding */
+			padding = (4 - (skb->len + sizeof(subframe_hdr))) & 0x3;
+			skb_put(skb, padding);
 
-			/* original A-MSDU header has the bit set but we're
-			 * not including A-MSDU subframe header */
-			hdr = (struct ieee80211_hdr *)skb->data;
-			qos = ieee80211_get_qos_ctl(hdr);
-			qos[0] &= ~IEEE80211_QOS_CTL_A_MSDU_PRESENT;
+			/* build amsdu subframe header */
+			memcpy(&subframe_hdr.dst, addr, ETH_ALEN);
+			memcpy(&subframe_hdr.src, ieee80211_get_SA(hdr), ETH_ALEN);
+			subframe_hdr.len = __cpu_to_be16(skb->len) ;
 
-			/* original 802.11 header has a different DA */
-			memcpy(ieee80211_get_DA(hdr), addr, ETH_ALEN);
+			/* push back amsdu hdr */
+			memcpy(skb_push(skb, sizeof(subframe_hdr)), &subframe_hdr,
+			       sizeof(subframe_hdr));
+
+			if (skb == first) {
+				/* push back orginal 80211 header */
+				hdr = (struct ieee80211_hdr *)hdr_buf;
+				hdr_len = ieee80211_hdrlen(hdr->frame_control);
+				memcpy(skb_push(skb, hdr_len), hdr, hdr_len);
+
+				/* original 802.11 header has a different DA */
+				memcpy(ieee80211_get_DA(hdr), addr, ETH_ALEN);
+			}
 			break;
 		case RX_MSDU_DECAP_ETHERNET2_DIX:
 			/* strip ethernet header and insert decapped 802.11
@@ -956,19 +984,21 @@ static void ath10k_htt_rx_amsdu(struct ath10k_htt *htt,
 		}
 
 		skb_in = skb;
-		ath10k_htt_rx_h_protected(htt, rx_status, skb_in, enctype, fmt,
-					  false);
+
+		if (skb_in == first)
+			ath10k_htt_rx_h_protected(htt, rx_status, skb_in,
+						  enctype, fmt, false);
+
+		memcpy(skb_put(frame, skb_in->len), skb_in->data, skb_in->len);
+
 		skb = skb->next;
 		skb_in->next = NULL;
 
-		if (skb)
-			rx_status->flag |= RX_FLAG_AMSDU_MORE;
-		else
-			rx_status->flag &= ~RX_FLAG_AMSDU_MORE;
-
-		ath10k_process_rx(htt->ar, rx_status, skb_in);
+		/* We don't need this skb anymore */
+		dev_kfree_skb(skb_in);
 	}
 
+	ath10k_process_rx(htt->ar, rx_status, frame);
 	/* FIXME: It might be nice to re-assemble the A-MSDU when there's a
 	 * monitor interface active for sniffing purposes. */
 }
-- 
1.7.9.5


[-- Attachment #3: Type: text/plain, Size: 146 bytes --]

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Denton Gentry]

The initial results are not promising: a MacOS 802.11ac client gets
between 0-2 Mbps with this change, where it was getting about 8 Mbps
prior to this change and ~170 Mbps prior to the reordering fix. A pcap
from the receiving system shows a very large number of out of order
frames, likely due to TCP retransmission.

An 802.11n MacBook gets very good throughput, only the 802.11ac
MacBook shows the very poor result. I'm trying to figure out why.


One specific note (and probably not related to the throughput): I
believe ath10k_htt_rx_amsdu runs in the tasklet, which means it would
need to use GFP_ATOMIC instead of GFP_KERNEL. Do I understand it
correctly?


On Mon, Jul 7, 2014 at 1:30 AM, Janusz Dziedzic
<janusz.dziedzic@tieto.com> wrote:
> On 6 July 2014 04:27, Adrian Chadd <adrian@freebsd.org> wrote:
>> I think you may have to tell mac80211 that it's okay and not to drop the frames.
>>
>> The earlier atheros chips would just give you the AMSDU frames as
>> deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
>> net80211 and it'd ull it apart. But if the driver is doing it (or,
>> well, the chip is doing it) then mac80211 needs to know not to drop
>> those sub-frames.
>>
>> I wonder if you'll ever get notified before the complete A-MPDU has
>> been received. That happens on previous chips. eg, you have an A-MPDU
>> of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
>> half of one MPDU before you hit EOL, the next notification you process
>> will be the next MSDU in the same MPDU - and then you'll hit the
>> reordering hilarity again.
>>
>> So hm, i will face the same issue in FreeBSD at some point, so I'd
>> likely do what you're thinking of doing - pass up a chain of mbufs
>> representing the current MPDU and treat the whole lot as the frame(s)
>> to care about. Honestly though, I'm kind of wondering whether I should
>> mostly just do what the Atheros reference driver does and treat it as
>> ethernet payload frames (ie, it's already de-encapsulated and the
>> reordering is already done) and just tack on the wifi header bit via
>> another of the DMA rings.
>>
>> Ugh, I really should sit down and write the FreeBSD version of this.
>>
>>
>>
>> -a
>>
>> (I'm still having flashbacks from working on this firmware at QCA. Aiee.)
>>
>>
>> On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
>>> There are two issues in handling the dis-aggregated A-MSDU subframes
>>> in ieee80211_sta_manage_reorder_buf:
>>>
>>> 1. The out-of-date check:
>>>
>>>         /* frame with out of date sequence number */
>>>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>>>                 dev_kfree_skb(skb);
>>>                 goto out;
>>>         }
>>>
>>> As all of the subframes carry the same sequence number, the first
>>> subframe will be delivered and increment head_seq_num and then all
>>> subsequent subframes will be discarded.
>>>
>>>
>>> 2. The duplicates check a bit later in the routine:
>>>
>>>         /* check if we already stored this frame */
>>>         if (tid_agg_rx->reorder_buf[index]) {
>>>                 dev_kfree_skb(skb);
>>>                 goto out;
>>>         }
>>>
>>> If there is enough packet loss that packets are queued in the reorder
>>> buffer and not immediately released, then only the first subframe will
>>> be stored. Subsequent subframes will be discarded as duplicates.
>>>
>>>
>>> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
>>> the reordering buffer changes, and dropped to about 8 Mbps with the
>>> reorder buffer. Hacking around the out-of-date sequence number check
>>> to allow all subframes to egress restores it to 170 Mbps.
>>>
>>> In the area where I'm testing there isn't enough 5 GHz noise to make
>>> the duplicates-check issue happen very often. By adding a printk I can
>>> see that it does happen, but it doesn't impact the throughput and I
>>> can't report the impact of fixing it.
>>>
>>> ----
>>>
>>> I do wonder if both of these are symptoms of dis-aggregating the
>>> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
>>> at a time, and the ath10k A-MSDU case is sending it subframes instead.
>>> Trying to make the ath10k code send up all of the subframes as a chain
>>> of skbs didn't immediately work, but I do wonder if that would better
>>> match the mac80211 expectations.
>>>
>>>
>>>
>>> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>> Yes, after some more testing it does look like an unfortunate
>>>> interaction between the reorder buffer and A-MSDU. The disaggregated
>>>> subframes all carry the same sequence number. The first subframe is
>>>> released from the reorder buffer and increments the head_seq_num.
>>>> Subsequent subframes are all discarded as being out of date.
>>>>
>>>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>>>> head=0xb06
>>>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>>>> head=0xb0b
>>>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>>>> head=0xb10
>>>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>>>> head=0xb15
>>>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>>>> head=0xb1a
>>>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>>>> head=0xb1f
>>>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>>>> head=0xb24
>>>>
>>>>
>>>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>>>
>>>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>>>> second is consistently lost. The MacBook generally retransmits the
>>>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>>>> frame makes it through.
>>>>>>
>>>>>> This became more noticeable after the reordering fixes in
>>>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>>>
>>>>>> I see this A-MSDU packet loss behavior both with and without the
>>>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>>>> second is dropped. However, *without* the reordering fixes (and
>>>>>> therefore with packets delivered out of order) the MacBook sends
>>>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>>>> therefore has to deal with more packet loss. I suspect it is an
>>>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>>>> never going to fully understand, its stuck in a region of the
>>>>>> congestion window where the Wifi driver keeps choosing to using
>>>>>> A-MSDU.
>>>>>
>>>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>>>
>>>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>>>> copying/allocation overhead each subframe is reported as a singly
>>>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>>>> A-MPDU reordering intereferes with A-MSDU now?
>>>>>
> Denton could you try attached patch: report amsdu in one big frame.
> If help, we can add amsdu skb list support to mac80211/cfg80211 - to
> improve performance and reduce memcpy, while currently we have skb
> frames, put them in one big skb and next cfg80211 split them again
> into msdus and report to stack.
>
> BR
> Janusz

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Adrian Chadd]

.. why not disable AMSDU reception for now? Until these things are fixed?



-a


On 7 July 2014 12:26, Denton Gentry <denton.gentry@gmail.com> wrote:
> The initial results are not promising: a MacOS 802.11ac client gets
> between 0-2 Mbps with this change, where it was getting about 8 Mbps
> prior to this change and ~170 Mbps prior to the reordering fix. A pcap
> from the receiving system shows a very large number of out of order
> frames, likely due to TCP retransmission.
>
> An 802.11n MacBook gets very good throughput, only the 802.11ac
> MacBook shows the very poor result. I'm trying to figure out why.
>
>
> One specific note (and probably not related to the throughput): I
> believe ath10k_htt_rx_amsdu runs in the tasklet, which means it would
> need to use GFP_ATOMIC instead of GFP_KERNEL. Do I understand it
> correctly?
>
>
> On Mon, Jul 7, 2014 at 1:30 AM, Janusz Dziedzic
> <janusz.dziedzic@tieto.com> wrote:
>> On 6 July 2014 04:27, Adrian Chadd <adrian@freebsd.org> wrote:
>>> I think you may have to tell mac80211 that it's okay and not to drop the frames.
>>>
>>> The earlier atheros chips would just give you the AMSDU frames as
>>> deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
>>> net80211 and it'd ull it apart. But if the driver is doing it (or,
>>> well, the chip is doing it) then mac80211 needs to know not to drop
>>> those sub-frames.
>>>
>>> I wonder if you'll ever get notified before the complete A-MPDU has
>>> been received. That happens on previous chips. eg, you have an A-MPDU
>>> of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
>>> half of one MPDU before you hit EOL, the next notification you process
>>> will be the next MSDU in the same MPDU - and then you'll hit the
>>> reordering hilarity again.
>>>
>>> So hm, i will face the same issue in FreeBSD at some point, so I'd
>>> likely do what you're thinking of doing - pass up a chain of mbufs
>>> representing the current MPDU and treat the whole lot as the frame(s)
>>> to care about. Honestly though, I'm kind of wondering whether I should
>>> mostly just do what the Atheros reference driver does and treat it as
>>> ethernet payload frames (ie, it's already de-encapsulated and the
>>> reordering is already done) and just tack on the wifi header bit via
>>> another of the DMA rings.
>>>
>>> Ugh, I really should sit down and write the FreeBSD version of this.
>>>
>>>
>>>
>>> -a
>>>
>>> (I'm still having flashbacks from working on this firmware at QCA. Aiee.)
>>>
>>>
>>> On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>> There are two issues in handling the dis-aggregated A-MSDU subframes
>>>> in ieee80211_sta_manage_reorder_buf:
>>>>
>>>> 1. The out-of-date check:
>>>>
>>>>         /* frame with out of date sequence number */
>>>>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>>>>                 dev_kfree_skb(skb);
>>>>                 goto out;
>>>>         }
>>>>
>>>> As all of the subframes carry the same sequence number, the first
>>>> subframe will be delivered and increment head_seq_num and then all
>>>> subsequent subframes will be discarded.
>>>>
>>>>
>>>> 2. The duplicates check a bit later in the routine:
>>>>
>>>>         /* check if we already stored this frame */
>>>>         if (tid_agg_rx->reorder_buf[index]) {
>>>>                 dev_kfree_skb(skb);
>>>>                 goto out;
>>>>         }
>>>>
>>>> If there is enough packet loss that packets are queued in the reorder
>>>> buffer and not immediately released, then only the first subframe will
>>>> be stored. Subsequent subframes will be discarded as duplicates.
>>>>
>>>>
>>>> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
>>>> the reordering buffer changes, and dropped to about 8 Mbps with the
>>>> reorder buffer. Hacking around the out-of-date sequence number check
>>>> to allow all subframes to egress restores it to 170 Mbps.
>>>>
>>>> In the area where I'm testing there isn't enough 5 GHz noise to make
>>>> the duplicates-check issue happen very often. By adding a printk I can
>>>> see that it does happen, but it doesn't impact the throughput and I
>>>> can't report the impact of fixing it.
>>>>
>>>> ----
>>>>
>>>> I do wonder if both of these are symptoms of dis-aggregating the
>>>> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
>>>> at a time, and the ath10k A-MSDU case is sending it subframes instead.
>>>> Trying to make the ath10k code send up all of the subframes as a chain
>>>> of skbs didn't immediately work, but I do wonder if that would better
>>>> match the mac80211 expectations.
>>>>
>>>>
>>>>
>>>> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>> Yes, after some more testing it does look like an unfortunate
>>>>> interaction between the reorder buffer and A-MSDU. The disaggregated
>>>>> subframes all carry the same sequence number. The first subframe is
>>>>> released from the reorder buffer and increments the head_seq_num.
>>>>> Subsequent subframes are all discarded as being out of date.
>>>>>
>>>>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>>>>> head=0xb06
>>>>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>>>>> head=0xb0b
>>>>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>>>>> head=0xb10
>>>>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>>>>> head=0xb15
>>>>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>>>>> head=0xb1a
>>>>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>>>>> head=0xb1f
>>>>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>>>>> head=0xb24
>>>>>
>>>>>
>>>>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>>>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>>>>
>>>>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>>>>> second is consistently lost. The MacBook generally retransmits the
>>>>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>>>>> frame makes it through.
>>>>>>>
>>>>>>> This became more noticeable after the reordering fixes in
>>>>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>>>>
>>>>>>> I see this A-MSDU packet loss behavior both with and without the
>>>>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>>>>> second is dropped. However, *without* the reordering fixes (and
>>>>>>> therefore with packets delivered out of order) the MacBook sends
>>>>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>>>>> therefore has to deal with more packet loss. I suspect it is an
>>>>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>>>>> never going to fully understand, its stuck in a region of the
>>>>>>> congestion window where the Wifi driver keeps choosing to using
>>>>>>> A-MSDU.
>>>>>>
>>>>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>>>>
>>>>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>>>>> copying/allocation overhead each subframe is reported as a singly
>>>>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>>>>> A-MPDU reordering intereferes with A-MSDU now?
>>>>>>
>> Denton could you try attached patch: report amsdu in one big frame.
>> If help, we can add amsdu skb list support to mac80211/cfg80211 - to
>> improve performance and reduce memcpy, while currently we have skb
>> frames, put them in one big skb and next cfg80211 split them again
>> into msdus and report to stack.
>>
>> BR
>> Janusz
>
> _______________________________________________
> ath10k mailing list
> ath10k@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/ath10k

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Denton Gentry]

I don't know of a way to tell the STA to not use A-MSDU; is there one?
A-MSDU reception is mandatory in the specifications, it didn't look
like there was a way to communicate that it isn't allowed.

We can disable our own sending of A-MSDUs, but that isn't the issue here.

With 802.11ac the minimum MPDU size appears to be 3895 bytes, which is
big enough to hold two 1500 byte subframes. If there were a way to
force it the MPDU size smaller that could effectively prevent the STA
from using A-MSDU, but I don't see a way to do that either.



On Mon, Jul 7, 2014 at 12:41 PM, Adrian Chadd <adrian@freebsd.org> wrote:
> .. why not disable AMSDU reception for now? Until these things are fixed?
>
>
>
> -a
>
>
> On 7 July 2014 12:26, Denton Gentry <denton.gentry@gmail.com> wrote:
>> The initial results are not promising: a MacOS 802.11ac client gets
>> between 0-2 Mbps with this change, where it was getting about 8 Mbps
>> prior to this change and ~170 Mbps prior to the reordering fix. A pcap
>> from the receiving system shows a very large number of out of order
>> frames, likely due to TCP retransmission.
>>
>> An 802.11n MacBook gets very good throughput, only the 802.11ac
>> MacBook shows the very poor result. I'm trying to figure out why.
>>
>>
>> One specific note (and probably not related to the throughput): I
>> believe ath10k_htt_rx_amsdu runs in the tasklet, which means it would
>> need to use GFP_ATOMIC instead of GFP_KERNEL. Do I understand it
>> correctly?
>>
>>
>> On Mon, Jul 7, 2014 at 1:30 AM, Janusz Dziedzic
>> <janusz.dziedzic@tieto.com> wrote:
>>> On 6 July 2014 04:27, Adrian Chadd <adrian@freebsd.org> wrote:
>>>> I think you may have to tell mac80211 that it's okay and not to drop the frames.
>>>>
>>>> The earlier atheros chips would just give you the AMSDU frames as
>>>> deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
>>>> net80211 and it'd ull it apart. But if the driver is doing it (or,
>>>> well, the chip is doing it) then mac80211 needs to know not to drop
>>>> those sub-frames.
>>>>
>>>> I wonder if you'll ever get notified before the complete A-MPDU has
>>>> been received. That happens on previous chips. eg, you have an A-MPDU
>>>> of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
>>>> half of one MPDU before you hit EOL, the next notification you process
>>>> will be the next MSDU in the same MPDU - and then you'll hit the
>>>> reordering hilarity again.
>>>>
>>>> So hm, i will face the same issue in FreeBSD at some point, so I'd
>>>> likely do what you're thinking of doing - pass up a chain of mbufs
>>>> representing the current MPDU and treat the whole lot as the frame(s)
>>>> to care about. Honestly though, I'm kind of wondering whether I should
>>>> mostly just do what the Atheros reference driver does and treat it as
>>>> ethernet payload frames (ie, it's already de-encapsulated and the
>>>> reordering is already done) and just tack on the wifi header bit via
>>>> another of the DMA rings.
>>>>
>>>> Ugh, I really should sit down and write the FreeBSD version of this.
>>>>
>>>>
>>>>
>>>> -a
>>>>
>>>> (I'm still having flashbacks from working on this firmware at QCA. Aiee.)
>>>>
>>>>
>>>> On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>> There are two issues in handling the dis-aggregated A-MSDU subframes
>>>>> in ieee80211_sta_manage_reorder_buf:
>>>>>
>>>>> 1. The out-of-date check:
>>>>>
>>>>>         /* frame with out of date sequence number */
>>>>>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>>>>>                 dev_kfree_skb(skb);
>>>>>                 goto out;
>>>>>         }
>>>>>
>>>>> As all of the subframes carry the same sequence number, the first
>>>>> subframe will be delivered and increment head_seq_num and then all
>>>>> subsequent subframes will be discarded.
>>>>>
>>>>>
>>>>> 2. The duplicates check a bit later in the routine:
>>>>>
>>>>>         /* check if we already stored this frame */
>>>>>         if (tid_agg_rx->reorder_buf[index]) {
>>>>>                 dev_kfree_skb(skb);
>>>>>                 goto out;
>>>>>         }
>>>>>
>>>>> If there is enough packet loss that packets are queued in the reorder
>>>>> buffer and not immediately released, then only the first subframe will
>>>>> be stored. Subsequent subframes will be discarded as duplicates.
>>>>>
>>>>>
>>>>> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
>>>>> the reordering buffer changes, and dropped to about 8 Mbps with the
>>>>> reorder buffer. Hacking around the out-of-date sequence number check
>>>>> to allow all subframes to egress restores it to 170 Mbps.
>>>>>
>>>>> In the area where I'm testing there isn't enough 5 GHz noise to make
>>>>> the duplicates-check issue happen very often. By adding a printk I can
>>>>> see that it does happen, but it doesn't impact the throughput and I
>>>>> can't report the impact of fixing it.
>>>>>
>>>>> ----
>>>>>
>>>>> I do wonder if both of these are symptoms of dis-aggregating the
>>>>> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
>>>>> at a time, and the ath10k A-MSDU case is sending it subframes instead.
>>>>> Trying to make the ath10k code send up all of the subframes as a chain
>>>>> of skbs didn't immediately work, but I do wonder if that would better
>>>>> match the mac80211 expectations.
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>> Yes, after some more testing it does look like an unfortunate
>>>>>> interaction between the reorder buffer and A-MSDU. The disaggregated
>>>>>> subframes all carry the same sequence number. The first subframe is
>>>>>> released from the reorder buffer and increments the head_seq_num.
>>>>>> Subsequent subframes are all discarded as being out of date.
>>>>>>
>>>>>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>>>>>> head=0xb06
>>>>>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>>>>>> head=0xb0b
>>>>>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>>>>>> head=0xb10
>>>>>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>>>>>> head=0xb15
>>>>>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>>>>>> head=0xb1a
>>>>>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>>>>>> head=0xb1f
>>>>>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>>>>>> head=0xb24
>>>>>>
>>>>>>
>>>>>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>>>>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>>>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>>>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>>>>>
>>>>>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>>>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>>>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>>>>>> second is consistently lost. The MacBook generally retransmits the
>>>>>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>>>>>> frame makes it through.
>>>>>>>>
>>>>>>>> This became more noticeable after the reordering fixes in
>>>>>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>>>>>
>>>>>>>> I see this A-MSDU packet loss behavior both with and without the
>>>>>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>>>>>> second is dropped. However, *without* the reordering fixes (and
>>>>>>>> therefore with packets delivered out of order) the MacBook sends
>>>>>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>>>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>>>>>> therefore has to deal with more packet loss. I suspect it is an
>>>>>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>>>>>> never going to fully understand, its stuck in a region of the
>>>>>>>> congestion window where the Wifi driver keeps choosing to using
>>>>>>>> A-MSDU.
>>>>>>>
>>>>>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>>>>>
>>>>>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>>>>>> copying/allocation overhead each subframe is reported as a singly
>>>>>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>>>>>> A-MPDU reordering intereferes with A-MSDU now?
>>>>>>>
>>> Denton could you try attached patch: report amsdu in one big frame.
>>> If help, we can add amsdu skb list support to mac80211/cfg80211 - to
>>> improve performance and reduce memcpy, while currently we have skb
>>> frames, put them in one big skb and next cfg80211 split them again
>>> into msdus and report to stack.
>>>
>>> BR
>>> Janusz
>>
>> _______________________________________________
>> ath10k mailing list
>> ath10k@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/ath10k

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Liu CF/TW]

I wonder is it the firmware or mac80211 at AP side that handles ADDBA
requests from MacBook in this case?

- If it is mac80211, then we can try disabling AMSDU-in-AMPDU. in
agg_rx.c by setting the 'policy' argument to 0:

ieee80211_send_addba_resp(sta->sdata, sta->sta.addr, tid,
dialog_token, status, 0, buf_size, timeout);

- if it is inside the firmware, then in ath10k WMI, it seems FW does
support the following BA related commands (but alas there are no
details how these commands could be used ie. no C structure defined to
know how to fill the command)

If ADDBA req is handled by FW when AP is the recipient,  I guess
WMI_10X_ADDBA_SET_RESP_CMDID could be used to disable AMSDU-in-AMPDU
during BA negotiation:

/* commands to directly control ba negotiation directly from host. */
WMI_10X_ADDBA_CLEAR_RESP_CMDID,
WMI_10X_ADDBA_SEND_CMDID,
WMI_10X_ADDBA_STATUS_CMDID,
WMI_10X_DELBA_SEND_CMDID,
WMI_10X_ADDBA_SET_RESP_CMDID,
WMI_10X_SEND_SINGLEAMSDU_CMDID,

In general, I think AMSDU-in-AMPDU shouldn't be hard coded as always
enabled in mac80211 but should be a policy. Iterating through AMSDU
subframes is not just slow on embedded CPUs, but the nature they all
share the same PN and same CRC makes it inferior to AMPDU, where MPDUs
aggregated using AMPDU could independently acknowledged, encrypted &
integrity-checked. (Although it is indeed a valid workaround to sender
larger PPDU  (4Kx64=256K) since BA bitmap size hasn't been increased
since 11n)

Does anyone know more details about how WMI_10X_ADDBA_SET_RESP_CMDID
could be used?

Thanks,
cfliu

2014-07-07 12:53 GMT-07:00 Denton Gentry <denton.gentry@gmail.com>:
> I don't know of a way to tell the STA to not use A-MSDU; is there one?
> A-MSDU reception is mandatory in the specifications, it didn't look
> like there was a way to communicate that it isn't allowed.
>
> We can disable our own sending of A-MSDUs, but that isn't the issue here.
>
> With 802.11ac the minimum MPDU size appears to be 3895 bytes, which is
> big enough to hold two 1500 byte subframes. If there were a way to
> force it the MPDU size smaller that could effectively prevent the STA
> from using A-MSDU, but I don't see a way to do that either.
>
>
>
> On Mon, Jul 7, 2014 at 12:41 PM, Adrian Chadd <adrian@freebsd.org> wrote:
>> .. why not disable AMSDU reception for now? Until these things are fixed?
>>
>>
>>
>> -a
>>
>>
>> On 7 July 2014 12:26, Denton Gentry <denton.gentry@gmail.com> wrote:
>>> The initial results are not promising: a MacOS 802.11ac client gets
>>> between 0-2 Mbps with this change, where it was getting about 8 Mbps
>>> prior to this change and ~170 Mbps prior to the reordering fix. A pcap
>>> from the receiving system shows a very large number of out of order
>>> frames, likely due to TCP retransmission.
>>>
>>> An 802.11n MacBook gets very good throughput, only the 802.11ac
>>> MacBook shows the very poor result. I'm trying to figure out why.
>>>
>>>
>>> One specific note (and probably not related to the throughput): I
>>> believe ath10k_htt_rx_amsdu runs in the tasklet, which means it would
>>> need to use GFP_ATOMIC instead of GFP_KERNEL. Do I understand it
>>> correctly?
>>>
>>>
>>> On Mon, Jul 7, 2014 at 1:30 AM, Janusz Dziedzic
>>> <janusz.dziedzic@tieto.com> wrote:
>>>> On 6 July 2014 04:27, Adrian Chadd <adrian@freebsd.org> wrote:
>>>>> I think you may have to tell mac80211 that it's okay and not to drop the frames.
>>>>>
>>>>> The earlier atheros chips would just give you the AMSDU frames as
>>>>> deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
>>>>> net80211 and it'd ull it apart. But if the driver is doing it (or,
>>>>> well, the chip is doing it) then mac80211 needs to know not to drop
>>>>> those sub-frames.
>>>>>
>>>>> I wonder if you'll ever get notified before the complete A-MPDU has
>>>>> been received. That happens on previous chips. eg, you have an A-MPDU
>>>>> of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
>>>>> half of one MPDU before you hit EOL, the next notification you process
>>>>> will be the next MSDU in the same MPDU - and then you'll hit the
>>>>> reordering hilarity again.
>>>>>
>>>>> So hm, i will face the same issue in FreeBSD at some point, so I'd
>>>>> likely do what you're thinking of doing - pass up a chain of mbufs
>>>>> representing the current MPDU and treat the whole lot as the frame(s)
>>>>> to care about. Honestly though, I'm kind of wondering whether I should
>>>>> mostly just do what the Atheros reference driver does and treat it as
>>>>> ethernet payload frames (ie, it's already de-encapsulated and the
>>>>> reordering is already done) and just tack on the wifi header bit via
>>>>> another of the DMA rings.
>>>>>
>>>>> Ugh, I really should sit down and write the FreeBSD version of this.
>>>>>
>>>>>
>>>>>
>>>>> -a
>>>>>
>>>>> (I'm still having flashbacks from working on this firmware at QCA. Aiee.)
>>>>>
>>>>>
>>>>> On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>> There are two issues in handling the dis-aggregated A-MSDU subframes
>>>>>> in ieee80211_sta_manage_reorder_buf:
>>>>>>
>>>>>> 1. The out-of-date check:
>>>>>>
>>>>>>         /* frame with out of date sequence number */
>>>>>>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>>>>>>                 dev_kfree_skb(skb);
>>>>>>                 goto out;
>>>>>>         }
>>>>>>
>>>>>> As all of the subframes carry the same sequence number, the first
>>>>>> subframe will be delivered and increment head_seq_num and then all
>>>>>> subsequent subframes will be discarded.
>>>>>>
>>>>>>
>>>>>> 2. The duplicates check a bit later in the routine:
>>>>>>
>>>>>>         /* check if we already stored this frame */
>>>>>>         if (tid_agg_rx->reorder_buf[index]) {
>>>>>>                 dev_kfree_skb(skb);
>>>>>>                 goto out;
>>>>>>         }
>>>>>>
>>>>>> If there is enough packet loss that packets are queued in the reorder
>>>>>> buffer and not immediately released, then only the first subframe will
>>>>>> be stored. Subsequent subframes will be discarded as duplicates.
>>>>>>
>>>>>>
>>>>>> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
>>>>>> the reordering buffer changes, and dropped to about 8 Mbps with the
>>>>>> reorder buffer. Hacking around the out-of-date sequence number check
>>>>>> to allow all subframes to egress restores it to 170 Mbps.
>>>>>>
>>>>>> In the area where I'm testing there isn't enough 5 GHz noise to make
>>>>>> the duplicates-check issue happen very often. By adding a printk I can
>>>>>> see that it does happen, but it doesn't impact the throughput and I
>>>>>> can't report the impact of fixing it.
>>>>>>
>>>>>> ----
>>>>>>
>>>>>> I do wonder if both of these are symptoms of dis-aggregating the
>>>>>> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
>>>>>> at a time, and the ath10k A-MSDU case is sending it subframes instead.
>>>>>> Trying to make the ath10k code send up all of the subframes as a chain
>>>>>> of skbs didn't immediately work, but I do wonder if that would better
>>>>>> match the mac80211 expectations.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>> Yes, after some more testing it does look like an unfortunate
>>>>>>> interaction between the reorder buffer and A-MSDU. The disaggregated
>>>>>>> subframes all carry the same sequence number. The first subframe is
>>>>>>> released from the reorder buffer and increments the head_seq_num.
>>>>>>> Subsequent subframes are all discarded as being out of date.
>>>>>>>
>>>>>>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>>>>>>> head=0xb06
>>>>>>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>>>>>>> head=0xb0b
>>>>>>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>>>>>>> head=0xb10
>>>>>>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>>>>>>> head=0xb15
>>>>>>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>>>>>>> head=0xb1a
>>>>>>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>>>>>>> head=0xb1f
>>>>>>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>>>>>>> head=0xb24
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>>>>>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>>>>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>>>>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>>>>>>
>>>>>>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>>>>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>>>>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>>>>>>> second is consistently lost. The MacBook generally retransmits the
>>>>>>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>>>>>>> frame makes it through.
>>>>>>>>>
>>>>>>>>> This became more noticeable after the reordering fixes in
>>>>>>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>>>>>>
>>>>>>>>> I see this A-MSDU packet loss behavior both with and without the
>>>>>>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>>>>>>> second is dropped. However, *without* the reordering fixes (and
>>>>>>>>> therefore with packets delivered out of order) the MacBook sends
>>>>>>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>>>>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>>>>>>> therefore has to deal with more packet loss. I suspect it is an
>>>>>>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>>>>>>> never going to fully understand, its stuck in a region of the
>>>>>>>>> congestion window where the Wifi driver keeps choosing to using
>>>>>>>>> A-MSDU.
>>>>>>>>
>>>>>>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>>>>>>
>>>>>>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>>>>>>> copying/allocation overhead each subframe is reported as a singly
>>>>>>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>>>>>>> A-MPDU reordering intereferes with A-MSDU now?
>>>>>>>>
>>>> Denton could you try attached patch: report amsdu in one big frame.
>>>> If help, we can add amsdu skb list support to mac80211/cfg80211 - to
>>>> improve performance and reduce memcpy, while currently we have skb
>>>> frames, put them in one big skb and next cfg80211 split them again
>>>> into msdus and report to stack.
>>>>
>>>> BR
>>>> Janusz
>>>
>>> _______________________________________________
>>> ath10k mailing list
>>> ath10k@lists.infradead.org
>>> http://lists.infradead.org/mailman/listinfo/ath10k
>
> _______________________________________________
> ath10k mailing list
> ath10k@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/ath10k

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Janusz Dziedzic]

On 7 July 2014 21:26, Denton Gentry <denton.gentry@gmail.com> wrote:
> The initial results are not promising: a MacOS 802.11ac client gets
> between 0-2 Mbps with this change, where it was getting about 8 Mbps
> prior to this change and ~170 Mbps prior to the reordering fix. A pcap
> from the receiving system shows a very large number of out of order
> frames, likely due to TCP retransmission.
>
Thanks for test. I used ath10k as a STA with forced AMSDU (almost all
the time 3 A-MSDU packets used when TCP STA --> AP).
But seems still some issues with your card. I will check how fast we
can get MacOS 80211ac.
Could you share airlogs (best using additional sniffer)?

BTW.
We can try to disable AMSDU reordering in mac80211 for drivers that
prefer to report AMSDU as a separate frames (RX_FLAG_AMSDU_MORE),
while this case is not handled correctly (few packets with the same
serial number) in mac80211. So, before we will add correct
implementation we can disable this I think. Could you try this +
Michal`s reordering patch (without big frame patch):

index 394e201..914e22c 100644
--- a/net/mac80211/rx.c
+++ b/net/mac80211/rx.c
@@ -924,6 +924,10 @@ static void ieee80211_rx_reorder_ampdu(struct
ieee80211_rx_data *rx,
        if (!(status->rx_flags & IEEE80211_RX_RA_MATCH))
                goto dont_reorder;

+       /* not impelemented correctly when driver report separete A-MSDUs */
+        if (status->flag & RX_FLAG_AMSDU_MORE)
+               goto dont_reorder;
+

This disabling patch I think is still not complete while last A-MSDU
frame don't have this MORE flag, but anyway we can try. I will check
how to disable this correctly.


> An 802.11n MacBook gets very good throughput, only the 802.11ac
> MacBook shows the very poor result. I'm trying to figure out why.
>
This is also with/without reordering/big_frame patches? Or 80211n
don't use amsdu/reordering?

>
> One specific note (and probably not related to the throughput): I
> believe ath10k_htt_rx_amsdu runs in the tasklet, which means it would
> need to use GFP_ATOMIC instead of GFP_KERNEL. Do I understand it
> correctly?
Yes you right this is tasklet and should be ATOMIC.

>
>
> On Mon, Jul 7, 2014 at 1:30 AM, Janusz Dziedzic
> <janusz.dziedzic@tieto.com> wrote:
>> On 6 July 2014 04:27, Adrian Chadd <adrian@freebsd.org> wrote:
>>> I think you may have to tell mac80211 that it's okay and not to drop the frames.
>>>
>>> The earlier atheros chips would just give you the AMSDU frames as
>>> deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
>>> net80211 and it'd ull it apart. But if the driver is doing it (or,
>>> well, the chip is doing it) then mac80211 needs to know not to drop
>>> those sub-frames.
>>>
>>> I wonder if you'll ever get notified before the complete A-MPDU has
>>> been received. That happens on previous chips. eg, you have an A-MPDU
>>> of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
>>> half of one MPDU before you hit EOL, the next notification you process
>>> will be the next MSDU in the same MPDU - and then you'll hit the
>>> reordering hilarity again.
>>>
>>> So hm, i will face the same issue in FreeBSD at some point, so I'd
>>> likely do what you're thinking of doing - pass up a chain of mbufs
>>> representing the current MPDU and treat the whole lot as the frame(s)
>>> to care about. Honestly though, I'm kind of wondering whether I should
>>> mostly just do what the Atheros reference driver does and treat it as
>>> ethernet payload frames (ie, it's already de-encapsulated and the
>>> reordering is already done) and just tack on the wifi header bit via
>>> another of the DMA rings.
>>>
>>> Ugh, I really should sit down and write the FreeBSD version of this.
>>>
>>>
>>>
>>> -a
>>>
>>> (I'm still having flashbacks from working on this firmware at QCA. Aiee.)
>>>
>>>
>>> On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>> There are two issues in handling the dis-aggregated A-MSDU subframes
>>>> in ieee80211_sta_manage_reorder_buf:
>>>>
>>>> 1. The out-of-date check:
>>>>
>>>>         /* frame with out of date sequence number */
>>>>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>>>>                 dev_kfree_skb(skb);
>>>>                 goto out;
>>>>         }
>>>>
>>>> As all of the subframes carry the same sequence number, the first
>>>> subframe will be delivered and increment head_seq_num and then all
>>>> subsequent subframes will be discarded.
>>>>
>>>>
>>>> 2. The duplicates check a bit later in the routine:
>>>>
>>>>         /* check if we already stored this frame */
>>>>         if (tid_agg_rx->reorder_buf[index]) {
>>>>                 dev_kfree_skb(skb);
>>>>                 goto out;
>>>>         }
>>>>
>>>> If there is enough packet loss that packets are queued in the reorder
>>>> buffer and not immediately released, then only the first subframe will
>>>> be stored. Subsequent subframes will be discarded as duplicates.
>>>>
>>>>
>>>> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
>>>> the reordering buffer changes, and dropped to about 8 Mbps with the
>>>> reorder buffer. Hacking around the out-of-date sequence number check
>>>> to allow all subframes to egress restores it to 170 Mbps.
>>>>
>>>> In the area where I'm testing there isn't enough 5 GHz noise to make
>>>> the duplicates-check issue happen very often. By adding a printk I can
>>>> see that it does happen, but it doesn't impact the throughput and I
>>>> can't report the impact of fixing it.
>>>>
>>>> ----
>>>>
>>>> I do wonder if both of these are symptoms of dis-aggregating the
>>>> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
>>>> at a time, and the ath10k A-MSDU case is sending it subframes instead.
>>>> Trying to make the ath10k code send up all of the subframes as a chain
>>>> of skbs didn't immediately work, but I do wonder if that would better
>>>> match the mac80211 expectations.
>>>>
>>>>
>>>>
>>>> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>> Yes, after some more testing it does look like an unfortunate
>>>>> interaction between the reorder buffer and A-MSDU. The disaggregated
>>>>> subframes all carry the same sequence number. The first subframe is
>>>>> released from the reorder buffer and increments the head_seq_num.
>>>>> Subsequent subframes are all discarded as being out of date.
>>>>>
>>>>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>>>>> head=0xb06
>>>>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>>>>> head=0xb0b
>>>>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>>>>> head=0xb10
>>>>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>>>>> head=0xb15
>>>>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>>>>> head=0xb1a
>>>>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>>>>> head=0xb1f
>>>>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>>>>> head=0xb24
>>>>>
>>>>>
>>>>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>>>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>>>>
>>>>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>>>>> second is consistently lost. The MacBook generally retransmits the
>>>>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>>>>> frame makes it through.
>>>>>>>
>>>>>>> This became more noticeable after the reordering fixes in
>>>>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>>>>
>>>>>>> I see this A-MSDU packet loss behavior both with and without the
>>>>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>>>>> second is dropped. However, *without* the reordering fixes (and
>>>>>>> therefore with packets delivered out of order) the MacBook sends
>>>>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>>>>> therefore has to deal with more packet loss. I suspect it is an
>>>>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>>>>> never going to fully understand, its stuck in a region of the
>>>>>>> congestion window where the Wifi driver keeps choosing to using
>>>>>>> A-MSDU.
>>>>>>
>>>>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>>>>
>>>>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>>>>> copying/allocation overhead each subframe is reported as a singly
>>>>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>>>>> A-MPDU reordering intereferes with A-MSDU now?
>>>>>>
>> Denton could you try attached patch: report amsdu in one big frame.
>> If help, we can add amsdu skb list support to mac80211/cfg80211 - to
>> improve performance and reduce memcpy, while currently we have skb
>> frames, put them in one big skb and next cfg80211 split them again
>> into msdus and report to stack.
>>
>> BR
>> Janusz

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Denton Gentry]

I think I know what is happening now, though I've no idea why. The
throughput is low because we have many TCP retransmissions. We have
retransmissions because the TCP checksum is wrong on a number of
frames, and I do find data corruption in the payload so the checksum
definitely should be wrong. All of the corrupted frames were
originally one of the subframes in an A-MSDU packet.

An example follows at the end of this message, as dissected by
Wireshark. iperf sends a very regular data pattern of "0123456789..."
over and over. Note how in subframe #2 offset 0x1e0 the bytes "32 33
34" have been replaced by "72 36 35"

01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567

I added printks at the bottom of ath10k_htt_rx_amsdu immediately
before the call to ath10k_process_rx. I found this same packet, and we
see the "72 36 35" corruption in the printk. So I think it happened in
ath10k_process_rx or before, not anything weird after passing it up to
mac80211.

[  101.863712] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
[  101.863727] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
[  101.863742] ath10k: 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32
[  101.863757] ath10k: 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37 38
[  101.863773] ath10k: 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34
[  101.863788] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
[  101.863803] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36


I've found a number of examples of similar corruption, always with
between one and four bytes replaced.

35363738 -> e52c6e07
3435 -> b43f
3839 -> c238
31 -> 7f
3435 -> 7436
30 -> 50
3233 -> bc37


The packet described above, dissected by Wireshark:

No.     Time        Source                Destination
Protocol Length Info
   2235 18.953349   192.168.144.79        192.168.144.13        TCP
  3112   52697 > 5001 [ACK] Seq=1391553 Ack=1 Win=131760 [TCP CHECKSUM
INCORRECT] Len=1448 TSval=1298580657 TSecr=4294947481

Frame 2235: 3112 bytes on wire (24896 bits), 3112 bytes captured (24896 bits)
    Encapsulation type: IEEE 802.11 plus radiotap radio header (23)
    Arrival Time: Jul  7, 2014 23:03:37.763365000 PDT
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1404799417.763365000 seconds
    [Time delta from previous captured frame: 0.003476000 seconds]
    [Time delta from previous displayed frame: 0.515641000 seconds]
    [Time since reference or first frame: 18.953349000 seconds]
    Frame Number: 2235
    Frame Length: 3112 bytes (24896 bits)
    Capture Length: 3112 bytes (24896 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    [Protocols in frame: radiotap:wlan:llc:ip:tcp:data:llc:ip:tcp:data]
    [Coloring Rule Name: TCP]
    [Coloring Rule String: tcp]
Radiotap Header v0, Length 38
    Header revision: 0
    Header pad: 0
    Header length: 38
    Present flags
        .... .... .... .... .... .... .... ...1 = TSFT: True
        .... .... .... .... .... .... .... ..1. = Flags: True
        .... .... .... .... .... .... .... .0.. = Rate: False
        .... .... .... .... .... .... .... 1... = Channel: True
        .... .... .... .... .... .... ...0 .... = FHSS: False
        .... .... .... .... .... .... ..1. .... = dBm Antenna Signal: True
        .... .... .... .... .... .... .0.. .... = dBm Antenna Noise: False
        .... .... .... .... .... .... 0... .... = Lock Quality: False
        .... .... .... .... .... ...0 .... .... = TX Attenuation: False
        .... .... .... .... .... ..0. .... .... = dB TX Attenuation: False
        .... .... .... .... .... .0.. .... .... = dBm TX Power: False
        .... .... .... .... .... 1... .... .... = Antenna: True
        .... .... .... .... ...0 .... .... .... = dB Antenna Signal: False
        .... .... .... .... ..0. .... .... .... = dB Antenna Noise: False
        .... .... .... .... .1.. .... .... .... = RX flags: True
        .... .... .... .0.. .... .... .... .... = Channel+: False
        .... .... .... 0... .... .... .... .... = HT information: False
        .... .... ...0 .... .... .... .... .... = A-MPDU Status: False
        .... .... ..1. .... .... .... .... .... = VHT information: True
        ...0 0000 00.. .... .... .... .... .... = Reserved: 0x00000000
        ..0. .... .... .... .... .... .... .... = Radiotap NS next: False
        .0.. .... .... .... .... .... .... .... = Vendor NS next: False
        0... .... .... .... .... .... .... .... = Ext: False
    MAC timestamp: 78051063
    Flags: 0x00
        .... ...0 = CFP: False
        .... ..0. = Preamble: Long
        .... .0.. = WEP: False
        .... 0... = Fragmentation: False
        ...0 .... = FCS at end: False
        ..0. .... = Data Pad: False
        .0.. .... = Bad FCS: False
        0... .... = Short GI: False
    Channel frequency: 5745 [A 149]
    Channel type: 802.11a (0x0140)
        .... .... ...0 .... = Turbo: False
        .... .... ..0. .... = Complementary Code Keying (CCK): False
        .... .... .1.. .... = Orthogonal Frequency-Division
Multiplexing (OFDM): True
        .... .... 0... .... = 2 GHz spectrum: False
        .... ...1 .... .... = 5 GHz spectrum: True
        .... ..0. .... .... = Passive: False
        .... .0.. .... .... = Dynamic CCK-OFDM: False
        .... 0... .... .... = Gaussian Frequency Shift Keying (GFSK): False
        ...0 .... .... .... = GSM (900MHz): False
        ..0. .... .... .... = Static Turbo: False
        .0.. .... .... .... = Half Rate Channel (10MHz Channel Width): False
        0... .... .... .... = Quarter Rate Channel (5MHz Channel Width): False
    SSI Signal: -53 dBm
    Antenna: 0
    RX flags: 0x0000
        .... .... .... .... .... ..0. = Bad PLCP: False
    VHT information
        Known VHT information: 0x44
            .... .... .... ...0 = STBC: False
            .... .... .... ..0. = TXOP_PS_NOT_ALLOWED: False
            .... .... .... .1.. = Guard interval: True
            .... .... .... 0... = SGI Nsym disambiguation: False
            .... .... ...0 .... = LDPC extra OFDM symbol: False
            .... .... ..0. .... = Beamformed: False
            .... .... .1.. .... = Bandwidth: True
            .... .... 0... .... = Group ID: False
            .... ...0 .... .... = Partial AID: False
        .... .0.. = Guard interval: long (0)
        Bandwidth: 80 MHz (4)
        User 0: MCS 8
            1000 .... = MCS index 0: 8 (256-QAM 3/4)
            .... 0010 = Spatial streams 0: 2
            Space-time streams 0: 2
            Coding 0: BCC (0)
            [Data Rate: 702.0 Mb/s]
IEEE 802.11 QoS Data, Flags: .......T
    Type/Subtype: QoS Data (0x28)
    Frame Control Field: 0x8801
        .... ..00 = Version: 0
        .... 10.. = Type: Data frame (2)
        1000 .... = Subtype: 8
        Flags: 0x01
            .... ..01 = DS status: Frame from STA to DS via an AP (To
DS: 1 From DS: 0) (0x01)
            .... .0.. = More Fragments: This is the last fragment
            .... 0... = Retry: Frame is not being retransmitted
            ...0 .... = PWR MGT: STA will stay up
            ..0. .... = More Data: No data buffered
            .0.. .... = Protected flag: Data is not protected
            0... .... = Order flag: Not strictly ordered
    .000 0000 0011 0000 = Duration: 48 microseconds
    Receiver address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
    BSS Id: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
    Transmitter address: Apple_67:24:54 (84:38:35:67:24:54)
    Source address: Apple_67:24:54 (84:38:35:67:24:54)
    Destination address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
    Fragment number: 0
    Sequence number: 1021
    Qos Control: 0x0080
        .... .... .... 0000 = TID: 0
        [.... .... .... .000 = Priority: Best Effort (Best Effort) (0)]
        .... .... ...0 .... = QoS bit 4: Bits 8-15 of QoS Control
field are TXOP Duration Requested
        .... .... .00. .... = Ack Policy: Normal Ack (0x0000)
        .... .... 1... .... = Payload Type: A-MSDU
        0000 0000 .... .... = TXOP Duration Requested: 0 (no TXOP requested)
IEEE 802.11 Aggregate MSDU
    A-MSDU Subframe #1
        Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
        Source address: Apple_67:24:54 (84:38:35:67:24:54)
        A-MSDU Length: 1510
        Logical-Link Control
            DSAP: SNAP (0xaa)
            IG Bit: Individual
            SSAP: SNAP (0xaa)
            CR Bit: Command
            Control field: U, func=UI (0x03)
                000. 00.. = Command: Unnumbered Information (0x00)
                .... ..11 = Frame type: Unnumbered frame (0x03)
            Organization Code: Encapsulated Ethernet (0x000000)
            Type: IP (0x0800)
        Internet Protocol Version 4, Src: 192.168.144.79
(192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
            Version: 4
            Header length: 20 bytes
            Differentiated Services Field: 0x00 (DSCP 0x00: Default;
ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
                0000 00.. = Differentiated Services Codepoint: Default (0x00)
                .... ..00 = Explicit Congestion Notification: Not-ECT
(Not ECN-Capable Transport) (0x00)
            Total Length: 1500
            Identification: 0xc622 (50722)
            Flags: 0x00
                0... .... = Reserved bit: Not set
                .0.. .... = Don't fragment: Not set
                ..0. .... = More fragments: Not set
            Fragment offset: 0
            Time to live: 64
            Protocol: TCP (6)
            Header checksum: 0x0d4c [correct]
                [Good: True]
                [Bad: False]
            Source: 192.168.144.79 (192.168.144.79)
            Destination: 192.168.144.13 (192.168.144.13)
            [Source GeoIP: Unknown]
            [Destination GeoIP: Unknown]
        Transmission Control Protocol, Src Port: 52697 (52697), Dst
Port: 5001 (5001), Seq: 1390105, Ack: 1, Len: 1448
            Source port: 52697 (52697)
            Destination port: 5001 (5001)
            [Stream index: 0]
            Sequence number: 1390105    (relative sequence number)
            [Next sequence number: 1391553    (relative sequence number)]
            Acknowledgment number: 1    (relative ack number)
            Header length: 32 bytes
            Flags: 0x010 (ACK)
                000. .... .... = Reserved: Not set
                ...0 .... .... = Nonce: Not set
                .... 0... .... = Congestion Window Reduced (CWR): Not set
                .... .0.. .... = ECN-Echo: Not set
                .... ..0. .... = Urgent: Not set
                .... ...1 .... = Acknowledgment: Set
                .... .... 0... = Push: Not set
                .... .... .0.. = Reset: Not set
                .... .... ..0. = Syn: Not set
                .... .... ...0 = Fin: Not set
            Window size value: 8235
            [Calculated window size: 131760]
            [Window size scaling factor: 16]
            Checksum: 0xa1c0 [correct]
                [Good Checksum: True]
                [Bad Checksum: False]
            Options: (12 bytes), No-Operation (NOP), No-Operation
(NOP), Timestamps
                No-Operation (NOP)
                    Type: 1
                        0... .... = Copy on fragmentation: No
                        .00. .... = Class: Control (0)
                        ...0 0001 = Number: No-Operation (NOP) (1)
                No-Operation (NOP)
                    Type: 1
                        0... .... = Copy on fragmentation: No
                        .00. .... = Class: Control (0)
                        ...0 0001 = Number: No-Operation (NOP) (1)
                Timestamps: TSval 1298580657, TSecr 4294947481
                    Kind: Timestamp (8)
                    Length: 10
                    Timestamp value: 1298580657
                    Timestamp echo reply: 4294947481
        Data (1448 bytes)

0000  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0010  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0020  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0030  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0040  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0050  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0060  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0070  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0080  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0090  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
00a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
00b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
00c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
00d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
00e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
00f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0100  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0110  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0120  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0130  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0140  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0150  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0160  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0170  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0180  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0190  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
01a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
01b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
01c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
01d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
01e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
01f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0200  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0210  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0220  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0230  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0240  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0250  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0260  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0270  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0280  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0290  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
02a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
02b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
02c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
02d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
02e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
02f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0300  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0310  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0320  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0330  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0340  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0350  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0360  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0370  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0380  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0390  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
03a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
03b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
03c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
03d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
03e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
03f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0400  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0410  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0420  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0430  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0440  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0450  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0460  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0470  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0480  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0490  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
04a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
04b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
04c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
04d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
04e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
04f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0500  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0510  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0520  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0530  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0540  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0550  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0560  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0570  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0580  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0590  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
05a0  34 35 36 37 38 39 30 31                           45678901
            Data: 343536373839303132333435363738393031323334353637...
            [Length: 1448]
    A-MSDU Subframe #2
        Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
        Source address: Apple_67:24:54 (84:38:35:67:24:54)
        A-MSDU Length: 1510
        Logical-Link Control
            DSAP: SNAP (0xaa)
            IG Bit: Individual
            SSAP: SNAP (0xaa)
            CR Bit: Command
            Control field: U, func=UI (0x03)
                000. 00.. = Command: Unnumbered Information (0x00)
                .... ..11 = Frame type: Unnumbered frame (0x03)
            Organization Code: Encapsulated Ethernet (0x000000)
            Type: IP (0x0800)
        Internet Protocol Version 4, Src: 192.168.144.79
(192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
            Version: 4
            Header length: 20 bytes
            Differentiated Services Field: 0x00 (DSCP 0x00: Default;
ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
                0000 00.. = Differentiated Services Codepoint: Default (0x00)
                .... ..00 = Explicit Congestion Notification: Not-ECT
(Not ECN-Capable Transport) (0x00)
            Total Length: 1500
            Identification: 0xda09 (55817)
            Flags: 0x00
                0... .... = Reserved bit: Not set
                .0.. .... = Don't fragment: Not set
                ..0. .... = More fragments: Not set
            Fragment offset: 0
            Time to live: 64
            Protocol: TCP (6)
            Header checksum: 0xf964 [correct]
                [Good: True]
                [Bad: False]
            Source: 192.168.144.79 (192.168.144.79)
            Destination: 192.168.144.13 (192.168.144.13)
            [Source GeoIP: Unknown]
            [Destination GeoIP: Unknown]
        Transmission Control Protocol, Src Port: 52697 (52697), Dst
Port: 5001 (5001), Seq: 1391553, Ack: 1, Len: 1448
            Source port: 52697 (52697)
            Destination port: 5001 (5001)
            [Stream index: 0]
            Sequence number: 1391553    (relative sequence number)
            [Next sequence number: 1393001    (relative sequence number)]
            Acknowledgment number: 1    (relative ack number)
            Header length: 32 bytes
            Flags: 0x010 (ACK)
                000. .... .... = Reserved: Not set
                ...0 .... .... = Nonce: Not set
                .... 0... .... = Congestion Window Reduced (CWR): Not set
                .... .0.. .... = ECN-Echo: Not set
                .... ..0. .... = Urgent: Not set
                .... ...1 .... = Acknowledgment: Set
                .... .... 0... = Push: Not set
                .... .... .0.. = Reset: Not set
                .... .... ..0. = Syn: Not set
                .... .... ...0 = Fin: Not set
            Window size value: 8235
            [Calculated window size: 131760]
            [Window size scaling factor: 16]
            Checksum: 0x9a16 [incorrect, should be 0x5913 (maybe
caused by "TCP checksum offload"?)]
                [Good Checksum: False]
                [Bad Checksum: True]
                    [Expert Info (Error/Checksum): Bad checksum]
                        [Message: Bad checksum]
                        [Severity level: Error]
                        [Group: Checksum]
            Options: (12 bytes), No-Operation (NOP), No-Operation
(NOP), Timestamps
                No-Operation (NOP)
                    Type: 1
                        0... .... = Copy on fragmentation: No
                        .00. .... = Class: Control (0)
                        ...0 0001 = Number: No-Operation (NOP) (1)
                No-Operation (NOP)
                    Type: 1
                        0... .... = Copy on fragmentation: No
                        .00. .... = Class: Control (0)
                        ...0 0001 = Number: No-Operation (NOP) (1)
                Timestamps: TSval 1298580657, TSecr 4294947481
                    Kind: Timestamp (8)
                    Length: 10
                    Timestamp value: 1298580657
                    Timestamp echo reply: 4294947481
        Data (1448 bytes)

0000  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0010  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0020  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0030  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0040  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0050  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0060  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0070  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0080  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0090  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
00a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
00b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
00c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
00d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
00e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
00f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0100  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0110  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0120  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0130  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0140  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0150  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0160  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0170  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0180  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0190  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
01a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
01b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
01c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
01d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
01f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0200  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0210  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0220  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0230  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0240  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0250  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0260  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0270  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0280  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0290  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
02a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
02b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
02c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
02d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
02e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
02f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0300  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0310  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0320  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0330  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0340  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0350  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0360  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0370  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0380  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0390  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
03a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
03b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
03c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
03d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
03e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
03f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0400  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0410  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0420  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0430  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0440  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0450  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0460  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0470  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0480  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0490  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
04a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
04b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
04c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
04d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
04e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
04f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0500  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0510  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0520  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0530  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0540  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0550  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0560  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0570  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0580  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0590  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
05a0  32 33 34 35 36 37 38 39                           23456789
            Data: 323334353637383930313233343536373839303132333435...
            [Length: 1448]

On Mon, Jul 7, 2014 at 12:26 PM, Denton Gentry <denton.gentry@gmail.com> wrote:
> The initial results are not promising: a MacOS 802.11ac client gets
> between 0-2 Mbps with this change, where it was getting about 8 Mbps
> prior to this change and ~170 Mbps prior to the reordering fix. A pcap
> from the receiving system shows a very large number of out of order
> frames, likely due to TCP retransmission.
>
> An 802.11n MacBook gets very good throughput, only the 802.11ac
> MacBook shows the very poor result. I'm trying to figure out why.
>
>
> One specific note (and probably not related to the throughput): I
> believe ath10k_htt_rx_amsdu runs in the tasklet, which means it would
> need to use GFP_ATOMIC instead of GFP_KERNEL. Do I understand it
> correctly?
>
>
> On Mon, Jul 7, 2014 at 1:30 AM, Janusz Dziedzic
> <janusz.dziedzic@tieto.com> wrote:
>> On 6 July 2014 04:27, Adrian Chadd <adrian@freebsd.org> wrote:
>>> I think you may have to tell mac80211 that it's okay and not to drop the frames.
>>>
>>> The earlier atheros chips would just give you the AMSDU frames as
>>> deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
>>> net80211 and it'd ull it apart. But if the driver is doing it (or,
>>> well, the chip is doing it) then mac80211 needs to know not to drop
>>> those sub-frames.
>>>
>>> I wonder if you'll ever get notified before the complete A-MPDU has
>>> been received. That happens on previous chips. eg, you have an A-MPDU
>>> of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
>>> half of one MPDU before you hit EOL, the next notification you process
>>> will be the next MSDU in the same MPDU - and then you'll hit the
>>> reordering hilarity again.
>>>
>>> So hm, i will face the same issue in FreeBSD at some point, so I'd
>>> likely do what you're thinking of doing - pass up a chain of mbufs
>>> representing the current MPDU and treat the whole lot as the frame(s)
>>> to care about. Honestly though, I'm kind of wondering whether I should
>>> mostly just do what the Atheros reference driver does and treat it as
>>> ethernet payload frames (ie, it's already de-encapsulated and the
>>> reordering is already done) and just tack on the wifi header bit via
>>> another of the DMA rings.
>>>
>>> Ugh, I really should sit down and write the FreeBSD version of this.
>>>
>>>
>>>
>>> -a
>>>
>>> (I'm still having flashbacks from working on this firmware at QCA. Aiee.)
>>>
>>>
>>> On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>> There are two issues in handling the dis-aggregated A-MSDU subframes
>>>> in ieee80211_sta_manage_reorder_buf:
>>>>
>>>> 1. The out-of-date check:
>>>>
>>>>         /* frame with out of date sequence number */
>>>>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>>>>                 dev_kfree_skb(skb);
>>>>                 goto out;
>>>>         }
>>>>
>>>> As all of the subframes carry the same sequence number, the first
>>>> subframe will be delivered and increment head_seq_num and then all
>>>> subsequent subframes will be discarded.
>>>>
>>>>
>>>> 2. The duplicates check a bit later in the routine:
>>>>
>>>>         /* check if we already stored this frame */
>>>>         if (tid_agg_rx->reorder_buf[index]) {
>>>>                 dev_kfree_skb(skb);
>>>>                 goto out;
>>>>         }
>>>>
>>>> If there is enough packet loss that packets are queued in the reorder
>>>> buffer and not immediately released, then only the first subframe will
>>>> be stored. Subsequent subframes will be discarded as duplicates.
>>>>
>>>>
>>>> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
>>>> the reordering buffer changes, and dropped to about 8 Mbps with the
>>>> reorder buffer. Hacking around the out-of-date sequence number check
>>>> to allow all subframes to egress restores it to 170 Mbps.
>>>>
>>>> In the area where I'm testing there isn't enough 5 GHz noise to make
>>>> the duplicates-check issue happen very often. By adding a printk I can
>>>> see that it does happen, but it doesn't impact the throughput and I
>>>> can't report the impact of fixing it.
>>>>
>>>> ----
>>>>
>>>> I do wonder if both of these are symptoms of dis-aggregating the
>>>> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
>>>> at a time, and the ath10k A-MSDU case is sending it subframes instead.
>>>> Trying to make the ath10k code send up all of the subframes as a chain
>>>> of skbs didn't immediately work, but I do wonder if that would better
>>>> match the mac80211 expectations.
>>>>
>>>>
>>>>
>>>> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>> Yes, after some more testing it does look like an unfortunate
>>>>> interaction between the reorder buffer and A-MSDU. The disaggregated
>>>>> subframes all carry the same sequence number. The first subframe is
>>>>> released from the reorder buffer and increments the head_seq_num.
>>>>> Subsequent subframes are all discarded as being out of date.
>>>>>
>>>>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>>>>> head=0xb06
>>>>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>>>>> head=0xb0b
>>>>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>>>>> head=0xb10
>>>>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>>>>> head=0xb15
>>>>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>>>>> head=0xb1a
>>>>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>>>>> head=0xb1f
>>>>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>>>>> head=0xb24
>>>>>
>>>>>
>>>>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>>>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>>>>
>>>>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>>>>> second is consistently lost. The MacBook generally retransmits the
>>>>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>>>>> frame makes it through.
>>>>>>>
>>>>>>> This became more noticeable after the reordering fixes in
>>>>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>>>>
>>>>>>> I see this A-MSDU packet loss behavior both with and without the
>>>>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>>>>> second is dropped. However, *without* the reordering fixes (and
>>>>>>> therefore with packets delivered out of order) the MacBook sends
>>>>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>>>>> therefore has to deal with more packet loss. I suspect it is an
>>>>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>>>>> never going to fully understand, its stuck in a region of the
>>>>>>> congestion window where the Wifi driver keeps choosing to using
>>>>>>> A-MSDU.
>>>>>>
>>>>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>>>>
>>>>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>>>>> copying/allocation overhead each subframe is reported as a singly
>>>>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>>>>> A-MPDU reordering intereferes with A-MSDU now?
>>>>>>
>> Denton could you try attached patch: report amsdu in one big frame.
>> If help, we can add amsdu skb list support to mac80211/cfg80211 - to
>> improve performance and reduce memcpy, while currently we have skb
>> frames, put them in one big skb and next cfg80211 split them again
>> into msdus and report to stack.
>>
>> BR
>> Janusz

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Janusz Dziedzic]

On 8 July 2014 08:43, Denton Gentry <denton.gentry@gmail.com> wrote:
> I think I know what is happening now, though I've no idea why. The
> throughput is low because we have many TCP retransmissions. We have
> retransmissions because the TCP checksum is wrong on a number of
> frames, and I do find data corruption in the payload so the checksum
> definitely should be wrong. All of the corrupted frames were
> originally one of the subframes in an A-MSDU packet.
>
> An example follows at the end of this message, as dissected by
> Wireshark. iperf sends a very regular data pattern of "0123456789..."
> over and over. Note how in subframe #2 offset 0x1e0 the bytes "32 33
> 34" have been replaced by "72 36 35"
>
> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
>
> I added printks at the bottom of ath10k_htt_rx_amsdu immediately
> before the call to ath10k_process_rx. I found this same packet, and we
> see the "72 36 35" corruption in the printk. So I think it happened in
> ath10k_process_rx or before, not anything weird after passing it up to
> mac80211.
>
> [  101.863712] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
> [  101.863727] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
> [  101.863742] ath10k: 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32
> [  101.863757] ath10k: 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37 38
> [  101.863773] ath10k: 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34
> [  101.863788] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
> [  101.863803] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
>
>
> I've found a number of examples of similar corruption, always with
> between one and four bytes replaced.
>
> 35363738 -> e52c6e07
> 3435 -> b43f
> 3839 -> c238
> 31 -> 7f
> 3435 -> 7436
> 30 -> 50
> 3233 -> bc37
>
Seems this could be because of:

+ /* cfg80211 expect this padding */
+ padding = (4 - (skb->len + sizeof(subframe_hdr))) & 0x3;
+ skb_put(skb, padding);

>
> The packet described above, dissected by Wireshark:
>
> No.     Time        Source                Destination
> Protocol Length Info
>    2235 18.953349   192.168.144.79        192.168.144.13        TCP
>   3112   52697 > 5001 [ACK] Seq=1391553 Ack=1 Win=131760 [TCP CHECKSUM
> INCORRECT] Len=1448 TSval=1298580657 TSecr=4294947481
>
> Frame 2235: 3112 bytes on wire (24896 bits), 3112 bytes captured (24896 bits)
>     Encapsulation type: IEEE 802.11 plus radiotap radio header (23)
>     Arrival Time: Jul  7, 2014 23:03:37.763365000 PDT
>     [Time shift for this packet: 0.000000000 seconds]
>     Epoch Time: 1404799417.763365000 seconds
>     [Time delta from previous captured frame: 0.003476000 seconds]
>     [Time delta from previous displayed frame: 0.515641000 seconds]
>     [Time since reference or first frame: 18.953349000 seconds]
>     Frame Number: 2235
>     Frame Length: 3112 bytes (24896 bits)
>     Capture Length: 3112 bytes (24896 bits)
>     [Frame is marked: False]
>     [Frame is ignored: False]
>     [Protocols in frame: radiotap:wlan:llc:ip:tcp:data:llc:ip:tcp:data]
>     [Coloring Rule Name: TCP]
>     [Coloring Rule String: tcp]
> Radiotap Header v0, Length 38
>     Header revision: 0
>     Header pad: 0
>     Header length: 38
>     Present flags
>         .... .... .... .... .... .... .... ...1 = TSFT: True
>         .... .... .... .... .... .... .... ..1. = Flags: True
>         .... .... .... .... .... .... .... .0.. = Rate: False
>         .... .... .... .... .... .... .... 1... = Channel: True
>         .... .... .... .... .... .... ...0 .... = FHSS: False
>         .... .... .... .... .... .... ..1. .... = dBm Antenna Signal: True
>         .... .... .... .... .... .... .0.. .... = dBm Antenna Noise: False
>         .... .... .... .... .... .... 0... .... = Lock Quality: False
>         .... .... .... .... .... ...0 .... .... = TX Attenuation: False
>         .... .... .... .... .... ..0. .... .... = dB TX Attenuation: False
>         .... .... .... .... .... .0.. .... .... = dBm TX Power: False
>         .... .... .... .... .... 1... .... .... = Antenna: True
>         .... .... .... .... ...0 .... .... .... = dB Antenna Signal: False
>         .... .... .... .... ..0. .... .... .... = dB Antenna Noise: False
>         .... .... .... .... .1.. .... .... .... = RX flags: True
>         .... .... .... .0.. .... .... .... .... = Channel+: False
>         .... .... .... 0... .... .... .... .... = HT information: False
>         .... .... ...0 .... .... .... .... .... = A-MPDU Status: False
>         .... .... ..1. .... .... .... .... .... = VHT information: True
>         ...0 0000 00.. .... .... .... .... .... = Reserved: 0x00000000
>         ..0. .... .... .... .... .... .... .... = Radiotap NS next: False
>         .0.. .... .... .... .... .... .... .... = Vendor NS next: False
>         0... .... .... .... .... .... .... .... = Ext: False
>     MAC timestamp: 78051063
>     Flags: 0x00
>         .... ...0 = CFP: False
>         .... ..0. = Preamble: Long
>         .... .0.. = WEP: False
>         .... 0... = Fragmentation: False
>         ...0 .... = FCS at end: False
>         ..0. .... = Data Pad: False
>         .0.. .... = Bad FCS: False
>         0... .... = Short GI: False
>     Channel frequency: 5745 [A 149]
>     Channel type: 802.11a (0x0140)
>         .... .... ...0 .... = Turbo: False
>         .... .... ..0. .... = Complementary Code Keying (CCK): False
>         .... .... .1.. .... = Orthogonal Frequency-Division
> Multiplexing (OFDM): True
>         .... .... 0... .... = 2 GHz spectrum: False
>         .... ...1 .... .... = 5 GHz spectrum: True
>         .... ..0. .... .... = Passive: False
>         .... .0.. .... .... = Dynamic CCK-OFDM: False
>         .... 0... .... .... = Gaussian Frequency Shift Keying (GFSK): False
>         ...0 .... .... .... = GSM (900MHz): False
>         ..0. .... .... .... = Static Turbo: False
>         .0.. .... .... .... = Half Rate Channel (10MHz Channel Width): False
>         0... .... .... .... = Quarter Rate Channel (5MHz Channel Width): False
>     SSI Signal: -53 dBm
>     Antenna: 0
>     RX flags: 0x0000
>         .... .... .... .... .... ..0. = Bad PLCP: False
>     VHT information
>         Known VHT information: 0x44
>             .... .... .... ...0 = STBC: False
>             .... .... .... ..0. = TXOP_PS_NOT_ALLOWED: False
>             .... .... .... .1.. = Guard interval: True
>             .... .... .... 0... = SGI Nsym disambiguation: False
>             .... .... ...0 .... = LDPC extra OFDM symbol: False
>             .... .... ..0. .... = Beamformed: False
>             .... .... .1.. .... = Bandwidth: True
>             .... .... 0... .... = Group ID: False
>             .... ...0 .... .... = Partial AID: False
>         .... .0.. = Guard interval: long (0)
>         Bandwidth: 80 MHz (4)
>         User 0: MCS 8
>             1000 .... = MCS index 0: 8 (256-QAM 3/4)
>             .... 0010 = Spatial streams 0: 2
>             Space-time streams 0: 2
>             Coding 0: BCC (0)
>             [Data Rate: 702.0 Mb/s]
> IEEE 802.11 QoS Data, Flags: .......T
>     Type/Subtype: QoS Data (0x28)
>     Frame Control Field: 0x8801
>         .... ..00 = Version: 0
>         .... 10.. = Type: Data frame (2)
>         1000 .... = Subtype: 8
>         Flags: 0x01
>             .... ..01 = DS status: Frame from STA to DS via an AP (To
> DS: 1 From DS: 0) (0x01)
>             .... .0.. = More Fragments: This is the last fragment
>             .... 0... = Retry: Frame is not being retransmitted
>             ...0 .... = PWR MGT: STA will stay up
>             ..0. .... = More Data: No data buffered
>             .0.. .... = Protected flag: Data is not protected
>             0... .... = Order flag: Not strictly ordered
>     .000 0000 0011 0000 = Duration: 48 microseconds
>     Receiver address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>     BSS Id: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>     Transmitter address: Apple_67:24:54 (84:38:35:67:24:54)
>     Source address: Apple_67:24:54 (84:38:35:67:24:54)
>     Destination address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>     Fragment number: 0
>     Sequence number: 1021
>     Qos Control: 0x0080
>         .... .... .... 0000 = TID: 0
>         [.... .... .... .000 = Priority: Best Effort (Best Effort) (0)]
>         .... .... ...0 .... = QoS bit 4: Bits 8-15 of QoS Control
> field are TXOP Duration Requested
>         .... .... .00. .... = Ack Policy: Normal Ack (0x0000)
>         .... .... 1... .... = Payload Type: A-MSDU
>         0000 0000 .... .... = TXOP Duration Requested: 0 (no TXOP requested)
> IEEE 802.11 Aggregate MSDU
>     A-MSDU Subframe #1
>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>         A-MSDU Length: 1510
>         Logical-Link Control
>             DSAP: SNAP (0xaa)
>             IG Bit: Individual
>             SSAP: SNAP (0xaa)
>             CR Bit: Command
>             Control field: U, func=UI (0x03)
>                 000. 00.. = Command: Unnumbered Information (0x00)
>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>             Organization Code: Encapsulated Ethernet (0x000000)
>             Type: IP (0x0800)
>         Internet Protocol Version 4, Src: 192.168.144.79
> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>             Version: 4
>             Header length: 20 bytes
>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>                 .... ..00 = Explicit Congestion Notification: Not-ECT
> (Not ECN-Capable Transport) (0x00)
>             Total Length: 1500
>             Identification: 0xc622 (50722)
>             Flags: 0x00
>                 0... .... = Reserved bit: Not set
>                 .0.. .... = Don't fragment: Not set
>                 ..0. .... = More fragments: Not set
>             Fragment offset: 0
>             Time to live: 64
>             Protocol: TCP (6)
>             Header checksum: 0x0d4c [correct]
>                 [Good: True]
>                 [Bad: False]
>             Source: 192.168.144.79 (192.168.144.79)
>             Destination: 192.168.144.13 (192.168.144.13)
>             [Source GeoIP: Unknown]
>             [Destination GeoIP: Unknown]
>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
> Port: 5001 (5001), Seq: 1390105, Ack: 1, Len: 1448
>             Source port: 52697 (52697)
>             Destination port: 5001 (5001)
>             [Stream index: 0]
>             Sequence number: 1390105    (relative sequence number)
>             [Next sequence number: 1391553    (relative sequence number)]
>             Acknowledgment number: 1    (relative ack number)
>             Header length: 32 bytes
>             Flags: 0x010 (ACK)
>                 000. .... .... = Reserved: Not set
>                 ...0 .... .... = Nonce: Not set
>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>                 .... .0.. .... = ECN-Echo: Not set
>                 .... ..0. .... = Urgent: Not set
>                 .... ...1 .... = Acknowledgment: Set
>                 .... .... 0... = Push: Not set
>                 .... .... .0.. = Reset: Not set
>                 .... .... ..0. = Syn: Not set
>                 .... .... ...0 = Fin: Not set
>             Window size value: 8235
>             [Calculated window size: 131760]
>             [Window size scaling factor: 16]
>             Checksum: 0xa1c0 [correct]
>                 [Good Checksum: True]
>                 [Bad Checksum: False]
>             Options: (12 bytes), No-Operation (NOP), No-Operation
> (NOP), Timestamps
>                 No-Operation (NOP)
>                     Type: 1
>                         0... .... = Copy on fragmentation: No
>                         .00. .... = Class: Control (0)
>                         ...0 0001 = Number: No-Operation (NOP) (1)
>                 No-Operation (NOP)
>                     Type: 1
>                         0... .... = Copy on fragmentation: No
>                         .00. .... = Class: Control (0)
>                         ...0 0001 = Number: No-Operation (NOP) (1)
>                 Timestamps: TSval 1298580657, TSecr 4294947481
>                     Kind: Timestamp (8)
>                     Length: 10
>                     Timestamp value: 1298580657
>                     Timestamp echo reply: 4294947481
>         Data (1448 bytes)
>
> 0000  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0010  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0020  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0030  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0040  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0050  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0060  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0070  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0080  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0090  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 00a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 00b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 00c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 00d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 00e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 00f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0100  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0110  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0120  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0130  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0140  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0150  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0160  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0170  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0180  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0190  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 01a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 01b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 01c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 01d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 01e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 01f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0200  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0210  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0220  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0230  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0240  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0250  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0260  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0270  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0280  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0290  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 02a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 02b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 02c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 02d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 02e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 02f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0300  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0310  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0320  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0330  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0340  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0350  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0360  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0370  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0380  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0390  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 03a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 03b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 03c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 03d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 03e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 03f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0400  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0410  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0420  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0430  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0440  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0450  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0460  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0470  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0480  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0490  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 04a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 04b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 04c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 04d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 04e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 04f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0500  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0510  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0520  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0530  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0540  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0550  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0560  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0570  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0580  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0590  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 05a0  34 35 36 37 38 39 30 31                           45678901
>             Data: 343536373839303132333435363738393031323334353637...
>             [Length: 1448]
>     A-MSDU Subframe #2
>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>         A-MSDU Length: 1510
>         Logical-Link Control
>             DSAP: SNAP (0xaa)
>             IG Bit: Individual
>             SSAP: SNAP (0xaa)
>             CR Bit: Command
>             Control field: U, func=UI (0x03)
>                 000. 00.. = Command: Unnumbered Information (0x00)
>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>             Organization Code: Encapsulated Ethernet (0x000000)
>             Type: IP (0x0800)
>         Internet Protocol Version 4, Src: 192.168.144.79
> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>             Version: 4
>             Header length: 20 bytes
>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>                 .... ..00 = Explicit Congestion Notification: Not-ECT
> (Not ECN-Capable Transport) (0x00)
>             Total Length: 1500
>             Identification: 0xda09 (55817)
>             Flags: 0x00
>                 0... .... = Reserved bit: Not set
>                 .0.. .... = Don't fragment: Not set
>                 ..0. .... = More fragments: Not set
>             Fragment offset: 0
>             Time to live: 64
>             Protocol: TCP (6)
>             Header checksum: 0xf964 [correct]
>                 [Good: True]
>                 [Bad: False]
>             Source: 192.168.144.79 (192.168.144.79)
>             Destination: 192.168.144.13 (192.168.144.13)
>             [Source GeoIP: Unknown]
>             [Destination GeoIP: Unknown]
>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
> Port: 5001 (5001), Seq: 1391553, Ack: 1, Len: 1448
>             Source port: 52697 (52697)
>             Destination port: 5001 (5001)
>             [Stream index: 0]
>             Sequence number: 1391553    (relative sequence number)
>             [Next sequence number: 1393001    (relative sequence number)]
>             Acknowledgment number: 1    (relative ack number)
>             Header length: 32 bytes
>             Flags: 0x010 (ACK)
>                 000. .... .... = Reserved: Not set
>                 ...0 .... .... = Nonce: Not set
>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>                 .... .0.. .... = ECN-Echo: Not set
>                 .... ..0. .... = Urgent: Not set
>                 .... ...1 .... = Acknowledgment: Set
>                 .... .... 0... = Push: Not set
>                 .... .... .0.. = Reset: Not set
>                 .... .... ..0. = Syn: Not set
>                 .... .... ...0 = Fin: Not set
>             Window size value: 8235
>             [Calculated window size: 131760]
>             [Window size scaling factor: 16]
>             Checksum: 0x9a16 [incorrect, should be 0x5913 (maybe
> caused by "TCP checksum offload"?)]
>                 [Good Checksum: False]
>                 [Bad Checksum: True]
>                     [Expert Info (Error/Checksum): Bad checksum]
>                         [Message: Bad checksum]
>                         [Severity level: Error]
>                         [Group: Checksum]
>             Options: (12 bytes), No-Operation (NOP), No-Operation
> (NOP), Timestamps
>                 No-Operation (NOP)
>                     Type: 1
>                         0... .... = Copy on fragmentation: No
>                         .00. .... = Class: Control (0)
>                         ...0 0001 = Number: No-Operation (NOP) (1)
>                 No-Operation (NOP)
>                     Type: 1
>                         0... .... = Copy on fragmentation: No
>                         .00. .... = Class: Control (0)
>                         ...0 0001 = Number: No-Operation (NOP) (1)
>                 Timestamps: TSval 1298580657, TSecr 4294947481
>                     Kind: Timestamp (8)
>                     Length: 10
>                     Timestamp value: 1298580657
>                     Timestamp echo reply: 4294947481
>         Data (1448 bytes)
>
> 0000  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0010  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0020  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0030  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0040  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0050  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0060  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0070  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0080  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0090  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 00a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 00b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 00c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 00d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 00e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 00f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0100  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0110  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0120  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0130  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0140  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0150  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0160  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0170  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0180  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0190  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 01a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 01b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 01c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 01d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
> 01f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0200  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0210  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0220  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0230  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0240  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0250  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0260  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0270  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0280  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0290  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 02a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 02b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 02c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 02d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 02e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 02f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0300  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0310  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0320  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0330  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0340  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0350  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0360  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0370  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0380  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0390  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 03a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 03b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 03c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 03d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 03e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 03f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0400  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0410  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0420  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0430  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0440  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0450  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0460  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0470  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0480  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0490  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 04a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 04b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 04c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 04d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 04e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 04f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0500  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0510  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0520  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0530  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0540  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 0550  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
> 0560  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
> 0570  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
> 0580  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
> 0590  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
> 05a0  32 33 34 35 36 37 38 39                           23456789
>             Data: 323334353637383930313233343536373839303132333435...
>             [Length: 1448]
>
> On Mon, Jul 7, 2014 at 12:26 PM, Denton Gentry <denton.gentry@gmail.com> wrote:
>> The initial results are not promising: a MacOS 802.11ac client gets
>> between 0-2 Mbps with this change, where it was getting about 8 Mbps
>> prior to this change and ~170 Mbps prior to the reordering fix. A pcap
>> from the receiving system shows a very large number of out of order
>> frames, likely due to TCP retransmission.
>>
>> An 802.11n MacBook gets very good throughput, only the 802.11ac
>> MacBook shows the very poor result. I'm trying to figure out why.
>>
>>
>> One specific note (and probably not related to the throughput): I
>> believe ath10k_htt_rx_amsdu runs in the tasklet, which means it would
>> need to use GFP_ATOMIC instead of GFP_KERNEL. Do I understand it
>> correctly?
>>
>>
>> On Mon, Jul 7, 2014 at 1:30 AM, Janusz Dziedzic
>> <janusz.dziedzic@tieto.com> wrote:
>>> On 6 July 2014 04:27, Adrian Chadd <adrian@freebsd.org> wrote:
>>>> I think you may have to tell mac80211 that it's okay and not to drop the frames.
>>>>
>>>> The earlier atheros chips would just give you the AMSDU frames as
>>>> deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
>>>> net80211 and it'd ull it apart. But if the driver is doing it (or,
>>>> well, the chip is doing it) then mac80211 needs to know not to drop
>>>> those sub-frames.
>>>>
>>>> I wonder if you'll ever get notified before the complete A-MPDU has
>>>> been received. That happens on previous chips. eg, you have an A-MPDU
>>>> of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
>>>> half of one MPDU before you hit EOL, the next notification you process
>>>> will be the next MSDU in the same MPDU - and then you'll hit the
>>>> reordering hilarity again.
>>>>
>>>> So hm, i will face the same issue in FreeBSD at some point, so I'd
>>>> likely do what you're thinking of doing - pass up a chain of mbufs
>>>> representing the current MPDU and treat the whole lot as the frame(s)
>>>> to care about. Honestly though, I'm kind of wondering whether I should
>>>> mostly just do what the Atheros reference driver does and treat it as
>>>> ethernet payload frames (ie, it's already de-encapsulated and the
>>>> reordering is already done) and just tack on the wifi header bit via
>>>> another of the DMA rings.
>>>>
>>>> Ugh, I really should sit down and write the FreeBSD version of this.
>>>>
>>>>
>>>>
>>>> -a
>>>>
>>>> (I'm still having flashbacks from working on this firmware at QCA. Aiee.)
>>>>
>>>>
>>>> On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>> There are two issues in handling the dis-aggregated A-MSDU subframes
>>>>> in ieee80211_sta_manage_reorder_buf:
>>>>>
>>>>> 1. The out-of-date check:
>>>>>
>>>>>         /* frame with out of date sequence number */
>>>>>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>>>>>                 dev_kfree_skb(skb);
>>>>>                 goto out;
>>>>>         }
>>>>>
>>>>> As all of the subframes carry the same sequence number, the first
>>>>> subframe will be delivered and increment head_seq_num and then all
>>>>> subsequent subframes will be discarded.
>>>>>
>>>>>
>>>>> 2. The duplicates check a bit later in the routine:
>>>>>
>>>>>         /* check if we already stored this frame */
>>>>>         if (tid_agg_rx->reorder_buf[index]) {
>>>>>                 dev_kfree_skb(skb);
>>>>>                 goto out;
>>>>>         }
>>>>>
>>>>> If there is enough packet loss that packets are queued in the reorder
>>>>> buffer and not immediately released, then only the first subframe will
>>>>> be stored. Subsequent subframes will be discarded as duplicates.
>>>>>
>>>>>
>>>>> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
>>>>> the reordering buffer changes, and dropped to about 8 Mbps with the
>>>>> reorder buffer. Hacking around the out-of-date sequence number check
>>>>> to allow all subframes to egress restores it to 170 Mbps.
>>>>>
>>>>> In the area where I'm testing there isn't enough 5 GHz noise to make
>>>>> the duplicates-check issue happen very often. By adding a printk I can
>>>>> see that it does happen, but it doesn't impact the throughput and I
>>>>> can't report the impact of fixing it.
>>>>>
>>>>> ----
>>>>>
>>>>> I do wonder if both of these are symptoms of dis-aggregating the
>>>>> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
>>>>> at a time, and the ath10k A-MSDU case is sending it subframes instead.
>>>>> Trying to make the ath10k code send up all of the subframes as a chain
>>>>> of skbs didn't immediately work, but I do wonder if that would better
>>>>> match the mac80211 expectations.
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>> Yes, after some more testing it does look like an unfortunate
>>>>>> interaction between the reorder buffer and A-MSDU. The disaggregated
>>>>>> subframes all carry the same sequence number. The first subframe is
>>>>>> released from the reorder buffer and increments the head_seq_num.
>>>>>> Subsequent subframes are all discarded as being out of date.
>>>>>>
>>>>>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>>>>>> head=0xb06
>>>>>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>>>>>> head=0xb0b
>>>>>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>>>>>> head=0xb10
>>>>>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>>>>>> head=0xb15
>>>>>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>>>>>> head=0xb1a
>>>>>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>>>>>> head=0xb1f
>>>>>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>>>>>> head=0xb24
>>>>>>
>>>>>>
>>>>>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>>>>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>>>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>>>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>>>>>
>>>>>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>>>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>>>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>>>>>> second is consistently lost. The MacBook generally retransmits the
>>>>>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>>>>>> frame makes it through.
>>>>>>>>
>>>>>>>> This became more noticeable after the reordering fixes in
>>>>>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>>>>>
>>>>>>>> I see this A-MSDU packet loss behavior both with and without the
>>>>>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>>>>>> second is dropped. However, *without* the reordering fixes (and
>>>>>>>> therefore with packets delivered out of order) the MacBook sends
>>>>>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>>>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>>>>>> therefore has to deal with more packet loss. I suspect it is an
>>>>>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>>>>>> never going to fully understand, its stuck in a region of the
>>>>>>>> congestion window where the Wifi driver keeps choosing to using
>>>>>>>> A-MSDU.
>>>>>>>
>>>>>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>>>>>
>>>>>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>>>>>> copying/allocation overhead each subframe is reported as a singly
>>>>>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>>>>>> A-MPDU reordering intereferes with A-MSDU now?
>>>>>>>
>>> Denton could you try attached patch: report amsdu in one big frame.
>>> If help, we can add amsdu skb list support to mac80211/cfg80211 - to
>>> improve performance and reduce memcpy, while currently we have skb
>>> frames, put them in one big skb and next cfg80211 split them again
>>> into msdus and report to stack.
>>>
>>> BR
>>> Janusz

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Janusz Dziedzic]

On 8 July 2014 08:50, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:
> On 8 July 2014 08:43, Denton Gentry <denton.gentry@gmail.com> wrote:
>> I think I know what is happening now, though I've no idea why. The
>> throughput is low because we have many TCP retransmissions. We have
>> retransmissions because the TCP checksum is wrong on a number of
>> frames, and I do find data corruption in the payload so the checksum
>> definitely should be wrong. All of the corrupted frames were
>> originally one of the subframes in an A-MSDU packet.
>>
>> An example follows at the end of this message, as dissected by
>> Wireshark. iperf sends a very regular data pattern of "0123456789..."
>> over and over. Note how in subframe #2 offset 0x1e0 the bytes "32 33
>> 34" have been replaced by "72 36 35"
>>
>> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
>>
>> I added printks at the bottom of ath10k_htt_rx_amsdu immediately
>> before the call to ath10k_process_rx. I found this same packet, and we
>> see the "72 36 35" corruption in the printk. So I think it happened in
>> ath10k_process_rx or before, not anything weird after passing it up to
>> mac80211.
>>
>> [  101.863712] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
>> [  101.863727] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
>> [  101.863742] ath10k: 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32
>> [  101.863757] ath10k: 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37 38
>> [  101.863773] ath10k: 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34
>> [  101.863788] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
>> [  101.863803] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
>>
>>
>> I've found a number of examples of similar corruption, always with
>> between one and four bytes replaced.
>>
>> 35363738 -> e52c6e07
>> 3435 -> b43f
>> 3839 -> c238
>> 31 -> 7f
>> 3435 -> 7436
>> 30 -> 50
>> 3233 -> bc37
>>
> Seems this could be because of:
>
> + /* cfg80211 expect this padding */
> + padding = (4 - (skb->len + sizeof(subframe_hdr))) & 0x3;
> + skb_put(skb, padding);
>

BTW, when AP (ath10k) send TCP ACK - are this packets also AMSDU?
I see in my case AP can send 3 x A-MSDU (with total size 304 bytes)
small frames aggregated. Maybe your HW have problems with that.
As I remember correctly someone some time ago report problems with
MacBook pro retina but I am not sure this is the same, while no one
tests the fix.

>>
>> The packet described above, dissected by Wireshark:
>>
>> No.     Time        Source                Destination
>> Protocol Length Info
>>    2235 18.953349   192.168.144.79        192.168.144.13        TCP
>>   3112   52697 > 5001 [ACK] Seq=1391553 Ack=1 Win=131760 [TCP CHECKSUM
>> INCORRECT] Len=1448 TSval=1298580657 TSecr=4294947481
>>
>> Frame 2235: 3112 bytes on wire (24896 bits), 3112 bytes captured (24896 bits)
>>     Encapsulation type: IEEE 802.11 plus radiotap radio header (23)
>>     Arrival Time: Jul  7, 2014 23:03:37.763365000 PDT
>>     [Time shift for this packet: 0.000000000 seconds]
>>     Epoch Time: 1404799417.763365000 seconds
>>     [Time delta from previous captured frame: 0.003476000 seconds]
>>     [Time delta from previous displayed frame: 0.515641000 seconds]
>>     [Time since reference or first frame: 18.953349000 seconds]
>>     Frame Number: 2235
>>     Frame Length: 3112 bytes (24896 bits)
>>     Capture Length: 3112 bytes (24896 bits)
>>     [Frame is marked: False]
>>     [Frame is ignored: False]
>>     [Protocols in frame: radiotap:wlan:llc:ip:tcp:data:llc:ip:tcp:data]
>>     [Coloring Rule Name: TCP]
>>     [Coloring Rule String: tcp]
>> Radiotap Header v0, Length 38
>>     Header revision: 0
>>     Header pad: 0
>>     Header length: 38
>>     Present flags
>>         .... .... .... .... .... .... .... ...1 = TSFT: True
>>         .... .... .... .... .... .... .... ..1. = Flags: True
>>         .... .... .... .... .... .... .... .0.. = Rate: False
>>         .... .... .... .... .... .... .... 1... = Channel: True
>>         .... .... .... .... .... .... ...0 .... = FHSS: False
>>         .... .... .... .... .... .... ..1. .... = dBm Antenna Signal: True
>>         .... .... .... .... .... .... .0.. .... = dBm Antenna Noise: False
>>         .... .... .... .... .... .... 0... .... = Lock Quality: False
>>         .... .... .... .... .... ...0 .... .... = TX Attenuation: False
>>         .... .... .... .... .... ..0. .... .... = dB TX Attenuation: False
>>         .... .... .... .... .... .0.. .... .... = dBm TX Power: False
>>         .... .... .... .... .... 1... .... .... = Antenna: True
>>         .... .... .... .... ...0 .... .... .... = dB Antenna Signal: False
>>         .... .... .... .... ..0. .... .... .... = dB Antenna Noise: False
>>         .... .... .... .... .1.. .... .... .... = RX flags: True
>>         .... .... .... .0.. .... .... .... .... = Channel+: False
>>         .... .... .... 0... .... .... .... .... = HT information: False
>>         .... .... ...0 .... .... .... .... .... = A-MPDU Status: False
>>         .... .... ..1. .... .... .... .... .... = VHT information: True
>>         ...0 0000 00.. .... .... .... .... .... = Reserved: 0x00000000
>>         ..0. .... .... .... .... .... .... .... = Radiotap NS next: False
>>         .0.. .... .... .... .... .... .... .... = Vendor NS next: False
>>         0... .... .... .... .... .... .... .... = Ext: False
>>     MAC timestamp: 78051063
>>     Flags: 0x00
>>         .... ...0 = CFP: False
>>         .... ..0. = Preamble: Long
>>         .... .0.. = WEP: False
>>         .... 0... = Fragmentation: False
>>         ...0 .... = FCS at end: False
>>         ..0. .... = Data Pad: False
>>         .0.. .... = Bad FCS: False
>>         0... .... = Short GI: False
>>     Channel frequency: 5745 [A 149]
>>     Channel type: 802.11a (0x0140)
>>         .... .... ...0 .... = Turbo: False
>>         .... .... ..0. .... = Complementary Code Keying (CCK): False
>>         .... .... .1.. .... = Orthogonal Frequency-Division
>> Multiplexing (OFDM): True
>>         .... .... 0... .... = 2 GHz spectrum: False
>>         .... ...1 .... .... = 5 GHz spectrum: True
>>         .... ..0. .... .... = Passive: False
>>         .... .0.. .... .... = Dynamic CCK-OFDM: False
>>         .... 0... .... .... = Gaussian Frequency Shift Keying (GFSK): False
>>         ...0 .... .... .... = GSM (900MHz): False
>>         ..0. .... .... .... = Static Turbo: False
>>         .0.. .... .... .... = Half Rate Channel (10MHz Channel Width): False
>>         0... .... .... .... = Quarter Rate Channel (5MHz Channel Width): False
>>     SSI Signal: -53 dBm
>>     Antenna: 0
>>     RX flags: 0x0000
>>         .... .... .... .... .... ..0. = Bad PLCP: False
>>     VHT information
>>         Known VHT information: 0x44
>>             .... .... .... ...0 = STBC: False
>>             .... .... .... ..0. = TXOP_PS_NOT_ALLOWED: False
>>             .... .... .... .1.. = Guard interval: True
>>             .... .... .... 0... = SGI Nsym disambiguation: False
>>             .... .... ...0 .... = LDPC extra OFDM symbol: False
>>             .... .... ..0. .... = Beamformed: False
>>             .... .... .1.. .... = Bandwidth: True
>>             .... .... 0... .... = Group ID: False
>>             .... ...0 .... .... = Partial AID: False
>>         .... .0.. = Guard interval: long (0)
>>         Bandwidth: 80 MHz (4)
>>         User 0: MCS 8
>>             1000 .... = MCS index 0: 8 (256-QAM 3/4)
>>             .... 0010 = Spatial streams 0: 2
>>             Space-time streams 0: 2
>>             Coding 0: BCC (0)
>>             [Data Rate: 702.0 Mb/s]
>> IEEE 802.11 QoS Data, Flags: .......T
>>     Type/Subtype: QoS Data (0x28)
>>     Frame Control Field: 0x8801
>>         .... ..00 = Version: 0
>>         .... 10.. = Type: Data frame (2)
>>         1000 .... = Subtype: 8
>>         Flags: 0x01
>>             .... ..01 = DS status: Frame from STA to DS via an AP (To
>> DS: 1 From DS: 0) (0x01)
>>             .... .0.. = More Fragments: This is the last fragment
>>             .... 0... = Retry: Frame is not being retransmitted
>>             ...0 .... = PWR MGT: STA will stay up
>>             ..0. .... = More Data: No data buffered
>>             .0.. .... = Protected flag: Data is not protected
>>             0... .... = Order flag: Not strictly ordered
>>     .000 0000 0011 0000 = Duration: 48 microseconds
>>     Receiver address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>     BSS Id: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>     Transmitter address: Apple_67:24:54 (84:38:35:67:24:54)
>>     Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>     Destination address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>     Fragment number: 0
>>     Sequence number: 1021
>>     Qos Control: 0x0080
>>         .... .... .... 0000 = TID: 0
>>         [.... .... .... .000 = Priority: Best Effort (Best Effort) (0)]
>>         .... .... ...0 .... = QoS bit 4: Bits 8-15 of QoS Control
>> field are TXOP Duration Requested
>>         .... .... .00. .... = Ack Policy: Normal Ack (0x0000)
>>         .... .... 1... .... = Payload Type: A-MSDU
>>         0000 0000 .... .... = TXOP Duration Requested: 0 (no TXOP requested)
>> IEEE 802.11 Aggregate MSDU
>>     A-MSDU Subframe #1
>>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>         A-MSDU Length: 1510
>>         Logical-Link Control
>>             DSAP: SNAP (0xaa)
>>             IG Bit: Individual
>>             SSAP: SNAP (0xaa)
>>             CR Bit: Command
>>             Control field: U, func=UI (0x03)
>>                 000. 00.. = Command: Unnumbered Information (0x00)
>>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>>             Organization Code: Encapsulated Ethernet (0x000000)
>>             Type: IP (0x0800)
>>         Internet Protocol Version 4, Src: 192.168.144.79
>> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>>             Version: 4
>>             Header length: 20 bytes
>>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
>> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>>                 .... ..00 = Explicit Congestion Notification: Not-ECT
>> (Not ECN-Capable Transport) (0x00)
>>             Total Length: 1500
>>             Identification: 0xc622 (50722)
>>             Flags: 0x00
>>                 0... .... = Reserved bit: Not set
>>                 .0.. .... = Don't fragment: Not set
>>                 ..0. .... = More fragments: Not set
>>             Fragment offset: 0
>>             Time to live: 64
>>             Protocol: TCP (6)
>>             Header checksum: 0x0d4c [correct]
>>                 [Good: True]
>>                 [Bad: False]
>>             Source: 192.168.144.79 (192.168.144.79)
>>             Destination: 192.168.144.13 (192.168.144.13)
>>             [Source GeoIP: Unknown]
>>             [Destination GeoIP: Unknown]
>>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
>> Port: 5001 (5001), Seq: 1390105, Ack: 1, Len: 1448
>>             Source port: 52697 (52697)
>>             Destination port: 5001 (5001)
>>             [Stream index: 0]
>>             Sequence number: 1390105    (relative sequence number)
>>             [Next sequence number: 1391553    (relative sequence number)]
>>             Acknowledgment number: 1    (relative ack number)
>>             Header length: 32 bytes
>>             Flags: 0x010 (ACK)
>>                 000. .... .... = Reserved: Not set
>>                 ...0 .... .... = Nonce: Not set
>>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>>                 .... .0.. .... = ECN-Echo: Not set
>>                 .... ..0. .... = Urgent: Not set
>>                 .... ...1 .... = Acknowledgment: Set
>>                 .... .... 0... = Push: Not set
>>                 .... .... .0.. = Reset: Not set
>>                 .... .... ..0. = Syn: Not set
>>                 .... .... ...0 = Fin: Not set
>>             Window size value: 8235
>>             [Calculated window size: 131760]
>>             [Window size scaling factor: 16]
>>             Checksum: 0xa1c0 [correct]
>>                 [Good Checksum: True]
>>                 [Bad Checksum: False]
>>             Options: (12 bytes), No-Operation (NOP), No-Operation
>> (NOP), Timestamps
>>                 No-Operation (NOP)
>>                     Type: 1
>>                         0... .... = Copy on fragmentation: No
>>                         .00. .... = Class: Control (0)
>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>                 No-Operation (NOP)
>>                     Type: 1
>>                         0... .... = Copy on fragmentation: No
>>                         .00. .... = Class: Control (0)
>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>                 Timestamps: TSval 1298580657, TSecr 4294947481
>>                     Kind: Timestamp (8)
>>                     Length: 10
>>                     Timestamp value: 1298580657
>>                     Timestamp echo reply: 4294947481
>>         Data (1448 bytes)
>>
>> 0000  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0010  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0020  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0030  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0040  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0050  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0060  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0070  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0080  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0090  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 00a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 00b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 00c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 00d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 00e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 00f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0100  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0110  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0120  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0130  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0140  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0150  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0160  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0170  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0180  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0190  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 01a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 01b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 01c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 01d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 01e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 01f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0200  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0210  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0220  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0230  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0240  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0250  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0260  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0270  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0280  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0290  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 02a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 02b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 02c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 02d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 02e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 02f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0300  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0310  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0320  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0330  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0340  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0350  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0360  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0370  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0380  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0390  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 03a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 03b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 03c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 03d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 03e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 03f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0400  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0410  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0420  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0430  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0440  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0450  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0460  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0470  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0480  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0490  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 04a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 04b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 04c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 04d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 04e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 04f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0500  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0510  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0520  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0530  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0540  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0550  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0560  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0570  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0580  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0590  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 05a0  34 35 36 37 38 39 30 31                           45678901
>>             Data: 343536373839303132333435363738393031323334353637...
>>             [Length: 1448]
>>     A-MSDU Subframe #2
>>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>         A-MSDU Length: 1510
>>         Logical-Link Control
>>             DSAP: SNAP (0xaa)
>>             IG Bit: Individual
>>             SSAP: SNAP (0xaa)
>>             CR Bit: Command
>>             Control field: U, func=UI (0x03)
>>                 000. 00.. = Command: Unnumbered Information (0x00)
>>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>>             Organization Code: Encapsulated Ethernet (0x000000)
>>             Type: IP (0x0800)
>>         Internet Protocol Version 4, Src: 192.168.144.79
>> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>>             Version: 4
>>             Header length: 20 bytes
>>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
>> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>>                 .... ..00 = Explicit Congestion Notification: Not-ECT
>> (Not ECN-Capable Transport) (0x00)
>>             Total Length: 1500
>>             Identification: 0xda09 (55817)
>>             Flags: 0x00
>>                 0... .... = Reserved bit: Not set
>>                 .0.. .... = Don't fragment: Not set
>>                 ..0. .... = More fragments: Not set
>>             Fragment offset: 0
>>             Time to live: 64
>>             Protocol: TCP (6)
>>             Header checksum: 0xf964 [correct]
>>                 [Good: True]
>>                 [Bad: False]
>>             Source: 192.168.144.79 (192.168.144.79)
>>             Destination: 192.168.144.13 (192.168.144.13)
>>             [Source GeoIP: Unknown]
>>             [Destination GeoIP: Unknown]
>>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
>> Port: 5001 (5001), Seq: 1391553, Ack: 1, Len: 1448
>>             Source port: 52697 (52697)
>>             Destination port: 5001 (5001)
>>             [Stream index: 0]
>>             Sequence number: 1391553    (relative sequence number)
>>             [Next sequence number: 1393001    (relative sequence number)]
>>             Acknowledgment number: 1    (relative ack number)
>>             Header length: 32 bytes
>>             Flags: 0x010 (ACK)
>>                 000. .... .... = Reserved: Not set
>>                 ...0 .... .... = Nonce: Not set
>>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>>                 .... .0.. .... = ECN-Echo: Not set
>>                 .... ..0. .... = Urgent: Not set
>>                 .... ...1 .... = Acknowledgment: Set
>>                 .... .... 0... = Push: Not set
>>                 .... .... .0.. = Reset: Not set
>>                 .... .... ..0. = Syn: Not set
>>                 .... .... ...0 = Fin: Not set
>>             Window size value: 8235
>>             [Calculated window size: 131760]
>>             [Window size scaling factor: 16]
>>             Checksum: 0x9a16 [incorrect, should be 0x5913 (maybe
>> caused by "TCP checksum offload"?)]
>>                 [Good Checksum: False]
>>                 [Bad Checksum: True]
>>                     [Expert Info (Error/Checksum): Bad checksum]
>>                         [Message: Bad checksum]
>>                         [Severity level: Error]
>>                         [Group: Checksum]
>>             Options: (12 bytes), No-Operation (NOP), No-Operation
>> (NOP), Timestamps
>>                 No-Operation (NOP)
>>                     Type: 1
>>                         0... .... = Copy on fragmentation: No
>>                         .00. .... = Class: Control (0)
>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>                 No-Operation (NOP)
>>                     Type: 1
>>                         0... .... = Copy on fragmentation: No
>>                         .00. .... = Class: Control (0)
>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>                 Timestamps: TSval 1298580657, TSecr 4294947481
>>                     Kind: Timestamp (8)
>>                     Length: 10
>>                     Timestamp value: 1298580657
>>                     Timestamp echo reply: 4294947481
>>         Data (1448 bytes)
>>
>> 0000  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0010  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0020  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0030  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0040  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0050  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0060  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0070  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0080  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0090  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 00a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 00b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 00c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 00d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 00e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 00f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0100  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0110  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0120  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0130  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0140  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0150  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0160  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0170  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0180  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0190  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 01a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 01b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 01c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 01d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
>> 01f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0200  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0210  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0220  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0230  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0240  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0250  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0260  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0270  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0280  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0290  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 02a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 02b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 02c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 02d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 02e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 02f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0300  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0310  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0320  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0330  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0340  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0350  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0360  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0370  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0380  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0390  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 03a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 03b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 03c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 03d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 03e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 03f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0400  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0410  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0420  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0430  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0440  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0450  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0460  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0470  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0480  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0490  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 04a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 04b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 04c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 04d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 04e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 04f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0500  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0510  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0520  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0530  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0540  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 0550  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>> 0560  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>> 0570  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>> 0580  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>> 0590  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>> 05a0  32 33 34 35 36 37 38 39                           23456789
>>             Data: 323334353637383930313233343536373839303132333435...
>>             [Length: 1448]
>>
>> On Mon, Jul 7, 2014 at 12:26 PM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>> The initial results are not promising: a MacOS 802.11ac client gets
>>> between 0-2 Mbps with this change, where it was getting about 8 Mbps
>>> prior to this change and ~170 Mbps prior to the reordering fix. A pcap
>>> from the receiving system shows a very large number of out of order
>>> frames, likely due to TCP retransmission.
>>>
>>> An 802.11n MacBook gets very good throughput, only the 802.11ac
>>> MacBook shows the very poor result. I'm trying to figure out why.
>>>
>>>
>>> One specific note (and probably not related to the throughput): I
>>> believe ath10k_htt_rx_amsdu runs in the tasklet, which means it would
>>> need to use GFP_ATOMIC instead of GFP_KERNEL. Do I understand it
>>> correctly?
>>>
>>>
>>> On Mon, Jul 7, 2014 at 1:30 AM, Janusz Dziedzic
>>> <janusz.dziedzic@tieto.com> wrote:
>>>> On 6 July 2014 04:27, Adrian Chadd <adrian@freebsd.org> wrote:
>>>>> I think you may have to tell mac80211 that it's okay and not to drop the frames.
>>>>>
>>>>> The earlier atheros chips would just give you the AMSDU frames as
>>>>> deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
>>>>> net80211 and it'd ull it apart. But if the driver is doing it (or,
>>>>> well, the chip is doing it) then mac80211 needs to know not to drop
>>>>> those sub-frames.
>>>>>
>>>>> I wonder if you'll ever get notified before the complete A-MPDU has
>>>>> been received. That happens on previous chips. eg, you have an A-MPDU
>>>>> of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
>>>>> half of one MPDU before you hit EOL, the next notification you process
>>>>> will be the next MSDU in the same MPDU - and then you'll hit the
>>>>> reordering hilarity again.
>>>>>
>>>>> So hm, i will face the same issue in FreeBSD at some point, so I'd
>>>>> likely do what you're thinking of doing - pass up a chain of mbufs
>>>>> representing the current MPDU and treat the whole lot as the frame(s)
>>>>> to care about. Honestly though, I'm kind of wondering whether I should
>>>>> mostly just do what the Atheros reference driver does and treat it as
>>>>> ethernet payload frames (ie, it's already de-encapsulated and the
>>>>> reordering is already done) and just tack on the wifi header bit via
>>>>> another of the DMA rings.
>>>>>
>>>>> Ugh, I really should sit down and write the FreeBSD version of this.
>>>>>
>>>>>
>>>>>
>>>>> -a
>>>>>
>>>>> (I'm still having flashbacks from working on this firmware at QCA. Aiee.)
>>>>>
>>>>>
>>>>> On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>> There are two issues in handling the dis-aggregated A-MSDU subframes
>>>>>> in ieee80211_sta_manage_reorder_buf:
>>>>>>
>>>>>> 1. The out-of-date check:
>>>>>>
>>>>>>         /* frame with out of date sequence number */
>>>>>>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>>>>>>                 dev_kfree_skb(skb);
>>>>>>                 goto out;
>>>>>>         }
>>>>>>
>>>>>> As all of the subframes carry the same sequence number, the first
>>>>>> subframe will be delivered and increment head_seq_num and then all
>>>>>> subsequent subframes will be discarded.
>>>>>>
>>>>>>
>>>>>> 2. The duplicates check a bit later in the routine:
>>>>>>
>>>>>>         /* check if we already stored this frame */
>>>>>>         if (tid_agg_rx->reorder_buf[index]) {
>>>>>>                 dev_kfree_skb(skb);
>>>>>>                 goto out;
>>>>>>         }
>>>>>>
>>>>>> If there is enough packet loss that packets are queued in the reorder
>>>>>> buffer and not immediately released, then only the first subframe will
>>>>>> be stored. Subsequent subframes will be discarded as duplicates.
>>>>>>
>>>>>>
>>>>>> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
>>>>>> the reordering buffer changes, and dropped to about 8 Mbps with the
>>>>>> reorder buffer. Hacking around the out-of-date sequence number check
>>>>>> to allow all subframes to egress restores it to 170 Mbps.
>>>>>>
>>>>>> In the area where I'm testing there isn't enough 5 GHz noise to make
>>>>>> the duplicates-check issue happen very often. By adding a printk I can
>>>>>> see that it does happen, but it doesn't impact the throughput and I
>>>>>> can't report the impact of fixing it.
>>>>>>
>>>>>> ----
>>>>>>
>>>>>> I do wonder if both of these are symptoms of dis-aggregating the
>>>>>> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
>>>>>> at a time, and the ath10k A-MSDU case is sending it subframes instead.
>>>>>> Trying to make the ath10k code send up all of the subframes as a chain
>>>>>> of skbs didn't immediately work, but I do wonder if that would better
>>>>>> match the mac80211 expectations.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>> Yes, after some more testing it does look like an unfortunate
>>>>>>> interaction between the reorder buffer and A-MSDU. The disaggregated
>>>>>>> subframes all carry the same sequence number. The first subframe is
>>>>>>> released from the reorder buffer and increments the head_seq_num.
>>>>>>> Subsequent subframes are all discarded as being out of date.
>>>>>>>
>>>>>>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>>>>>>> head=0xb06
>>>>>>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>>>>>>> head=0xb0b
>>>>>>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>>>>>>> head=0xb10
>>>>>>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>>>>>>> head=0xb15
>>>>>>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>>>>>>> head=0xb1a
>>>>>>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>>>>>>> head=0xb1f
>>>>>>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>>>>>>> head=0xb24
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>>>>>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>>>>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>>>>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>>>>>>
>>>>>>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>>>>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>>>>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>>>>>>> second is consistently lost. The MacBook generally retransmits the
>>>>>>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>>>>>>> frame makes it through.
>>>>>>>>>
>>>>>>>>> This became more noticeable after the reordering fixes in
>>>>>>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>>>>>>
>>>>>>>>> I see this A-MSDU packet loss behavior both with and without the
>>>>>>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>>>>>>> second is dropped. However, *without* the reordering fixes (and
>>>>>>>>> therefore with packets delivered out of order) the MacBook sends
>>>>>>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>>>>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>>>>>>> therefore has to deal with more packet loss. I suspect it is an
>>>>>>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>>>>>>> never going to fully understand, its stuck in a region of the
>>>>>>>>> congestion window where the Wifi driver keeps choosing to using
>>>>>>>>> A-MSDU.
>>>>>>>>
>>>>>>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>>>>>>
>>>>>>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>>>>>>> copying/allocation overhead each subframe is reported as a singly
>>>>>>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>>>>>>> A-MPDU reordering intereferes with A-MSDU now?
>>>>>>>>
>>>> Denton could you try attached patch: report amsdu in one big frame.
>>>> If help, we can add amsdu skb list support to mac80211/cfg80211 - to
>>>> improve performance and reduce memcpy, while currently we have skb
>>>> frames, put them in one big skb and next cfg80211 split them again
>>>> into msdus and report to stack.
>>>>
>>>> BR
>>>> Janusz

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Janusz Dziedzic]

On 8 July 2014 09:02, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:
> On 8 July 2014 08:50, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:
>> On 8 July 2014 08:43, Denton Gentry <denton.gentry@gmail.com> wrote:
>>> I think I know what is happening now, though I've no idea why. The
>>> throughput is low because we have many TCP retransmissions. We have
>>> retransmissions because the TCP checksum is wrong on a number of
>>> frames, and I do find data corruption in the payload so the checksum
>>> definitely should be wrong. All of the corrupted frames were
>>> originally one of the subframes in an A-MSDU packet.
>>>
>>> An example follows at the end of this message, as dissected by
>>> Wireshark. iperf sends a very regular data pattern of "0123456789..."
>>> over and over. Note how in subframe #2 offset 0x1e0 the bytes "32 33
>>> 34" have been replaced by "72 36 35"
>>>
>>> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
>>>
>>> I added printks at the bottom of ath10k_htt_rx_amsdu immediately
>>> before the call to ath10k_process_rx. I found this same packet, and we
>>> see the "72 36 35" corruption in the printk. So I think it happened in
>>> ath10k_process_rx or before, not anything weird after passing it up to
>>> mac80211.
>>>
>>> [  101.863712] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
>>> [  101.863727] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
>>> [  101.863742] ath10k: 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32
>>> [  101.863757] ath10k: 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37 38
>>> [  101.863773] ath10k: 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34
>>> [  101.863788] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
>>> [  101.863803] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
>>>
>>>
>>> I've found a number of examples of similar corruption, always with
>>> between one and four bytes replaced.
>>>
>>> 35363738 -> e52c6e07
>>> 3435 -> b43f
>>> 3839 -> c238
>>> 31 -> 7f
>>> 3435 -> 7436
>>> 30 -> 50
>>> 3233 -> bc37
>>>
>> Seems this could be because of:
>>
>> + /* cfg80211 expect this padding */
>> + padding = (4 - (skb->len + sizeof(subframe_hdr))) & 0x3;
>> + skb_put(skb, padding);
>>
>
> BTW, when AP (ath10k) send TCP ACK - are this packets also AMSDU?
> I see in my case AP can send 3 x A-MSDU (with total size 304 bytes)
> small frames aggregated. Maybe your HW have problems with that.
> As I remember correctly someone some time ago report problems with
> MacBook pro retina but I am not sure this is the same, while no one
> tests the fix.
>
>>>
>>> The packet described above, dissected by Wireshark:
>>>
>>> No.     Time        Source                Destination
>>> Protocol Length Info
>>>    2235 18.953349   192.168.144.79        192.168.144.13        TCP
>>>   3112   52697 > 5001 [ACK] Seq=1391553 Ack=1 Win=131760 [TCP CHECKSUM
>>> INCORRECT] Len=1448 TSval=1298580657 TSecr=4294947481
>>>
>>> Frame 2235: 3112 bytes on wire (24896 bits), 3112 bytes captured (24896 bits)
>>>     Encapsulation type: IEEE 802.11 plus radiotap radio header (23)
>>>     Arrival Time: Jul  7, 2014 23:03:37.763365000 PDT
>>>     [Time shift for this packet: 0.000000000 seconds]
>>>     Epoch Time: 1404799417.763365000 seconds
>>>     [Time delta from previous captured frame: 0.003476000 seconds]
>>>     [Time delta from previous displayed frame: 0.515641000 seconds]
>>>     [Time since reference or first frame: 18.953349000 seconds]
>>>     Frame Number: 2235
>>>     Frame Length: 3112 bytes (24896 bits)
>>>     Capture Length: 3112 bytes (24896 bits)
>>>     [Frame is marked: False]
>>>     [Frame is ignored: False]
>>>     [Protocols in frame: radiotap:wlan:llc:ip:tcp:data:llc:ip:tcp:data]
>>>     [Coloring Rule Name: TCP]
>>>     [Coloring Rule String: tcp]
>>> Radiotap Header v0, Length 38
>>>     Header revision: 0
>>>     Header pad: 0
>>>     Header length: 38
>>>     Present flags
>>>         .... .... .... .... .... .... .... ...1 = TSFT: True
>>>         .... .... .... .... .... .... .... ..1. = Flags: True
>>>         .... .... .... .... .... .... .... .0.. = Rate: False
>>>         .... .... .... .... .... .... .... 1... = Channel: True
>>>         .... .... .... .... .... .... ...0 .... = FHSS: False
>>>         .... .... .... .... .... .... ..1. .... = dBm Antenna Signal: True
>>>         .... .... .... .... .... .... .0.. .... = dBm Antenna Noise: False
>>>         .... .... .... .... .... .... 0... .... = Lock Quality: False
>>>         .... .... .... .... .... ...0 .... .... = TX Attenuation: False
>>>         .... .... .... .... .... ..0. .... .... = dB TX Attenuation: False
>>>         .... .... .... .... .... .0.. .... .... = dBm TX Power: False
>>>         .... .... .... .... .... 1... .... .... = Antenna: True
>>>         .... .... .... .... ...0 .... .... .... = dB Antenna Signal: False
>>>         .... .... .... .... ..0. .... .... .... = dB Antenna Noise: False
>>>         .... .... .... .... .1.. .... .... .... = RX flags: True
>>>         .... .... .... .0.. .... .... .... .... = Channel+: False
>>>         .... .... .... 0... .... .... .... .... = HT information: False
>>>         .... .... ...0 .... .... .... .... .... = A-MPDU Status: False
>>>         .... .... ..1. .... .... .... .... .... = VHT information: True
>>>         ...0 0000 00.. .... .... .... .... .... = Reserved: 0x00000000
>>>         ..0. .... .... .... .... .... .... .... = Radiotap NS next: False
>>>         .0.. .... .... .... .... .... .... .... = Vendor NS next: False
>>>         0... .... .... .... .... .... .... .... = Ext: False
>>>     MAC timestamp: 78051063
>>>     Flags: 0x00
>>>         .... ...0 = CFP: False
>>>         .... ..0. = Preamble: Long
>>>         .... .0.. = WEP: False
>>>         .... 0... = Fragmentation: False
>>>         ...0 .... = FCS at end: False
>>>         ..0. .... = Data Pad: False
>>>         .0.. .... = Bad FCS: False
>>>         0... .... = Short GI: False
>>>     Channel frequency: 5745 [A 149]
>>>     Channel type: 802.11a (0x0140)
>>>         .... .... ...0 .... = Turbo: False
>>>         .... .... ..0. .... = Complementary Code Keying (CCK): False
>>>         .... .... .1.. .... = Orthogonal Frequency-Division
>>> Multiplexing (OFDM): True
>>>         .... .... 0... .... = 2 GHz spectrum: False
>>>         .... ...1 .... .... = 5 GHz spectrum: True
>>>         .... ..0. .... .... = Passive: False
>>>         .... .0.. .... .... = Dynamic CCK-OFDM: False
>>>         .... 0... .... .... = Gaussian Frequency Shift Keying (GFSK): False
>>>         ...0 .... .... .... = GSM (900MHz): False
>>>         ..0. .... .... .... = Static Turbo: False
>>>         .0.. .... .... .... = Half Rate Channel (10MHz Channel Width): False
>>>         0... .... .... .... = Quarter Rate Channel (5MHz Channel Width): False
>>>     SSI Signal: -53 dBm
>>>     Antenna: 0
>>>     RX flags: 0x0000
>>>         .... .... .... .... .... ..0. = Bad PLCP: False
>>>     VHT information
>>>         Known VHT information: 0x44
>>>             .... .... .... ...0 = STBC: False
>>>             .... .... .... ..0. = TXOP_PS_NOT_ALLOWED: False
>>>             .... .... .... .1.. = Guard interval: True
>>>             .... .... .... 0... = SGI Nsym disambiguation: False
>>>             .... .... ...0 .... = LDPC extra OFDM symbol: False
>>>             .... .... ..0. .... = Beamformed: False
>>>             .... .... .1.. .... = Bandwidth: True
>>>             .... .... 0... .... = Group ID: False
>>>             .... ...0 .... .... = Partial AID: False
>>>         .... .0.. = Guard interval: long (0)
>>>         Bandwidth: 80 MHz (4)
>>>         User 0: MCS 8
>>>             1000 .... = MCS index 0: 8 (256-QAM 3/4)
>>>             .... 0010 = Spatial streams 0: 2
>>>             Space-time streams 0: 2
>>>             Coding 0: BCC (0)
>>>             [Data Rate: 702.0 Mb/s]
>>> IEEE 802.11 QoS Data, Flags: .......T
>>>     Type/Subtype: QoS Data (0x28)
>>>     Frame Control Field: 0x8801
>>>         .... ..00 = Version: 0
>>>         .... 10.. = Type: Data frame (2)
>>>         1000 .... = Subtype: 8
>>>         Flags: 0x01
>>>             .... ..01 = DS status: Frame from STA to DS via an AP (To
>>> DS: 1 From DS: 0) (0x01)
>>>             .... .0.. = More Fragments: This is the last fragment
>>>             .... 0... = Retry: Frame is not being retransmitted
>>>             ...0 .... = PWR MGT: STA will stay up
>>>             ..0. .... = More Data: No data buffered
>>>             .0.. .... = Protected flag: Data is not protected
>>>             0... .... = Order flag: Not strictly ordered
>>>     .000 0000 0011 0000 = Duration: 48 microseconds
>>>     Receiver address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>     BSS Id: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>     Transmitter address: Apple_67:24:54 (84:38:35:67:24:54)
>>>     Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>     Destination address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>     Fragment number: 0
>>>     Sequence number: 1021
>>>     Qos Control: 0x0080
>>>         .... .... .... 0000 = TID: 0
>>>         [.... .... .... .000 = Priority: Best Effort (Best Effort) (0)]
>>>         .... .... ...0 .... = QoS bit 4: Bits 8-15 of QoS Control
>>> field are TXOP Duration Requested
>>>         .... .... .00. .... = Ack Policy: Normal Ack (0x0000)
>>>         .... .... 1... .... = Payload Type: A-MSDU
>>>         0000 0000 .... .... = TXOP Duration Requested: 0 (no TXOP requested)
>>> IEEE 802.11 Aggregate MSDU
>>>     A-MSDU Subframe #1
>>>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>>>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>         A-MSDU Length: 1510
>>>         Logical-Link Control
>>>             DSAP: SNAP (0xaa)
>>>             IG Bit: Individual
>>>             SSAP: SNAP (0xaa)
>>>             CR Bit: Command
>>>             Control field: U, func=UI (0x03)
>>>                 000. 00.. = Command: Unnumbered Information (0x00)
>>>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>>>             Organization Code: Encapsulated Ethernet (0x000000)
>>>             Type: IP (0x0800)
>>>         Internet Protocol Version 4, Src: 192.168.144.79
>>> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>>>             Version: 4
>>>             Header length: 20 bytes
>>>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
>>> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>>>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>>>                 .... ..00 = Explicit Congestion Notification: Not-ECT
>>> (Not ECN-Capable Transport) (0x00)
>>>             Total Length: 1500
>>>             Identification: 0xc622 (50722)
>>>             Flags: 0x00
>>>                 0... .... = Reserved bit: Not set
>>>                 .0.. .... = Don't fragment: Not set
>>>                 ..0. .... = More fragments: Not set
>>>             Fragment offset: 0
>>>             Time to live: 64
>>>             Protocol: TCP (6)
>>>             Header checksum: 0x0d4c [correct]
>>>                 [Good: True]
>>>                 [Bad: False]
>>>             Source: 192.168.144.79 (192.168.144.79)
>>>             Destination: 192.168.144.13 (192.168.144.13)
>>>             [Source GeoIP: Unknown]
>>>             [Destination GeoIP: Unknown]
>>>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
>>> Port: 5001 (5001), Seq: 1390105, Ack: 1, Len: 1448
>>>             Source port: 52697 (52697)
>>>             Destination port: 5001 (5001)
>>>             [Stream index: 0]
>>>             Sequence number: 1390105    (relative sequence number)
>>>             [Next sequence number: 1391553    (relative sequence number)]
>>>             Acknowledgment number: 1    (relative ack number)
>>>             Header length: 32 bytes
>>>             Flags: 0x010 (ACK)
>>>                 000. .... .... = Reserved: Not set
>>>                 ...0 .... .... = Nonce: Not set
>>>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>>>                 .... .0.. .... = ECN-Echo: Not set
>>>                 .... ..0. .... = Urgent: Not set
>>>                 .... ...1 .... = Acknowledgment: Set
>>>                 .... .... 0... = Push: Not set
>>>                 .... .... .0.. = Reset: Not set
>>>                 .... .... ..0. = Syn: Not set
>>>                 .... .... ...0 = Fin: Not set
>>>             Window size value: 8235
>>>             [Calculated window size: 131760]
>>>             [Window size scaling factor: 16]
>>>             Checksum: 0xa1c0 [correct]
>>>                 [Good Checksum: True]
>>>                 [Bad Checksum: False]
>>>             Options: (12 bytes), No-Operation (NOP), No-Operation
>>> (NOP), Timestamps
>>>                 No-Operation (NOP)
>>>                     Type: 1
>>>                         0... .... = Copy on fragmentation: No
>>>                         .00. .... = Class: Control (0)
>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>                 No-Operation (NOP)
>>>                     Type: 1
>>>                         0... .... = Copy on fragmentation: No
>>>                         .00. .... = Class: Control (0)
>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>                 Timestamps: TSval 1298580657, TSecr 4294947481
>>>                     Kind: Timestamp (8)
>>>                     Length: 10
>>>                     Timestamp value: 1298580657
>>>                     Timestamp echo reply: 4294947481
>>>         Data (1448 bytes)
>>>
>>> 0000  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0010  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0020  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0030  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0040  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0050  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0060  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0070  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0080  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0090  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 00a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 00b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 00c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 00d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 00e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 00f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0100  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0110  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0120  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0130  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0140  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0150  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0160  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0170  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0180  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0190  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 01a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 01b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 01c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 01d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 01e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 01f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0200  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0210  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0220  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0230  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0240  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0250  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0260  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0270  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0280  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0290  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 02a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 02b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 02c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 02d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 02e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 02f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0300  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0310  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0320  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0330  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0340  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0350  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0360  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0370  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0380  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0390  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 03a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 03b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 03c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 03d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 03e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 03f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0400  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0410  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0420  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0430  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0440  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0450  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0460  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0470  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0480  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0490  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 04a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 04b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 04c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 04d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 04e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 04f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0500  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0510  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0520  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0530  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0540  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0550  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0560  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0570  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0580  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0590  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 05a0  34 35 36 37 38 39 30 31                           45678901
>>>             Data: 343536373839303132333435363738393031323334353637...
>>>             [Length: 1448]
>>>     A-MSDU Subframe #2
>>>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>>>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>         A-MSDU Length: 1510
>>>         Logical-Link Control
>>>             DSAP: SNAP (0xaa)
>>>             IG Bit: Individual
>>>             SSAP: SNAP (0xaa)
>>>             CR Bit: Command
>>>             Control field: U, func=UI (0x03)
>>>                 000. 00.. = Command: Unnumbered Information (0x00)
>>>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>>>             Organization Code: Encapsulated Ethernet (0x000000)
>>>             Type: IP (0x0800)
>>>         Internet Protocol Version 4, Src: 192.168.144.79
>>> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>>>             Version: 4
>>>             Header length: 20 bytes
>>>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
>>> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>>>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>>>                 .... ..00 = Explicit Congestion Notification: Not-ECT
>>> (Not ECN-Capable Transport) (0x00)
>>>             Total Length: 1500
>>>             Identification: 0xda09 (55817)
>>>             Flags: 0x00
>>>                 0... .... = Reserved bit: Not set
>>>                 .0.. .... = Don't fragment: Not set
>>>                 ..0. .... = More fragments: Not set
>>>             Fragment offset: 0
>>>             Time to live: 64
>>>             Protocol: TCP (6)
>>>             Header checksum: 0xf964 [correct]
>>>                 [Good: True]
>>>                 [Bad: False]
>>>             Source: 192.168.144.79 (192.168.144.79)
>>>             Destination: 192.168.144.13 (192.168.144.13)
>>>             [Source GeoIP: Unknown]
>>>             [Destination GeoIP: Unknown]
>>>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
>>> Port: 5001 (5001), Seq: 1391553, Ack: 1, Len: 1448
>>>             Source port: 52697 (52697)
>>>             Destination port: 5001 (5001)
>>>             [Stream index: 0]
>>>             Sequence number: 1391553    (relative sequence number)
>>>             [Next sequence number: 1393001    (relative sequence number)]
>>>             Acknowledgment number: 1    (relative ack number)
>>>             Header length: 32 bytes
>>>             Flags: 0x010 (ACK)
>>>                 000. .... .... = Reserved: Not set
>>>                 ...0 .... .... = Nonce: Not set
>>>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>>>                 .... .0.. .... = ECN-Echo: Not set
>>>                 .... ..0. .... = Urgent: Not set
>>>                 .... ...1 .... = Acknowledgment: Set
>>>                 .... .... 0... = Push: Not set
>>>                 .... .... .0.. = Reset: Not set
>>>                 .... .... ..0. = Syn: Not set
>>>                 .... .... ...0 = Fin: Not set
>>>             Window size value: 8235
>>>             [Calculated window size: 131760]
>>>             [Window size scaling factor: 16]
>>>             Checksum: 0x9a16 [incorrect, should be 0x5913 (maybe
>>> caused by "TCP checksum offload"?)]

And yes we are using checksum offload in ath10k.
Best is using standalone 80211ac sniffer for that case to be sure
about checksum.

>>>                 [Good Checksum: False]
>>>                 [Bad Checksum: True]
>>>                     [Expert Info (Error/Checksum): Bad checksum]
>>>                         [Message: Bad checksum]
>>>                         [Severity level: Error]
>>>                         [Group: Checksum]
>>>             Options: (12 bytes), No-Operation (NOP), No-Operation
>>> (NOP), Timestamps
>>>                 No-Operation (NOP)
>>>                     Type: 1
>>>                         0... .... = Copy on fragmentation: No
>>>                         .00. .... = Class: Control (0)
>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>                 No-Operation (NOP)
>>>                     Type: 1
>>>                         0... .... = Copy on fragmentation: No
>>>                         .00. .... = Class: Control (0)
>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>                 Timestamps: TSval 1298580657, TSecr 4294947481
>>>                     Kind: Timestamp (8)
>>>                     Length: 10
>>>                     Timestamp value: 1298580657
>>>                     Timestamp echo reply: 4294947481
>>>         Data (1448 bytes)
>>>
>>> 0000  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0010  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0020  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0030  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0040  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0050  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0060  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0070  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0080  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0090  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 00a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 00b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 00c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 00d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 00e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 00f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0100  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0110  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0120  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0130  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0140  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0150  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0160  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0170  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0180  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0190  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 01a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 01b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 01c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 01d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
>>> 01f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0200  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0210  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0220  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0230  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0240  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0250  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0260  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0270  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0280  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0290  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 02a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 02b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 02c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 02d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 02e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 02f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0300  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0310  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0320  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0330  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0340  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0350  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0360  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0370  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0380  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0390  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 03a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 03b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 03c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 03d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 03e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 03f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0400  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0410  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0420  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0430  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0440  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0450  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0460  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0470  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0480  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0490  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 04a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 04b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 04c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 04d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 04e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 04f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0500  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0510  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0520  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0530  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0540  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 0550  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>> 0560  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>> 0570  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>> 0580  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>> 0590  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>> 05a0  32 33 34 35 36 37 38 39                           23456789
>>>             Data: 323334353637383930313233343536373839303132333435...
>>>             [Length: 1448]
>>>
>>> On Mon, Jul 7, 2014 at 12:26 PM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>> The initial results are not promising: a MacOS 802.11ac client gets
>>>> between 0-2 Mbps with this change, where it was getting about 8 Mbps
>>>> prior to this change and ~170 Mbps prior to the reordering fix. A pcap
>>>> from the receiving system shows a very large number of out of order
>>>> frames, likely due to TCP retransmission.
>>>>
>>>> An 802.11n MacBook gets very good throughput, only the 802.11ac
>>>> MacBook shows the very poor result. I'm trying to figure out why.
>>>>
>>>>
>>>> One specific note (and probably not related to the throughput): I
>>>> believe ath10k_htt_rx_amsdu runs in the tasklet, which means it would
>>>> need to use GFP_ATOMIC instead of GFP_KERNEL. Do I understand it
>>>> correctly?
>>>>
>>>>
>>>> On Mon, Jul 7, 2014 at 1:30 AM, Janusz Dziedzic
>>>> <janusz.dziedzic@tieto.com> wrote:
>>>>> On 6 July 2014 04:27, Adrian Chadd <adrian@freebsd.org> wrote:
>>>>>> I think you may have to tell mac80211 that it's okay and not to drop the frames.
>>>>>>
>>>>>> The earlier atheros chips would just give you the AMSDU frames as
>>>>>> deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
>>>>>> net80211 and it'd ull it apart. But if the driver is doing it (or,
>>>>>> well, the chip is doing it) then mac80211 needs to know not to drop
>>>>>> those sub-frames.
>>>>>>
>>>>>> I wonder if you'll ever get notified before the complete A-MPDU has
>>>>>> been received. That happens on previous chips. eg, you have an A-MPDU
>>>>>> of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
>>>>>> half of one MPDU before you hit EOL, the next notification you process
>>>>>> will be the next MSDU in the same MPDU - and then you'll hit the
>>>>>> reordering hilarity again.
>>>>>>
>>>>>> So hm, i will face the same issue in FreeBSD at some point, so I'd
>>>>>> likely do what you're thinking of doing - pass up a chain of mbufs
>>>>>> representing the current MPDU and treat the whole lot as the frame(s)
>>>>>> to care about. Honestly though, I'm kind of wondering whether I should
>>>>>> mostly just do what the Atheros reference driver does and treat it as
>>>>>> ethernet payload frames (ie, it's already de-encapsulated and the
>>>>>> reordering is already done) and just tack on the wifi header bit via
>>>>>> another of the DMA rings.
>>>>>>
>>>>>> Ugh, I really should sit down and write the FreeBSD version of this.
>>>>>>
>>>>>>
>>>>>>
>>>>>> -a
>>>>>>
>>>>>> (I'm still having flashbacks from working on this firmware at QCA. Aiee.)
>>>>>>
>>>>>>
>>>>>> On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>> There are two issues in handling the dis-aggregated A-MSDU subframes
>>>>>>> in ieee80211_sta_manage_reorder_buf:
>>>>>>>
>>>>>>> 1. The out-of-date check:
>>>>>>>
>>>>>>>         /* frame with out of date sequence number */
>>>>>>>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>>>>>>>                 dev_kfree_skb(skb);
>>>>>>>                 goto out;
>>>>>>>         }
>>>>>>>
>>>>>>> As all of the subframes carry the same sequence number, the first
>>>>>>> subframe will be delivered and increment head_seq_num and then all
>>>>>>> subsequent subframes will be discarded.
>>>>>>>
>>>>>>>
>>>>>>> 2. The duplicates check a bit later in the routine:
>>>>>>>
>>>>>>>         /* check if we already stored this frame */
>>>>>>>         if (tid_agg_rx->reorder_buf[index]) {
>>>>>>>                 dev_kfree_skb(skb);
>>>>>>>                 goto out;
>>>>>>>         }
>>>>>>>
>>>>>>> If there is enough packet loss that packets are queued in the reorder
>>>>>>> buffer and not immediately released, then only the first subframe will
>>>>>>> be stored. Subsequent subframes will be discarded as duplicates.
>>>>>>>
>>>>>>>
>>>>>>> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
>>>>>>> the reordering buffer changes, and dropped to about 8 Mbps with the
>>>>>>> reorder buffer. Hacking around the out-of-date sequence number check
>>>>>>> to allow all subframes to egress restores it to 170 Mbps.
>>>>>>>
>>>>>>> In the area where I'm testing there isn't enough 5 GHz noise to make
>>>>>>> the duplicates-check issue happen very often. By adding a printk I can
>>>>>>> see that it does happen, but it doesn't impact the throughput and I
>>>>>>> can't report the impact of fixing it.
>>>>>>>
>>>>>>> ----
>>>>>>>
>>>>>>> I do wonder if both of these are symptoms of dis-aggregating the
>>>>>>> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
>>>>>>> at a time, and the ath10k A-MSDU case is sending it subframes instead.
>>>>>>> Trying to make the ath10k code send up all of the subframes as a chain
>>>>>>> of skbs didn't immediately work, but I do wonder if that would better
>>>>>>> match the mac80211 expectations.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>> Yes, after some more testing it does look like an unfortunate
>>>>>>>> interaction between the reorder buffer and A-MSDU. The disaggregated
>>>>>>>> subframes all carry the same sequence number. The first subframe is
>>>>>>>> released from the reorder buffer and increments the head_seq_num.
>>>>>>>> Subsequent subframes are all discarded as being out of date.
>>>>>>>>
>>>>>>>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>>>>>>>> head=0xb06
>>>>>>>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>>>>>>>> head=0xb0b
>>>>>>>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>>>>>>>> head=0xb10
>>>>>>>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>>>>>>>> head=0xb15
>>>>>>>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>>>>>>>> head=0xb1a
>>>>>>>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>>>>>>>> head=0xb1f
>>>>>>>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>>>>>>>> head=0xb24
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>>>>>>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>>>>>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>>>>>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>>>>>>>
>>>>>>>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>>>>>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>>>>>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>>>>>>>> second is consistently lost. The MacBook generally retransmits the
>>>>>>>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>>>>>>>> frame makes it through.
>>>>>>>>>>
>>>>>>>>>> This became more noticeable after the reordering fixes in
>>>>>>>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>>>>>>>
>>>>>>>>>> I see this A-MSDU packet loss behavior both with and without the
>>>>>>>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>>>>>>>> second is dropped. However, *without* the reordering fixes (and
>>>>>>>>>> therefore with packets delivered out of order) the MacBook sends
>>>>>>>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>>>>>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>>>>>>>> therefore has to deal with more packet loss. I suspect it is an
>>>>>>>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>>>>>>>> never going to fully understand, its stuck in a region of the
>>>>>>>>>> congestion window where the Wifi driver keeps choosing to using
>>>>>>>>>> A-MSDU.
>>>>>>>>>
>>>>>>>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>>>>>>>
>>>>>>>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>>>>>>>> copying/allocation overhead each subframe is reported as a singly
>>>>>>>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>>>>>>>> A-MPDU reordering intereferes with A-MSDU now?
>>>>>>>>>
>>>>> Denton could you try attached patch: report amsdu in one big frame.
>>>>> If help, we can add amsdu skb list support to mac80211/cfg80211 - to
>>>>> improve performance and reduce memcpy, while currently we have skb
>>>>> frames, put them in one big skb and next cfg80211 split them again
>>>>> into msdus and report to stack.
>>>>>
>>>>> BR
>>>>> Janusz

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Denton Gentry]

[-- Attachment #1: Type: text/plain, Size: 44500 bytes --]

I ran iperf on an 802.11ac MacBook Air, through my ath10 AP, and to an
Ubuntu system connected to the AP via Ethernet. The AP was running the
reorder buffer patches, the patch to make A-MSDU bypass the reorder
buffer, and the patch to make amsdu aggregation configurable via
debugfs.

http://lists.infradead.org/pipermail/ath10k/2014-June/002551.html
http://lists.infradead.org/pipermail/ath10k/2014-June/002553.html
http://lists.infradead.org/pipermail/ath10k/2014-July/002597.html
http://permalink.gmane.org/gmane.linux.kernel.wireless.general/124128

It was notably not running the patch from earlier in this thread to
copy A-MSDU subframes into one skb.


I used a second ath10k AP configured in monitor mode as a Wifi
sniffer. I also gathered pcaps on the MacBook Air and on the Ubuntu
system. Wireshark on the MacBook cannot gather radiotap headers nor
put the interface into monitor mode, so we see only Ethernet frames
and we don't see retransmissions on the Wifi link.

I've attached decodes of the same two packets as captured by the
MacBook, by the sniffer, and by the Ubuntu server.

The MacBook sends two frames, TCP sequence numbers 3065441 and 3066889
(the Wireshark "relative sequence number"). In the Wifi sniffer we see
these being aggregated as two subframes in an A-MSDU frame. The first
time this A-MSDU is sent it is corrupted in the air. We see the TCP
checksum of one of the subframes is wrong, and a few bytes of the
payload replaced with "&gt%'g9(1$".

The A-MSDU is retransmitted a short time later, and the second time
the TCP checksum of both subframes is correct.

This is all fine so far.

However in the pcap taken from the Ubuntu server, we see TCP sequence
number 3065441 being delivered *twice.* The first time, the TCP
checksum is wrong. The second time the TCP checksum is correct.

To me, it looks like A-MSDU frames with bad FCS are not being
discarded after ieee80211_rx_monitor(). The corrupted frames are being
delivered. Delivering the corrupted frames results in sending more TCP
Dup ACKs for the same sequence number back to the MacBook, and I think
this is what causes the MacBook to decide there is congestion and slow
down.

One note: the wifi sniffer does not show the same corruption to the
same packet as the pcap from the Ubuntu system shows. I think that is
normal: the sniffer won't see precisely the same noise, won't have
precisely the same receive sensitivity, and its antennas are not
pointing in the same direction as the primary AP. If I do this test
again, I'll gather a pcap on the primary AP as well to compare to what
we see on the Ubuntu system.


On Tue, Jul 8, 2014 at 12:29 AM, Janusz Dziedzic
<janusz.dziedzic@tieto.com> wrote:
> On 8 July 2014 09:02, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:
>> On 8 July 2014 08:50, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:
>>> On 8 July 2014 08:43, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>> I think I know what is happening now, though I've no idea why. The
>>>> throughput is low because we have many TCP retransmissions. We have
>>>> retransmissions because the TCP checksum is wrong on a number of
>>>> frames, and I do find data corruption in the payload so the checksum
>>>> definitely should be wrong. All of the corrupted frames were
>>>> originally one of the subframes in an A-MSDU packet.
>>>>
>>>> An example follows at the end of this message, as dissected by
>>>> Wireshark. iperf sends a very regular data pattern of "0123456789..."
>>>> over and over. Note how in subframe #2 offset 0x1e0 the bytes "32 33
>>>> 34" have been replaced by "72 36 35"
>>>>
>>>> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
>>>>
>>>> I added printks at the bottom of ath10k_htt_rx_amsdu immediately
>>>> before the call to ath10k_process_rx. I found this same packet, and we
>>>> see the "72 36 35" corruption in the printk. So I think it happened in
>>>> ath10k_process_rx or before, not anything weird after passing it up to
>>>> mac80211.
>>>>
>>>> [  101.863712] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
>>>> [  101.863727] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
>>>> [  101.863742] ath10k: 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32
>>>> [  101.863757] ath10k: 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37 38
>>>> [  101.863773] ath10k: 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34
>>>> [  101.863788] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
>>>> [  101.863803] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
>>>>
>>>>
>>>> I've found a number of examples of similar corruption, always with
>>>> between one and four bytes replaced.
>>>>
>>>> 35363738 -> e52c6e07
>>>> 3435 -> b43f
>>>> 3839 -> c238
>>>> 31 -> 7f
>>>> 3435 -> 7436
>>>> 30 -> 50
>>>> 3233 -> bc37
>>>>
>>> Seems this could be because of:
>>>
>>> + /* cfg80211 expect this padding */
>>> + padding = (4 - (skb->len + sizeof(subframe_hdr))) & 0x3;
>>> + skb_put(skb, padding);
>>>
>>
>> BTW, when AP (ath10k) send TCP ACK - are this packets also AMSDU?
>> I see in my case AP can send 3 x A-MSDU (with total size 304 bytes)
>> small frames aggregated. Maybe your HW have problems with that.
>> As I remember correctly someone some time ago report problems with
>> MacBook pro retina but I am not sure this is the same, while no one
>> tests the fix.
>>
>>>>
>>>> The packet described above, dissected by Wireshark:
>>>>
>>>> No.     Time        Source                Destination
>>>> Protocol Length Info
>>>>    2235 18.953349   192.168.144.79        192.168.144.13        TCP
>>>>   3112   52697 > 5001 [ACK] Seq=1391553 Ack=1 Win=131760 [TCP CHECKSUM
>>>> INCORRECT] Len=1448 TSval=1298580657 TSecr=4294947481
>>>>
>>>> Frame 2235: 3112 bytes on wire (24896 bits), 3112 bytes captured (24896 bits)
>>>>     Encapsulation type: IEEE 802.11 plus radiotap radio header (23)
>>>>     Arrival Time: Jul  7, 2014 23:03:37.763365000 PDT
>>>>     [Time shift for this packet: 0.000000000 seconds]
>>>>     Epoch Time: 1404799417.763365000 seconds
>>>>     [Time delta from previous captured frame: 0.003476000 seconds]
>>>>     [Time delta from previous displayed frame: 0.515641000 seconds]
>>>>     [Time since reference or first frame: 18.953349000 seconds]
>>>>     Frame Number: 2235
>>>>     Frame Length: 3112 bytes (24896 bits)
>>>>     Capture Length: 3112 bytes (24896 bits)
>>>>     [Frame is marked: False]
>>>>     [Frame is ignored: False]
>>>>     [Protocols in frame: radiotap:wlan:llc:ip:tcp:data:llc:ip:tcp:data]
>>>>     [Coloring Rule Name: TCP]
>>>>     [Coloring Rule String: tcp]
>>>> Radiotap Header v0, Length 38
>>>>     Header revision: 0
>>>>     Header pad: 0
>>>>     Header length: 38
>>>>     Present flags
>>>>         .... .... .... .... .... .... .... ...1 = TSFT: True
>>>>         .... .... .... .... .... .... .... ..1. = Flags: True
>>>>         .... .... .... .... .... .... .... .0.. = Rate: False
>>>>         .... .... .... .... .... .... .... 1... = Channel: True
>>>>         .... .... .... .... .... .... ...0 .... = FHSS: False
>>>>         .... .... .... .... .... .... ..1. .... = dBm Antenna Signal: True
>>>>         .... .... .... .... .... .... .0.. .... = dBm Antenna Noise: False
>>>>         .... .... .... .... .... .... 0... .... = Lock Quality: False
>>>>         .... .... .... .... .... ...0 .... .... = TX Attenuation: False
>>>>         .... .... .... .... .... ..0. .... .... = dB TX Attenuation: False
>>>>         .... .... .... .... .... .0.. .... .... = dBm TX Power: False
>>>>         .... .... .... .... .... 1... .... .... = Antenna: True
>>>>         .... .... .... .... ...0 .... .... .... = dB Antenna Signal: False
>>>>         .... .... .... .... ..0. .... .... .... = dB Antenna Noise: False
>>>>         .... .... .... .... .1.. .... .... .... = RX flags: True
>>>>         .... .... .... .0.. .... .... .... .... = Channel+: False
>>>>         .... .... .... 0... .... .... .... .... = HT information: False
>>>>         .... .... ...0 .... .... .... .... .... = A-MPDU Status: False
>>>>         .... .... ..1. .... .... .... .... .... = VHT information: True
>>>>         ...0 0000 00.. .... .... .... .... .... = Reserved: 0x00000000
>>>>         ..0. .... .... .... .... .... .... .... = Radiotap NS next: False
>>>>         .0.. .... .... .... .... .... .... .... = Vendor NS next: False
>>>>         0... .... .... .... .... .... .... .... = Ext: False
>>>>     MAC timestamp: 78051063
>>>>     Flags: 0x00
>>>>         .... ...0 = CFP: False
>>>>         .... ..0. = Preamble: Long
>>>>         .... .0.. = WEP: False
>>>>         .... 0... = Fragmentation: False
>>>>         ...0 .... = FCS at end: False
>>>>         ..0. .... = Data Pad: False
>>>>         .0.. .... = Bad FCS: False
>>>>         0... .... = Short GI: False
>>>>     Channel frequency: 5745 [A 149]
>>>>     Channel type: 802.11a (0x0140)
>>>>         .... .... ...0 .... = Turbo: False
>>>>         .... .... ..0. .... = Complementary Code Keying (CCK): False
>>>>         .... .... .1.. .... = Orthogonal Frequency-Division
>>>> Multiplexing (OFDM): True
>>>>         .... .... 0... .... = 2 GHz spectrum: False
>>>>         .... ...1 .... .... = 5 GHz spectrum: True
>>>>         .... ..0. .... .... = Passive: False
>>>>         .... .0.. .... .... = Dynamic CCK-OFDM: False
>>>>         .... 0... .... .... = Gaussian Frequency Shift Keying (GFSK): False
>>>>         ...0 .... .... .... = GSM (900MHz): False
>>>>         ..0. .... .... .... = Static Turbo: False
>>>>         .0.. .... .... .... = Half Rate Channel (10MHz Channel Width): False
>>>>         0... .... .... .... = Quarter Rate Channel (5MHz Channel Width): False
>>>>     SSI Signal: -53 dBm
>>>>     Antenna: 0
>>>>     RX flags: 0x0000
>>>>         .... .... .... .... .... ..0. = Bad PLCP: False
>>>>     VHT information
>>>>         Known VHT information: 0x44
>>>>             .... .... .... ...0 = STBC: False
>>>>             .... .... .... ..0. = TXOP_PS_NOT_ALLOWED: False
>>>>             .... .... .... .1.. = Guard interval: True
>>>>             .... .... .... 0... = SGI Nsym disambiguation: False
>>>>             .... .... ...0 .... = LDPC extra OFDM symbol: False
>>>>             .... .... ..0. .... = Beamformed: False
>>>>             .... .... .1.. .... = Bandwidth: True
>>>>             .... .... 0... .... = Group ID: False
>>>>             .... ...0 .... .... = Partial AID: False
>>>>         .... .0.. = Guard interval: long (0)
>>>>         Bandwidth: 80 MHz (4)
>>>>         User 0: MCS 8
>>>>             1000 .... = MCS index 0: 8 (256-QAM 3/4)
>>>>             .... 0010 = Spatial streams 0: 2
>>>>             Space-time streams 0: 2
>>>>             Coding 0: BCC (0)
>>>>             [Data Rate: 702.0 Mb/s]
>>>> IEEE 802.11 QoS Data, Flags: .......T
>>>>     Type/Subtype: QoS Data (0x28)
>>>>     Frame Control Field: 0x8801
>>>>         .... ..00 = Version: 0
>>>>         .... 10.. = Type: Data frame (2)
>>>>         1000 .... = Subtype: 8
>>>>         Flags: 0x01
>>>>             .... ..01 = DS status: Frame from STA to DS via an AP (To
>>>> DS: 1 From DS: 0) (0x01)
>>>>             .... .0.. = More Fragments: This is the last fragment
>>>>             .... 0... = Retry: Frame is not being retransmitted
>>>>             ...0 .... = PWR MGT: STA will stay up
>>>>             ..0. .... = More Data: No data buffered
>>>>             .0.. .... = Protected flag: Data is not protected
>>>>             0... .... = Order flag: Not strictly ordered
>>>>     .000 0000 0011 0000 = Duration: 48 microseconds
>>>>     Receiver address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>>     BSS Id: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>>     Transmitter address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>     Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>     Destination address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>>     Fragment number: 0
>>>>     Sequence number: 1021
>>>>     Qos Control: 0x0080
>>>>         .... .... .... 0000 = TID: 0
>>>>         [.... .... .... .000 = Priority: Best Effort (Best Effort) (0)]
>>>>         .... .... ...0 .... = QoS bit 4: Bits 8-15 of QoS Control
>>>> field are TXOP Duration Requested
>>>>         .... .... .00. .... = Ack Policy: Normal Ack (0x0000)
>>>>         .... .... 1... .... = Payload Type: A-MSDU
>>>>         0000 0000 .... .... = TXOP Duration Requested: 0 (no TXOP requested)
>>>> IEEE 802.11 Aggregate MSDU
>>>>     A-MSDU Subframe #1
>>>>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>>>>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>         A-MSDU Length: 1510
>>>>         Logical-Link Control
>>>>             DSAP: SNAP (0xaa)
>>>>             IG Bit: Individual
>>>>             SSAP: SNAP (0xaa)
>>>>             CR Bit: Command
>>>>             Control field: U, func=UI (0x03)
>>>>                 000. 00.. = Command: Unnumbered Information (0x00)
>>>>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>>>>             Organization Code: Encapsulated Ethernet (0x000000)
>>>>             Type: IP (0x0800)
>>>>         Internet Protocol Version 4, Src: 192.168.144.79
>>>> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>>>>             Version: 4
>>>>             Header length: 20 bytes
>>>>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
>>>> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>>>>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>>>>                 .... ..00 = Explicit Congestion Notification: Not-ECT
>>>> (Not ECN-Capable Transport) (0x00)
>>>>             Total Length: 1500
>>>>             Identification: 0xc622 (50722)
>>>>             Flags: 0x00
>>>>                 0... .... = Reserved bit: Not set
>>>>                 .0.. .... = Don't fragment: Not set
>>>>                 ..0. .... = More fragments: Not set
>>>>             Fragment offset: 0
>>>>             Time to live: 64
>>>>             Protocol: TCP (6)
>>>>             Header checksum: 0x0d4c [correct]
>>>>                 [Good: True]
>>>>                 [Bad: False]
>>>>             Source: 192.168.144.79 (192.168.144.79)
>>>>             Destination: 192.168.144.13 (192.168.144.13)
>>>>             [Source GeoIP: Unknown]
>>>>             [Destination GeoIP: Unknown]
>>>>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
>>>> Port: 5001 (5001), Seq: 1390105, Ack: 1, Len: 1448
>>>>             Source port: 52697 (52697)
>>>>             Destination port: 5001 (5001)
>>>>             [Stream index: 0]
>>>>             Sequence number: 1390105    (relative sequence number)
>>>>             [Next sequence number: 1391553    (relative sequence number)]
>>>>             Acknowledgment number: 1    (relative ack number)
>>>>             Header length: 32 bytes
>>>>             Flags: 0x010 (ACK)
>>>>                 000. .... .... = Reserved: Not set
>>>>                 ...0 .... .... = Nonce: Not set
>>>>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>>>>                 .... .0.. .... = ECN-Echo: Not set
>>>>                 .... ..0. .... = Urgent: Not set
>>>>                 .... ...1 .... = Acknowledgment: Set
>>>>                 .... .... 0... = Push: Not set
>>>>                 .... .... .0.. = Reset: Not set
>>>>                 .... .... ..0. = Syn: Not set
>>>>                 .... .... ...0 = Fin: Not set
>>>>             Window size value: 8235
>>>>             [Calculated window size: 131760]
>>>>             [Window size scaling factor: 16]
>>>>             Checksum: 0xa1c0 [correct]
>>>>                 [Good Checksum: True]
>>>>                 [Bad Checksum: False]
>>>>             Options: (12 bytes), No-Operation (NOP), No-Operation
>>>> (NOP), Timestamps
>>>>                 No-Operation (NOP)
>>>>                     Type: 1
>>>>                         0... .... = Copy on fragmentation: No
>>>>                         .00. .... = Class: Control (0)
>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>                 No-Operation (NOP)
>>>>                     Type: 1
>>>>                         0... .... = Copy on fragmentation: No
>>>>                         .00. .... = Class: Control (0)
>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>                 Timestamps: TSval 1298580657, TSecr 4294947481
>>>>                     Kind: Timestamp (8)
>>>>                     Length: 10
>>>>                     Timestamp value: 1298580657
>>>>                     Timestamp echo reply: 4294947481
>>>>         Data (1448 bytes)
>>>>
>>>> 0000  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0010  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0020  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0030  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0040  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0050  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0060  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0070  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0080  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0090  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 00a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 00b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 00c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 00d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 00e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 00f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0100  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0110  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0120  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0130  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0140  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0150  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0160  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0170  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0180  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0190  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 01a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 01b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 01c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 01d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 01e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 01f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0200  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0210  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0220  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0230  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0240  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0250  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0260  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0270  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0280  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0290  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 02a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 02b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 02c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 02d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 02e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 02f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0300  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0310  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0320  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0330  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0340  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0350  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0360  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0370  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0380  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0390  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 03a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 03b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 03c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 03d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 03e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 03f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0400  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0410  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0420  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0430  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0440  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0450  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0460  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0470  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0480  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0490  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 04a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 04b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 04c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 04d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 04e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 04f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0500  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0510  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0520  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0530  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0540  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0550  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0560  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0570  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0580  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0590  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 05a0  34 35 36 37 38 39 30 31                           45678901
>>>>             Data: 343536373839303132333435363738393031323334353637...
>>>>             [Length: 1448]
>>>>     A-MSDU Subframe #2
>>>>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>>>>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>         A-MSDU Length: 1510
>>>>         Logical-Link Control
>>>>             DSAP: SNAP (0xaa)
>>>>             IG Bit: Individual
>>>>             SSAP: SNAP (0xaa)
>>>>             CR Bit: Command
>>>>             Control field: U, func=UI (0x03)
>>>>                 000. 00.. = Command: Unnumbered Information (0x00)
>>>>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>>>>             Organization Code: Encapsulated Ethernet (0x000000)
>>>>             Type: IP (0x0800)
>>>>         Internet Protocol Version 4, Src: 192.168.144.79
>>>> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>>>>             Version: 4
>>>>             Header length: 20 bytes
>>>>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
>>>> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>>>>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>>>>                 .... ..00 = Explicit Congestion Notification: Not-ECT
>>>> (Not ECN-Capable Transport) (0x00)
>>>>             Total Length: 1500
>>>>             Identification: 0xda09 (55817)
>>>>             Flags: 0x00
>>>>                 0... .... = Reserved bit: Not set
>>>>                 .0.. .... = Don't fragment: Not set
>>>>                 ..0. .... = More fragments: Not set
>>>>             Fragment offset: 0
>>>>             Time to live: 64
>>>>             Protocol: TCP (6)
>>>>             Header checksum: 0xf964 [correct]
>>>>                 [Good: True]
>>>>                 [Bad: False]
>>>>             Source: 192.168.144.79 (192.168.144.79)
>>>>             Destination: 192.168.144.13 (192.168.144.13)
>>>>             [Source GeoIP: Unknown]
>>>>             [Destination GeoIP: Unknown]
>>>>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
>>>> Port: 5001 (5001), Seq: 1391553, Ack: 1, Len: 1448
>>>>             Source port: 52697 (52697)
>>>>             Destination port: 5001 (5001)
>>>>             [Stream index: 0]
>>>>             Sequence number: 1391553    (relative sequence number)
>>>>             [Next sequence number: 1393001    (relative sequence number)]
>>>>             Acknowledgment number: 1    (relative ack number)
>>>>             Header length: 32 bytes
>>>>             Flags: 0x010 (ACK)
>>>>                 000. .... .... = Reserved: Not set
>>>>                 ...0 .... .... = Nonce: Not set
>>>>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>>>>                 .... .0.. .... = ECN-Echo: Not set
>>>>                 .... ..0. .... = Urgent: Not set
>>>>                 .... ...1 .... = Acknowledgment: Set
>>>>                 .... .... 0... = Push: Not set
>>>>                 .... .... .0.. = Reset: Not set
>>>>                 .... .... ..0. = Syn: Not set
>>>>                 .... .... ...0 = Fin: Not set
>>>>             Window size value: 8235
>>>>             [Calculated window size: 131760]
>>>>             [Window size scaling factor: 16]
>>>>             Checksum: 0x9a16 [incorrect, should be 0x5913 (maybe
>>>> caused by "TCP checksum offload"?)]
>
> And yes we are using checksum offload in ath10k.
> Best is using standalone 80211ac sniffer for that case to be sure
> about checksum.
>
>>>>                 [Good Checksum: False]
>>>>                 [Bad Checksum: True]
>>>>                     [Expert Info (Error/Checksum): Bad checksum]
>>>>                         [Message: Bad checksum]
>>>>                         [Severity level: Error]
>>>>                         [Group: Checksum]
>>>>             Options: (12 bytes), No-Operation (NOP), No-Operation
>>>> (NOP), Timestamps
>>>>                 No-Operation (NOP)
>>>>                     Type: 1
>>>>                         0... .... = Copy on fragmentation: No
>>>>                         .00. .... = Class: Control (0)
>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>                 No-Operation (NOP)
>>>>                     Type: 1
>>>>                         0... .... = Copy on fragmentation: No
>>>>                         .00. .... = Class: Control (0)
>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>                 Timestamps: TSval 1298580657, TSecr 4294947481
>>>>                     Kind: Timestamp (8)
>>>>                     Length: 10
>>>>                     Timestamp value: 1298580657
>>>>                     Timestamp echo reply: 4294947481
>>>>         Data (1448 bytes)
>>>>
>>>> 0000  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0010  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0020  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0030  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0040  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0050  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0060  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0070  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0080  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0090  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 00a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 00b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 00c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 00d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 00e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 00f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0100  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0110  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0120  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0130  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0140  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0150  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0160  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0170  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0180  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0190  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 01a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 01b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 01c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 01d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
>>>> 01f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0200  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0210  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0220  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0230  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0240  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0250  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0260  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0270  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0280  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0290  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 02a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 02b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 02c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 02d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 02e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 02f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0300  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0310  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0320  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0330  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0340  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0350  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0360  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0370  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0380  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0390  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 03a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 03b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 03c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 03d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 03e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 03f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0400  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0410  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0420  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0430  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0440  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0450  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0460  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0470  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0480  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0490  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 04a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 04b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 04c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 04d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 04e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 04f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0500  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0510  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0520  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0530  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0540  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 0550  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>> 0560  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>> 0570  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>> 0580  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>> 0590  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>> 05a0  32 33 34 35 36 37 38 39                           23456789
>>>>             Data: 323334353637383930313233343536373839303132333435...
>>>>             [Length: 1448]
>>>>
>>>> On Mon, Jul 7, 2014 at 12:26 PM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>> The initial results are not promising: a MacOS 802.11ac client gets
>>>>> between 0-2 Mbps with this change, where it was getting about 8 Mbps
>>>>> prior to this change and ~170 Mbps prior to the reordering fix. A pcap
>>>>> from the receiving system shows a very large number of out of order
>>>>> frames, likely due to TCP retransmission.
>>>>>
>>>>> An 802.11n MacBook gets very good throughput, only the 802.11ac
>>>>> MacBook shows the very poor result. I'm trying to figure out why.
>>>>>
>>>>>
>>>>> One specific note (and probably not related to the throughput): I
>>>>> believe ath10k_htt_rx_amsdu runs in the tasklet, which means it would
>>>>> need to use GFP_ATOMIC instead of GFP_KERNEL. Do I understand it
>>>>> correctly?
>>>>>
>>>>>
>>>>> On Mon, Jul 7, 2014 at 1:30 AM, Janusz Dziedzic
>>>>> <janusz.dziedzic@tieto.com> wrote:
>>>>>> On 6 July 2014 04:27, Adrian Chadd <adrian@freebsd.org> wrote:
>>>>>>> I think you may have to tell mac80211 that it's okay and not to drop the frames.
>>>>>>>
>>>>>>> The earlier atheros chips would just give you the AMSDU frames as
>>>>>>> deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
>>>>>>> net80211 and it'd ull it apart. But if the driver is doing it (or,
>>>>>>> well, the chip is doing it) then mac80211 needs to know not to drop
>>>>>>> those sub-frames.
>>>>>>>
>>>>>>> I wonder if you'll ever get notified before the complete A-MPDU has
>>>>>>> been received. That happens on previous chips. eg, you have an A-MPDU
>>>>>>> of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
>>>>>>> half of one MPDU before you hit EOL, the next notification you process
>>>>>>> will be the next MSDU in the same MPDU - and then you'll hit the
>>>>>>> reordering hilarity again.
>>>>>>>
>>>>>>> So hm, i will face the same issue in FreeBSD at some point, so I'd
>>>>>>> likely do what you're thinking of doing - pass up a chain of mbufs
>>>>>>> representing the current MPDU and treat the whole lot as the frame(s)
>>>>>>> to care about. Honestly though, I'm kind of wondering whether I should
>>>>>>> mostly just do what the Atheros reference driver does and treat it as
>>>>>>> ethernet payload frames (ie, it's already de-encapsulated and the
>>>>>>> reordering is already done) and just tack on the wifi header bit via
>>>>>>> another of the DMA rings.
>>>>>>>
>>>>>>> Ugh, I really should sit down and write the FreeBSD version of this.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -a
>>>>>>>
>>>>>>> (I'm still having flashbacks from working on this firmware at QCA. Aiee.)
>>>>>>>
>>>>>>>
>>>>>>> On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>> There are two issues in handling the dis-aggregated A-MSDU subframes
>>>>>>>> in ieee80211_sta_manage_reorder_buf:
>>>>>>>>
>>>>>>>> 1. The out-of-date check:
>>>>>>>>
>>>>>>>>         /* frame with out of date sequence number */
>>>>>>>>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>>>>>>>>                 dev_kfree_skb(skb);
>>>>>>>>                 goto out;
>>>>>>>>         }
>>>>>>>>
>>>>>>>> As all of the subframes carry the same sequence number, the first
>>>>>>>> subframe will be delivered and increment head_seq_num and then all
>>>>>>>> subsequent subframes will be discarded.
>>>>>>>>
>>>>>>>>
>>>>>>>> 2. The duplicates check a bit later in the routine:
>>>>>>>>
>>>>>>>>         /* check if we already stored this frame */
>>>>>>>>         if (tid_agg_rx->reorder_buf[index]) {
>>>>>>>>                 dev_kfree_skb(skb);
>>>>>>>>                 goto out;
>>>>>>>>         }
>>>>>>>>
>>>>>>>> If there is enough packet loss that packets are queued in the reorder
>>>>>>>> buffer and not immediately released, then only the first subframe will
>>>>>>>> be stored. Subsequent subframes will be discarded as duplicates.
>>>>>>>>
>>>>>>>>
>>>>>>>> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
>>>>>>>> the reordering buffer changes, and dropped to about 8 Mbps with the
>>>>>>>> reorder buffer. Hacking around the out-of-date sequence number check
>>>>>>>> to allow all subframes to egress restores it to 170 Mbps.
>>>>>>>>
>>>>>>>> In the area where I'm testing there isn't enough 5 GHz noise to make
>>>>>>>> the duplicates-check issue happen very often. By adding a printk I can
>>>>>>>> see that it does happen, but it doesn't impact the throughput and I
>>>>>>>> can't report the impact of fixing it.
>>>>>>>>
>>>>>>>> ----
>>>>>>>>
>>>>>>>> I do wonder if both of these are symptoms of dis-aggregating the
>>>>>>>> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
>>>>>>>> at a time, and the ath10k A-MSDU case is sending it subframes instead.
>>>>>>>> Trying to make the ath10k code send up all of the subframes as a chain
>>>>>>>> of skbs didn't immediately work, but I do wonder if that would better
>>>>>>>> match the mac80211 expectations.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>> Yes, after some more testing it does look like an unfortunate
>>>>>>>>> interaction between the reorder buffer and A-MSDU. The disaggregated
>>>>>>>>> subframes all carry the same sequence number. The first subframe is
>>>>>>>>> released from the reorder buffer and increments the head_seq_num.
>>>>>>>>> Subsequent subframes are all discarded as being out of date.
>>>>>>>>>
>>>>>>>>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>>>>>>>>> head=0xb06
>>>>>>>>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>>>>>>>>> head=0xb0b
>>>>>>>>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>>>>>>>>> head=0xb10
>>>>>>>>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>>>>>>>>> head=0xb15
>>>>>>>>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>>>>>>>>> head=0xb1a
>>>>>>>>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>>>>>>>>> head=0xb1f
>>>>>>>>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>>>>>>>>> head=0xb24
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>>>>>>>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>>>>>>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>>>>>>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>>>>>>>>
>>>>>>>>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>>>>>>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>>>>>>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>>>>>>>>> second is consistently lost. The MacBook generally retransmits the
>>>>>>>>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>>>>>>>>> frame makes it through.
>>>>>>>>>>>
>>>>>>>>>>> This became more noticeable after the reordering fixes in
>>>>>>>>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>>>>>>>>
>>>>>>>>>>> I see this A-MSDU packet loss behavior both with and without the
>>>>>>>>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>>>>>>>>> second is dropped. However, *without* the reordering fixes (and
>>>>>>>>>>> therefore with packets delivered out of order) the MacBook sends
>>>>>>>>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>>>>>>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>>>>>>>>> therefore has to deal with more packet loss. I suspect it is an
>>>>>>>>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>>>>>>>>> never going to fully understand, its stuck in a region of the
>>>>>>>>>>> congestion window where the Wifi driver keeps choosing to using
>>>>>>>>>>> A-MSDU.
>>>>>>>>>>
>>>>>>>>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>>>>>>>>
>>>>>>>>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>>>>>>>>> copying/allocation overhead each subframe is reported as a singly
>>>>>>>>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>>>>>>>>> A-MPDU reordering intereferes with A-MSDU now?
>>>>>>>>>>
>>>>>> Denton could you try attached patch: report amsdu in one big frame.
>>>>>> If help, we can add amsdu skb list support to mac80211/cfg80211 - to
>>>>>> improve performance and reduce memcpy, while currently we have skb
>>>>>> frames, put them in one big skb and next cfg80211 split them again
>>>>>> into msdus and report to stack.
>>>>>>
>>>>>> BR
>>>>>> Janusz

[-- Attachment #2: macbook.txt --]
[-- Type: text/plain, Size: 22301 bytes --]

No.     Time           Source                Destination           Protocol Length Info
   3793 5.784865000    192.168.254.52        192.168.254.1         TCP      1514   52734 > 5001 [ACK] Seq=3065441 Ack=1 Win=131760 Len=1448 TSval=1302514011 TSecr=328746582

Frame 3793: 1514 bytes on wire (12112 bits), 1514 bytes captured (12112 bits) on interface 0
    Interface id: 0
    Encapsulation type: Ethernet (1)
    Arrival Time: Jul  8, 2014 11:24:38.171552000 PDT
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1404843878.171552000 seconds
    [Time delta from previous captured frame: 0.000001000 seconds]
    [Time delta from previous displayed frame: 0.000001000 seconds]
    [Time since reference or first frame: 5.784865000 seconds]
    Frame Number: 3793
    Frame Length: 1514 bytes (12112 bits)
    Capture Length: 1514 bytes (12112 bits)
    [Frame is marked: True]
    [Frame is ignored: False]
    [Protocols in frame: eth:ip:tcp:data]
    [Coloring Rule Name: TCP]
    [Coloring Rule String: tcp]
Ethernet II, Src: Apple_67:24:54 (84:38:35:67:24:54), Dst: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
    Destination: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
        Address: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Source: Apple_67:24:54 (84:38:35:67:24:54)
        Address: Apple_67:24:54 (84:38:35:67:24:54)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: IP (0x0800)
Internet Protocol Version 4, Src: 192.168.254.52 (192.168.254.52), Dst: 192.168.254.1 (192.168.254.1)
    Version: 4
    Header length: 20 bytes
    Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
        0000 00.. = Differentiated Services Codepoint: Default (0x00)
        .... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
    Total Length: 1500
    Identification: 0xa03d (41021)
    Flags: 0x02 (Don't Fragment)
        0... .... = Reserved bit: Not set
        .1.. .... = Don't fragment: Set
        ..0. .... = More fragments: Not set
    Fragment offset: 0
    Time to live: 64
    Protocol: TCP (6)
    Header checksum: 0x1757 [correct]
        [Good: True]
        [Bad: False]
    Source: 192.168.254.52 (192.168.254.52)
    Destination: 192.168.254.1 (192.168.254.1)
    [Source GeoIP: Unknown]
    [Destination GeoIP: Unknown]
Transmission Control Protocol, Src Port: 52734 (52734), Dst Port: 5001 (5001), Seq: 3065441, Ack: 1, Len: 1448
    Source port: 52734 (52734)
    Destination port: 5001 (5001)
    [Stream index: 0]
    Sequence number: 3065441    (relative sequence number)
    [Next sequence number: 3066889    (relative sequence number)]
    Acknowledgment number: 1    (relative ack number)
    Header length: 32 bytes
    Flags: 0x010 (ACK)
        000. .... .... = Reserved: Not set
        ...0 .... .... = Nonce: Not set
        .... 0... .... = Congestion Window Reduced (CWR): Not set
        .... .0.. .... = ECN-Echo: Not set
        .... ..0. .... = Urgent: Not set
        .... ...1 .... = Acknowledgment: Set
        .... .... 0... = Push: Not set
        .... .... .0.. = Reset: Not set
        .... .... ..0. = Syn: Not set
        .... .... ...0 = Fin: Not set
    Window size value: 8235
    [Calculated window size: 131760]
    [Window size scaling factor: 16]
    Checksum: 0x1b72 [correct]
        [Good Checksum: True]
        [Bad Checksum: False]
    Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
        No-Operation (NOP)
            Type: 1
                0... .... = Copy on fragmentation: No
                .00. .... = Class: Control (0)
                ...0 0001 = Number: No-Operation (NOP) (1)
        No-Operation (NOP)
            Type: 1
                0... .... = Copy on fragmentation: No
                .00. .... = Class: Control (0)
                ...0 0001 = Number: No-Operation (NOP) (1)
        Timestamps: TSval 1302514011, TSecr 328746582
            Kind: Timestamp (8)
            Length: 10
            Timestamp value: 1302514011
            Timestamp echo reply: 328746582
    [SEQ/ACK analysis]
        [Bytes in flight: 5792]
Data (1448 bytes)

0000  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0010  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0020  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0030  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0040  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0050  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0060  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0070  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0080  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0090  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
00a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
00b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
00c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
00d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
00e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
00f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0100  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0110  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0120  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0130  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0140  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0150  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0160  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0170  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0180  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0190  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
01a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
01b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
01c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
01d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
01e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
01f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0200  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0210  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0220  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0230  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0240  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0250  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0260  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0270  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0280  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0290  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
02a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
02b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
02c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
02d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
02e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
02f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0300  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0310  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0320  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0330  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0340  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0350  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0360  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0370  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0380  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0390  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
03a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
03b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
03c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
03d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
03e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
03f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0400  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0410  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0420  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0430  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0440  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0450  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0460  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0470  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0480  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0490  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
04a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
04b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
04c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
04d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
04e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
04f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0500  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0510  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0520  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0530  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0540  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0550  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0560  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0570  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0580  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0590  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
05a0  30 31 32 33 34 35 36 37                           01234567
    Data: 303132333435363738393031323334353637383930313233...
    [Length: 1448]

No.     Time           Source                Destination           Protocol Length Info
   3794 5.784894000    192.168.254.52        192.168.254.1         TCP      1514   52734 > 5001 [ACK] Seq=3066889 Ack=1 Win=131760 Len=1448 TSval=1302514011 TSecr=328746582

Frame 3794: 1514 bytes on wire (12112 bits), 1514 bytes captured (12112 bits) on interface 0
    Interface id: 0
    Encapsulation type: Ethernet (1)
    Arrival Time: Jul  8, 2014 11:24:38.171581000 PDT
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1404843878.171581000 seconds
    [Time delta from previous captured frame: 0.000029000 seconds]
    [Time delta from previous displayed frame: 0.000029000 seconds]
    [Time since reference or first frame: 5.784894000 seconds]
    Frame Number: 3794
    Frame Length: 1514 bytes (12112 bits)
    Capture Length: 1514 bytes (12112 bits)
    [Frame is marked: True]
    [Frame is ignored: False]
    [Protocols in frame: eth:ip:tcp:data]
    [Coloring Rule Name: TCP]
    [Coloring Rule String: tcp]
Ethernet II, Src: Apple_67:24:54 (84:38:35:67:24:54), Dst: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
    Destination: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
        Address: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Source: Apple_67:24:54 (84:38:35:67:24:54)
        Address: Apple_67:24:54 (84:38:35:67:24:54)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: IP (0x0800)
Internet Protocol Version 4, Src: 192.168.254.52 (192.168.254.52), Dst: 192.168.254.1 (192.168.254.1)
    Version: 4
    Header length: 20 bytes
    Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
        0000 00.. = Differentiated Services Codepoint: Default (0x00)
        .... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
    Total Length: 1500
    Identification: 0xc127 (49447)
    Flags: 0x02 (Don't Fragment)
        0... .... = Reserved bit: Not set
        .1.. .... = Don't fragment: Set
        ..0. .... = More fragments: Not set
    Fragment offset: 0
    Time to live: 64
    Protocol: TCP (6)
    Header checksum: 0xf66c [correct]
        [Good: True]
        [Bad: False]
    Source: 192.168.254.52 (192.168.254.52)
    Destination: 192.168.254.1 (192.168.254.1)
    [Source GeoIP: Unknown]
    [Destination GeoIP: Unknown]
Transmission Control Protocol, Src Port: 52734 (52734), Dst Port: 5001 (5001), Seq: 3066889, Ack: 1, Len: 1448
    Source port: 52734 (52734)
    Destination port: 5001 (5001)
    [Stream index: 0]
    Sequence number: 3066889    (relative sequence number)
    [Next sequence number: 3068337    (relative sequence number)]
    Acknowledgment number: 1    (relative ack number)
    Header length: 32 bytes
    Flags: 0x010 (ACK)
        000. .... .... = Reserved: Not set
        ...0 .... .... = Nonce: Not set
        .... 0... .... = Congestion Window Reduced (CWR): Not set
        .... .0.. .... = ECN-Echo: Not set
        .... ..0. .... = Urgent: Not set
        .... ...1 .... = Acknowledgment: Set
        .... .... 0... = Push: Not set
        .... .... .0.. = Reset: Not set
        .... .... ..0. = Syn: Not set
        .... .... ...0 = Fin: Not set
    Window size value: 8235
    [Calculated window size: 131760]
    [Window size scaling factor: 16]
    Checksum: 0x13c8 [correct]
        [Good Checksum: True]
        [Bad Checksum: False]
    Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
        No-Operation (NOP)
            Type: 1
                0... .... = Copy on fragmentation: No
                .00. .... = Class: Control (0)
                ...0 0001 = Number: No-Operation (NOP) (1)
        No-Operation (NOP)
            Type: 1
                0... .... = Copy on fragmentation: No
                .00. .... = Class: Control (0)
                ...0 0001 = Number: No-Operation (NOP) (1)
        Timestamps: TSval 1302514011, TSecr 328746582
            Kind: Timestamp (8)
            Length: 10
            Timestamp value: 1302514011
            Timestamp echo reply: 328746582
    [SEQ/ACK analysis]
        [Bytes in flight: 7240]
Data (1448 bytes)

0000  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0010  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0020  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0030  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0040  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0050  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0060  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0070  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0080  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0090  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
00a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
00b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
00c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
00d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
00e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
00f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0100  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0110  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0120  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0130  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0140  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0150  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0160  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0170  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0180  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0190  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
01a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
01b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
01c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
01d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
01e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
01f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0200  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0210  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0220  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0230  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0240  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0250  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0260  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0270  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0280  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0290  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
02a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
02b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
02c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
02d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
02e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
02f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0300  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0310  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0320  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0330  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0340  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0350  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0360  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0370  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0380  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0390  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
03a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
03b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
03c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
03d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
03e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
03f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0400  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0410  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0420  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0430  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0440  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0450  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0460  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0470  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0480  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0490  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
04a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
04b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
04c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
04d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
04e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
04f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0500  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0510  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0520  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0530  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0540  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0550  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0560  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0570  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0580  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0590  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
05a0  38 39 30 31 32 33 34 35                           89012345
    Data: 383930313233343536373839303132333435363738393031...
    [Length: 1448]

[-- Attachment #3: server.txt --]
[-- Type: text/plain, Size: 23196 bytes --]

No.     Time        Source                Destination           Protocol Length Info
   3886 4.594969    192.168.254.52        192.168.254.1         TCP      1514   [TCP Retransmission] 52734 > 5001 [ACK] Seq=3065441 Ack=1 Win=131760 [TCP CHECKSUM INCORRECT] Len=1448 TSval=1302514011 TSecr=328746582

Frame 3886: 1514 bytes on wire (12112 bits), 1514 bytes captured (12112 bits)
    Encapsulation type: Ethernet (1)
    Arrival Time: Jul  8, 2014 11:24:39.084614000 PDT
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1404843879.084614000 seconds
    [Time delta from previous captured frame: 0.000041000 seconds]
    [Time delta from previous displayed frame: 0.000041000 seconds]
    [Time since reference or first frame: 4.594969000 seconds]
    Frame Number: 3886
    Frame Length: 1514 bytes (12112 bits)
    Capture Length: 1514 bytes (12112 bits)
    [Frame is marked: True]
    [Frame is ignored: False]
    [Protocols in frame: eth:ip:tcp:data]
    [Coloring Rule Name: TCP]
    [Coloring Rule String: tcp]
Ethernet II, Src: Apple_67:24:54 (84:38:35:67:24:54), Dst: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
    Destination: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
        Address: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Source: Apple_67:24:54 (84:38:35:67:24:54)
        Address: Apple_67:24:54 (84:38:35:67:24:54)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: IP (0x0800)
Internet Protocol Version 4, Src: 192.168.254.52 (192.168.254.52), Dst: 192.168.254.1 (192.168.254.1)
    Version: 4
    Header length: 20 bytes
    Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
        0000 00.. = Differentiated Services Codepoint: Default (0x00)
        .... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
    Total Length: 1500
    Identification: 0xa03d (41021)
    Flags: 0x02 (Don't Fragment)
        0... .... = Reserved bit: Not set
        .1.. .... = Don't fragment: Set
        ..0. .... = More fragments: Not set
    Fragment offset: 0
    Time to live: 64
    Protocol: TCP (6)
    Header checksum: 0x1757 [correct]
        [Good: True]
        [Bad: False]
    Source: 192.168.254.52 (192.168.254.52)
    Destination: 192.168.254.1 (192.168.254.1)
    [Source GeoIP: Unknown]
    [Destination GeoIP: Unknown]
Transmission Control Protocol, Src Port: 52734 (52734), Dst Port: 5001 (5001), Seq: 3065441, Ack: 1, Len: 1448
    Source port: 52734 (52734)
    Destination port: 5001 (5001)
    [Stream index: 0]
    Sequence number: 3065441    (relative sequence number)
    [Next sequence number: 3066889    (relative sequence number)]
    Acknowledgment number: 1    (relative ack number)
    Header length: 32 bytes
    Flags: 0x010 (ACK)
        000. .... .... = Reserved: Not set
        ...0 .... .... = Nonce: Not set
        .... 0... .... = Congestion Window Reduced (CWR): Not set
        .... .0.. .... = ECN-Echo: Not set
        .... ..0. .... = Urgent: Not set
        .... ...1 .... = Acknowledgment: Set
        .... .... 0... = Push: Not set
        .... .... .0.. = Reset: Not set
        .... .... ..0. = Syn: Not set
        .... .... ...0 = Fin: Not set
    Window size value: 8235
    [Calculated window size: 131760]
    [Window size scaling factor: 16]
    Checksum: 0x1b72 [incorrect, should be 0x2ac9 (maybe caused by "TCP checksum offload"?)]
        [Good Checksum: False]
        [Bad Checksum: True]
            [Expert Info (Error/Checksum): Bad checksum]
                [Message: Bad checksum]
                [Severity level: Error]
                [Group: Checksum]
    Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
        No-Operation (NOP)
            Type: 1
                0... .... = Copy on fragmentation: No
                .00. .... = Class: Control (0)
                ...0 0001 = Number: No-Operation (NOP) (1)
        No-Operation (NOP)
            Type: 1
                0... .... = Copy on fragmentation: No
                .00. .... = Class: Control (0)
                ...0 0001 = Number: No-Operation (NOP) (1)
        Timestamps: TSval 1302514011, TSecr 328746582
            Kind: Timestamp (8)
            Length: 10
            Timestamp value: 1302514011
            Timestamp echo reply: 328746582
    [SEQ/ACK analysis]
        [Bytes in flight: 814161224]
        [TCP Analysis Flags]
            [This frame is a (suspected) retransmission]
                [Expert Info (Note/Sequence): Retransmission (suspected)]
                    [Message: Retransmission (suspected)]
                    [Severity level: Note]
                    [Group: Sequence]
            [The RTO for this segment was: 1.371819000 seconds]
            [RTO based on delta from frame: 1038]
Data (1448 bytes)

0000  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0010  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0020  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0030  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0040  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0050  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0060  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0070  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0080  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0090  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
00a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
00b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
00c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
00d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
00e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
00f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0100  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0110  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0120  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0130  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0140  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0150  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0160  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0170  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0180  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0190  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
01a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
01b0  32 33 34 35 36 37 38 39 9a 31 38 19 94 b7 36 37   23456789.18...67
01c0  38 39 30 31 32 33 34 35 36 37 38 39 30 91 92 b1   8901234567890...
01d0  9c 1f 36 37 38 39 30 31 32 33 34 35 36 37 38 39   ..67890123456789
01e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
01f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0200  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0210  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0220  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0230  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0240  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0250  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0260  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0270  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0280  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0290  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
02a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
02b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
02c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
02d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
02e0  36 37 38 39 30 31 32 33 20 61 23 22 3c 39 30 31   67890123 a#"<901
02f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0300  38 39 30 31 32 11 36 3f 9c 3d 3a b1 38 19 9a 99   89012.6?.=:.8...
0310  34 bd 14 9d b2 b3 98 b3 38 3b 16 b7 14 9d b2 1b   4.......8;......
0320  3a 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   :123456789012345
0330  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0340  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0350  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0360  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0370  30 31 32 33 34 35 36 37 38 39 30 31 32 33 74 30   01234567890123t0
0380  32 36 38 39 30 31 32 33 34 35 36 37 38 39 30 31   2689012345678901
0390  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
03a0  38 39 30 31 32 33 34 35 36 37 1a 3b 3a 9b 9a bb   8901234567.;:...
03b0  36 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   6567890123456789
03c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
03d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
03e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
03f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0400  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0410  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0420  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0430  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0440  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0450  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0460  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0470  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0480  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0490  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
04a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
04b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
04c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
04d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
04e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
04f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0500  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0510  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0520  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0530  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0540  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0550  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0560  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0570  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0580  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0590  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
05a0  30 31 32 33 34 35 36 37                           01234567
    Data: 303132333435363738393031323334353637383930313233...
    [Length: 1448]

No.     Time        Source                Destination           Protocol Length Info
   3891 4.595330    192.168.254.52        192.168.254.1         TCP      1514   [TCP Out-Of-Order] 52734 > 5001 [ACK] Seq=3065441 Ack=1 Win=131760 Len=1448 TSval=1302514011 TSecr=328746582

Frame 3891: 1514 bytes on wire (12112 bits), 1514 bytes captured (12112 bits)
    Encapsulation type: Ethernet (1)
    Arrival Time: Jul  8, 2014 11:24:39.084975000 PDT
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1404843879.084975000 seconds
    [Time delta from previous captured frame: 0.000006000 seconds]
    [Time delta from previous displayed frame: 0.000006000 seconds]
    [Time since reference or first frame: 4.595330000 seconds]
    Frame Number: 3891
    Frame Length: 1514 bytes (12112 bits)
    Capture Length: 1514 bytes (12112 bits)
    [Frame is marked: True]
    [Frame is ignored: False]
    [Protocols in frame: eth:ip:tcp:data]
    [Coloring Rule Name: TCP]
    [Coloring Rule String: tcp]
Ethernet II, Src: Apple_67:24:54 (84:38:35:67:24:54), Dst: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
    Destination: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
        Address: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Source: Apple_67:24:54 (84:38:35:67:24:54)
        Address: Apple_67:24:54 (84:38:35:67:24:54)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: IP (0x0800)
Internet Protocol Version 4, Src: 192.168.254.52 (192.168.254.52), Dst: 192.168.254.1 (192.168.254.1)
    Version: 4
    Header length: 20 bytes
    Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
        0000 00.. = Differentiated Services Codepoint: Default (0x00)
        .... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
    Total Length: 1500
    Identification: 0xa03d (41021)
    Flags: 0x02 (Don't Fragment)
        0... .... = Reserved bit: Not set
        .1.. .... = Don't fragment: Set
        ..0. .... = More fragments: Not set
    Fragment offset: 0
    Time to live: 64
    Protocol: TCP (6)
    Header checksum: 0x1757 [correct]
        [Good: True]
        [Bad: False]
    Source: 192.168.254.52 (192.168.254.52)
    Destination: 192.168.254.1 (192.168.254.1)
    [Source GeoIP: Unknown]
    [Destination GeoIP: Unknown]
Transmission Control Protocol, Src Port: 52734 (52734), Dst Port: 5001 (5001), Seq: 3065441, Ack: 1, Len: 1448
    Source port: 52734 (52734)
    Destination port: 5001 (5001)
    [Stream index: 0]
    Sequence number: 3065441    (relative sequence number)
    [Next sequence number: 3066889    (relative sequence number)]
    Acknowledgment number: 1    (relative ack number)
    Header length: 32 bytes
    Flags: 0x010 (ACK)
        000. .... .... = Reserved: Not set
        ...0 .... .... = Nonce: Not set
        .... 0... .... = Congestion Window Reduced (CWR): Not set
        .... .0.. .... = ECN-Echo: Not set
        .... ..0. .... = Urgent: Not set
        .... ...1 .... = Acknowledgment: Set
        .... .... 0... = Push: Not set
        .... .... .0.. = Reset: Not set
        .... .... ..0. = Syn: Not set
        .... .... ...0 = Fin: Not set
    Window size value: 8235
    [Calculated window size: 131760]
    [Window size scaling factor: 16]
    Checksum: 0x1b72 [correct]
        [Good Checksum: True]
        [Bad Checksum: False]
    Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
        No-Operation (NOP)
            Type: 1
                0... .... = Copy on fragmentation: No
                .00. .... = Class: Control (0)
                ...0 0001 = Number: No-Operation (NOP) (1)
        No-Operation (NOP)
            Type: 1
                0... .... = Copy on fragmentation: No
                .00. .... = Class: Control (0)
                ...0 0001 = Number: No-Operation (NOP) (1)
        Timestamps: TSval 1302514011, TSecr 328746582
            Kind: Timestamp (8)
            Length: 10
            Timestamp value: 1302514011
            Timestamp echo reply: 328746582
    [SEQ/ACK analysis]
        [TCP Analysis Flags]
            [This frame is a (suspected) out-of-order segment]
                [Expert Info (Warn/Sequence): Out-Of-Order segment]
                    [Message: Out-Of-Order segment]
                    [Severity level: Warn]
                    [Group: Sequence]
Data (1448 bytes)

0000  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0010  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0020  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0030  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0040  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0050  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0060  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0070  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0080  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0090  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
00a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
00b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
00c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
00d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
00e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
00f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0100  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0110  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0120  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0130  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0140  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0150  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0160  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0170  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0180  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0190  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
01a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
01b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
01c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
01d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
01e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
01f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0200  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0210  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0220  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0230  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0240  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0250  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0260  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0270  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0280  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0290  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
02a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
02b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
02c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
02d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
02e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
02f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0300  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0310  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0320  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0330  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0340  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0350  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0360  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0370  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0380  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0390  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
03a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
03b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
03c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
03d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
03e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
03f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0400  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0410  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0420  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0430  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0440  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0450  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0460  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0470  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0480  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0490  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
04a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
04b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
04c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
04d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
04e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
04f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0500  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0510  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0520  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0530  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0540  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0550  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0560  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0570  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0580  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0590  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
05a0  30 31 32 33 34 35 36 37                           01234567
    Data: 303132333435363738393031323334353637383930313233...
    [Length: 1448]

[-- Attachment #4: wifisniffer.txt --]
[-- Type: text/plain, Size: 55323 bytes --]

No.     Time        Source                Destination           Protocol Length Info
   5549 15.608617   192.168.254.52        192.168.254.1         TCP      3110   52734 > 5001 [ACK] Seq=3066889 Ack=1 Win=131760 [TCP CHECKSUM INCORRECT] Len=1448 TSval=1302514011 TSecr=328746582

Frame 5549: 3110 bytes on wire (24880 bits), 3110 bytes captured (24880 bits)
    Encapsulation type: IEEE 802.11 plus radiotap radio header (23)
    Arrival Time: Jul  8, 2014 11:24:39.082371000 PDT
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1404843879.082371000 seconds
    [Time delta from previous captured frame: 0.000079000 seconds]
    [Time delta from previous displayed frame: 0.000079000 seconds]
    [Time since reference or first frame: 15.608617000 seconds]
    Frame Number: 5549
    Frame Length: 3110 bytes (24880 bits)
    Capture Length: 3110 bytes (24880 bits)
    [Frame is marked: True]
    [Frame is ignored: False]
    [Protocols in frame: radiotap:wlan:llc:ip:tcp:data:llc:ip:tcp:data]
    [Coloring Rule Name: TCP]
    [Coloring Rule String: tcp]
Radiotap Header v0, Length 38
    Header revision: 0
    Header pad: 0
    Header length: 38
    Present flags
        .... .... .... .... .... .... .... ...1 = TSFT: True
        .... .... .... .... .... .... .... ..1. = Flags: True
        .... .... .... .... .... .... .... .0.. = Rate: False
        .... .... .... .... .... .... .... 1... = Channel: True
        .... .... .... .... .... .... ...0 .... = FHSS: False
        .... .... .... .... .... .... ..1. .... = dBm Antenna Signal: True
        .... .... .... .... .... .... .0.. .... = dBm Antenna Noise: False
        .... .... .... .... .... .... 0... .... = Lock Quality: False
        .... .... .... .... .... ...0 .... .... = TX Attenuation: False
        .... .... .... .... .... ..0. .... .... = dB TX Attenuation: False
        .... .... .... .... .... .0.. .... .... = dBm TX Power: False
        .... .... .... .... .... 1... .... .... = Antenna: True
        .... .... .... .... ...0 .... .... .... = dB Antenna Signal: False
        .... .... .... .... ..0. .... .... .... = dB Antenna Noise: False
        .... .... .... .... .1.. .... .... .... = RX flags: True
        .... .... .... .0.. .... .... .... .... = Channel+: False
        .... .... .... 0... .... .... .... .... = HT information: False
        .... .... ...0 .... .... .... .... .... = A-MPDU Status: False
        .... .... ..1. .... .... .... .... .... = VHT information: True
        ...0 0000 00.. .... .... .... .... .... = Reserved: 0x00000000
        ..0. .... .... .... .... .... .... .... = Radiotap NS next: False
        .0.. .... .... .... .... .... .... .... = Vendor NS next: False
        0... .... .... .... .... .... .... .... = Ext: False
    MAC timestamp: 1395542308
    Flags: 0x40
        .... ...0 = CFP: False
        .... ..0. = Preamble: Long
        .... .0.. = WEP: False
        .... 0... = Fragmentation: False
        ...0 .... = FCS at end: False
        ..0. .... = Data Pad: False
        .1.. .... = Bad FCS: True
        0... .... = Short GI: False
    Channel frequency: 5745 [A 149]
    Channel type: 802.11a (0x0140)
        .... .... ...0 .... = Turbo: False
        .... .... ..0. .... = Complementary Code Keying (CCK): False
        .... .... .1.. .... = Orthogonal Frequency-Division Multiplexing (OFDM): True
        .... .... 0... .... = 2 GHz spectrum: False
        .... ...1 .... .... = 5 GHz spectrum: True
        .... ..0. .... .... = Passive: False
        .... .0.. .... .... = Dynamic CCK-OFDM: False
        .... 0... .... .... = Gaussian Frequency Shift Keying (GFSK): False
        ...0 .... .... .... = GSM (900MHz): False
        ..0. .... .... .... = Static Turbo: False
        .0.. .... .... .... = Half Rate Channel (10MHz Channel Width): False
        0... .... .... .... = Quarter Rate Channel (5MHz Channel Width): False
    SSI Signal: -46 dBm
    Antenna: 0
    RX flags: 0x0000
        .... .... .... .... .... ..0. = Bad PLCP: False
    VHT information
        Known VHT information: 0x44
            .... .... .... ...0 = STBC: False
            .... .... .... ..0. = TXOP_PS_NOT_ALLOWED: False
            .... .... .... .1.. = Guard interval: True
            .... .... .... 0... = SGI Nsym disambiguation: False
            .... .... ...0 .... = LDPC extra OFDM symbol: False
            .... .... ..0. .... = Beamformed: False
            .... .... .1.. .... = Bandwidth: True
            .... .... 0... .... = Group ID: False
            .... ...0 .... .... = Partial AID: False
        .... .0.. = Guard interval: long (0)
        Bandwidth: 80 MHz (4)
        User 0: MCS 9
            1001 .... = MCS index 0: 9 (256-QAM 5/6)
            .... 0010 = Spatial streams 0: 2
            Space-time streams 0: 2
            Coding 0: BCC (0)
            [Data Rate: 780.0 Mb/s]
IEEE 802.11 QoS Data, Flags: .......T
    Type/Subtype: QoS Data (0x28)
    Frame Control Field: 0x8801
        .... ..00 = Version: 0
        .... 10.. = Type: Data frame (2)
        1000 .... = Subtype: 8
        Flags: 0x01
            .... ..01 = DS status: Frame from STA to DS via an AP (To DS: 1 From DS: 0) (0x01)
            .... .0.. = More Fragments: This is the last fragment
            .... 0... = Retry: Frame is not being retransmitted
            ...0 .... = PWR MGT: STA will stay up
            ..0. .... = More Data: No data buffered
            .0.. .... = Protected flag: Data is not protected
            0... .... = Order flag: Not strictly ordered
    .000 0000 0011 0000 = Duration: 48 microseconds
    Receiver address: SenaoNet_18:a7:d0 (88:dc:96:18:a7:d0)
    BSS Id: SenaoNet_18:a7:d0 (88:dc:96:18:a7:d0)
    Transmitter address: Apple_67:24:54 (84:38:35:67:24:54)
    Source address: Apple_67:24:54 (84:38:35:67:24:54)
    Destination address: SenaoNet_18:a7:d0 (88:dc:96:18:a7:d0)
    Fragment number: 0
    Sequence number: 1391
    Qos Control: 0x0080
        .... .... .... 0000 = TID: 0
        [.... .... .... .000 = Priority: Best Effort (Best Effort) (0)]
        .... .... ...0 .... = QoS bit 4: Bits 8-15 of QoS Control field are TXOP Duration Requested
        .... .... .00. .... = Ack Policy: Normal Ack (0x0000)
        .... .... 1... .... = Payload Type: A-MSDU
        0000 0000 .... .... = TXOP Duration Requested: 0 (no TXOP requested)
IEEE 802.11 Aggregate MSDU
    A-MSDU Subframe #1
        Destination address: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
        Source address: Apple_67:24:54 (84:38:35:67:24:54)
        A-MSDU Length: 1508
        Logical-Link Control
            DSAP: SNAP (0xaa)
            IG Bit: Individual
            SSAP: SNAP (0xaa)
            CR Bit: Command
            Control field: U, func=UI (0x03)
                000. 00.. = Command: Unnumbered Information (0x00)
                .... ..11 = Frame type: Unnumbered frame (0x03)
            Organization Code: Encapsulated Ethernet (0x000000)
            Type: IP (0x0800)
        Internet Protocol Version 4, Src: 192.168.254.52 (192.168.254.52), Dst: 192.168.254.1 (192.168.254.1)
            Version: 4
            Header length: 20 bytes
            Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
                0000 00.. = Differentiated Services Codepoint: Default (0x00)
                .... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
            Total Length: 1500
            Identification: 0xa03d (41021)
            Flags: 0x02 (Don't Fragment)
                0... .... = Reserved bit: Not set
                .1.. .... = Don't fragment: Set
                ..0. .... = More fragments: Not set
            Fragment offset: 0
            Time to live: 64
            Protocol: TCP (6)
            Header checksum: 0x1757 [correct]
                [Good: True]
                [Bad: False]
            Source: 192.168.254.52 (192.168.254.52)
            Destination: 192.168.254.1 (192.168.254.1)
            [Source GeoIP: Unknown]
            [Destination GeoIP: Unknown]
        Transmission Control Protocol, Src Port: 52734 (52734), Dst Port: 5001 (5001), Seq: 3065441, Ack: 1, Len: 1448
            Source port: 52734 (52734)
            Destination port: 5001 (5001)
            [Stream index: 0]
            Sequence number: 3065441    (relative sequence number)
            [Next sequence number: 3066889    (relative sequence number)]
            Acknowledgment number: 1    (relative ack number)
            Header length: 32 bytes
            Flags: 0x010 (ACK)
                000. .... .... = Reserved: Not set
                ...0 .... .... = Nonce: Not set
                .... 0... .... = Congestion Window Reduced (CWR): Not set
                .... .0.. .... = ECN-Echo: Not set
                .... ..0. .... = Urgent: Not set
                .... ...1 .... = Acknowledgment: Set
                .... .... 0... = Push: Not set
                .... .... .0.. = Reset: Not set
                .... .... ..0. = Syn: Not set
                .... .... ...0 = Fin: Not set
            Window size value: 8235
            [Calculated window size: 131760]
            [Window size scaling factor: 16]
            Checksum: 0x1b72 [correct]
                [Good Checksum: True]
                [Bad Checksum: False]
            Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
                No-Operation (NOP)
                    Type: 1
                        0... .... = Copy on fragmentation: No
                        .00. .... = Class: Control (0)
                        ...0 0001 = Number: No-Operation (NOP) (1)
                No-Operation (NOP)
                    Type: 1
                        0... .... = Copy on fragmentation: No
                        .00. .... = Class: Control (0)
                        ...0 0001 = Number: No-Operation (NOP) (1)
                Timestamps: TSval 1302514011, TSecr 328746582
                    Kind: Timestamp (8)
                    Length: 10
                    Timestamp value: 1302514011
                    Timestamp echo reply: 328746582
        Data (1448 bytes)

0000  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0010  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0020  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0030  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0040  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0050  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0060  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0070  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0080  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0090  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
00a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
00b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
00c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
00d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
00e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
00f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0100  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0110  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0120  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0130  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0140  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0150  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0160  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0170  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0180  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0190  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
01a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
01b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
01c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
01d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
01e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
01f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0200  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0210  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0220  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0230  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0240  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0250  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0260  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0270  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0280  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0290  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
02a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
02b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
02c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
02d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
02e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
02f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0300  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0310  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0320  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0330  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0340  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0350  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0360  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0370  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0380  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0390  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
03a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
03b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
03c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
03d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
03e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
03f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0400  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0410  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0420  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0430  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0440  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0450  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0460  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0470  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0480  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0490  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
04a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
04b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
04c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
04d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
04e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
04f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0500  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0510  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0520  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0530  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0540  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0550  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0560  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0570  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0580  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0590  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
05a0  30 31 32 33 34 35 36 37                           01234567
            Data: 303132333435363738393031323334353637383930313233...
            [Length: 1448]
    A-MSDU Subframe #2
        Destination address: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
        Source address: Apple_67:24:54 (84:38:35:67:24:54)
        A-MSDU Length: 1508
        Logical-Link Control
            DSAP: SNAP (0xaa)
            IG Bit: Individual
            SSAP: SNAP (0xaa)
            CR Bit: Command
            Control field: U, func=UI (0x03)
                000. 00.. = Command: Unnumbered Information (0x00)
                .... ..11 = Frame type: Unnumbered frame (0x03)
            Organization Code: Encapsulated Ethernet (0x000000)
            Type: IP (0x0800)
        Internet Protocol Version 4, Src: 192.168.254.52 (192.168.254.52), Dst: 192.168.254.1 (192.168.254.1)
            Version: 4
            Header length: 20 bytes
            Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
                0000 00.. = Differentiated Services Codepoint: Default (0x00)
                .... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
            Total Length: 1500
            Identification: 0xc127 (49447)
            Flags: 0x02 (Don't Fragment)
                0... .... = Reserved bit: Not set
                .1.. .... = Don't fragment: Set
                ..0. .... = More fragments: Not set
            Fragment offset: 0
            Time to live: 64
            Protocol: TCP (6)
            Header checksum: 0xf66c [correct]
                [Good: True]
                [Bad: False]
            Source: 192.168.254.52 (192.168.254.52)
            Destination: 192.168.254.1 (192.168.254.1)
            [Source GeoIP: Unknown]
            [Destination GeoIP: Unknown]
        Transmission Control Protocol, Src Port: 52734 (52734), Dst Port: 5001 (5001), Seq: 3066889, Ack: 1, Len: 1448
            Source port: 52734 (52734)
            Destination port: 5001 (5001)
            [Stream index: 0]
            Sequence number: 3066889    (relative sequence number)
            [Next sequence number: 3068337    (relative sequence number)]
            Acknowledgment number: 1    (relative ack number)
            Header length: 32 bytes
            Flags: 0x010 (ACK)
                000. .... .... = Reserved: Not set
                ...0 .... .... = Nonce: Not set
                .... 0... .... = Congestion Window Reduced (CWR): Not set
                .... .0.. .... = ECN-Echo: Not set
                .... ..0. .... = Urgent: Not set
                .... ...1 .... = Acknowledgment: Set
                .... .... 0... = Push: Not set
                .... .... .0.. = Reset: Not set
                .... .... ..0. = Syn: Not set
                .... .... ...0 = Fin: Not set
            Window size value: 8235
            [Calculated window size: 131760]
            [Window size scaling factor: 16]
            Checksum: 0x13c8 [incorrect, should be 0xe891 (maybe caused by "TCP checksum offload"?)]
                [Good Checksum: False]
                [Bad Checksum: True]
                    [Expert Info (Error/Checksum): Bad checksum]
                        [Message: Bad checksum]
                        [Severity level: Error]
                        [Group: Checksum]
            Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
                No-Operation (NOP)
                    Type: 1
                        0... .... = Copy on fragmentation: No
                        .00. .... = Class: Control (0)
                        ...0 0001 = Number: No-Operation (NOP) (1)
                No-Operation (NOP)
                    Type: 1
                        0... .... = Copy on fragmentation: No
                        .00. .... = Class: Control (0)
                        ...0 0001 = Number: No-Operation (NOP) (1)
                Timestamps: TSval 1302514011, TSecr 328746582
                    Kind: Timestamp (8)
                    Length: 10
                    Timestamp value: 1302514011
                    Timestamp echo reply: 328746582
        Data (1448 bytes)

0000  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0010  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0020  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0030  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0040  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0050  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0060  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0070  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0080  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0090  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
00a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
00b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
00c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
00d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
00e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
00f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0100  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0110  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0120  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0130  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0140  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0150  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0160  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0170  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0180  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0190  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
01a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
01b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
01c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
01d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
01e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
01f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0200  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0210  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0220  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0230  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0240  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0250  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0260  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0270  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0280  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0290  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
02a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
02b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
02c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
02d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
02e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
02f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0300  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0310  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0320  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0330  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0340  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0350  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0360  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0370  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0380  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0390  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
03a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
03b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
03c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
03d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
03e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
03f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0400  32 33 34 35 36 37 38 39 30 31 26 67 74 25 27 67   2345678901&gt%'g
0410  39 28 31 24 36 33 34 35 36 37 38 39 30 31 32 33   9(1$634567890123
0420  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0430  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0440  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0450  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0460  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0470  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0480  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0490  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
04a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
04b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
04c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
04d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
04e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
04f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0500  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0510  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0520  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0530  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0540  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0550  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0560  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0570  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0580  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0590  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
05a0  38 39 30 31 32 33 34 35                           89012345
            Data: 383930313233343536373839303132333435363738393031...
            [Length: 1448]

No.     Time        Source                Destination           Protocol Length Info
   5556 15.609203   192.168.254.52        192.168.254.1         TCP      3110   52734 > 5001 [ACK] Seq=3066889 Ack=1 Win=131760 Len=1448 TSval=1302514011 TSecr=328746582

Frame 5556: 3110 bytes on wire (24880 bits), 3110 bytes captured (24880 bits)
    Encapsulation type: IEEE 802.11 plus radiotap radio header (23)
    Arrival Time: Jul  8, 2014 11:24:39.082957000 PDT
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1404843879.082957000 seconds
    [Time delta from previous captured frame: 0.000066000 seconds]
    [Time delta from previous displayed frame: 0.000066000 seconds]
    [Time since reference or first frame: 15.609203000 seconds]
    Frame Number: 5556
    Frame Length: 3110 bytes (24880 bits)
    Capture Length: 3110 bytes (24880 bits)
    [Frame is marked: True]
    [Frame is ignored: False]
    [Protocols in frame: radiotap:wlan:llc:ip:tcp:data:llc:ip:tcp:data]
    [Coloring Rule Name: TCP]
    [Coloring Rule String: tcp]
Radiotap Header v0, Length 38
    Header revision: 0
    Header pad: 0
    Header length: 38
    Present flags
        .... .... .... .... .... .... .... ...1 = TSFT: True
        .... .... .... .... .... .... .... ..1. = Flags: True
        .... .... .... .... .... .... .... .0.. = Rate: False
        .... .... .... .... .... .... .... 1... = Channel: True
        .... .... .... .... .... .... ...0 .... = FHSS: False
        .... .... .... .... .... .... ..1. .... = dBm Antenna Signal: True
        .... .... .... .... .... .... .0.. .... = dBm Antenna Noise: False
        .... .... .... .... .... .... 0... .... = Lock Quality: False
        .... .... .... .... .... ...0 .... .... = TX Attenuation: False
        .... .... .... .... .... ..0. .... .... = dB TX Attenuation: False
        .... .... .... .... .... .0.. .... .... = dBm TX Power: False
        .... .... .... .... .... 1... .... .... = Antenna: True
        .... .... .... .... ...0 .... .... .... = dB Antenna Signal: False
        .... .... .... .... ..0. .... .... .... = dB Antenna Noise: False
        .... .... .... .... .1.. .... .... .... = RX flags: True
        .... .... .... .0.. .... .... .... .... = Channel+: False
        .... .... .... 0... .... .... .... .... = HT information: False
        .... .... ...0 .... .... .... .... .... = A-MPDU Status: False
        .... .... ..1. .... .... .... .... .... = VHT information: True
        ...0 0000 00.. .... .... .... .... .... = Reserved: 0x00000000
        ..0. .... .... .... .... .... .... .... = Radiotap NS next: False
        .0.. .... .... .... .... .... .... .... = Vendor NS next: False
        0... .... .... .... .... .... .... .... = Ext: False
    MAC timestamp: 1395542664
    Flags: 0x00
        .... ...0 = CFP: False
        .... ..0. = Preamble: Long
        .... .0.. = WEP: False
        .... 0... = Fragmentation: False
        ...0 .... = FCS at end: False
        ..0. .... = Data Pad: False
        .0.. .... = Bad FCS: False
        0... .... = Short GI: False
    Channel frequency: 5745 [A 149]
    Channel type: 802.11a (0x0140)
        .... .... ...0 .... = Turbo: False
        .... .... ..0. .... = Complementary Code Keying (CCK): False
        .... .... .1.. .... = Orthogonal Frequency-Division Multiplexing (OFDM): True
        .... .... 0... .... = 2 GHz spectrum: False
        .... ...1 .... .... = 5 GHz spectrum: True
        .... ..0. .... .... = Passive: False
        .... .0.. .... .... = Dynamic CCK-OFDM: False
        .... 0... .... .... = Gaussian Frequency Shift Keying (GFSK): False
        ...0 .... .... .... = GSM (900MHz): False
        ..0. .... .... .... = Static Turbo: False
        .0.. .... .... .... = Half Rate Channel (10MHz Channel Width): False
        0... .... .... .... = Quarter Rate Channel (5MHz Channel Width): False
    SSI Signal: -46 dBm
    Antenna: 0
    RX flags: 0x0000
        .... .... .... .... .... ..0. = Bad PLCP: False
    VHT information
        Known VHT information: 0x44
            .... .... .... ...0 = STBC: False
            .... .... .... ..0. = TXOP_PS_NOT_ALLOWED: False
            .... .... .... .1.. = Guard interval: True
            .... .... .... 0... = SGI Nsym disambiguation: False
            .... .... ...0 .... = LDPC extra OFDM symbol: False
            .... .... ..0. .... = Beamformed: False
            .... .... .1.. .... = Bandwidth: True
            .... .... 0... .... = Group ID: False
            .... ...0 .... .... = Partial AID: False
        .... .0.. = Guard interval: long (0)
        Bandwidth: 80 MHz (4)
        User 0: MCS 9
            1001 .... = MCS index 0: 9 (256-QAM 5/6)
            .... 0010 = Spatial streams 0: 2
            Space-time streams 0: 2
            Coding 0: BCC (0)
            [Data Rate: 780.0 Mb/s]
IEEE 802.11 QoS Data, Flags: ....R..T
    Type/Subtype: QoS Data (0x28)
    Frame Control Field: 0x8809
        .... ..00 = Version: 0
        .... 10.. = Type: Data frame (2)
        1000 .... = Subtype: 8
        Flags: 0x09
            .... ..01 = DS status: Frame from STA to DS via an AP (To DS: 1 From DS: 0) (0x01)
            .... .0.. = More Fragments: This is the last fragment
            .... 1... = Retry: Frame is being retransmitted
                [Expert Info (Note/Sequence): Retransmission (retry)]
                    [Message: Retransmission (retry)]
                    [Severity level: Note]
                    [Group: Sequence]
            ...0 .... = PWR MGT: STA will stay up
            ..0. .... = More Data: No data buffered
            .0.. .... = Protected flag: Data is not protected
            0... .... = Order flag: Not strictly ordered
    .000 0000 0011 0000 = Duration: 48 microseconds
    Receiver address: SenaoNet_18:a7:d0 (88:dc:96:18:a7:d0)
    BSS Id: SenaoNet_18:a7:d0 (88:dc:96:18:a7:d0)
    Transmitter address: Apple_67:24:54 (84:38:35:67:24:54)
    Source address: Apple_67:24:54 (84:38:35:67:24:54)
    Destination address: SenaoNet_18:a7:d0 (88:dc:96:18:a7:d0)
    Fragment number: 0
    Sequence number: 1391
    Qos Control: 0x0080
        .... .... .... 0000 = TID: 0
        [.... .... .... .000 = Priority: Best Effort (Best Effort) (0)]
        .... .... ...0 .... = QoS bit 4: Bits 8-15 of QoS Control field are TXOP Duration Requested
        .... .... .00. .... = Ack Policy: Normal Ack (0x0000)
        .... .... 1... .... = Payload Type: A-MSDU
        0000 0000 .... .... = TXOP Duration Requested: 0 (no TXOP requested)
IEEE 802.11 Aggregate MSDU
    A-MSDU Subframe #1
        Destination address: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
        Source address: Apple_67:24:54 (84:38:35:67:24:54)
        A-MSDU Length: 1508
        Logical-Link Control
            DSAP: SNAP (0xaa)
            IG Bit: Individual
            SSAP: SNAP (0xaa)
            CR Bit: Command
            Control field: U, func=UI (0x03)
                000. 00.. = Command: Unnumbered Information (0x00)
                .... ..11 = Frame type: Unnumbered frame (0x03)
            Organization Code: Encapsulated Ethernet (0x000000)
            Type: IP (0x0800)
        Internet Protocol Version 4, Src: 192.168.254.52 (192.168.254.52), Dst: 192.168.254.1 (192.168.254.1)
            Version: 4
            Header length: 20 bytes
            Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
                0000 00.. = Differentiated Services Codepoint: Default (0x00)
                .... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
            Total Length: 1500
            Identification: 0xa03d (41021)
            Flags: 0x02 (Don't Fragment)
                0... .... = Reserved bit: Not set
                .1.. .... = Don't fragment: Set
                ..0. .... = More fragments: Not set
            Fragment offset: 0
            Time to live: 64
            Protocol: TCP (6)
            Header checksum: 0x1757 [correct]
                [Good: True]
                [Bad: False]
            Source: 192.168.254.52 (192.168.254.52)
            Destination: 192.168.254.1 (192.168.254.1)
            [Source GeoIP: Unknown]
            [Destination GeoIP: Unknown]
        Transmission Control Protocol, Src Port: 52734 (52734), Dst Port: 5001 (5001), Seq: 3065441, Ack: 1, Len: 1448
            Source port: 52734 (52734)
            Destination port: 5001 (5001)
            [Stream index: 0]
            Sequence number: 3065441    (relative sequence number)
            [Next sequence number: 3066889    (relative sequence number)]
            Acknowledgment number: 1    (relative ack number)
            Header length: 32 bytes
            Flags: 0x010 (ACK)
                000. .... .... = Reserved: Not set
                ...0 .... .... = Nonce: Not set
                .... 0... .... = Congestion Window Reduced (CWR): Not set
                .... .0.. .... = ECN-Echo: Not set
                .... ..0. .... = Urgent: Not set
                .... ...1 .... = Acknowledgment: Set
                .... .... 0... = Push: Not set
                .... .... .0.. = Reset: Not set
                .... .... ..0. = Syn: Not set
                .... .... ...0 = Fin: Not set
            Window size value: 8235
            [Calculated window size: 131760]
            [Window size scaling factor: 16]
            Checksum: 0x1b72 [correct]
                [Good Checksum: True]
                [Bad Checksum: False]
            Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
                No-Operation (NOP)
                    Type: 1
                        0... .... = Copy on fragmentation: No
                        .00. .... = Class: Control (0)
                        ...0 0001 = Number: No-Operation (NOP) (1)
                No-Operation (NOP)
                    Type: 1
                        0... .... = Copy on fragmentation: No
                        .00. .... = Class: Control (0)
                        ...0 0001 = Number: No-Operation (NOP) (1)
                Timestamps: TSval 1302514011, TSecr 328746582
                    Kind: Timestamp (8)
                    Length: 10
                    Timestamp value: 1302514011
                    Timestamp echo reply: 328746582
        Data (1448 bytes)

0000  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0010  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0020  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0030  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0040  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0050  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0060  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0070  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0080  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0090  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
00a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
00b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
00c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
00d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
00e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
00f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0100  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0110  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0120  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0130  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0140  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0150  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0160  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0170  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0180  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0190  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
01a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
01b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
01c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
01d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
01e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
01f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0200  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0210  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0220  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0230  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0240  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0250  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0260  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0270  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0280  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0290  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
02a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
02b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
02c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
02d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
02e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
02f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0300  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0310  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0320  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0330  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0340  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0350  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0360  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0370  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0380  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0390  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
03a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
03b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
03c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
03d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
03e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
03f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0400  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0410  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0420  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0430  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0440  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0450  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0460  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0470  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0480  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0490  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
04a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
04b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
04c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
04d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
04e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
04f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0500  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0510  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0520  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0530  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0540  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0550  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0560  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0570  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0580  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0590  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
05a0  30 31 32 33 34 35 36 37                           01234567
            Data: 303132333435363738393031323334353637383930313233...
            [Length: 1448]
    A-MSDU Subframe #2
        Destination address: Intel_5e:bd:85 (00:0e:0c:5e:bd:85)
        Source address: Apple_67:24:54 (84:38:35:67:24:54)
        A-MSDU Length: 1508
        Logical-Link Control
            DSAP: SNAP (0xaa)
            IG Bit: Individual
            SSAP: SNAP (0xaa)
            CR Bit: Command
            Control field: U, func=UI (0x03)
                000. 00.. = Command: Unnumbered Information (0x00)
                .... ..11 = Frame type: Unnumbered frame (0x03)
            Organization Code: Encapsulated Ethernet (0x000000)
            Type: IP (0x0800)
        Internet Protocol Version 4, Src: 192.168.254.52 (192.168.254.52), Dst: 192.168.254.1 (192.168.254.1)
            Version: 4
            Header length: 20 bytes
            Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
                0000 00.. = Differentiated Services Codepoint: Default (0x00)
                .... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
            Total Length: 1500
            Identification: 0xc127 (49447)
            Flags: 0x02 (Don't Fragment)
                0... .... = Reserved bit: Not set
                .1.. .... = Don't fragment: Set
                ..0. .... = More fragments: Not set
            Fragment offset: 0
            Time to live: 64
            Protocol: TCP (6)
            Header checksum: 0xf66c [correct]
                [Good: True]
                [Bad: False]
            Source: 192.168.254.52 (192.168.254.52)
            Destination: 192.168.254.1 (192.168.254.1)
            [Source GeoIP: Unknown]
            [Destination GeoIP: Unknown]
        Transmission Control Protocol, Src Port: 52734 (52734), Dst Port: 5001 (5001), Seq: 3066889, Ack: 1, Len: 1448
            Source port: 52734 (52734)
            Destination port: 5001 (5001)
            [Stream index: 0]
            Sequence number: 3066889    (relative sequence number)
            [Next sequence number: 3068337    (relative sequence number)]
            Acknowledgment number: 1    (relative ack number)
            Header length: 32 bytes
            Flags: 0x010 (ACK)
                000. .... .... = Reserved: Not set
                ...0 .... .... = Nonce: Not set
                .... 0... .... = Congestion Window Reduced (CWR): Not set
                .... .0.. .... = ECN-Echo: Not set
                .... ..0. .... = Urgent: Not set
                .... ...1 .... = Acknowledgment: Set
                .... .... 0... = Push: Not set
                .... .... .0.. = Reset: Not set
                .... .... ..0. = Syn: Not set
                .... .... ...0 = Fin: Not set
            Window size value: 8235
            [Calculated window size: 131760]
            [Window size scaling factor: 16]
            Checksum: 0x13c8 [correct]
                [Good Checksum: True]
                [Bad Checksum: False]
            Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
                No-Operation (NOP)
                    Type: 1
                        0... .... = Copy on fragmentation: No
                        .00. .... = Class: Control (0)
                        ...0 0001 = Number: No-Operation (NOP) (1)
                No-Operation (NOP)
                    Type: 1
                        0... .... = Copy on fragmentation: No
                        .00. .... = Class: Control (0)
                        ...0 0001 = Number: No-Operation (NOP) (1)
                Timestamps: TSval 1302514011, TSecr 328746582
                    Kind: Timestamp (8)
                    Length: 10
                    Timestamp value: 1302514011
                    Timestamp echo reply: 328746582
        Data (1448 bytes)

0000  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0010  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0020  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0030  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0040  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0050  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0060  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0070  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0080  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0090  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
00a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
00b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
00c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
00d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
00e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
00f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0100  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0110  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0120  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0130  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0140  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0150  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0160  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0170  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0180  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0190  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
01a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
01b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
01c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
01d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
01e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
01f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0200  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0210  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0220  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0230  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0240  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0250  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0260  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0270  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0280  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0290  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
02a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
02b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
02c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
02d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
02e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
02f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0300  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0310  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0320  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0330  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0340  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0350  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0360  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0370  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0380  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0390  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
03a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
03b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
03c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
03d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
03e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
03f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0400  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0410  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0420  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0430  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0440  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0450  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0460  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0470  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0480  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0490  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
04a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
04b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
04c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
04d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
04e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
04f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0500  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0510  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0520  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0530  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0540  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
0550  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
0560  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
0570  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
0580  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
0590  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
05a0  38 39 30 31 32 33 34 35                           89012345
            Data: 383930313233343536373839303132333435363738393031...
            [Length: 1448]

[-- Attachment #5: Type: text/plain, Size: 146 bytes --]

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Janusz Dziedzic]

On 9 July 2014 08:09, Denton Gentry <denton.gentry@gmail.com> wrote:
> I ran iperf on an 802.11ac MacBook Air, through my ath10 AP, and to an
> Ubuntu system connected to the AP via Ethernet. The AP was running the
> reorder buffer patches, the patch to make A-MSDU bypass the reorder
> buffer, and the patch to make amsdu aggregation configurable via
> debugfs.
>
> http://lists.infradead.org/pipermail/ath10k/2014-June/002551.html
> http://lists.infradead.org/pipermail/ath10k/2014-June/002553.html
> http://lists.infradead.org/pipermail/ath10k/2014-July/002597.html
> http://permalink.gmane.org/gmane.linux.kernel.wireless.general/124128
>
> It was notably not running the patch from earlier in this thread to
> copy A-MSDU subframes into one skb.
>
>
> I used a second ath10k AP configured in monitor mode as a Wifi
> sniffer. I also gathered pcaps on the MacBook Air and on the Ubuntu
> system. Wireshark on the MacBook cannot gather radiotap headers nor
> put the interface into monitor mode, so we see only Ethernet frames
> and we don't see retransmissions on the Wifi link.
>
> I've attached decodes of the same two packets as captured by the
> MacBook, by the sniffer, and by the Ubuntu server.
>
> The MacBook sends two frames, TCP sequence numbers 3065441 and 3066889
> (the Wireshark "relative sequence number"). In the Wifi sniffer we see
> these being aggregated as two subframes in an A-MSDU frame. The first
> time this A-MSDU is sent it is corrupted in the air. We see the TCP
> checksum of one of the subframes is wrong, and a few bytes of the
> payload replaced with "&gt%'g9(1$".
>
> The A-MSDU is retransmitted a short time later, and the second time
> the TCP checksum of both subframes is correct.
>
> This is all fine so far.
>
> However in the pcap taken from the Ubuntu server, we see TCP sequence
> number 3065441 being delivered *twice.* The first time, the TCP
> checksum is wrong. The second time the TCP checksum is correct.
>
> To me, it looks like A-MSDU frames with bad FCS are not being
> discarded after ieee80211_rx_monitor(). The corrupted frames are being
> delivered. Delivering the corrupted frames results in sending more TCP
> Dup ACKs for the same sequence number back to the MacBook, and I think
> this is what causes the MacBook to decide there is congestion and slow
> down.
>
OK, to be sure this is the main issue we can just drop frames with wrong FCS.
Could you check this:

@@ -1267,10 +1267,10 @@ static void ath10k_htt_rx_handler(struct
ath10k_htt *htt,
                                continue;
                        }

-                       if (attention & RX_ATTENTION_FLAGS_FCS_ERR)
-                               rx_status->flag |= RX_FLAG_FAILED_FCS_CRC;
-                       else
-                               rx_status->flag &= ~RX_FLAG_FAILED_FCS_CRC;
+                       if (attention & RX_ATTENTION_FLAGS_FCS_ERR) {
+                               ath10k_htt_rx_free_msdu_chain(msdu_head);
+                               continue;
+                       }

                        if (attention & RX_ATTENTION_FLAGS_TKIP_MIC_ERR)

If will help we will have to check how to handle monitor + ap case
correctly, one of idea is to not drop frames with wrong FCS only in
case of standalone monitor mode.
But, please try this patch to be sure.

BR
Janusz


> One note: the wifi sniffer does not show the same corruption to the
> same packet as the pcap from the Ubuntu system shows. I think that is
> normal: the sniffer won't see precisely the same noise, won't have
> precisely the same receive sensitivity, and its antennas are not
> pointing in the same direction as the primary AP. If I do this test
> again, I'll gather a pcap on the primary AP as well to compare to what
> we see on the Ubuntu system.
>
>
> On Tue, Jul 8, 2014 at 12:29 AM, Janusz Dziedzic
> <janusz.dziedzic@tieto.com> wrote:
>> On 8 July 2014 09:02, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:
>>> On 8 July 2014 08:50, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:
>>>> On 8 July 2014 08:43, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>> I think I know what is happening now, though I've no idea why. The
>>>>> throughput is low because we have many TCP retransmissions. We have
>>>>> retransmissions because the TCP checksum is wrong on a number of
>>>>> frames, and I do find data corruption in the payload so the checksum
>>>>> definitely should be wrong. All of the corrupted frames were
>>>>> originally one of the subframes in an A-MSDU packet.
>>>>>
>>>>> An example follows at the end of this message, as dissected by
>>>>> Wireshark. iperf sends a very regular data pattern of "0123456789..."
>>>>> over and over. Note how in subframe #2 offset 0x1e0 the bytes "32 33
>>>>> 34" have been replaced by "72 36 35"
>>>>>
>>>>> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
>>>>>
>>>>> I added printks at the bottom of ath10k_htt_rx_amsdu immediately
>>>>> before the call to ath10k_process_rx. I found this same packet, and we
>>>>> see the "72 36 35" corruption in the printk. So I think it happened in
>>>>> ath10k_process_rx or before, not anything weird after passing it up to
>>>>> mac80211.
>>>>>
>>>>> [  101.863712] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
>>>>> [  101.863727] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
>>>>> [  101.863742] ath10k: 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32
>>>>> [  101.863757] ath10k: 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37 38
>>>>> [  101.863773] ath10k: 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34
>>>>> [  101.863788] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
>>>>> [  101.863803] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
>>>>>
>>>>>
>>>>> I've found a number of examples of similar corruption, always with
>>>>> between one and four bytes replaced.
>>>>>
>>>>> 35363738 -> e52c6e07
>>>>> 3435 -> b43f
>>>>> 3839 -> c238
>>>>> 31 -> 7f
>>>>> 3435 -> 7436
>>>>> 30 -> 50
>>>>> 3233 -> bc37
>>>>>
>>>> Seems this could be because of:
>>>>
>>>> + /* cfg80211 expect this padding */
>>>> + padding = (4 - (skb->len + sizeof(subframe_hdr))) & 0x3;
>>>> + skb_put(skb, padding);
>>>>
>>>
>>> BTW, when AP (ath10k) send TCP ACK - are this packets also AMSDU?
>>> I see in my case AP can send 3 x A-MSDU (with total size 304 bytes)
>>> small frames aggregated. Maybe your HW have problems with that.
>>> As I remember correctly someone some time ago report problems with
>>> MacBook pro retina but I am not sure this is the same, while no one
>>> tests the fix.
>>>
>>>>>
>>>>> The packet described above, dissected by Wireshark:
>>>>>
>>>>> No.     Time        Source                Destination
>>>>> Protocol Length Info
>>>>>    2235 18.953349   192.168.144.79        192.168.144.13        TCP
>>>>>   3112   52697 > 5001 [ACK] Seq=1391553 Ack=1 Win=131760 [TCP CHECKSUM
>>>>> INCORRECT] Len=1448 TSval=1298580657 TSecr=4294947481
>>>>>
>>>>> Frame 2235: 3112 bytes on wire (24896 bits), 3112 bytes captured (24896 bits)
>>>>>     Encapsulation type: IEEE 802.11 plus radiotap radio header (23)
>>>>>     Arrival Time: Jul  7, 2014 23:03:37.763365000 PDT
>>>>>     [Time shift for this packet: 0.000000000 seconds]
>>>>>     Epoch Time: 1404799417.763365000 seconds
>>>>>     [Time delta from previous captured frame: 0.003476000 seconds]
>>>>>     [Time delta from previous displayed frame: 0.515641000 seconds]
>>>>>     [Time since reference or first frame: 18.953349000 seconds]
>>>>>     Frame Number: 2235
>>>>>     Frame Length: 3112 bytes (24896 bits)
>>>>>     Capture Length: 3112 bytes (24896 bits)
>>>>>     [Frame is marked: False]
>>>>>     [Frame is ignored: False]
>>>>>     [Protocols in frame: radiotap:wlan:llc:ip:tcp:data:llc:ip:tcp:data]
>>>>>     [Coloring Rule Name: TCP]
>>>>>     [Coloring Rule String: tcp]
>>>>> Radiotap Header v0, Length 38
>>>>>     Header revision: 0
>>>>>     Header pad: 0
>>>>>     Header length: 38
>>>>>     Present flags
>>>>>         .... .... .... .... .... .... .... ...1 = TSFT: True
>>>>>         .... .... .... .... .... .... .... ..1. = Flags: True
>>>>>         .... .... .... .... .... .... .... .0.. = Rate: False
>>>>>         .... .... .... .... .... .... .... 1... = Channel: True
>>>>>         .... .... .... .... .... .... ...0 .... = FHSS: False
>>>>>         .... .... .... .... .... .... ..1. .... = dBm Antenna Signal: True
>>>>>         .... .... .... .... .... .... .0.. .... = dBm Antenna Noise: False
>>>>>         .... .... .... .... .... .... 0... .... = Lock Quality: False
>>>>>         .... .... .... .... .... ...0 .... .... = TX Attenuation: False
>>>>>         .... .... .... .... .... ..0. .... .... = dB TX Attenuation: False
>>>>>         .... .... .... .... .... .0.. .... .... = dBm TX Power: False
>>>>>         .... .... .... .... .... 1... .... .... = Antenna: True
>>>>>         .... .... .... .... ...0 .... .... .... = dB Antenna Signal: False
>>>>>         .... .... .... .... ..0. .... .... .... = dB Antenna Noise: False
>>>>>         .... .... .... .... .1.. .... .... .... = RX flags: True
>>>>>         .... .... .... .0.. .... .... .... .... = Channel+: False
>>>>>         .... .... .... 0... .... .... .... .... = HT information: False
>>>>>         .... .... ...0 .... .... .... .... .... = A-MPDU Status: False
>>>>>         .... .... ..1. .... .... .... .... .... = VHT information: True
>>>>>         ...0 0000 00.. .... .... .... .... .... = Reserved: 0x00000000
>>>>>         ..0. .... .... .... .... .... .... .... = Radiotap NS next: False
>>>>>         .0.. .... .... .... .... .... .... .... = Vendor NS next: False
>>>>>         0... .... .... .... .... .... .... .... = Ext: False
>>>>>     MAC timestamp: 78051063
>>>>>     Flags: 0x00
>>>>>         .... ...0 = CFP: False
>>>>>         .... ..0. = Preamble: Long
>>>>>         .... .0.. = WEP: False
>>>>>         .... 0... = Fragmentation: False
>>>>>         ...0 .... = FCS at end: False
>>>>>         ..0. .... = Data Pad: False
>>>>>         .0.. .... = Bad FCS: False
>>>>>         0... .... = Short GI: False
>>>>>     Channel frequency: 5745 [A 149]
>>>>>     Channel type: 802.11a (0x0140)
>>>>>         .... .... ...0 .... = Turbo: False
>>>>>         .... .... ..0. .... = Complementary Code Keying (CCK): False
>>>>>         .... .... .1.. .... = Orthogonal Frequency-Division
>>>>> Multiplexing (OFDM): True
>>>>>         .... .... 0... .... = 2 GHz spectrum: False
>>>>>         .... ...1 .... .... = 5 GHz spectrum: True
>>>>>         .... ..0. .... .... = Passive: False
>>>>>         .... .0.. .... .... = Dynamic CCK-OFDM: False
>>>>>         .... 0... .... .... = Gaussian Frequency Shift Keying (GFSK): False
>>>>>         ...0 .... .... .... = GSM (900MHz): False
>>>>>         ..0. .... .... .... = Static Turbo: False
>>>>>         .0.. .... .... .... = Half Rate Channel (10MHz Channel Width): False
>>>>>         0... .... .... .... = Quarter Rate Channel (5MHz Channel Width): False
>>>>>     SSI Signal: -53 dBm
>>>>>     Antenna: 0
>>>>>     RX flags: 0x0000
>>>>>         .... .... .... .... .... ..0. = Bad PLCP: False
>>>>>     VHT information
>>>>>         Known VHT information: 0x44
>>>>>             .... .... .... ...0 = STBC: False
>>>>>             .... .... .... ..0. = TXOP_PS_NOT_ALLOWED: False
>>>>>             .... .... .... .1.. = Guard interval: True
>>>>>             .... .... .... 0... = SGI Nsym disambiguation: False
>>>>>             .... .... ...0 .... = LDPC extra OFDM symbol: False
>>>>>             .... .... ..0. .... = Beamformed: False
>>>>>             .... .... .1.. .... = Bandwidth: True
>>>>>             .... .... 0... .... = Group ID: False
>>>>>             .... ...0 .... .... = Partial AID: False
>>>>>         .... .0.. = Guard interval: long (0)
>>>>>         Bandwidth: 80 MHz (4)
>>>>>         User 0: MCS 8
>>>>>             1000 .... = MCS index 0: 8 (256-QAM 3/4)
>>>>>             .... 0010 = Spatial streams 0: 2
>>>>>             Space-time streams 0: 2
>>>>>             Coding 0: BCC (0)
>>>>>             [Data Rate: 702.0 Mb/s]
>>>>> IEEE 802.11 QoS Data, Flags: .......T
>>>>>     Type/Subtype: QoS Data (0x28)
>>>>>     Frame Control Field: 0x8801
>>>>>         .... ..00 = Version: 0
>>>>>         .... 10.. = Type: Data frame (2)
>>>>>         1000 .... = Subtype: 8
>>>>>         Flags: 0x01
>>>>>             .... ..01 = DS status: Frame from STA to DS via an AP (To
>>>>> DS: 1 From DS: 0) (0x01)
>>>>>             .... .0.. = More Fragments: This is the last fragment
>>>>>             .... 0... = Retry: Frame is not being retransmitted
>>>>>             ...0 .... = PWR MGT: STA will stay up
>>>>>             ..0. .... = More Data: No data buffered
>>>>>             .0.. .... = Protected flag: Data is not protected
>>>>>             0... .... = Order flag: Not strictly ordered
>>>>>     .000 0000 0011 0000 = Duration: 48 microseconds
>>>>>     Receiver address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>>>     BSS Id: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>>>     Transmitter address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>     Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>     Destination address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>>>     Fragment number: 0
>>>>>     Sequence number: 1021
>>>>>     Qos Control: 0x0080
>>>>>         .... .... .... 0000 = TID: 0
>>>>>         [.... .... .... .000 = Priority: Best Effort (Best Effort) (0)]
>>>>>         .... .... ...0 .... = QoS bit 4: Bits 8-15 of QoS Control
>>>>> field are TXOP Duration Requested
>>>>>         .... .... .00. .... = Ack Policy: Normal Ack (0x0000)
>>>>>         .... .... 1... .... = Payload Type: A-MSDU
>>>>>         0000 0000 .... .... = TXOP Duration Requested: 0 (no TXOP requested)
>>>>> IEEE 802.11 Aggregate MSDU
>>>>>     A-MSDU Subframe #1
>>>>>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>>>>>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>         A-MSDU Length: 1510
>>>>>         Logical-Link Control
>>>>>             DSAP: SNAP (0xaa)
>>>>>             IG Bit: Individual
>>>>>             SSAP: SNAP (0xaa)
>>>>>             CR Bit: Command
>>>>>             Control field: U, func=UI (0x03)
>>>>>                 000. 00.. = Command: Unnumbered Information (0x00)
>>>>>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>>>>>             Organization Code: Encapsulated Ethernet (0x000000)
>>>>>             Type: IP (0x0800)
>>>>>         Internet Protocol Version 4, Src: 192.168.144.79
>>>>> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>>>>>             Version: 4
>>>>>             Header length: 20 bytes
>>>>>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
>>>>> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>>>>>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>>>>>                 .... ..00 = Explicit Congestion Notification: Not-ECT
>>>>> (Not ECN-Capable Transport) (0x00)
>>>>>             Total Length: 1500
>>>>>             Identification: 0xc622 (50722)
>>>>>             Flags: 0x00
>>>>>                 0... .... = Reserved bit: Not set
>>>>>                 .0.. .... = Don't fragment: Not set
>>>>>                 ..0. .... = More fragments: Not set
>>>>>             Fragment offset: 0
>>>>>             Time to live: 64
>>>>>             Protocol: TCP (6)
>>>>>             Header checksum: 0x0d4c [correct]
>>>>>                 [Good: True]
>>>>>                 [Bad: False]
>>>>>             Source: 192.168.144.79 (192.168.144.79)
>>>>>             Destination: 192.168.144.13 (192.168.144.13)
>>>>>             [Source GeoIP: Unknown]
>>>>>             [Destination GeoIP: Unknown]
>>>>>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
>>>>> Port: 5001 (5001), Seq: 1390105, Ack: 1, Len: 1448
>>>>>             Source port: 52697 (52697)
>>>>>             Destination port: 5001 (5001)
>>>>>             [Stream index: 0]
>>>>>             Sequence number: 1390105    (relative sequence number)
>>>>>             [Next sequence number: 1391553    (relative sequence number)]
>>>>>             Acknowledgment number: 1    (relative ack number)
>>>>>             Header length: 32 bytes
>>>>>             Flags: 0x010 (ACK)
>>>>>                 000. .... .... = Reserved: Not set
>>>>>                 ...0 .... .... = Nonce: Not set
>>>>>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>>>>>                 .... .0.. .... = ECN-Echo: Not set
>>>>>                 .... ..0. .... = Urgent: Not set
>>>>>                 .... ...1 .... = Acknowledgment: Set
>>>>>                 .... .... 0... = Push: Not set
>>>>>                 .... .... .0.. = Reset: Not set
>>>>>                 .... .... ..0. = Syn: Not set
>>>>>                 .... .... ...0 = Fin: Not set
>>>>>             Window size value: 8235
>>>>>             [Calculated window size: 131760]
>>>>>             [Window size scaling factor: 16]
>>>>>             Checksum: 0xa1c0 [correct]
>>>>>                 [Good Checksum: True]
>>>>>                 [Bad Checksum: False]
>>>>>             Options: (12 bytes), No-Operation (NOP), No-Operation
>>>>> (NOP), Timestamps
>>>>>                 No-Operation (NOP)
>>>>>                     Type: 1
>>>>>                         0... .... = Copy on fragmentation: No
>>>>>                         .00. .... = Class: Control (0)
>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>                 No-Operation (NOP)
>>>>>                     Type: 1
>>>>>                         0... .... = Copy on fragmentation: No
>>>>>                         .00. .... = Class: Control (0)
>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>                 Timestamps: TSval 1298580657, TSecr 4294947481
>>>>>                     Kind: Timestamp (8)
>>>>>                     Length: 10
>>>>>                     Timestamp value: 1298580657
>>>>>                     Timestamp echo reply: 4294947481
>>>>>         Data (1448 bytes)
>>>>>
>>>>> 0000  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0010  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0020  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0030  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0040  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0050  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0060  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0070  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0080  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0090  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 00a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 00b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 00c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 00d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 00e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 00f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0100  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0110  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0120  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0130  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0140  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0150  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0160  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0170  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0180  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0190  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 01a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 01b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 01c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 01d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 01e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 01f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0200  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0210  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0220  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0230  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0240  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0250  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0260  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0270  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0280  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0290  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 02a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 02b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 02c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 02d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 02e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 02f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0300  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0310  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0320  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0330  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0340  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0350  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0360  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0370  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0380  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0390  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 03a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 03b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 03c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 03d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 03e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 03f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0400  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0410  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0420  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0430  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0440  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0450  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0460  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0470  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0480  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0490  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 04a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 04b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 04c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 04d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 04e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 04f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0500  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0510  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0520  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0530  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0540  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0550  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0560  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0570  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0580  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0590  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 05a0  34 35 36 37 38 39 30 31                           45678901
>>>>>             Data: 343536373839303132333435363738393031323334353637...
>>>>>             [Length: 1448]
>>>>>     A-MSDU Subframe #2
>>>>>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>>>>>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>         A-MSDU Length: 1510
>>>>>         Logical-Link Control
>>>>>             DSAP: SNAP (0xaa)
>>>>>             IG Bit: Individual
>>>>>             SSAP: SNAP (0xaa)
>>>>>             CR Bit: Command
>>>>>             Control field: U, func=UI (0x03)
>>>>>                 000. 00.. = Command: Unnumbered Information (0x00)
>>>>>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>>>>>             Organization Code: Encapsulated Ethernet (0x000000)
>>>>>             Type: IP (0x0800)
>>>>>         Internet Protocol Version 4, Src: 192.168.144.79
>>>>> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>>>>>             Version: 4
>>>>>             Header length: 20 bytes
>>>>>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
>>>>> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>>>>>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>>>>>                 .... ..00 = Explicit Congestion Notification: Not-ECT
>>>>> (Not ECN-Capable Transport) (0x00)
>>>>>             Total Length: 1500
>>>>>             Identification: 0xda09 (55817)
>>>>>             Flags: 0x00
>>>>>                 0... .... = Reserved bit: Not set
>>>>>                 .0.. .... = Don't fragment: Not set
>>>>>                 ..0. .... = More fragments: Not set
>>>>>             Fragment offset: 0
>>>>>             Time to live: 64
>>>>>             Protocol: TCP (6)
>>>>>             Header checksum: 0xf964 [correct]
>>>>>                 [Good: True]
>>>>>                 [Bad: False]
>>>>>             Source: 192.168.144.79 (192.168.144.79)
>>>>>             Destination: 192.168.144.13 (192.168.144.13)
>>>>>             [Source GeoIP: Unknown]
>>>>>             [Destination GeoIP: Unknown]
>>>>>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
>>>>> Port: 5001 (5001), Seq: 1391553, Ack: 1, Len: 1448
>>>>>             Source port: 52697 (52697)
>>>>>             Destination port: 5001 (5001)
>>>>>             [Stream index: 0]
>>>>>             Sequence number: 1391553    (relative sequence number)
>>>>>             [Next sequence number: 1393001    (relative sequence number)]
>>>>>             Acknowledgment number: 1    (relative ack number)
>>>>>             Header length: 32 bytes
>>>>>             Flags: 0x010 (ACK)
>>>>>                 000. .... .... = Reserved: Not set
>>>>>                 ...0 .... .... = Nonce: Not set
>>>>>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>>>>>                 .... .0.. .... = ECN-Echo: Not set
>>>>>                 .... ..0. .... = Urgent: Not set
>>>>>                 .... ...1 .... = Acknowledgment: Set
>>>>>                 .... .... 0... = Push: Not set
>>>>>                 .... .... .0.. = Reset: Not set
>>>>>                 .... .... ..0. = Syn: Not set
>>>>>                 .... .... ...0 = Fin: Not set
>>>>>             Window size value: 8235
>>>>>             [Calculated window size: 131760]
>>>>>             [Window size scaling factor: 16]
>>>>>             Checksum: 0x9a16 [incorrect, should be 0x5913 (maybe
>>>>> caused by "TCP checksum offload"?)]
>>
>> And yes we are using checksum offload in ath10k.
>> Best is using standalone 80211ac sniffer for that case to be sure
>> about checksum.
>>
>>>>>                 [Good Checksum: False]
>>>>>                 [Bad Checksum: True]
>>>>>                     [Expert Info (Error/Checksum): Bad checksum]
>>>>>                         [Message: Bad checksum]
>>>>>                         [Severity level: Error]
>>>>>                         [Group: Checksum]
>>>>>             Options: (12 bytes), No-Operation (NOP), No-Operation
>>>>> (NOP), Timestamps
>>>>>                 No-Operation (NOP)
>>>>>                     Type: 1
>>>>>                         0... .... = Copy on fragmentation: No
>>>>>                         .00. .... = Class: Control (0)
>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>                 No-Operation (NOP)
>>>>>                     Type: 1
>>>>>                         0... .... = Copy on fragmentation: No
>>>>>                         .00. .... = Class: Control (0)
>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>                 Timestamps: TSval 1298580657, TSecr 4294947481
>>>>>                     Kind: Timestamp (8)
>>>>>                     Length: 10
>>>>>                     Timestamp value: 1298580657
>>>>>                     Timestamp echo reply: 4294947481
>>>>>         Data (1448 bytes)
>>>>>
>>>>> 0000  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0010  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0020  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0030  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0040  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0050  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0060  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0070  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0080  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0090  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 00a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 00b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 00c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 00d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 00e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 00f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0100  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0110  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0120  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0130  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0140  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0150  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0160  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0170  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0180  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0190  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 01a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 01b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 01c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 01d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
>>>>> 01f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0200  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0210  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0220  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0230  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0240  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0250  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0260  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0270  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0280  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0290  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 02a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 02b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 02c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 02d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 02e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 02f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0300  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0310  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0320  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0330  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0340  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0350  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0360  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0370  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0380  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0390  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 03a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 03b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 03c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 03d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 03e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 03f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0400  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0410  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0420  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0430  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0440  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0450  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0460  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0470  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0480  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0490  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 04a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 04b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 04c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 04d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 04e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 04f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0500  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0510  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0520  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0530  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0540  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 0550  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>> 0560  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>> 0570  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>> 0580  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>> 0590  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>> 05a0  32 33 34 35 36 37 38 39                           23456789
>>>>>             Data: 323334353637383930313233343536373839303132333435...
>>>>>             [Length: 1448]
>>>>>
>>>>> On Mon, Jul 7, 2014 at 12:26 PM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>> The initial results are not promising: a MacOS 802.11ac client gets
>>>>>> between 0-2 Mbps with this change, where it was getting about 8 Mbps
>>>>>> prior to this change and ~170 Mbps prior to the reordering fix. A pcap
>>>>>> from the receiving system shows a very large number of out of order
>>>>>> frames, likely due to TCP retransmission.
>>>>>>
>>>>>> An 802.11n MacBook gets very good throughput, only the 802.11ac
>>>>>> MacBook shows the very poor result. I'm trying to figure out why.
>>>>>>
>>>>>>
>>>>>> One specific note (and probably not related to the throughput): I
>>>>>> believe ath10k_htt_rx_amsdu runs in the tasklet, which means it would
>>>>>> need to use GFP_ATOMIC instead of GFP_KERNEL. Do I understand it
>>>>>> correctly?
>>>>>>
>>>>>>
>>>>>> On Mon, Jul 7, 2014 at 1:30 AM, Janusz Dziedzic
>>>>>> <janusz.dziedzic@tieto.com> wrote:
>>>>>>> On 6 July 2014 04:27, Adrian Chadd <adrian@freebsd.org> wrote:
>>>>>>>> I think you may have to tell mac80211 that it's okay and not to drop the frames.
>>>>>>>>
>>>>>>>> The earlier atheros chips would just give you the AMSDU frames as
>>>>>>>> deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
>>>>>>>> net80211 and it'd ull it apart. But if the driver is doing it (or,
>>>>>>>> well, the chip is doing it) then mac80211 needs to know not to drop
>>>>>>>> those sub-frames.
>>>>>>>>
>>>>>>>> I wonder if you'll ever get notified before the complete A-MPDU has
>>>>>>>> been received. That happens on previous chips. eg, you have an A-MPDU
>>>>>>>> of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
>>>>>>>> half of one MPDU before you hit EOL, the next notification you process
>>>>>>>> will be the next MSDU in the same MPDU - and then you'll hit the
>>>>>>>> reordering hilarity again.
>>>>>>>>
>>>>>>>> So hm, i will face the same issue in FreeBSD at some point, so I'd
>>>>>>>> likely do what you're thinking of doing - pass up a chain of mbufs
>>>>>>>> representing the current MPDU and treat the whole lot as the frame(s)
>>>>>>>> to care about. Honestly though, I'm kind of wondering whether I should
>>>>>>>> mostly just do what the Atheros reference driver does and treat it as
>>>>>>>> ethernet payload frames (ie, it's already de-encapsulated and the
>>>>>>>> reordering is already done) and just tack on the wifi header bit via
>>>>>>>> another of the DMA rings.
>>>>>>>>
>>>>>>>> Ugh, I really should sit down and write the FreeBSD version of this.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> -a
>>>>>>>>
>>>>>>>> (I'm still having flashbacks from working on this firmware at QCA. Aiee.)
>>>>>>>>
>>>>>>>>
>>>>>>>> On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>> There are two issues in handling the dis-aggregated A-MSDU subframes
>>>>>>>>> in ieee80211_sta_manage_reorder_buf:
>>>>>>>>>
>>>>>>>>> 1. The out-of-date check:
>>>>>>>>>
>>>>>>>>>         /* frame with out of date sequence number */
>>>>>>>>>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>>>>>>>>>                 dev_kfree_skb(skb);
>>>>>>>>>                 goto out;
>>>>>>>>>         }
>>>>>>>>>
>>>>>>>>> As all of the subframes carry the same sequence number, the first
>>>>>>>>> subframe will be delivered and increment head_seq_num and then all
>>>>>>>>> subsequent subframes will be discarded.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2. The duplicates check a bit later in the routine:
>>>>>>>>>
>>>>>>>>>         /* check if we already stored this frame */
>>>>>>>>>         if (tid_agg_rx->reorder_buf[index]) {
>>>>>>>>>                 dev_kfree_skb(skb);
>>>>>>>>>                 goto out;
>>>>>>>>>         }
>>>>>>>>>
>>>>>>>>> If there is enough packet loss that packets are queued in the reorder
>>>>>>>>> buffer and not immediately released, then only the first subframe will
>>>>>>>>> be stored. Subsequent subframes will be discarded as duplicates.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
>>>>>>>>> the reordering buffer changes, and dropped to about 8 Mbps with the
>>>>>>>>> reorder buffer. Hacking around the out-of-date sequence number check
>>>>>>>>> to allow all subframes to egress restores it to 170 Mbps.
>>>>>>>>>
>>>>>>>>> In the area where I'm testing there isn't enough 5 GHz noise to make
>>>>>>>>> the duplicates-check issue happen very often. By adding a printk I can
>>>>>>>>> see that it does happen, but it doesn't impact the throughput and I
>>>>>>>>> can't report the impact of fixing it.
>>>>>>>>>
>>>>>>>>> ----
>>>>>>>>>
>>>>>>>>> I do wonder if both of these are symptoms of dis-aggregating the
>>>>>>>>> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
>>>>>>>>> at a time, and the ath10k A-MSDU case is sending it subframes instead.
>>>>>>>>> Trying to make the ath10k code send up all of the subframes as a chain
>>>>>>>>> of skbs didn't immediately work, but I do wonder if that would better
>>>>>>>>> match the mac80211 expectations.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>>> Yes, after some more testing it does look like an unfortunate
>>>>>>>>>> interaction between the reorder buffer and A-MSDU. The disaggregated
>>>>>>>>>> subframes all carry the same sequence number. The first subframe is
>>>>>>>>>> released from the reorder buffer and increments the head_seq_num.
>>>>>>>>>> Subsequent subframes are all discarded as being out of date.
>>>>>>>>>>
>>>>>>>>>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>>>>>>>>>> head=0xb06
>>>>>>>>>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>>>>>>>>>> head=0xb0b
>>>>>>>>>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>>>>>>>>>> head=0xb10
>>>>>>>>>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>>>>>>>>>> head=0xb15
>>>>>>>>>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>>>>>>>>>> head=0xb1a
>>>>>>>>>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>>>>>>>>>> head=0xb1f
>>>>>>>>>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>>>>>>>>>> head=0xb24
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>>>>>>>>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>>>>>>>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>>>>>>>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>>>>>>>>>
>>>>>>>>>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>>>>>>>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>>>>>>>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>>>>>>>>>> second is consistently lost. The MacBook generally retransmits the
>>>>>>>>>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>>>>>>>>>> frame makes it through.
>>>>>>>>>>>>
>>>>>>>>>>>> This became more noticeable after the reordering fixes in
>>>>>>>>>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>>>>>>>>>
>>>>>>>>>>>> I see this A-MSDU packet loss behavior both with and without the
>>>>>>>>>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>>>>>>>>>> second is dropped. However, *without* the reordering fixes (and
>>>>>>>>>>>> therefore with packets delivered out of order) the MacBook sends
>>>>>>>>>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>>>>>>>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>>>>>>>>>> therefore has to deal with more packet loss. I suspect it is an
>>>>>>>>>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>>>>>>>>>> never going to fully understand, its stuck in a region of the
>>>>>>>>>>>> congestion window where the Wifi driver keeps choosing to using
>>>>>>>>>>>> A-MSDU.
>>>>>>>>>>>
>>>>>>>>>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>>>>>>>>>
>>>>>>>>>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>>>>>>>>>> copying/allocation overhead each subframe is reported as a singly
>>>>>>>>>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>>>>>>>>>> A-MPDU reordering intereferes with A-MSDU now?
>>>>>>>>>>>
>>>>>>> Denton could you try attached patch: report amsdu in one big frame.
>>>>>>> If help, we can add amsdu skb list support to mac80211/cfg80211 - to
>>>>>>> improve performance and reduce memcpy, while currently we have skb
>>>>>>> frames, put them in one big skb and next cfg80211 split them again
>>>>>>> into msdus and report to stack.
>>>>>>>
>>>>>>> BR
>>>>>>> Janusz

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Denton Gentry]

[-- Attachment #1: Type: text/plain, Size: 51307 bytes --]

I added this check for RX_ATTENTION_FLAGS_FCS_ERR in
ath10k_htt_rx_handler, along with a printk to verify that the code was
being run.

                        if (attention & RX_ATTENTION_FLAGS_FCS_ERR) {
                                ath10k_htt_rx_free_msdu_chain(msdu_head);
                                printk(KERN_WARNING
"ath10k_htt_rx_handler: amsdu RX_ATTENTION_FLAGS_FCS_ERR");
                                continue;
                        } else
                                rx_status->flag &= ~RX_FLAG_FAILED_FCS_CRC;

I see the printk appear in dmesg, so I know some packets are hitting
it. The throughput from the MacOS 802.11ac STA does improve, from 5-20
Mbps without to 50-60 Mbps with this change. Prior to adding the
reorder buffer fix, the MacBook was getting 170 Mbps.

However, packets with incorrect TCP checksums are still being
delivered to the Ethernet connected server at the other end. I
gathered pcaps at the MacBook, at the ath10k AP, on another ath10k
device configured in monitor mode as a sniffer, and at the
Ethernet-connected Ubuntu server at the other end.


server.pcap
Captured from the Ubuntu iperf server, connected to the AP via Ethernet.

Packet #10: TCP seq 4369, IP ID 0x6373, TCP checksum incorrect
At offset 208 bytes into the data we see “32 33 34 35” replaced by “98 19 1e b7”

Packet #17: TCP seq 4369, IP ID 0x6373, TCP checksum correct
Packet #23: TCP seq 4369, IP ID 0xe152, TCP checksum correct


ap.pcap
Captured from the AP.

Packet #10: TCP seq 4369, IP ID 0x6373, TCP checksum incorrect
At offset 208 bytes into the data we see “32 33 34 35” replaced by “98 19 1e b7”
This is the same pattern we see in the Ubuntu server's trace.

Packet #14: TCP seq 4369, IP ID 0x6373, TCP checksum correct
This was a Wifi-layer retransmit, which I think means the firmware
knew that something was wrong with packet #10. In the wifisniffer
trace, packet #19 is a Block ACK from the AP which requests one packet
be retransmitted.
However it appears that RX_ATTENTION_FLAGS_FCS_ERR was not set on
packet #10, so it was forwarded to the Ethernet.

Packet #24: TCP seq 4369, IP ID 0xe152, TCP checksum correct
This is a TCP-layer retransmit from the MacBook, as the IP ID is different.


wifisniffer.pcap
A second ath10 configured in monitor mode.
Packet #17 AMSDU#1: TCP seq 4369, IP ID 0x6373, TCP checksum correct
This is packet #10 in the AP trace. Note that the sniffer did not see
“98 19 1e b7” like the AP did.

Packet #20 AMSDU#1: TCP seq 4369, IP ID 0x6373, TCP checksum correct
This is packet #14 in the AP trace.

Packet #39 AMSDU#2: TCP seq 4369, IP ID 0xe152, TCP checksum correct
This is packet #24 in the AP trace.


macbook.pcapng
56: TCP seq 4369, IP ID 0x6373, TCP checksum correct
This is the original transmission. Wireshark on MacOS cannot put the
interface into monitor mode, so we do not see the Wifi-layer
retransmission of this packet.

65: TCP seq 4369, IP ID 0xe152, TCP checksum correct
This is a TCP retransmission, triggered because it received duplicate
TCP ACKs back from the Ubuntu server.


On Wed, Jul 9, 2014 at 12:39 AM, Janusz Dziedzic
<janusz.dziedzic@tieto.com> wrote:
> On 9 July 2014 08:09, Denton Gentry <denton.gentry@gmail.com> wrote:
>> I ran iperf on an 802.11ac MacBook Air, through my ath10 AP, and to an
>> Ubuntu system connected to the AP via Ethernet. The AP was running the
>> reorder buffer patches, the patch to make A-MSDU bypass the reorder
>> buffer, and the patch to make amsdu aggregation configurable via
>> debugfs.
>>
>> http://lists.infradead.org/pipermail/ath10k/2014-June/002551.html
>> http://lists.infradead.org/pipermail/ath10k/2014-June/002553.html
>> http://lists.infradead.org/pipermail/ath10k/2014-July/002597.html
>> http://permalink.gmane.org/gmane.linux.kernel.wireless.general/124128
>>
>> It was notably not running the patch from earlier in this thread to
>> copy A-MSDU subframes into one skb.
>>
>>
>> I used a second ath10k AP configured in monitor mode as a Wifi
>> sniffer. I also gathered pcaps on the MacBook Air and on the Ubuntu
>> system. Wireshark on the MacBook cannot gather radiotap headers nor
>> put the interface into monitor mode, so we see only Ethernet frames
>> and we don't see retransmissions on the Wifi link.
>>
>> I've attached decodes of the same two packets as captured by the
>> MacBook, by the sniffer, and by the Ubuntu server.
>>
>> The MacBook sends two frames, TCP sequence numbers 3065441 and 3066889
>> (the Wireshark "relative sequence number"). In the Wifi sniffer we see
>> these being aggregated as two subframes in an A-MSDU frame. The first
>> time this A-MSDU is sent it is corrupted in the air. We see the TCP
>> checksum of one of the subframes is wrong, and a few bytes of the
>> payload replaced with "&gt%'g9(1$".
>>
>> The A-MSDU is retransmitted a short time later, and the second time
>> the TCP checksum of both subframes is correct.
>>
>> This is all fine so far.
>>
>> However in the pcap taken from the Ubuntu server, we see TCP sequence
>> number 3065441 being delivered *twice.* The first time, the TCP
>> checksum is wrong. The second time the TCP checksum is correct.
>>
>> To me, it looks like A-MSDU frames with bad FCS are not being
>> discarded after ieee80211_rx_monitor(). The corrupted frames are being
>> delivered. Delivering the corrupted frames results in sending more TCP
>> Dup ACKs for the same sequence number back to the MacBook, and I think
>> this is what causes the MacBook to decide there is congestion and slow
>> down.
>>
> OK, to be sure this is the main issue we can just drop frames with wrong FCS.
> Could you check this:
>
> @@ -1267,10 +1267,10 @@ static void ath10k_htt_rx_handler(struct
> ath10k_htt *htt,
>                                 continue;
>                         }
>
> -                       if (attention & RX_ATTENTION_FLAGS_FCS_ERR)
> -                               rx_status->flag |= RX_FLAG_FAILED_FCS_CRC;
> -                       else
> -                               rx_status->flag &= ~RX_FLAG_FAILED_FCS_CRC;
> +                       if (attention & RX_ATTENTION_FLAGS_FCS_ERR) {
> +                               ath10k_htt_rx_free_msdu_chain(msdu_head);
> +                               continue;
> +                       }
>
>                         if (attention & RX_ATTENTION_FLAGS_TKIP_MIC_ERR)
>
> If will help we will have to check how to handle monitor + ap case
> correctly, one of idea is to not drop frames with wrong FCS only in
> case of standalone monitor mode.
> But, please try this patch to be sure.
>
> BR
> Janusz
>
>
>> One note: the wifi sniffer does not show the same corruption to the
>> same packet as the pcap from the Ubuntu system shows. I think that is
>> normal: the sniffer won't see precisely the same noise, won't have
>> precisely the same receive sensitivity, and its antennas are not
>> pointing in the same direction as the primary AP. If I do this test
>> again, I'll gather a pcap on the primary AP as well to compare to what
>> we see on the Ubuntu system.
>>
>>
>> On Tue, Jul 8, 2014 at 12:29 AM, Janusz Dziedzic
>> <janusz.dziedzic@tieto.com> wrote:
>>> On 8 July 2014 09:02, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:
>>>> On 8 July 2014 08:50, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:
>>>>> On 8 July 2014 08:43, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>> I think I know what is happening now, though I've no idea why. The
>>>>>> throughput is low because we have many TCP retransmissions. We have
>>>>>> retransmissions because the TCP checksum is wrong on a number of
>>>>>> frames, and I do find data corruption in the payload so the checksum
>>>>>> definitely should be wrong. All of the corrupted frames were
>>>>>> originally one of the subframes in an A-MSDU packet.
>>>>>>
>>>>>> An example follows at the end of this message, as dissected by
>>>>>> Wireshark. iperf sends a very regular data pattern of "0123456789..."
>>>>>> over and over. Note how in subframe #2 offset 0x1e0 the bytes "32 33
>>>>>> 34" have been replaced by "72 36 35"
>>>>>>
>>>>>> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
>>>>>>
>>>>>> I added printks at the bottom of ath10k_htt_rx_amsdu immediately
>>>>>> before the call to ath10k_process_rx. I found this same packet, and we
>>>>>> see the "72 36 35" corruption in the printk. So I think it happened in
>>>>>> ath10k_process_rx or before, not anything weird after passing it up to
>>>>>> mac80211.
>>>>>>
>>>>>> [  101.863712] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
>>>>>> [  101.863727] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
>>>>>> [  101.863742] ath10k: 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32
>>>>>> [  101.863757] ath10k: 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37 38
>>>>>> [  101.863773] ath10k: 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34
>>>>>> [  101.863788] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
>>>>>> [  101.863803] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
>>>>>>
>>>>>>
>>>>>> I've found a number of examples of similar corruption, always with
>>>>>> between one and four bytes replaced.
>>>>>>
>>>>>> 35363738 -> e52c6e07
>>>>>> 3435 -> b43f
>>>>>> 3839 -> c238
>>>>>> 31 -> 7f
>>>>>> 3435 -> 7436
>>>>>> 30 -> 50
>>>>>> 3233 -> bc37
>>>>>>
>>>>> Seems this could be because of:
>>>>>
>>>>> + /* cfg80211 expect this padding */
>>>>> + padding = (4 - (skb->len + sizeof(subframe_hdr))) & 0x3;
>>>>> + skb_put(skb, padding);
>>>>>
>>>>
>>>> BTW, when AP (ath10k) send TCP ACK - are this packets also AMSDU?
>>>> I see in my case AP can send 3 x A-MSDU (with total size 304 bytes)
>>>> small frames aggregated. Maybe your HW have problems with that.
>>>> As I remember correctly someone some time ago report problems with
>>>> MacBook pro retina but I am not sure this is the same, while no one
>>>> tests the fix.
>>>>
>>>>>>
>>>>>> The packet described above, dissected by Wireshark:
>>>>>>
>>>>>> No.     Time        Source                Destination
>>>>>> Protocol Length Info
>>>>>>    2235 18.953349   192.168.144.79        192.168.144.13        TCP
>>>>>>   3112   52697 > 5001 [ACK] Seq=1391553 Ack=1 Win=131760 [TCP CHECKSUM
>>>>>> INCORRECT] Len=1448 TSval=1298580657 TSecr=4294947481
>>>>>>
>>>>>> Frame 2235: 3112 bytes on wire (24896 bits), 3112 bytes captured (24896 bits)
>>>>>>     Encapsulation type: IEEE 802.11 plus radiotap radio header (23)
>>>>>>     Arrival Time: Jul  7, 2014 23:03:37.763365000 PDT
>>>>>>     [Time shift for this packet: 0.000000000 seconds]
>>>>>>     Epoch Time: 1404799417.763365000 seconds
>>>>>>     [Time delta from previous captured frame: 0.003476000 seconds]
>>>>>>     [Time delta from previous displayed frame: 0.515641000 seconds]
>>>>>>     [Time since reference or first frame: 18.953349000 seconds]
>>>>>>     Frame Number: 2235
>>>>>>     Frame Length: 3112 bytes (24896 bits)
>>>>>>     Capture Length: 3112 bytes (24896 bits)
>>>>>>     [Frame is marked: False]
>>>>>>     [Frame is ignored: False]
>>>>>>     [Protocols in frame: radiotap:wlan:llc:ip:tcp:data:llc:ip:tcp:data]
>>>>>>     [Coloring Rule Name: TCP]
>>>>>>     [Coloring Rule String: tcp]
>>>>>> Radiotap Header v0, Length 38
>>>>>>     Header revision: 0
>>>>>>     Header pad: 0
>>>>>>     Header length: 38
>>>>>>     Present flags
>>>>>>         .... .... .... .... .... .... .... ...1 = TSFT: True
>>>>>>         .... .... .... .... .... .... .... ..1. = Flags: True
>>>>>>         .... .... .... .... .... .... .... .0.. = Rate: False
>>>>>>         .... .... .... .... .... .... .... 1... = Channel: True
>>>>>>         .... .... .... .... .... .... ...0 .... = FHSS: False
>>>>>>         .... .... .... .... .... .... ..1. .... = dBm Antenna Signal: True
>>>>>>         .... .... .... .... .... .... .0.. .... = dBm Antenna Noise: False
>>>>>>         .... .... .... .... .... .... 0... .... = Lock Quality: False
>>>>>>         .... .... .... .... .... ...0 .... .... = TX Attenuation: False
>>>>>>         .... .... .... .... .... ..0. .... .... = dB TX Attenuation: False
>>>>>>         .... .... .... .... .... .0.. .... .... = dBm TX Power: False
>>>>>>         .... .... .... .... .... 1... .... .... = Antenna: True
>>>>>>         .... .... .... .... ...0 .... .... .... = dB Antenna Signal: False
>>>>>>         .... .... .... .... ..0. .... .... .... = dB Antenna Noise: False
>>>>>>         .... .... .... .... .1.. .... .... .... = RX flags: True
>>>>>>         .... .... .... .0.. .... .... .... .... = Channel+: False
>>>>>>         .... .... .... 0... .... .... .... .... = HT information: False
>>>>>>         .... .... ...0 .... .... .... .... .... = A-MPDU Status: False
>>>>>>         .... .... ..1. .... .... .... .... .... = VHT information: True
>>>>>>         ...0 0000 00.. .... .... .... .... .... = Reserved: 0x00000000
>>>>>>         ..0. .... .... .... .... .... .... .... = Radiotap NS next: False
>>>>>>         .0.. .... .... .... .... .... .... .... = Vendor NS next: False
>>>>>>         0... .... .... .... .... .... .... .... = Ext: False
>>>>>>     MAC timestamp: 78051063
>>>>>>     Flags: 0x00
>>>>>>         .... ...0 = CFP: False
>>>>>>         .... ..0. = Preamble: Long
>>>>>>         .... .0.. = WEP: False
>>>>>>         .... 0... = Fragmentation: False
>>>>>>         ...0 .... = FCS at end: False
>>>>>>         ..0. .... = Data Pad: False
>>>>>>         .0.. .... = Bad FCS: False
>>>>>>         0... .... = Short GI: False
>>>>>>     Channel frequency: 5745 [A 149]
>>>>>>     Channel type: 802.11a (0x0140)
>>>>>>         .... .... ...0 .... = Turbo: False
>>>>>>         .... .... ..0. .... = Complementary Code Keying (CCK): False
>>>>>>         .... .... .1.. .... = Orthogonal Frequency-Division
>>>>>> Multiplexing (OFDM): True
>>>>>>         .... .... 0... .... = 2 GHz spectrum: False
>>>>>>         .... ...1 .... .... = 5 GHz spectrum: True
>>>>>>         .... ..0. .... .... = Passive: False
>>>>>>         .... .0.. .... .... = Dynamic CCK-OFDM: False
>>>>>>         .... 0... .... .... = Gaussian Frequency Shift Keying (GFSK): False
>>>>>>         ...0 .... .... .... = GSM (900MHz): False
>>>>>>         ..0. .... .... .... = Static Turbo: False
>>>>>>         .0.. .... .... .... = Half Rate Channel (10MHz Channel Width): False
>>>>>>         0... .... .... .... = Quarter Rate Channel (5MHz Channel Width): False
>>>>>>     SSI Signal: -53 dBm
>>>>>>     Antenna: 0
>>>>>>     RX flags: 0x0000
>>>>>>         .... .... .... .... .... ..0. = Bad PLCP: False
>>>>>>     VHT information
>>>>>>         Known VHT information: 0x44
>>>>>>             .... .... .... ...0 = STBC: False
>>>>>>             .... .... .... ..0. = TXOP_PS_NOT_ALLOWED: False
>>>>>>             .... .... .... .1.. = Guard interval: True
>>>>>>             .... .... .... 0... = SGI Nsym disambiguation: False
>>>>>>             .... .... ...0 .... = LDPC extra OFDM symbol: False
>>>>>>             .... .... ..0. .... = Beamformed: False
>>>>>>             .... .... .1.. .... = Bandwidth: True
>>>>>>             .... .... 0... .... = Group ID: False
>>>>>>             .... ...0 .... .... = Partial AID: False
>>>>>>         .... .0.. = Guard interval: long (0)
>>>>>>         Bandwidth: 80 MHz (4)
>>>>>>         User 0: MCS 8
>>>>>>             1000 .... = MCS index 0: 8 (256-QAM 3/4)
>>>>>>             .... 0010 = Spatial streams 0: 2
>>>>>>             Space-time streams 0: 2
>>>>>>             Coding 0: BCC (0)
>>>>>>             [Data Rate: 702.0 Mb/s]
>>>>>> IEEE 802.11 QoS Data, Flags: .......T
>>>>>>     Type/Subtype: QoS Data (0x28)
>>>>>>     Frame Control Field: 0x8801
>>>>>>         .... ..00 = Version: 0
>>>>>>         .... 10.. = Type: Data frame (2)
>>>>>>         1000 .... = Subtype: 8
>>>>>>         Flags: 0x01
>>>>>>             .... ..01 = DS status: Frame from STA to DS via an AP (To
>>>>>> DS: 1 From DS: 0) (0x01)
>>>>>>             .... .0.. = More Fragments: This is the last fragment
>>>>>>             .... 0... = Retry: Frame is not being retransmitted
>>>>>>             ...0 .... = PWR MGT: STA will stay up
>>>>>>             ..0. .... = More Data: No data buffered
>>>>>>             .0.. .... = Protected flag: Data is not protected
>>>>>>             0... .... = Order flag: Not strictly ordered
>>>>>>     .000 0000 0011 0000 = Duration: 48 microseconds
>>>>>>     Receiver address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>>>>     BSS Id: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>>>>     Transmitter address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>>     Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>>     Destination address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>>>>     Fragment number: 0
>>>>>>     Sequence number: 1021
>>>>>>     Qos Control: 0x0080
>>>>>>         .... .... .... 0000 = TID: 0
>>>>>>         [.... .... .... .000 = Priority: Best Effort (Best Effort) (0)]
>>>>>>         .... .... ...0 .... = QoS bit 4: Bits 8-15 of QoS Control
>>>>>> field are TXOP Duration Requested
>>>>>>         .... .... .00. .... = Ack Policy: Normal Ack (0x0000)
>>>>>>         .... .... 1... .... = Payload Type: A-MSDU
>>>>>>         0000 0000 .... .... = TXOP Duration Requested: 0 (no TXOP requested)
>>>>>> IEEE 802.11 Aggregate MSDU
>>>>>>     A-MSDU Subframe #1
>>>>>>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>>>>>>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>>         A-MSDU Length: 1510
>>>>>>         Logical-Link Control
>>>>>>             DSAP: SNAP (0xaa)
>>>>>>             IG Bit: Individual
>>>>>>             SSAP: SNAP (0xaa)
>>>>>>             CR Bit: Command
>>>>>>             Control field: U, func=UI (0x03)
>>>>>>                 000. 00.. = Command: Unnumbered Information (0x00)
>>>>>>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>>>>>>             Organization Code: Encapsulated Ethernet (0x000000)
>>>>>>             Type: IP (0x0800)
>>>>>>         Internet Protocol Version 4, Src: 192.168.144.79
>>>>>> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>>>>>>             Version: 4
>>>>>>             Header length: 20 bytes
>>>>>>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
>>>>>> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>>>>>>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>>>>>>                 .... ..00 = Explicit Congestion Notification: Not-ECT
>>>>>> (Not ECN-Capable Transport) (0x00)
>>>>>>             Total Length: 1500
>>>>>>             Identification: 0xc622 (50722)
>>>>>>             Flags: 0x00
>>>>>>                 0... .... = Reserved bit: Not set
>>>>>>                 .0.. .... = Don't fragment: Not set
>>>>>>                 ..0. .... = More fragments: Not set
>>>>>>             Fragment offset: 0
>>>>>>             Time to live: 64
>>>>>>             Protocol: TCP (6)
>>>>>>             Header checksum: 0x0d4c [correct]
>>>>>>                 [Good: True]
>>>>>>                 [Bad: False]
>>>>>>             Source: 192.168.144.79 (192.168.144.79)
>>>>>>             Destination: 192.168.144.13 (192.168.144.13)
>>>>>>             [Source GeoIP: Unknown]
>>>>>>             [Destination GeoIP: Unknown]
>>>>>>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
>>>>>> Port: 5001 (5001), Seq: 1390105, Ack: 1, Len: 1448
>>>>>>             Source port: 52697 (52697)
>>>>>>             Destination port: 5001 (5001)
>>>>>>             [Stream index: 0]
>>>>>>             Sequence number: 1390105    (relative sequence number)
>>>>>>             [Next sequence number: 1391553    (relative sequence number)]
>>>>>>             Acknowledgment number: 1    (relative ack number)
>>>>>>             Header length: 32 bytes
>>>>>>             Flags: 0x010 (ACK)
>>>>>>                 000. .... .... = Reserved: Not set
>>>>>>                 ...0 .... .... = Nonce: Not set
>>>>>>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>>>>>>                 .... .0.. .... = ECN-Echo: Not set
>>>>>>                 .... ..0. .... = Urgent: Not set
>>>>>>                 .... ...1 .... = Acknowledgment: Set
>>>>>>                 .... .... 0... = Push: Not set
>>>>>>                 .... .... .0.. = Reset: Not set
>>>>>>                 .... .... ..0. = Syn: Not set
>>>>>>                 .... .... ...0 = Fin: Not set
>>>>>>             Window size value: 8235
>>>>>>             [Calculated window size: 131760]
>>>>>>             [Window size scaling factor: 16]
>>>>>>             Checksum: 0xa1c0 [correct]
>>>>>>                 [Good Checksum: True]
>>>>>>                 [Bad Checksum: False]
>>>>>>             Options: (12 bytes), No-Operation (NOP), No-Operation
>>>>>> (NOP), Timestamps
>>>>>>                 No-Operation (NOP)
>>>>>>                     Type: 1
>>>>>>                         0... .... = Copy on fragmentation: No
>>>>>>                         .00. .... = Class: Control (0)
>>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>>                 No-Operation (NOP)
>>>>>>                     Type: 1
>>>>>>                         0... .... = Copy on fragmentation: No
>>>>>>                         .00. .... = Class: Control (0)
>>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>>                 Timestamps: TSval 1298580657, TSecr 4294947481
>>>>>>                     Kind: Timestamp (8)
>>>>>>                     Length: 10
>>>>>>                     Timestamp value: 1298580657
>>>>>>                     Timestamp echo reply: 4294947481
>>>>>>         Data (1448 bytes)
>>>>>>
>>>>>> 0000  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0010  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0020  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0030  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0040  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0050  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0060  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0070  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0080  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0090  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 00a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 00b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 00c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 00d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 00e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 00f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0100  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0110  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0120  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0130  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0140  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0150  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0160  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0170  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0180  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0190  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 01a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 01b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 01c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 01d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 01e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 01f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0200  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0210  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0220  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0230  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0240  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0250  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0260  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0270  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0280  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0290  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 02a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 02b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 02c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 02d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 02e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 02f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0300  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0310  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0320  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0330  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0340  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0350  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0360  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0370  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0380  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0390  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 03a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 03b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 03c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 03d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 03e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 03f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0400  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0410  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0420  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0430  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0440  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0450  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0460  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0470  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0480  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0490  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 04a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 04b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 04c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 04d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 04e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 04f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0500  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0510  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0520  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0530  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0540  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0550  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0560  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0570  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0580  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0590  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 05a0  34 35 36 37 38 39 30 31                           45678901
>>>>>>             Data: 343536373839303132333435363738393031323334353637...
>>>>>>             [Length: 1448]
>>>>>>     A-MSDU Subframe #2
>>>>>>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>>>>>>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>>         A-MSDU Length: 1510
>>>>>>         Logical-Link Control
>>>>>>             DSAP: SNAP (0xaa)
>>>>>>             IG Bit: Individual
>>>>>>             SSAP: SNAP (0xaa)
>>>>>>             CR Bit: Command
>>>>>>             Control field: U, func=UI (0x03)
>>>>>>                 000. 00.. = Command: Unnumbered Information (0x00)
>>>>>>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>>>>>>             Organization Code: Encapsulated Ethernet (0x000000)
>>>>>>             Type: IP (0x0800)
>>>>>>         Internet Protocol Version 4, Src: 192.168.144.79
>>>>>> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>>>>>>             Version: 4
>>>>>>             Header length: 20 bytes
>>>>>>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
>>>>>> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>>>>>>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>>>>>>                 .... ..00 = Explicit Congestion Notification: Not-ECT
>>>>>> (Not ECN-Capable Transport) (0x00)
>>>>>>             Total Length: 1500
>>>>>>             Identification: 0xda09 (55817)
>>>>>>             Flags: 0x00
>>>>>>                 0... .... = Reserved bit: Not set
>>>>>>                 .0.. .... = Don't fragment: Not set
>>>>>>                 ..0. .... = More fragments: Not set
>>>>>>             Fragment offset: 0
>>>>>>             Time to live: 64
>>>>>>             Protocol: TCP (6)
>>>>>>             Header checksum: 0xf964 [correct]
>>>>>>                 [Good: True]
>>>>>>                 [Bad: False]
>>>>>>             Source: 192.168.144.79 (192.168.144.79)
>>>>>>             Destination: 192.168.144.13 (192.168.144.13)
>>>>>>             [Source GeoIP: Unknown]
>>>>>>             [Destination GeoIP: Unknown]
>>>>>>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
>>>>>> Port: 5001 (5001), Seq: 1391553, Ack: 1, Len: 1448
>>>>>>             Source port: 52697 (52697)
>>>>>>             Destination port: 5001 (5001)
>>>>>>             [Stream index: 0]
>>>>>>             Sequence number: 1391553    (relative sequence number)
>>>>>>             [Next sequence number: 1393001    (relative sequence number)]
>>>>>>             Acknowledgment number: 1    (relative ack number)
>>>>>>             Header length: 32 bytes
>>>>>>             Flags: 0x010 (ACK)
>>>>>>                 000. .... .... = Reserved: Not set
>>>>>>                 ...0 .... .... = Nonce: Not set
>>>>>>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>>>>>>                 .... .0.. .... = ECN-Echo: Not set
>>>>>>                 .... ..0. .... = Urgent: Not set
>>>>>>                 .... ...1 .... = Acknowledgment: Set
>>>>>>                 .... .... 0... = Push: Not set
>>>>>>                 .... .... .0.. = Reset: Not set
>>>>>>                 .... .... ..0. = Syn: Not set
>>>>>>                 .... .... ...0 = Fin: Not set
>>>>>>             Window size value: 8235
>>>>>>             [Calculated window size: 131760]
>>>>>>             [Window size scaling factor: 16]
>>>>>>             Checksum: 0x9a16 [incorrect, should be 0x5913 (maybe
>>>>>> caused by "TCP checksum offload"?)]
>>>
>>> And yes we are using checksum offload in ath10k.
>>> Best is using standalone 80211ac sniffer for that case to be sure
>>> about checksum.
>>>
>>>>>>                 [Good Checksum: False]
>>>>>>                 [Bad Checksum: True]
>>>>>>                     [Expert Info (Error/Checksum): Bad checksum]
>>>>>>                         [Message: Bad checksum]
>>>>>>                         [Severity level: Error]
>>>>>>                         [Group: Checksum]
>>>>>>             Options: (12 bytes), No-Operation (NOP), No-Operation
>>>>>> (NOP), Timestamps
>>>>>>                 No-Operation (NOP)
>>>>>>                     Type: 1
>>>>>>                         0... .... = Copy on fragmentation: No
>>>>>>                         .00. .... = Class: Control (0)
>>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>>                 No-Operation (NOP)
>>>>>>                     Type: 1
>>>>>>                         0... .... = Copy on fragmentation: No
>>>>>>                         .00. .... = Class: Control (0)
>>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>>                 Timestamps: TSval 1298580657, TSecr 4294947481
>>>>>>                     Kind: Timestamp (8)
>>>>>>                     Length: 10
>>>>>>                     Timestamp value: 1298580657
>>>>>>                     Timestamp echo reply: 4294947481
>>>>>>         Data (1448 bytes)
>>>>>>
>>>>>> 0000  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0010  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0020  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0030  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0040  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0050  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0060  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0070  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0080  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0090  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 00a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 00b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 00c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 00d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 00e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 00f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0100  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0110  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0120  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0130  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0140  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0150  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0160  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0170  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0180  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0190  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 01a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 01b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 01c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 01d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
>>>>>> 01f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0200  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0210  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0220  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0230  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0240  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0250  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0260  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0270  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0280  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0290  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 02a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 02b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 02c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 02d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 02e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 02f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0300  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0310  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0320  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0330  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0340  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0350  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0360  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0370  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0380  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0390  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 03a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 03b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 03c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 03d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 03e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 03f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0400  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0410  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0420  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0430  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0440  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0450  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0460  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0470  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0480  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0490  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 04a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 04b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 04c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 04d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 04e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 04f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0500  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0510  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0520  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0530  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0540  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 0550  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>> 0560  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>> 0570  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>> 0580  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>> 0590  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>> 05a0  32 33 34 35 36 37 38 39                           23456789
>>>>>>             Data: 323334353637383930313233343536373839303132333435...
>>>>>>             [Length: 1448]
>>>>>>
>>>>>> On Mon, Jul 7, 2014 at 12:26 PM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>> The initial results are not promising: a MacOS 802.11ac client gets
>>>>>>> between 0-2 Mbps with this change, where it was getting about 8 Mbps
>>>>>>> prior to this change and ~170 Mbps prior to the reordering fix. A pcap
>>>>>>> from the receiving system shows a very large number of out of order
>>>>>>> frames, likely due to TCP retransmission.
>>>>>>>
>>>>>>> An 802.11n MacBook gets very good throughput, only the 802.11ac
>>>>>>> MacBook shows the very poor result. I'm trying to figure out why.
>>>>>>>
>>>>>>>
>>>>>>> One specific note (and probably not related to the throughput): I
>>>>>>> believe ath10k_htt_rx_amsdu runs in the tasklet, which means it would
>>>>>>> need to use GFP_ATOMIC instead of GFP_KERNEL. Do I understand it
>>>>>>> correctly?
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jul 7, 2014 at 1:30 AM, Janusz Dziedzic
>>>>>>> <janusz.dziedzic@tieto.com> wrote:
>>>>>>>> On 6 July 2014 04:27, Adrian Chadd <adrian@freebsd.org> wrote:
>>>>>>>>> I think you may have to tell mac80211 that it's okay and not to drop the frames.
>>>>>>>>>
>>>>>>>>> The earlier atheros chips would just give you the AMSDU frames as
>>>>>>>>> deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
>>>>>>>>> net80211 and it'd ull it apart. But if the driver is doing it (or,
>>>>>>>>> well, the chip is doing it) then mac80211 needs to know not to drop
>>>>>>>>> those sub-frames.
>>>>>>>>>
>>>>>>>>> I wonder if you'll ever get notified before the complete A-MPDU has
>>>>>>>>> been received. That happens on previous chips. eg, you have an A-MPDU
>>>>>>>>> of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
>>>>>>>>> half of one MPDU before you hit EOL, the next notification you process
>>>>>>>>> will be the next MSDU in the same MPDU - and then you'll hit the
>>>>>>>>> reordering hilarity again.
>>>>>>>>>
>>>>>>>>> So hm, i will face the same issue in FreeBSD at some point, so I'd
>>>>>>>>> likely do what you're thinking of doing - pass up a chain of mbufs
>>>>>>>>> representing the current MPDU and treat the whole lot as the frame(s)
>>>>>>>>> to care about. Honestly though, I'm kind of wondering whether I should
>>>>>>>>> mostly just do what the Atheros reference driver does and treat it as
>>>>>>>>> ethernet payload frames (ie, it's already de-encapsulated and the
>>>>>>>>> reordering is already done) and just tack on the wifi header bit via
>>>>>>>>> another of the DMA rings.
>>>>>>>>>
>>>>>>>>> Ugh, I really should sit down and write the FreeBSD version of this.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -a
>>>>>>>>>
>>>>>>>>> (I'm still having flashbacks from working on this firmware at QCA. Aiee.)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>>> There are two issues in handling the dis-aggregated A-MSDU subframes
>>>>>>>>>> in ieee80211_sta_manage_reorder_buf:
>>>>>>>>>>
>>>>>>>>>> 1. The out-of-date check:
>>>>>>>>>>
>>>>>>>>>>         /* frame with out of date sequence number */
>>>>>>>>>>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>>>>>>>>>>                 dev_kfree_skb(skb);
>>>>>>>>>>                 goto out;
>>>>>>>>>>         }
>>>>>>>>>>
>>>>>>>>>> As all of the subframes carry the same sequence number, the first
>>>>>>>>>> subframe will be delivered and increment head_seq_num and then all
>>>>>>>>>> subsequent subframes will be discarded.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2. The duplicates check a bit later in the routine:
>>>>>>>>>>
>>>>>>>>>>         /* check if we already stored this frame */
>>>>>>>>>>         if (tid_agg_rx->reorder_buf[index]) {
>>>>>>>>>>                 dev_kfree_skb(skb);
>>>>>>>>>>                 goto out;
>>>>>>>>>>         }
>>>>>>>>>>
>>>>>>>>>> If there is enough packet loss that packets are queued in the reorder
>>>>>>>>>> buffer and not immediately released, then only the first subframe will
>>>>>>>>>> be stored. Subsequent subframes will be discarded as duplicates.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
>>>>>>>>>> the reordering buffer changes, and dropped to about 8 Mbps with the
>>>>>>>>>> reorder buffer. Hacking around the out-of-date sequence number check
>>>>>>>>>> to allow all subframes to egress restores it to 170 Mbps.
>>>>>>>>>>
>>>>>>>>>> In the area where I'm testing there isn't enough 5 GHz noise to make
>>>>>>>>>> the duplicates-check issue happen very often. By adding a printk I can
>>>>>>>>>> see that it does happen, but it doesn't impact the throughput and I
>>>>>>>>>> can't report the impact of fixing it.
>>>>>>>>>>
>>>>>>>>>> ----
>>>>>>>>>>
>>>>>>>>>> I do wonder if both of these are symptoms of dis-aggregating the
>>>>>>>>>> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
>>>>>>>>>> at a time, and the ath10k A-MSDU case is sending it subframes instead.
>>>>>>>>>> Trying to make the ath10k code send up all of the subframes as a chain
>>>>>>>>>> of skbs didn't immediately work, but I do wonder if that would better
>>>>>>>>>> match the mac80211 expectations.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>>>> Yes, after some more testing it does look like an unfortunate
>>>>>>>>>>> interaction between the reorder buffer and A-MSDU. The disaggregated
>>>>>>>>>>> subframes all carry the same sequence number. The first subframe is
>>>>>>>>>>> released from the reorder buffer and increments the head_seq_num.
>>>>>>>>>>> Subsequent subframes are all discarded as being out of date.
>>>>>>>>>>>
>>>>>>>>>>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>>>>>>>>>>> head=0xb06
>>>>>>>>>>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>>>>>>>>>>> head=0xb0b
>>>>>>>>>>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>>>>>>>>>>> head=0xb10
>>>>>>>>>>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>>>>>>>>>>> head=0xb15
>>>>>>>>>>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>>>>>>>>>>> head=0xb1a
>>>>>>>>>>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>>>>>>>>>>> head=0xb1f
>>>>>>>>>>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>>>>>>>>>>> head=0xb24
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>>>>>>>>>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>>>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>>>>>>>>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>>>>>>>>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>>>>>>>>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>>>>>>>>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>>>>>>>>>>> second is consistently lost. The MacBook generally retransmits the
>>>>>>>>>>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>>>>>>>>>>> frame makes it through.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This became more noticeable after the reordering fixes in
>>>>>>>>>>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>>>>>>>>>>
>>>>>>>>>>>>> I see this A-MSDU packet loss behavior both with and without the
>>>>>>>>>>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>>>>>>>>>>> second is dropped. However, *without* the reordering fixes (and
>>>>>>>>>>>>> therefore with packets delivered out of order) the MacBook sends
>>>>>>>>>>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>>>>>>>>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>>>>>>>>>>> therefore has to deal with more packet loss. I suspect it is an
>>>>>>>>>>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>>>>>>>>>>> never going to fully understand, its stuck in a region of the
>>>>>>>>>>>>> congestion window where the Wifi driver keeps choosing to using
>>>>>>>>>>>>> A-MSDU.
>>>>>>>>>>>>
>>>>>>>>>>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>>>>>>>>>>
>>>>>>>>>>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>>>>>>>>>>> copying/allocation overhead each subframe is reported as a singly
>>>>>>>>>>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>>>>>>>>>>> A-MPDU reordering intereferes with A-MSDU now?
>>>>>>>>>>>>
>>>>>>>> Denton could you try attached patch: report amsdu in one big frame.
>>>>>>>> If help, we can add amsdu skb list support to mac80211/cfg80211 - to
>>>>>>>> improve performance and reduce memcpy, while currently we have skb
>>>>>>>> frames, put them in one big skb and next cfg80211 split them again
>>>>>>>> into msdus and report to stack.
>>>>>>>>
>>>>>>>> BR
>>>>>>>> Janusz

[-- Attachment #2: ap.pcap --]
[-- Type: application/octet-stream, Size: 35655 bytes --]

[-- Attachment #3: macbook.pcap --]
[-- Type: application/octet-stream, Size: 39182 bytes --]

[-- Attachment #4: server.pcap --]
[-- Type: application/octet-stream, Size: 24744 bytes --]

[-- Attachment #5: wifisniffer.pcap --]
[-- Type: application/octet-stream, Size: 47148 bytes --]

[-- Attachment #6: Type: text/plain, Size: 146 bytes --]

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Janusz Dziedzic]

On 10 July 2014 15:40, Denton Gentry <denton.gentry@gmail.com> wrote:
> I added this check for RX_ATTENTION_FLAGS_FCS_ERR in
> ath10k_htt_rx_handler, along with a printk to verify that the code was
> being run.
>
>                         if (attention & RX_ATTENTION_FLAGS_FCS_ERR) {
>                                 ath10k_htt_rx_free_msdu_chain(msdu_head);
>                                 printk(KERN_WARNING
> "ath10k_htt_rx_handler: amsdu RX_ATTENTION_FLAGS_FCS_ERR");
>                                 continue;
>                         } else
>                                 rx_status->flag &= ~RX_FLAG_FAILED_FCS_CRC;
>
> I see the printk appear in dmesg, so I know some packets are hitting
> it. The throughput from the MacOS 802.11ac STA does improve, from 5-20
> Mbps without to 50-60 Mbps with this change. Prior to adding the
> reorder buffer fix, the MacBook was getting 170 Mbps.
>
> However, packets with incorrect TCP checksums are still being
> delivered to the Ethernet connected server at the other end. I
> gathered pcaps at the MacBook, at the ath10k AP, on another ath10k
> device configured in monitor mode as a sniffer, and at the
> Ethernet-connected Ubuntu server at the other end.
>
Thanks for logs.

This is strange, this suggest MPDU have correct FCS, and only data in
MSDU are incorrect.
Seems like a FW/HW bug, while FCS was correct and other sniffer see
correct frame.
Seems also like not driver issue while this incorrect data are at
offset ~0x14C. In driver we touch mostly head and  tail of the data.
FW report IP/TCP checksum - interesting what status you get for such packet.
This is line:
skb->ip_summed = ath10k_htt_rx_get_csum_state(skb);

I see also this is QAM256 (5/6) - but FCS was correct so this suggest
MPDU frame was received correctly.
Interesting issue, I am curious who and when insert this wrong data.
Will check this more tomorrow.

BTW - this patch below is wrong, when we will start reorder we will
deliver only one MSDU - last one from AMSDU - instead of all, and all
with MORE flag will go directly to stack not waiting for reorder
decision. So could introduce other problems.

+       /* not impelemented correctly when driver report separete A-MSDUs */
+        if (status->flag & RX_FLAG_AMSDU_MORE)
+               goto dont_reorder;
+

So we have 3 options in case of using mac80211 reordering:
1) use big frame (memcpy overhead)
2) report amsd as a skb list (not sure mac80211 patches will be merged
- under discussion)
3) rewrite mac80211 reordering code to support AMSDU_MORE correctly

Other option can be, put reordering code in the driver (table for each
mpdu for peer and for each tid - resources required - while we have
almost the same allocated in mac80211 already)

BR
Janusz

>
> server.pcap
> Captured from the Ubuntu iperf server, connected to the AP via Ethernet.
>
> Packet #10: TCP seq 4369, IP ID 0x6373, TCP checksum incorrect
> At offset 208 bytes into the data we see “32 33 34 35” replaced by “98 19 1e b7”
>
> Packet #17: TCP seq 4369, IP ID 0x6373, TCP checksum correct
> Packet #23: TCP seq 4369, IP ID 0xe152, TCP checksum correct
>
>
> ap.pcap
> Captured from the AP.
>
> Packet #10: TCP seq 4369, IP ID 0x6373, TCP checksum incorrect
> At offset 208 bytes into the data we see “32 33 34 35” replaced by “98 19 1e b7”
> This is the same pattern we see in the Ubuntu server's trace.
>
> Packet #14: TCP seq 4369, IP ID 0x6373, TCP checksum correct
> This was a Wifi-layer retransmit, which I think means the firmware
> knew that something was wrong with packet #10. In the wifisniffer
> trace, packet #19 is a Block ACK from the AP which requests one packet
> be retransmitted.
> However it appears that RX_ATTENTION_FLAGS_FCS_ERR was not set on
> packet #10, so it was forwarded to the Ethernet.
>
> Packet #24: TCP seq 4369, IP ID 0xe152, TCP checksum correct
> This is a TCP-layer retransmit from the MacBook, as the IP ID is different.
>
>
> wifisniffer.pcap
> A second ath10 configured in monitor mode.
> Packet #17 AMSDU#1: TCP seq 4369, IP ID 0x6373, TCP checksum correct
> This is packet #10 in the AP trace. Note that the sniffer did not see
> “98 19 1e b7” like the AP did.
>
> Packet #20 AMSDU#1: TCP seq 4369, IP ID 0x6373, TCP checksum correct
> This is packet #14 in the AP trace.
>
> Packet #39 AMSDU#2: TCP seq 4369, IP ID 0xe152, TCP checksum correct
> This is packet #24 in the AP trace.
>
>
> macbook.pcapng
> 56: TCP seq 4369, IP ID 0x6373, TCP checksum correct
> This is the original transmission. Wireshark on MacOS cannot put the
> interface into monitor mode, so we do not see the Wifi-layer
> retransmission of this packet.
>
> 65: TCP seq 4369, IP ID 0xe152, TCP checksum correct
> This is a TCP retransmission, triggered because it received duplicate
> TCP ACKs back from the Ubuntu server.
>
>
> On Wed, Jul 9, 2014 at 12:39 AM, Janusz Dziedzic
> <janusz.dziedzic@tieto.com> wrote:
>> On 9 July 2014 08:09, Denton Gentry <denton.gentry@gmail.com> wrote:
>>> I ran iperf on an 802.11ac MacBook Air, through my ath10 AP, and to an
>>> Ubuntu system connected to the AP via Ethernet. The AP was running the
>>> reorder buffer patches, the patch to make A-MSDU bypass the reorder
>>> buffer, and the patch to make amsdu aggregation configurable via
>>> debugfs.
>>>
>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002551.html
>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002553.html
>>> http://lists.infradead.org/pipermail/ath10k/2014-July/002597.html
>>> http://permalink.gmane.org/gmane.linux.kernel.wireless.general/124128
>>>
>>> It was notably not running the patch from earlier in this thread to
>>> copy A-MSDU subframes into one skb.
>>>
>>>
>>> I used a second ath10k AP configured in monitor mode as a Wifi
>>> sniffer. I also gathered pcaps on the MacBook Air and on the Ubuntu
>>> system. Wireshark on the MacBook cannot gather radiotap headers nor
>>> put the interface into monitor mode, so we see only Ethernet frames
>>> and we don't see retransmissions on the Wifi link.
>>>
>>> I've attached decodes of the same two packets as captured by the
>>> MacBook, by the sniffer, and by the Ubuntu server.
>>>
>>> The MacBook sends two frames, TCP sequence numbers 3065441 and 3066889
>>> (the Wireshark "relative sequence number"). In the Wifi sniffer we see
>>> these being aggregated as two subframes in an A-MSDU frame. The first
>>> time this A-MSDU is sent it is corrupted in the air. We see the TCP
>>> checksum of one of the subframes is wrong, and a few bytes of the
>>> payload replaced with "&gt%'g9(1$".
>>>
>>> The A-MSDU is retransmitted a short time later, and the second time
>>> the TCP checksum of both subframes is correct.
>>>
>>> This is all fine so far.
>>>
>>> However in the pcap taken from the Ubuntu server, we see TCP sequence
>>> number 3065441 being delivered *twice.* The first time, the TCP
>>> checksum is wrong. The second time the TCP checksum is correct.
>>>
>>> To me, it looks like A-MSDU frames with bad FCS are not being
>>> discarded after ieee80211_rx_monitor(). The corrupted frames are being
>>> delivered. Delivering the corrupted frames results in sending more TCP
>>> Dup ACKs for the same sequence number back to the MacBook, and I think
>>> this is what causes the MacBook to decide there is congestion and slow
>>> down.
>>>
>> OK, to be sure this is the main issue we can just drop frames with wrong FCS.
>> Could you check this:
>>
>> @@ -1267,10 +1267,10 @@ static void ath10k_htt_rx_handler(struct
>> ath10k_htt *htt,
>>                                 continue;
>>                         }
>>
>> -                       if (attention & RX_ATTENTION_FLAGS_FCS_ERR)
>> -                               rx_status->flag |= RX_FLAG_FAILED_FCS_CRC;
>> -                       else
>> -                               rx_status->flag &= ~RX_FLAG_FAILED_FCS_CRC;
>> +                       if (attention & RX_ATTENTION_FLAGS_FCS_ERR) {
>> +                               ath10k_htt_rx_free_msdu_chain(msdu_head);
>> +                               continue;
>> +                       }
>>
>>                         if (attention & RX_ATTENTION_FLAGS_TKIP_MIC_ERR)
>>
>> If will help we will have to check how to handle monitor + ap case
>> correctly, one of idea is to not drop frames with wrong FCS only in
>> case of standalone monitor mode.
>> But, please try this patch to be sure.
>>
>> BR
>> Janusz
>>
>>
>>> One note: the wifi sniffer does not show the same corruption to the
>>> same packet as the pcap from the Ubuntu system shows. I think that is
>>> normal: the sniffer won't see precisely the same noise, won't have
>>> precisely the same receive sensitivity, and its antennas are not
>>> pointing in the same direction as the primary AP. If I do this test
>>> again, I'll gather a pcap on the primary AP as well to compare to what
>>> we see on the Ubuntu system.
>>>
>>>
>>> On Tue, Jul 8, 2014 at 12:29 AM, Janusz Dziedzic
>>> <janusz.dziedzic@tieto.com> wrote:
>>>> On 8 July 2014 09:02, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:
>>>>> On 8 July 2014 08:50, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:
>>>>>> On 8 July 2014 08:43, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>> I think I know what is happening now, though I've no idea why. The
>>>>>>> throughput is low because we have many TCP retransmissions. We have
>>>>>>> retransmissions because the TCP checksum is wrong on a number of
>>>>>>> frames, and I do find data corruption in the payload so the checksum
>>>>>>> definitely should be wrong. All of the corrupted frames were
>>>>>>> originally one of the subframes in an A-MSDU packet.
>>>>>>>
>>>>>>> An example follows at the end of this message, as dissected by
>>>>>>> Wireshark. iperf sends a very regular data pattern of "0123456789..."
>>>>>>> over and over. Note how in subframe #2 offset 0x1e0 the bytes "32 33
>>>>>>> 34" have been replaced by "72 36 35"
>>>>>>>
>>>>>>> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
>>>>>>>
>>>>>>> I added printks at the bottom of ath10k_htt_rx_amsdu immediately
>>>>>>> before the call to ath10k_process_rx. I found this same packet, and we
>>>>>>> see the "72 36 35" corruption in the printk. So I think it happened in
>>>>>>> ath10k_process_rx or before, not anything weird after passing it up to
>>>>>>> mac80211.
>>>>>>>
>>>>>>> [  101.863712] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
>>>>>>> [  101.863727] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
>>>>>>> [  101.863742] ath10k: 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32
>>>>>>> [  101.863757] ath10k: 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37 38
>>>>>>> [  101.863773] ath10k: 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34
>>>>>>> [  101.863788] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
>>>>>>> [  101.863803] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
>>>>>>>
>>>>>>>
>>>>>>> I've found a number of examples of similar corruption, always with
>>>>>>> between one and four bytes replaced.
>>>>>>>
>>>>>>> 35363738 -> e52c6e07
>>>>>>> 3435 -> b43f
>>>>>>> 3839 -> c238
>>>>>>> 31 -> 7f
>>>>>>> 3435 -> 7436
>>>>>>> 30 -> 50
>>>>>>> 3233 -> bc37
>>>>>>>
>>>>>> Seems this could be because of:
>>>>>>
>>>>>> + /* cfg80211 expect this padding */
>>>>>> + padding = (4 - (skb->len + sizeof(subframe_hdr))) & 0x3;
>>>>>> + skb_put(skb, padding);
>>>>>>
>>>>>
>>>>> BTW, when AP (ath10k) send TCP ACK - are this packets also AMSDU?
>>>>> I see in my case AP can send 3 x A-MSDU (with total size 304 bytes)
>>>>> small frames aggregated. Maybe your HW have problems with that.
>>>>> As I remember correctly someone some time ago report problems with
>>>>> MacBook pro retina but I am not sure this is the same, while no one
>>>>> tests the fix.
>>>>>
>>>>>>>
>>>>>>> The packet described above, dissected by Wireshark:
>>>>>>>
>>>>>>> No.     Time        Source                Destination
>>>>>>> Protocol Length Info
>>>>>>>    2235 18.953349   192.168.144.79        192.168.144.13        TCP
>>>>>>>   3112   52697 > 5001 [ACK] Seq=1391553 Ack=1 Win=131760 [TCP CHECKSUM
>>>>>>> INCORRECT] Len=1448 TSval=1298580657 TSecr=4294947481
>>>>>>>
>>>>>>> Frame 2235: 3112 bytes on wire (24896 bits), 3112 bytes captured (24896 bits)
>>>>>>>     Encapsulation type: IEEE 802.11 plus radiotap radio header (23)
>>>>>>>     Arrival Time: Jul  7, 2014 23:03:37.763365000 PDT
>>>>>>>     [Time shift for this packet: 0.000000000 seconds]
>>>>>>>     Epoch Time: 1404799417.763365000 seconds
>>>>>>>     [Time delta from previous captured frame: 0.003476000 seconds]
>>>>>>>     [Time delta from previous displayed frame: 0.515641000 seconds]
>>>>>>>     [Time since reference or first frame: 18.953349000 seconds]
>>>>>>>     Frame Number: 2235
>>>>>>>     Frame Length: 3112 bytes (24896 bits)
>>>>>>>     Capture Length: 3112 bytes (24896 bits)
>>>>>>>     [Frame is marked: False]
>>>>>>>     [Frame is ignored: False]
>>>>>>>     [Protocols in frame: radiotap:wlan:llc:ip:tcp:data:llc:ip:tcp:data]
>>>>>>>     [Coloring Rule Name: TCP]
>>>>>>>     [Coloring Rule String: tcp]
>>>>>>> Radiotap Header v0, Length 38
>>>>>>>     Header revision: 0
>>>>>>>     Header pad: 0
>>>>>>>     Header length: 38
>>>>>>>     Present flags
>>>>>>>         .... .... .... .... .... .... .... ...1 = TSFT: True
>>>>>>>         .... .... .... .... .... .... .... ..1. = Flags: True
>>>>>>>         .... .... .... .... .... .... .... .0.. = Rate: False
>>>>>>>         .... .... .... .... .... .... .... 1... = Channel: True
>>>>>>>         .... .... .... .... .... .... ...0 .... = FHSS: False
>>>>>>>         .... .... .... .... .... .... ..1. .... = dBm Antenna Signal: True
>>>>>>>         .... .... .... .... .... .... .0.. .... = dBm Antenna Noise: False
>>>>>>>         .... .... .... .... .... .... 0... .... = Lock Quality: False
>>>>>>>         .... .... .... .... .... ...0 .... .... = TX Attenuation: False
>>>>>>>         .... .... .... .... .... ..0. .... .... = dB TX Attenuation: False
>>>>>>>         .... .... .... .... .... .0.. .... .... = dBm TX Power: False
>>>>>>>         .... .... .... .... .... 1... .... .... = Antenna: True
>>>>>>>         .... .... .... .... ...0 .... .... .... = dB Antenna Signal: False
>>>>>>>         .... .... .... .... ..0. .... .... .... = dB Antenna Noise: False
>>>>>>>         .... .... .... .... .1.. .... .... .... = RX flags: True
>>>>>>>         .... .... .... .0.. .... .... .... .... = Channel+: False
>>>>>>>         .... .... .... 0... .... .... .... .... = HT information: False
>>>>>>>         .... .... ...0 .... .... .... .... .... = A-MPDU Status: False
>>>>>>>         .... .... ..1. .... .... .... .... .... = VHT information: True
>>>>>>>         ...0 0000 00.. .... .... .... .... .... = Reserved: 0x00000000
>>>>>>>         ..0. .... .... .... .... .... .... .... = Radiotap NS next: False
>>>>>>>         .0.. .... .... .... .... .... .... .... = Vendor NS next: False
>>>>>>>         0... .... .... .... .... .... .... .... = Ext: False
>>>>>>>     MAC timestamp: 78051063
>>>>>>>     Flags: 0x00
>>>>>>>         .... ...0 = CFP: False
>>>>>>>         .... ..0. = Preamble: Long
>>>>>>>         .... .0.. = WEP: False
>>>>>>>         .... 0... = Fragmentation: False
>>>>>>>         ...0 .... = FCS at end: False
>>>>>>>         ..0. .... = Data Pad: False
>>>>>>>         .0.. .... = Bad FCS: False
>>>>>>>         0... .... = Short GI: False
>>>>>>>     Channel frequency: 5745 [A 149]
>>>>>>>     Channel type: 802.11a (0x0140)
>>>>>>>         .... .... ...0 .... = Turbo: False
>>>>>>>         .... .... ..0. .... = Complementary Code Keying (CCK): False
>>>>>>>         .... .... .1.. .... = Orthogonal Frequency-Division
>>>>>>> Multiplexing (OFDM): True
>>>>>>>         .... .... 0... .... = 2 GHz spectrum: False
>>>>>>>         .... ...1 .... .... = 5 GHz spectrum: True
>>>>>>>         .... ..0. .... .... = Passive: False
>>>>>>>         .... .0.. .... .... = Dynamic CCK-OFDM: False
>>>>>>>         .... 0... .... .... = Gaussian Frequency Shift Keying (GFSK): False
>>>>>>>         ...0 .... .... .... = GSM (900MHz): False
>>>>>>>         ..0. .... .... .... = Static Turbo: False
>>>>>>>         .0.. .... .... .... = Half Rate Channel (10MHz Channel Width): False
>>>>>>>         0... .... .... .... = Quarter Rate Channel (5MHz Channel Width): False
>>>>>>>     SSI Signal: -53 dBm
>>>>>>>     Antenna: 0
>>>>>>>     RX flags: 0x0000
>>>>>>>         .... .... .... .... .... ..0. = Bad PLCP: False
>>>>>>>     VHT information
>>>>>>>         Known VHT information: 0x44
>>>>>>>             .... .... .... ...0 = STBC: False
>>>>>>>             .... .... .... ..0. = TXOP_PS_NOT_ALLOWED: False
>>>>>>>             .... .... .... .1.. = Guard interval: True
>>>>>>>             .... .... .... 0... = SGI Nsym disambiguation: False
>>>>>>>             .... .... ...0 .... = LDPC extra OFDM symbol: False
>>>>>>>             .... .... ..0. .... = Beamformed: False
>>>>>>>             .... .... .1.. .... = Bandwidth: True
>>>>>>>             .... .... 0... .... = Group ID: False
>>>>>>>             .... ...0 .... .... = Partial AID: False
>>>>>>>         .... .0.. = Guard interval: long (0)
>>>>>>>         Bandwidth: 80 MHz (4)
>>>>>>>         User 0: MCS 8
>>>>>>>             1000 .... = MCS index 0: 8 (256-QAM 3/4)
>>>>>>>             .... 0010 = Spatial streams 0: 2
>>>>>>>             Space-time streams 0: 2
>>>>>>>             Coding 0: BCC (0)
>>>>>>>             [Data Rate: 702.0 Mb/s]
>>>>>>> IEEE 802.11 QoS Data, Flags: .......T
>>>>>>>     Type/Subtype: QoS Data (0x28)
>>>>>>>     Frame Control Field: 0x8801
>>>>>>>         .... ..00 = Version: 0
>>>>>>>         .... 10.. = Type: Data frame (2)
>>>>>>>         1000 .... = Subtype: 8
>>>>>>>         Flags: 0x01
>>>>>>>             .... ..01 = DS status: Frame from STA to DS via an AP (To
>>>>>>> DS: 1 From DS: 0) (0x01)
>>>>>>>             .... .0.. = More Fragments: This is the last fragment
>>>>>>>             .... 0... = Retry: Frame is not being retransmitted
>>>>>>>             ...0 .... = PWR MGT: STA will stay up
>>>>>>>             ..0. .... = More Data: No data buffered
>>>>>>>             .0.. .... = Protected flag: Data is not protected
>>>>>>>             0... .... = Order flag: Not strictly ordered
>>>>>>>     .000 0000 0011 0000 = Duration: 48 microseconds
>>>>>>>     Receiver address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>>>>>     BSS Id: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>>>>>     Transmitter address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>>>     Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>>>     Destination address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>>>>>     Fragment number: 0
>>>>>>>     Sequence number: 1021
>>>>>>>     Qos Control: 0x0080
>>>>>>>         .... .... .... 0000 = TID: 0
>>>>>>>         [.... .... .... .000 = Priority: Best Effort (Best Effort) (0)]
>>>>>>>         .... .... ...0 .... = QoS bit 4: Bits 8-15 of QoS Control
>>>>>>> field are TXOP Duration Requested
>>>>>>>         .... .... .00. .... = Ack Policy: Normal Ack (0x0000)
>>>>>>>         .... .... 1... .... = Payload Type: A-MSDU
>>>>>>>         0000 0000 .... .... = TXOP Duration Requested: 0 (no TXOP requested)
>>>>>>> IEEE 802.11 Aggregate MSDU
>>>>>>>     A-MSDU Subframe #1
>>>>>>>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>>>>>>>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>>>         A-MSDU Length: 1510
>>>>>>>         Logical-Link Control
>>>>>>>             DSAP: SNAP (0xaa)
>>>>>>>             IG Bit: Individual
>>>>>>>             SSAP: SNAP (0xaa)
>>>>>>>             CR Bit: Command
>>>>>>>             Control field: U, func=UI (0x03)
>>>>>>>                 000. 00.. = Command: Unnumbered Information (0x00)
>>>>>>>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>>>>>>>             Organization Code: Encapsulated Ethernet (0x000000)
>>>>>>>             Type: IP (0x0800)
>>>>>>>         Internet Protocol Version 4, Src: 192.168.144.79
>>>>>>> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>>>>>>>             Version: 4
>>>>>>>             Header length: 20 bytes
>>>>>>>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
>>>>>>> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>>>>>>>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>>>>>>>                 .... ..00 = Explicit Congestion Notification: Not-ECT
>>>>>>> (Not ECN-Capable Transport) (0x00)
>>>>>>>             Total Length: 1500
>>>>>>>             Identification: 0xc622 (50722)
>>>>>>>             Flags: 0x00
>>>>>>>                 0... .... = Reserved bit: Not set
>>>>>>>                 .0.. .... = Don't fragment: Not set
>>>>>>>                 ..0. .... = More fragments: Not set
>>>>>>>             Fragment offset: 0
>>>>>>>             Time to live: 64
>>>>>>>             Protocol: TCP (6)
>>>>>>>             Header checksum: 0x0d4c [correct]
>>>>>>>                 [Good: True]
>>>>>>>                 [Bad: False]
>>>>>>>             Source: 192.168.144.79 (192.168.144.79)
>>>>>>>             Destination: 192.168.144.13 (192.168.144.13)
>>>>>>>             [Source GeoIP: Unknown]
>>>>>>>             [Destination GeoIP: Unknown]
>>>>>>>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
>>>>>>> Port: 5001 (5001), Seq: 1390105, Ack: 1, Len: 1448
>>>>>>>             Source port: 52697 (52697)
>>>>>>>             Destination port: 5001 (5001)
>>>>>>>             [Stream index: 0]
>>>>>>>             Sequence number: 1390105    (relative sequence number)
>>>>>>>             [Next sequence number: 1391553    (relative sequence number)]
>>>>>>>             Acknowledgment number: 1    (relative ack number)
>>>>>>>             Header length: 32 bytes
>>>>>>>             Flags: 0x010 (ACK)
>>>>>>>                 000. .... .... = Reserved: Not set
>>>>>>>                 ...0 .... .... = Nonce: Not set
>>>>>>>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>>>>>>>                 .... .0.. .... = ECN-Echo: Not set
>>>>>>>                 .... ..0. .... = Urgent: Not set
>>>>>>>                 .... ...1 .... = Acknowledgment: Set
>>>>>>>                 .... .... 0... = Push: Not set
>>>>>>>                 .... .... .0.. = Reset: Not set
>>>>>>>                 .... .... ..0. = Syn: Not set
>>>>>>>                 .... .... ...0 = Fin: Not set
>>>>>>>             Window size value: 8235
>>>>>>>             [Calculated window size: 131760]
>>>>>>>             [Window size scaling factor: 16]
>>>>>>>             Checksum: 0xa1c0 [correct]
>>>>>>>                 [Good Checksum: True]
>>>>>>>                 [Bad Checksum: False]
>>>>>>>             Options: (12 bytes), No-Operation (NOP), No-Operation
>>>>>>> (NOP), Timestamps
>>>>>>>                 No-Operation (NOP)
>>>>>>>                     Type: 1
>>>>>>>                         0... .... = Copy on fragmentation: No
>>>>>>>                         .00. .... = Class: Control (0)
>>>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>>>                 No-Operation (NOP)
>>>>>>>                     Type: 1
>>>>>>>                         0... .... = Copy on fragmentation: No
>>>>>>>                         .00. .... = Class: Control (0)
>>>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>>>                 Timestamps: TSval 1298580657, TSecr 4294947481
>>>>>>>                     Kind: Timestamp (8)
>>>>>>>                     Length: 10
>>>>>>>                     Timestamp value: 1298580657
>>>>>>>                     Timestamp echo reply: 4294947481
>>>>>>>         Data (1448 bytes)
>>>>>>>
>>>>>>> 0000  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0010  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0020  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0030  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0040  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0050  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0060  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0070  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0080  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0090  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 00a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 00b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 00c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 00d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 00e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 00f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0100  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0110  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0120  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0130  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0140  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0150  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0160  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0170  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0180  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0190  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 01a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 01b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 01c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 01d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 01e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 01f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0200  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0210  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0220  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0230  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0240  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0250  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0260  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0270  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0280  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0290  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 02a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 02b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 02c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 02d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 02e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 02f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0300  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0310  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0320  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0330  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0340  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0350  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0360  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0370  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0380  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0390  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 03a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 03b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 03c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 03d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 03e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 03f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0400  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0410  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0420  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0430  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0440  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0450  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0460  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0470  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0480  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0490  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 04a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 04b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 04c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 04d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 04e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 04f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0500  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0510  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0520  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0530  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0540  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0550  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0560  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0570  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0580  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0590  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 05a0  34 35 36 37 38 39 30 31                           45678901
>>>>>>>             Data: 343536373839303132333435363738393031323334353637...
>>>>>>>             [Length: 1448]
>>>>>>>     A-MSDU Subframe #2
>>>>>>>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>>>>>>>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>>>         A-MSDU Length: 1510
>>>>>>>         Logical-Link Control
>>>>>>>             DSAP: SNAP (0xaa)
>>>>>>>             IG Bit: Individual
>>>>>>>             SSAP: SNAP (0xaa)
>>>>>>>             CR Bit: Command
>>>>>>>             Control field: U, func=UI (0x03)
>>>>>>>                 000. 00.. = Command: Unnumbered Information (0x00)
>>>>>>>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>>>>>>>             Organization Code: Encapsulated Ethernet (0x000000)
>>>>>>>             Type: IP (0x0800)
>>>>>>>         Internet Protocol Version 4, Src: 192.168.144.79
>>>>>>> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>>>>>>>             Version: 4
>>>>>>>             Header length: 20 bytes
>>>>>>>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
>>>>>>> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>>>>>>>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>>>>>>>                 .... ..00 = Explicit Congestion Notification: Not-ECT
>>>>>>> (Not ECN-Capable Transport) (0x00)
>>>>>>>             Total Length: 1500
>>>>>>>             Identification: 0xda09 (55817)
>>>>>>>             Flags: 0x00
>>>>>>>                 0... .... = Reserved bit: Not set
>>>>>>>                 .0.. .... = Don't fragment: Not set
>>>>>>>                 ..0. .... = More fragments: Not set
>>>>>>>             Fragment offset: 0
>>>>>>>             Time to live: 64
>>>>>>>             Protocol: TCP (6)
>>>>>>>             Header checksum: 0xf964 [correct]
>>>>>>>                 [Good: True]
>>>>>>>                 [Bad: False]
>>>>>>>             Source: 192.168.144.79 (192.168.144.79)
>>>>>>>             Destination: 192.168.144.13 (192.168.144.13)
>>>>>>>             [Source GeoIP: Unknown]
>>>>>>>             [Destination GeoIP: Unknown]
>>>>>>>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
>>>>>>> Port: 5001 (5001), Seq: 1391553, Ack: 1, Len: 1448
>>>>>>>             Source port: 52697 (52697)
>>>>>>>             Destination port: 5001 (5001)
>>>>>>>             [Stream index: 0]
>>>>>>>             Sequence number: 1391553    (relative sequence number)
>>>>>>>             [Next sequence number: 1393001    (relative sequence number)]
>>>>>>>             Acknowledgment number: 1    (relative ack number)
>>>>>>>             Header length: 32 bytes
>>>>>>>             Flags: 0x010 (ACK)
>>>>>>>                 000. .... .... = Reserved: Not set
>>>>>>>                 ...0 .... .... = Nonce: Not set
>>>>>>>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>>>>>>>                 .... .0.. .... = ECN-Echo: Not set
>>>>>>>                 .... ..0. .... = Urgent: Not set
>>>>>>>                 .... ...1 .... = Acknowledgment: Set
>>>>>>>                 .... .... 0... = Push: Not set
>>>>>>>                 .... .... .0.. = Reset: Not set
>>>>>>>                 .... .... ..0. = Syn: Not set
>>>>>>>                 .... .... ...0 = Fin: Not set
>>>>>>>             Window size value: 8235
>>>>>>>             [Calculated window size: 131760]
>>>>>>>             [Window size scaling factor: 16]
>>>>>>>             Checksum: 0x9a16 [incorrect, should be 0x5913 (maybe
>>>>>>> caused by "TCP checksum offload"?)]
>>>>
>>>> And yes we are using checksum offload in ath10k.
>>>> Best is using standalone 80211ac sniffer for that case to be sure
>>>> about checksum.
>>>>
>>>>>>>                 [Good Checksum: False]
>>>>>>>                 [Bad Checksum: True]
>>>>>>>                     [Expert Info (Error/Checksum): Bad checksum]
>>>>>>>                         [Message: Bad checksum]
>>>>>>>                         [Severity level: Error]
>>>>>>>                         [Group: Checksum]
>>>>>>>             Options: (12 bytes), No-Operation (NOP), No-Operation
>>>>>>> (NOP), Timestamps
>>>>>>>                 No-Operation (NOP)
>>>>>>>                     Type: 1
>>>>>>>                         0... .... = Copy on fragmentation: No
>>>>>>>                         .00. .... = Class: Control (0)
>>>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>>>                 No-Operation (NOP)
>>>>>>>                     Type: 1
>>>>>>>                         0... .... = Copy on fragmentation: No
>>>>>>>                         .00. .... = Class: Control (0)
>>>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>>>                 Timestamps: TSval 1298580657, TSecr 4294947481
>>>>>>>                     Kind: Timestamp (8)
>>>>>>>                     Length: 10
>>>>>>>                     Timestamp value: 1298580657
>>>>>>>                     Timestamp echo reply: 4294947481
>>>>>>>         Data (1448 bytes)
>>>>>>>
>>>>>>> 0000  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0010  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0020  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0030  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0040  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0050  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0060  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0070  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0080  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0090  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 00a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 00b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 00c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 00d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 00e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 00f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0100  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0110  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0120  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0130  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0140  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0150  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0160  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0170  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0180  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0190  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 01a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 01b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 01c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 01d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
>>>>>>> 01f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0200  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0210  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0220  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0230  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0240  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0250  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0260  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0270  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0280  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0290  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 02a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 02b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 02c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 02d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 02e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 02f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0300  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0310  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0320  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0330  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0340  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0350  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0360  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0370  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0380  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0390  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 03a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 03b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 03c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 03d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 03e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 03f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0400  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0410  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0420  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0430  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0440  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0450  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0460  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0470  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0480  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0490  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 04a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 04b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 04c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 04d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 04e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 04f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0500  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0510  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0520  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0530  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0540  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 0550  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>> 0560  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>> 0570  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>> 0580  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>> 0590  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>> 05a0  32 33 34 35 36 37 38 39                           23456789
>>>>>>>             Data: 323334353637383930313233343536373839303132333435...
>>>>>>>             [Length: 1448]
>>>>>>>
>>>>>>> On Mon, Jul 7, 2014 at 12:26 PM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>> The initial results are not promising: a MacOS 802.11ac client gets
>>>>>>>> between 0-2 Mbps with this change, where it was getting about 8 Mbps
>>>>>>>> prior to this change and ~170 Mbps prior to the reordering fix. A pcap
>>>>>>>> from the receiving system shows a very large number of out of order
>>>>>>>> frames, likely due to TCP retransmission.
>>>>>>>>
>>>>>>>> An 802.11n MacBook gets very good throughput, only the 802.11ac
>>>>>>>> MacBook shows the very poor result. I'm trying to figure out why.
>>>>>>>>
>>>>>>>>
>>>>>>>> One specific note (and probably not related to the throughput): I
>>>>>>>> believe ath10k_htt_rx_amsdu runs in the tasklet, which means it would
>>>>>>>> need to use GFP_ATOMIC instead of GFP_KERNEL. Do I understand it
>>>>>>>> correctly?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jul 7, 2014 at 1:30 AM, Janusz Dziedzic
>>>>>>>> <janusz.dziedzic@tieto.com> wrote:
>>>>>>>>> On 6 July 2014 04:27, Adrian Chadd <adrian@freebsd.org> wrote:
>>>>>>>>>> I think you may have to tell mac80211 that it's okay and not to drop the frames.
>>>>>>>>>>
>>>>>>>>>> The earlier atheros chips would just give you the AMSDU frames as
>>>>>>>>>> deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
>>>>>>>>>> net80211 and it'd ull it apart. But if the driver is doing it (or,
>>>>>>>>>> well, the chip is doing it) then mac80211 needs to know not to drop
>>>>>>>>>> those sub-frames.
>>>>>>>>>>
>>>>>>>>>> I wonder if you'll ever get notified before the complete A-MPDU has
>>>>>>>>>> been received. That happens on previous chips. eg, you have an A-MPDU
>>>>>>>>>> of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
>>>>>>>>>> half of one MPDU before you hit EOL, the next notification you process
>>>>>>>>>> will be the next MSDU in the same MPDU - and then you'll hit the
>>>>>>>>>> reordering hilarity again.
>>>>>>>>>>
>>>>>>>>>> So hm, i will face the same issue in FreeBSD at some point, so I'd
>>>>>>>>>> likely do what you're thinking of doing - pass up a chain of mbufs
>>>>>>>>>> representing the current MPDU and treat the whole lot as the frame(s)
>>>>>>>>>> to care about. Honestly though, I'm kind of wondering whether I should
>>>>>>>>>> mostly just do what the Atheros reference driver does and treat it as
>>>>>>>>>> ethernet payload frames (ie, it's already de-encapsulated and the
>>>>>>>>>> reordering is already done) and just tack on the wifi header bit via
>>>>>>>>>> another of the DMA rings.
>>>>>>>>>>
>>>>>>>>>> Ugh, I really should sit down and write the FreeBSD version of this.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -a
>>>>>>>>>>
>>>>>>>>>> (I'm still having flashbacks from working on this firmware at QCA. Aiee.)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>>>> There are two issues in handling the dis-aggregated A-MSDU subframes
>>>>>>>>>>> in ieee80211_sta_manage_reorder_buf:
>>>>>>>>>>>
>>>>>>>>>>> 1. The out-of-date check:
>>>>>>>>>>>
>>>>>>>>>>>         /* frame with out of date sequence number */
>>>>>>>>>>>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>>>>>>>>>>>                 dev_kfree_skb(skb);
>>>>>>>>>>>                 goto out;
>>>>>>>>>>>         }
>>>>>>>>>>>
>>>>>>>>>>> As all of the subframes carry the same sequence number, the first
>>>>>>>>>>> subframe will be delivered and increment head_seq_num and then all
>>>>>>>>>>> subsequent subframes will be discarded.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2. The duplicates check a bit later in the routine:
>>>>>>>>>>>
>>>>>>>>>>>         /* check if we already stored this frame */
>>>>>>>>>>>         if (tid_agg_rx->reorder_buf[index]) {
>>>>>>>>>>>                 dev_kfree_skb(skb);
>>>>>>>>>>>                 goto out;
>>>>>>>>>>>         }
>>>>>>>>>>>
>>>>>>>>>>> If there is enough packet loss that packets are queued in the reorder
>>>>>>>>>>> buffer and not immediately released, then only the first subframe will
>>>>>>>>>>> be stored. Subsequent subframes will be discarded as duplicates.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
>>>>>>>>>>> the reordering buffer changes, and dropped to about 8 Mbps with the
>>>>>>>>>>> reorder buffer. Hacking around the out-of-date sequence number check
>>>>>>>>>>> to allow all subframes to egress restores it to 170 Mbps.
>>>>>>>>>>>
>>>>>>>>>>> In the area where I'm testing there isn't enough 5 GHz noise to make
>>>>>>>>>>> the duplicates-check issue happen very often. By adding a printk I can
>>>>>>>>>>> see that it does happen, but it doesn't impact the throughput and I
>>>>>>>>>>> can't report the impact of fixing it.
>>>>>>>>>>>
>>>>>>>>>>> ----
>>>>>>>>>>>
>>>>>>>>>>> I do wonder if both of these are symptoms of dis-aggregating the
>>>>>>>>>>> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
>>>>>>>>>>> at a time, and the ath10k A-MSDU case is sending it subframes instead.
>>>>>>>>>>> Trying to make the ath10k code send up all of the subframes as a chain
>>>>>>>>>>> of skbs didn't immediately work, but I do wonder if that would better
>>>>>>>>>>> match the mac80211 expectations.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>>>>> Yes, after some more testing it does look like an unfortunate
>>>>>>>>>>>> interaction between the reorder buffer and A-MSDU. The disaggregated
>>>>>>>>>>>> subframes all carry the same sequence number. The first subframe is
>>>>>>>>>>>> released from the reorder buffer and increments the head_seq_num.
>>>>>>>>>>>> Subsequent subframes are all discarded as being out of date.
>>>>>>>>>>>>
>>>>>>>>>>>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>>>>>>>>>>>> head=0xb06
>>>>>>>>>>>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>>>>>>>>>>>> head=0xb0b
>>>>>>>>>>>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>>>>>>>>>>>> head=0xb10
>>>>>>>>>>>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>>>>>>>>>>>> head=0xb15
>>>>>>>>>>>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>>>>>>>>>>>> head=0xb1a
>>>>>>>>>>>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>>>>>>>>>>>> head=0xb1f
>>>>>>>>>>>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>>>>>>>>>>>> head=0xb24
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>>>>>>>>>>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>>>>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>>>>>>>>>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>>>>>>>>>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>>>>>>>>>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>>>>>>>>>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>>>>>>>>>>>> second is consistently lost. The MacBook generally retransmits the
>>>>>>>>>>>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>>>>>>>>>>>> frame makes it through.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This became more noticeable after the reordering fixes in
>>>>>>>>>>>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I see this A-MSDU packet loss behavior both with and without the
>>>>>>>>>>>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>>>>>>>>>>>> second is dropped. However, *without* the reordering fixes (and
>>>>>>>>>>>>>> therefore with packets delivered out of order) the MacBook sends
>>>>>>>>>>>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>>>>>>>>>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>>>>>>>>>>>> therefore has to deal with more packet loss. I suspect it is an
>>>>>>>>>>>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>>>>>>>>>>>> never going to fully understand, its stuck in a region of the
>>>>>>>>>>>>>> congestion window where the Wifi driver keeps choosing to using
>>>>>>>>>>>>>> A-MSDU.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>>>>>>>>>>>
>>>>>>>>>>>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>>>>>>>>>>>> copying/allocation overhead each subframe is reported as a singly
>>>>>>>>>>>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>>>>>>>>>>>> A-MPDU reordering intereferes with A-MSDU now?
>>>>>>>>>>>>>
>>>>>>>>> Denton could you try attached patch: report amsdu in one big frame.
>>>>>>>>> If help, we can add amsdu skb list support to mac80211/cfg80211 - to
>>>>>>>>> improve performance and reduce memcpy, while currently we have skb
>>>>>>>>> frames, put them in one big skb and next cfg80211 split them again
>>>>>>>>> into msdus and report to stack.
>>>>>>>>>
>>>>>>>>> BR
>>>>>>>>> Janusz

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Janusz Dziedzic]

[-- Attachment #1: Type: text/plain, Size: 56306 bytes --]

On 10 July 2014 20:48, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:
> On 10 July 2014 15:40, Denton Gentry <denton.gentry@gmail.com> wrote:
>> I added this check for RX_ATTENTION_FLAGS_FCS_ERR in
>> ath10k_htt_rx_handler, along with a printk to verify that the code was
>> being run.
>>
>>                         if (attention & RX_ATTENTION_FLAGS_FCS_ERR) {
>>                                 ath10k_htt_rx_free_msdu_chain(msdu_head);
>>                                 printk(KERN_WARNING
>> "ath10k_htt_rx_handler: amsdu RX_ATTENTION_FLAGS_FCS_ERR");
>>                                 continue;
>>                         } else
>>                                 rx_status->flag &= ~RX_FLAG_FAILED_FCS_CRC;
>>
>> I see the printk appear in dmesg, so I know some packets are hitting
>> it. The throughput from the MacOS 802.11ac STA does improve, from 5-20
>> Mbps without to 50-60 Mbps with this change. Prior to adding the
>> reorder buffer fix, the MacBook was getting 170 Mbps.
>>
>> However, packets with incorrect TCP checksums are still being
>> delivered to the Ethernet connected server at the other end. I
>> gathered pcaps at the MacBook, at the ath10k AP, on another ath10k
>> device configured in monitor mode as a sniffer, and at the
>> Ethernet-connected Ubuntu server at the other end.
>>
> Thanks for logs.
>
> This is strange, this suggest MPDU have correct FCS, and only data in
> MSDU are incorrect.
> Seems like a FW/HW bug, while FCS was correct and other sniffer see
> correct frame.
> Seems also like not driver issue while this incorrect data are at
> offset ~0x14C. In driver we touch mostly head and  tail of the data.
> FW report IP/TCP checksum - interesting what status you get for such packet.
> This is line:
> skb->ip_summed = ath10k_htt_rx_get_csum_state(skb);
>
> I see also this is QAM256 (5/6) - but FCS was correct so this suggest
> MPDU frame was received correctly.
> Interesting issue, I am curious who and when insert this wrong data.
> Will check this more tomorrow.
>
> BTW - this patch below is wrong, when we will start reorder we will
> deliver only one MSDU - last one from AMSDU - instead of all, and all
> with MORE flag will go directly to stack not waiting for reorder
> decision. So could introduce other problems.
>
> +       /* not impelemented correctly when driver report separete A-MSDUs */
> +        if (status->flag & RX_FLAG_AMSDU_MORE)
> +               goto dont_reorder;
> +
>
> So we have 3 options in case of using mac80211 reordering:
> 1) use big frame (memcpy overhead)
> 2) report amsd as a skb list (not sure mac80211 patches will be merged
> - under discussion)
> 3) rewrite mac80211 reordering code to support AMSDU_MORE correctly
>
> Other option can be, put reordering code in the driver (table for each
> mpdu for peer and for each tid - resources required - while we have
> almost the same allocated in mac80211 already)
>
> BR
> Janusz
>
>>
>> server.pcap
>> Captured from the Ubuntu iperf server, connected to the AP via Ethernet.
>>
>> Packet #10: TCP seq 4369, IP ID 0x6373, TCP checksum incorrect
>> At offset 208 bytes into the data we see “32 33 34 35” replaced by “98 19 1e b7”
>>
>> Packet #17: TCP seq 4369, IP ID 0x6373, TCP checksum correct
>> Packet #23: TCP seq 4369, IP ID 0xe152, TCP checksum correct
>>
>>
>> ap.pcap
>> Captured from the AP.
>>
>> Packet #10: TCP seq 4369, IP ID 0x6373, TCP checksum incorrect
>> At offset 208 bytes into the data we see “32 33 34 35” replaced by “98 19 1e b7”
>> This is the same pattern we see in the Ubuntu server's trace.
>>
>> Packet #14: TCP seq 4369, IP ID 0x6373, TCP checksum correct
>> This was a Wifi-layer retransmit, which I think means the firmware
>> knew that something was wrong with packet #10. In the wifisniffer
>> trace, packet #19 is a Block ACK from the AP which requests one packet
>> be retransmitted.
>> However it appears that RX_ATTENTION_FLAGS_FCS_ERR was not set on
>> packet #10, so it was forwarded to the Ethernet.
>>
>> Packet #24: TCP seq 4369, IP ID 0xe152, TCP checksum correct
>> This is a TCP-layer retransmit from the MacBook, as the IP ID is different.
>>
>>
>> wifisniffer.pcap
>> A second ath10 configured in monitor mode.
>> Packet #17 AMSDU#1: TCP seq 4369, IP ID 0x6373, TCP checksum correct
>> This is packet #10 in the AP trace. Note that the sniffer did not see
>> “98 19 1e b7” like the AP did.
>>
>> Packet #20 AMSDU#1: TCP seq 4369, IP ID 0x6373, TCP checksum correct
>> This is packet #14 in the AP trace.
>>
>> Packet #39 AMSDU#2: TCP seq 4369, IP ID 0xe152, TCP checksum correct
>> This is packet #24 in the AP trace.
>>
OK, this is what I see in wifi sniffer
seq_num = 68, packet #17 - AMSDU1, AMSDU2
blockACK - missing packet 68 - this suggest AP get this frame but
probably FCS was wrong?
seq_num = 68, retransmitted, packet #20, AMSDU1, AMSDU2
blockAck OK

Seems FW report missing packet, but didn't set INVALID_FCS??


Added some logs to my test env and activate monitor interface on AP.
This is what I see in logs:

[12990.863184] ath10k: xxx mpdu 3882 fcs err 0
[12990.863187] ath10k: xxx v4 1 v6 0 tcp 1 udp 0 ip_ck_ok 1 tcp_ck_ok 0
[12990.863189] ath10k: xxx mpdu 3882 msdu 0 ip_summed 0

[12990.863205] ath10k: xxx mpdu 3883 fcs err 0
[12990.863208] ath10k: xxx v4 1 v6 0 tcp 1 udp 0 ip_ck_ok 1 tcp_ck_ok 0
[12990.863210] ath10k: xxx mpdu 3883 msdu 0 ip_summed 0
[12990.863212] ath10k: xxx v4 1 v6 0 tcp 1 udp 0 ip_ck_ok 1 tcp_ck_ok 0
[12990.863215] ath10k: xxx mpdu 3883 msdu 1 ip_summed 0
[12990.863217] ath10k: xxx v4 1 v6 0 tcp 1 udp 0 ip_ck_ok 1 tcp_ck_ok 0
[12990.863219] ath10k: xxx mpdu 3883 msdu 2 ip_summed 0

Which mean we get MPDU with 3 MSDU subframes. FCS for SN 3883 was correct!
But IP/TCP checksum in msdus subframes not.

Seems FW report FCS incorrectly in some cases.
Will check this more today.

BTW as a workaround we can skip all packets with wrong FCS and wrong
TCP/IP checksum.
Patch attached.

BR
Janusz


>>
>> macbook.pcapng
>> 56: TCP seq 4369, IP ID 0x6373, TCP checksum correct
>> This is the original transmission. Wireshark on MacOS cannot put the
>> interface into monitor mode, so we do not see the Wifi-layer
>> retransmission of this packet.
>>
>> 65: TCP seq 4369, IP ID 0xe152, TCP checksum correct
>> This is a TCP retransmission, triggered because it received duplicate
>> TCP ACKs back from the Ubuntu server.
>>
>>
>> On Wed, Jul 9, 2014 at 12:39 AM, Janusz Dziedzic
>> <janusz.dziedzic@tieto.com> wrote:
>>> On 9 July 2014 08:09, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>> I ran iperf on an 802.11ac MacBook Air, through my ath10 AP, and to an
>>>> Ubuntu system connected to the AP via Ethernet. The AP was running the
>>>> reorder buffer patches, the patch to make A-MSDU bypass the reorder
>>>> buffer, and the patch to make amsdu aggregation configurable via
>>>> debugfs.
>>>>
>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002551.html
>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002553.html
>>>> http://lists.infradead.org/pipermail/ath10k/2014-July/002597.html
>>>> http://permalink.gmane.org/gmane.linux.kernel.wireless.general/124128
>>>>
>>>> It was notably not running the patch from earlier in this thread to
>>>> copy A-MSDU subframes into one skb.
>>>>
>>>>
>>>> I used a second ath10k AP configured in monitor mode as a Wifi
>>>> sniffer. I also gathered pcaps on the MacBook Air and on the Ubuntu
>>>> system. Wireshark on the MacBook cannot gather radiotap headers nor
>>>> put the interface into monitor mode, so we see only Ethernet frames
>>>> and we don't see retransmissions on the Wifi link.
>>>>
>>>> I've attached decodes of the same two packets as captured by the
>>>> MacBook, by the sniffer, and by the Ubuntu server.
>>>>
>>>> The MacBook sends two frames, TCP sequence numbers 3065441 and 3066889
>>>> (the Wireshark "relative sequence number"). In the Wifi sniffer we see
>>>> these being aggregated as two subframes in an A-MSDU frame. The first
>>>> time this A-MSDU is sent it is corrupted in the air. We see the TCP
>>>> checksum of one of the subframes is wrong, and a few bytes of the
>>>> payload replaced with "&gt%'g9(1$".
>>>>
>>>> The A-MSDU is retransmitted a short time later, and the second time
>>>> the TCP checksum of both subframes is correct.
>>>>
>>>> This is all fine so far.
>>>>
>>>> However in the pcap taken from the Ubuntu server, we see TCP sequence
>>>> number 3065441 being delivered *twice.* The first time, the TCP
>>>> checksum is wrong. The second time the TCP checksum is correct.
>>>>
>>>> To me, it looks like A-MSDU frames with bad FCS are not being
>>>> discarded after ieee80211_rx_monitor(). The corrupted frames are being
>>>> delivered. Delivering the corrupted frames results in sending more TCP
>>>> Dup ACKs for the same sequence number back to the MacBook, and I think
>>>> this is what causes the MacBook to decide there is congestion and slow
>>>> down.
>>>>
>>> OK, to be sure this is the main issue we can just drop frames with wrong FCS.
>>> Could you check this:
>>>
>>> @@ -1267,10 +1267,10 @@ static void ath10k_htt_rx_handler(struct
>>> ath10k_htt *htt,
>>>                                 continue;
>>>                         }
>>>
>>> -                       if (attention & RX_ATTENTION_FLAGS_FCS_ERR)
>>> -                               rx_status->flag |= RX_FLAG_FAILED_FCS_CRC;
>>> -                       else
>>> -                               rx_status->flag &= ~RX_FLAG_FAILED_FCS_CRC;
>>> +                       if (attention & RX_ATTENTION_FLAGS_FCS_ERR) {
>>> +                               ath10k_htt_rx_free_msdu_chain(msdu_head);
>>> +                               continue;
>>> +                       }
>>>
>>>                         if (attention & RX_ATTENTION_FLAGS_TKIP_MIC_ERR)
>>>
>>> If will help we will have to check how to handle monitor + ap case
>>> correctly, one of idea is to not drop frames with wrong FCS only in
>>> case of standalone monitor mode.
>>> But, please try this patch to be sure.
>>>
>>> BR
>>> Janusz
>>>
>>>
>>>> One note: the wifi sniffer does not show the same corruption to the
>>>> same packet as the pcap from the Ubuntu system shows. I think that is
>>>> normal: the sniffer won't see precisely the same noise, won't have
>>>> precisely the same receive sensitivity, and its antennas are not
>>>> pointing in the same direction as the primary AP. If I do this test
>>>> again, I'll gather a pcap on the primary AP as well to compare to what
>>>> we see on the Ubuntu system.
>>>>
>>>>
>>>> On Tue, Jul 8, 2014 at 12:29 AM, Janusz Dziedzic
>>>> <janusz.dziedzic@tieto.com> wrote:
>>>>> On 8 July 2014 09:02, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:
>>>>>> On 8 July 2014 08:50, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:
>>>>>>> On 8 July 2014 08:43, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>> I think I know what is happening now, though I've no idea why. The
>>>>>>>> throughput is low because we have many TCP retransmissions. We have
>>>>>>>> retransmissions because the TCP checksum is wrong on a number of
>>>>>>>> frames, and I do find data corruption in the payload so the checksum
>>>>>>>> definitely should be wrong. All of the corrupted frames were
>>>>>>>> originally one of the subframes in an A-MSDU packet.
>>>>>>>>
>>>>>>>> An example follows at the end of this message, as dissected by
>>>>>>>> Wireshark. iperf sends a very regular data pattern of "0123456789..."
>>>>>>>> over and over. Note how in subframe #2 offset 0x1e0 the bytes "32 33
>>>>>>>> 34" have been replaced by "72 36 35"
>>>>>>>>
>>>>>>>> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
>>>>>>>>
>>>>>>>> I added printks at the bottom of ath10k_htt_rx_amsdu immediately
>>>>>>>> before the call to ath10k_process_rx. I found this same packet, and we
>>>>>>>> see the "72 36 35" corruption in the printk. So I think it happened in
>>>>>>>> ath10k_process_rx or before, not anything weird after passing it up to
>>>>>>>> mac80211.
>>>>>>>>
>>>>>>>> [  101.863712] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
>>>>>>>> [  101.863727] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
>>>>>>>> [  101.863742] ath10k: 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32
>>>>>>>> [  101.863757] ath10k: 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37 38
>>>>>>>> [  101.863773] ath10k: 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34
>>>>>>>> [  101.863788] ath10k: 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30
>>>>>>>> [  101.863803] ath10k: 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36
>>>>>>>>
>>>>>>>>
>>>>>>>> I've found a number of examples of similar corruption, always with
>>>>>>>> between one and four bytes replaced.
>>>>>>>>
>>>>>>>> 35363738 -> e52c6e07
>>>>>>>> 3435 -> b43f
>>>>>>>> 3839 -> c238
>>>>>>>> 31 -> 7f
>>>>>>>> 3435 -> 7436
>>>>>>>> 30 -> 50
>>>>>>>> 3233 -> bc37
>>>>>>>>
>>>>>>> Seems this could be because of:
>>>>>>>
>>>>>>> + /* cfg80211 expect this padding */
>>>>>>> + padding = (4 - (skb->len + sizeof(subframe_hdr))) & 0x3;
>>>>>>> + skb_put(skb, padding);
>>>>>>>
>>>>>>
>>>>>> BTW, when AP (ath10k) send TCP ACK - are this packets also AMSDU?
>>>>>> I see in my case AP can send 3 x A-MSDU (with total size 304 bytes)
>>>>>> small frames aggregated. Maybe your HW have problems with that.
>>>>>> As I remember correctly someone some time ago report problems with
>>>>>> MacBook pro retina but I am not sure this is the same, while no one
>>>>>> tests the fix.
>>>>>>
>>>>>>>>
>>>>>>>> The packet described above, dissected by Wireshark:
>>>>>>>>
>>>>>>>> No.     Time        Source                Destination
>>>>>>>> Protocol Length Info
>>>>>>>>    2235 18.953349   192.168.144.79        192.168.144.13        TCP
>>>>>>>>   3112   52697 > 5001 [ACK] Seq=1391553 Ack=1 Win=131760 [TCP CHECKSUM
>>>>>>>> INCORRECT] Len=1448 TSval=1298580657 TSecr=4294947481
>>>>>>>>
>>>>>>>> Frame 2235: 3112 bytes on wire (24896 bits), 3112 bytes captured (24896 bits)
>>>>>>>>     Encapsulation type: IEEE 802.11 plus radiotap radio header (23)
>>>>>>>>     Arrival Time: Jul  7, 2014 23:03:37.763365000 PDT
>>>>>>>>     [Time shift for this packet: 0.000000000 seconds]
>>>>>>>>     Epoch Time: 1404799417.763365000 seconds
>>>>>>>>     [Time delta from previous captured frame: 0.003476000 seconds]
>>>>>>>>     [Time delta from previous displayed frame: 0.515641000 seconds]
>>>>>>>>     [Time since reference or first frame: 18.953349000 seconds]
>>>>>>>>     Frame Number: 2235
>>>>>>>>     Frame Length: 3112 bytes (24896 bits)
>>>>>>>>     Capture Length: 3112 bytes (24896 bits)
>>>>>>>>     [Frame is marked: False]
>>>>>>>>     [Frame is ignored: False]
>>>>>>>>     [Protocols in frame: radiotap:wlan:llc:ip:tcp:data:llc:ip:tcp:data]
>>>>>>>>     [Coloring Rule Name: TCP]
>>>>>>>>     [Coloring Rule String: tcp]
>>>>>>>> Radiotap Header v0, Length 38
>>>>>>>>     Header revision: 0
>>>>>>>>     Header pad: 0
>>>>>>>>     Header length: 38
>>>>>>>>     Present flags
>>>>>>>>         .... .... .... .... .... .... .... ...1 = TSFT: True
>>>>>>>>         .... .... .... .... .... .... .... ..1. = Flags: True
>>>>>>>>         .... .... .... .... .... .... .... .0.. = Rate: False
>>>>>>>>         .... .... .... .... .... .... .... 1... = Channel: True
>>>>>>>>         .... .... .... .... .... .... ...0 .... = FHSS: False
>>>>>>>>         .... .... .... .... .... .... ..1. .... = dBm Antenna Signal: True
>>>>>>>>         .... .... .... .... .... .... .0.. .... = dBm Antenna Noise: False
>>>>>>>>         .... .... .... .... .... .... 0... .... = Lock Quality: False
>>>>>>>>         .... .... .... .... .... ...0 .... .... = TX Attenuation: False
>>>>>>>>         .... .... .... .... .... ..0. .... .... = dB TX Attenuation: False
>>>>>>>>         .... .... .... .... .... .0.. .... .... = dBm TX Power: False
>>>>>>>>         .... .... .... .... .... 1... .... .... = Antenna: True
>>>>>>>>         .... .... .... .... ...0 .... .... .... = dB Antenna Signal: False
>>>>>>>>         .... .... .... .... ..0. .... .... .... = dB Antenna Noise: False
>>>>>>>>         .... .... .... .... .1.. .... .... .... = RX flags: True
>>>>>>>>         .... .... .... .0.. .... .... .... .... = Channel+: False
>>>>>>>>         .... .... .... 0... .... .... .... .... = HT information: False
>>>>>>>>         .... .... ...0 .... .... .... .... .... = A-MPDU Status: False
>>>>>>>>         .... .... ..1. .... .... .... .... .... = VHT information: True
>>>>>>>>         ...0 0000 00.. .... .... .... .... .... = Reserved: 0x00000000
>>>>>>>>         ..0. .... .... .... .... .... .... .... = Radiotap NS next: False
>>>>>>>>         .0.. .... .... .... .... .... .... .... = Vendor NS next: False
>>>>>>>>         0... .... .... .... .... .... .... .... = Ext: False
>>>>>>>>     MAC timestamp: 78051063
>>>>>>>>     Flags: 0x00
>>>>>>>>         .... ...0 = CFP: False
>>>>>>>>         .... ..0. = Preamble: Long
>>>>>>>>         .... .0.. = WEP: False
>>>>>>>>         .... 0... = Fragmentation: False
>>>>>>>>         ...0 .... = FCS at end: False
>>>>>>>>         ..0. .... = Data Pad: False
>>>>>>>>         .0.. .... = Bad FCS: False
>>>>>>>>         0... .... = Short GI: False
>>>>>>>>     Channel frequency: 5745 [A 149]
>>>>>>>>     Channel type: 802.11a (0x0140)
>>>>>>>>         .... .... ...0 .... = Turbo: False
>>>>>>>>         .... .... ..0. .... = Complementary Code Keying (CCK): False
>>>>>>>>         .... .... .1.. .... = Orthogonal Frequency-Division
>>>>>>>> Multiplexing (OFDM): True
>>>>>>>>         .... .... 0... .... = 2 GHz spectrum: False
>>>>>>>>         .... ...1 .... .... = 5 GHz spectrum: True
>>>>>>>>         .... ..0. .... .... = Passive: False
>>>>>>>>         .... .0.. .... .... = Dynamic CCK-OFDM: False
>>>>>>>>         .... 0... .... .... = Gaussian Frequency Shift Keying (GFSK): False
>>>>>>>>         ...0 .... .... .... = GSM (900MHz): False
>>>>>>>>         ..0. .... .... .... = Static Turbo: False
>>>>>>>>         .0.. .... .... .... = Half Rate Channel (10MHz Channel Width): False
>>>>>>>>         0... .... .... .... = Quarter Rate Channel (5MHz Channel Width): False
>>>>>>>>     SSI Signal: -53 dBm
>>>>>>>>     Antenna: 0
>>>>>>>>     RX flags: 0x0000
>>>>>>>>         .... .... .... .... .... ..0. = Bad PLCP: False
>>>>>>>>     VHT information
>>>>>>>>         Known VHT information: 0x44
>>>>>>>>             .... .... .... ...0 = STBC: False
>>>>>>>>             .... .... .... ..0. = TXOP_PS_NOT_ALLOWED: False
>>>>>>>>             .... .... .... .1.. = Guard interval: True
>>>>>>>>             .... .... .... 0... = SGI Nsym disambiguation: False
>>>>>>>>             .... .... ...0 .... = LDPC extra OFDM symbol: False
>>>>>>>>             .... .... ..0. .... = Beamformed: False
>>>>>>>>             .... .... .1.. .... = Bandwidth: True
>>>>>>>>             .... .... 0... .... = Group ID: False
>>>>>>>>             .... ...0 .... .... = Partial AID: False
>>>>>>>>         .... .0.. = Guard interval: long (0)
>>>>>>>>         Bandwidth: 80 MHz (4)
>>>>>>>>         User 0: MCS 8
>>>>>>>>             1000 .... = MCS index 0: 8 (256-QAM 3/4)
>>>>>>>>             .... 0010 = Spatial streams 0: 2
>>>>>>>>             Space-time streams 0: 2
>>>>>>>>             Coding 0: BCC (0)
>>>>>>>>             [Data Rate: 702.0 Mb/s]
>>>>>>>> IEEE 802.11 QoS Data, Flags: .......T
>>>>>>>>     Type/Subtype: QoS Data (0x28)
>>>>>>>>     Frame Control Field: 0x8801
>>>>>>>>         .... ..00 = Version: 0
>>>>>>>>         .... 10.. = Type: Data frame (2)
>>>>>>>>         1000 .... = Subtype: 8
>>>>>>>>         Flags: 0x01
>>>>>>>>             .... ..01 = DS status: Frame from STA to DS via an AP (To
>>>>>>>> DS: 1 From DS: 0) (0x01)
>>>>>>>>             .... .0.. = More Fragments: This is the last fragment
>>>>>>>>             .... 0... = Retry: Frame is not being retransmitted
>>>>>>>>             ...0 .... = PWR MGT: STA will stay up
>>>>>>>>             ..0. .... = More Data: No data buffered
>>>>>>>>             .0.. .... = Protected flag: Data is not protected
>>>>>>>>             0... .... = Order flag: Not strictly ordered
>>>>>>>>     .000 0000 0011 0000 = Duration: 48 microseconds
>>>>>>>>     Receiver address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>>>>>>     BSS Id: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>>>>>>     Transmitter address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>>>>     Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>>>>     Destination address: SenaoNet_18:a8:00 (88:dc:96:18:a8:00)
>>>>>>>>     Fragment number: 0
>>>>>>>>     Sequence number: 1021
>>>>>>>>     Qos Control: 0x0080
>>>>>>>>         .... .... .... 0000 = TID: 0
>>>>>>>>         [.... .... .... .000 = Priority: Best Effort (Best Effort) (0)]
>>>>>>>>         .... .... ...0 .... = QoS bit 4: Bits 8-15 of QoS Control
>>>>>>>> field are TXOP Duration Requested
>>>>>>>>         .... .... .00. .... = Ack Policy: Normal Ack (0x0000)
>>>>>>>>         .... .... 1... .... = Payload Type: A-MSDU
>>>>>>>>         0000 0000 .... .... = TXOP Duration Requested: 0 (no TXOP requested)
>>>>>>>> IEEE 802.11 Aggregate MSDU
>>>>>>>>     A-MSDU Subframe #1
>>>>>>>>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>>>>>>>>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>>>>         A-MSDU Length: 1510
>>>>>>>>         Logical-Link Control
>>>>>>>>             DSAP: SNAP (0xaa)
>>>>>>>>             IG Bit: Individual
>>>>>>>>             SSAP: SNAP (0xaa)
>>>>>>>>             CR Bit: Command
>>>>>>>>             Control field: U, func=UI (0x03)
>>>>>>>>                 000. 00.. = Command: Unnumbered Information (0x00)
>>>>>>>>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>>>>>>>>             Organization Code: Encapsulated Ethernet (0x000000)
>>>>>>>>             Type: IP (0x0800)
>>>>>>>>         Internet Protocol Version 4, Src: 192.168.144.79
>>>>>>>> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>>>>>>>>             Version: 4
>>>>>>>>             Header length: 20 bytes
>>>>>>>>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
>>>>>>>> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>>>>>>>>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>>>>>>>>                 .... ..00 = Explicit Congestion Notification: Not-ECT
>>>>>>>> (Not ECN-Capable Transport) (0x00)
>>>>>>>>             Total Length: 1500
>>>>>>>>             Identification: 0xc622 (50722)
>>>>>>>>             Flags: 0x00
>>>>>>>>                 0... .... = Reserved bit: Not set
>>>>>>>>                 .0.. .... = Don't fragment: Not set
>>>>>>>>                 ..0. .... = More fragments: Not set
>>>>>>>>             Fragment offset: 0
>>>>>>>>             Time to live: 64
>>>>>>>>             Protocol: TCP (6)
>>>>>>>>             Header checksum: 0x0d4c [correct]
>>>>>>>>                 [Good: True]
>>>>>>>>                 [Bad: False]
>>>>>>>>             Source: 192.168.144.79 (192.168.144.79)
>>>>>>>>             Destination: 192.168.144.13 (192.168.144.13)
>>>>>>>>             [Source GeoIP: Unknown]
>>>>>>>>             [Destination GeoIP: Unknown]
>>>>>>>>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
>>>>>>>> Port: 5001 (5001), Seq: 1390105, Ack: 1, Len: 1448
>>>>>>>>             Source port: 52697 (52697)
>>>>>>>>             Destination port: 5001 (5001)
>>>>>>>>             [Stream index: 0]
>>>>>>>>             Sequence number: 1390105    (relative sequence number)
>>>>>>>>             [Next sequence number: 1391553    (relative sequence number)]
>>>>>>>>             Acknowledgment number: 1    (relative ack number)
>>>>>>>>             Header length: 32 bytes
>>>>>>>>             Flags: 0x010 (ACK)
>>>>>>>>                 000. .... .... = Reserved: Not set
>>>>>>>>                 ...0 .... .... = Nonce: Not set
>>>>>>>>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>>>>>>>>                 .... .0.. .... = ECN-Echo: Not set
>>>>>>>>                 .... ..0. .... = Urgent: Not set
>>>>>>>>                 .... ...1 .... = Acknowledgment: Set
>>>>>>>>                 .... .... 0... = Push: Not set
>>>>>>>>                 .... .... .0.. = Reset: Not set
>>>>>>>>                 .... .... ..0. = Syn: Not set
>>>>>>>>                 .... .... ...0 = Fin: Not set
>>>>>>>>             Window size value: 8235
>>>>>>>>             [Calculated window size: 131760]
>>>>>>>>             [Window size scaling factor: 16]
>>>>>>>>             Checksum: 0xa1c0 [correct]
>>>>>>>>                 [Good Checksum: True]
>>>>>>>>                 [Bad Checksum: False]
>>>>>>>>             Options: (12 bytes), No-Operation (NOP), No-Operation
>>>>>>>> (NOP), Timestamps
>>>>>>>>                 No-Operation (NOP)
>>>>>>>>                     Type: 1
>>>>>>>>                         0... .... = Copy on fragmentation: No
>>>>>>>>                         .00. .... = Class: Control (0)
>>>>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>>>>                 No-Operation (NOP)
>>>>>>>>                     Type: 1
>>>>>>>>                         0... .... = Copy on fragmentation: No
>>>>>>>>                         .00. .... = Class: Control (0)
>>>>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>>>>                 Timestamps: TSval 1298580657, TSecr 4294947481
>>>>>>>>                     Kind: Timestamp (8)
>>>>>>>>                     Length: 10
>>>>>>>>                     Timestamp value: 1298580657
>>>>>>>>                     Timestamp echo reply: 4294947481
>>>>>>>>         Data (1448 bytes)
>>>>>>>>
>>>>>>>> 0000  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0010  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0020  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0030  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0040  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0050  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0060  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0070  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0080  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0090  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 00a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 00b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 00c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 00d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 00e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 00f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0100  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0110  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0120  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0130  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0140  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0150  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0160  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0170  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0180  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0190  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 01a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 01b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 01c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 01d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 01e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 01f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0200  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0210  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0220  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0230  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0240  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0250  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0260  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0270  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0280  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0290  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 02a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 02b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 02c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 02d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 02e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 02f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0300  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0310  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0320  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0330  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0340  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0350  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0360  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0370  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0380  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0390  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 03a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 03b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 03c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 03d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 03e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 03f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0400  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0410  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0420  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0430  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0440  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0450  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0460  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0470  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0480  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0490  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 04a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 04b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 04c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 04d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 04e0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 04f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0500  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0510  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0520  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0530  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0540  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0550  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0560  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0570  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0580  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0590  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 05a0  34 35 36 37 38 39 30 31                           45678901
>>>>>>>>             Data: 343536373839303132333435363738393031323334353637...
>>>>>>>>             [Length: 1448]
>>>>>>>>     A-MSDU Subframe #2
>>>>>>>>         Destination address: GoogleFi_00:14:cd (f8:8f:ca:00:14:cd)
>>>>>>>>         Source address: Apple_67:24:54 (84:38:35:67:24:54)
>>>>>>>>         A-MSDU Length: 1510
>>>>>>>>         Logical-Link Control
>>>>>>>>             DSAP: SNAP (0xaa)
>>>>>>>>             IG Bit: Individual
>>>>>>>>             SSAP: SNAP (0xaa)
>>>>>>>>             CR Bit: Command
>>>>>>>>             Control field: U, func=UI (0x03)
>>>>>>>>                 000. 00.. = Command: Unnumbered Information (0x00)
>>>>>>>>                 .... ..11 = Frame type: Unnumbered frame (0x03)
>>>>>>>>             Organization Code: Encapsulated Ethernet (0x000000)
>>>>>>>>             Type: IP (0x0800)
>>>>>>>>         Internet Protocol Version 4, Src: 192.168.144.79
>>>>>>>> (192.168.144.79), Dst: 192.168.144.13 (192.168.144.13)
>>>>>>>>             Version: 4
>>>>>>>>             Header length: 20 bytes
>>>>>>>>             Differentiated Services Field: 0x00 (DSCP 0x00: Default;
>>>>>>>> ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
>>>>>>>>                 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>>>>>>>>                 .... ..00 = Explicit Congestion Notification: Not-ECT
>>>>>>>> (Not ECN-Capable Transport) (0x00)
>>>>>>>>             Total Length: 1500
>>>>>>>>             Identification: 0xda09 (55817)
>>>>>>>>             Flags: 0x00
>>>>>>>>                 0... .... = Reserved bit: Not set
>>>>>>>>                 .0.. .... = Don't fragment: Not set
>>>>>>>>                 ..0. .... = More fragments: Not set
>>>>>>>>             Fragment offset: 0
>>>>>>>>             Time to live: 64
>>>>>>>>             Protocol: TCP (6)
>>>>>>>>             Header checksum: 0xf964 [correct]
>>>>>>>>                 [Good: True]
>>>>>>>>                 [Bad: False]
>>>>>>>>             Source: 192.168.144.79 (192.168.144.79)
>>>>>>>>             Destination: 192.168.144.13 (192.168.144.13)
>>>>>>>>             [Source GeoIP: Unknown]
>>>>>>>>             [Destination GeoIP: Unknown]
>>>>>>>>         Transmission Control Protocol, Src Port: 52697 (52697), Dst
>>>>>>>> Port: 5001 (5001), Seq: 1391553, Ack: 1, Len: 1448
>>>>>>>>             Source port: 52697 (52697)
>>>>>>>>             Destination port: 5001 (5001)
>>>>>>>>             [Stream index: 0]
>>>>>>>>             Sequence number: 1391553    (relative sequence number)
>>>>>>>>             [Next sequence number: 1393001    (relative sequence number)]
>>>>>>>>             Acknowledgment number: 1    (relative ack number)
>>>>>>>>             Header length: 32 bytes
>>>>>>>>             Flags: 0x010 (ACK)
>>>>>>>>                 000. .... .... = Reserved: Not set
>>>>>>>>                 ...0 .... .... = Nonce: Not set
>>>>>>>>                 .... 0... .... = Congestion Window Reduced (CWR): Not set
>>>>>>>>                 .... .0.. .... = ECN-Echo: Not set
>>>>>>>>                 .... ..0. .... = Urgent: Not set
>>>>>>>>                 .... ...1 .... = Acknowledgment: Set
>>>>>>>>                 .... .... 0... = Push: Not set
>>>>>>>>                 .... .... .0.. = Reset: Not set
>>>>>>>>                 .... .... ..0. = Syn: Not set
>>>>>>>>                 .... .... ...0 = Fin: Not set
>>>>>>>>             Window size value: 8235
>>>>>>>>             [Calculated window size: 131760]
>>>>>>>>             [Window size scaling factor: 16]
>>>>>>>>             Checksum: 0x9a16 [incorrect, should be 0x5913 (maybe
>>>>>>>> caused by "TCP checksum offload"?)]
>>>>>
>>>>> And yes we are using checksum offload in ath10k.
>>>>> Best is using standalone 80211ac sniffer for that case to be sure
>>>>> about checksum.
>>>>>
>>>>>>>>                 [Good Checksum: False]
>>>>>>>>                 [Bad Checksum: True]
>>>>>>>>                     [Expert Info (Error/Checksum): Bad checksum]
>>>>>>>>                         [Message: Bad checksum]
>>>>>>>>                         [Severity level: Error]
>>>>>>>>                         [Group: Checksum]
>>>>>>>>             Options: (12 bytes), No-Operation (NOP), No-Operation
>>>>>>>> (NOP), Timestamps
>>>>>>>>                 No-Operation (NOP)
>>>>>>>>                     Type: 1
>>>>>>>>                         0... .... = Copy on fragmentation: No
>>>>>>>>                         .00. .... = Class: Control (0)
>>>>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>>>>                 No-Operation (NOP)
>>>>>>>>                     Type: 1
>>>>>>>>                         0... .... = Copy on fragmentation: No
>>>>>>>>                         .00. .... = Class: Control (0)
>>>>>>>>                         ...0 0001 = Number: No-Operation (NOP) (1)
>>>>>>>>                 Timestamps: TSval 1298580657, TSecr 4294947481
>>>>>>>>                     Kind: Timestamp (8)
>>>>>>>>                     Length: 10
>>>>>>>>                     Timestamp value: 1298580657
>>>>>>>>                     Timestamp echo reply: 4294947481
>>>>>>>>         Data (1448 bytes)
>>>>>>>>
>>>>>>>> 0000  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0010  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0020  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0030  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0040  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0050  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0060  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0070  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0080  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0090  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 00a0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 00b0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 00c0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 00d0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 00e0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 00f0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0100  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0110  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0120  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0130  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0140  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0150  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0160  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0170  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0180  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0190  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 01a0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 01b0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 01c0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 01d0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 01e0  32 33 34 35 36 37 38 39 30 31 72 36 35 35 36 37   2345678901r65567
>>>>>>>> 01f0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0200  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0210  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0220  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0230  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0240  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0250  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0260  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0270  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0280  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0290  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 02a0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 02b0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 02c0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 02d0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 02e0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 02f0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0300  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0310  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0320  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0330  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0340  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0350  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0360  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0370  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0380  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0390  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 03a0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 03b0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 03c0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 03d0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 03e0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 03f0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0400  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0410  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0420  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0430  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0440  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0450  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0460  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0470  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0480  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0490  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 04a0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 04b0  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 04c0  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 04d0  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 04e0  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 04f0  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0500  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0510  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0520  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0530  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0540  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 0550  32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37   2345678901234567
>>>>>>>> 0560  38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33   8901234567890123
>>>>>>>> 0570  34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39   4567890123456789
>>>>>>>> 0580  30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35   0123456789012345
>>>>>>>> 0590  36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31   6789012345678901
>>>>>>>> 05a0  32 33 34 35 36 37 38 39                           23456789
>>>>>>>>             Data: 323334353637383930313233343536373839303132333435...
>>>>>>>>             [Length: 1448]
>>>>>>>>
>>>>>>>> On Mon, Jul 7, 2014 at 12:26 PM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>> The initial results are not promising: a MacOS 802.11ac client gets
>>>>>>>>> between 0-2 Mbps with this change, where it was getting about 8 Mbps
>>>>>>>>> prior to this change and ~170 Mbps prior to the reordering fix. A pcap
>>>>>>>>> from the receiving system shows a very large number of out of order
>>>>>>>>> frames, likely due to TCP retransmission.
>>>>>>>>>
>>>>>>>>> An 802.11n MacBook gets very good throughput, only the 802.11ac
>>>>>>>>> MacBook shows the very poor result. I'm trying to figure out why.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> One specific note (and probably not related to the throughput): I
>>>>>>>>> believe ath10k_htt_rx_amsdu runs in the tasklet, which means it would
>>>>>>>>> need to use GFP_ATOMIC instead of GFP_KERNEL. Do I understand it
>>>>>>>>> correctly?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jul 7, 2014 at 1:30 AM, Janusz Dziedzic
>>>>>>>>> <janusz.dziedzic@tieto.com> wrote:
>>>>>>>>>> On 6 July 2014 04:27, Adrian Chadd <adrian@freebsd.org> wrote:
>>>>>>>>>>> I think you may have to tell mac80211 that it's okay and not to drop the frames.
>>>>>>>>>>>
>>>>>>>>>>> The earlier atheros chips would just give you the AMSDU frames as
>>>>>>>>>>> deaggregated A-MPDU sub-frames - you'd just pass the A-MSDU up to
>>>>>>>>>>> net80211 and it'd ull it apart. But if the driver is doing it (or,
>>>>>>>>>>> well, the chip is doing it) then mac80211 needs to know not to drop
>>>>>>>>>>> those sub-frames.
>>>>>>>>>>>
>>>>>>>>>>> I wonder if you'll ever get notified before the complete A-MPDU has
>>>>>>>>>>> been received. That happens on previous chips. eg, you have an A-MPDU
>>>>>>>>>>> of 16 frames with 4 MSDUs in each MPDU. If you get notified and handle
>>>>>>>>>>> half of one MPDU before you hit EOL, the next notification you process
>>>>>>>>>>> will be the next MSDU in the same MPDU - and then you'll hit the
>>>>>>>>>>> reordering hilarity again.
>>>>>>>>>>>
>>>>>>>>>>> So hm, i will face the same issue in FreeBSD at some point, so I'd
>>>>>>>>>>> likely do what you're thinking of doing - pass up a chain of mbufs
>>>>>>>>>>> representing the current MPDU and treat the whole lot as the frame(s)
>>>>>>>>>>> to care about. Honestly though, I'm kind of wondering whether I should
>>>>>>>>>>> mostly just do what the Atheros reference driver does and treat it as
>>>>>>>>>>> ethernet payload frames (ie, it's already de-encapsulated and the
>>>>>>>>>>> reordering is already done) and just tack on the wifi header bit via
>>>>>>>>>>> another of the DMA rings.
>>>>>>>>>>>
>>>>>>>>>>> Ugh, I really should sit down and write the FreeBSD version of this.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> -a
>>>>>>>>>>>
>>>>>>>>>>> (I'm still having flashbacks from working on this firmware at QCA. Aiee.)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 5 July 2014 06:55, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>>>>> There are two issues in handling the dis-aggregated A-MSDU subframes
>>>>>>>>>>>> in ieee80211_sta_manage_reorder_buf:
>>>>>>>>>>>>
>>>>>>>>>>>> 1. The out-of-date check:
>>>>>>>>>>>>
>>>>>>>>>>>>         /* frame with out of date sequence number */
>>>>>>>>>>>>         if (ieee80211_sn_less(mpdu_seq_num, head_seq_num)) {
>>>>>>>>>>>>                 dev_kfree_skb(skb);
>>>>>>>>>>>>                 goto out;
>>>>>>>>>>>>         }
>>>>>>>>>>>>
>>>>>>>>>>>> As all of the subframes carry the same sequence number, the first
>>>>>>>>>>>> subframe will be delivered and increment head_seq_num and then all
>>>>>>>>>>>> subsequent subframes will be discarded.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2. The duplicates check a bit later in the routine:
>>>>>>>>>>>>
>>>>>>>>>>>>         /* check if we already stored this frame */
>>>>>>>>>>>>         if (tid_agg_rx->reorder_buf[index]) {
>>>>>>>>>>>>                 dev_kfree_skb(skb);
>>>>>>>>>>>>                 goto out;
>>>>>>>>>>>>         }
>>>>>>>>>>>>
>>>>>>>>>>>> If there is enough packet loss that packets are queued in the reorder
>>>>>>>>>>>> buffer and not immediately released, then only the first subframe will
>>>>>>>>>>>> be stored. Subsequent subframes will be discarded as duplicates.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> An 802.11ac MacBook is able to get about 170 Mbps with iperf prior to
>>>>>>>>>>>> the reordering buffer changes, and dropped to about 8 Mbps with the
>>>>>>>>>>>> reorder buffer. Hacking around the out-of-date sequence number check
>>>>>>>>>>>> to allow all subframes to egress restores it to 170 Mbps.
>>>>>>>>>>>>
>>>>>>>>>>>> In the area where I'm testing there isn't enough 5 GHz noise to make
>>>>>>>>>>>> the duplicates-check issue happen very often. By adding a printk I can
>>>>>>>>>>>> see that it does happen, but it doesn't impact the throughput and I
>>>>>>>>>>>> can't report the impact of fixing it.
>>>>>>>>>>>>
>>>>>>>>>>>> ----
>>>>>>>>>>>>
>>>>>>>>>>>> I do wonder if both of these are symptoms of dis-aggregating the
>>>>>>>>>>>> A-MSDU too early. mac80211 expects to be dealing with the whole MPDU
>>>>>>>>>>>> at a time, and the ath10k A-MSDU case is sending it subframes instead.
>>>>>>>>>>>> Trying to make the ath10k code send up all of the subframes as a chain
>>>>>>>>>>>> of skbs didn't immediately work, but I do wonder if that would better
>>>>>>>>>>>> match the mac80211 expectations.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jul 4, 2014 at 11:58 AM, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>>>>>> Yes, after some more testing it does look like an unfortunate
>>>>>>>>>>>>> interaction between the reorder buffer and A-MSDU. The disaggregated
>>>>>>>>>>>>> subframes all carry the same sequence number. The first subframe is
>>>>>>>>>>>>> released from the reorder buffer and increments the head_seq_num.
>>>>>>>>>>>>> Subsequent subframes are all discarded as being out of date.
>>>>>>>>>>>>>
>>>>>>>>>>>>> [  308.514021] ieee80211_sta_manage_reorder_buf: out of date seq=0xb05
>>>>>>>>>>>>> head=0xb06
>>>>>>>>>>>>> [  308.520577] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0a
>>>>>>>>>>>>> head=0xb0b
>>>>>>>>>>>>> [  308.527198] ieee80211_sta_manage_reorder_buf: out of date seq=0xb0f
>>>>>>>>>>>>> head=0xb10
>>>>>>>>>>>>> [  308.533857] ieee80211_sta_manage_reorder_buf: out of date seq=0xb14
>>>>>>>>>>>>> head=0xb15
>>>>>>>>>>>>> [  308.540480] ieee80211_sta_manage_reorder_buf: out of date seq=0xb19
>>>>>>>>>>>>> head=0xb1a
>>>>>>>>>>>>> [  308.547730] ieee80211_sta_manage_reorder_buf: out of date seq=0xb1e
>>>>>>>>>>>>> head=0xb1f
>>>>>>>>>>>>> [  308.554069] ieee80211_sta_manage_reorder_buf: out of date seq=0xb23
>>>>>>>>>>>>> head=0xb24
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jul 2, 2014 at 9:49 AM, Michal Kazior <michal.kazior@tieto.com> wrote:
>>>>>>>>>>>>>> On 30 June 2014 22:15, Denton Gentry <denton.gentry@gmail.com> wrote:
>>>>>>>>>>>>>>> In iperf tests using a MacBook STA bridging through an ath10k AP to an
>>>>>>>>>>>>>>> Ethernet server, I'm noticing very selective packet loss. The second
>>>>>>>>>>>>>>> and subsequent frames in an A-MSDU packet appear to be dropped.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The AP sets the A-MSDU size to 3839 bytes, and the MacBook frequently
>>>>>>>>>>>>>>> sends A-MSDU packets containing two TCP frames. So far as I can tell,
>>>>>>>>>>>>>>> the first TCP frame from an A-MSDU aggregate is delivered and the
>>>>>>>>>>>>>>> second is consistently lost. The MacBook generally retransmits the
>>>>>>>>>>>>>>> lost frame as a singleton with no aggregation, and the retransmitted
>>>>>>>>>>>>>>> frame makes it through.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This became more noticeable after the reordering fixes in
>>>>>>>>>>>>>>> http://lists.infradead.org/pipermail/ath10k/2014-June/002552.html
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I see this A-MSDU packet loss behavior both with and without the
>>>>>>>>>>>>>>> reordering fixes, the first packet in an A-MSDU is delivered while the
>>>>>>>>>>>>>>> second is dropped. However, *without* the reordering fixes (and
>>>>>>>>>>>>>>> therefore with packets delivered out of order) the MacBook sends
>>>>>>>>>>>>>>> relatively few A-MSDU frames. *With* the reordering fixes, so all
>>>>>>>>>>>>>>> packets are delivered in order, the MacBook keeps sending A-MSDU and
>>>>>>>>>>>>>>> therefore has to deal with more packet loss. I suspect it is an
>>>>>>>>>>>>>>> interaction with the MacOS TCP congestion window which I'm likely
>>>>>>>>>>>>>>> never going to fully understand, its stuck in a region of the
>>>>>>>>>>>>>>> congestion window where the Wifi driver keeps choosing to using
>>>>>>>>>>>>>>> A-MSDU.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I was actually worried about A-MSDU once A-MPDU re-ordering issue was raised.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ath10k fw reports A-MSDU subframes separately. To avoid memory
>>>>>>>>>>>>>> copying/allocation overhead each subframe is reported as a singly
>>>>>>>>>>>>>> A-MSDU MSDU to mac80211 with an extra rx_flag AMSDU_MORE. Perhaps
>>>>>>>>>>>>>> A-MPDU reordering intereferes with A-MSDU now?
>>>>>>>>>>>>>>
>>>>>>>>>> Denton could you try attached patch: report amsdu in one big frame.
>>>>>>>>>> If help, we can add amsdu skb list support to mac80211/cfg80211 - to
>>>>>>>>>> improve performance and reduce memcpy, while currently we have skb
>>>>>>>>>> frames, put them in one big skb and next cfg80211 split them again
>>>>>>>>>> into msdus and report to stack.
>>>>>>>>>>
>>>>>>>>>> BR
>>>>>>>>>> Janusz

[-- Attachment #2: 0001-ath10k-skip-frames-with-tcp-ip-checksum.patch --]
[-- Type: text/x-diff, Size: 2513 bytes --]

From 2e9ca6310e76847bdd648f8210065f58ddaa6572 Mon Sep 17 00:00:00 2001
From: Janusz Dziedzic <janusz.dziedzic@tieto.com>
Date: Fri, 11 Jul 2014 11:00:00 +0200
Subject: [PATCH] ath10k: skip frames with tcp/ip checksum

Seems FW don't report FCS errors correctly.
As a workaround check tpc/ip checksum also
and remove frames with wrong CRC.

Signed-off-by: Janusz Dziedzic <janusz.dziedzic@tieto.com>
---
 drivers/net/wireless/ath/ath10k/htt_rx.c |   22 ++++++++++++++++++----
 1 file changed, 18 insertions(+), 4 deletions(-)

diff --git a/drivers/net/wireless/ath/ath10k/htt_rx.c b/drivers/net/wireless/ath/ath10k/htt_rx.c
index 9dec96f..0cb5b44 100644
--- a/drivers/net/wireless/ath/ath10k/htt_rx.c
+++ b/drivers/net/wireless/ath/ath10k/htt_rx.c
@@ -882,6 +882,7 @@ static void ath10k_htt_rx_amsdu(struct ath10k_htt *htt,
 	unsigned int hdr_len;
 	struct amsdu_subframe_hdr subframe_hdr;
 	struct ieee80211_rx_status *status;
+	int ip_summed;
 
 	rxd = (void *)skb->data - sizeof(*rxd);
 	enctype = MS(__le32_to_cpu(rxd->mpdu_start.info0),
@@ -902,7 +903,13 @@ static void ath10k_htt_rx_amsdu(struct ath10k_htt *htt,
 			 RX_MSDU_START_INFO1_DECAP_FORMAT);
 		decap_hdr = (void *)rxd->rx_hdr_status;
 
-		skb->ip_summed = ath10k_htt_rx_get_csum_state(skb);
+		ip_summed = ath10k_htt_rx_get_csum_state(skb);
+		if (ip_summed == -1) {
+			ath10k_htt_rx_free_msdu_chain(first);
+			return;
+		}
+
+		skb->ip_summed = ip_summed;
 
 		/* First frame in an A-MSDU chain has more decapped data. */
 		if (skb == first) {
@@ -998,6 +1005,7 @@ static void ath10k_htt_rx_msdu(struct ath10k_htt *htt,
 	enum htt_rx_mpdu_encrypt_type enctype;
 	int hdr_len;
 	void *rfc1042;
+	int ip_summed;
 
 	/* This shouldn't happen. If it does than it may be a FW bug. */
 	if (skb->next) {
@@ -1014,7 +1022,13 @@ static void ath10k_htt_rx_msdu(struct ath10k_htt *htt,
 	hdr = (struct ieee80211_hdr *)rxd->rx_hdr_status;
 	hdr_len = ieee80211_hdrlen(hdr->frame_control);
 
-	skb->ip_summed = ath10k_htt_rx_get_csum_state(skb);
+	ip_summed = ath10k_htt_rx_get_csum_state(skb);
+	if (ip_summed == -1) {
+		ath10k_htt_rx_free_msdu_chain(skb);
+		return;
+	}
+
+	skb->ip_summed = ip_summed;
 
 	switch (fmt) {
 	case RX_MSDU_DECAP_RAW:
@@ -1083,9 +1097,9 @@ static int ath10k_htt_rx_get_csum_state(struct sk_buff *skb)
 	if (!is_tcp && !is_udp)
 		return CHECKSUM_NONE;
 	if (!ip_csum_ok)
-		return CHECKSUM_NONE;
+		return -1;
 	if (!tcpudp_csum_ok)
-		return CHECKSUM_NONE;
+		return -1;
 
 	return CHECKSUM_UNNECESSARY;
 }
-- 
1.7.9.5


[-- Attachment #3: Type: text/plain, Size: 146 bytes --]

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Bartosz Markowski]

On 8 July 2014 09:02, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:

[...]

> BTW, when AP (ath10k) send TCP ACK - are this packets also AMSDU?
> I see in my case AP can send 3 x A-MSDU (with total size 304 bytes)
> small frames aggregated. Maybe your HW have problems with that.

@Denton - could you please try to apply this patch -->
http://permalink.gmane.org/gmane.linux.kernel.wireless.general/124128
(adds debugfs entires so you can manually configure ampdu/amsdu
settings) and check whether setting the amsdu to 1 will bypass the
problem (gets rid of aggregated TCP ACKs)

echo "1 64" > htt_max_amsdu_ampdu

As Janusz wrote ath10k has enabled checksum offloads, so we do not
think the retransmissions you noticed, by sniffing via ath10k device,
are due to the CRC erorrs.

> As I remember correctly someone some time ago report problems with
> MacBook pro retina but I am not sure this is the same, while no one
> tests the fix.

This is it: http://lists.infradead.org/pipermail/ath10k/2014-May/001917.html.
We checked that time and got a hint from firmware guys there was an
IOT problem with amsdu 3 setting in our AP with BRCM sta. Can follow
up on this if that's the case here also.

-Bartosz

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: A-MSDU reception not working?

[Denton Gentry]

On Tue, Jul 8, 2014 at 3:59 AM, Bartosz Markowski
<bartosz.markowski@tieto.com> wrote:
> On 8 July 2014 09:02, Janusz Dziedzic <janusz.dziedzic@tieto.com> wrote:
>
> [...]
>
>> BTW, when AP (ath10k) send TCP ACK - are this packets also AMSDU?
>> I see in my case AP can send 3 x A-MSDU (with total size 304 bytes)
>> small frames aggregated. Maybe your HW have problems with that.
>
> @Denton - could you please try to apply this patch -->
> http://permalink.gmane.org/gmane.linux.kernel.wireless.general/124128
> (adds debugfs entires so you can manually configure ampdu/amsdu
> settings) and check whether setting the amsdu to 1 will bypass the
> problem (gets rid of aggregated TCP ACKs)
>
> echo "1 64" > htt_max_amsdu_ampdu

I applied the patch and used echo "1 64" >
/sys/.../htt_max_amsdu_ampdu. It did not improve the throughput of the
802.11ac MacBook Air.

Even without applying this patch, the sniffer trace does not show the
TCP ACKs going back to the MacBook being aggregated as A-MSDU. They
are either individual frames or A-MPDU.


> As Janusz wrote ath10k has enabled checksum offloads, so we do not
> think the retransmissions you noticed, by sniffing via ath10k device,
> are due to the CRC erorrs.
>
>> As I remember correctly someone some time ago report problems with
>> MacBook pro retina but I am not sure this is the same, while no one
>> tests the fix.
>
> This is it: http://lists.infradead.org/pipermail/ath10k/2014-May/001917.html.
> We checked that time and got a hint from firmware guys there was an
> IOT problem with amsdu 3 setting in our AP with BRCM sta. Can follow
> up on this if that's the case here also.
>
> -Bartosz

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k