high RTT in 40 MHz channel width

high RTT in 40 MHz channel width

[Iñaki Pascual]

Hi,

I am measuring TCP and UDP latency (actually RTT) and I am getting too 
high values when working with channels with 40MHz (also with 80MHz) width.

I am using hping3 for testing and these are the RTT avg values:

BW 20 MHz: TCP 0.9 UDP 1.1
BW 40 MHz: TCP 7.7 UDP 1.3
BW 80 MHz: TCP 3.5 UDP 1.3

I have tried different channels with similar results. According hping 
there are no packet loss and with wireshark I don't see any retransmission.

I am using:
driver: ath10k_pci
version: 4.2.0+
firmware-version: 10.1.467-ct-com-full-014-96d543

Any ideas?

Bests,

Iñaki

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: high RTT in 40 MHz channel width

[Jose Núñez-Martínez (CTTC)]

Hi all,
just to clarify, results are ms. And stations are in IBSS mode.

BW 20 MHz: TCP 0.9ms UDP 1.1ms
BW 40 MHz: TCP 7.7ms UDP 1.3ms
BW 80 MHz: TCP 3.5ms UDP 1.3ms

Behavior with 40MHz and 80MHz channel bandwidth is really weird taking 
into account there are no retransmissions. Anyone facing this kind of 
problem?

Thanks,
Jose

On 04/29/2016 07:12 PM, Iñaki Pascual wrote:
> Hi,
>
> I am measuring TCP and UDP latency (actually RTT) and I am getting too 
> high values when working with channels with 40MHz (also with 80MHz) 
> width.
>
> I am using hping3 for testing and these are the RTT avg values:
>
> BW 20 MHz: TCP 0.9 UDP 1.1
> BW 40 MHz: TCP 7.7 UDP 1.3
> BW 80 MHz: TCP 3.5 UDP 1.3
>
> I have tried different channels with similar results. According hping 
> there are no packet loss and with wireshark I don't see any 
> retransmission.
>
> I am using:
> driver: ath10k_pci
> version: 4.2.0+
> firmware-version: 10.1.467-ct-com-full-014-96d543
>
> Any ideas?
>
> Bests,
>
> Iñaki
>
> _______________________________________________
> ath10k mailing list
> ath10k@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/ath10k

-- 
Jose Núñez-Martínez, PhD
Senior Researcher   email: jose.nunez@cttc.cat
COMNET Division     Web: http://networks.cttc.cat

Centre Tecnològic de Telecomunicacions de Catalunya (CTTC)
Parc Mediterrani de la Tecnologia
Av. Carl Friedrich Gauss, 7                Ph.:+34 936452927
08860 Castelldefels - Barcelona            Fax: +34 936452901


_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: high RTT in 40 MHz channel width

[Ben Greear]

Since UDP seems to mostly not be effected, maybe it has something
to do with your tcp stack and/or congestion control?

You could sniff on the wlan0 interface, as well as on-air, and compare timestamps
to see if the firmware is being slow about putting the TCP frames on the
air?

What TCP congestion control are you using?

And also, if there is any interference on the secondary channels, the NIC will hold
off transmitting.  So, you would not even see retransmits in this case.

Have you tried in regular managed mode (AP/STA) to see if it is the same with CT firmware?

You could try stock firmware in AP/STA mode.  If CT firmware shows
a regression, I will be happy to look at it.

Finally, you might try the latest CT firmware in case it acts better.  I
ripped out some queue control in the non-full builds somewhat recently,
and possibly that would decrease latency.

Thanks,
Ben


On 04/30/2016 09:39 AM, Jose Núñez-Martínez (CTTC) wrote:
> Hi all,
> just to clarify, results are ms. And stations are in IBSS mode.
>
> BW 20 MHz: TCP 0.9ms UDP 1.1ms
> BW 40 MHz: TCP 7.7ms UDP 1.3ms
> BW 80 MHz: TCP 3.5ms UDP 1.3ms
>
> Behavior with 40MHz and 80MHz channel bandwidth is really weird taking into account there are no retransmissions. Anyone facing this kind of problem?
>
> Thanks,
> Jose
>
> On 04/29/2016 07:12 PM, Iñaki Pascual wrote:
>> Hi,
>>
>> I am measuring TCP and UDP latency (actually RTT) and I am getting too high values when working with channels with 40MHz (also with 80MHz) width.
>>
>> I am using hping3 for testing and these are the RTT avg values:
>>
>> BW 20 MHz: TCP 0.9 UDP 1.1
>> BW 40 MHz: TCP 7.7 UDP 1.3
>> BW 80 MHz: TCP 3.5 UDP 1.3
>>
>> I have tried different channels with similar results. According hping there are no packet loss and with wireshark I don't see any retransmission.
>>
>> I am using:
>> driver: ath10k_pci
>> version: 4.2.0+
>> firmware-version: 10.1.467-ct-com-full-014-96d543
>>
>> Any ideas?
>>
>> Bests,
>>
>> Iñaki
>>
>> _______________________________________________
>> ath10k mailing list
>> ath10k@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/ath10k
>

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: high RTT in 40 MHz channel width

[Iñaki Pascual]

Hi Ben,

thanks for your comments. Please see my in-line answers in your email below.

Best regards,

Iñaki

On 30/04/16 18:50, Ben Greear wrote:
> Since UDP seems to mostly not be effected, maybe it has something
> to do with your tcp stack and/or congestion control?
Yes, we agree the problem is TCP-related so we have performed some tests 
(described below) using hping3 TCP
>
> You could sniff on the wlan0 interface, as well as on-air, and compare 
> timestamps
> to see if the firmware is being slow about putting the TCP frames on the
> air?
>
we did as you say and the results are
TCP hping request on-air timeStamp:             15:11:19,194250000
TCP hping request on-interface timeStamp:   15:11:19,194253000
TCP hping reply on-interface timeStamp:       15:11:19,194280000
TCP hping reply on-air timeStamp:                 15:11:19,197582000

So it looks like the firmware takes 3.3 ms to process/send the reply 
frame. This number is in line with the aprox. 7.7 ms RTT we are 
measuring (3,3 ms *2 + aprox 1 ms delay)
We repeated the experiment with other machines and channels and the 
results are consistent.

> What TCP congestion control are you using?
>
reno
> And also, if there is any interference on the secondary channels, the 
> NIC will hold
> off transmitting.  So, you would not even see retransmits in this case.
>
for these tests, in order to to avoid interferences, we put down all 
other wireless interfaces and equipments in our testbed
> Have you tried in regular managed mode (AP/STA) to see if it is the 
> same with CT firmware?
>
we tried this test, but we were not able to configure the client with BW 
40MHz nor 20MHz, only 80MHz. The average measured RTT for 80 MHz was 3.5 
ms, same as ad-hoc mode.
> You could try stock firmware in AP/STA mode.  If CT firmware shows
> a regression, I will be happy to look at it.
we tried with an Ubuntu 14.04 fresh install, kernel and firmware are:
driver: ath10k_pci
version: 4.2.0-34-generic
firmware-version: 1.0.0.636
we run the test in ad-hoc mode, we could not configure BW 80 MHz (not 
supported)
results:
40MHz
round-trip min/avg/max = 0.5/2.8/5.5 ms
20MHz
round-trip min/avg/max = 0.4/2.5/4.4 ms
> Finally, you might try the latest CT firmware in case it acts better.  I
> ripped out some queue control in the non-full builds somewhat recently,
> and possibly that would decrease latency.
>
we tried with
firmware-version: 10.1.467-ct-_fH-016-b797ef5
results show almost same RTT values

> Thanks,
> Ben
>
>
> On 04/30/2016 09:39 AM, Jose Núñez-Martínez (CTTC) wrote:
>> Hi all,
>> just to clarify, results are ms. And stations are in IBSS mode.
>>
>> BW 20 MHz: TCP 0.9ms UDP 1.1ms
>> BW 40 MHz: TCP 7.7ms UDP 1.3ms
>> BW 80 MHz: TCP 3.5ms UDP 1.3ms
>>
>> Behavior with 40MHz and 80MHz channel bandwidth is really weird 
>> taking into account there are no retransmissions. Anyone facing this 
>> kind of problem?
>>
>> Thanks,
>> Jose
>>
>> On 04/29/2016 07:12 PM, Iñaki Pascual wrote:
>>> Hi,
>>>
>>> I am measuring TCP and UDP latency (actually RTT) and I am getting 
>>> too high values when working with channels with 40MHz (also with 
>>> 80MHz) width.
>>>
>>> I am using hping3 for testing and these are the RTT avg values:
>>>
>>> BW 20 MHz: TCP 0.9 UDP 1.1
>>> BW 40 MHz: TCP 7.7 UDP 1.3
>>> BW 80 MHz: TCP 3.5 UDP 1.3
>>>
>>> I have tried different channels with similar results. According 
>>> hping there are no packet loss and with wireshark I don't see any 
>>> retransmission.
>>>
>>> I am using:
>>> driver: ath10k_pci
>>> version: 4.2.0+
>>> firmware-version: 10.1.467-ct-com-full-014-96d543
>>>
>>> Any ideas?
>>>
>>> Bests,
>>>
>>> Iñaki
>>>
>>> _______________________________________________
>>> ath10k mailing list
>>> ath10k@lists.infradead.org
>>> http://lists.infradead.org/mailman/listinfo/ath10k
>>
>


_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: high RTT in 40 MHz channel width

[Michal Kazior]

On 29 April 2016 at 19:12, Iñaki Pascual <ipascual@cttc.cat> wrote:
> Hi,
>
> I am measuring TCP and UDP latency (actually RTT) and I am getting too high
> values when working with channels with 40MHz (also with 80MHz) width.
>
> I am using hping3 for testing and these are the RTT avg values:
>
> BW 20 MHz: TCP 0.9 UDP 1.1
> BW 40 MHz: TCP 7.7 UDP 1.3
> BW 80 MHz: TCP 3.5 UDP 1.3

Just 2cc. Did you try changing the a-msdu/a-mpdu limits in firmware?
You can tune it via debugfs file:

  echo 3 64 > /sys/kernel/debug/ieee80211/phy0/ath10k/htt_max_amsdu_ampdu

The "3" stands for A-MSDU limit, "64" stands for A-MPDU limit. "3 64"
is the default. You can test, e.g.

 - "1 64"
 - "3 8"
 - "1 1"
 - "1 8"

And see if this changes TCP RTT in any way.

You didn't really tell your setup (or I missed it). Are you using
ath10k as AP, client, or both (i.e. have two ath10k supported
devices)?

Are you bridging traffic or is to locally generated? Perhaps there's a
problem with hw offloaded ip/tcp checksumming. You might want to
remove it from the driver (manually) to verify that or check tcpdump
to confirm whether OTA frames have correct checksums.


Michał

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

Re: high RTT in 40 MHz channel width

[Iñaki Pascual]

Hi Michael,

no, we did not try changing a-msdu/a-mpdu limits. We will take a look at 
this, thanks.

Regarding the setup, these tests were run with two machines with three 
ath10k devices each (QCA988x 802.11ac Wireless Network Adapter).
Kernel and firmware versions:
driver: ath10k_pci
version: 4.2.0+
firmware-version: 10.1.467-ct-com-full-014-96d543

We just enabled one interface on each machine and configured them in 
ibss mode. Then we run hping3 (TCP and UDP) from one to another. The 
equipments are at 4 meters distance with good line view.

Bests,

Iñaki


On 05/05/16 12:47, Michal Kazior wrote:
> On 29 April 2016 at 19:12, Iñaki Pascual <ipascual@cttc.cat> wrote:
>> Hi,
>>
>> I am measuring TCP and UDP latency (actually RTT) and I am getting too high
>> values when working with channels with 40MHz (also with 80MHz) width.
>>
>> I am using hping3 for testing and these are the RTT avg values:
>>
>> BW 20 MHz: TCP 0.9 UDP 1.1
>> BW 40 MHz: TCP 7.7 UDP 1.3
>> BW 80 MHz: TCP 3.5 UDP 1.3
> Just 2cc. Did you try changing the a-msdu/a-mpdu limits in firmware?
> You can tune it via debugfs file:
>
>    echo 3 64 > /sys/kernel/debug/ieee80211/phy0/ath10k/htt_max_amsdu_ampdu
>
> The "3" stands for A-MSDU limit, "64" stands for A-MPDU limit. "3 64"
> is the default. You can test, e.g.
>
>   - "1 64"
>   - "3 8"
>   - "1 1"
>   - "1 8"
>
> And see if this changes TCP RTT in any way.
>
> You didn't really tell your setup (or I missed it). Are you using
> ath10k as AP, client, or both (i.e. have two ath10k supported
> devices)?
>
> Are you bridging traffic or is to locally generated? Perhaps there's a
> problem with hw offloaded ip/tcp checksumming. You might want to
> remove it from the driver (manually) to verify that or check tcpdump
> to confirm whether OTA frames have correct checksums.
>
>
> Michał
>
> _______________________________________________
> ath10k mailing list
> ath10k@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/ath10k


_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k