[TCP] Severe CUBIC single-flow throughput degradation over a WAN path with Linux 6.8

[TCP] Severe CUBIC single-flow throughput degradation over a WAN path with Linux 6.8

[RZ Z]

Hi,

I've been investigating a TCP throughput issue on a VPS running Ubuntu
6.8.0-31-generic.

With CUBIC, a single TCP connection only achieves around 3-6 Mbps,
while four parallel CUBIC connections reach approximately 192-204 Mbps
combined. Switching to BBR brings single-flow throughput up to around
200 Mbps.

I first noticed this while using Xray (VLESS + REALITY), but
reproduced the slowdown with plain iperf3, so the issue doesn't appear
to be specific to Xray.

The tests were in the VPS-to-client direction, with the VPS acting as
the TCP sender.

Here are the main results:

CUBIC, 1 flow:    ~3-6 Mbps
CUBIC, 4 flows:   ~192-204 Mbps aggregate
BBR, 1 flow:      ~200 Mbps

One CUBIC run achieved 4.25 Mbps with 107 retransmissions. I also
observed cwnd values around 2-9 segments under CUBIC, compared with
around 539-666 under BBR in separate high-throughput tests.

The VPS configuration:

Kernel:           6.8.0-31-generic (Ubuntu)
iperf3:           3.16
Default CC:       bbr (changed from cubic)
Available CC:     reno cubic bbr
Egress qdisc:     pfifo_fast (enp1s0)
TCP ECN:          2
TCP SACK:         1
TCP timestamps:   1

After switching to BBR, I also checked TCP statistics and noticed
considerable reordering-related activity.

For example, one long-lived TCP connection reported:

rtt:          83.675 ms
cwnd:         8
retrans:      0/29
reordering:   114
reord_seen:   428246
rcv_ooopack:  94147

These statistics came from a different connection, not the original
iperf3 test, so I'm not sure whether the reordering activity is
related to the throughput issue.

I also encountered a similar problem on another server with a
different configuration and an entirely independent client network.
Switching to BBR helped there too, although I haven't collected
equivalent plain-TCP iperf3 results from that setup.

Could packet reordering or TCP loss recovery explain why CUBIC
performs so poorly here? I'm also wondering whether any relevant
changes have been made since Linux 6.8.

I understand that CUBIC can be sensitive to packet loss, but the
difference between single-flow CUBIC and both parallel CUBIC and
single-flow BBR seems unusually large.

Any suggestions on what to investigate next would be appreciated.

Thanks!

Re: [TCP] Severe CUBIC single-flow throughput degradation over a WAN path with Linux 6.8

[Stephen Hemminger]

On Mon, 9 Nov 2026 05:41:51 +0800
RZ Z <zrzluck99@gmail.com> wrote:

> Hi,
> 
> I've been investigating a TCP throughput issue on a VPS running Ubuntu
> 6.8.0-31-generic.
> 
> With CUBIC, a single TCP connection only achieves around 3-6 Mbps,
> while four parallel CUBIC connections reach approximately 192-204 Mbps
> combined. Switching to BBR brings single-flow throughput up to around
> 200 Mbps.
> 
> I first noticed this while using Xray (VLESS + REALITY), but
> reproduced the slowdown with plain iperf3, so the issue doesn't appear
> to be specific to Xray.
> 
> The tests were in the VPS-to-client direction, with the VPS acting as
> the TCP sender.
> 
> Here are the main results:
> 
> CUBIC, 1 flow:    ~3-6 Mbps
> CUBIC, 4 flows:   ~192-204 Mbps aggregate
> BBR, 1 flow:      ~200 Mbps
> 
> One CUBIC run achieved 4.25 Mbps with 107 retransmissions. I also
> observed cwnd values around 2-9 segments under CUBIC, compared with
> around 539-666 under BBR in separate high-throughput tests.
> 
> The VPS configuration:
> 
> Kernel:           6.8.0-31-generic (Ubuntu)
> iperf3:           3.16
> Default CC:       bbr (changed from cubic)
> Available CC:     reno cubic bbr
> Egress qdisc:     pfifo_fast (enp1s0)
> TCP ECN:          2
> TCP SACK:         1
> TCP timestamps:   1
> 
> After switching to BBR, I also checked TCP statistics and noticed
> considerable reordering-related activity.
> 
> For example, one long-lived TCP connection reported:
> 
> rtt:          83.675 ms
> cwnd:         8
> retrans:      0/29
> reordering:   114
> reord_seen:   428246
> rcv_ooopack:  94147
> 
> These statistics came from a different connection, not the original
> iperf3 test, so I'm not sure whether the reordering activity is
> related to the throughput issue.
> 
> I also encountered a similar problem on another server with a
> different configuration and an entirely independent client network.
> Switching to BBR helped there too, although I haven't collected
> equivalent plain-TCP iperf3 results from that setup.
> 
> Could packet reordering or TCP loss recovery explain why CUBIC
> performs so poorly here? I'm also wondering whether any relevant
> changes have been made since Linux 6.8.
> 
> I understand that CUBIC can be sensitive to packet loss, but the
> difference between single-flow CUBIC and both parallel CUBIC and
> single-flow BBR seems unusually large.
> 
> Any suggestions on what to investigate next would be appreciated.
> 
> Thanks!

Is this a regression, was it always this way? 
It would also help to look at pcap traces and/or TCP statistics.