public inbox for netdev@vger.kernel.org
 help / color / mirror / Atom feed
From: Ido Schimmel <idosch@idosch.org>
To: Ricard Bejarano <ricard@bejarano.io>
Cc: Andrew Lunn <andrew@lunn.ch>,
	Mika Westerberg <mika.westerberg@linux.intel.com>,
	netdev@vger.kernel.org, michael.jamet@intel.com,
	YehezkelShB@gmail.com, andrew+netdev@lunn.ch,
	davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
	pabeni@redhat.com
Subject: Re: Poor thunderbolt-net interface performance when bridged
Date: Mon, 1 Sep 2025 23:20:56 +0300	[thread overview]
Message-ID: <aLYAKG2Aw5t7GKtu@shredder> (raw)
In-Reply-To: <75EA103A-A9B8-4924-938B-5F41DD4491CE@bejarano.io>

On Thu, Aug 28, 2025 at 09:59:25AM +0200, Ricard Bejarano wrote:
> Anything we could further test?

Disclaimer:
I am not familiar with thunderbolt and tbnet.

tl;dr:
Can you try disabling TSO on your thunderbolt devices and see if it
helps? Like so:
# ethtool -K tb0 tcp-segmentation-offload off

Details:
The driver advertises support for TSO but admits that it's not
implementing it correctly:

"ThunderboltIP takes advantage of TSO packets but instead of segmenting
them we just split the packet into Thunderbolt frames (maximum payload
size of each frame is 4084 bytes) and calculate checksum over the whole
packet here.

The receiving side does the opposite if the host OS supports LRO,
otherwise it needs to split the large packet into MTU sized smaller
packets."

So, what I *think* ends up happening is that the receiver (blue)
receives large TCP packets from tbnet instead of MTU sized TCP packets.
This might be OK for locally received traffic, but not for forwarded
traffic.

The bridge/router/whatever on blue will try to forward the oversized
packets towards purple and drop them because they exceed the size of the
MTU of your Ethernet interface (1500).

The above can explain why you only see it with TCP and only when
forwarding from tbnet to regular Ethernet devices and not in the other
direction.

You can try to start recording packet drops on blue *before* running the
iperf3 test:

# perf record -a -g -e skb:kfree_skb

And then view the traces with "perf script". If the above theory is
correct (and it might not be), you should see the drops in
br_dev_queue_push_xmit().

  reply	other threads:[~2025-09-01 20:21 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-05-22 17:19 Poor thunderbolt-net interface performance when bridged Ricard Bejarano
2025-05-23 11:07 ` Mika Westerberg
2025-05-23 15:07   ` Ricard Bejarano
2025-05-26  4:50     ` Mika Westerberg
2025-05-26  8:50       ` Ricard Bejarano
2025-05-26  9:22         ` Mika Westerberg
2025-05-26 11:47           ` Ricard Bejarano
2025-05-26 12:04             ` Mika Westerberg
2025-05-26 16:10               ` Ricard Bejarano
2025-05-27 10:33                 ` Mika Westerberg
2025-05-27 12:36                   ` Ricard Bejarano
2025-05-26 14:28             ` Andrew Lunn
2025-05-26 15:09               ` Stephen Hemminger
2025-05-26 19:36                 ` Ricard Bejarano
2025-05-26 19:34               ` Ricard Bejarano
2025-05-26 20:19                 ` Andrew Lunn
2025-05-27  8:47                   ` Ricard Bejarano
2025-05-27 12:51                     ` Andrew Lunn
2025-05-27 14:25                       ` Ricard Bejarano
2025-05-27 15:02                         ` Andrew Lunn
2025-05-27 18:57                           ` Ricard Bejarano
2025-05-27 19:08                             ` Andrew Lunn
2025-05-27 19:17                               ` Ricard Bejarano
2025-05-27 19:32                               ` Ricard Bejarano
2025-05-28  6:38                                 ` Ricard Bejarano
2025-05-28 11:57                                   ` Andrew Lunn
2025-05-28 13:08                                     ` Ricard Bejarano
2025-05-29 12:45                                       ` Ricard Bejarano
2025-06-14  9:13                                         ` Ricard Bejarano
2025-06-14 14:11                                           ` Andrew Lunn
2025-06-15 13:56                                             ` Ricard Bejarano
2025-06-21 11:00                                             ` Ricard Bejarano
2025-06-30  7:28                                               ` Ricard Bejarano
2025-08-28  7:59                                                 ` Ricard Bejarano
2025-09-01 20:20                                                   ` Ido Schimmel [this message]
2025-09-02 10:18                                                     ` Ricard Bejarano
2025-09-03  7:43                                                       ` Ido Schimmel
2025-09-04  8:56                                                         ` Ricard Bejarano
2025-09-04 10:33                                                           ` Ido Schimmel
2025-05-29  8:38                                     ` Ricard Bejarano
2025-05-29 10:06                                     ` Ricard Bejarano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aLYAKG2Aw5t7GKtu@shredder \
    --to=idosch@idosch.org \
    --cc=YehezkelShB@gmail.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=andrew@lunn.ch \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=kuba@kernel.org \
    --cc=michael.jamet@intel.com \
    --cc=mika.westerberg@linux.intel.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=ricard@bejarano.io \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox