From: Eric Dumazet <eric.dumazet@gmail.com>
To: Timo Teras <timo.teras@iki.fi>
Cc: netdev@vger.kernel.org, Francois Romieu <romieu@fr.zoreil.com>
Subject: Re: linux-3.0.18+r8169+ipv4/tcp forwarding = tso/gso weirdness and performance degration
Date: Wed, 14 Mar 2012 10:15:14 -0700 [thread overview]
Message-ID: <1331745314.6022.27.camel@edumazet-glaptop> (raw)
In-Reply-To: <20120314190156.622c8cd5@vostro>
On Wed, 2012-03-14 at 19:01 +0200, Timo Teras wrote:
> Hi,
>
> I have a router box running linux-3.0.18 (with grsec patches).
>
> with the NIC hardware:
> r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
> r8169 0000:00:09.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
> r8169 0000:00:09.0: (unregistered net_device): no PCI Express capability
> r8169 0000:00:09.0: eth0: RTL8169sc/8110sc at 0xf82f8000, 00:30:18:ab:6b:54, XID 18000000 IRQ 18
> r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
> r8169 0000:00:0b.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
> r8169 0000:00:0b.0: (unregistered net_device): no PCI Express capability
> r8169 0000:00:0b.0: eth1: RTL8169sc/8110sc at 0xf82fa000, 00:30:18:ab:6b:55, XID 18000000 IRQ 19
> r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
> r8169 0000:00:0c.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
> r8169 0000:00:0c.0: (unregistered net_device): no PCI Express capability
> r8169 0000:00:0c.0: eth2: RTL8169sc/8110sc at 0xf82fc000, 00:30:18:ab:6b:56, XID 18000000 IRQ 16
>
> This box is working just as a plain IPv4 router (internal RFC1918
> address space) forwarding packets.
>
> It routes basically from eth2 to multiple vlans over bond0 consisting of eth0 and eth1.
>
> I have most hw accel stuff turned off, and "ethtool -k eth0" says:
> Offload parameters for eth0:
> rx-checksumming: on
> tx-checksumming: on
> scatter-gather: off
> tcp segmentation offload: off
> udp fragmentation offload: off
> generic segmentation offload: off
>
> The same applies for all interfaces (except lo).
>
> However, tcpdump on this box indicates that I'm receiving very
> long (tcp length more than mtu) incoming packets on eth2 implying that
> gso/tso got turned on somehow. eth2 is connected with cross-over cable
> to similar box running a bit older linux box; but gso/tso is turned off
> there too. When dumping simultaneously on the other side, it indicates
> that all packets sent are normal length, and no merging was performed
> earlier (fits mtu 1500).
>
> So it would appear that the router box somehow insists on doing gso/tso,
> and sadly it will also mess up on the send path (the incoming merged
> packet is forwarded, but sent out short) causing lost segments and
> serious performance degration.
>
> Any pointers how to next debug/fix/workaround this issue?
>
You are fighting the wrong side ;)
Here, its GRO doing the aggregation on receiver.
What kind of problems do you experiment because of this ?
ethtool -k eth2
next prev parent reply other threads:[~2012-03-14 17:15 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-03-14 17:01 linux-3.0.18+r8169+ipv4/tcp forwarding = tso/gso weirdness and performance degration Timo Teras
2012-03-14 17:15 ` Eric Dumazet [this message]
2012-03-14 17:29 ` Timo Teras
2012-03-14 18:25 ` Eric Dumazet
2012-03-14 19:29 ` Ben Hutchings
2012-03-14 19:51 ` Timo Teras
2012-03-14 20:12 ` Eric Dumazet
2012-03-14 20:33 ` Timo Teras
2012-03-14 20:52 ` Eric Dumazet
2012-03-14 20:53 ` Francois Romieu
2012-03-15 6:06 ` Timo Teras
2012-03-15 15:11 ` Timo Teras
2012-03-15 16:11 ` Eric Dumazet
2012-03-15 18:47 ` Timo Teras
2012-03-15 19:11 ` Francois Romieu
2012-03-16 20:15 ` Timo Teras
2012-03-17 9:56 ` Timo Teras
2012-03-17 11:35 ` Francois Romieu
2012-03-17 22:20 ` Francois Romieu
2012-03-18 7:00 ` Timo Teras
2012-03-20 15:31 ` Timo Teras
2012-03-20 18:20 ` Francois Romieu
2012-03-14 21:16 ` Francois Romieu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1331745314.6022.27.camel@edumazet-glaptop \
--to=eric.dumazet@gmail.com \
--cc=netdev@vger.kernel.org \
--cc=romieu@fr.zoreil.com \
--cc=timo.teras@iki.fi \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).