From: Rick Jones <rick.jones2@hpe.com>
To: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Yuval Mintz <Yuval.Mintz@qlogic.com>,
Alexander Duyck <alexander.duyck@gmail.com>,
Manish Chopra <manish.chopra@qlogic.com>,
David Miller <davem@davemloft.net>,
netdev <netdev@vger.kernel.org>,
Ariel Elior <Ariel.Elior@qlogic.com>,
Tom Herbert <tom@herbertland.com>,
Hannes Frederic Sowa <hannes@redhat.com>
Subject: Re: [PATCH net-next 0/5] qed/qede: Tunnel hardware GRO support
Date: Wed, 22 Jun 2016 17:48:00 -0700 [thread overview]
Message-ID: <576B31C0.8090301@hpe.com> (raw)
In-Reply-To: <576B1AE8.6030309@hpe.com>
On 06/22/2016 04:10 PM, Rick Jones wrote:
> My systems are presently in the midst of an install but I should be able
> to demonstrate it in the morning (US Pacific time, modulo the shuttle
> service of a car repair place)
The installs finished sooner than I thought. So, receiver:
root@np-cp1-comp0001-mgmt:/home/stack# uname -a
Linux np-cp1-comp0001-mgmt 4.4.11-2-amd64-hpelinux #hpelinux1 SMP Mon
May 23 15:39:22 UTC 2016 x86_64 GNU/Linux
root@np-cp1-comp0001-mgmt:/home/stack# ethtool -i hed2
driver: bnx2x
version: 1.712.30-0
firmware-version: bc 7.10.10
bus-info: 0000:05:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
the hed2 interface is a port of an HPE 630M NIC, based on the BCM57840:
05:00.0 Ethernet controller: Broadcom Corporation BCM57840 NetXtreme II
10/20-Gigabit Ethernet (rev 11)
Subsystem: Hewlett-Packard Company HP FlexFabric 20Gb 2-port 630M Adapter
(The pci.ids entry being from before that 10 GbE IP was purchased from
Broadcom by QLogic...)
Verify that LRO is disabled (IIRC it is enabled by default):
root@np-cp1-comp0001-mgmt:/home/stack# ethtool -k hed2 | grep large
large-receive-offload: off
Verify that disable_tpa is not set:
root@np-cp1-comp0001-mgmt:/home/stack# cat
/sys/module/bnx2x/parameters/disable_tpa
0
So this means we will see NIC-firmware GRO.
Start a tcpdump on the receiver:
root@np-cp1-comp0001-mgmt:/home/stack# tcpdump -s 96 -c 2000000 -i hed2
-w foo.pcap port 12867
tcpdump: listening on hed2, link-type EN10MB (Ethernet), capture size 96
bytes
Start a netperf test targeting that system, specifying a smaller MSS:
stack@np-cp1-comp0002-mgmt:~$ ./netperf -H np-cp1-comp0001-guest -- -G
1400 -P 12867 -O throughput,transport_mss
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 12867 AF_INET to
np-cp1-comp0001-guest () port 12867 AF_INET : demo
Throughput Transport
MSS
bytes
3372.82 1388
Come back to the receiver and post-process the tcpdump capture to get
the average segment size for the data segments:
2000000 packets captured
2000916 packets received by filter
0 packets dropped by kernel
root@np-cp1-comp0001-mgmt:/home/stack# tcpdump -n -r foo.pcap | fgrep -v
"length 0" | awk '{sum += $NF}END{print "Average:",sum/NR}'
reading from file foo.pcap, link-type EN10MB (Ethernet)
Average: 2741.93
and finally a snippet of the capture:
00:37:47.333414 IP 192.168.2.8.12867 > 192.168.2.7.12867: Flags [S], seq
1236484791, win 28000, options [mss 1400,sackOK,TS val 1491134 ecr
0,nop,wscale 7], length 0
00:37:47.333488 IP 192.168.2.7.12867 > 192.168.2.8.12867: Flags [S.],
seq 134167501, ack 1236484792, win 28960, options [mss 1460,sackOK,TS
val 1499053 ecr 1491134,nop,wscale 7], length 0
00:37:47.333731 IP 192.168.2.8.12867 > 192.168.2.7.12867: Flags [.], ack
1, win 219, options [nop,nop,TS val 1491134 ecr 1499053], length 0
00:37:47.333788 IP 192.168.2.8.12867 > 192.168.2.7.12867: Flags [.], seq
1:2777, ack 1, win 219, options [nop,nop,TS val 1491134 ecr 1499053],
length 2776
00:37:47.333815 IP 192.168.2.7.12867 > 192.168.2.8.12867: Flags [.], ack
2777, win 270, options [nop,nop,TS val 1499053 ecr 1491134], length 0
00:37:47.333822 IP 192.168.2.8.12867 > 192.168.2.7.12867: Flags [.], seq
2777:5553, ack 1, win 219, options [nop,nop,TS val 1491134 ecr 1499053],
length 2776
00:37:47.333837 IP 192.168.2.7.12867 > 192.168.2.8.12867: Flags [.], ack
5553, win 313, options [nop,nop,TS val 1499053 ecr 1491134], length 0
00:37:47.333842 IP 192.168.2.8.12867 > 192.168.2.7.12867: Flags [.], seq
5553:8329, ack 1, win 219, options [nop,nop,TS val 1491134 ecr 1499053],
length 2776
00:37:47.333856 IP 192.168.2.8.12867 > 192.168.2.7.12867: Flags [.], seq
8329:11105, ack 1, win 219, options [nop,nop,TS val 1491134 ecr
1499053], length 2776
00:37:47.333869 IP 192.168.2.7.12867 > 192.168.2.8.12867: Flags [.], ack
8329, win 357, options [nop,nop,TS val 1499053 ecr 1491134], length 0
00:37:47.333879 IP 192.168.2.8.12867 > 192.168.2.7.12867: Flags [.], seq
11105:13881, ack 1, win 219, options [nop,nop,TS val 1491134 ecr
1499053], length 2776
00:37:47.333891 IP 192.168.2.7.12867 > 192.168.2.8.12867: Flags [.], ack
11105, win 400, options [nop,nop,TS val 1499053 ecr 1491134], length 0
00:37:47.333911 IP 192.168.2.7.12867 > 192.168.2.8.12867: Flags [.], ack
13881, win 444, options [nop,nop,TS val 1499053 ecr 1491134], length 0
00:37:47.333964 IP 192.168.2.8.12867 > 192.168.2.7.12867: Flags [.], seq
13881:16657, ack 1, win 219, options [nop,nop,TS val 1491134 ecr
1499053], length 2776
00:37:47.333982 IP 192.168.2.8.12867 > 192.168.2.7.12867: Flags [.], seq
16657:19433, ack 1, win 219, options [nop,nop,TS val 1491134 ecr
1499053], length 2776
00:37:47.333989 IP 192.168.2.8.12867 > 192.168.2.7.12867: Flags [.], seq
19433:22209, ack 1, win 219, options [nop,nop,TS val 1491134 ecr
1499053], length 2776
00:37:47.333994 IP 192.168.2.8.12867 > 192.168.2.7.12867: Flags [.], seq
22209:24985, ack 1, win 219, options [nop,nop,TS val 1491134 ecr
1499053], length 2776
00:37:47.334011 IP 192.168.2.8.12867 > 192.168.2.7.12867: Flags [.], seq
24985:27761, ack 1, win 219, options [nop,nop,TS val 1491134 ecr
1499053], length 2776
00:37:47.334018 IP 192.168.2.8.12867 > 192.168.2.7.12867: Flags [.], seq
27761:30537, ack 1, win 219, options [nop,nop,TS val 1491134 ecr
1499053], length 2776
00:37:47.334025 IP 192.168.2.8.12867 > 192.168.2.7.12867: Flags [.], seq
30537:33313, ack 1, win 219, options [nop,nop,TS val 1491134 ecr
1499053], length 2776
00:37:47.334031 IP 192.168.2.8.12867 > 192.168.2.7.12867: Flags [.], seq
33313:36089, ack 1, win 219, options [nop,nop,TS val 1491134 ecr
1499053], length 2776
2776 being twice 1388.
happy benchmarking,
rick jones
root@np-cp1-comp0001-mgmt:/home/stack# tcpdump -n -r foo.pcap | fgrep -v
-e "length 0" | fgrep length | awk '{print $NF}' | sort | uniq -c | sort -nr
reading from file foo.pcap, link-type EN10MB (Ethernet)
1400584 2776
5930 1388
17 2544
15 1620
13 2560
13 2000
12 2604
12 2504
12 2456
12 1708
...
next prev parent reply other threads:[~2016-06-23 0:48 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-06-22 8:25 [PATCH net-next 0/5] qed/qede: Tunnel hardware GRO support Manish Chopra
2016-06-22 8:25 ` [PATCH net-next 1/5] net: export udp and gre gro_complete() APIs Manish Chopra
2016-06-22 8:25 ` [PATCH net-next 2/5] qede: Add support to handle VXLAN hardware GRO packets Manish Chopra
2016-06-22 8:25 ` [PATCH net-next 3/5] qede: Add support to handle GENEVE " Manish Chopra
2016-06-22 8:25 ` [PATCH net-next 4/5] qede: Add support to handle GRE " Manish Chopra
2016-06-22 8:25 ` [PATCH net-next 5/5] qed: Enable hardware GRO feature for encapsulated packets Manish Chopra
2016-06-22 16:27 ` [PATCH net-next 0/5] qed/qede: Tunnel hardware GRO support Alexander Duyck
2016-06-22 17:16 ` Yuval Mintz
2016-06-22 17:45 ` Alexander Duyck
2016-06-22 18:22 ` Yuval Mintz
2016-06-22 21:32 ` Alexander Duyck
2016-06-22 22:32 ` Hannes Frederic Sowa
2016-06-22 23:42 ` Eric Dumazet
2016-06-22 21:52 ` Rick Jones
2016-06-22 22:47 ` Eric Dumazet
2016-06-22 22:56 ` Alexander Duyck
2016-06-22 23:31 ` Eric Dumazet
2016-06-22 23:59 ` Tom Herbert
2016-06-23 0:11 ` Alexander Duyck
2016-06-23 4:10 ` Yuval Mintz
2016-06-23 4:17 ` Yuval Mintz
2016-06-23 17:07 ` Alexander Duyck
2016-06-23 21:06 ` Yuval Mintz
2016-06-23 23:20 ` Alexander Duyck
2016-06-24 5:20 ` Yuval Mintz
2016-06-24 16:44 ` Alexander Duyck
2016-06-24 13:09 ` Edward Cree
2016-06-24 16:31 ` Tom Herbert
2016-06-24 17:21 ` Edward Cree
2016-06-26 6:09 ` Yuval Mintz
2016-06-22 23:52 ` Rick Jones
2016-06-23 0:18 ` Alexander Duyck
2016-06-22 23:10 ` Rick Jones
2016-06-23 0:48 ` Rick Jones [this message]
2016-06-23 9:03 ` Yuval Mintz
2016-06-26 19:53 ` David Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=576B31C0.8090301@hpe.com \
--to=rick.jones2@hpe.com \
--cc=Ariel.Elior@qlogic.com \
--cc=Yuval.Mintz@qlogic.com \
--cc=alexander.duyck@gmail.com \
--cc=davem@davemloft.net \
--cc=eric.dumazet@gmail.com \
--cc=hannes@redhat.com \
--cc=manish.chopra@qlogic.com \
--cc=netdev@vger.kernel.org \
--cc=tom@herbertland.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.