From: Jasper Spaans <spaans@fox-it.com>
To: "netdev@vger.kernel.org" <netdev@vger.kernel.org>
Cc: <mchan@broadcom.com>
Subject: bnx2 rxhash
Date: Thu, 6 Oct 2011 11:57:25 +0200 [thread overview]
Message-ID: <4E8D7B85.7060702@fox-it.com> (raw)
[-- Attachment #1: Type: text/plain, Size: 4318 bytes --]
Hi list,
I'm doing some experiments with rxhash and packet capturing, and I'm
seeing some unexpected results when receiving data (which happens in
promiscuous mode). This behaviour seems to be similar to that seen in
the question asked previously by Christophe Ngo Van Duc:
http://kerneltrap.com/mailarchive/linux-netdev/2010/7/2/6280474
The data I'm processing consists of (almost 100% tcp) traffic between an
ssl-offloader and a cluster of webservers, so the range of mac-addresses
is rather limited. This ssl-offloader does preserve the IP-address and
tcp ports of the clients, so if the rxhash is based on that data, it
should be distributed evenly.
Is there anything I can do about this?
Thanks,
Jasper
The raw data:
spaans@dhcp-074:~$ uname -a
Linux dhcp-074 3.0.4 #7 SMP Tue Sep 13 12:52:01 CEST 2011 x86_64 GNU/Linux
spaans@dhcp-074:~$ grep eth6 /proc/interrupts | cut -b -30,270-
142: 57070 0 PCI-MSI-edge eth6-0
143: 0 0 PCI-MSI-edge eth6-1
144: 0 0 PCI-MSI-edge eth6-2
145: 0 0 PCI-MSI-edge eth6-3
146: 0 0 PCI-MSI-edge eth6-4
147: 17127 0 PCI-MSI-edge eth6-5
148: 24 0 PCI-MSI-edge eth6-6
149: 0 0 PCI-MSI-edge eth6-7
This is rather different from the behaviour I get with an intel card:
spaans@dhcp-074:~$ grep eth2 /proc/interrupts | cut -b -30,270-
98: 3 0 PCI-MSI-edge eth2
99: 4492 0 PCI-MSI-edge eth2-TxRx-0
100: 4647 0 PCI-MSI-edge eth2-TxRx-1
101: 4369 0 PCI-MSI-edge eth2-TxRx-2
102: 4579 0 PCI-MSI-edge eth2-TxRx-3
103: 4184 0 PCI-MSI-edge eth2-TxRx-4
104: 51268 0 PCI-MSI-edge eth2-TxRx-5
105: 4610 0 PCI-MSI-edge eth2-TxRx-6
106: 4528 0 PCI-MSI-edge eth2-TxRx-7
In both cases, only Tx-5 was used for sending data.
Information about the nics:
from dmesg:
[ 5.857778] igb 0000:08:00.0: Intel(R) Gigabit Ethernet Network
Connection
[ 5.857780] igb 0000:08:00.0: eth6: (PCIe:2.5Gb/s:Width x4)
00:1b:21:b9:b8:dc
[ 5.858101] igb 0000:08:00.0: eth6: PBA No: G18771-002
[ 5.858102] igb 0000:08:00.0: Using MSI-X interrupts. 8 rx queue(s),
8 tx queue(s)
note that this interface is renamed to eth2 after boot
spaans@dhcp-074:~$ ip link sh eth2
8: eth2: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq
state UP qlen 1000
link/ether 00:1b:21:b9:b8:dc brd ff:ff:ff:ff:ff:ff
spaans@dhcp-074:~$ sudo ethtool -k eth2
Offload parameters for eth2:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: off
large-receive-offload: off
ntuple-filters: off
receive-hashing: off
spaans@dhcp-074:~$ sudo ethtool -i eth2
driver: igb
version: 3.0.6-k2
firmware-version: 1.5-1
bus-info: 0000:08:00.0
vs
from dmesg:
[ 5.090860] bnx2 0000:02:00.0: eth2: Broadcom NetXtreme II BCM5709
1000Base-T (C0) PCI Express found at mem d8000000, IRQ 32, node addr
18:03:73:fa:0f:0e
note that this interface is renamed to eth6 after boot
spaans@dhcp-074:~$ ip link sh eth6
4: eth6: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq
state UP qlen 1000
link/ether 18:03:73:fa:0f:0e brd ff:ff:ff:ff:ff:ff
spaans@dhcp-074:~$ sudo ethtool -k eth6
Offload parameters for eth6:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
ntuple-filters: off
receive-hashing: on
spaans@dhcp-074:~$ sudo ethtool -i eth6
driver: bnx2
version: 2.1.6
firmware-version: 6.2.12 bc 5.2.3 NCSI 2.0.11
bus-info: 0000:02:00.0
--
/\____/\ Ir. Jasper Spaans
\ (_)/ Fox-IT Experts in IT Security!
\ X T: +31-15-2847999
\ / \ M: +31-6-41588725
\/ KvK Haaglanden 27301624
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2679 bytes --]
next reply other threads:[~2011-10-06 10:06 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-10-06 9:57 Jasper Spaans [this message]
2011-10-06 16:52 ` bnx2 rxhash Michael Chan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4E8D7B85.7060702@fox-it.com \
--to=spaans@fox-it.com \
--cc=mchan@broadcom.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).