* forcedeth doesn't pass traffic (3.0.x kernel and earliers)
@ 2011-12-10 21:26 Arkadiusz Miśkiewicz
2011-12-10 21:31 ` Arkadiusz Miśkiewicz
0 siblings, 1 reply; 4+ messages in thread
From: Arkadiusz Miśkiewicz @ 2011-12-10 21:26 UTC (permalink / raw)
To: netdev; +Cc: Ian Campbell, David Decotigny, Eric Dumazet
Hello,
I have few supermicro machines connected to edge-core switch using 1Gbps uplink.
The problem is that after boot I have to restart net to get network card pass traffic. The problem
is only with forcedeth network cards. It happens on different supermicro servers and it happens
for as far as I can remember (so, 2.6.3x kernels, too). Currently I'm on 3.0.13.
Logs below show state after fresh boot where network traffic doesn't get through.
Note that it shows that link is up but still - traffic doesn't get passed. tcpdump on eth0
interface sees only traffic initiated from this server and no traffic from outside.
Tried to set every offload (via ethtool -K) to off but that didn't help.
I have to make:
ip link set eth0 down; ip link set eth0 up
wait few second and then traffic starts to get passed. When I do that this shows in dmesg:
[ 1058.266480] forcedeth 0000:00:08.0: irq 40 for MSI/MSI-X
[ 1069.163238] eth0: no IPv6 routers present
(CC to few people shown by get_maintainer.pl)
LOGS/OUTPUTs from non working state after fresh boot.
# dmesg|grep eth
[ 34.528326] forcedeth: Reverse Engineered nForce ethernet driver. Version 0.64.
[ 34.528542] forcedeth 0000:00:08.0: PCI INT A -> Link[LMAC] -> GSI 22 (level, low) -> IRQ 22
[ 34.528547] forcedeth 0000:00:08.0: setting latency timer to 64
[ 35.055319] forcedeth 0000:00:08.0: ifname eth0, PHY OUI 0x5043 @ 2, addr 00:30:48:7a:b4:88
[ 35.055324] forcedeth 0000:00:08.0: highdma csum vlan pwrctl mgmt gbit lnktim msi desc-v3
[ 35.056480] forcedeth 0000:00:09.0: PCI INT A -> Link[LMAD] -> GSI 21 (level, low) -> IRQ 21
[ 35.056485] forcedeth 0000:00:09.0: setting latency timer to 64
[ 35.581960] forcedeth 0000:00:09.0: ifname eth1, PHY OUI 0x5043 @ 3, addr 00:30:48:7a:b4:89
[ 35.581965] forcedeth 0000:00:09.0: highdma csum vlan pwrctl mgmt gbit lnktim msi desc-v3
[ 35.584211] Loading kernel module for a network device with CAP_SYS_MODULE (deprecated). Use CAP_NET_ADMIN and alias netdev-eth0 instead
[ 35.591778] forcedeth 0000:00:08.0: irq 40 for MSI/MSI-X
[ 35.591970] forcedeth 0000:00:08.0: eth0: no link during initialization
[ 35.593030] ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 37.666757] forcedeth 0000:00:08.0: eth0: link up
[ 37.668624] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 48.150006] eth0: no IPv6 routers present
lspci -vv:
00:09.0 Bridge: nVidia Corporation MCP55 Ethernet (rev a3)
Subsystem: Super Micro Computer Inc Device 1611
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0 (250ns min, 5000ns max)
Interrupt: pin A routed to IRQ 21
Region 0: Memory at feaf5000 (32-bit, non-prefetchable) [size=4K]
Region 1: I/O ports at b080 [size=8]
Region 2: Memory at feafa000 (32-bit, non-prefetchable) [size=256]
Region 3: Memory at feaf4c00 (32-bit, non-prefetchable) [size=16]
Capabilities: [44] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [70] MSI-X: Enable- Count=8 Masked-
Vector table: BAR=2 offset=00000000
PBA: BAR=3 offset=00000000
Capabilities: [50] MSI: Enable- Count=1/8 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [6c] HyperTransport: MSI Mapping Enable- Fixed+
Kernel driver in use: forcedeth
# ethtool eth0
Settings for eth0:
Supported ports: [ MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: MII
PHYAD: 2
Transceiver: external
Auto-negotiation: on
Supports Wake-on: g
Wake-on: d
Link detected: yes
# ethtool -S eth0
NIC statistics:
tx_bytes: 46352
tx_zero_rexmt: 704
tx_one_rexmt: 0
tx_many_rexmt: 0
tx_late_collision: 0
tx_fifo_errors: 0
tx_carrier_errors: 0
tx_excess_deferral: 0
tx_retry_error: 0
rx_frame_error: 0
rx_extra_byte: 0
rx_late_collision: 0
rx_runt: 0
rx_frame_too_long: 0
rx_over_errors: 1
rx_crc_errors: 0
rx_frame_align_error: 0
rx_length_error: 0
rx_unicast: 31
rx_multicast: 0
rx_broadcast: 436
rx_packets: 467
rx_errors_total: 1
tx_errors_total: 0
tx_deferral: 0
tx_packets: 704
rx_bytes: 35872
tx_pause: 0
rx_pause: 0
rx_drop_frame: 170
# ethtool -k eth0
Offload parameters for eth0:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: off
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc sfq state UP qlen 1000
link/ether 00:30:48:7a:b4:88 brd ff:ff:ff:ff:ff:ff
--
Arkadiusz Miśkiewicz PLD/Linux Team
arekm / maven.pl http://ftp.pld-linux.org/
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: forcedeth doesn't pass traffic (3.0.x kernel and earliers)
2011-12-10 21:26 forcedeth doesn't pass traffic (3.0.x kernel and earliers) Arkadiusz Miśkiewicz
@ 2011-12-10 21:31 ` Arkadiusz Miśkiewicz
2012-01-13 17:33 ` Arkadiusz Miśkiewicz
0 siblings, 1 reply; 4+ messages in thread
From: Arkadiusz Miśkiewicz @ 2011-12-10 21:31 UTC (permalink / raw)
To: netdev; +Cc: Ian Campbell, David Decotigny, Eric Dumazet
On Saturday 10 of December 2011, Arkadiusz Miśkiewicz wrote:
> Hello,
>
> I have few supermicro machines connected to edge-core switch using 1Gbps
> uplink. The problem is that after boot I have to restart net to get
> network card pass traffic. The problem is only with forcedeth network
> cards. It happens on different supermicro servers and it happens for as
> far as I can remember (so, 2.6.3x kernels, too). Currently I'm on 3.0.13.
>
> Logs below show state after fresh boot where network traffic doesn't get
> through. Note that it shows that link is up but still - traffic doesn't
> get passed. tcpdump on eth0 interface sees only traffic initiated from
> this server and no traffic from outside.
>
> Tried to set every offload (via ethtool -K) to off but that didn't help.
>
> I have to make:
> ip link set eth0 down; ip link set eth0 up
> wait few second and then traffic starts to get passed. When I do that this
> shows in dmesg: [ 1058.266480] forcedeth 0000:00:08.0: irq 40 for
> MSI/MSI-X
> [ 1069.163238] eth0: no IPv6 routers present
One more note. This doesn't always happens like that. Sometimes I boot
and I get link working, traffic gets passed fine:
[ 34.038476] forcedeth: Reverse Engineered nForce ethernet driver. Version 0.64.
[ 34.038698] ACPI: PCI Interrupt Link [LMAC] enabled at IRQ 22
[ 34.038705] forcedeth 0000:00:08.0: PCI INT A -> Link[LMAC] -> GSI 22 (level, low) -> IRQ 22
[ 34.038710] forcedeth 0000:00:08.0: setting latency timer to 64
[ 34.565309] forcedeth 0000:00:08.0: ifname eth0, PHY OUI 0x5043 @ 2, addr 00:30:48:7a:b4:88
[ 34.565314] forcedeth 0000:00:08.0: highdma csum vlan pwrctl mgmt gbit lnktim msi desc-v3
[ 34.566823] ACPI: PCI Interrupt Link [LMAD] enabled at IRQ 21
[ 34.566831] forcedeth 0000:00:09.0: PCI INT A -> Link[LMAD] -> GSI 21 (level, low) -> IRQ 21
[ 34.566836] forcedeth 0000:00:09.0: setting latency timer to 64
[ 35.091341] forcedeth 0000:00:09.0: ifname eth1, PHY OUI 0x5043 @ 3, addr 00:30:48:7a:b4:89
[ 35.091346] forcedeth 0000:00:09.0: highdma csum vlan pwrctl mgmt gbit lnktim msi desc-v3
[ 35.093492] Loading kernel module for a network device with CAP_SYS_MODULE (deprecated). Use CAP_NET_ADMIN and alias netdev-eth0 instead
[ 35.101074] forcedeth 0000:00:08.0: irq 40 for MSI/MSI-X
[ 35.101267] forcedeth 0000:00:08.0: eth0: no link during initialization
[ 35.102322] ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 35.663785] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 35.756083] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[ 37.147869] forcedeth 0000:00:08.0: eth0: link up
[ 37.149108] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 47.176677] eth0: no IPv6 routers present
>
> (CC to few people shown by get_maintainer.pl)
>
> LOGS/OUTPUTs from non working state after fresh boot.
> # dmesg|grep eth
> [ 34.528326] forcedeth: Reverse Engineered nForce ethernet driver.
> Version 0.64. [ 34.528542] forcedeth 0000:00:08.0: PCI INT A ->
> Link[LMAC] -> GSI 22 (level, low) -> IRQ 22 [ 34.528547] forcedeth
> 0000:00:08.0: setting latency timer to 64 [ 35.055319] forcedeth
> 0000:00:08.0: ifname eth0, PHY OUI 0x5043 @ 2, addr 00:30:48:7a:b4:88 [
> 35.055324] forcedeth 0000:00:08.0: highdma csum vlan pwrctl mgmt gbit
> lnktim msi desc-v3 [ 35.056480] forcedeth 0000:00:09.0: PCI INT A ->
> Link[LMAD] -> GSI 21 (level, low) -> IRQ 21 [ 35.056485] forcedeth
> 0000:00:09.0: setting latency timer to 64 [ 35.581960] forcedeth
> 0000:00:09.0: ifname eth1, PHY OUI 0x5043 @ 3, addr 00:30:48:7a:b4:89 [
> 35.581965] forcedeth 0000:00:09.0: highdma csum vlan pwrctl mgmt gbit
> lnktim msi desc-v3 [ 35.584211] Loading kernel module for a network
> device with CAP_SYS_MODULE (deprecated). Use CAP_NET_ADMIN and alias
> netdev-eth0 instead [ 35.591778] forcedeth 0000:00:08.0: irq 40 for
> MSI/MSI-X
> [ 35.591970] forcedeth 0000:00:08.0: eth0: no link during initialization
> [ 35.593030] ADDRCONF(NETDEV_UP): eth0: link is not ready
> [ 37.666757] forcedeth 0000:00:08.0: eth0: link up
> [ 37.668624] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> [ 48.150006] eth0: no IPv6 routers present
>
> lspci -vv:
> 00:09.0 Bridge: nVidia Corporation MCP55 Ethernet (rev a3)
> Subsystem: Super Micro Computer Inc Device 1611
> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
> Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF- FastB2B+
> ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> Latency: 0 (250ns min, 5000ns max)
> Interrupt: pin A routed to IRQ 21
> Region 0: Memory at feaf5000 (32-bit, non-prefetchable) [size=4K]
> Region 1: I/O ports at b080 [size=8]
> Region 2: Memory at feafa000 (32-bit, non-prefetchable) [size=256]
> Region 3: Memory at feaf4c00 (32-bit, non-prefetchable) [size=16]
> Capabilities: [44] Power Management version 2
> Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA
> PME(D0+,D1+,D2+,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0
> DScale=0 PME- Capabilities: [70] MSI-X: Enable- Count=8 Masked-
> Vector table: BAR=2 offset=00000000
> PBA: BAR=3 offset=00000000
> Capabilities: [50] MSI: Enable- Count=1/8 Maskable+ 64bit+
> Address: 0000000000000000 Data: 0000
> Masking: 00000000 Pending: 00000000
> Capabilities: [6c] HyperTransport: MSI Mapping Enable- Fixed+
> Kernel driver in use: forcedeth
>
> # ethtool eth0
> Settings for eth0:
> Supported ports: [ MII ]
> Supported link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Full
> Supported pause frame use: No
> Supports auto-negotiation: Yes
> Advertised link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Full
> Advertised pause frame use: No
> Advertised auto-negotiation: Yes
> Speed: 1000Mb/s
> Duplex: Full
> Port: MII
> PHYAD: 2
> Transceiver: external
> Auto-negotiation: on
> Supports Wake-on: g
> Wake-on: d
> Link detected: yes
>
> # ethtool -S eth0
> NIC statistics:
> tx_bytes: 46352
> tx_zero_rexmt: 704
> tx_one_rexmt: 0
> tx_many_rexmt: 0
> tx_late_collision: 0
> tx_fifo_errors: 0
> tx_carrier_errors: 0
> tx_excess_deferral: 0
> tx_retry_error: 0
> rx_frame_error: 0
> rx_extra_byte: 0
> rx_late_collision: 0
> rx_runt: 0
> rx_frame_too_long: 0
> rx_over_errors: 1
> rx_crc_errors: 0
> rx_frame_align_error: 0
> rx_length_error: 0
> rx_unicast: 31
> rx_multicast: 0
> rx_broadcast: 436
> rx_packets: 467
> rx_errors_total: 1
> tx_errors_total: 0
> tx_deferral: 0
> tx_packets: 704
> rx_bytes: 35872
> tx_pause: 0
> rx_pause: 0
> rx_drop_frame: 170
>
> # ethtool -k eth0
> Offload parameters for eth0:
> rx-checksumming: on
> tx-checksumming: on
> scatter-gather: on
> tcp-segmentation-offload: on
> udp-fragmentation-offload: off
> generic-segmentation-offload: on
> generic-receive-offload: on
> large-receive-offload: off
> rx-vlan-offload: on
> tx-vlan-offload: on
> ntuple-filters: off
> receive-hashing: off
>
>
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc sfq state UP qlen
> 1000 link/ether 00:30:48:7a:b4:88 brd ff:ff:ff:ff:ff:ff
--
Arkadiusz Miśkiewicz PLD/Linux Team
arekm / maven.pl http://ftp.pld-linux.org/
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: forcedeth doesn't pass traffic (3.0.x kernel and earliers)
2011-12-10 21:31 ` Arkadiusz Miśkiewicz
@ 2012-01-13 17:33 ` Arkadiusz Miśkiewicz
2012-01-13 18:34 ` John A. Sullivan III
0 siblings, 1 reply; 4+ messages in thread
From: Arkadiusz Miśkiewicz @ 2012-01-13 17:33 UTC (permalink / raw)
To: netdev
On Saturday 10 of December 2011, Arkadiusz Miśkiewicz wrote:
> On Saturday 10 of December 2011, Arkadiusz Miśkiewicz wrote:
> > Hello,
> >
> > I have few supermicro machines connected to edge-core switch using 1Gbps
> > uplink. The problem is that after boot I have to restart net to get
> > network card pass traffic. The problem is only with forcedeth network
> > cards. It happens on different supermicro servers and it happens for as
> > far as I can remember (so, 2.6.3x kernels, too). Currently I'm on 3.0.13.
> >
> > Logs below show state after fresh boot where network traffic doesn't get
> > through. Note that it shows that link is up but still - traffic doesn't
> > get passed. tcpdump on eth0 interface sees only traffic initiated from
> > this server and no traffic from outside.
> >
> > Tried to set every offload (via ethtool -K) to off but that didn't help.
> >
> > I have to make:
> > ip link set eth0 down; ip link set eth0 up
> > wait few second and then traffic starts to get passed. When I do that
> > this shows in dmesg: [ 1058.266480] forcedeth 0000:00:08.0: irq 40 for
> > MSI/MSI-X
> > [ 1069.163238] eth0: no IPv6 routers present
>
> One more note. This doesn't always happens like that. Sometimes I boot
> and I get link working, traffic gets passed fine:
rmmod forcedeth; modprobe forcedeth; ifup doesn't help
Counters on switch side after forcedeth machine reboot, no traffic being
passed, switch counters reset.
#show interfaces counters ethernet 1/19
Ethernet 1/19
Iftable Stats:
Octets Input: 0, Octets Output: 11116437
Unicast Input: 0, Unicast Output: 59
Discard Input: 0, Discard Output: 0
Error Input: 0, Error Output: 0
Unknown Protos Input: 0, QLen Output: 0
Extended Iftable Stats:
Multi-cast Input: 0, Multi-cast Output: 24742
Broadcast Input: 0, Broadcast Output: 790
Ether-like Stats:
Alignment Errors: 0, FCS Errors: 0
Single Collision Frames: 0, Multiple Collision Frames: 0
SQE Test Errors: 0, Deferred Transmissions: 0
Late Collisions: 0, Excessive Collisions: 0
Internal Mac Transmit Errors: 0, Internal Mac Receive Errors: 0
Frames Too Long: 0, Carrier Sense Errors: 0
Symbol Errors: 0
RMON Stats:
Drop Events: 0, Octets: 11075170, Packets: 25549
Broadcast PKTS: 790, Multi-cast PKTS: 24742
Undersize PKTS: 0, Oversize PKTS: 0
Fragments: 0, Jabbers: 0
CRC Align Errors: 0, Collisions: 0
Packet Size <= 64 Octets: 2060, Packet Size 65 to 127 Octets: 12764
Packet Size 128 to 255 Octets: 364, Packet Size 256 to 511 Octets: 292
Packet Size 512 to 1023 Octets: 8755, Packet Size 1024 to 1518 Octets: 1314
No errors here.
# dmesg|grep eth0
[ 33.432266] forcedeth 0000:00:08.0: ifname eth0, PHY OUI 0x5043 @ 2, addr
00:30:48:7a:57:a6
[ 33.960287] Loading kernel module for a network device with CAP_SYS_MODULE
(deprecated). Use CAP_NET_ADMIN and alias netdev-eth0 instead
[ 33.967925] forcedeth 0000:00:08.0: eth0: no link during initialization
[ 33.970995] ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 36.380311] forcedeth 0000:00:08.0: eth0: link up
[ 36.384016] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 47.150014] eth0: no IPv6 routers present
forcedeth side:
# ethtool -S eth0
NIC statistics:
tx_bytes: 69926
tx_zero_rexmt: 1011
tx_one_rexmt: 0
tx_many_rexmt: 0
tx_late_collision: 0
tx_fifo_errors: 0
tx_carrier_errors: 0
tx_excess_deferral: 0
tx_retry_error: 0
rx_frame_error: 0
rx_extra_byte: 0
rx_late_collision: 0
rx_runt: 0
rx_frame_too_long: 0
rx_over_errors: 0
rx_crc_errors: 0
rx_frame_align_error: 0
rx_length_error: 0
rx_unicast: 3047
rx_multicast: 0
rx_broadcast: 2113
rx_packets: 5160
rx_errors_total: 0
tx_errors_total: 0
tx_deferral: 0
tx_packets: 1011
rx_bytes: 382259
tx_pause: 0
rx_pause: 0
rx_drop_frame: 0
>
> [ 34.038476] forcedeth: Reverse Engineered nForce ethernet driver.
> Version 0.64. [ 34.038698] ACPI: PCI Interrupt Link [LMAC] enabled at
> IRQ 22
> [ 34.038705] forcedeth 0000:00:08.0: PCI INT A -> Link[LMAC] -> GSI 22
> (level, low) -> IRQ 22 [ 34.038710] forcedeth 0000:00:08.0: setting
> latency timer to 64 [ 34.565309] forcedeth 0000:00:08.0: ifname eth0,
> PHY OUI 0x5043 @ 2, addr 00:30:48:7a:b4:88 [ 34.565314] forcedeth
> 0000:00:08.0: highdma csum vlan pwrctl mgmt gbit lnktim msi desc-v3 [
> 34.566823] ACPI: PCI Interrupt Link [LMAD] enabled at IRQ 21
> [ 34.566831] forcedeth 0000:00:09.0: PCI INT A -> Link[LMAD] -> GSI 21
> (level, low) -> IRQ 21 [ 34.566836] forcedeth 0000:00:09.0: setting
> latency timer to 64 [ 35.091341] forcedeth 0000:00:09.0: ifname eth1,
> PHY OUI 0x5043 @ 3, addr 00:30:48:7a:b4:89 [ 35.091346] forcedeth
> 0000:00:09.0: highdma csum vlan pwrctl mgmt gbit lnktim msi desc-v3 [
> 35.093492] Loading kernel module for a network device with CAP_SYS_MODULE
> (deprecated). Use CAP_NET_ADMIN and alias netdev-eth0 instead [
> 35.101074] forcedeth 0000:00:08.0: irq 40 for MSI/MSI-X
> [ 35.101267] forcedeth 0000:00:08.0: eth0: no link during initialization
> [ 35.102322] ADDRCONF(NETDEV_UP): eth0: link is not ready
> [ 35.663785] ip_tables: (C) 2000-2006 Netfilter Core Team
> [ 35.756083] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
> [ 37.147869] forcedeth 0000:00:08.0: eth0: link up
> [ 37.149108] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> [ 47.176677] eth0: no IPv6 routers present
>
> > (CC to few people shown by get_maintainer.pl)
> >
> > LOGS/OUTPUTs from non working state after fresh boot.
> > # dmesg|grep eth
> > [ 34.528326] forcedeth: Reverse Engineered nForce ethernet driver.
> > Version 0.64. [ 34.528542] forcedeth 0000:00:08.0: PCI INT A ->
> > Link[LMAC] -> GSI 22 (level, low) -> IRQ 22 [ 34.528547] forcedeth
> > 0000:00:08.0: setting latency timer to 64 [ 35.055319] forcedeth
> > 0000:00:08.0: ifname eth0, PHY OUI 0x5043 @ 2, addr 00:30:48:7a:b4:88 [
> > 35.055324] forcedeth 0000:00:08.0: highdma csum vlan pwrctl mgmt gbit
> > lnktim msi desc-v3 [ 35.056480] forcedeth 0000:00:09.0: PCI INT A ->
> > Link[LMAD] -> GSI 21 (level, low) -> IRQ 21 [ 35.056485] forcedeth
> > 0000:00:09.0: setting latency timer to 64 [ 35.581960] forcedeth
> > 0000:00:09.0: ifname eth1, PHY OUI 0x5043 @ 3, addr 00:30:48:7a:b4:89 [
> > 35.581965] forcedeth 0000:00:09.0: highdma csum vlan pwrctl mgmt gbit
> > lnktim msi desc-v3 [ 35.584211] Loading kernel module for a network
> > device with CAP_SYS_MODULE (deprecated). Use CAP_NET_ADMIN and alias
> > netdev-eth0 instead [ 35.591778] forcedeth 0000:00:08.0: irq 40 for
> > MSI/MSI-X
> > [ 35.591970] forcedeth 0000:00:08.0: eth0: no link during
> > initialization [ 35.593030] ADDRCONF(NETDEV_UP): eth0: link is not
> > ready
> > [ 37.666757] forcedeth 0000:00:08.0: eth0: link up
> > [ 37.668624] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> > [ 48.150006] eth0: no IPv6 routers present
> >
> > lspci -vv:
> > 00:09.0 Bridge: nVidia Corporation MCP55 Ethernet (rev a3)
> >
> > Subsystem: Super Micro Computer Inc Device 1611
> > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> > ParErr-
> >
> > Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF- FastB2B+
> > ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> > Latency: 0 (250ns min, 5000ns max)
> >
> > Interrupt: pin A routed to IRQ 21
> > Region 0: Memory at feaf5000 (32-bit, non-prefetchable) [size=4K]
> > Region 1: I/O ports at b080 [size=8]
> > Region 2: Memory at feafa000 (32-bit, non-prefetchable)
> > [size=256] Region 3: Memory at feaf4c00 (32-bit,
> > non-prefetchable) [size=16] Capabilities: [44] Power Management
> > version 2
> >
> > Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA
> >
> > PME(D0+,D1+,D2+,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0
> > DScale=0 PME- Capabilities: [70] MSI-X: Enable- Count=8 Masked-
> >
> > Vector table: BAR=2 offset=00000000
> > PBA: BAR=3 offset=00000000
> >
> > Capabilities: [50] MSI: Enable- Count=1/8 Maskable+ 64bit+
> >
> > Address: 0000000000000000 Data: 0000
> > Masking: 00000000 Pending: 00000000
> >
> > Capabilities: [6c] HyperTransport: MSI Mapping Enable- Fixed+
> > Kernel driver in use: forcedeth
> >
> > # ethtool eth0
> >
> > Settings for eth0:
> > Supported ports: [ MII ]
> > Supported link modes: 10baseT/Half 10baseT/Full
> >
> > 100baseT/Half 100baseT/Full
> > 1000baseT/Full
> >
> > Supported pause frame use: No
> > Supports auto-negotiation: Yes
> > Advertised link modes: 10baseT/Half 10baseT/Full
> >
> > 100baseT/Half 100baseT/Full
> > 1000baseT/Full
> >
> > Advertised pause frame use: No
> > Advertised auto-negotiation: Yes
> > Speed: 1000Mb/s
> > Duplex: Full
> > Port: MII
> > PHYAD: 2
> > Transceiver: external
> > Auto-negotiation: on
> > Supports Wake-on: g
> > Wake-on: d
> > Link detected: yes
> >
> > # ethtool -S eth0
> >
> > NIC statistics:
> > tx_bytes: 46352
> > tx_zero_rexmt: 704
> > tx_one_rexmt: 0
> > tx_many_rexmt: 0
> > tx_late_collision: 0
> > tx_fifo_errors: 0
> > tx_carrier_errors: 0
> > tx_excess_deferral: 0
> > tx_retry_error: 0
> > rx_frame_error: 0
> > rx_extra_byte: 0
> > rx_late_collision: 0
> > rx_runt: 0
> > rx_frame_too_long: 0
> > rx_over_errors: 1
> > rx_crc_errors: 0
> > rx_frame_align_error: 0
> > rx_length_error: 0
> > rx_unicast: 31
> > rx_multicast: 0
> > rx_broadcast: 436
> > rx_packets: 467
> > rx_errors_total: 1
> > tx_errors_total: 0
> > tx_deferral: 0
> > tx_packets: 704
> > rx_bytes: 35872
> > tx_pause: 0
> > rx_pause: 0
> > rx_drop_frame: 170
> >
> > # ethtool -k eth0
> > Offload parameters for eth0:
> > rx-checksumming: on
> > tx-checksumming: on
> > scatter-gather: on
> > tcp-segmentation-offload: on
> > udp-fragmentation-offload: off
> > generic-segmentation-offload: on
> > generic-receive-offload: on
> > large-receive-offload: off
> > rx-vlan-offload: on
> > tx-vlan-offload: on
> > ntuple-filters: off
> > receive-hashing: off
> >
> >
> > 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc sfq state UP
> > qlen 1000 link/ether 00:30:48:7a:b4:88 brd ff:ff:ff:ff:ff:ff
--
Arkadiusz Miśkiewicz PLD/Linux Team
arekm / maven.pl http://ftp.pld-linux.org/
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: forcedeth doesn't pass traffic (3.0.x kernel and earliers)
2012-01-13 17:33 ` Arkadiusz Miśkiewicz
@ 2012-01-13 18:34 ` John A. Sullivan III
0 siblings, 0 replies; 4+ messages in thread
From: John A. Sullivan III @ 2012-01-13 18:34 UTC (permalink / raw)
To: Arkadiusz Miśkiewicz; +Cc: netdev
On Fri, 2012-01-13 at 18:33 +0100, Arkadiusz Miśkiewicz wrote:
> On Saturday 10 of December 2011, Arkadiusz Miśkiewicz wrote:
> > On Saturday 10 of December 2011, Arkadiusz Miśkiewicz wrote:
> > > Hello,
> > >
> > > I have few supermicro machines connected to edge-core switch using 1Gbps
> > > uplink. The problem is that after boot I have to restart net to get
> > > network card pass traffic. The problem is only with forcedeth network
> > > cards. It happens on different supermicro servers and it happens for as
> > > far as I can remember (so, 2.6.3x kernels, too). Currently I'm on 3.0.13.
> > >
> > > Logs below show state after fresh boot where network traffic doesn't get
> > > through. Note that it shows that link is up but still - traffic doesn't
> > > get passed. tcpdump on eth0 interface sees only traffic initiated from
> > > this server and no traffic from outside.
> > >
> > > Tried to set every offload (via ethtool -K) to off but that didn't help.
> > >
> > > I have to make:
> > > ip link set eth0 down; ip link set eth0 up
> > > wait few second and then traffic starts to get passed. When I do that
> > > this shows in dmesg: [ 1058.266480] forcedeth 0000:00:08.0: irq 40 for
> > > MSI/MSI-X
> > > [ 1069.163238] eth0: no IPv6 routers present
> >
> > One more note. This doesn't always happens like that. Sometimes I boot
> > and I get link working, traffic gets passed fine:
>
<snip>
We have noticed similar behavior on some of our SuperMicro systems. It
was worse for us; the interfaces would work fine on boot but then
inexplicably stop after weeks of running causing production outages. We
simply stopped using them - John
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2012-01-13 18:35 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-12-10 21:26 forcedeth doesn't pass traffic (3.0.x kernel and earliers) Arkadiusz Miśkiewicz
2011-12-10 21:31 ` Arkadiusz Miśkiewicz
2012-01-13 17:33 ` Arkadiusz Miśkiewicz
2012-01-13 18:34 ` John A. Sullivan III
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).