* niu interface automatically goes up then down
@ 2013-01-09 22:58 Jan Engelhardt
2013-01-09 23:56 ` David Miller
2013-01-10 0:24 ` Julian Calaby
0 siblings, 2 replies; 6+ messages in thread
From: Jan Engelhardt @ 2013-01-09 22:58 UTC (permalink / raw)
To: Linux Networking Developer Mailing List; +Cc: sparclinux
Hi,
I am observing that a machine with a niu card/driver automatically
enables itself and then goes dormant again.
[ 4410.930078] niu 0000:10:00.0 eth4: Link is up at 100Mbit/sec, full duplex
[ 4410.930137] br0: port 4(eth4) entered forwarding state
[ 4410.930156] br0: port 4(eth4) entered forwarding state
[ 4420.957745] niu 0000:10:00.0 eth4: Link is down
The interface itself is marked UP, and is part of a bridge,
if that matters. The kernel version is 3.7.1 on sparc64.
6: eth4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500
qdisc mq master br0 state DOWN qlen 1000
link/ether 00:21:28:71:32:5a brd ff:ff:ff:ff:ff:ff
inet6 fe80::221:28ff:fe71:325a/64 scope link
valid_lft forever preferred_lft forever
At the same time, eth5, to which a cable+machine _is_ connected,
randomly goes out. At first I thought it might be the connected peer,
but seeing that eth4 randomly does a up-down cycle leads me to assume
that niu is doing a cycle here rather than the peer.
[ 3839.724721] niu 0000:10:00.1 eth5: Link is down
[ 3839.725077] br0: port 5(eth5) entered disabled state
[ 3840.725016] niu 0000:10:00.1 eth5: Link is up at 100Mbit/sec, half duplex
[ 3840.725195] br0: port 5(eth5) entered forwarding state
[ 3840.725327] br0: port 5(eth5) entered forwarding state
[ 3855.762171] br0: port 5(eth5) entered forwarding state
dmesg for niu:
[ 60.158957] niu: niu.c:v1.1 (Apr 22, 2010)
[ 60.160457] niu: niu0: Found PHY 002060b1 type MII at phy_port 10
[ 60.160887] niu: niu0: Found PHY 002060b1 type MII at phy_port 11
[ 60.161314] niu: niu0: Found PHY 002060b1 type MII at phy_port 12
[ 60.161739] niu: niu0: Found PHY 002060b1 type MII at phy_port 13
[ 60.168250] niu: niu0: Port 0 [4 RX chans] [6 TX chans]
[ 60.168319] niu: niu0: Port 1 [4 RX chans] [6 TX chans]
[ 60.168386] niu: niu0: Port 2 [4 RX chans] [6 TX chans]
[ 60.168452] niu: niu0: Port 3 [4 RX chans] [6 TX chans]
[ 60.168519] niu: niu0: Port 0 RDC tbl(0) [ 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 ]
[ 60.168913] niu: niu0: Port 0 RDC tbl(1) [ 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 ]
[ 60.169213] niu: niu0: Port 1 RDC tbl(2) [ 4 5 6 7 4 5 6 7 4 5 6 7 4 5 6 7 ]
[ 60.169503] niu: niu0: Port 1 RDC tbl(3) [ 4 5 6 7 4 5 6 7 4 5 6 7 4 5 6 7 ]
[ 60.169790] niu: niu0: Port 2 RDC tbl(4) [ 8 9 10 11 8 9 10 11 8 9 10 11 8 9 10 11 ]
[ 60.170078] niu: niu0: Port 2 RDC tbl(5) [ 8 9 10 11 8 9 10 11 8 9 10 11 8 9 10 11 ]
[ 60.170366] niu: niu0: Port 3 RDC tbl(6) [ 12 13 14 15 12 13 14 15 12 13 14 15 12 13 14 15 ]
[ 60.170657] niu: niu0: Port 3 RDC tbl(7) [ 12 13 14 15 12 13 14 15 12 13 14 15 12 13 14 15 ]
[ 60.416607] niu: eth4: NIU Ethernet 00:21:28:71:32:5a
[ 60.416679] niu: eth4: Port type[XMAC] mode[1G:COPPER] XCVR[MII] phy[mif]
[ 60.624669] niu: eth5: NIU Ethernet 00:21:28:71:32:5b
[ 60.624741] niu: eth5: Port type[XMAC] mode[1G:COPPER] XCVR[MII] phy[mif]
[ 60.832774] niu: eth6: NIU Ethernet 00:21:28:71:32:5c
[ 60.832845] niu: eth6: Port type[BMAC] mode[1G:COPPER] XCVR[MII] phy[mif]
[ 61.040806] niu: eth7: NIU Ethernet 00:21:28:71:32:5d
[ 61.040877] niu: eth7: Port type[BMAC] mode[1G:COPPER] XCVR[MII] phy[mif]
lspci:
10:00.0 Ethernet controller: Sun Microsystems Computer Corp. Multithreaded 10-Gigabit Ethernet Network Controller (rev 01)
Subsystem: Sun Microsystems Computer Corp. Device 0000
Flags: bus master, fast devsel, latency 0, IRQ 00000015
Memory at 01000000 (64-bit, non-prefetchable) [size=16M]
Memory at 00d00000 (64-bit, non-prefetchable) [size=32K]
Memory at 00d08000 (64-bit, non-prefetchable) [size=32K]
Expansion ROM at 00e00000 [disabled] [size=1M]
Capabilities: [40] Power Management version 2
Capabilities: [50] MSI: Enable- Count=1/32 Maskable+ 64bit+
Capabilities: [70] MSI-X: Enable+ Count=32 Masked-
Capabilities: [80] Express Endpoint, MSI 00
Capabilities: [94] Vendor Specific Information: Len=08 <?>
Capabilities: [9c] Vendor Specific Information: Len=40 <?>
Capabilities: [100] Advanced Error Reporting
Kernel driver in use: niu
10:00.1 Ethernet controller: Sun Microsystems Computer Corp. Multithreaded 10-Gigabit Ethernet Network Controller (rev 01)
Subsystem: Sun Microsystems Computer Corp. Device 0000
Flags: bus master, fast devsel, latency 0, IRQ 00000016
Memory at 02000000 (64-bit, non-prefetchable) [size=16M]
Memory at 00d10000 (64-bit, non-prefetchable) [size=32K]
Memory at 00d18000 (64-bit, non-prefetchable) [size=32K]
Expansion ROM at 00f00000 [disabled] [size=1M]
Capabilities: [40] Power Management version 2
Capabilities: [50] MSI: Enable- Count=1/32 Maskable+ 64bit+
Capabilities: [70] MSI-X: Enable+ Count=32 Masked-
Capabilities: [80] Express Endpoint, MSI 00
Capabilities: [94] Vendor Specific Information: Len=08 <?>
Capabilities: [9c] Vendor Specific Information: Len=40 <?>
Capabilities: [100] Advanced Error Reporting
Kernel driver in use: niu
10:00.2 Ethernet controller: Sun Microsystems Computer Corp. Multithreaded 10-Gigabit Ethernet Network Controller (rev 01)
Subsystem: Sun Microsystems Computer Corp. Device 0000
Flags: bus master, fast devsel, latency 0, IRQ 00000017
Memory at 03000000 (64-bit, non-prefetchable) [size=16M]
Memory at 00d20000 (64-bit, non-prefetchable) [size=32K]
Memory at 00d28000 (64-bit, non-prefetchable) [size=32K]
Expansion ROM at 04000000 [disabled] [size=1M]
Capabilities: [40] Power Management version 2
Capabilities: [50] MSI: Enable- Count=1/32 Maskable+ 64bit+
Capabilities: [70] MSI-X: Enable+ Count=32 Masked-
Capabilities: [80] Express Endpoint, MSI 00
Capabilities: [94] Vendor Specific Information: Len=08 <?>
Capabilities: [9c] Vendor Specific Information: Len=40 <?>
Capabilities: [100] Advanced Error Reporting
Kernel driver in use: niu
10:00.3 Ethernet controller: Sun Microsystems Computer Corp. Multithreaded 10-Gigabit Ethernet Network Controller (rev 01)
Subsystem: Sun Microsystems Computer Corp. Device 0000
Flags: bus master, fast devsel, latency 0, IRQ 00000014
Memory at 05000000 (64-bit, non-prefetchable) [size=16M]
Memory at 00d30000 (64-bit, non-prefetchable) [size=32K]
Memory at 00d38000 (64-bit, non-prefetchable) [size=32K]
Expansion ROM at 04100000 [disabled] [size=1M]
Capabilities: [40] Power Management version 2
Capabilities: [50] MSI: Enable- Count=1/32 Maskable+ 64bit+
Capabilities: [70] MSI-X: Enable+ Count=32 Masked-
Capabilities: [80] Express Endpoint, MSI 00
Capabilities: [94] Vendor Specific Information: Len=08 <?>
Capabilities: [9c] Vendor Specific Information: Len=40 <?>
Capabilities: [100] Advanced Error Reporting
Kernel driver in use: niu
Any other info needed?
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: niu interface automatically goes up then down
2013-01-09 22:58 niu interface automatically goes up then down Jan Engelhardt
@ 2013-01-09 23:56 ` David Miller
2013-01-10 0:24 ` Julian Calaby
1 sibling, 0 replies; 6+ messages in thread
From: David Miller @ 2013-01-09 23:56 UTC (permalink / raw)
To: jengelh; +Cc: netdev, sparclinux
I'm just letting you know ahead of time that, given my current
workload and travel schedule, I will not be able to look into
this problem at all and someone else will need to do so.
Thanks.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: niu interface automatically goes up then down
2013-01-09 22:58 niu interface automatically goes up then down Jan Engelhardt
2013-01-09 23:56 ` David Miller
@ 2013-01-10 0:24 ` Julian Calaby
2013-01-10 0:55 ` Jan Engelhardt
1 sibling, 1 reply; 6+ messages in thread
From: Julian Calaby @ 2013-01-10 0:24 UTC (permalink / raw)
To: Jan Engelhardt; +Cc: Linux Networking Developer Mailing List, sparclinux
Hi Jan,
I'm not familiar with the hardware or drivers, so I won't be able to
help you that much either, but I can ask a couple of pointed
questions. =)
On Thu, Jan 10, 2013 at 9:58 AM, Jan Engelhardt <jengelh@inai.de> wrote:
> Hi,
>
> I am observing that a machine with a niu card/driver automatically
> enables itself and then goes dormant again.
>
> [ 4410.930078] niu 0000:10:00.0 eth4: Link is up at 100Mbit/sec, full duplex
> [ 4410.930137] br0: port 4(eth4) entered forwarding state
> [ 4410.930156] br0: port 4(eth4) entered forwarding state
> [ 4420.957745] niu 0000:10:00.0 eth4: Link is down
>
> The interface itself is marked UP, and is part of a bridge,
> if that matters. The kernel version is 3.7.1 on sparc64.
>
> 6: eth4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500
> qdisc mq master br0 state DOWN qlen 1000
> link/ether 00:21:28:71:32:5a brd ff:ff:ff:ff:ff:ff
> inet6 fe80::221:28ff:fe71:325a/64 scope link
> valid_lft forever preferred_lft forever
That is rather odd. Is this particular interface connected to anything
that could be causing this?
> At the same time, eth5, to which a cable+machine _is_ connected,
> randomly goes out. At first I thought it might be the connected peer,
> but seeing that eth4 randomly does a up-down cycle leads me to assume
> that niu is doing a cycle here rather than the peer.
>
> [ 3839.724721] niu 0000:10:00.1 eth5: Link is down
> [ 3839.725077] br0: port 5(eth5) entered disabled state
> [ 3840.725016] niu 0000:10:00.1 eth5: Link is up at 100Mbit/sec, half duplex
> [ 3840.725195] br0: port 5(eth5) entered forwarding state
> [ 3840.725327] br0: port 5(eth5) entered forwarding state
> [ 3855.762171] br0: port 5(eth5) entered forwarding state
Again, could it be the device at the end of the link?
Out of curiosity, why do you have these two ports bridged together and
what is the purpose of this configuration?
Thanks,
--
Julian Calaby
Email: julian.calaby@gmail.com
Profile: http://www.google.com/profiles/julian.calaby/
.Plan: http://sites.google.com/site/juliancalaby/
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: niu interface automatically goes up then down
2013-01-10 0:24 ` Julian Calaby
@ 2013-01-10 0:55 ` Jan Engelhardt
2013-01-10 1:27 ` Julian Calaby
0 siblings, 1 reply; 6+ messages in thread
From: Jan Engelhardt @ 2013-01-10 0:55 UTC (permalink / raw)
To: Julian Calaby; +Cc: Linux Networking Developer Mailing List, sparclinux
On Thursday 2013-01-10 01:24, Julian Calaby wrote:
>>
>> The interface itself is marked UP, and is part of a bridge,
>> if that matters. The kernel version is 3.7.1 on sparc64.
>>
>> 6: eth4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500
>> qdisc mq master br0 state DOWN qlen 1000
>> link/ether 00:21:28:71:32:5a brd ff:ff:ff:ff:ff:ff
>> inet6 fe80::221:28ff:fe71:325a/64 scope link
>> valid_lft forever preferred_lft forever
>
>That is rather odd. Is this particular interface connected to anything
>that could be causing this?
Nope, there is nothing connected to eth4. UP indicates that
I had issue `ip link set dev eth4 up`.
>> At the same time, eth5, to which a cable+machine _is_ connected,
>> randomly goes out. At first I thought it might be the connected peer,
>> but seeing that eth4 randomly does a up-down cycle leads me to assume
>> that niu is doing a cycle here rather than the peer.
>>
>> [ 3839.724721] niu 0000:10:00.1 eth5: Link is down
>> [ 3839.725077] br0: port 5(eth5) entered disabled state
>> [ 3840.725016] niu 0000:10:00.1 eth5: Link is up at 100Mbit/sec, half duplex
>> [ 3840.725195] br0: port 5(eth5) entered forwarding state
>> [ 3840.725327] br0: port 5(eth5) entered forwarding state
>> [ 3855.762171] br0: port 5(eth5) entered forwarding state
>
>Again, could it be the device at the end of the link?
>Out of curiosity, why do you have these two ports bridged together and
>what is the purpose of this configuration?
Nobody expects the... lack of a switch. And since there are
plenty of ports in the machine anyway, might as well use them as a
software switch as an intermediate solution.
eth0 through eth3 is a quad-port e1000e.
eth4 through eth7 is the quad-port niu.
The e1000e ports don't flake out at all, therefore I rate
the peer(s) being at fault with a very low probability.
The issue is not pressing, since it's just service processors
which are connected.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: niu interface automatically goes up then down
2013-01-10 0:55 ` Jan Engelhardt
@ 2013-01-10 1:27 ` Julian Calaby
2013-01-10 1:52 ` Jan Engelhardt
0 siblings, 1 reply; 6+ messages in thread
From: Julian Calaby @ 2013-01-10 1:27 UTC (permalink / raw)
To: Jan Engelhardt; +Cc: Linux Networking Developer Mailing List, sparclinux
Hi Jan,
Thanks for the answers.
On Thu, Jan 10, 2013 at 11:55 AM, Jan Engelhardt <jengelh@inai.de> wrote:
>
> On Thursday 2013-01-10 01:24, Julian Calaby wrote:
>>>
>>> The interface itself is marked UP, and is part of a bridge,
>>> if that matters. The kernel version is 3.7.1 on sparc64.
>>>
>>> 6: eth4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500
>>> qdisc mq master br0 state DOWN qlen 1000
>>> link/ether 00:21:28:71:32:5a brd ff:ff:ff:ff:ff:ff
>>> inet6 fe80::221:28ff:fe71:325a/64 scope link
>>> valid_lft forever preferred_lft forever
>>
>>That is rather odd. Is this particular interface connected to anything
>>that could be causing this?
>
> Nope, there is nothing connected to eth4. UP indicates that
> I had issue `ip link set dev eth4 up`.
I assumed that from the "NO-CARRIER" status, but I had to check: for
all I know it's some sort of failover connection and is hooked up to a
disabled port on a different switch.
>>> At the same time, eth5, to which a cable+machine _is_ connected,
>>> randomly goes out. At first I thought it might be the connected peer,
>>> but seeing that eth4 randomly does a up-down cycle leads me to assume
>>> that niu is doing a cycle here rather than the peer.
>>>
>>> [ 3839.724721] niu 0000:10:00.1 eth5: Link is down
>>> [ 3839.725077] br0: port 5(eth5) entered disabled state
>>> [ 3840.725016] niu 0000:10:00.1 eth5: Link is up at 100Mbit/sec, half duplex
>>> [ 3840.725195] br0: port 5(eth5) entered forwarding state
>>> [ 3840.725327] br0: port 5(eth5) entered forwarding state
>>> [ 3855.762171] br0: port 5(eth5) entered forwarding state
>>
>>Again, could it be the device at the end of the link?
>>Out of curiosity, why do you have these two ports bridged together and
>>what is the purpose of this configuration?
>
> Nobody expects the... lack of a switch. And since there are
> plenty of ports in the machine anyway, might as well use them as a
> software switch as an intermediate solution.
True that. I have an overabundance of PCI network cards and have built
temporary servers with two of them bridged together so I'm not
stealing a port that might be needed.
> eth0 through eth3 is a quad-port e1000e.
> eth4 through eth7 is the quad-port niu.
>
> The e1000e ports don't flake out at all, therefore I rate
> the peer(s) being at fault with a very low probability.
Makes sense.
> The issue is not pressing, since it's just service processors
> which are connected.
Ok, I'm guessing that this is production so heavy debugging isn't
going to happen.
I'm out of ideas. Tag?
Thanks,
--
Julian Calaby
Email: julian.calaby@gmail.com
Profile: http://www.google.com/profiles/julian.calaby/
.Plan: http://sites.google.com/site/juliancalaby/
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: niu interface automatically goes up then down
2013-01-10 1:27 ` Julian Calaby
@ 2013-01-10 1:52 ` Jan Engelhardt
0 siblings, 0 replies; 6+ messages in thread
From: Jan Engelhardt @ 2013-01-10 1:52 UTC (permalink / raw)
To: Julian Calaby; +Cc: Linux Networking Developer Mailing List, sparclinux
On Thursday 2013-01-10 02:27, Julian Calaby wrote:
>
>> The issue is not pressing, since it's just service processors
>> which are connected.
>
>Ok, I'm guessing that this is production so heavy debugging isn't
>going to happen.
No production, free to toy.
>I'm out of ideas. Tag?
Here is a fun fact. If I disable autonegotiation (via ethtool) on the
link on the niu side, the remote side will deassert the carrier and
won't return it until autoneg is turned on again.
In fact, if I do `ethtool -s eth7 autoneg off; ethtool -s eth7 autoneg
on;`, it will take a good exact second (~1.000400 s) for the HW to
reestablish an autonegotiated link. This delay is of the same magnitude
and value as when the link bounces on its own.
Hypothesis: maybe the niu HW drops autoneg for a splitsecond (or it
gets lost due to some other reason), causing the observed bounces.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-01-10 1:52 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-01-09 22:58 niu interface automatically goes up then down Jan Engelhardt
2013-01-09 23:56 ` David Miller
2013-01-10 0:24 ` Julian Calaby
2013-01-10 0:55 ` Jan Engelhardt
2013-01-10 1:27 ` Julian Calaby
2013-01-10 1:52 ` Jan Engelhardt
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).