MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[marcmicalizzi]

I’ve just been playing with the mcbin for a little while now, and this
kernel tree (mcbin branch) seems to be the only way (using a modern kernel)
I’ve found that actually gets the 10GB SFP ports working on the SingleShot. 

Unfortunately, however, the GPON SFP I have (Huawei MA5671, provided by my
ISP) does not seem to work on eth3 (mvpp2x driver). Currently I’m using it
successfully in a different machine using a Broadcom BCM57810S (bnx2x
warpcore) with a couple modifications to the driver to link at 2500BASE-X,
and support SC modules, but was hoping to move it over to the MacchiatoBin
and try it as a router.

Worth noting is one quirk of this SFP is that it does not have a physical
EEPROM, and instead has a soft EEPROM that takes some time to load after
powering.

On boot I receive: 
[    5.376559] sfp sfp-eth3: failed to read EEPROM: -6
Which also keeps posting to dmesg afterwards.

And with ethtool -m: 
# ethtool -m eth3
Cannot get Module EEPROM data: No such device or address

I also noticed from `ethtool eth3` that it does not explicitly list
2500Base-X as a supported link or advertisement mode with the SFP plugged,
and afterwards (otherwise it does):
mcbin # ethtool eth3
Settings for eth3:
        Supported ports: [ MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Speed: 10Mb/s
        Duplex: Half
        Port: MII
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: on
        Link detected: no

From the machine I have that does work with the SFP:
server # ethtool -m eth0
        Identifier                                : 0x03 (SFP)
        Extended identifier                       : 0x04 (GBIC/SFP defined
by 2-wire interface ID)
        Connector                                 : 0x01 (SC)
        Transceiver codes                         : 0x00 0x00 0x00 0x02 0x00
0x00 0x00 0x00
        Transceiver type                          : Ethernet: 1000BASE-LX
        Encoding                                  : 0x03 (NRZ)
        BR, Nominal                               : 1200MBd
        Rate identifier                           : 0x00 (unspecified)
        Length (SMF,km)                           : 20km
        Length (SMF)                              : 20000m
        Length (50um)                             : 0m
        Length (62.5um)                           : 0m
        Length (Copper)                           : 0m
        Length (OM3)                              : 0m
        Laser wavelength                          : 1310nm
        Vendor name                               : HUAWEI
        Vendor OUI                                : 00:00:00
        Vendor PN                                 : MA5671A
        Vendor rev                                : 0000
        Option values                             : 0x00 0x1a
        Option                                    : RX_LOS implemented
        Option                                    : TX_FAULT implemented
        Option                                    : TX_DISABLE implemented
        BR margin, max                            : 0%
        BR margin, min                            : 0%
        Vendor SN                                 : xxxxxxxxxxxxxxxxx
        Date code                                 : 180105
        Optical diagnostics support               : Yes
        Laser bias current                        : 8.058 mA
        Laser output power                        : 1.4902 mW / 1.73 dBm
        Receiver signal average optical power     : 0.0104 mW / -19.83 dBm
        Module temperature                        : 41.66 degrees C / 106.99
degrees F
        Module voltage                            : 3.2717 V
        Alarm/warning flags implemented           : Yes
        Laser bias current high alarm             : Off
        Laser bias current low alarm              : Off
        Laser bias current high warning           : Off
        Laser bias current low warning            : Off
        Laser output power high alarm             : Off
        Laser output power low alarm              : Off
        Laser output power high warning           : Off
        Laser output power low warning            : Off
        Module temperature high alarm             : Off
        Module temperature low alarm              : Off
        Module temperature high warning           : Off
        Module temperature low warning            : Off
        Module voltage high alarm                 : Off
        Module voltage low alarm                  : Off
        Module voltage high warning               : Off
        Module voltage low warning                : Off
        Laser rx power high alarm                 : Off
        Laser rx power low alarm                  : Off
        Laser rx power high warning               : Off
        Laser rx power low warning                : Off
        Laser bias current high alarm threshold   : 90.000 mA
        Laser bias current low alarm threshold    : 0.000 mA
        Laser bias current high warning threshold : 70.000 mA
        Laser bias current low warning threshold  : 0.000 mA
        Laser output power high alarm threshold   : 3.9810 mW / 6.00 dBm
        Laser output power low alarm threshold    : 0.8912 mW / -0.50 dBm
        Laser output power high warning threshold : 3.1622 mW / 5.00 dBm
        Laser output power low warning threshold  : 1.1220 mW / 0.50 dBm
        Module temperature high alarm threshold   : 95.00 degrees C / 203.00
degrees F
        Module temperature low alarm threshold    : -50.00 degrees C /
-58.00 degrees F
        Module temperature high warning threshold : 90.00 degrees C / 194.00
degrees F
        Module temperature low warning threshold  : -45.00 degrees C /
-49.00 degrees F
        Module voltage high alarm threshold       : 3.6000 V
        Module voltage low alarm threshold        : 3.0000 V
        Module voltage high warning threshold     : 3.5000 V
        Module voltage low warning threshold      : 3.1000 V
        Laser rx power high alarm threshold       : 0.2511 mW / -6.00 dBm
        Laser rx power low alarm threshold        : 0.0013 mW / -28.86 dBm
        Laser rx power high warning threshold     : 0.1995 mW / -7.00 dBm
        Laser rx power low warning threshold      : 0.0016 mW / -27.96 dBm
server # ethtool eth0
Settings for eth0:
        Supported ports: [ FIBRE ]
        Supported link modes:   1000baseT/Full
                                2500baseX/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: No
        Advertised link modes:  2500baseX/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Speed: 2500Mb/s
        Duplex: Full
        Port: FIBRE
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: g
        Wake-on: d
        Current message level: 0x00000000 (0)

        Link detected: yes

If you have any ideas or can provide any direction, I would greatly
appreciate it.

Thanks,
Marc

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Russell King - ARM Linux admin]

On Sat, Mar 02, 2019 at 01:00:28PM -0500, marcmicalizzi@gmail.com wrote:
> I’ve just been playing with the mcbin for a little while now, and this
> kernel tree (mcbin branch) seems to be the only way (using a modern kernel)
> I’ve found that actually gets the 10GB SFP ports working on the SingleShot. 
> 
> Unfortunately, however, the GPON SFP I have (Huawei MA5671, provided by my
> ISP) does not seem to work on eth3 (mvpp2x driver). Currently I’m using it
> successfully in a different machine using a Broadcom BCM57810S (bnx2x
> warpcore) with a couple modifications to the driver to link at 2500BASE-X,
> and support SC modules, but was hoping to move it over to the MacchiatoBin
> and try it as a router.
> 
> Worth noting is one quirk of this SFP is that it does not have a physical
> EEPROM, and instead has a soft EEPROM that takes some time to load after
> powering.
> 
> On boot I receive: 
> [    5.376559] sfp sfp-eth3: failed to read EEPROM: -6
> Which also keeps posting to dmesg afterwards.

If it can't read the EEPROM, then we have no way to know the capabilities
of the device.

> And with ethtool -m: 
> # ethtool -m eth3
> Cannot get Module EEPROM data: No such device or address
> 
> I also noticed from `ethtool eth3` that it does not explicitly list
> 2500Base-X as a supported link or advertisement mode with the SFP plugged,
> and afterwards (otherwise it does):
> mcbin # ethtool eth3
> Settings for eth3:
>         Supported ports: [ MII ]
>         Supported link modes:   10baseT/Half 10baseT/Full
>                                 100baseT/Half 100baseT/Full
>                                 1000baseT/Full
>         Supported pause frame use: Symmetric Receive-only
>         Supports auto-negotiation: Yes
>         Advertised link modes:  10baseT/Half 10baseT/Full
>                                 100baseT/Half 100baseT/Full
>                                 1000baseT/Full
>         Advertised pause frame use: Symmetric Receive-only
>         Advertised auto-negotiation: Yes
>         Speed: 10Mb/s
>         Duplex: Half
>         Port: MII
>         PHYAD: 0
>         Transceiver: internal
>         Auto-negotiation: on
>         Link detected: no
> 
> From the machine I have that does work with the SFP:
> server # ethtool -m eth0
>         Identifier                                : 0x03 (SFP)
>         Extended identifier                       : 0x04 (GBIC/SFP defined
> by 2-wire interface ID)
>         Connector                                 : 0x01 (SC)
>         Transceiver codes                         : 0x00 0x00 0x00 0x02 0x00
> 0x00 0x00 0x00
>         Transceiver type                          : Ethernet: 1000BASE-LX
>         Encoding                                  : 0x03 (NRZ)
>         BR, Nominal                               : 1200MBd
>         Rate identifier                           : 0x00 (unspecified)
>         Length (SMF,km)                           : 20km
>         Length (SMF)                              : 20000m
>         Length (50um)                             : 0m
>         Length (62.5um)                           : 0m
>         Length (Copper)                           : 0m
>         Length (OM3)                              : 0m
>         Laser wavelength                          : 1310nm
>         Vendor name                               : HUAWEI
>         Vendor OUI                                : 00:00:00
>         Vendor PN                                 : MA5671A
>         Vendor rev                                : 0000
>         Option values                             : 0x00 0x1a
>         Option                                    : RX_LOS implemented
>         Option                                    : TX_FAULT implemented
>         Option                                    : TX_DISABLE implemented
>         BR margin, max                            : 0%
>         BR margin, min                            : 0%
>         Vendor SN                                 : xxxxxxxxxxxxxxxxx
>         Date code                                 : 180105
>         Optical diagnostics support               : Yes
>         Laser bias current                        : 8.058 mA
>         Laser output power                        : 1.4902 mW / 1.73 dBm
>         Receiver signal average optical power     : 0.0104 mW / -19.83 dBm
>         Module temperature                        : 41.66 degrees C / 106.99
> degrees F
>         Module voltage                            : 3.2717 V
>         Alarm/warning flags implemented           : Yes
>         Laser bias current high alarm             : Off
>         Laser bias current low alarm              : Off
>         Laser bias current high warning           : Off
>         Laser bias current low warning            : Off
>         Laser output power high alarm             : Off
>         Laser output power low alarm              : Off
>         Laser output power high warning           : Off
>         Laser output power low warning            : Off
>         Module temperature high alarm             : Off
>         Module temperature low alarm              : Off
>         Module temperature high warning           : Off
>         Module temperature low warning            : Off
>         Module voltage high alarm                 : Off
>         Module voltage low alarm                  : Off
>         Module voltage high warning               : Off
>         Module voltage low warning                : Off
>         Laser rx power high alarm                 : Off
>         Laser rx power low alarm                  : Off
>         Laser rx power high warning               : Off
>         Laser rx power low warning                : Off
>         Laser bias current high alarm threshold   : 90.000 mA
>         Laser bias current low alarm threshold    : 0.000 mA
>         Laser bias current high warning threshold : 70.000 mA
>         Laser bias current low warning threshold  : 0.000 mA
>         Laser output power high alarm threshold   : 3.9810 mW / 6.00 dBm
>         Laser output power low alarm threshold    : 0.8912 mW / -0.50 dBm
>         Laser output power high warning threshold : 3.1622 mW / 5.00 dBm
>         Laser output power low warning threshold  : 1.1220 mW / 0.50 dBm
>         Module temperature high alarm threshold   : 95.00 degrees C / 203.00
> degrees F
>         Module temperature low alarm threshold    : -50.00 degrees C /
> -58.00 degrees F
>         Module temperature high warning threshold : 90.00 degrees C / 194.00
> degrees F
>         Module temperature low warning threshold  : -45.00 degrees C /
> -49.00 degrees F
>         Module voltage high alarm threshold       : 3.6000 V
>         Module voltage low alarm threshold        : 3.0000 V
>         Module voltage high warning threshold     : 3.5000 V
>         Module voltage low warning threshold      : 3.1000 V
>         Laser rx power high alarm threshold       : 0.2511 mW / -6.00 dBm
>         Laser rx power low alarm threshold        : 0.0013 mW / -28.86 dBm
>         Laser rx power high warning threshold     : 0.1995 mW / -7.00 dBm
>         Laser rx power low warning threshold      : 0.0016 mW / -27.96 dBm

That looks like it is able to read the EEPROM, including the
diagnostics.

Our SFP initialisation conforms to the specifications, so I'm at a loss
to make any suggestions.  Many SFPs have been tested, and some others
have tested GPON modules which afaik have worked.

However, I have no GPON modules, and even if I did, I wouldn't be able
to test them beyond merely plugging them in - it is unlikely that I'd
be able to get GPON based Internet here for anything under a few 
thousand pounds sterling (to pay for the roads to be dug up to lay a
FTTP fiber) with a heafty monthly fee on top, if it's even available
(which it isn't.)  The UK - or at least the area just outside London -
tends to be an Internet backwater (see my signature for how our Super!
Fast! Fiber! Optic! Broadband!, aka FTTC, performs.)

As I presently have no income, I really don't want to go and buy GPON
modules for one-off testing.  However, if one were to drop through the
letterbox...

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Andrew Lunn]

On Sat, Mar 02, 2019 at 01:00:28PM -0500, marcmicalizzi@gmail.com wrote:
> I’ve just been playing with the mcbin for a little while now, and this
> kernel tree (mcbin branch) seems to be the only way (using a modern kernel)
> I’ve found that actually gets the 10GB SFP ports working on the SingleShot. 
> 
> Unfortunately, however, the GPON SFP I have (Huawei MA5671, provided by my
> ISP) does not seem to work on eth3 (mvpp2x driver). Currently I’m using it
> successfully in a different machine using a Broadcom BCM57810S (bnx2x
> warpcore) with a couple modifications to the driver to link at 2500BASE-X,
> and support SC modules, but was hoping to move it over to the MacchiatoBin
> and try it as a router.
> 
> Worth noting is one quirk of this SFP is that it does not have a physical
> EEPROM, and instead has a soft EEPROM that takes some time to load after
> powering.
 
I've received reports that some GPON SFPs don't like block reads of
their i2c bus. You have to do 16bit word reads.

There was a patch a while ago to deal with i2c bus drivers which could
also not handle big block transfers:

https://www.spinics.net/lists/netdev/msg546524.html

I don't think the comments were every addresses, so it did not get
merged. But you can try reviving the patch, fix it up, and see if
smaller reads help with your problems.

	Andrew

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Marc Micalizzi]

On Sat, Mar 2, 2019 at 3:26 PM Andrew Lunn <andrew@lunn.ch> wrote:
> I've received reports that some GPON SFPs don't like block reads of
> their i2c bus. You have to do 16bit word reads.
>
> There was a patch a while ago to deal with i2c bus drivers which could
> also not handle big block transfers:
>
> https://www.spinics.net/lists/netdev/msg546524.html


I looked into that patch, and after doing a bit of investigation in to
where it's actually failing, I'm not sure it would make any
difference. I added a debug message in i2c_mv64xxx.c
mv64xxx_i2c_fsm(), which is where my -6 ENXIO is originating from,
immediately after the fallthrough for
MV64XXX_I2C_STATUS_MAST_WR_ADDR_NO_ACK (on line 314), and that
provided me with:
```i2c i2c-1: No device at other end-- state: 0x4, status: 0x20, addr:
0x50, flags: 0x0```

I'm not really too sure where to go from there, both due to the fact I
have _very_ limited kernel development experience (read: just slightly
more than none), and the only comparison I have to make, that I have
any familiarity with, is the bnx2x driver (from attempting to get it
to link at 2500BASE-X, which someone else suceeded at), which makes
use of it's own registers for i2c.
I forgot to mention in the original message, and it is worth noting
that the ixgbe driver works out of the box on an x520 (82599) with the
module as well (albeit linked at 1000BASE-X and not 2500BASE-X, as
there is no 2500BASE-X support for the 82599).

I am going to see if I can get my hands on a second GPON SFP of the
same make and model somehow, so at the very least I don't have to
bring my internet down for my family every single time I want to test
something. Also, it doesn't actually need a fiber link to bring it up,
as the GPON SFP is really an SoC running lantiq openwrt, pretending to
be a proper SFP.

Thanks,
Marc

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Andrew Lunn]

On Sat, Mar 02, 2019 at 09:17:56PM -0500, Marc Micalizzi wrote:
> On Sat, Mar 2, 2019 at 3:26 PM Andrew Lunn <andrew@lunn.ch> wrote:
> > I've received reports that some GPON SFPs don't like block reads of
> > their i2c bus. You have to do 16bit word reads.
> >
> > There was a patch a while ago to deal with i2c bus drivers which could
> > also not handle big block transfers:
> >
> > https://www.spinics.net/lists/netdev/msg546524.html
> 
> 
> I looked into that patch, and after doing a bit of investigation in to
> where it's actually failing, I'm not sure it would make any
> difference. I added a debug message in i2c_mv64xxx.c
> mv64xxx_i2c_fsm(), which is where my -6 ENXIO is originating from,
> immediately after the fallthrough for
> MV64XXX_I2C_STATUS_MAST_WR_ADDR_NO_ACK (on line 314), and that
> provided me with:
> ```i2c i2c-1: No device at other end-- state: 0x4, status: 0x20, addr:
> 0x50, flags: 0x0```

This fits with your comment about it taking a bit of time to boot
before it starts answering on the i2c bus.

You might want to play with the value of T_PROBE_INIT in
drivers/net/phy/sfp.c.

The driver will keep trying to talk to the SFP, but it waits 300ms
before making its first try, and then retries every 100ms. Maybe a
longer first wait will help.

At the moment, you need to play around and see if you can make it
reliable. Once you know what to do to make it reliable, we can figure
out a clean way to implement it.

> I am going to see if I can get my hands on a second GPON SFP of the
> same make and model somehow, so at the very least I don't have to
> bring my internet down for my family every single time I want to test
> something.

Yes, buy one to use and a second one to hack on :-)

     Andrew

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Russell King - ARM Linux admin]

On Sun, Mar 03, 2019 at 03:48:33AM +0100, Andrew Lunn wrote:
> On Sat, Mar 02, 2019 at 09:17:56PM -0500, Marc Micalizzi wrote:
> > On Sat, Mar 2, 2019 at 3:26 PM Andrew Lunn <andrew@lunn.ch> wrote:
> > > I've received reports that some GPON SFPs don't like block reads of
> > > their i2c bus. You have to do 16bit word reads.
> > >
> > > There was a patch a while ago to deal with i2c bus drivers which could
> > > also not handle big block transfers:
> > >
> > > https://www.spinics.net/lists/netdev/msg546524.html
> > 
> > 
> > I looked into that patch, and after doing a bit of investigation in to
> > where it's actually failing, I'm not sure it would make any
> > difference. I added a debug message in i2c_mv64xxx.c
> > mv64xxx_i2c_fsm(), which is where my -6 ENXIO is originating from,
> > immediately after the fallthrough for
> > MV64XXX_I2C_STATUS_MAST_WR_ADDR_NO_ACK (on line 314), and that
> > provided me with:
> > ```i2c i2c-1: No device at other end-- state: 0x4, status: 0x20, addr:
> > 0x50, flags: 0x0```
> 
> This fits with your comment about it taking a bit of time to boot
> before it starts answering on the i2c bus.

The original report is that it _never_ responds on the I2C bus when
plugged into the Macchiatobin, but it does when plugged into other
systems.

Thinking a little more about this, this morning, I've remembered that
there's the pull-up resistor issue on the Macchiatobin, which are way
too strong.  (IMHO, even the reworked boards are out of spec for SFP
modules.)

There are two very small resistor packs (8 pin black devices with
three digits on the top) labelled RN3001 and RN3004 near the reset
button.  They're directly next to the PCA9548 I2C multiplexer U44.
The three digits will be either with 222 or 112 - please can you
check which they are?

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Russell King - ARM Linux admin]

On Sun, Mar 03, 2019 at 10:01:28AM +0000, Russell King - ARM Linux admin wrote:
> The original report is that it _never_ responds on the I2C bus when
> plugged into the Macchiatobin, but it does when plugged into other
> systems.
> 
> Thinking a little more about this, this morning, I've remembered that
> there's the pull-up resistor issue on the Macchiatobin, which are way
> too strong.  (IMHO, even the reworked boards are out of spec for SFP
> modules.)
> 
> There are two very small resistor packs (8 pin black devices with
> three digits on the top) labelled RN3001 and RN3004 near the reset
> button.  They're directly next to the PCA9548 I2C multiplexer U44.
> The three digits will be either with 222 or 112 - please can you
> check which they are?

The following patch should avoid endlessly trying to probe the module,
eventually dropping into error state when all retries are exhausted.

I have one module where there the I2S lines are not wired (it's a
Metanoia VDSL module) but it's also unresponsive over Serdes, and the
information for it is not available.  It's been a while since I tested
with that module, but I have replaced the last "SFP_MOD_ERROR" below
with "SFP_MOD_PRESENT" so we believe that the module is present.

8<====
From: Russell King <rmk+kernel@armlinux.org.uk>
Subject: net: sfp: avoid endlessly retrying SFP module probe

Rather than endlessly retrying to probe a SFP module, place a limit of
ten tries (each attempted at 100ms intervals) before deciding that the
module is unresponsive, and dropping into error state. This leads to a
maximum of 1s after insertion for the EEPROM to become readable.

There have been reports that endlessly retrying eventually leads to RCU
lockups - although given that we run these in a workqueue and relinquish
the CPU between attempts (which are triggered by a timer) it's hard to
see why the RCU lockups are happening.

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
---
 drivers/net/phy/sfp.c |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/net/phy/sfp.c b/drivers/net/phy/sfp.c
index 024578a81112..9a3be348d791 100644
--- a/drivers/net/phy/sfp.c
+++ b/drivers/net/phy/sfp.c
@@ -1745,6 +1745,7 @@ static void sfp_sm_event(struct sfp *sfp, unsigned int event)
 		if (event == SFP_E_INSERT && sfp->attached) {
 			sfp_module_tx_disable(sfp);
 			sfp_sm_ins_next(sfp, SFP_MOD_PROBE, T_PROBE_INIT);
+			sfp->sm_retries = 10;
 		}
 		break;
 
@@ -1760,8 +1761,10 @@ static void sfp_sm_event(struct sfp *sfp, unsigned int event)
 				sfp_sm_ins_next(sfp, SFP_MOD_HPOWER, val);
 			else if (val != -EAGAIN)
 				sfp_sm_ins_next(sfp, SFP_MOD_ERROR, 0);
-			else
+			else if (--sfp->sm_retries)
 				sfp_sm_set_timer(sfp, T_PROBE_RETRY);
+			else
+				sfp_sm_ins_next(sfp, SFP_MOD_ERROR, 0);
 		}
 		break;
 

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Marc Micalizzi]

On Sun, Mar 3, 2019 at 5:31 AM Russell King - ARM Linux admin
<linux@armlinux.org.uk> wrote:
>
> On Sun, Mar 03, 2019 at 10:01:28AM +0000, Russell King - ARM Linux admin wrote:
> > The original report is that it _never_ responds on the I2C bus when
> > plugged into the Macchiatobin, but it does when plugged into other
> > systems.

Yes, it never does respond. Just in case -- I'm not sure if there
would be any change in power state between retries -- I did increase
T_PROBE_INIT to 3000ms and T_PROBE_RETRY to 1000, but it didn't make
any difference.

> > Thinking a little more about this, this morning, I've remembered that
> > there's the pull-up resistor issue on the Macchiatobin, which are way
> > too strong.  (IMHO, even the reworked boards are out of spec for SFP
> > modules.)
> >
> > There are two very small resistor packs (8 pin black devices with
> > three digits on the top) labelled RN3001 and RN3004 near the reset
> > button.  They're directly next to the PCA9548 I2C multiplexer U44.
> > The three digits will be either with 222 or 112 - please can you
> > check which they are?

After several attempts to take a picture with flash (I have the
macchiatobin in a mini-itx case in a wallmount rack), I was able to
see that both have 222 printed on them.

> The following patch should avoid endlessly trying to probe the module,
> eventually dropping into error state when all retries are exhausted.

That patch does resolve the flood of failed to read EEPROM messages,
which did allow me to also see that it does see module removed when I
pull the SFP at least. Still no EEPROM read though.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Marc Micalizzi]

> Yes, it never does respond. Just in case -- I'm not sure if there
> would be any change in power state between retries -- I did increase
> T_PROBE_INIT to 3000ms and T_PROBE_RETRY to 1000, but it didn't make
> any difference.

So I did get 2 more of the same SFP modules, and plugged one into the
macchiatobin, and it looks like it actually does come up eventually.
On plugging it in I still got:

Mar  6 19:31:24 mbin kernel: [290926.751494] i2c i2c-1: No device at
other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
Mar  6 19:31:24 mbin kernel: [290926.751509] sfp sfp-eth3: failed to
read EEPROM: -6
Mar  6 19:31:24 mbin kernel: [290926.855277] i2c i2c-1: No device at
other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
Mar  6 19:31:24 mbin kernel: [290926.855291] sfp sfp-eth3: failed to
read EEPROM: -6
Mar  6 19:31:24 mbin kernel: [290926.959276] i2c i2c-1: No device at
other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
Mar  6 19:31:24 mbin kernel: [290926.959288] sfp sfp-eth3: failed to
read EEPROM: -6
Mar  6 19:31:24 mbin kernel: [290927.063271] i2c i2c-1: No device at
other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
Mar  6 19:31:24 mbin kernel: [290927.063283] sfp sfp-eth3: failed to
read EEPROM: -6
Mar  6 19:31:24 mbin kernel: [290927.167272] i2c i2c-1: No device at
other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
Mar  6 19:31:24 mbin kernel: [290927.167283] sfp sfp-eth3: failed to
read EEPROM: -6
Mar  6 19:31:24 mbin kernel: [290927.271269] i2c i2c-1: No device at
other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
Mar  6 19:31:24 mbin kernel: [290927.271280] sfp sfp-eth3: failed to
read EEPROM: -6
Mar  6 19:31:25 mbin kernel: [290927.375268] i2c i2c-1: No device at
other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
Mar  6 19:31:25 mbin kernel: [290927.375279] sfp sfp-eth3: failed to
read EEPROM: -6
Mar  6 19:31:25 mbin kernel: [290927.479268] i2c i2c-1: No device at
other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
Mar  6 19:31:25 mbin kernel: [290927.479279] sfp sfp-eth3: failed to
read EEPROM: -6
Mar  6 19:31:25 mbin kernel: [290927.583268] i2c i2c-1: No device at
other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
Mar  6 19:31:25 mbin kernel: [290927.583279] sfp sfp-eth3: failed to
read EEPROM: -6
Mar  6 19:31:25 mbin kernel: [290927.687278] i2c i2c-1: No device at
other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
Mar  6 19:31:25 mbin kernel: [290927.687289] sfp sfp-eth3: failed to
read EEPROM: -6

But then just now, I tried ethtool -m eth3, and it returned me the
EEPROM information correctly. After a reboot (without hard power off),
it was also seen right away. I imagine after a hard poweroff it may
take longer again.

Revisiting some logs posted by someone else from the boot logs from
the Huawei SFP (i think via serial), it appears that the soft EEPROM
takes about 27 seconds to become available, after power-up, which I
had never even left the module plugged for that long, as it was
interrupting internet for my family.
[   21.388000] GPON SFP I2C Slave Driver, Version 2.2.1 (c) Copyright 2015, LanG
[   21.404000] [sfp_i2c] vpe code <sfp_i2c_vpe.bin> with size <4108 bytes> load!
[   21.412000] VPE loader: VPE1 running successfully
[   21.496000] FALC(tm) ON Optic Driver, version 7.5.1.0 (c) Copyright 2015, LaG
[   21.896000] FALC(tm) ON Base Driver, Version 7.5.1.0 (c) Copyright 2017, Int5
[   21.932000] FALC(tm) ON Ethernet Driver, Version 7.5.1.0 (c) Copyright 2017,5
MIPS: set unaligned_action to 'SHOW'
[   27.880000] i2c /dev entries driver
[   27.904000] Custom GPIO-based I2C driver version 0.1.1
[   27.916000] i2c-gpio i2c-gpio.0: using pins 37 (SDA) and 38 (SCL)


So this leads to the next problem now: ethtool only show and allows
1000BASE-X with this SFP.
The EEPROM does say it is:
        Transceiver type                          : Ethernet: 1000BASE-LX
        Encoding                                  : 0x03 (NRZ)
        BR, Nominal                               : 1200MBd

However, the module does link at 2500BASE-X as well.
`ethtool -s eth3 speed 2500 duplex full autoneg off` doesn't throw an
error, but doesn't change anything either.
 I see `[ 1095.238452] mvpp2x f4000000.ppv22 eth3: reconfig: pm 4->4
cm 0->c f a->a` in dmesg.
Any suggestions?




Also, on an unrelated sidenote, I am, and have been, also seeing a
flood of, but not all the time, after a reboot or, disconnect and
reconnect of the fiber, it stops showing up for a while, sometimes.
Bad SFP maybe? It doesn't interrupt my ssh sessions at least...
Mar  7 12:47:42 mbin kernel: [353105.209698] mvpp2x f2000000.ppv22
eth0: Link is Down
Mar  7 12:47:42 mbin kernel: [353105.209712] mvpp2x f2000000.ppv22
eth0: Link is Up - 10Gbps/Full - flow control rx
Mar  7 12:47:45 mbin kernel: [353107.393313] br0: port 1(eth0) entered
disabled state
Mar  7 12:47:45 mbin kernel: [353107.394241] mvpp2x f2000000.ppv22
eth0: Link is Down
Mar  7 12:47:45 mbin kernel: [353107.394257] mvpp2x f2000000.ppv22
eth0: Link is Up - 10Gbps/Full - flow control rx
Mar  7 12:47:45 mbin kernel: [353107.394271] br0: port 1(eth0) entered
blocking state
Mar  7 12:47:45 mbin kernel: [353107.394274] br0: port 1(eth0) entered
forwarding state
Mar  7 12:47:50 mbin kernel: [353112.852100] mvpp2x f2000000.ppv22
eth0: Link is Down
Mar  7 12:47:50 mbin kernel: [353112.852115] mvpp2x f2000000.ppv22
eth0: Link is Up - 10Gbps/Full - flow control rx
Mar  7 12:48:59 mbin kernel: [353181.748384] mvpp2x f2000000.ppv22
eth0: Link is Down
Mar  7 12:48:59 mbin kernel: [353181.748399] mvpp2x f2000000.ppv22
eth0: Link is Up - 10Gbps/Full - flow control rx
Mar  7 12:49:19 mbin kernel: [353201.953776] mvpp2x f2000000.ppv22
eth0: MVPP2_RXD_ERR_SUMMARY
Mar  7 12:49:19 mbin kernel: [353201.953787] mvpp2x f2000000.ppv22
eth0: bad rx status 14008514 (crc error), size=291

that I'm not sure about either.
mbin # ethtool eth0
Settings for eth0:
        Supported ports: [ FIBRE ]
        Supported link modes:   10000baseSR/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Advertised link modes:  10000baseSR/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Speed: 10000Mb/s
        Duplex: Full
        Port: FIBRE
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: on
        Link detected: yes
mbin # ethtool -m eth0
        Identifier                                : 0x03 (SFP)
        Extended identifier                       : 0x04 (GBIC/SFP
defined by 2-wire interface ID)
        Connector                                 : 0x07 (LC)
        Transceiver codes                         : 0x10 0x00 0x00
0x00 0x00 0x00 0x00 0x00
        Transceiver type                          : 10G Ethernet: 10G Base-SR
        Encoding                                  : 0x06 (64B/66B)
        BR, Nominal                               : 10300MBd
        Rate identifier                           : 0x00 (unspecified)
        Length (SMF,km)                           : 0km
        Length (SMF)                              : 0m
        Length (50um)                             : 80m
        Length (62.5um)                           : 30m
        Length (Copper)                           : 0m
        Length (OM3)                              : 300m
        Laser wavelength                          : 850nm
        Vendor name                               : FINISAR CORP.
        Vendor OUI                                : 00:90:65
        Vendor PN                                 : FTLX8571D3BCL
        Vendor rev                                : A
        Option values                             : 0x00 0x1a
        Option                                    : RX_LOS implemented
        Option                                    : TX_FAULT implemented
        Option                                    : TX_DISABLE implemented
        BR margin, max                            : 0%
        BR margin, min                            : 0%
        Vendor SN                                 : ANQ064Q
        Date code                                 : 121210
        Optical diagnostics support               : Yes
        Laser bias current                        : 8.108 mA
        Laser output power                        : 0.6311 mW / -2.00 dBm
        Receiver signal average optical power     : 0.5401 mW / -2.68 dBm
        Module temperature                        : 39.85 degrees C /
103.73 degrees F
        Module voltage                            : 3.2966 V
        Alarm/warning flags implemented           : Yes
        Laser bias current high alarm             : Off
        Laser bias current low alarm              : Off
        Laser bias current high warning           : Off
        Laser bias current low warning            : Off
        Laser output power high alarm             : Off
        Laser output power low alarm              : Off
        Laser output power high warning           : Off
        Laser output power low warning            : Off
        Module temperature high alarm             : Off
        Module temperature low alarm              : Off
        Module temperature high warning           : Off
        Module temperature low warning            : Off
        Module voltage high alarm                 : Off
        Module voltage low alarm                  : Off
        Module voltage high warning               : Off
        Module voltage low warning                : Off
        Laser rx power high alarm                 : Off
        Laser rx power low alarm                  : Off
        Laser rx power high warning               : Off
        Laser rx power low warning                : Off
        Laser bias current high alarm threshold   : 13.200 mA
        Laser bias current low alarm threshold    : 4.000 mA
        Laser bias current high warning threshold : 12.600 mA
        Laser bias current low warning threshold  : 5.000 mA
        Laser output power high alarm threshold   : 1.0000 mW / 0.00 dBm
        Laser output power low alarm threshold    : 0.2512 mW / -6.00 dBm
        Laser output power high warning threshold : 0.7943 mW / -1.00 dBm
        Laser output power low warning threshold  : 0.3162 mW / -5.00 dBm
        Module temperature high alarm threshold   : 78.00 degrees C /
172.40 degrees F
        Module temperature low alarm threshold    : -13.00 degrees C /
8.60 degrees F
        Module temperature high warning threshold : 73.00 degrees C /
163.40 degrees F
        Module temperature low warning threshold  : -8.00 degrees C /
17.60 degrees F
        Module voltage high alarm threshold       : 3.7000 V
        Module voltage low alarm threshold        : 2.9000 V
        Module voltage high warning threshold     : 3.6000 V
        Module voltage low warning threshold      : 3.0000 V
        Laser rx power high alarm threshold       : 1.0000 mW / 0.00 dBm
        Laser rx power low alarm threshold        : 0.0100 mW / -20.00 dBm
        Laser rx power high warning threshold     : 0.7943 mW / -1.00 dBm
        Laser rx power low warning threshold      : 0.0158 mW / -18.01 dBm

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Russell King - ARM Linux admin]

On Thu, Mar 07, 2019 at 01:42:42PM -0500, Marc Micalizzi wrote:
> > Yes, it never does respond. Just in case -- I'm not sure if there
> > would be any change in power state between retries -- I did increase
> > T_PROBE_INIT to 3000ms and T_PROBE_RETRY to 1000, but it didn't make
> > any difference.
> 
> So I did get 2 more of the same SFP modules, and plugged one into the
> macchiatobin, and it looks like it actually does come up eventually.
> On plugging it in I still got:
> 
> Mar  6 19:31:24 mbin kernel: [290926.751494] i2c i2c-1: No device at
> other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
> Mar  6 19:31:24 mbin kernel: [290926.751509] sfp sfp-eth3: failed to
> read EEPROM: -6
> Mar  6 19:31:24 mbin kernel: [290926.855277] i2c i2c-1: No device at
> other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
> Mar  6 19:31:24 mbin kernel: [290926.855291] sfp sfp-eth3: failed to
> read EEPROM: -6
> Mar  6 19:31:24 mbin kernel: [290926.959276] i2c i2c-1: No device at
> other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
> Mar  6 19:31:24 mbin kernel: [290926.959288] sfp sfp-eth3: failed to
> read EEPROM: -6
> Mar  6 19:31:24 mbin kernel: [290927.063271] i2c i2c-1: No device at
> other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
> Mar  6 19:31:24 mbin kernel: [290927.063283] sfp sfp-eth3: failed to
> read EEPROM: -6
> Mar  6 19:31:24 mbin kernel: [290927.167272] i2c i2c-1: No device at
> other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
> Mar  6 19:31:24 mbin kernel: [290927.167283] sfp sfp-eth3: failed to
> read EEPROM: -6
> Mar  6 19:31:24 mbin kernel: [290927.271269] i2c i2c-1: No device at
> other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
> Mar  6 19:31:24 mbin kernel: [290927.271280] sfp sfp-eth3: failed to
> read EEPROM: -6
> Mar  6 19:31:25 mbin kernel: [290927.375268] i2c i2c-1: No device at
> other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
> Mar  6 19:31:25 mbin kernel: [290927.375279] sfp sfp-eth3: failed to
> read EEPROM: -6
> Mar  6 19:31:25 mbin kernel: [290927.479268] i2c i2c-1: No device at
> other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
> Mar  6 19:31:25 mbin kernel: [290927.479279] sfp sfp-eth3: failed to
> read EEPROM: -6
> Mar  6 19:31:25 mbin kernel: [290927.583268] i2c i2c-1: No device at
> other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
> Mar  6 19:31:25 mbin kernel: [290927.583279] sfp sfp-eth3: failed to
> read EEPROM: -6
> Mar  6 19:31:25 mbin kernel: [290927.687278] i2c i2c-1: No device at
> other end-- state: 0x4, status: 0x20, addr: 0x50, flags: 0x0
> Mar  6 19:31:25 mbin kernel: [290927.687289] sfp sfp-eth3: failed to
> read EEPROM: -6
> 
> But then just now, I tried ethtool -m eth3, and it returned me the
> EEPROM information correctly. After a reboot (without hard power off),
> it was also seen right away. I imagine after a hard poweroff it may
> take longer again.
> 
> Revisiting some logs posted by someone else from the boot logs from
> the Huawei SFP (i think via serial), it appears that the soft EEPROM
> takes about 27 seconds to become available, after power-up, which I
> had never even left the module plugged for that long, as it was
> interrupting internet for my family.
> [   21.388000] GPON SFP I2C Slave Driver, Version 2.2.1 (c) Copyright 2015, LanG
> [   21.404000] [sfp_i2c] vpe code <sfp_i2c_vpe.bin> with size <4108 bytes> load!
> [   21.412000] VPE loader: VPE1 running successfully
> [   21.496000] FALC(tm) ON Optic Driver, version 7.5.1.0 (c) Copyright 2015, LaG
> [   21.896000] FALC(tm) ON Base Driver, Version 7.5.1.0 (c) Copyright 2017, Int5
> [   21.932000] FALC(tm) ON Ethernet Driver, Version 7.5.1.0 (c) Copyright 2017,5
> MIPS: set unaligned_action to 'SHOW'
> [   27.880000] i2c /dev entries driver
> [   27.904000] Custom GPIO-based I2C driver version 0.1.1
> [   27.916000] i2c-gpio i2c-gpio.0: using pins 37 (SDA) and 38 (SCL)
> 
> 
> So this leads to the next problem now: ethtool only show and allows
> 1000BASE-X with this SFP.
> The EEPROM does say it is:
>         Transceiver type                          : Ethernet: 1000BASE-LX
>         Encoding                                  : 0x03 (NRZ)
>         BR, Nominal                               : 1200MBd

Well, there's a couple of things wrong there to the SFP EEPROM
standard (SFF8472):

1. 1000base-X is not "NRZ", it's 8b/10b encoding, so the value in the
   encoding field is incorrect.
2. 1G speed is 1.25MBaud, which is supposed to be rounded up, so this
   should be 1300MBd (another illustration of the sloppy nature of
   these EEPROMs.)

There should also be two more fields, which may be:

        BR margin, max                            : 0%
	BR margin, min                            : 0%

If they are, there is no generic way to identify that this module
supports 2.5G, and the only possible way we could do so is to explicitly
identify the module by the vendor and part number.

> However, the module does link at 2500BASE-X as well.
> `ethtool -s eth3 speed 2500 duplex full autoneg off` doesn't throw an
> error, but doesn't change anything either.
>  I see `[ 1095.238452] mvpp2x f4000000.ppv22 eth3: reconfig: pm 4->4
> cm 0->c f a->a` in dmesg.
> Any suggestions?

Since we decide the module doesn't support 2.5G, we won't switch to 2.5G
speeds even with an explicit request.  That explicit request should have
been refused and errored out.

> Also, on an unrelated sidenote, I am, and have been, also seeing a
> flood of, but not all the time, after a reboot or, disconnect and
> reconnect of the fiber, it stops showing up for a while, sometimes.
> Bad SFP maybe? It doesn't interrupt my ssh sessions at least...
> Mar  7 12:47:42 mbin kernel: [353105.209698] mvpp2x f2000000.ppv22
> eth0: Link is Down
> Mar  7 12:47:42 mbin kernel: [353105.209712] mvpp2x f2000000.ppv22
> eth0: Link is Up - 10Gbps/Full - flow control rx

The link status is a culmination of several sources, and knowing which
source caused it isn't possible without having phylink debug enabled.
Can you enable phylink debug by adding "#define DEBUG" at the top of
drivers/net/phy/phylink.c and rebuilding please?

>         Laser output power                        : 0.6311 mW / -2.00 dBm
>         Receiver signal average optical power     : 0.5401 mW / -2.68 dBm

Looks like the module is getting good signal from the other end.  Does
the other end also report these sporadic link failures too?

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Marc Micalizzi]

On Thu, Mar 7, 2019 at 2:01 PM Russell King - ARM Linux admin
<linux@armlinux.org.uk> wrote:
> > So this leads to the next problem now: ethtool only show and allows
> > 1000BASE-X with this SFP.
> > The EEPROM does say it is:
> >         Transceiver type                          : Ethernet: 1000BASE-LX
> >         Encoding                                  : 0x03 (NRZ)
> >         BR, Nominal                               : 1200MBd
>
> Well, there's a couple of things wrong there to the SFP EEPROM
> standard (SFF8472):
>
> 1. 1000base-X is not "NRZ", it's 8b/10b encoding, so the value in the
>    encoding field is incorrect.
> 2. 1G speed is 1.25MBaud, which is supposed to be rounded up, so this
>    should be 1300MBd (another illustration of the sloppy nature of
>    these EEPROMs.)
>
> There should also be two more fields, which may be:
>
>         BR margin, max                            : 0%
>         BR margin, min                            : 0%
>
> If they are, there is no generic way to identify that this module
> supports 2.5G, and the only possible way we could do so is to explicitly
> identify the module by the vendor and part number.

They are both 0%, yes. So it would need a explicit capabilities set.
Is this already done anywhere for any modules that I could append on
to? Or nothing else that has come up so far conforms so poorly to the
standards? :)

> > Also, on an unrelated sidenote, I am, and have been, also seeing a
> > flood of, but not all the time, after a reboot or, disconnect and
> > reconnect of the fiber, it stops showing up for a while, sometimes.
> > Bad SFP maybe? It doesn't interrupt my ssh sessions at least...
> > Mar  7 12:47:42 mbin kernel: [353105.209698] mvpp2x f2000000.ppv22
> > eth0: Link is Down
> > Mar  7 12:47:42 mbin kernel: [353105.209712] mvpp2x f2000000.ppv22
> > eth0: Link is Up - 10Gbps/Full - flow control rx
>
> The link status is a culmination of several sources, and knowing which
> source caused it isn't possible without having phylink debug enabled.
> Can you enable phylink debug by adding "#define DEBUG" at the top of
> drivers/net/phy/phylink.c and rebuilding please?

With phylink debug I get:
[   20.871636] mvpp2x f2000000.ppv22 eth0: mac link up
[   29.973143] mvpp2x f2000000.ppv22 eth0: mac link up
[   45.494580] mvpp2x f2000000.ppv22 eth0: mac link up
[   45.494595] mvpp2x f2000000.ppv22 eth0: mac link up
[   74.038370] mvpp2x f2000000.ppv22 eth0: mac link up
[   74.038385] mvpp2x f2000000.ppv22 eth0: mac link up
[   95.683112] random: crng init done
[   97.177052] mvpp2x f2000000.ppv22 eth0: mac link up
[  107.968144] mvpp2x f2000000.ppv22 eth0: mac link down
[  107.968163] mvpp2x f2000000.ppv22 eth0: mac link up
[  107.968217] br0: port 1(eth0) entered disabled state
[  107.969434] mvpp2x f2000000.ppv22 eth0: Link is Down
[  107.969452] mvpp2x f2000000.ppv22 eth0: Link is Up - 10Gbps/Full -
flow control rx
[  107.969470] br0: port 1(eth0) entered blocking state
[  107.969474] br0: port 1(eth0) entered forwarding state
INIT: Id "f0" respawning too fast: disabled for 5 minutes
[  153.962990] mvpp2x f2000000.ppv22 eth0: mac link down
[  153.963009] mvpp2x f2000000.ppv22 eth0: mac link up
[  153.963053] br0: port 1(eth0) entered disabled state
[  153.964190] mvpp2x f2000000.ppv22 eth0: Link is Down
[  153.964208] mvpp2x f2000000.ppv22 eth0: Link is Up - 10Gbps/Full -
flow control rx
[  153.964223] br0: port 1(eth0) entered blocking state
[  153.964226] br0: port 1(eth0) entered forwarding state
[  166.416711] mvpp2x f2000000.ppv22 eth0: mac link up
[  179.966442] mvpp2x f2000000.ppv22 eth0: mac link up
[  179.966462] mvpp2x f2000000.ppv22 eth0: mac link up
[  185.842717] mvpp2x f2000000.ppv22 eth0: mac link up
[  185.842732] mvpp2x f2000000.ppv22 eth0: mac link up
[  190.853434] mvpp2x f2000000.ppv22 eth0: mac link up
[  190.853449] mvpp2x f2000000.ppv22 eth0: mac link up
[  229.960363] mvpp2x f2000000.ppv22 eth0: mac link up
[  229.960378] mvpp2x f2000000.ppv22 eth0: mac link up
[  241.589448] mvpp2x f2000000.ppv22 eth0: mac link up
[  241.589462] mvpp2x f2000000.ppv22 eth0: mac link up
[  278.890967] mvpp2x f2000000.ppv22 eth0: mac link down
[  278.890982] mvpp2x f2000000.ppv22 eth0: mac link up
[  278.891018] br0: port 1(eth0) entered disabled state
[  278.892190] mvpp2x f2000000.ppv22 eth0: Link is Down
[  278.892206] mvpp2x f2000000.ppv22 eth0: Link is Up - 10Gbps/Full -
flow control rx
[  278.892221] br0: port 1(eth0) entered blocking state
[  278.892224] br0: port 1(eth0) entered forwarding state
[  288.111348] mvpp2x f2000000.ppv22 eth0: mac link down
[  288.111363] mvpp2x f2000000.ppv22 eth0: mac link up
[  288.111393] br0: port 1(eth0) entered disabled state
[  288.112475] mvpp2x f2000000.ppv22 eth0: Link is Down
[  288.112490] mvpp2x f2000000.ppv22 eth0: Link is Up - 10Gbps/Full -
flow control rx
[  288.112504] br0: port 1(eth0) entered blocking state
[  288.112507] br0: port 1(eth0) entered forwarding state
[  290.825608] mvpp2x f2000000.ppv22 eth0: mac link up
[  290.825622] mvpp2x f2000000.ppv22 eth0: mac link up
[  299.467488] mvpp2x f2000000.ppv22 eth0: mac link up
[  460.806719] mvpp2x f2000000.ppv22 eth0: mac link down
[  460.806726] mvpp2x f2000000.ppv22 eth0: mac link up


> >         Laser output power                        : 0.6311 mW / -2.00 dBm
> >         Receiver signal average optical power     : 0.5401 mW / -2.68 dBm
>
> Looks like the module is getting good signal from the other end.  Does
> the other end also report these sporadic link failures too?

The other end doesn't report any status changes for the switchport for
these link failures, just the macchiatobin.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Russell King - ARM Linux admin]

On Thu, Mar 07, 2019 at 02:40:53PM -0500, Marc Micalizzi wrote:
> On Thu, Mar 7, 2019 at 2:01 PM Russell King - ARM Linux admin
> <linux@armlinux.org.uk> wrote:
> > > So this leads to the next problem now: ethtool only show and allows
> > > 1000BASE-X with this SFP.
> > > The EEPROM does say it is:
> > >         Transceiver type                          : Ethernet: 1000BASE-LX
> > >         Encoding                                  : 0x03 (NRZ)
> > >         BR, Nominal                               : 1200MBd
> >
> > Well, there's a couple of things wrong there to the SFP EEPROM
> > standard (SFF8472):
> >
> > 1. 1000base-X is not "NRZ", it's 8b/10b encoding, so the value in the
> >    encoding field is incorrect.
> > 2. 1G speed is 1.25MBaud, which is supposed to be rounded up, so this
> >    should be 1300MBd (another illustration of the sloppy nature of
> >    these EEPROMs.)
> >
> > There should also be two more fields, which may be:
> >
> >         BR margin, max                            : 0%
> >         BR margin, min                            : 0%
> >
> > If they are, there is no generic way to identify that this module
> > supports 2.5G, and the only possible way we could do so is to explicitly
> > identify the module by the vendor and part number.
> 
> They are both 0%, yes. So it would need a explicit capabilities set.
> Is this already done anywhere for any modules that I could append on
> to? Or nothing else that has come up so far conforms so poorly to the
> standards? :)

So far, we have one manufacturer's modules that fail the EEPROM checksum
validation due to their production line modifying the EEPROM contents
with the part number and serial number _without_ updating the checksum.

We don't have any other work-arounds present, so it's going to need an
addition.  Expect a patch in the next few days.

> With phylink debug I get:
> [   20.871636] mvpp2x f2000000.ppv22 eth0: mac link up
> [   29.973143] mvpp2x f2000000.ppv22 eth0: mac link up
> [   45.494580] mvpp2x f2000000.ppv22 eth0: mac link up
> [   45.494595] mvpp2x f2000000.ppv22 eth0: mac link up
> [   74.038370] mvpp2x f2000000.ppv22 eth0: mac link up
> [   74.038385] mvpp2x f2000000.ppv22 eth0: mac link up
> [   95.683112] random: crng init done
> [   97.177052] mvpp2x f2000000.ppv22 eth0: mac link up
> [  107.968144] mvpp2x f2000000.ppv22 eth0: mac link down
> [  107.968163] mvpp2x f2000000.ppv22 eth0: mac link up
> [  107.968217] br0: port 1(eth0) entered disabled state
> [  107.969434] mvpp2x f2000000.ppv22 eth0: Link is Down
> [  107.969452] mvpp2x f2000000.ppv22 eth0: Link is Up - 10Gbps/Full -
> flow control rx
> [  107.969470] br0: port 1(eth0) entered blocking state
> [  107.969474] br0: port 1(eth0) entered forwarding state
> INIT: Id "f0" respawning too fast: disabled for 5 minutes
> [  153.962990] mvpp2x f2000000.ppv22 eth0: mac link down
> [  153.963009] mvpp2x f2000000.ppv22 eth0: mac link up
> [  153.963053] br0: port 1(eth0) entered disabled state
> [  153.964190] mvpp2x f2000000.ppv22 eth0: Link is Down
> [  153.964208] mvpp2x f2000000.ppv22 eth0: Link is Up - 10Gbps/Full -
> flow control rx
> [  153.964223] br0: port 1(eth0) entered blocking state
> [  153.964226] br0: port 1(eth0) entered forwarding state
> [  166.416711] mvpp2x f2000000.ppv22 eth0: mac link up
> [  179.966442] mvpp2x f2000000.ppv22 eth0: mac link up
> [  179.966462] mvpp2x f2000000.ppv22 eth0: mac link up
> [  185.842717] mvpp2x f2000000.ppv22 eth0: mac link up
> [  185.842732] mvpp2x f2000000.ppv22 eth0: mac link up
> [  190.853434] mvpp2x f2000000.ppv22 eth0: mac link up
> [  190.853449] mvpp2x f2000000.ppv22 eth0: mac link up
> [  229.960363] mvpp2x f2000000.ppv22 eth0: mac link up
> [  229.960378] mvpp2x f2000000.ppv22 eth0: mac link up
> [  241.589448] mvpp2x f2000000.ppv22 eth0: mac link up
> [  241.589462] mvpp2x f2000000.ppv22 eth0: mac link up
> [  278.890967] mvpp2x f2000000.ppv22 eth0: mac link down
> [  278.890982] mvpp2x f2000000.ppv22 eth0: mac link up
> [  278.891018] br0: port 1(eth0) entered disabled state
> [  278.892190] mvpp2x f2000000.ppv22 eth0: Link is Down
> [  278.892206] mvpp2x f2000000.ppv22 eth0: Link is Up - 10Gbps/Full -
> flow control rx
> [  278.892221] br0: port 1(eth0) entered blocking state
> [  278.892224] br0: port 1(eth0) entered forwarding state
> [  288.111348] mvpp2x f2000000.ppv22 eth0: mac link down
> [  288.111363] mvpp2x f2000000.ppv22 eth0: mac link up
> [  288.111393] br0: port 1(eth0) entered disabled state
> [  288.112475] mvpp2x f2000000.ppv22 eth0: Link is Down
> [  288.112490] mvpp2x f2000000.ppv22 eth0: Link is Up - 10Gbps/Full -
> flow control rx
> [  288.112504] br0: port 1(eth0) entered blocking state
> [  288.112507] br0: port 1(eth0) entered forwarding state
> [  290.825608] mvpp2x f2000000.ppv22 eth0: mac link up
> [  290.825622] mvpp2x f2000000.ppv22 eth0: mac link up
> [  299.467488] mvpp2x f2000000.ppv22 eth0: mac link up
> [  460.806719] mvpp2x f2000000.ppv22 eth0: mac link down
> [  460.806726] mvpp2x f2000000.ppv22 eth0: mac link up

Ok, that is basically pointing at the quality/reliability of the
received waveform at the serdes is impaired.  The repeated "mac link up"
messages basically means that the link up bit briefly deasserted before
we could read the deassertion.

> > >         Laser output power                        : 0.6311 mW / -2.00 dBm
> > >         Receiver signal average optical power     : 0.5401 mW / -2.68 dBm
> >
> > Looks like the module is getting good signal from the other end.  Does
> > the other end also report these sporadic link failures too?
> 
> The other end doesn't report any status changes for the switchport for
> these link failures, just the macchiatobin.

The received signal level doesn't suggest dirty optics, which would've
been my first suggestion.  I don't have a second suggestion at the
moment, but that may change.  I'll keep you posted when I send the
patch for working around the speed issue.

Thanks.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Russell King - ARM Linux admin]

On Sat, Mar 02, 2019 at 01:00:28PM -0500, marcmicalizzi@gmail.com wrote:
> From the machine I have that does work with the SFP:
> server # ethtool -m eth0
>         Identifier                                : 0x03 (SFP)
>         Extended identifier                       : 0x04 (GBIC/SFP defined
> by 2-wire interface ID)
>         Connector                                 : 0x01 (SC)
>         Transceiver codes                         : 0x00 0x00 0x00 0x02 0x00
> 0x00 0x00 0x00
>         Transceiver type                          : Ethernet: 1000BASE-LX
>         Encoding                                  : 0x03 (NRZ)
>         BR, Nominal                               : 1200MBd
>         Rate identifier                           : 0x00 (unspecified)
>         Length (SMF,km)                           : 20km
>         Length (SMF)                              : 20000m
>         Length (50um)                             : 0m
>         Length (62.5um)                           : 0m
>         Length (Copper)                           : 0m
>         Length (OM3)                              : 0m
>         Laser wavelength                          : 1310nm
>         Vendor name                               : HUAWEI
>         Vendor OUI                                : 00:00:00
>         Vendor PN                                 : MA5671A
>         Vendor rev                                : 0000
>         Option values                             : 0x00 0x1a
>         Option                                    : RX_LOS implemented
>         Option                                    : TX_FAULT implemented
>         Option                                    : TX_DISABLE implemented
>         BR margin, max                            : 0%
>         BR margin, min                            : 0%
>         Vendor SN                                 : xxxxxxxxxxxxxxxxx
>         Date code                                 : 180105
>         Optical diagnostics support               : Yes
>         Laser bias current                        : 8.058 mA
>         Laser output power                        : 1.4902 mW / 1.73 dBm
>         Receiver signal average optical power     : 0.0104 mW / -19.83 dBm
>         Module temperature                        : 41.66 degrees C / 106.99
> degrees F
>         Module voltage                            : 3.2717 V
>         Alarm/warning flags implemented           : Yes
>         Laser bias current high alarm             : Off
>         Laser bias current low alarm              : Off
>         Laser bias current high warning           : Off
>         Laser bias current low warning            : Off
>         Laser output power high alarm             : Off
>         Laser output power low alarm              : Off
>         Laser output power high warning           : Off
>         Laser output power low warning            : Off
>         Module temperature high alarm             : Off
>         Module temperature low alarm              : Off
>         Module temperature high warning           : Off
>         Module temperature low warning            : Off
>         Module voltage high alarm                 : Off
>         Module voltage low alarm                  : Off
>         Module voltage high warning               : Off
>         Module voltage low warning                : Off
>         Laser rx power high alarm                 : Off
>         Laser rx power low alarm                  : Off
>         Laser rx power high warning               : Off
>         Laser rx power low warning                : Off
>         Laser bias current high alarm threshold   : 90.000 mA
>         Laser bias current low alarm threshold    : 0.000 mA
>         Laser bias current high warning threshold : 70.000 mA
>         Laser bias current low warning threshold  : 0.000 mA
>         Laser output power high alarm threshold   : 3.9810 mW / 6.00 dBm
>         Laser output power low alarm threshold    : 0.8912 mW / -0.50 dBm
>         Laser output power high warning threshold : 3.1622 mW / 5.00 dBm
>         Laser output power low warning threshold  : 1.1220 mW / 0.50 dBm
>         Module temperature high alarm threshold   : 95.00 degrees C / 203.00
> degrees F
>         Module temperature low alarm threshold    : -50.00 degrees C /
> -58.00 degrees F
>         Module temperature high warning threshold : 90.00 degrees C / 194.00
> degrees F
>         Module temperature low warning threshold  : -45.00 degrees C /
> -49.00 degrees F
>         Module voltage high alarm threshold       : 3.6000 V
>         Module voltage low alarm threshold        : 3.0000 V
>         Module voltage high warning threshold     : 3.5000 V
>         Module voltage low warning threshold      : 3.1000 V
>         Laser rx power high alarm threshold       : 0.2511 mW / -6.00 dBm
>         Laser rx power low alarm threshold        : 0.0013 mW / -28.86 dBm
>         Laser rx power high warning threshold     : 0.1995 mW / -7.00 dBm
>         Laser rx power low warning threshold      : 0.0016 mW / -27.96 dBm

As I understand from the bare information I can glean from google, this
module supports operating at both 1000base-X and 2500base-X speeds
despite being capable of receiving at 2.4Gb/s ?

> server # ethtool eth0
> Settings for eth0:
>         Supported ports: [ FIBRE ]
>         Supported link modes:   1000baseT/Full
>                                 2500baseX/Full
>         Supported pause frame use: Symmetric Receive-only
>         Supports auto-negotiation: No
>         Advertised link modes:  2500baseX/Full
>         Advertised pause frame use: No
>         Advertised auto-negotiation: No
>         Speed: 2500Mb/s
>         Duplex: Full
>         Port: FIBRE
>         PHYAD: 1
>         Transceiver: internal
>         Auto-negotiation: off
>         Supports Wake-on: g
>         Wake-on: d
>         Current message level: 0x00000000 (0)
> 
>         Link detected: yes

Which network driver is being used for this?

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Marc Micalizzi]

On Fri, Mar 8, 2019 at 6:09 AM Russell King - ARM Linux admin
<linux@armlinux.org.uk> wrote:

> >         Laser wavelength                          : 1310nm
> >         Vendor name                               : HUAWEI
> >         Vendor OUI                                : 00:00:00
> >         Vendor PN                                 : MA5671A
>
> As I understand from the bare information I can glean from google, this
> module supports operating at both 1000base-X and 2500base-X speeds
> despite being capable of receiving at 2.4Gb/s ?

That's correct. It will link at both 1000base-X and 2500base-X. On the
box it came in from the ISP it say 2.488G(rx)/1.244G(tx) which is the
GPON specification as far as I know. I haven't been able to test the
upper limit of this, as the internet package I have (which is also the
fastest offered in Canada for non-dedicated) is 1.5Gbps down/940Mbps
up, so that obviously won't reach the upper limit. Also, the SFP does
have a shell running on 192.168.1.10, however it is very limited and
locked down, so I haven't been able to run iperf from it to see what
the link actually supports. The author of the patch for the bnx2x
referenced below might be able to do that at some point though, as he
removed the serial flash from the SFP and was able to write to it if I
recall correctly.

> > server # ethtool eth0
> > Settings for eth0:
> >         Supported ports: [ FIBRE ]
> >         Supported link modes:   1000baseT/Full
> >                                 2500baseX/Full
> >         Supported pause frame use: Symmetric Receive-only
> >         Supports auto-negotiation: No
> >         Advertised link modes:  2500baseX/Full
> >         Advertised pause frame use: No
> >         Advertised auto-negotiation: No
> >         Speed: 2500Mb/s
> >         Duplex: Full
> >         Port: FIBRE
> >         PHYAD: 1
> >         Transceiver: internal
> >         Auto-negotiation: off
> >         Supports Wake-on: g
> >         Wake-on: d
> >         Current message level: 0x00000000 (0)
> >
> >         Link detected: yes
>
> Which network driver is being used for this?

I'm using bnx2x with the patch in the first post at
https://www.dslreports.com/forum/r32230041-Internet-Bypassing-the-HH3K-up-to-2-5Gbps-using-a-BCM57810S-NIC
As well, if I were using the ISP's hardware for the modem, which is
also running a broadcom chipset (I believe it's the BCM53134 for the
switch portion) that does link at 2500base-X as well using the module,
however it runs a script that sets a series of registers to switch to
2.5G SGMII

Thanks,
Marc

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Marc Micalizzi]

So for testing I patched sfp-bus.c with the special case for the
module (not sure if the patch would conform to good form for the
kernel or not, but it seems functional)--shows up on ethtool as 2500
now, and seems to work--but I'm not able to fully test it
unfortunately (so I'm not sure if there would need to be additional
changes to have it working or not).

------
diff --git a/drivers/net/phy/sfp-bus.c b/drivers/net/phy/sfp-bus.c
index ad9db65..079ee5c 100644
--- a/drivers/net/phy/sfp-bus.c
+++ b/drivers/net/phy/sfp-bus.c
@@ -233,6 +233,11 @@ void sfp_parse_support(struct sfp_bus *bus, const
struct sfp_eeprom_id *id,
                        phylink_set(modes, 1000baseX_Full);
        }

+       if (!memcmp(id->base.vendor_name, "HUAWEI          ", 16) &&
+           !memcmp(id->base.vendor_pn  , "MA5671A         ", 16)) {
+               phylink_set(modes, 2500baseX_Full);
+       }
+
        bitmap_or(support, support, modes, __ETHTOOL_LINK_MODE_MASK_NBITS);

        phylink_set(support, Autoneg);
------

What I've discovered though, when I went to test it, is while the 2
SFPs that I bought do come up eventually (between 120-300 seconds or
so after plug/cold boot for the eeprom to be readable, followed by mac
link up), my ISP provided SFP, despite being the same model number and
revision, actually does never come up (or at least not over 10 minutes
or so). No EEPROM read and no mac link up. It does, however, still
work in my server with the modified (to support SC modules and link at
2500BaseX) bnx2x, and also with ixgbe (albeit not at 2500BaseX).

I'll see if I can find any useful information off the SFP for figuring
out why one works and the other doesn't, maybe it's a different
software version running on the SFP? But at this point I'm at a bit of
a loss again. Unfortunately, I'd have to have my ISP switch the SLIC
ID to the other SFP to test with it, if they were even willing to, and
then there would be no guarantee that a software update pushed over
OMCI wouldn't break it too.

Thanks,
Marc

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Russell King - ARM Linux admin]

Hi Marc,

Sorry I haven't been back to you yet, I've been struggling with some
other issues.

On Wed, Mar 13, 2019 at 11:45:47AM -0400, Marc Micalizzi wrote:
> So for testing I patched sfp-bus.c with the special case for the
> module (not sure if the patch would conform to good form for the
> kernel or not, but it seems functional)--shows up on ethtool as 2500
> now, and seems to work--but I'm not able to fully test it
> unfortunately (so I'm not sure if there would need to be additional
> changes to have it working or not).
> 
> ------
> diff --git a/drivers/net/phy/sfp-bus.c b/drivers/net/phy/sfp-bus.c
> index ad9db65..079ee5c 100644
> --- a/drivers/net/phy/sfp-bus.c
> +++ b/drivers/net/phy/sfp-bus.c
> @@ -233,6 +233,11 @@ void sfp_parse_support(struct sfp_bus *bus, const
> struct sfp_eeprom_id *id,
>                         phylink_set(modes, 1000baseX_Full);
>         }
> 
> +       if (!memcmp(id->base.vendor_name, "HUAWEI          ", 16) &&
> +           !memcmp(id->base.vendor_pn  , "MA5671A         ", 16)) {
> +               phylink_set(modes, 2500baseX_Full);
> +       }
> +
>         bitmap_or(support, support, modes, __ETHTOOL_LINK_MODE_MASK_NBITS);
> 
>         phylink_set(support, Autoneg);

That is good enough.

> ------
> 
> What I've discovered though, when I went to test it, is while the 2
> SFPs that I bought do come up eventually (between 120-300 seconds or
> so after plug/cold boot for the eeprom to be readable, followed by mac
> link up), my ISP provided SFP, despite being the same model number and
> revision, actually does never come up (or at least not over 10 minutes
> or so). No EEPROM read and no mac link up. It does, however, still
> work in my server with the modified (to support SC modules and link at
> 2500BaseX) bnx2x, and also with ixgbe (albeit not at 2500BaseX).

Presumably when the module is plugged into these other cards, it comes
up faster, and the EEPROM is readable before the 120-300 seconds it
seems to take on the mcbin?  Maybe if we knew why it takes that long
we may have a clue why the other module never comes up.

> I'll see if I can find any useful information off the SFP for figuring
> out why one works and the other doesn't, maybe it's a different
> software version running on the SFP? But at this point I'm at a bit of
> a loss again. Unfortunately, I'd have to have my ISP switch the SLIC
> ID to the other SFP to test with it, if they were even willing to, and
> then there would be no guarantee that a software update pushed over
> OMCI wouldn't break it too.

There are two different initialisation scenarios that the SFP
specifications permit - powering on/hotplugging with TX_DISABLE
asserted (which is what we do) and powering on/hotplugging with
TX_DISABLE negated.

I really don't want to change to the negated style because it means
that an optical modules laser will become active as soon as the module
is plugged in, and if you happen to be looking into the module while
plugging it in (eg, it's at eye-level height) that wouldn't be good.
LED based modules (as opposed to laser modules) aren't a concern of
course.

If you want to try the "other" style of initialisation, towards the
top of sfp_sm_event(), change:

                if (event == SFP_E_INSERT && sfp->attached) {
-                       sfp_module_tx_disable(sfp);
+                       sfp_module_tx_enable(sfp);
                        sfp_sm_ins_next(sfp, SFP_MOD_PROBE, T_PROBE_INIT);

and see whether that makes a difference.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Marc Micalizzi]

On Thu, Mar 14, 2019 at 7:30 AM Russell King - ARM Linux admin
<linux@armlinux.org.uk> wrote:
>
> Hi Marc,
>
> Sorry I haven't been back to you yet, I've been struggling with some
> other issues.

No worries, I know everyone is busy, and I'm just appreciative that I
get any response whatsoever--so thank you for the help you have given
so far, I really appreciate it.

> > What I've discovered though, when I went to test it, is while the 2
> > SFPs that I bought do come up eventually (between 120-300 seconds or
> > so after plug/cold boot for the eeprom to be readable, followed by mac
> > link up), my ISP provided SFP, despite being the same model number and
> > revision, actually does never come up (or at least not over 10 minutes
> > or so). No EEPROM read and no mac link up. It does, however, still
> > work in my server with the modified (to support SC modules and link at
> > 2500BaseX) bnx2x, and also with ixgbe (albeit not at 2500BaseX).
>
> Presumably when the module is plugged into these other cards, it comes
> up faster, and the EEPROM is readable before the 120-300 seconds it
> seems to take on the mcbin?  Maybe if we knew why it takes that long
> we may have a clue why the other module never comes up.

Yes, on bnx2x it come up quicker, from a hot plug it is 59 seconds
(for the original ISP provided SFP), and from cold boot it appears to
be around 17-18s into dmesg.
Attached at the bottom are the kernel logs from bnx2x.debug for the
hotplug of my original isp provided sfp, just in case there's anything
useful there (line numbers might be slightly off, seeing as there are
a few modifications to the bnx2x module).

Are there any other debug defines that would be useful in the kernel
to try and figure out what's going on on the mcbin for the two working
SFPs?

> If you want to try the "other" style of initialisation, towards the
> top of sfp_sm_event(), change:
>
>                 if (event == SFP_E_INSERT && sfp->attached) {
> -                       sfp_module_tx_disable(sfp);
> +                       sfp_module_tx_enable(sfp);
>                         sfp_sm_ins_next(sfp, SFP_MOD_PROBE, T_PROBE_INIT);
>
> and see whether that makes a difference.

So with tx_enable, (Let's call the modules module A, B and C; B and C
being the ones I purchased for testing, and A being the original SFP
provided by my ISP),
On hot plug, Module B took 120s for the eeprom to be readable, Module
C took 481s, and Module A didn't come up
From cold boot (power pulled), Module B took 832s followed by 615s on
the second run (???), Module C took 180s both runs, Module A hadn't
come up after 1500 seconds (family needed the internet, so I couldn't
run it even longer).

Thanks,
Marc

-----------------------------------------------
bnx2x log
-----------------------------------------------
Mar 14 14:42:13 empire kernel: [   72.929107] bnx2x:
[bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0
Mar 14 14:42:14 empire kernel: [   73.953159] bnx2x:
[bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0
Mar 14 14:42:15 empire kernel: [   74.977202] bnx2x:
[bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0
Mar 14 14:42:16 empire kernel: [   76.001218] bnx2x:
[bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0
Mar 14 14:42:17 empire kernel: [   77.025300] bnx2x:
[bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0
Mar 14 14:42:18 empire kernel: [   78.049345] bnx2x:
[bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0
Mar 14 14:42:19 empire kernel: [   79.073392] bnx2x:
[bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0
Mar 14 14:42:20 empire kernel: [   80.097447] bnx2x:
[bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0
++++++++++++I plugged the SFP in here++++++++++++
Mar 14 14:42:21 empire kernel: [   81.073242] bnx2x:
[bnx2x_igu_ack_sb_gen:652(eth0)]write 0x02200000 to IGU addr 0x442000
Mar 14 14:42:21 empire kernel: [   81.073294] bnx2x:
[bnx2x_sp_task:5681(eth0)]sp task invoked
Mar 14 14:42:21 empire kernel: [   81.073298] bnx2x:
[bnx2x_sp_task:5690(eth0)]status 1
Mar 14 14:42:21 empire kernel: [   81.073300] bnx2x:
[bnx2x_sp_task:5691(eth0)]setting interrupt_occurred to 0
Mar 14 14:42:21 empire kernel: [   81.073303] bnx2x:
[bnx2x_attn_int:5221(eth0)]attn_bits 801  attn_ack 0  asserted 801
deasserted 0
Mar 14 14:42:21 empire kernel: [   81.073308] bnx2x:
[bnx2x_attn_int_asserted:4026(eth0)]aeu_mask 17  newly asserted 801
Mar 14 14:42:21 empire kernel: [   81.073310] bnx2x:
[bnx2x_attn_int_asserted:4028(eth0)]new mask 16
Mar 14 14:42:21 empire kernel: [   81.073313] bnx2x:
[bnx2x_attn_int_asserted:4033(eth0)]attn_state 0
Mar 14 14:42:21 empire kernel: [   81.073315] bnx2x:
[bnx2x_attn_int_asserted:4035(eth0)]new state 801
Mar 14 14:42:21 empire kernel: [   81.073317] bnx2x:
[bnx2x_attn_int_asserted:4063(eth0)]GPIO_3_FUNC!
Mar 14 14:42:21 empire kernel: [   81.073320] bnx2x:
[bnx2x_attn_int_asserted:4105(eth0)]about to mask 0x00000801 at IGU
addr 0x442d08
Mar 14 14:42:21 empire kernel: [   81.073323] bnx2x:
[bnx2x_igu_ack_sb_gen:652(eth0)]write 0x01400001 to IGU addr 0x442000
Mar 14 14:42:21 empire kernel: [   81.073329] bnx2x:
[bnx2x_igu_ack_sb_gen:652(eth0)]write 0x02200000 to IGU addr 0x442000
Mar 14 14:42:21 empire kernel: [   81.073337] bnx2x:
[bnx2x_sp_task:5681(eth0)]sp task invoked
Mar 14 14:42:21 empire kernel: [   81.073339] bnx2x:
[bnx2x_sp_task:5690(eth0)]status 1
Mar 14 14:42:21 empire kernel: [   81.073341] bnx2x:
[bnx2x_sp_task:5691(eth0)]setting interrupt_occurred to 0
Mar 14 14:42:21 empire kernel: [   81.073344] bnx2x:
[bnx2x_attn_int:5221(eth0)]attn_bits 800  attn_ack 801  asserted 0
deasserted 1
Mar 14 14:42:21 empire kernel: [   81.073356] bnx2x:
[bnx2x_attn_int_deasserted:5146(eth0)]attn: 00000010 00000000 02180000
00000000 00000000
Mar 14 14:42:21 empire kernel: [   81.073359] bnx2x:
[bnx2x_attn_int_deasserted:5156(eth0)]group[0]: 00008010 fff55fff
0000ffff f00003e0 000000fc
Mar 14 14:42:21 empire kernel: [   81.073364] bnx2x:
[bnx2x_sfp_set_transmitter:7855(eth0)]Setting SFP+ transmitter to 1
Mar 14 14:42:21 empire kernel: [   81.073367] bnx2x:
[bnx2x_sfp_e3_set_transmitter:4492(eth0)]Setting WC TX to 1
Mar 14 14:42:21 empire kernel: [   81.073369] bnx2x:
[bnx2x_set_epio:395(eth0)]Setting EPIO pin 21 to 0
Mar 14 14:42:21 empire kernel: [   81.073374] bnx2x:
[bnx2x_set_sfp_module_fault_led:8593(eth0)]Setting SFP+ module fault
LED to 1
Mar 14 14:42:21 empire kernel: [   81.073377] bnx2x:
[bnx2x_set_e3_module_fault_led:8585(eth0)]Setting Fault LED to 1 using
pin cfg 5
Mar 14 14:42:21 empire kernel: [   81.073382] bnx2x:
[bnx2x_set_gpio:2132(eth0)]Set GPIO 0 (shift 4) -> output high
Mar 14 14:42:21 empire kernel: [   81.073479] bnx2x:
[bnx2x_power_sfp_module:8622(eth0)]Setting SFP+ power to 1
Mar 14 14:42:21 empire kernel: [   81.073490] bnx2x:
[bnx2x_warpcore_power_module:7944(eth0)]Setting SFP+ module power to 1
using pin cfg 16
Mar 14 14:42:21 empire kernel: [   81.073499] bnx2x:
[bnx2x_set_epio:395(eth0)]Setting EPIO pin 7 to 0
Mar 14 14:42:21 empire kernel: [   81.073512] bnx2x:
[bnx2x_set_gpio_int:2226(eth0)]Clear GPIO INT 2 (shift 2) -> output
low
Mar 14 14:42:21 empire kernel: [   81.073516] bnx2x:
[bnx2x_bsc_module_sel:3080(eth0)]Setting BSC switch
Mar 14 14:42:21 empire kernel: [   81.073521] bnx2x:
[bnx2x_set_epio:395(eth0)]Setting EPIO pin 29 to 1
Mar 14 14:42:21 empire kernel: [   81.084303] bnx2x:
[bnx2x_bsc_read:3129(eth0)]wr 0 byte timed out after 1002 try
Mar 14 14:42:21 empire kernel: [   81.084306] bnx2x:
[bnx2x_bsc_module_sel:3080(eth0)]Setting BSC switch
Mar 14 14:42:21 empire kernel: [   81.084307] bnx2x:
[bnx2x_set_epio:395(eth0)]Setting EPIO pin 29 to 1
Mar 14 14:42:21 empire kernel: [   81.094984] bnx2x:
[bnx2x_bsc_read:3129(eth0)]wr 0 byte timed out after 1002 try
Mar 14 14:42:21 empire kernel: [   81.094986] bnx2x:
[bnx2x_bsc_module_sel:3080(eth0)]Setting BSC switch
++++++++++++repeats 50-100 times++++++++++++
Mar 14 14:42:23 empire kernel: [   83.583680] bnx2x:
[bnx2x_bsc_read:3129(eth0)]wr 0 byte timed out after 1002 try
Mar 14 14:42:23 empire kernel: [   83.593587] bnx2x:
[bnx2x_bsc_module_sel:3080(eth0)]Setting BSC switch
Mar 14 14:42:23 empire kernel: [   83.593611] bnx2x:
[bnx2x_set_epio:395(eth0)]Setting EPIO pin 29 to 1
Mar 14 14:42:23 empire kernel: [   83.604300] bnx2x:
[bnx2x_bsc_read:3129(eth0)]wr 0 byte timed out after 1002 try
Mar 14 14:42:23 empire kernel: [   83.604303] bnx2x:
[bnx2x_bsc_module_sel:3080(eth0)]Setting BSC switch
Mar 14 14:42:23 empire kernel: [   83.604304] bnx2x:
[bnx2x_set_epio:395(eth0)]Setting EPIO pin 29 to 1
Mar 14 14:42:23 empire kernel: [   83.614990] bnx2x:
[bnx2x_bsc_read:3129(eth0)]wr 0 byte timed out after 1002 try
Mar 14 14:42:23 empire kernel: [   83.614992] bnx2x:
[bnx2x_warpcore_power_module:7944(eth0)]Setting SFP+ module power to 0
using pin cfg 16
Mar 14 14:42:23 empire kernel: [   83.614993] bnx2x:
[bnx2x_set_epio:395(eth0)]Setting EPIO pin 7 to 1
Mar 14 14:42:23 empire kernel: [   83.616568] bnx2x:
[bnx2x_warpcore_power_module:7944(eth0)]Setting SFP+ module power to 1
using pin cfg 16
Mar 14 14:42:23 empire kernel: [   83.616571] bnx2x:
[bnx2x_set_epio:395(eth0)]Setting EPIO pin 7 to 0
Mar 14 14:42:23 empire kernel: [   83.616575] bnx2x:
[bnx2x_bsc_module_sel:3080(eth0)]Setting BSC switch
Mar 14 14:42:23 empire kernel: [   83.616576] bnx2x:
[bnx2x_set_epio:395(eth0)]Setting EPIO pin 29 to 1
Mar 14 14:42:23 empire kernel: [   83.627247] bnx2x:
[bnx2x_bsc_read:3129(eth0)]wr 0 byte timed out after 1002 try
Mar 14 14:42:23 empire kernel: [   83.627248] bnx2x:
[bnx2x_handle_module_detect_int:8805(eth0)]SFP+ module is not
initialized
Mar 14 14:42:23 empire kernel: [   83.627251] bnx2x:
[bnx2x_attn_int_deasserted:5181(eth0)]about to mask 0xfffffffe at IGU
addr 0x442d10
Mar 14 14:42:23 empire kernel: [   83.627255] bnx2x:
[bnx2x_attn_int_deasserted:5194(eth0)]aeu_mask 16  newly deasserted 1
Mar 14 14:42:23 empire kernel: [   83.627255] bnx2x:
[bnx2x_attn_int_deasserted:5196(eth0)]new mask 17
Mar 14 14:42:23 empire kernel: [   83.627257] bnx2x:
[bnx2x_attn_int_deasserted:5201(eth0)]attn_state 801
Mar 14 14:42:23 empire kernel: [   83.627258] bnx2x:
[bnx2x_attn_int_deasserted:5203(eth0)]new state 800
Mar 14 14:42:23 empire kernel: [   83.627259] bnx2x:
[bnx2x_igu_ack_sb_gen:652(eth0)]write 0x01400002 to IGU addr 0x442000
Mar 14 14:42:23 empire kernel: [   83.627287] bnx2x:
[bnx2x_igu_ack_sb_gen:652(eth0)]write 0x02200000 to IGU addr 0x442000
Mar 14 14:42:23 empire kernel: [   83.627422] bnx2x:
[bnx2x_analyze_link_error:13741(eth0)]Analyze TX Fault
Mar 14 14:42:23 empire kernel: [   83.627423] bnx2x:
[bnx2x_analyze_link_error:13747(eth0)]Link changed:[0 0]->1
Mar 14 14:42:23 empire kernel: [   83.627424] bnx2x:
[bnx2x_sfp_tx_fault_detection:13895(eth0)]Change TX_Fault LED: ->0
Mar 14 14:42:23 empire kernel: [   83.627426] bnx2x:
[bnx2x_set_e3_module_fault_led:8585(eth0)]Setting Fault LED to 0 using
pin cfg 5
Mar 14 14:42:23 empire kernel: [   83.627429] bnx2x:
[bnx2x_set_gpio:2123(eth0)]Set GPIO 0 (shift 4) -> output low
Mar 14 14:42:23 empire kernel: [   83.627432] bnx2x:
[bnx2x_sp_task:5681(eth0)]sp task invoked
Mar 14 14:42:23 empire kernel: [   83.627434] bnx2x:
[bnx2x_sp_task:5690(eth0)]status 1
Mar 14 14:42:23 empire kernel: [   83.627434] bnx2x:
[bnx2x_sp_task:5691(eth0)]setting interrupt_occurred to 0
Mar 14 14:42:23 empire kernel: [   83.627436] bnx2x:
[bnx2x_attn_int:5221(eth0)]attn_bits 0  attn_ack 801  asserted 0
deasserted 800
Mar 14 14:42:23 empire kernel: [   83.627444] bnx2x:
[bnx2x_attn_int_deasserted:5146(eth0)]attn: 00000000 00000000 02180000
00000000 00000000
Mar 14 14:42:23 empire kernel: [   83.627446] bnx2x:
[bnx2x_attn_int_deasserted:5181(eth0)]about to mask 0xfffff7ff at IGU
addr 0x442d10
Mar 14 14:42:23 empire kernel: [   83.627449] bnx2x:
[bnx2x_attn_int_deasserted:5194(eth0)]aeu_mask 17  newly deasserted
800
Mar 14 14:42:23 empire kernel: [   83.627449] bnx2x:
[bnx2x_attn_int_deasserted:5196(eth0)]new mask 17
Mar 14 14:42:23 empire kernel: [   83.627451] bnx2x:
[bnx2x_attn_int_deasserted:5201(eth0)]attn_state 800
Mar 14 14:42:23 empire kernel: [   83.627452] bnx2x:
[bnx2x_attn_int_deasserted:5203(eth0)]new state 0
Mar 14 14:42:23 empire kernel: [   83.627453] bnx2x:
[bnx2x_igu_ack_sb_gen:652(eth0)]write 0x01400003 to IGU addr 0x442000
Mar 14 14:42:24 empire kernel: [   84.193629] bnx2x:
[bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0
Mar 14 14:42:24 empire kernel: [   84.641795] bnx2x:
[bnx2x_analyze_link_error:13741(eth0)]Analyze TX Fault
Mar 14 14:42:24 empire kernel: [   84.641798] bnx2x:
[bnx2x_analyze_link_error:13747(eth0)]Link changed:[0 0]->1
Mar 14 14:42:24 empire kernel: [   84.641801] bnx2x:
[bnx2x_sfp_tx_fault_detection:13895(eth0)]Change TX_Fault LED: ->0
Mar 14 14:42:24 empire kernel: [   84.641804] bnx2x:
[bnx2x_set_e3_module_fault_led:8585(eth0)]Setting Fault LED to 0 using
pin cfg 5
Mar 14 14:42:24 empire kernel: [   84.641809] bnx2x:
[bnx2x_set_gpio:2123(eth0)]Set GPIO 0 (shift 4) -> output low
Mar 14 14:42:25 empire kernel: [   85.217675] bnx2x:
[bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0
Mar 14 14:42:25 empire kernel: [   85.665873] bnx2x:
[bnx2x_analyze_link_error:13741(eth0)]Analyze TX Fault
Mar 14 14:42:25 empire kernel: [   85.665876] bnx2x:
[bnx2x_analyze_link_error:13747(eth0)]Link changed:[0 0]->1
Mar 14 14:42:25 empire kernel: [   85.665878] bnx2x:
[bnx2x_sfp_tx_fault_detection:13895(eth0)]Change TX_Fault LED: ->0
Mar 14 14:42:25 empire kernel: [   85.665882] bnx2x:
[bnx2x_set_e3_module_fault_led:8585(eth0)]Setting Fault LED to 0 using
pin cfg 5
++++++++++++fault repeats for a while++++++++++++
Mar 14 14:43:09 empire kernel: [  129.251715] bnx2x:
[bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0
Mar 14 14:43:09 empire kernel: [  129.699906] bnx2x:
[bnx2x_analyze_link_error:13741(eth0)]Analyze TX Fault
Mar 14 14:43:09 empire kernel: [  129.699910] bnx2x:
[bnx2x_analyze_link_error:13747(eth0)]Link changed:[0 0]->1
Mar 14 14:43:09 empire kernel: [  129.699912] bnx2x:
[bnx2x_sfp_tx_fault_detection:13895(eth0)]Change TX_Fault LED: ->0
Mar 14 14:43:09 empire kernel: [  129.699915] bnx2x:
[bnx2x_set_e3_module_fault_led:8585(eth0)]Setting Fault LED to 0 using
pin cfg 5
Mar 14 14:43:09 empire kernel: [  129.699920] bnx2x:
[bnx2x_set_gpio:2123(eth0)]Set GPIO 0 (shift 4) -> output low
Mar 14 14:43:10 empire kernel: [  130.275764] bnx2x:
[bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0
Mar 14 14:43:10 empire kernel: [  130.439097] bnx2x:
[bnx2x_igu_ack_sb_gen:652(eth0)]write 0x02200000 to IGU addr 0x442000
Mar 14 14:43:10 empire kernel: [  130.439148] bnx2x:
[bnx2x_sp_task:5681(eth0)]sp task invoked
Mar 14 14:43:10 empire kernel: [  130.439152] bnx2x:
[bnx2x_sp_task:5690(eth0)]status 1
Mar 14 14:43:10 empire kernel: [  130.439153] bnx2x:
[bnx2x_sp_task:5691(eth0)]setting interrupt_occurred to 0
Mar 14 14:43:10 empire kernel: [  130.439157] bnx2x:
[bnx2x_attn_int:5221(eth0)]attn_bits 100  attn_ack 0  asserted 100
deasserted 0
Mar 14 14:43:10 empire kernel: [  130.439162] bnx2x:
[bnx2x_attn_int_asserted:4026(eth0)]aeu_mask 17  newly asserted 100
Mar 14 14:43:10 empire kernel: [  130.439164] bnx2x:
[bnx2x_attn_int_asserted:4028(eth0)]new mask 17
Mar 14 14:43:10 empire kernel: [  130.439166] bnx2x:
[bnx2x_attn_int_asserted:4033(eth0)]attn_state 0
Mar 14 14:43:10 empire kernel: [  130.439168] bnx2x:
[bnx2x_attn_int_asserted:4035(eth0)]new state 100
Mar 14 14:43:10 empire kernel: [  130.439174] bnx2x:
[bnx2x_stats_handle:1397(eth0)]state 0 -> event 3 -> state 0
Mar 14 14:43:10 empire kernel: [  130.439255] bnx2x:
[bnx2x_link_update:6841(eth0)]port 0, XGXS?1, int_status 0x0
Mar 14 14:43:10 empire kernel: [  130.439259] bnx2x:
[bnx2x_link_update:6848(eth0)]int_mask 0x0 MI_INT 0, SERDES_LINK 0
Mar 14 14:43:10 empire kernel: [  130.439262] bnx2x:
[bnx2x_link_update:6852(eth0)] 10G 0, XGXS_LINK f
Mar 14 14:43:10 empire kernel: [  130.439337] bnx2x:
[bnx2x_warpcore_read_status:5735(eth0)]0x81d1 = 0x400
Mar 14 14:43:10 empire kernel: [  130.439411] bnx2x:
[bnx2x_warpcore_read_status:5808(eth0)]lane 2 gp_speed 0x203
Mar 14 14:43:10 empire kernel: [  130.439413] bnx2x:
[bnx2x_get_link_speed_duplex:5548(eth0)]phy link up
Mar 14 14:43:10 empire kernel: [  130.439416] bnx2x:
[bnx2x_get_link_speed_duplex:5624(eth0)] phy_link_up 1 line_speed 2500
Mar 14 14:43:10 empire kernel: [  130.439418] bnx2x:
[bnx2x_warpcore_read_status:5824(eth0)]duplex 1  flow_ctrl 0x300
link_status 0x100013
Mar 14 14:43:10 empire kernel: [  130.439422] bnx2x:
[bnx2x_link_update:6982(eth0)]vars->flow_ctrl = 0x300,
vars->link_status = 0x100013, ext_phy_line_speed = 0
Mar 14 14:43:10 empire kernel: [  130.439424] bnx2x:
[bnx2x_link_int_ack:6167(eth0)]Ack link up interrupt with mask
0x3c0000
Mar 14 14:43:10 empire kernel: [  130.440753] bnx2x:
[bnx2x_umac_enable:1564(eth0)]enabling UMAC
Mar 14 14:43:10 empire kernel: [  130.440857] bnx2x:
[bnx2x_set_led:6316(eth0)]bnx2x_set_led: port 0, mode 2
Mar 14 14:43:10 empire kernel: [  130.440860] bnx2x:
[bnx2x_set_led:6318(eth0)]speed 0x9c4, hw_led_mode 0x1
Mar 14 14:43:10 empire kernel: [  130.462726] bnx2x:
[bnx2x_fw_command:3028(eth0)]wrote command (1000015) to FW MB param
0x00000000
Mar 14 14:43:10 empire kernel: [  130.486802] bnx2x:
[bnx2x_fw_command:3040(eth0)][after 20 ms] read (1100015) seq is (15)
from FW MB
Mar 14 14:43:10 empire kernel: [  130.486805] bnx2x:
[bnx2x_init_dropless_fc:2344(eth0)]dropless_fc is disabled
Mar 14 14:43:10 empire kernel: [  130.486813] bnx2x:
[bnx2x_storm_stats_post:137(eth0)]Sending statistics ramrod 0
Mar 14 14:43:10 empire kernel: [  130.486817] bnx2x:
[bnx2x_dp_stats:101(eth0)]dumping stats:
Mar 14 14:43:10 empire kernel: [  130.486817] fw_stats_req
Mar 14 14:43:10 empire kernel: [  130.486817]     hdr
Mar 14 14:43:10 empire kernel: [  130.486817]         cmd_num 12
Mar 14 14:43:10 empire kernel: [  130.486817]         reserved0 0
Mar 14 14:43:10 empire kernel: [  130.486817]         drv_stats_counter 0
Mar 14 14:43:10 empire kernel: [  130.486817]         reserved1 0
Mar 14 14:43:10 empire kernel: [  130.486817]
stats_counters_addrs 8 43995110
Mar 14 14:43:10 empire kernel: [  130.486820] bnx2x:
[bnx2x_dp_stats:116(eth0)]query[0]
Mar 14 14:43:10 empire kernel: [  130.486820]               kind 1
Mar 14 14:43:10 empire kernel: [  130.486820]               index 0
Mar 14 14:43:10 empire kernel: [  130.486820]               funcID 0
Mar 14 14:43:10 empire kernel: [  130.486820]               reserved 0
Mar 14 14:43:10 empire kernel: [  130.486820]               address 8 43995130
Mar 14 14:43:10 empire kernel: [  130.486824] bnx2x:
[bnx2x_dp_stats:116(eth0)]query[1]
Mar 14 14:43:10 empire kernel: [  130.486824]               kind 2
Mar 14 14:43:10 empire kernel: [  130.486824]               index 0
Mar 14 14:43:10 empire kernel: [  130.486824]               funcID 0
Mar 14 14:43:10 empire kernel: [  130.486824]               reserved 0
Mar 14 14:43:10 empire kernel: [  130.486824]               address 8 43995148
Mar 14 14:43:10 empire kernel: [  130.486827] bnx2x:
[bnx2x_dp_stats:116(eth0)]query[2]
Mar 14 14:43:10 empire kernel: [  130.486827]               kind 4
Mar 14 14:43:10 empire kernel: [  130.486827]               index 0
Mar 14 14:43:10 empire kernel: [  130.486827]               funcID 0
Mar 14 14:43:10 empire kernel: [  130.486827]               reserved 0
Mar 14 14:43:10 empire kernel: [  130.486827]               address 8 43995150
Mar 14 14:43:10 empire kernel: [  130.486831] bnx2x:
[bnx2x_dp_stats:116(eth0)]query[3]
Mar 14 14:43:10 empire kernel: [  130.486831]               kind 0
Mar 14 14:43:10 empire kernel: [  130.486831]               index 2
Mar 14 14:43:10 empire kernel: [  130.486831]               funcID 0
Mar 14 14:43:10 empire kernel: [  130.486831]               reserved 0
Mar 14 14:43:10 empire kernel: [  130.486831]               address 8 43995190
Mar 14 14:43:10 empire kernel: [  130.486834] bnx2x:
[bnx2x_dp_stats:116(eth0)]query[4]
Mar 14 14:43:10 empire kernel: [  130.486834]               kind 0
Mar 14 14:43:10 empire kernel: [  130.486834]               index 3
Mar 14 14:43:10 empire kernel: [  130.486834]               funcID 0
Mar 14 14:43:10 empire kernel: [  130.486834]               reserved 0
Mar 14 14:43:10 empire kernel: [  130.486834]               address 8 43995228
Mar 14 14:43:10 empire kernel: [  130.486837] bnx2x:
[bnx2x_dp_stats:116(eth0)]query[5]
Mar 14 14:43:10 empire kernel: [  130.486837]               kind 0
Mar 14 14:43:10 empire kernel: [  130.486837]               index 4
Mar 14 14:43:10 empire kernel: [  130.486837]               funcID 0
Mar 14 14:43:10 empire kernel: [  130.486837]               reserved 0
Mar 14 14:43:10 empire kernel: [  130.486837]               address 8 439952c0
Mar 14 14:43:10 empire kernel: [  130.486840] bnx2x:
[bnx2x_dp_stats:116(eth0)]query[6]
Mar 14 14:43:10 empire kernel: [  130.486840]               kind 0
Mar 14 14:43:10 empire kernel: [  130.486840]               index 5
Mar 14 14:43:10 empire kernel: [  130.486840]               funcID 0
Mar 14 14:43:10 empire kernel: [  130.486840]               reserved 0
Mar 14 14:43:10 empire kernel: [  130.486840]               address 8 43995358
Mar 14 14:43:10 empire kernel: [  130.486844] bnx2x:
[bnx2x_dp_stats:116(eth0)]query[7]
Mar 14 14:43:10 empire kernel: [  130.486844]               kind 0
Mar 14 14:43:10 empire kernel: [  130.486844]               index 6
Mar 14 14:43:10 empire kernel: [  130.486844]               funcID 0
Mar 14 14:43:10 empire kernel: [  130.486844]               reserved 0
Mar 14 14:43:10 empire kernel: [  130.486844]               address 8 439953f0
Mar 14 14:43:10 empire kernel: [  130.486860] bnx2x:
[bnx2x_dp_stats:116(eth0)]query[8]
Mar 14 14:43:10 empire kernel: [  130.486860]               kind 0
Mar 14 14:43:10 empire kernel: [  130.486860]               index 7
Mar 14 14:43:10 empire kernel: [  130.486860]               funcID 0
Mar 14 14:43:10 empire kernel: [  130.486860]               reserved 0
Mar 14 14:43:10 empire kernel: [  130.486860]               address 8 43995488
Mar 14 14:43:10 empire kernel: [  130.486864] bnx2x:
[bnx2x_dp_stats:116(eth0)]query[9]
Mar 14 14:43:10 empire kernel: [  130.486864]               kind 0
Mar 14 14:43:10 empire kernel: [  130.486864]               index 8
Mar 14 14:43:10 empire kernel: [  130.486864]               funcID 0
Mar 14 14:43:10 empire kernel: [  130.486864]               reserved 0
Mar 14 14:43:10 empire kernel: [  130.486864]               address 8 43995520
Mar 14 14:43:10 empire kernel: [  130.486867] bnx2x:
[bnx2x_dp_stats:116(eth0)]query[10]
Mar 14 14:43:10 empire kernel: [  130.486867]               kind 0
Mar 14 14:43:10 empire kernel: [  130.486867]               index 9
Mar 14 14:43:10 empire kernel: [  130.486867]               funcID 0
Mar 14 14:43:10 empire kernel: [  130.486867]               reserved 0
Mar 14 14:43:10 empire kernel: [  130.486867]               address 8 439955b8
Mar 14 14:43:10 empire kernel: [  130.486871] bnx2x:
[bnx2x_dp_stats:116(eth0)]query[11]
Mar 14 14:43:10 empire kernel: [  130.486871]               kind 0
Mar 14 14:43:10 empire kernel: [  130.486871]               index 136
Mar 14 14:43:10 empire kernel: [  130.486871]               funcID 0
Mar 14 14:43:10 empire kernel: [  130.486871]               reserved 0
Mar 14 14:43:10 empire kernel: [  130.486871]               address 8 43995650
Mar 14 14:43:10 empire kernel: [  130.486876] bnx2x:
[bnx2x_sp_post:3944(eth0)]SPQE[2d] (8:5703d2d0)  (cmd, common?) (6,1)
hw_cid 0  data (8:43995000) type(0x8) left (CQ, EQ) (8,f5)
Mar 14 14:43:10 empire kernel: [  130.486880] bnx2x:
[bnx2x_stats_handle:1397(eth0)]state 0 -> event 1 -> state 1
Mar 14 14:43:10 empire kernel: [  130.486882] bnx2x:
[bnx2x_set_local_cmng:2638(eth0)]single function mode without fairness
Mar 14 14:43:10 empire kernel: [  130.486887] bnx2x:
[bnx2x_read_mf_cfg:2558(eth0)]mf_cfg function enabled
Mar 14 14:43:10 empire kernel: [  130.486900] bnx2x 0000:02:00.0 eth0:
NIC Link is Up, 2500 Mbps full duplex, Flow control: ON - receive &
transmit
Mar 14 14:43:10 empire kernel: [  130.486904] bnx2x:
[bnx2x_attn_int_asserted:4105(eth0)]about to mask 0x00000100 at IGU
addr 0x442d08
Mar 14 14:43:10 empire kernel: [  130.486909] bnx2x:
[bnx2x_igu_ack_sb_gen:652(eth0)]write 0x01400004 to IGU addr 0x442000
Mar 14 14:43:10 empire kernel: [  130.486931] IPv6:
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Russell King - ARM Linux admin]

On Thu, Mar 14, 2019 at 03:08:18PM -0400, Marc Micalizzi wrote:
> Mar 14 14:42:21 empire kernel: [   81.073356] bnx2x:
> [bnx2x_attn_int_deasserted:5146(eth0)]attn: 00000010 00000000 02180000
> 00000000 00000000
> Mar 14 14:42:21 empire kernel: [   81.073359] bnx2x:
> [bnx2x_attn_int_deasserted:5156(eth0)]group[0]: 00008010 fff55fff
> 0000ffff f00003e0 000000fc

This is the start of processing the module insertion

> Mar 14 14:42:21 empire kernel: [   81.073364] bnx2x:
> [bnx2x_sfp_set_transmitter:7855(eth0)]Setting SFP+ transmitter to 1
> Mar 14 14:42:21 empire kernel: [   81.073367] bnx2x:
> [bnx2x_sfp_e3_set_transmitter:4492(eth0)]Setting WC TX to 1
> Mar 14 14:42:21 empire kernel: [   81.073369] bnx2x:
> [bnx2x_set_epio:395(eth0)]Setting EPIO pin 21 to 0
> Mar 14 14:42:21 empire kernel: [   81.073374] bnx2x:
> [bnx2x_set_sfp_module_fault_led:8593(eth0)]Setting SFP+ module fault
> LED to 1
> Mar 14 14:42:21 empire kernel: [   81.073377] bnx2x:
> [bnx2x_set_e3_module_fault_led:8585(eth0)]Setting Fault LED to 1 using
> pin cfg 5
> Mar 14 14:42:21 empire kernel: [   81.073382] bnx2x:
> [bnx2x_set_gpio:2132(eth0)]Set GPIO 0 (shift 4) -> output high
> Mar 14 14:42:21 empire kernel: [   81.073479] bnx2x:
> [bnx2x_power_sfp_module:8622(eth0)]Setting SFP+ power to 1
> Mar 14 14:42:21 empire kernel: [   81.073490] bnx2x:
> [bnx2x_warpcore_power_module:7944(eth0)]Setting SFP+ module power to 1
> using pin cfg 16
> Mar 14 14:42:21 empire kernel: [   81.073499] bnx2x:
> [bnx2x_set_epio:395(eth0)]Setting EPIO pin 7 to 0

Here, we power up the module - no idea what state the modules
TX_DISABLE signal is.

> Mar 14 14:42:21 empire kernel: [   81.073512] bnx2x:
> [bnx2x_set_gpio_int:2226(eth0)]Clear GPIO INT 2 (shift 2) -> output
> low
> Mar 14 14:42:21 empire kernel: [   81.073516] bnx2x:
> [bnx2x_bsc_module_sel:3080(eth0)]Setting BSC switch
> Mar 14 14:42:21 empire kernel: [   81.073521] bnx2x:
> [bnx2x_set_epio:395(eth0)]Setting EPIO pin 29 to 1
> Mar 14 14:42:21 empire kernel: [   81.084303] bnx2x:
> [bnx2x_bsc_read:3129(eth0)]wr 0 byte timed out after 1002 try

We're trying to read the module EEPROM, which fails each time we see
the above message.

> Mar 14 14:42:21 empire kernel: [   81.084306] bnx2x:
> [bnx2x_bsc_module_sel:3080(eth0)]Setting BSC switch
> Mar 14 14:42:21 empire kernel: [   81.084307] bnx2x:
> [bnx2x_set_epio:395(eth0)]Setting EPIO pin 29 to 1
> Mar 14 14:42:21 empire kernel: [   81.094984] bnx2x:
> [bnx2x_bsc_read:3129(eth0)]wr 0 byte timed out after 1002 try
> Mar 14 14:42:21 empire kernel: [   81.094986] bnx2x:
> [bnx2x_bsc_module_sel:3080(eth0)]Setting BSC switch
> ++++++++++++repeats 50-100 times++++++++++++

It'll only try twice, then power down the module and power it back up,
then try one more time, and repeat that whole process 60 times.

...
> [bnx2x_bsc_read:3129(eth0)]wr 0 byte timed out after 1002 try
> Mar 14 14:42:23 empire kernel: [   83.614992] bnx2x:
> [bnx2x_warpcore_power_module:7944(eth0)]Setting SFP+ module power to 0
> using pin cfg 16
> Mar 14 14:42:23 empire kernel: [   83.614993] bnx2x:
> [bnx2x_set_epio:395(eth0)]Setting EPIO pin 7 to 1
> Mar 14 14:42:23 empire kernel: [   83.616568] bnx2x:
> [bnx2x_warpcore_power_module:7944(eth0)]Setting SFP+ module power to 1
> using pin cfg 16
> Mar 14 14:42:23 empire kernel: [   83.616571] bnx2x:
> [bnx2x_set_epio:395(eth0)]Setting EPIO pin 7 to 0

That's one of the module power cycles.

> Mar 14 14:42:23 empire kernel: [   83.616575] bnx2x:
> [bnx2x_bsc_module_sel:3080(eth0)]Setting BSC switch
> Mar 14 14:42:23 empire kernel: [   83.616576] bnx2x:
> [bnx2x_set_epio:395(eth0)]Setting EPIO pin 29 to 1
> Mar 14 14:42:23 empire kernel: [   83.627247] bnx2x:
> [bnx2x_bsc_read:3129(eth0)]wr 0 byte timed out after 1002 try
> Mar 14 14:42:23 empire kernel: [   83.627248] bnx2x:
> [bnx2x_handle_module_detect_int:8805(eth0)]SFP+ module is not
> initialized

and here, we give up trying in bnx2x_handle_module_detect_int().
So, at this point, we have not read the EEPROM, and I don't see
any further attempts to try and read the EEPROM.

> Mar 14 14:42:23 empire kernel: [   83.627251] bnx2x:
> [bnx2x_attn_int_deasserted:5181(eth0)]about to mask 0xfffffffe at IGU
> addr 0x442d10
> Mar 14 14:42:23 empire kernel: [   83.627255] bnx2x:
> [bnx2x_attn_int_deasserted:5194(eth0)]aeu_mask 16  newly deasserted 1
> Mar 14 14:42:23 empire kernel: [   83.627255] bnx2x:
> [bnx2x_attn_int_deasserted:5196(eth0)]new mask 17
> Mar 14 14:42:23 empire kernel: [   83.627257] bnx2x:
> [bnx2x_attn_int_deasserted:5201(eth0)]attn_state 801
> Mar 14 14:42:23 empire kernel: [   83.627258] bnx2x:
> [bnx2x_attn_int_deasserted:5203(eth0)]new state 800
> Mar 14 14:42:23 empire kernel: [   83.627259] bnx2x:
> [bnx2x_igu_ack_sb_gen:652(eth0)]write 0x01400002 to IGU addr 0x442000
> Mar 14 14:42:23 empire kernel: [   83.627287] bnx2x:
> [bnx2x_igu_ack_sb_gen:652(eth0)]write 0x02200000 to IGU addr 0x442000
> Mar 14 14:42:23 empire kernel: [   83.627422] bnx2x:
> [bnx2x_analyze_link_error:13741(eth0)]Analyze TX Fault

This appears to be a periodic task executed once per second.

> Mar 14 14:42:23 empire kernel: [   83.627423] bnx2x:
> [bnx2x_analyze_link_error:13747(eth0)]Link changed:[0 0]->1
> Mar 14 14:42:23 empire kernel: [   83.627424] bnx2x:
> [bnx2x_sfp_tx_fault_detection:13895(eth0)]Change TX_Fault LED: ->0
> Mar 14 14:42:23 empire kernel: [   83.627426] bnx2x:
> [bnx2x_set_e3_module_fault_led:8585(eth0)]Setting Fault LED to 0 using
> pin cfg 5
> Mar 14 14:42:23 empire kernel: [   83.627429] bnx2x:
> [bnx2x_set_gpio:2123(eth0)]Set GPIO 0 (shift 4) -> output low
> Mar 14 14:42:23 empire kernel: [   83.627432] bnx2x:
> [bnx2x_sp_task:5681(eth0)]sp task invoked
> Mar 14 14:42:23 empire kernel: [   83.627434] bnx2x:
> [bnx2x_sp_task:5690(eth0)]status 1
> Mar 14 14:42:23 empire kernel: [   83.627434] bnx2x:
> [bnx2x_sp_task:5691(eth0)]setting interrupt_occurred to 0
> Mar 14 14:42:23 empire kernel: [   83.627436] bnx2x:
> [bnx2x_attn_int:5221(eth0)]attn_bits 0  attn_ack 801  asserted 0
> deasserted 800
> Mar 14 14:42:23 empire kernel: [   83.627444] bnx2x:
> [bnx2x_attn_int_deasserted:5146(eth0)]attn: 00000000 00000000 02180000
> 00000000 00000000
> Mar 14 14:42:23 empire kernel: [   83.627446] bnx2x:
> [bnx2x_attn_int_deasserted:5181(eth0)]about to mask 0xfffff7ff at IGU
> addr 0x442d10
> Mar 14 14:42:23 empire kernel: [   83.627449] bnx2x:
> [bnx2x_attn_int_deasserted:5194(eth0)]aeu_mask 17  newly deasserted
> 800
> Mar 14 14:42:23 empire kernel: [   83.627449] bnx2x:
> [bnx2x_attn_int_deasserted:5196(eth0)]new mask 17
> Mar 14 14:42:23 empire kernel: [   83.627451] bnx2x:
> [bnx2x_attn_int_deasserted:5201(eth0)]attn_state 800
> Mar 14 14:42:23 empire kernel: [   83.627452] bnx2x:
> [bnx2x_attn_int_deasserted:5203(eth0)]new state 0
> Mar 14 14:42:23 empire kernel: [   83.627453] bnx2x:
> [bnx2x_igu_ack_sb_gen:652(eth0)]write 0x01400003 to IGU addr 0x442000
> Mar 14 14:42:24 empire kernel: [   84.193629] bnx2x:
> [bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0
> Mar 14 14:42:24 empire kernel: [   84.641795] bnx2x:
> [bnx2x_analyze_link_error:13741(eth0)]Analyze TX Fault
> Mar 14 14:42:24 empire kernel: [   84.641798] bnx2x:
> [bnx2x_analyze_link_error:13747(eth0)]Link changed:[0 0]->1
> Mar 14 14:42:24 empire kernel: [   84.641801] bnx2x:
> [bnx2x_sfp_tx_fault_detection:13895(eth0)]Change TX_Fault LED: ->0
> Mar 14 14:42:24 empire kernel: [   84.641804] bnx2x:
> [bnx2x_set_e3_module_fault_led:8585(eth0)]Setting Fault LED to 0 using
> pin cfg 5
> Mar 14 14:42:24 empire kernel: [   84.641809] bnx2x:
> [bnx2x_set_gpio:2123(eth0)]Set GPIO 0 (shift 4) -> output low
> Mar 14 14:42:25 empire kernel: [   85.217675] bnx2x:
> [bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0
> Mar 14 14:42:25 empire kernel: [   85.665873] bnx2x:
> [bnx2x_analyze_link_error:13741(eth0)]Analyze TX Fault
> Mar 14 14:42:25 empire kernel: [   85.665876] bnx2x:
> [bnx2x_analyze_link_error:13747(eth0)]Link changed:[0 0]->1
> Mar 14 14:42:25 empire kernel: [   85.665878] bnx2x:
> [bnx2x_sfp_tx_fault_detection:13895(eth0)]Change TX_Fault LED: ->0
> Mar 14 14:42:25 empire kernel: [   85.665882] bnx2x:
> [bnx2x_set_e3_module_fault_led:8585(eth0)]Setting Fault LED to 0 using
> pin cfg 5
> ++++++++++++fault repeats for a while++++++++++++
> Mar 14 14:43:09 empire kernel: [  129.251715] bnx2x:
> [bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0
> Mar 14 14:43:09 empire kernel: [  129.699906] bnx2x:
> [bnx2x_analyze_link_error:13741(eth0)]Analyze TX Fault
> Mar 14 14:43:09 empire kernel: [  129.699910] bnx2x:
> [bnx2x_analyze_link_error:13747(eth0)]Link changed:[0 0]->1
> Mar 14 14:43:09 empire kernel: [  129.699912] bnx2x:
> [bnx2x_sfp_tx_fault_detection:13895(eth0)]Change TX_Fault LED: ->0
> Mar 14 14:43:09 empire kernel: [  129.699915] bnx2x:
> [bnx2x_set_e3_module_fault_led:8585(eth0)]Setting Fault LED to 0 using
> pin cfg 5
> Mar 14 14:43:09 empire kernel: [  129.699920] bnx2x:
> [bnx2x_set_gpio:2123(eth0)]Set GPIO 0 (shift 4) -> output low
> Mar 14 14:43:10 empire kernel: [  130.275764] bnx2x:
> [bnx2x_stats_handle:1397(eth0)]state 0 -> event 2 -> state 0

This being the last of them...

> Mar 14 14:43:10 empire kernel: [  130.439097] bnx2x:
> [bnx2x_igu_ack_sb_gen:652(eth0)]write 0x02200000 to IGU addr 0x442000
> Mar 14 14:43:10 empire kernel: [  130.439148] bnx2x:
> [bnx2x_sp_task:5681(eth0)]sp task invoked
> Mar 14 14:43:10 empire kernel: [  130.439152] bnx2x:
> [bnx2x_sp_task:5690(eth0)]status 1
> Mar 14 14:43:10 empire kernel: [  130.439153] bnx2x:
> [bnx2x_sp_task:5691(eth0)]setting interrupt_occurred to 0
> Mar 14 14:43:10 empire kernel: [  130.439157] bnx2x:
> [bnx2x_attn_int:5221(eth0)]attn_bits 100  attn_ack 0  asserted 100
> deasserted 0
> Mar 14 14:43:10 empire kernel: [  130.439162] bnx2x:
> [bnx2x_attn_int_asserted:4026(eth0)]aeu_mask 17  newly asserted 100
> Mar 14 14:43:10 empire kernel: [  130.439164] bnx2x:
> [bnx2x_attn_int_asserted:4028(eth0)]new mask 17
> Mar 14 14:43:10 empire kernel: [  130.439166] bnx2x:
> [bnx2x_attn_int_asserted:4033(eth0)]attn_state 0
> Mar 14 14:43:10 empire kernel: [  130.439168] bnx2x:
> [bnx2x_attn_int_asserted:4035(eth0)]new state 100
> Mar 14 14:43:10 empire kernel: [  130.439174] bnx2x:
> [bnx2x_stats_handle:1397(eth0)]state 0 -> event 3 -> state 0
> Mar 14 14:43:10 empire kernel: [  130.439255] bnx2x:
> [bnx2x_link_update:6841(eth0)]port 0, XGXS?1, int_status 0x0

This looks like we got an XGXS interrupt, indicating that the link
came up at 2500Mbps - it looks like bnx2x has the ability to
detect the link speed from the module, which we don't have on the
mcbin.

> Mar 14 14:43:10 empire kernel: [  130.439259] bnx2x:
> [bnx2x_link_update:6848(eth0)]int_mask 0x0 MI_INT 0, SERDES_LINK 0
> Mar 14 14:43:10 empire kernel: [  130.439262] bnx2x:
> [bnx2x_link_update:6852(eth0)] 10G 0, XGXS_LINK f
> Mar 14 14:43:10 empire kernel: [  130.439337] bnx2x:
> [bnx2x_warpcore_read_status:5735(eth0)]0x81d1 = 0x400
> Mar 14 14:43:10 empire kernel: [  130.439411] bnx2x:
> [bnx2x_warpcore_read_status:5808(eth0)]lane 2 gp_speed 0x203
> Mar 14 14:43:10 empire kernel: [  130.439413] bnx2x:
> [bnx2x_get_link_speed_duplex:5548(eth0)]phy link up
> Mar 14 14:43:10 empire kernel: [  130.439416] bnx2x:
> [bnx2x_get_link_speed_duplex:5624(eth0)] phy_link_up 1 line_speed 2500

and here it decodes the speed by reading from the PHY.

So, it seems bnx2x also tries to read the module EEPROM, but fails
to, and while trying it keeps power cycling the module, eventually
giving up.  It doesn't power down the module, but leaves it powered.
After a while, the PHY on the board detects that one of its SERDES
lanes (which is connected to the SFP port) has link at 2.5Gbps, and
configures appropriately for that.

So, bnx2x is not doing this through being able to read the EEPROM
and configure based on what it found, but by having the capability
to detect the SERDES signal coming from the SFP, work out what
speed and (presumably) negotiation protocol (802.3z vs SGMII) is
in use and use that to bring up the link.

The Armada 8040 does not have that capability: it is unable to
automatically detect the SERDES protocol, needing software to
program several different hardware blocks according to the
desired speed.  The only thing I can think of doing in this
circumstance is to allow complete manual configuration. Hmm.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Marc Micalizzi]

On Thu, Mar 14, 2019 at 8:22 PM Russell King - ARM Linux admin
<linux@armlinux.org.uk> wrote:

> > Mar 14 14:43:10 empire kernel: [  130.439255] bnx2x:
> > [bnx2x_link_update:6841(eth0)]port 0, XGXS?1, int_status 0x0
>
> This looks like we got an XGXS interrupt, indicating that the link
> came up at 2500Mbps - it looks like bnx2x has the ability to
> detect the link speed from the module, which we don't have on the
> mcbin.

For the Broadcom NetExtreme II cards, they have a tool called ediag in
which you can set, among many other nvram parameters, the default
speed to use, OOB it is 1000Mbps, but for this I have it set to
2500Mbps, so it tries that first--from what I understand though, if it
is set to 1000Mbps, you can still force it to 2500Mbps using ethtool
(with the modifications to the driver to allow 2500BaseX with warpcore
of course)--so I'm not sure how much of it is detecting the speed from
the module, and how much of it is just setting the default speed from
the nvram and confirming that the link is up, or if that even matters.

> > Mar 14 14:43:10 empire kernel: [  130.439259] bnx2x:
> > [bnx2x_link_update:6848(eth0)]int_mask 0x0 MI_INT 0, SERDES_LINK 0
> > Mar 14 14:43:10 empire kernel: [  130.439262] bnx2x:
> > [bnx2x_link_update:6852(eth0)] 10G 0, XGXS_LINK f
> > Mar 14 14:43:10 empire kernel: [  130.439337] bnx2x:
> > [bnx2x_warpcore_read_status:5735(eth0)]0x81d1 = 0x400
> > Mar 14 14:43:10 empire kernel: [  130.439411] bnx2x:
> > [bnx2x_warpcore_read_status:5808(eth0)]lane 2 gp_speed 0x203
> > Mar 14 14:43:10 empire kernel: [  130.439413] bnx2x:
> > [bnx2x_get_link_speed_duplex:5548(eth0)]phy link up
> > Mar 14 14:43:10 empire kernel: [  130.439416] bnx2x:
> > [bnx2x_get_link_speed_duplex:5624(eth0)] phy_link_up 1 line_speed 2500
>
> and here it decodes the speed by reading from the PHY.
>
> So, it seems bnx2x also tries to read the module EEPROM, but fails
> to, and while trying it keeps power cycling the module, eventually
> giving up.  It doesn't power down the module, but leaves it powered.
> After a while, the PHY on the board detects that one of its SERDES
> lanes (which is connected to the SFP port) has link at 2.5Gbps, and
> configures appropriately for that.
>
> So, bnx2x is not doing this through being able to read the EEPROM
> and configure based on what it found, but by having the capability
> to detect the SERDES signal coming from the SFP, work out what
> speed and (presumably) negotiation protocol (802.3z vs SGMII) is
> in use and use that to bring up the link.

As with what I mentioned above about the nvram configutation, I don't
know enough about any of the negotiation protocols to know if, and
what role, they would play when the default speed from the card
configuration is being used. At any rate, it is able to eventually
read the EEPROM as I do get output from ethtool -m. My guess would be,
if it's power cycling to try to read the module as you mentioned
above, after it gives up and leaves the module powered the soft EEPROM
populates after a while and then works, albeit not in a way that is
involved in configuration.
(With the change to sfp_sm_event() to have sfp_module_tx_enable on
insert, would that mean that there's no power cycling to read the
EEPROM on the mcbin as well? Or is that unrelated?)

> The Armada 8040 does not have that capability: it is unable to
> automatically detect the SERDES protocol, needing software to
> program several different hardware blocks according to the
> desired speed.  The only thing I can think of doing in this
> circumstance is to allow complete manual configuration. Hmm.

I wouldn't be at all opposed to manual configuration, if that's what
makes the most sense. I would guess that would end up being via the
kernel (or modprobe if mvpp2x is compiled as a module) cmdline?

Thank you for the detailed analysis, insight and help,
Marc

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: MacchiatoBin SingleShot 2500BASE-X port (eth3) with GPON SFP on mvpp2x

[Marc Micalizzi]

I've made a bit of progress on this (it's mostly working, just a few
quirks) and just was hoping for some input. I switch the GPON SFP
module with my ISP for one that actually has a physical EEPROM (and is
otherwise the same hardware, albeit a labeled different vendor),
however is still not very standards compliant.
It appears the basis for the Huawei and this Alcatel module
https://www.sourcephotonics.com/wp-content/uploads/2017/08/DS-8085-02_SPS-34-24T-HP-TDFO.pdf

# ethtool -m eth3
        Identifier                                : 0x03 (SFP)
        Extended identifier                       : 0x04 (GBIC/SFP
defined by 2-wire interface ID)
        Connector                                 : 0x01 (SC)
        Transceiver codes                         : 0x00 0x00 0x00
0x02 0x00 0x00 0x00 0x00
        Transceiver type                          : Ethernet: 1000BASE-LX
        Encoding                                  : 0x03 (NRZ)
        BR, Nominal                               : 2500MBd
        Rate identifier                           : 0x0c (reserved or unknown)
        Length (SMF,km)                           : 20km
        Length (SMF)                              : 20000m
        Length (50um)                             : 0m
        Length (62.5um)                           : 0m
        Length (Copper)                           : 0m
        Length (OM3)                              : 0m
        Laser wavelength                          : 33685nm
        Vendor name                               : ALCATELLUCENT
        Vendor OUI                                : 00:00:00
        Vendor PN                                 : G010SP
        Vendor rev                                : 10
        Option values                             : 0x00 0x1a
        Option                                    : RX_LOS implemented
        Option                                    : TX_FAULT implemented
        Option                                    : TX_DISABLE implemented
        BR margin, max                            : 0%
        BR margin, min                            : 0%
        Vendor SN                                 : ALCLFXXXXXXX
        Date code                                 : 170515
        Optical diagnostics support               : Yes
        Laser bias current                        : 9.558 mA
        Laser output power                        : 1.6278 mW / 2.12 dBm
        Receiver signal average optical power     : 0.0157 mW / -18.04 dBm
        Module temperature                        : 49.95 degrees C /
121.91 degrees F
        Module voltage                            : 3.3399 V
        Alarm/warning flags implemented           : Yes
        Laser bias current high alarm             : Off
        Laser bias current low alarm              : Off
        Laser bias current high warning           : Off
        Laser bias current low warning            : Off
        Laser output power high alarm             : Off
        Laser output power low alarm              : Off
        Laser output power high warning           : Off
        Laser output power low warning            : Off
        Module temperature high alarm             : Off
        Module temperature low alarm              : Off
        Module temperature high warning           : Off
        Module temperature low warning            : Off
        Module voltage high alarm                 : Off
        Module voltage low alarm                  : Off
        Module voltage high warning               : Off
        Module voltage low warning                : Off
        Laser rx power high alarm                 : Off
        Laser rx power low alarm                  : Off
        Laser rx power high warning               : Off
        Laser rx power low warning                : Off
        Laser bias current high alarm threshold   : 90.000 mA
        Laser bias current low alarm threshold    : 0.000 mA
        Laser bias current high warning threshold : 70.000 mA
        Laser bias current low warning threshold  : 0.000 mA
        Laser output power high alarm threshold   : 3.1622 mW / 5.00 dBm
        Laser output power low alarm threshold    : 0.8912 mW / -0.50 dBm
        Laser output power high warning threshold : 2.8183 mW / 4.50 dBm
        Laser output power low warning threshold  : 1.0000 mW / 0.00 dBm
        Module temperature high alarm threshold   : 100.00 degrees C /
212.00 degrees F
        Module temperature low alarm threshold    : -50.00 degrees C /
-58.00 degrees F
        Module temperature high warning threshold : 95.00 degrees C /
203.00 degrees F
        Module temperature low warning threshold  : -40.00 degrees C /
-40.00 degrees F
        Module voltage high alarm threshold       : 3.6000 V
        Module voltage low alarm threshold        : 3.0000 V
        Module voltage high warning threshold     : 3.5000 V
        Module voltage low warning threshold      : 3.1000 V
        Laser rx power high alarm threshold       : 0.1995 mW / -7.00 dBm
        Laser rx power low alarm threshold        : 0.0015 mW / -28.24 dBm
        Laser rx power high warning threshold     : 0.1584 mW / -8.00 dBm
        Laser rx power low warning threshold      : 0.0020 mW / -26.99 dBm

I've managed to get it mostly working through a series of hacky
changes to drivers/net/phy/sfp.c, and just had a few observations that
maybe there's a better way to do. (These also likely hold true for the
Huawei MA5671A, albeit with a soft EEPROM, I never got this far)
-One thing I've come to understand, is the GPON SFP uses the Rate
Select pin as Dying Gasp, which presents obvious problems. Luckily, in
this case, Dying Gasp can be disabled by sshing into the SoC running
on the SFP, but this still is less than ideal.
-Additionally, the module uses the TXFAULT pin as serial tx when
asc0=0 in the uboot environment, which is not the default behaviour,
however there is a serial blast over the TXFAULT at power up time,
which also presents obvious problems.
-Like the Huawei MA5671A, the BR, Nominal and encoding reported by
this module is wrong, at 2500MBd and NRZ(which is used on the GPON
side, but not between host and sfp) instead of 3200MBd and 8b10
-sfp_read in sfp_hwmon_insert fails with -ENXIO for a few seconds on
cold boot at the very least
-the LOS signal from the module is questionable at best, some of which
may have been actually a result of Dying Gasp being enabled (I need to
further test it), but also considering that even with a LOS there may
be desire to access the SoC on the SFP
-the module takes about 70 seconds from power to fully booted, to
which end I've left tx_enable on at all times for the module

So the hacky patchset I've got it working with is as follows:
diff --git a/drivers/net/phy/sfp.c b/drivers/net/phy/sfp.c
index fd8bb998ae52..7a09aff9ae82 100644
--- a/drivers/net/phy/sfp.c
+++ b/drivers/net/phy/sfp.c
@@ -1229,6 +1418,12 @@ static void sfp_sm_link_check_los(struct sfp *sfp)
        else if (!(sfp->id.ext.options & cpu_to_be16(SFP_OPTIONS_LOS_NORMAL)))
                los = 0;
+        if (los && !memcmp(sfp->id.base.vendor_name, "ALCATELLUCENT   ", 16) &&
+                   !memcmp(sfp->id.base.vendor_pn  , "G010SP          ", 16)) {
+               dev_info(sfp->dev, "Ignoring LOS for GPON SFP");
+               los = 0;
+       }
+
        if (los)
                sfp_sm_next(sfp, SFP_S_WAIT_LOS, 0);
        else
@@ -1267,14 +1462,20 @@ static void sfp_sm_fault(struct sfp *sfp, bool warn)
 static void sfp_sm_mod_init(struct sfp *sfp)
 {
+       unsigned int delay = 0;
        sfp_module_tx_enable(sfp);
        /* Wait t_init before indicating that the link is up, provided the
         * current state indicates no TX_FAULT.  If TX_FAULT clears before
         * this time, that's fine too.
         */
-       sfp_sm_next(sfp, SFP_S_INIT, T_INIT_JIFFIES);
-       sfp->sm_retries = 5;
+       if (!memcmp(sfp->id.base.vendor_name, "ALCATELLUCENT   ", 16) &&
+            !memcmp(sfp->id.base.vendor_pn  , "G010SP          ", 16)) {
+                delay = msecs_to_jiffies(7500);
+                dev_info(sfp->dev, "Delaying INIT for 7.5 seconds");
+        }
+       sfp_sm_next(sfp, SFP_S_INIT, T_INIT_JIFFIES + delay);
+       sfp->sm_retries = delay ? 0 : 5;
        /* Setting the serdes link mode is guesswork: there's no
         * field in the EEPROM which indicates what mode should
@@ -1475,9 +1755,10 @@ static void sfp_sm_event(struct sfp *sfp,
unsigned int event)
         */
        switch (sfp->sm_mod_state) {
        default:
               if (event == SFP_E_INSERT && sfp->attached) {
-                        sfp_module_tx_disable(sfp);
+                       sfp_module_tx_enable(sfp);
                        sfp_sm_ins_next(sfp, SFP_MOD_PROBE, T_PROBE_INIT);
                }
                break;
@@ -1491,10 +1772,12 @@ static void sfp_sm_event(struct sfp *sfp,
unsigned int event)
                                sfp_sm_ins_next(sfp, SFP_MOD_PRESENT, 0);
                        else if (val > 0)
                                sfp_sm_ins_next(sfp, SFP_MOD_HPOWER, val);
-                       else if (val != -EAGAIN)
+                       else if (val != -EAGAIN && val != -ENXIO)
                                sfp_sm_ins_next(sfp, SFP_MOD_ERROR, 0);
-                       else
+                       else if (val == -EAGAIN)
                                sfp_sm_set_timer(sfp, T_PROBE_RETRY);
+                       else
+                               sfp_sm_set_timer(sfp, T_FAULT_RECOVER);
                }
                break;
@@ -1526,7 +1809,9 @@ static void sfp_sm_event(struct sfp *sfp,
unsigned int event)
                         * as this resets the PHY. Otherwise, raise it to
                         * turn the laser off.
                         */
-                       if (!sfp->mod_phy)
+                       if (!sfp->mod_phy &&
+                           (memcmp(sfp->id.base.vendor_name,
"ALCATELLUCENT   ", 16) ||
+                            memcmp(sfp->id.base.vendor_pn  , "G010SP
        ", 16)))
                                sfp_module_tx_disable(sfp);
                        sfp->sm_dev_state = SFP_DEV_DOWN;
                }
@@ -1847,11 +2151,13 @@ static int sfp_probe(struct platform_device *pdev)
            gpiod_get_value_cansleep(sfp->gpio[GPIO_RATE_SELECT]))
                sfp->state |= SFP_F_RATE_SELECT;
        sfp_set_state(sfp, sfp->state);
-       sfp_module_tx_disable(sfp);
+
+        if (!memcmp(sfp->id.base.vendor_name, "ALCATELLUCENT   ", 16) &&
+            !memcmp(sfp->id.base.vendor_pn  , "G010SP          ", 16)) {
+               sfp_module_tx_enable(sfp);
+       } else {
+               sfp_module_tx_disable(sfp);
+       }
        for (i = 0; i < GPIO_MAX; i++) {
                if (gpio_flags[i] != GPIOD_IN || !sfp->gpio[i])

diff --git a/drivers/net/phy/sfp-bus.c b/usr/src/linux/drivers/net/phy/sfp-bus.c
index ad9db652874d..6e31a37d909a 100644
--- a/drivers/net/phy/sfp-bus.c
+++ b/drivers/net/phy/sfp-bus.c
@@ -233,6 +233,14 @@ void sfp_parse_support(struct sfp_bus *bus, const
struct sfp_eeprom_id *id,
                        phylink_set(modes, 1000baseX_Full);
        }
+       /*GPON SFPs with incorrect EEPROM information*/
+       if ((!memcmp(id->base.vendor_name, "HUAWEI          ", 16) &&
+            !memcmp(id->base.vendor_pn  , "MA5671A         ", 16)) ||
+           (!memcmp(id->base.vendor_name, "ALCATELLUCENT   ", 16) &&
+             !memcmp(id->base.vendor_pn  , "G010SP          ", 16))) {
+               phylink_set(modes, 2500baseX_Full);
+       }
+
        bitmap_or(support, support, modes, __ETHTOOL_LINK_MODE_MASK_NBITS);
        phylink_set(support, Autoneg);

I'm wondering if there's any better way to work around those issues,
possibly in a way that would actually belong in the kernel instead of
being a hacked workaround like what I've put together.
Many of these seem like they would be persistent issues across many
different GPON SFPs, limiting their use outside of ISP provided
hardware.

(Additionally I'm running into a weird performance issue on the
macchiatobin where speeds from a remote pppoe session across a bridge
are fine at 1500Mbps line speed, but on a pppoe session on a vlan
subinterface of the bridge on the macchiatobin they aren't breaking
1070Mbps, but I doubt this mailing list is the place for that)

Thanks,
Marc

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel