b53 tags on bpi-r1 (bcm53125)

b53 tags on bpi-r1 (bcm53125)

[Jochen Friedrich]

Hi Florian,


i just tested the latest DSA patches for b53 on a bpi-r1 device (net: 
dsa: b53: Turn on Broadcom tags).

Unfortunately, the bcm53125 does not seem to handle the broadcom tags on 
CPU port although tuned on b53_brcm_hdr_setup(). The tag is still 
visible on a device connected to one of the switch ports.

I wonder if there is something else necessary to make the chip handle 
the tags or if the chip doesn't support broadcom tags after all. The 
documentation at 
https://sourceforge.net/p/ohvesw/svn/12/tree/document/bcm/53125U-DS02-R.pdf 
doesn't document register 3 in the management page, at all...


Thanks,

Jochen

Re: b53 tags on bpi-r1 (bcm53125)

[Florian Fainelli]

Hi Jochen,

On 11/21/2017 12:10 AM, Jochen Friedrich wrote:
> Hi Florian,
> 
> 
> i just tested the latest DSA patches for b53 on a bpi-r1 device (net:
> dsa: b53: Turn on Broadcom tags).
> 
> Unfortunately, the bcm53125 does not seem to handle the broadcom tags on
> CPU port although tuned on b53_brcm_hdr_setup(). The tag is still
> visible on a device connected to one of the switch ports.
> 
> I wonder if there is something else necessary to make the chip handle
> the tags or if the chip doesn't support broadcom tags after all. The
> documentation at
> https://sourceforge.net/p/ohvesw/svn/12/tree/document/bcm/53125U-DS02-R.pdf
> doesn't document register 3 in the management page, at all...

I am afraid U stands for unmanaged, which means we may not have any
possibility to enable Broadcom tags on that particular model. And you
are right, I also don't see any Broadcom HDR control register in the
datasheet, only the 53125M and S seem to have that, darn.

Does the following patch help at all (I don't really expect it to)?
Could you also provide me with two piieces of information that could
help detect such a switch model:

- what is the value of the revision register (should be printed along
with BCM53125 during boot)?
- what is the value of page 02, register 3 during b53_switch_detect()? A
managed switch should have this set to 0x3, whereas I would expect an
unmanaged switch to be 0x0

Thanks!

@@ -330,9 +331,9 @@ static void b53_set_forwarding(struct b53_device
*dev, int enable)
        b53_read8(dev, B53_CTRL_PAGE, B53_SWITCH_MODE, &mgmt);

        if (enable)
-               mgmt |= SM_SW_FWD_EN;
+               mgmt |= SM_SW_FWD_EN | SM_SW_FWD_MODE;
        else
-               mgmt &= ~SM_SW_FWD_EN;
+               mgmt &= ~(SM_SW_FWD_EN | SM_SW_FWD_MODE);

        b53_write8(dev, B53_CTRL_PAGE, B53_SWITCH_MODE, mgmt);


-- 
Florian

Re: b53 tags on bpi-r1 (bcm53125)

[scram287]

Hi Florian,

> Hi Jochen,
>
> On 11/21/2017 12:10 AM, Jochen Friedrich wrote:
>> Hi Florian,
>>
>>
>> i just tested the latest DSA patches for b53 on a bpi-r1 device (net:
>> dsa: b53: Turn on Broadcom tags).
>>
>> Unfortunately, the bcm53125 does not seem to handle the broadcom tags on
>> CPU port although tuned on b53_brcm_hdr_setup(). The tag is still
>> visible on a device connected to one of the switch ports.
>>
>> I wonder if there is something else necessary to make the chip handle
>> the tags or if the chip doesn't support broadcom tags after all. The
>> documentation at
>> https://sourceforge.net/p/ohvesw/svn/12/tree/document/bcm/53125U-DS02-R.pdf
>> doesn't document register 3 in the management page, at all...
>
> I am afraid U stands for unmanaged, which means we may not have any
> possibility to enable Broadcom tags on that particular model. And you
> are right, I also don't see any Broadcom HDR control register in the
> datasheet, only the 53125M and S seem to have that, darn.

I had a closer look at the board, and it really has the 53125S  
soldered. The U datasheet seems to be the only one publically  
available though.

> Does the following patch help at all (I don't really expect it to)?

The patch doesn't make a difference, unfortunately.

> Could you also provide me with two piieces of information that could
> help detect such a switch model:
>
> - what is the value of the revision register (should be printed along
> with BCM53125 during boot)?

b53_common: found switch: BCM53125, rev 4

> - what is the value of page 02, register 3 during b53_switch_detect()? A
> managed switch should have this set to 0x3, whereas I would expect an
> unmanaged switch to be 0x0

I'll check this and tell you.

Thanks, Jochen

Re: b53 tags on bpi-r1 (bcm53125)

[Jochen Friedrich]

Hallo Florian,

> - what is the value of page 02, register 3 during b53_switch_detect()? A
> managed switch should have this set to 0x3, whereas I would expect an
> unmanaged switch to be 0x0

The register has the value 3.

Thanks, Jochen

Re: b53 tags on bpi-r1 (bcm53125)

[Florian Fainelli]

On 11/21/2017 03:07 PM, Jochen Friedrich wrote:
> Hallo Florian,
> 
>> - what is the value of page 02, register 3 during b53_switch_detect()? A
>> managed switch should have this set to 0x3, whereas I would expect an
>> unmanaged switch to be 0x0
> 
> The register has the value 3.

OK, because this is a 53125S, it should support Broadcom tags correctly,
not clear yet what is happening. Just to make sure, you are using
arch/arm/boot/dts/sun7i-a20-lamobo-r1.dts as the DTS for this platform,
right?

If so, can you provide the ethtool -k eth0 output and/or see if
disabling HW VLAN tag support for eth0 helps in any way?
-- 
Florian

Re: b53 tags on bpi-r1 (bcm53125)

[Jochen Friedrich]

Hi Florian,


> OK, because this is a 53125S, it should support Broadcom tags correctly,
> not clear yet what is happening. Just to make sure, you are using
> arch/arm/boot/dts/sun7i-a20-lamobo-r1.dts as the DTS for this platform,
> right?

Yes, that's the DTS I'm using.

> If so, can you provide the ethtool -k eth0 output and/or see if
> disabling HW VLAN tag support for eth0 helps in any way?

It looks like gmac doesn't have HW VLAN support.

Thanks,
Jochen

Re: b53 tags on bpi-r1 (bcm53125)

[Florian Fainelli]

Hi Jochen,

On 11/23/2017 09:05 AM, Jochen Friedrich wrote:
> Hi Florian,
> 
> 
>> OK, because this is a 53125S, it should support Broadcom tags correctly,
>> not clear yet what is happening. Just to make sure, you are using
>> arch/arm/boot/dts/sun7i-a20-lamobo-r1.dts as the DTS for this platform,
>> right?
> 
> Yes, that's the DTS I'm using.
> 
>> If so, can you provide the ethtool -k eth0 output and/or see if
>> disabling HW VLAN tag support for eth0 helps in any way?
> 
> It looks like gmac doesn't have HW VLAN support.

OK, good. I was wondering if this could confuse the GMAC HW into
thinking Broadcom tags where VLAN tags.

With Broadcom tags (or any type of switch tagging protocol), eth0
becomes a conduit interface and no longer a "normal" network device for
applications/socket to use. This means that if you were obtaining an IP
address through a DHCP client using e.g: dhclient eth0, this now needs
to be replaced using dhclient lan1 for instance.

If you use the per-port network devices, do you actually see Broadcom
tags coming out of these interfaces, or is it just when you use eth0
like what you used before that you see that happening?

If this is the former, then we are still back to square one, as this
really should not happen, if this is the latter, it means we need to
update any network scripts to properly make use of DSA-enabled network
devices and use only the per-port network devices now.

Thanks!
-- 
Florian

Re: b53 tags on bpi-r1 (bcm53125)

[Jochen Friedrich]

Hi Florian,

> With Broadcom tags (or any type of switch tagging protocol), eth0
> becomes a conduit interface and no longer a "normal" network device for
> applications/socket to use. This means that if you were obtaining an IP
> address through a DHCP client using e.g: dhclient eth0, this now needs
> to be replaced using dhclient lan1 for instance.

Yes, that's what I would expect.

> If you use the per-port network devices, do you actually see Broadcom
> tags coming out of these interfaces, or is it just when you use eth0
> like what you used before that you see that happening?

For outgoing Packet, I see the tags on eth0, that's expected, but also  
an a laptop connected to any of the external ports (no matter which  
one, wan or lan1-4).
Incoming packets are seen on eth0 untagged and don't get forwarded to  
the corresponding child Interface.

So it seems the switch operates in dumb forwarding mode.

I also tried a pull down on LED Port 0 (mii_dumb_fwdg_en), but the  
switch still remains in dumb mode. However, I would have to double  
check I picked the correct pin, the Bpi-r1 schematics is not  
completely error free :-(.

Thanks, Jochen

Re: b53 tags on bpi-r1 (bcm53125)

[Florian Fainelli]

Hi Jochen,

On 11/23/2017 11:14 AM, Jochen Friedrich wrote:
> Hi Florian,
> 
>> With Broadcom tags (or any type of switch tagging protocol), eth0
>> becomes a conduit interface and no longer a "normal" network device for
>> applications/socket to use. This means that if you were obtaining an IP
>> address through a DHCP client using e.g: dhclient eth0, this now needs
>> to be replaced using dhclient lan1 for instance.
> 
> Yes, that's what I would expect.
> 
>> If you use the per-port network devices, do you actually see Broadcom
>> tags coming out of these interfaces, or is it just when you use eth0
>> like what you used before that you see that happening?
> 
> For outgoing Packet, I see the tags on eth0, that's expected, but also
> an a laptop connected to any of the external ports (no matter which one,
> wan or lan1-4).
> Incoming packets are seen on eth0 untagged and don't get forwarded to
> the corresponding child Interface.>
> So it seems the switch operates in dumb forwarding mode.

Humm, what if you comment that part from b53_set_forwarding:

        b53_read8(dev, B53_CTRL_PAGE, B53_SWITCH_CTRL, &mgmt);
        mgmt |= B53_MII_DUMB_FWDG_EN;
        b53_write8(dev, B53_CTRL_PAGE, B53_SWITCH_CTRL, mgmt);

does this help?

> 
> I also tried a pull down on LED Port 0 (mii_dumb_fwdg_en), but the
> switch still remains in dumb mode. However, I would have to double check
> I picked the correct pin, the Bpi-r1 schematics is not completely error
> free :-(.
> 
> Thanks, Jochen

-- 
Florian

Re: b53 tags on bpi-r1 (bcm53125)

[Jochen Friedrich]

Hi Florian,

unfortunately, this doesn't make any difference.

Just out of curiosity, BPI-R1 has pull-down resistors on LED6 and 7 
(MII_MODE0/1). However, the public available 53125U sheet doesn't 
document these pins:

LED[6] IMP_MODE[0] Pull-up Active low (since IMP Mode is not used)
LED[7] IMP_MODE[1] Pull-up Active low (since IMP Mode is not used)

Is this MII mode maybe incompatible with Broadcom tags?

Thanks,
Jochen

Am 15.12.2017 um 02:55 schrieb Florian Fainelli:
> Hi Jochen,
>
> On 11/23/2017 11:14 AM, Jochen Friedrich wrote:
>> Hi Florian,
>>
>>> With Broadcom tags (or any type of switch tagging protocol), eth0
>>> becomes a conduit interface and no longer a "normal" network device for
>>> applications/socket to use. This means that if you were obtaining an IP
>>> address through a DHCP client using e.g: dhclient eth0, this now needs
>>> to be replaced using dhclient lan1 for instance.
>> Yes, that's what I would expect.
>>
>>> If you use the per-port network devices, do you actually see Broadcom
>>> tags coming out of these interfaces, or is it just when you use eth0
>>> like what you used before that you see that happening?
>> For outgoing Packet, I see the tags on eth0, that's expected, but also
>> an a laptop connected to any of the external ports (no matter which one,
>> wan or lan1-4).
>> Incoming packets are seen on eth0 untagged and don't get forwarded to
>> the corresponding child Interface.>
>> So it seems the switch operates in dumb forwarding mode.
> Humm, what if you comment that part from b53_set_forwarding:
>
>          b53_read8(dev, B53_CTRL_PAGE, B53_SWITCH_CTRL, &mgmt);
>          mgmt |= B53_MII_DUMB_FWDG_EN;
>          b53_write8(dev, B53_CTRL_PAGE, B53_SWITCH_CTRL, mgmt);
>
> does this help?
>
>> I also tried a pull down on LED Port 0 (mii_dumb_fwdg_en), but the
>> switch still remains in dumb mode. However, I would have to double check
>> I picked the correct pin, the Bpi-r1 schematics is not completely error
>> free :-(.
>>
>> Thanks, Jochen

Re: b53 tags on bpi-r1 (bcm53125)

[Florian Fainelli]

Hi Jochen,

Le 12/18/17 à 02:44, Jochen Friedrich a écrit :
> Hi Florian,
> 
> unfortunately, this doesn't make any difference.
> 
> Just out of curiosity, BPI-R1 has pull-down resistors on LED6 and 7
> (MII_MODE0/1). However, the public available 53125U sheet doesn't
> document these pins:
> 
> LED[6] IMP_MODE[0] Pull-up Active low (since IMP Mode is not used)
> LED[7] IMP_MODE[1] Pull-up Active low (since IMP Mode is not used)
> 
> Is this MII mode maybe incompatible with Broadcom tags?

AFAICT, it should not, this is largely independent from enabling
Broadcom tags.

I have now reproduced this on my BPI-R1 as well and am looking at what
might be going wrong.

Thanks!
-- 
Florian

Re: b53 tags on bpi-r1 (bcm53125)

[Florian Fainelli]

Le 12/29/17 à 12:21, Florian Fainelli a écrit :
> Hi Jochen,
> 
> Le 12/18/17 à 02:44, Jochen Friedrich a écrit :
>> Hi Florian,
>>
>> unfortunately, this doesn't make any difference.
>>
>> Just out of curiosity, BPI-R1 has pull-down resistors on LED6 and 7
>> (MII_MODE0/1). However, the public available 53125U sheet doesn't
>> document these pins:
>>
>> LED[6] IMP_MODE[0] Pull-up Active low (since IMP Mode is not used)
>> LED[7] IMP_MODE[1] Pull-up Active low (since IMP Mode is not used)
>>
>> Is this MII mode maybe incompatible with Broadcom tags?
> 
> AFAICT, it should not, this is largely independent from enabling
> Broadcom tags.
> 
> I have now reproduced this on my BPI-R1 as well and am looking at what
> might be going wrong.

OK, so I have been able to find out what was going on. On BCM53125 and
earlier switches, we need to do a couple of things for Broadcom tags to
be usable:

- turn on managed mode (SM_SW_FWD_MODE)
- configure Port 8 for IMP mode (B53_GLOBAL_CONFIG, setting bit
GC_FRM_MGMT_PORT_MII)

After doing that, I can now see the correct outgoing packets on my host,
however, the replies don't seem to be delivered to the per-port DSA
network devices, and I suspect it's because of stmmac, so I am
investigating this now.
-- 
Florian

Re: b53 tags on bpi-r1 (bcm53125)

[Florian Fainelli]

Le 12/29/17 à 13:56, Florian Fainelli a écrit :
> Le 12/29/17 à 12:21, Florian Fainelli a écrit :
>> Hi Jochen,
>>
>> Le 12/18/17 à 02:44, Jochen Friedrich a écrit :
>>> Hi Florian,
>>>
>>> unfortunately, this doesn't make any difference.
>>>
>>> Just out of curiosity, BPI-R1 has pull-down resistors on LED6 and 7
>>> (MII_MODE0/1). However, the public available 53125U sheet doesn't
>>> document these pins:
>>>
>>> LED[6] IMP_MODE[0] Pull-up Active low (since IMP Mode is not used)
>>> LED[7] IMP_MODE[1] Pull-up Active low (since IMP Mode is not used)
>>>
>>> Is this MII mode maybe incompatible with Broadcom tags?
>>
>> AFAICT, it should not, this is largely independent from enabling
>> Broadcom tags.
>>
>> I have now reproduced this on my BPI-R1 as well and am looking at what
>> might be going wrong.
> 
> OK, so I have been able to find out what was going on. On BCM53125 and
> earlier switches, we need to do a couple of things for Broadcom tags to
> be usable:
> 
> - turn on managed mode (SM_SW_FWD_MODE)
> - configure Port 8 for IMP mode (B53_GLOBAL_CONFIG, setting bit
> GC_FRM_MGMT_PORT_MII)
> 
> After doing that, I can now see the correct outgoing packets on my host,
> however, the replies don't seem to be delivered to the per-port DSA
> network devices, and I suspect it's because of stmmac, so I am
> investigating this now.
> 

So stmmac was indeed part of the problem. I had to clear the
GMAC_CONTROL_ACS bit in GMAC_CORE_INIT in order to allow stmmac to
properly receive packets, otherwise, packets were truncated to 8 bytes
on reception which I assume is because the Broadcom tags may look like
some sort of weird LLC/SNAP packet which was confusing the hardware.
This is all working correctly now after a bunch of changes that I will
submit in the next few days.

Preliminary patches available here:

https://github.com/ffainelli/linux/commits/stmmac-fixes

Thank you very much for your patience!
-- 
Florian

Re: b53 tags on bpi-r1 (bcm53125)

[Florian Fainelli]

On 12/29/2017 07:22 PM, Florian Fainelli wrote:
> Le 12/29/17 à 13:56, Florian Fainelli a écrit :
>> Le 12/29/17 à 12:21, Florian Fainelli a écrit :
>>> Hi Jochen,
>>>
>>> Le 12/18/17 à 02:44, Jochen Friedrich a écrit :
>>>> Hi Florian,
>>>>
>>>> unfortunately, this doesn't make any difference.
>>>>
>>>> Just out of curiosity, BPI-R1 has pull-down resistors on LED6 and 7
>>>> (MII_MODE0/1). However, the public available 53125U sheet doesn't
>>>> document these pins:
>>>>
>>>> LED[6] IMP_MODE[0] Pull-up Active low (since IMP Mode is not used)
>>>> LED[7] IMP_MODE[1] Pull-up Active low (since IMP Mode is not used)
>>>>
>>>> Is this MII mode maybe incompatible with Broadcom tags?
>>>
>>> AFAICT, it should not, this is largely independent from enabling
>>> Broadcom tags.
>>>
>>> I have now reproduced this on my BPI-R1 as well and am looking at what
>>> might be going wrong.
>>
>> OK, so I have been able to find out what was going on. On BCM53125 and
>> earlier switches, we need to do a couple of things for Broadcom tags to
>> be usable:
>>
>> - turn on managed mode (SM_SW_FWD_MODE)
>> - configure Port 8 for IMP mode (B53_GLOBAL_CONFIG, setting bit
>> GC_FRM_MGMT_PORT_MII)
>>
>> After doing that, I can now see the correct outgoing packets on my host,
>> however, the replies don't seem to be delivered to the per-port DSA
>> network devices, and I suspect it's because of stmmac, so I am
>> investigating this now.
>>
> 
> So stmmac was indeed part of the problem. I had to clear the
> GMAC_CONTROL_ACS bit in GMAC_CORE_INIT in order to allow stmmac to
> properly receive packets, otherwise, packets were truncated to 8 bytes
> on reception which I assume is because the Broadcom tags may look like
> some sort of weird LLC/SNAP packet which was confusing the hardware.
> This is all working correctly now after a bunch of changes that I will
> submit in the next few days.
> 
> Preliminary patches available here:
> 
> https://github.com/ffainelli/linux/commits/stmmac-fixes
> 
> Thank you very much for your patience!

Turning on managed mode has some other effects, like how multicast
traffic needs to be handled and how to resolve ARL misses, which means
that the changes are too invasive for the upcoming release since we
would be essentially adding multicast support through a number of code
paths (including core DSA).

I will submit a simple change for now which does not turn on Broadcom
tags on switches that require managed mode (5395, 97, 98, 53125 and
friends) and target enabling Broadcom tags for these models for net-next.

Thanks
-- 
Florian

Re: b53 tags on bpi-r1 (bcm53125)

[Jochen Friedrich]

Hi Florian,

Am 04.01.2018 um 05:49 schrieb Florian Fainelli:
> On 12/29/2017 07:22 PM, Florian Fainelli wrote:
>> Le 12/29/17 à 13:56, Florian Fainelli a écrit :
>>> Le 12/29/17 à 12:21, Florian Fainelli a écrit :
>>>> Hi Jochen,
>>>>
>>>> Le 12/18/17 à 02:44, Jochen Friedrich a écrit :
>>>>> Hi Florian,
>>>>>
>>>>> unfortunately, this doesn't make any difference.
>>>>>
>>>>> Just out of curiosity, BPI-R1 has pull-down resistors on LED6 and 7
>>>>> (MII_MODE0/1). However, the public available 53125U sheet doesn't
>>>>> document these pins:
>>>>>
>>>>> LED[6] IMP_MODE[0] Pull-up Active low (since IMP Mode is not used)
>>>>> LED[7] IMP_MODE[1] Pull-up Active low (since IMP Mode is not used)
>>>>>
>>>>> Is this MII mode maybe incompatible with Broadcom tags?
>>>> AFAICT, it should not, this is largely independent from enabling
>>>> Broadcom tags.
>>>>
>>>> I have now reproduced this on my BPI-R1 as well and am looking at what
>>>> might be going wrong.
>>> OK, so I have been able to find out what was going on. On BCM53125 and
>>> earlier switches, we need to do a couple of things for Broadcom tags to
>>> be usable:
>>>
>>> - turn on managed mode (SM_SW_FWD_MODE)
>>> - configure Port 8 for IMP mode (B53_GLOBAL_CONFIG, setting bit
>>> GC_FRM_MGMT_PORT_MII)
>>>
>>> After doing that, I can now see the correct outgoing packets on my host,
>>> however, the replies don't seem to be delivered to the per-port DSA
>>> network devices, and I suspect it's because of stmmac, so I am
>>> investigating this now.
>>>
>> So stmmac was indeed part of the problem. I had to clear the
>> GMAC_CONTROL_ACS bit in GMAC_CORE_INIT in order to allow stmmac to
>> properly receive packets, otherwise, packets were truncated to 8 bytes
>> on reception which I assume is because the Broadcom tags may look like
>> some sort of weird LLC/SNAP packet which was confusing the hardware.
>> This is all working correctly now after a bunch of changes that I will
>> submit in the next few days.
>>
>> Preliminary patches available here:
>>
>> https://github.com/ffainelli/linux/commits/stmmac-fixes
>>
>> Thank you very much for your patience!
> Turning on managed mode has some other effects, like how multicast
> traffic needs to be handled and how to resolve ARL misses, which means
> that the changes are too invasive for the upcoming release since we
> would be essentially adding multicast support through a number of code
> paths (including core DSA).
>
> I will submit a simple change for now which does not turn on Broadcom
> tags on switches that require managed mode (5395, 97, 98, 53125 and
> friends) and target enabling Broadcom tags for these models for net-next.
>
> Thanks

Thanks. That's the best approach for now :-)