Re: DSA Switch: query: Switch configuration, data plane doesn't work whereas control plane works

[Vladimir Oltean <olteanv@gmail.com>]

On Tue, Dec 24, 2024 at 06:20:38PM +0530, sai kumar wrote:
> Thanks Vladimir for your inputs.
> 
> I tried checking the statistics, the in_broadcasts count increases
> with respect to client dhcp discover broadcasts.
> But the same is not happening with external cpu port eth1, the eth1
> in_broadcasts and in_accepted are 0 , Observing the rx frame errors on
> eth1 but
> this count doesn't match to the clients broadcast packets count.
> 
> ethtool -S lan1
> NIC statistics:
>      tx_packets: 7
>      tx_bytes: 626
>      rx_packets: 0
>      rx_bytes: 0
>      in_good_octets: 18112
>      in_bad_octets: 0
>      in_unicast: 0
>      in_broadcasts: 51
>      in_multicasts: 5
>      in_pause: 0
>      in_undersize: 0
>      in_fragments: 0
>      in_oversize: 0
>      in_jabber: 0
>      in_rx_error: 0
>      in_fcs_error: 0
>      out_octets: 7792
>      out_unicast: 0
>      out_broadcasts: 105
>      out_multicasts: 12
>      out_pause: 0
>      excessive: 0
>      collisions: 0
>      deferred: 0
>      single: 0
>      multiple: 0
>      out_fcs_error: 0
>      late: 0
>      hist_64bytes: 105
>      hist_65_127bytes: 17
>      hist_128_255bytes: 0
>      hist_256_511bytes: 51
>      hist_512_1023bytes: 0
>      hist_1024_max_bytes: 0
>      in_discards: 0
>      in_filtered: 0
>      in_accepted: 56
>      in_bad_accepted: 0
>      in_good_avb_class_a: 0
>      in_good_avb_class_b: 0
>      in_bad_avb_class_a: 0
>      in_bad_avb_class_b: 0
>      tcam_counter_0: 0
>      tcam_counter_1: 0
>      tcam_counter_2: 0
>      tcam_counter_3: 0
>      in_da_unknown: 5
>      in_management: 4
>      out_queue_0: 117
>      out_queue_1: 0
>      out_queue_2: 0
>      out_queue_3: 0
>      out_queue_4: 0
>      out_queue_5: 0
>      out_queue_6: 0
>      out_queue_7: 0
>      out_cut_through: 0
>      out_octets_a: 0
>      out_octets_b: 0
>      out_management: 0
>      atu_member_violation: 0
>      atu_miss_violation: 0
>      atu_full_violation: 0
>      vtu_member_violation: 0
>      vtu_miss_violation: 0
> 
> 
> ethtool -S eth1
> NIC statistics:
>      interrupts [CPU 0]: 12
>      interrupts [CPU 1]: 9
>      interrupts [CPU 2]: 5
>      interrupts [CPU 3]: 7
>      interrupts [TOTAL]: 33
>      rx packets [CPU 0]: 0
>      rx packets [CPU 1]: 0
>      rx packets [CPU 2]: 0
>      rx packets [CPU 3]: 0
>      rx packets [TOTAL]: 0
>      tx packets [CPU 0]: 2
>      tx packets [CPU 1]: 0
>      tx packets [CPU 2]: 12
>      tx packets [CPU 3]: 0
>      tx packets [TOTAL]: 14
>      tx recycled [CPU 0]: 0
>      tx recycled [CPU 1]: 0
>      tx recycled [CPU 2]: 0
>      tx recycled [CPU 3]: 0
>      tx recycled [TOTAL]: 0
>      tx confirm [CPU 0]: 6
>      tx confirm [CPU 1]: 3
>      tx confirm [CPU 2]: 2
>      tx confirm [CPU 3]: 3
>      tx confirm [TOTAL]: 14
>      tx S/G [CPU 0]: 0
>      tx S/G [CPU 1]: 0
>      tx S/G [CPU 2]: 0
>      tx S/G [CPU 3]: 0
>      tx S/G [TOTAL]: 0
>      rx S/G [CPU 0]: 0
>      rx S/G [CPU 1]: 0
>      rx S/G [CPU 2]: 0
>      rx S/G [CPU 3]: 0
>      rx S/G [TOTAL]: 0
>      tx error [CPU 0]: 0
>      tx error [CPU 1]: 0
>      tx error [CPU 2]: 0
>      tx error [CPU 3]: 0
>      tx error [TOTAL]: 0
>      rx error [CPU 0]: 6
>      rx error [CPU 1]: 6
>      rx error [CPU 2]: 3
>      rx error [CPU 3]: 4
>      rx error [TOTAL]: 19
>      bp count [CPU 0]: 128
>      bp count [CPU 1]: 128
>      bp count [CPU 2]: 128
>      bp count [CPU 3]: 128
>      bp count [TOTAL]: 512
>      rx dma error: 0
>      rx frame physical error: 2
>      rx frame size error: 17
>      rx header error: 0
>      rx csum error: 0
>      qman cg_tdrop: 0
>      qman wred: 0
>      qman error cond: 0
>      qman early window: 0
>      qman late window: 0
>      qman fq tdrop: 0
>      qman fq retired: 0
>      qman orp disabled: 0
>      congestion time (ms): 0
>      entered congestion: 0
>      congested (0/1): 0
>      p00_in_good_octets: 0
>      p00_in_bad_octets: 2
>      p00_in_unicast: 0
>      p00_in_broadcasts: 0
>      p00_in_multicasts: 0
>      p00_in_pause: 0
>      p00_in_undersize: 0
>      p00_in_fragments: 0
>      p00_in_oversize: 0
>      p00_in_jabber: 0
>      p00_in_rx_error: 1
>      p00_in_fcs_error: 0
>      p00_out_octets: 45440
>      p00_out_unicast: 0
>      p00_out_broadcasts: 312
>      p00_out_multicasts: 63
>      p00_out_pause: 0
>      p00_excessive: 0
>      p00_collisions: 0
>      p00_deferred: 0
>      p00_single: 0
>      p00_multiple: 0
>      p00_out_fcs_error: 0
>      p00_late: 0
>      p00_hist_64bytes: 0
>      p00_hist_65_127bytes: 306
>      p00_hist_128_255bytes: 8
>      p00_hist_256_511bytes: 61
>      p00_hist_512_1023bytes: 0
>      p00_hist_1024_max_bytes: 0
>      p00_in_discards: 0
>      p00_in_filtered: 0
>      p00_in_accepted: 0
>      p00_in_bad_accepted: 0
>      p00_in_good_avb_class_a: 0
>      p00_in_good_avb_class_b: 0
>      p00_in_bad_avb_class_a: 0
>      p00_in_bad_avb_class_b: 0
>      p00_tcam_counter_0: 0
>      p00_tcam_counter_1: 0
>      p00_tcam_counter_2: 0
>      p00_tcam_counter_3: 0
>      p00_in_da_unknown: 0
>      p00_in_management: 0
>      p00_out_queue_0: 373
>      p00_out_queue_1: 0
>      p00_out_queue_2: 0
>      p00_out_queue_3: 0
>      p00_out_queue_4: 0
>      p00_out_queue_5: 0
>      p00_out_queue_6: 2
>      p00_out_queue_7: 0
>      p00_out_cut_through: 0
>      p00_out_octets_a: 0
>      p00_out_octets_b: 0
>      p00_out_management: 11
>      p00_atu_member_violation: 0
>      p00_atu_miss_violation: 0
>      p00_atu_full_violation: 0
>      p00_vtu_member_violation: 0
>      p00_vtu_miss_violation: 0

Hmmm....

I recognize the ethtool stats as belonging to the DPAA1 Ethernet driver.

And I see that the Marvell 6390 has undocumented EDSA tag support, thus
using DSA (with no EtherType) by default. This reminds me of this
discussion with Tobias Waldekranz. The FMan parser sees errors due to
the somewhat controversial Marvell original DSA tag decisions:
https://lore.kernel.org/netdev/20210323102326.3677940-1-tobias@waldekranz.com/

Could you please force EDSA with this switch and see if the packet loss
situation improves? Simplest way would be:

ip link set eth0 down
echo edsa > /sys/class/net/eth0/dsa/tagging
ip link set lan1 up

If this helps, I would recommend reading
Documentation/devicetree/bindings/net/dsa/dsa-port.yaml to see how to
make this board use edsa persistently.