Re: [PATCH 3/3] net: stmmac: Add DWMAC glue layer for Renesas GBETH

["Russell King (Oracle)" <linux@armlinux.org.uk>]

On Sun, Mar 02, 2025 at 10:02:15PM +0000, Lad, Prabhakar wrote:
> Hi Russell,
> 
> On Sun, Mar 2, 2025 at 9:44 PM Russell King (Oracle)
> <linux@armlinux.org.uk> wrote:
> >
> > On Sun, Mar 02, 2025 at 09:20:49PM +0000, Lad, Prabhakar wrote:
> > > Hi Russell,
> > > > What is the reason for setting this flag? If it's because of suspend/
> > > > resume failures, does my "net: stmmac: fix resume failures due to
> > > > RX clock" series solve this for you without requiring this flag?
> > > >
> > > Ive set this flag based on the configuration supported by this IP.
> > > Unfortunately the platform which I am working on doesn't support s2r
> > > yet so I cannot test suspend/resume path yet. But I do see an issue
> > > when I unload and load just the glue module the DMA reset fails.
> >
> > Thanks for that feedback - that's a scenario I hadn't considered.
> >
> > I was trying to avoid having to disable LPI RX clock-stop on suspend by
> > ensuring that it was enabled at resume time. I think that's valid, but
> > you've brought up another similar scenario:
> >
> > - device is brought up, configures RX clock stop
> > - links with media, negotiates EEE
> > - driver is unloaded, link doesn't go down, but due to no traffic goes
> >   into idle, so RX clock is stopped
> > - driver reloaded, RX clock still stopped, reset fails
> >
> > I would like to solve that so we can get the power savings from
> > stopping the clock, but still have reset work when necessary.
> >
> I would be happy to test the patches ;)
> 
> > I'm guessing that the "DMA reset fails" refers to this path:
> >
> > stmmac_open() -> __stmmac_open() -> stmmac_hw_setup() ->
> > stmmac_init_dma_engine() -> stmmac_reset() ?
> >
> Yes.
> 
> > In other words, when the device is being brought back up
> > adminsitratively?
> >
> > What happens if you (replace $if):
> >
> > # ip li set dev $if down
> > # ip li set dev $if up
> >
> > Does that also fail without STMMAC_FLAG_RX_CLK_RUNS_IN_LPI set?
> >
> Logs without STMMAC_FLAG_RX_CLK_RUNS_IN_LPI flag set:
> --------------------------------------------------------------
> root@rzv2h-evk-alpha:~# ip li set dev eth1 down
> [   33.606549] renesas-gbeth 15c40000.ethernet eth1: Link is Down
> root@rzv2h-evk-alpha:~#
> root@rzv2h-evk-alpha:~# ip li set dev eth0 down
> [   37.356992] renesas-gbeth 15c30000.ethernet eth0: Link is Down
> root@rzv2h-evk-alpha:~#
> root@rzv2h-evk-alpha:~# ip li set dev eth1 up
> [   43.974803] renesas-gbeth 15c40000.ethernet eth1: Register
> MEM_TYPE_PAGE_POOL RxQ-0
> [   43.983189] renesas-gbeth 15c40000.ethernet eth1: Register
> MEM_TYPE_PAGE_POOL RxQ-1
> [   43.991155] renesas-gbeth 15c40000.ethernet eth1: Register
> MEM_TYPE_PAGE_POOL RxQ-2
> [   43.999128] renesas-gbeth 15c40000.ethernet eth1: Register
> MEM_TYPE_PAGE_POOL RxQ-3
> [   44.072079] renesas-gbeth 15c40000.ethernet eth1: PHY [stmmac-1:00]
> driver [Microchip KSZ9131 Gigabit PHY] (irq=POLL)
> [   44.094605] dwmac4: Master AXI performs fixed burst length
> [   44.100138] renesas-gbeth 15c40000.ethernet eth1: No Safety
> Features support found
> [   44.107748] renesas-gbeth 15c40000.ethernet eth1: IEEE 1588-2008
> Advanced Timestamp supported
> [   44.116725] renesas-gbeth 15c40000.ethernet eth1: registered PTP clock
> [   44.123352] renesas-gbeth 15c40000.ethernet eth1: configuring for
> phy/rgmii-id link mode
> root@rzv2h-evk-alpha:~#
> root@rzv2h-evk-alpha:~# ip li set dev eth1[   47.207761] renesas-gbeth
> 15c40000.ethernet eth1: Link is Up - 1Gbps/Full - flow control off
> ^C
> root@rzv2h-evk-alpha:~# ^C
> root@rzv2h-evk-alpha:~# ip li set dev eth0 up
> [   55.636722] renesas-gbeth 15c30000.ethernet eth0: Register
> MEM_TYPE_PAGE_POOL RxQ-0
> [   55.645139] renesas-gbeth 15c30000.ethernet eth0: Register
> MEM_TYPE_PAGE_POOL RxQ-1
> [   55.653111] renesas-gbeth 15c30000.ethernet eth0: Register
> MEM_TYPE_PAGE_POOL RxQ-2
> [   55.661073] renesas-gbeth 15c30000.ethernet eth0: Register
> MEM_TYPE_PAGE_POOL RxQ-3
> [   55.732087] renesas-gbeth 15c30000.ethernet eth0: PHY [stmmac-0:00]
> driver [Microchip KSZ9131 Gigabit PHY] (irq=POLL)
> [   55.754612] dwmac4: Master AXI performs fixed burst length
> [   55.760143] renesas-gbeth 15c30000.ethernet eth0: No Safety
> Features support found
> [   55.767740] renesas-gbeth 15c30000.ethernet eth0: IEEE 1588-2008
> Advanced Timestamp supported
> [   55.776705] renesas-gbeth 15c30000.ethernet eth0: registered PTP clock
> [   55.783333] renesas-gbeth 15c30000.ethernet eth0: configuring for
> phy/rgmii-id link mode
> root@rzv2h-evk-alpha:~#
> root@rzv2h-evk-alpha:~# [   58.855844] renesas-gbeth 15c30000.ethernet
> eth0: tx_clk_stop=1
> [   58.861989] renesas-gbeth 15c30000.ethernet eth0: Link is Up -
> 1Gbps/Full - flow control rx/tx
> 
> root@rzv2h-evk-alpha:~#
> root@rzv2h-evk-alpha:~#
> 
> Logs with STMMAC_FLAG_RX_CLK_RUNS_IN_LPI flag set:
> --------------------------------------------------------------
> root@rzv2h-evk-alpha:~# ip li set dev eth1 down
> [   30.053790] renesas-gbeth 15c40000.ethernet eth1: Link is Down
> root@rzv2h-evk-alpha:~# ip li set dev eth0 down
> [   35.366935] renesas-gbeth 15c30000.ethernet eth0: Link is Down
> root@rzv2h-evk-alpha:~# ip li set dev eth1 up
> [   40.448563] renesas-gbeth 15c40000.ethernet eth1: Register
> MEM_TYPE_PAGE_POOL RxQ-0
> [   40.456725] renesas-gbeth 15c40000.ethernet eth1: Register
> MEM_TYPE_PAGE_POOL RxQ-1
> [   40.464893] renesas-gbeth 15c40000.ethernet eth1: Register
> MEM_TYPE_PAGE_POOL RxQ-2
> [   40.472840] renesas-gbeth 15c40000.ethernet eth1: Register
> MEM_TYPE_PAGE_POOL RxQ-3
> [   40.543895] renesas-gbeth 15c40000.ethernet eth1: PHY [stmmac-1:00]
> driver [Microchip KSZ9131 Gigabit PHY] (irq=POLL)
> [   40.566419] dwmac4: Master AXI performs fixed burst length
> [   40.571949] renesas-gbeth 15c40000.ethernet eth1: No Safety
> Features support found
> [   40.579550] renesas-gbeth 15c40000.ethernet eth1: IEEE 1588-2008
> Advanced Timestamp supported
> [   40.588505] renesas-gbeth 15c40000.ethernet eth1: registered PTP clock
> [   40.595135] renesas-gbeth 15c40000.ethernet eth1: configuring for
> phy/rgmii-id link mode
> root@rzv2h-evk-alpha:~#
> root@rzv2h-evk-alpha:~# [   43.687551] renesas-gbeth 15c40000.ethernet
> eth1: Link is Up - 1Gbps/Full - flow control off
> 
> root@rzv2h-evk-alpha:~# ip li set dev eth0 up
> [   49.644479] renesas-gbeth 15c30000.ethernet eth0: Register
> MEM_TYPE_PAGE_POOL RxQ-0
> [   49.652719] renesas-gbeth 15c30000.ethernet eth0: Register
> MEM_TYPE_PAGE_POOL RxQ-1
> [   49.660681] renesas-gbeth 15c30000.ethernet eth0: Register
> MEM_TYPE_PAGE_POOL RxQ-2
> [   49.669059] renesas-gbeth 15c30000.ethernet eth0: Register
> MEM_TYPE_PAGE_POOL RxQ-3
> [   49.740011] renesas-gbeth 15c30000.ethernet eth0: PHY [stmmac-0:00]
> driver [Microchip KSZ9131 Gigabit PHY] (irq=POLL)
> [   49.762518] dwmac4: Master AXI performs fixed burst length
> [   49.768057] renesas-gbeth 15c30000.ethernet eth0: No Safety
> Features support found
> [   49.775655] renesas-gbeth 15c30000.ethernet eth0: IEEE 1588-2008
> Advanced Timestamp supported
> [   49.784609] renesas-gbeth 15c30000.ethernet eth0: registered PTP clock
> [   49.791236] renesas-gbeth 15c30000.ethernet eth0: configuring for
> phy/rgmii-id link mode
> root@rzv2h-evk-alpha:~#
> root@rzv2h-evk-alpha:~# [   52.871635] renesas-gbeth 15c30000.ethernet
> eth0: tx_clk_stop=1
> [   52.877777] renesas-gbeth 15c30000.ethernet eth0: Link is Up -
> 1Gbps/Full - flow control rx/tx

I would like to get to the bottom of why this fails for module removal/
insertion, but not for admistratively down/upping the interface.

Removal of your module will unregister the netdev, and part of that
work will bring the netdev administratively down. When re-inserting
the module, that will trigger various userspace events, and it will
be userspace bringing the network interface(s) back up. This should
be no different from administratively down/upping the interface but
it seems you get different behaviour.

I'd like to understand why that is, because at the moment I'm wondering
whether my patches that address the suspend/resume need further work
before I send them - but in order to assess that, I need to work out
why your issue only seems to occur in the module removal/insertion
and not down/up as well as I'd expect.

Please could you investigate this?

Thanks.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!