[RFC] net: stmmac: keep RXC running in LPI mode to avoid system overload

[RFC] net: stmmac: keep RXC running in LPI mode to avoid system overload

[Romain Baeriswyl]

In order to avoid system overload, the clock RXC from the Phy should not be 
stopped when in LPI mode.

With the RTL8211E PHY which support EEE mode and with Apple Airport Extreme that 
supports it also, the kernel get frozen as soon as some Ethernet transfers are 
on-going. System seems to be overloaded with too many interrupts. The 'top' 
command reports often around ~80% irq.

By letting the RXC clock running even in LPI mode as proposed below, the issue 
seems solved. Is it the right way to proceed?  

What is the power impact to not stop RXC in LPI mode?

---
 drivers/net/ethernet/stmicro/stmmac/stmmac_main.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index e9eab29..d40d26b 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -314,7 +314,8 @@ bool stmmac_eee_init(struct stmmac_priv *priv)
 	/* MAC core supports the EEE feature. */
 	if (priv->dma_cap.eee) {
 		/* Check if the PHY supports EEE */
-		if (phy_init_eee(priv->phydev, 1))
+		/* Keeps RXC running in LPI mode to avoid stability issue */
+		if (phy_init_eee(priv->phydev, 0))
 			goto out;
 
 		if (!priv->eee_active) {
@@ -770,7 +771,8 @@ static void stmmac_adjust_link(struct net_device *dev)
 	/* At this stage, it could be needed to setup the EEE or adjust some
 	 * MAC related HW registers.
 	 */
-	priv->eee_enabled = stmmac_eee_init(priv);
+	if (new_state)
+		priv->eee_enabled = stmmac_eee_init(priv);
 
 	spin_unlock_irqrestore(&priv->lock, flags);
 }
-- 
1.7.1

Re: [RFC] net: stmmac: keep RXC running in LPI mode to avoid system overload

[Giuseppe CAVALLARO]

Hello Romain

On 10/8/2013 5:06 PM, Romain Baeriswyl wrote:
> In order to avoid system overload, the clock RXC from the Phy should not be
> stopped when in LPI mode.
>
> With the RTL8211E PHY which support EEE mode and with Apple Airport Extreme that
> supports it also, the kernel get frozen as soon as some Ethernet transfers are


hmm, I have a board with this transceiver so I could do some tests
this could take a while, unfortunately.

> on-going. System seems to be overloaded with too many interrupts. The 'top'
> command reports often around ~80% irq.

do you mean lpi mac interrupts?

>
> By letting the RXC clock running even in LPI mode as proposed below, the issue
> seems solved. Is it the right way to proceed?

For EEE capability, RX_CLK may be halted for this reason i used it as 
default in the stmmac and never seen your issue.

>
> What is the power impact to not stop RXC in LPI mode?

I can point you to "22.2.2.8a Receive direction LPI transition"
in IEEE802-3az... where is it reported that the PHY  halt the RX_CLK
in LPI mode.

May I suspect some issues on your HW? or disable it with ethtool


peppe

>
> ---
>   drivers/net/ethernet/stmicro/stmmac/stmmac_main.c |    6 ++++--
>   1 files changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
> index e9eab29..d40d26b 100644
> --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
> +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
> @@ -314,7 +314,8 @@ bool stmmac_eee_init(struct stmmac_priv *priv)
>   	/* MAC core supports the EEE feature. */
>   	if (priv->dma_cap.eee) {
>   		/* Check if the PHY supports EEE */
> -		if (phy_init_eee(priv->phydev, 1))
> +		/* Keeps RXC running in LPI mode to avoid stability issue */
> +		if (phy_init_eee(priv->phydev, 0))
>   			goto out;
>
>   		if (!priv->eee_active) {
> @@ -770,7 +771,8 @@ static void stmmac_adjust_link(struct net_device *dev)
>   	/* At this stage, it could be needed to setup the EEE or adjust some
>   	 * MAC related HW registers.
>   	 */
> -	priv->eee_enabled = stmmac_eee_init(priv);
> +	if (new_state)
> +		priv->eee_enabled = stmmac_eee_init(priv);

this seems out of context and not necessary

>
>   	spin_unlock_irqrestore(&priv->lock, flags);
>   }
>

Re: [RFC] net: stmmac: keep RXC running in LPI mode to avoid system overload

[Romain Baeriswyl]

Hello Guiseppe,

Thanks for your answer. Please find below some details and answers.

> > In order to avoid system overload, the clock RXC from the Phy
> > should not be
> > stopped when in LPI mode.
> >
> > With the RTL8211E PHY which support EEE mode and with Apple Airport
> > Extreme that
> > supports it also, the kernel get frozen as soon as some Ethernet
> > transfers are
> 
> 
> hmm, I have a board with this transceiver so I could do some tests
> this could take a while, unfortunately.
> 
> > on-going. System seems to be overloaded with too many interrupts.
> > The 'top'
> > command reports often around ~80% irq.
> 
> do you mean lpi mac interrupts?
 
By disabling the LPI mode with ethtool --set-eee eth0 eee off, the 
interrupt overload remains. Both counters irq_rx_path_in_lpi_mode_n and 
irq_rx_path_exit_lpi_mode_n continue to run meaning the PHY continue
to put the RX path in LPI mode.

Only way I found to get ride of the overload is to keep the RX_CLK running
in LPI mode. In this configuration, the RX link still continue to go in
LPI mode and the two above counters continue to run.

> >
> > By letting the RXC clock running even in LPI mode as proposed
> > below, the issue
> > seems solved. Is it the right way to proceed?
> 
> For EEE capability, RX_CLK may be halted for this reason i used it as
> default in the stmmac and never seen your issue.
> 
> >
> > What is the power impact to not stop RXC in LPI mode?
> 
> I can point you to "22.2.2.8a Receive direction LPI transition"
> in IEEE802-3az... where is it reported that the PHY  halt the RX_CLK
> in LPI mode.

Actually the PHY "may" stop RX_CLK.

> 
> May I suspect some issues on your HW? or disable it with ethtool
> 

Yes, it could be HW issue. It would be very useful if you could reproduce 
the setup using a SoC with "DesignWare Cores Ethernet MAC" core, 
the RTL8211E PHY and an Apple Airport Extreme. The issue could well be between 
the controller and the PHY.

Which Ethernet switch or router did you use to test the EEE mode? 
I may try these equipments as well.

Romain

Re: [RFC] net: stmmac: keep RXC running in LPI mode to avoid system overload

[Giuseppe CAVALLARO]

Hello Romain
On 10/13/2013 10:02 PM, Romain Baeriswyl wrote:
> Hello Guiseppe,
>
> Thanks for your answer. Please find below some details and answers.
>
>>> In order to avoid system overload, the clock RXC from the Phy
>>> should not be
>>> stopped when in LPI mode.
>>>
>>> With the RTL8211E PHY which support EEE mode and with Apple Airport
>>> Extreme that
>>> supports it also, the kernel get frozen as soon as some Ethernet
>>> transfers are
>>
>>
>> hmm, I have a board with this transceiver so I could do some tests
>> this could take a while, unfortunately.
>>
>>> on-going. System seems to be overloaded with too many interrupts.
>>> The 'top'
>>> command reports often around ~80% irq.
>>
>> do you mean lpi mac interrupts?
>
> By disabling the LPI mode with ethtool --set-eee eth0 eee off, the
> interrupt overload remains. Both counters irq_rx_path_in_lpi_mode_n and
> irq_rx_path_exit_lpi_mode_n continue to run meaning the PHY continue
> to put the RX path in LPI mode.
>
> Only way I found to get ride of the overload is to keep the RX_CLK running
> in LPI mode. In this configuration, the RX link still continue to go in
> LPI mode and the two above counters continue to run.
>
>>>
>>> By letting the RXC clock running even in LPI mode as proposed
>>> below, the issue
>>> seems solved. Is it the right way to proceed?
>>
>> For EEE capability, RX_CLK may be halted for this reason i used it as
>> default in the stmmac and never seen your issue.
>>
>>>
>>> What is the power impact to not stop RXC in LPI mode?
>>
>> I can point you to "22.2.2.8a Receive direction LPI transition"
>> in IEEE802-3az... where is it reported that the PHY  halt the RX_CLK
>> in LPI mode.
>
> Actually the PHY "may" stop RX_CLK.

Another option it could be add a new parameter to enable/disable it.
We can decide a parameter for the stmmac or something for ethtool.

>
>>
>> May I suspect some issues on your HW? or disable it with ethtool
>>
>
> Yes, it could be HW issue. It would be very useful if you could reproduce
> the setup using a SoC with "DesignWare Cores Ethernet MAC" core,
> the RTL8211E PHY and an Apple Airport Extreme. The issue could well be between
> the controller and the PHY.
>
> Which Ethernet switch or router did you use to test the EEE mode?

when I tested EEE I connected with P2P two STi boards.
You can test it too.

> I may try these equipments as well.
>

peppe