[PATCH v2 0/1] Adjust macb max_tx_len for mpfs

[PATCH v2 0/1] Adjust macb max_tx_len for mpfs

[daire.mcnamara]

From: Daire McNamara <daire.mcnamara@microchip.com>

Several customers have reported unexpected ethernet issues whereby
the GEM stops transmitting and receiving. Performing an action such
as ifconfig <ethX> down; ifconfig <ethX> up clears this particular
condition.

The origin of the issue is a stream of AMBA_ERRORS (bit 6) from the
tx queues.

This patch sets the max_tx_length to SRAM size (16 KiB
in the case of mpfs) divided by num_queues (4 in the case of mpfs)
and then subtracts 56 bytes from that figure - resulting in max_tx_len
of 4040.  The max jumbo length is also set to 4040.  These figures
are derived from Cadence erratum 1686.

Change from v1
- Switched from using macb_is_gem() to hw_is_gem() as macb_is_gem()
  relies on capabilities being read and these have not been ascertained
  at this point of the probe routine.

Daire McNamara (1):
  net: macb: Shorten max_tx_len to 4KiB - 56 on mpfs

 drivers/net/ethernet/cadence/macb.h      |  1 +
 drivers/net/ethernet/cadence/macb_main.c | 16 ++++++++++++----
 2 files changed, 13 insertions(+), 4 deletions(-)


base-commit: 09a9639e56c01c7a00d6c0ca63f4c7c41abe075d
-- 
2.25.1

[PATCH v2 1/1] net: macb: Shorten max_tx_len to 4KiB - 56 on mpfs

[daire.mcnamara]

From: Daire McNamara <daire.mcnamara@microchip.com>

On mpfs, with SRAM configured for 4 queues, setting max_tx_len
to GEM_TX_MAX_LEN=0x3f0 results multiple AMBA errors.
Setting max_tx_len to (4KiB - 56) removes those errors.

The details are described in erratum 1686 by Cadence

The max jumbo frame size is also reduced for mpfs to (4KiB - 56).

Signed-off-by: Daire McNamara <daire.mcnamara@microchip.com>
---
 drivers/net/ethernet/cadence/macb.h      |  1 +
 drivers/net/ethernet/cadence/macb_main.c | 16 ++++++++++++----
 2 files changed, 13 insertions(+), 4 deletions(-)

diff --git a/drivers/net/ethernet/cadence/macb.h b/drivers/net/ethernet/cadence/macb.h
index 14dfec4db8f9..989e7c5db9b9 100644
--- a/drivers/net/ethernet/cadence/macb.h
+++ b/drivers/net/ethernet/cadence/macb.h
@@ -1175,6 +1175,7 @@ struct macb_config {
 			    struct clk **hclk, struct clk **tx_clk,
 			    struct clk **rx_clk, struct clk **tsu_clk);
 	int	(*init)(struct platform_device *pdev);
+	unsigned int		max_tx_length;
 	int	jumbo_max_len;
 	const struct macb_usrio_config *usrio;
 };
diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
index 66e30561569e..eb53dcc0c3cd 100644
--- a/drivers/net/ethernet/cadence/macb_main.c
+++ b/drivers/net/ethernet/cadence/macb_main.c
@@ -4095,14 +4095,12 @@ static int macb_init(struct platform_device *pdev)
 
 	/* setup appropriated routines according to adapter type */
 	if (macb_is_gem(bp)) {
-		bp->max_tx_length = GEM_MAX_TX_LEN;
 		bp->macbgem_ops.mog_alloc_rx_buffers = gem_alloc_rx_buffers;
 		bp->macbgem_ops.mog_free_rx_buffers = gem_free_rx_buffers;
 		bp->macbgem_ops.mog_init_rings = gem_init_rings;
 		bp->macbgem_ops.mog_rx = gem_rx;
 		dev->ethtool_ops = &gem_ethtool_ops;
 	} else {
-		bp->max_tx_length = MACB_MAX_TX_LEN;
 		bp->macbgem_ops.mog_alloc_rx_buffers = macb_alloc_rx_buffers;
 		bp->macbgem_ops.mog_free_rx_buffers = macb_free_rx_buffers;
 		bp->macbgem_ops.mog_init_rings = macb_init_rings;
@@ -4839,7 +4837,8 @@ static const struct macb_config mpfs_config = {
 	.clk_init = macb_clk_init,
 	.init = init_reset_optional,
 	.usrio = &macb_default_usrio,
-	.jumbo_max_len = 10240,
+	.max_tx_length = 4040, /* Cadence Erratum 1686 */
+	.jumbo_max_len = 4040,
 };
 
 static const struct macb_config sama7g5_gem_config = {
@@ -4986,8 +4985,17 @@ static int macb_probe(struct platform_device *pdev)
 	bp->tx_clk = tx_clk;
 	bp->rx_clk = rx_clk;
 	bp->tsu_clk = tsu_clk;
-	if (macb_config)
+	if (macb_config) {
+		if (hw_is_gem(bp->regs, bp->native_io)) {
+			if (macb_config->max_tx_length)
+				bp->max_tx_length = macb_config->max_tx_length;
+			else
+				bp->max_tx_length = GEM_MAX_TX_LEN;
+		} else {
+			bp->max_tx_length = MACB_MAX_TX_LEN;
+		}
 		bp->jumbo_max_len = macb_config->jumbo_max_len;
+	}
 
 	bp->wol = 0;
 	if (of_property_read_bool(np, "magic-packet"))
-- 
2.25.1

Re: [PATCH v2 1/1] net: macb: Shorten max_tx_len to 4KiB - 56 on mpfs

[Simon Horman]

On Mon, Apr 17, 2023 at 03:00:41PM +0100, daire.mcnamara@microchip.com wrote:
> From: Daire McNamara <daire.mcnamara@microchip.com>
> 
> On mpfs, with SRAM configured for 4 queues, setting max_tx_len
> to GEM_TX_MAX_LEN=0x3f0 results multiple AMBA errors.
> Setting max_tx_len to (4KiB - 56) removes those errors.
> 
> The details are described in erratum 1686 by Cadence
> 
> The max jumbo frame size is also reduced for mpfs to (4KiB - 56).
> 
> Signed-off-by: Daire McNamara <daire.mcnamara@microchip.com>

...

>  static const struct macb_config sama7g5_gem_config = {
> @@ -4986,8 +4985,17 @@ static int macb_probe(struct platform_device *pdev)
>  	bp->tx_clk = tx_clk;
>  	bp->rx_clk = rx_clk;
>  	bp->tsu_clk = tsu_clk;
> -	if (macb_config)
> +	if (macb_config) {
> +		if (hw_is_gem(bp->regs, bp->native_io)) {
> +			if (macb_config->max_tx_length)
> +				bp->max_tx_length = macb_config->max_tx_length;
> +			else
> +				bp->max_tx_length = GEM_MAX_TX_LEN;
> +		} else {
> +			bp->max_tx_length = MACB_MAX_TX_LEN;
> +		}

Hi Daire,

no need to refresh the patch on my account.
But can the above be simplified as:

               if (macb_is_gem(bp) && hw_is_gem(bp->regs, bp->native_io))
                       bp->max_tx_length = macb_config->max_tx_length;
               else
                       bp->max_tx_length = MACB_MAX_TX_LEN;

>  		bp->jumbo_max_len = macb_config->jumbo_max_len;
> +	}
>  
>  	bp->wol = 0;
>  	if (of_property_read_bool(np, "magic-packet"))
> -- 
> 2.25.1
> 

Re: [PATCH v2 1/1] net: macb: Shorten max_tx_len to 4KiB - 56 on mpfs

[Jakub Kicinski]

On Tue, 18 Apr 2023 16:28:25 +0200 Simon Horman wrote:
> no need to refresh the patch on my account.
> But can the above be simplified as:
> 
>                if (macb_is_gem(bp) && hw_is_gem(bp->regs, bp->native_io))
>                        bp->max_tx_length = macb_config->max_tx_length;
>                else
>                        bp->max_tx_length = MACB_MAX_TX_LEN;

I suspect that DaveM agreed, because patch is set to Changes Requested
in patchwork :) 

Daire, please respin with Simon's suggestion.

Re: [PATCH v2 1/1] net: macb: Shorten max_tx_len to 4KiB - 56 on mpfs

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 2236 bytes --]

Jaukb, Simon,

On Wed, Apr 19, 2023 at 06:02:22PM -0700, Jakub Kicinski wrote:
> On Tue, 18 Apr 2023 16:28:25 +0200 Simon Horman wrote:

[readding the context]

> > > static const struct macb_config sama7g5_gem_config = {
> > > @@ -4986,8 +4985,17 @@ static int macb_probe(struct platform_device *pdev)
> > >       bp->tx_clk = tx_clk;
> > >       bp->rx_clk = rx_clk;
> > >       bp->tsu_clk = tsu_clk;
> > > -     if (macb_config)
> > > +     if (macb_config) {
> > > +             if (hw_is_gem(bp->regs, bp->native_io)) {
> > > +                     if (macb_config->max_tx_length)
> > > +                             bp->max_tx_length = macb_config->max_tx_length;
> > > +                     else
> > > +                             bp->max_tx_length = GEM_MAX_TX_LEN;
> > > +             } else {
> > > +                     bp->max_tx_length = MACB_MAX_TX_LEN;
> > > +             }

> > no need to refresh the patch on my account.
> > But can the above be simplified as:
> > 
> >                if (macb_is_gem(bp) && hw_is_gem(bp->regs, bp->native_io))
> >                        bp->max_tx_length = macb_config->max_tx_length;
> >                else
> >                        bp->max_tx_length = MACB_MAX_TX_LEN;
> 
> I suspect that DaveM agreed, because patch is set to Changes Requested
> in patchwork :) 
> 
> Daire, please respin with Simon's suggestion.

I'm feeling a bit stupid reading this suggestion as I am not sure how it
is supposed to work :(

Firstly, why macb_is_gem() and hw_is_gem()? They both do the same thing,
except last time around we established that macb_is_gem() cannot return
anything other than false at this point.
What have I missed here?

Secondly, is it guaranteed that macb_config::max_tx_length is even
set?

Also, another question...
Is it even possible for `if (macb_config)` to be false?
Isn't it either going to be set to &default_gem_config or to
match->data, no? The driver is pretty inconsistent about if it checks
whether macb_config is non-NULL before accessing it, but from reading
.probe, it seems to be like it is always set to something valid at this
point.

(btw Daire, Nicolas' email has no h in it)

Cheers,
Conor.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v2 1/1] net: macb: Shorten max_tx_len to 4KiB - 56 on mpfs

[Simon Horman]

On Thu, Apr 20, 2023 at 08:18:35AM +0100, Conor Dooley wrote:
> Jaukb, Simon,
> 
> On Wed, Apr 19, 2023 at 06:02:22PM -0700, Jakub Kicinski wrote:
> > On Tue, 18 Apr 2023 16:28:25 +0200 Simon Horman wrote:
> 
> [readding the context]
> 
> > > > static const struct macb_config sama7g5_gem_config = {
> > > > @@ -4986,8 +4985,17 @@ static int macb_probe(struct platform_device *pdev)
> > > >       bp->tx_clk = tx_clk;
> > > >       bp->rx_clk = rx_clk;
> > > >       bp->tsu_clk = tsu_clk;
> > > > -     if (macb_config)
> > > > +     if (macb_config) {
> > > > +             if (hw_is_gem(bp->regs, bp->native_io)) {
> > > > +                     if (macb_config->max_tx_length)
> > > > +                             bp->max_tx_length = macb_config->max_tx_length;
> > > > +                     else
> > > > +                             bp->max_tx_length = GEM_MAX_TX_LEN;
> > > > +             } else {
> > > > +                     bp->max_tx_length = MACB_MAX_TX_LEN;
> > > > +             }
> 
> > > no need to refresh the patch on my account.
> > > But can the above be simplified as:
> > > 
> > >                if (macb_is_gem(bp) && hw_is_gem(bp->regs, bp->native_io))
> > >                        bp->max_tx_length = macb_config->max_tx_length;
> > >                else
> > >                        bp->max_tx_length = MACB_MAX_TX_LEN;
> > 
> > I suspect that DaveM agreed, because patch is set to Changes Requested
> > in patchwork :) 
> > 
> > Daire, please respin with Simon's suggestion.
> 
> I'm feeling a bit stupid reading this suggestion as I am not sure how it
> is supposed to work :(

Hi Conor, all,

just to clarify, my suggestion was at a slightly higher level regarding
the arrangement of logic statements:

	if (a)
		if (b)

	vs

	if (a && b)

I think your concerns are deeper and, in my reading of them, ought
to be addressed.

> Firstly, why macb_is_gem() and hw_is_gem()? They both do the same thing,
> except last time around we established that macb_is_gem() cannot return
> anything other than false at this point.
> What have I missed here?
> 
> Secondly, is it guaranteed that macb_config::max_tx_length is even
> set?
> 
> Also, another question...
> Is it even possible for `if (macb_config)` to be false?
> Isn't it either going to be set to &default_gem_config or to
> match->data, no? The driver is pretty inconsistent about if it checks
> whether macb_config is non-NULL before accessing it, but from reading
> .probe, it seems to be like it is always set to something valid at this
> point.
> 
> (btw Daire, Nicolas' email has no h in it)
> 
> Cheers,
> Conor.

Re: [PATCH v2 1/1] net: macb: Shorten max_tx_len to 4KiB - 56 on mpfs

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 3444 bytes --]

Hey Simon,

On Fri, Apr 21, 2023 at 04:05:19PM +0200, Simon Horman wrote:
> On Thu, Apr 20, 2023 at 08:18:35AM +0100, Conor Dooley wrote:
> > Jaukb, Simon,
> > 
> > On Wed, Apr 19, 2023 at 06:02:22PM -0700, Jakub Kicinski wrote:
> > > On Tue, 18 Apr 2023 16:28:25 +0200 Simon Horman wrote:
> > 
> > [readding the context]
> > 
> > > > > static const struct macb_config sama7g5_gem_config = {
> > > > > @@ -4986,8 +4985,17 @@ static int macb_probe(struct platform_device *pdev)
> > > > >       bp->tx_clk = tx_clk;
> > > > >       bp->rx_clk = rx_clk;
> > > > >       bp->tsu_clk = tsu_clk;
> > > > > -     if (macb_config)
> > > > > +     if (macb_config) {
> > > > > +             if (hw_is_gem(bp->regs, bp->native_io)) {
> > > > > +                     if (macb_config->max_tx_length)
> > > > > +                             bp->max_tx_length = macb_config->max_tx_length;
> > > > > +                     else
> > > > > +                             bp->max_tx_length = GEM_MAX_TX_LEN;
> > > > > +             } else {
> > > > > +                     bp->max_tx_length = MACB_MAX_TX_LEN;
> > > > > +             }
> > 
> > > > no need to refresh the patch on my account.
> > > > But can the above be simplified as:
> > > > 
> > > >                if (macb_is_gem(bp) && hw_is_gem(bp->regs, bp->native_io))
> > > >                        bp->max_tx_length = macb_config->max_tx_length;
> > > >                else
> > > >                        bp->max_tx_length = MACB_MAX_TX_LEN;
> > > 
> > > I suspect that DaveM agreed, because patch is set to Changes Requested
> > > in patchwork :) 
> > > 
> > > Daire, please respin with Simon's suggestion.
> > 
> > I'm feeling a bit stupid reading this suggestion as I am not sure how it
> > is supposed to work :(

> just to clarify, my suggestion was at a slightly higher level regarding
> the arrangement of logic statements:
> 
> 	if (a)
> 		if (b)
> 
> 	vs
> 
> 	if (a && b)

Ah, I do at least feel less stupid now!
There are 3 possible conditions though, you'd be left with something
like:
	if !hw_is_gem()
	else if macb_config->max_tx_length
	else
> 
> I think your concerns are deeper and, in my reading of them, ought
> to be addressed.
> 
> > Firstly, why macb_is_gem() and hw_is_gem()? They both do the same thing,
> > except last time around we established that macb_is_gem() cannot return
> > anything other than false at this point.
> > What have I missed here?
> > 
> > Secondly, is it guaranteed that macb_config::max_tx_length is even
> > set?

These two were concerns about your suggestion, so they can now be
disregarded as you'd not been seriously suggesting that particular
if (false && hw_is_gem()) test ;)

> > Also, another question...
> > Is it even possible for `if (macb_config)` to be false?
> > Isn't it either going to be set to &default_gem_config or to
> > match->data, no? The driver is pretty inconsistent about if it checks
> > whether macb_config is non-NULL before accessing it, but from reading
> > .probe, it seems to be like it is always set to something valid at this
> > point.

This one though is more of a question for the drivers's maintainers -
Daire's only gone and copied what's done about 4 lines above the top of
the diff. Removing useless NULL checks, assuming they are useless, is
surely out of scope for sorting out this erratum though, no?

Cheers,
Conor.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v2 1/1] net: macb: Shorten max_tx_len to 4KiB - 56 on mpfs

[Simon Horman]

On Fri, Apr 21, 2023 at 05:39:52PM +0100, Conor Dooley wrote:
> Hey Simon,
> 
> On Fri, Apr 21, 2023 at 04:05:19PM +0200, Simon Horman wrote:
> > On Thu, Apr 20, 2023 at 08:18:35AM +0100, Conor Dooley wrote:
> > > Jaukb, Simon,
> > > 
> > > On Wed, Apr 19, 2023 at 06:02:22PM -0700, Jakub Kicinski wrote:
> > > > On Tue, 18 Apr 2023 16:28:25 +0200 Simon Horman wrote:
> > > 
> > > [readding the context]
> > > 
> > > > > > static const struct macb_config sama7g5_gem_config = {
> > > > > > @@ -4986,8 +4985,17 @@ static int macb_probe(struct platform_device *pdev)
> > > > > >       bp->tx_clk = tx_clk;
> > > > > >       bp->rx_clk = rx_clk;
> > > > > >       bp->tsu_clk = tsu_clk;
> > > > > > -     if (macb_config)
> > > > > > +     if (macb_config) {
> > > > > > +             if (hw_is_gem(bp->regs, bp->native_io)) {
> > > > > > +                     if (macb_config->max_tx_length)
> > > > > > +                             bp->max_tx_length = macb_config->max_tx_length;
> > > > > > +                     else
> > > > > > +                             bp->max_tx_length = GEM_MAX_TX_LEN;
> > > > > > +             } else {
> > > > > > +                     bp->max_tx_length = MACB_MAX_TX_LEN;
> > > > > > +             }
> > > 
> > > > > no need to refresh the patch on my account.
> > > > > But can the above be simplified as:
> > > > > 
> > > > >                if (macb_is_gem(bp) && hw_is_gem(bp->regs, bp->native_io))
> > > > >                        bp->max_tx_length = macb_config->max_tx_length;
> > > > >                else
> > > > >                        bp->max_tx_length = MACB_MAX_TX_LEN;
> > > > 
> > > > I suspect that DaveM agreed, because patch is set to Changes Requested
> > > > in patchwork :) 
> > > > 
> > > > Daire, please respin with Simon's suggestion.
> > > 
> > > I'm feeling a bit stupid reading this suggestion as I am not sure how it
> > > is supposed to work :(
> 
> > just to clarify, my suggestion was at a slightly higher level regarding
> > the arrangement of logic statements:
> > 
> > 	if (a)
> > 		if (b)
> > 
> > 	vs
> > 
> > 	if (a && b)
> 
> Ah, I do at least feel less stupid now!
> There are 3 possible conditions though, you'd be left with something
> like:
> 	if !hw_is_gem()
> 	else if macb_config->max_tx_length
> 	else
> > 
> > I think your concerns are deeper and, in my reading of them, ought
> > to be addressed.
> > 
> > > Firstly, why macb_is_gem() and hw_is_gem()? They both do the same thing,
> > > except last time around we established that macb_is_gem() cannot return
> > > anything other than false at this point.
> > > What have I missed here?
> > > 
> > > Secondly, is it guaranteed that macb_config::max_tx_length is even
> > > set?
> 
> These two were concerns about your suggestion, so they can now be
> disregarded as you'd not been seriously suggesting that particular
> if (false && hw_is_gem()) test ;)

Yes, that's right. I would not have made the suggestion had I known that :)

> > > Also, another question...
> > > Is it even possible for `if (macb_config)` to be false?
> > > Isn't it either going to be set to &default_gem_config or to
> > > match->data, no? The driver is pretty inconsistent about if it checks
> > > whether macb_config is non-NULL before accessing it, but from reading
> > > .probe, it seems to be like it is always set to something valid at this
> > > point.
> 
> This one though is more of a question for the drivers's maintainers -
> Daire's only gone and copied what's done about 4 lines above the top of
> the diff. Removing useless NULL checks, assuming they are useless, is
> surely out of scope for sorting out this erratum though, no?

FWIIW, I would say that a patch to address an erratum should only
address the erratum.