[PATCH net] net: ethernet: ti: am65-cpsw: default to round-robin for host port receive

[PATCH net] net: ethernet: ti: am65-cpsw: default to round-robin for host port receive

[Siddharth Vadapalli]

The Host Port (i.e. CPU facing port) of CPSW receives traffic from Linux
via TX DMA Channels which are Hardware Queues consisting of traffic
categorized according to their priority. The Host Port is configured to
dequeue traffic from these Hardware Queues on the basis of priority i.e.
as long as traffic exists on a Hardware Queue of a higher priority, the
traffic on Hardware Queues of lower priority isn't dequeued. An alternate
operation is also supported wherein traffic can be dequeued by the Host
Port in a Round-Robin manner.

Until [0], the am65-cpsw driver enabled a single TX DMA Channel, due to
which, unless modified by user via "ethtool", all traffic from Linux is
transmitted on DMA Channel 0. Therefore, configuring the Host Port for
priority based dequeuing or Round-Robin operation is identical since
there is a single DMA Channel.

Since [0], all 8 TX DMA Channels are enabled by default. Additionally,
the default "tc mapping" doesn't take into account the possibility of
different traffic profiles which various users might have. This results
in traffic starvation at the Host Port due to the priority based dequeuing
which has been enabled by default since the inception of the driver. The
traffic starvation triggers NETDEV WATCHDOG timeout for all TX DMA Channels
that haven't been serviced due to the presence of traffic on the higher
priority TX DMA Channels.

Fix this by defaulting to Round-Robin dequeuing at the Host Port, which
shall ensure that traffic is dequeued from all TX DMA Channels irrespective
of the traffic profile. This will address the NETDEV WATCHDOG timeouts.
At the same time, users can still switch from Round-Robin to Priority
based dequeuing at the Host Port with the help of the "p0-rx-ptype-rrobin"
private flag of "ethtool". Users are expected to setup an appropriate
"tc mapping" that suits their traffic profile when switching to priority
based dequeuing at the Host Port.

[0] commit be397ea3473d ("net: ethernet: am65-cpsw: Set default TX channels to maximum")
Fixes: be397ea3473d ("net: ethernet: am65-cpsw: Set default TX channels to maximum")
Cc: <stable@vger.kernel.org>
Signed-off-by: Siddharth Vadapalli <s-vadapalli@ti.com>
---

Hello,

This patch is based on commit
8faabc041a00 Merge tag 'net-6.13-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net
of Mainline Linux.

Regards,
Siddharth.

 drivers/net/ethernet/ti/am65-cpsw-nuss.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.c b/drivers/net/ethernet/ti/am65-cpsw-nuss.c
index 14e1df721f2e..5465bf872734 100644
--- a/drivers/net/ethernet/ti/am65-cpsw-nuss.c
+++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.c
@@ -3551,7 +3551,7 @@ static int am65_cpsw_nuss_probe(struct platform_device *pdev)
 	init_completion(&common->tdown_complete);
 	common->tx_ch_num = AM65_CPSW_DEFAULT_TX_CHNS;
 	common->rx_ch_num_flows = AM65_CPSW_DEFAULT_RX_CHN_FLOWS;
-	common->pf_p0_rx_ptype_rrobin = false;
+	common->pf_p0_rx_ptype_rrobin = true;
 	common->default_vlan = 1;
 
 	common->ports = devm_kcalloc(dev, common->port_num,
-- 
2.43.0

Re: [PATCH net] net: ethernet: ti: am65-cpsw: default to round-robin for host port receive

[Przemek Kitszel]

On 12/20/24 08:56, Siddharth Vadapalli wrote:
> The Host Port (i.e. CPU facing port) of CPSW receives traffic from Linux
> via TX DMA Channels which are Hardware Queues consisting of traffic
> categorized according to their priority. The Host Port is configured to
> dequeue traffic from these Hardware Queues on the basis of priority i.e.
> as long as traffic exists on a Hardware Queue of a higher priority, the
> traffic on Hardware Queues of lower priority isn't dequeued. An alternate
> operation is also supported wherein traffic can be dequeued by the Host
> Port in a Round-Robin manner.
> 
> Until [0], the am65-cpsw driver enabled a single TX DMA Channel, due to
> which, unless modified by user via "ethtool", all traffic from Linux is
> transmitted on DMA Channel 0. Therefore, configuring the Host Port for
> priority based dequeuing or Round-Robin operation is identical since
> there is a single DMA Channel.
> 
> Since [0], all 8 TX DMA Channels are enabled by default. Additionally,
> the default "tc mapping" doesn't take into account the possibility of
> different traffic profiles which various users might have. This results
> in traffic starvation at the Host Port due to the priority based dequeuing
> which has been enabled by default since the inception of the driver. The
> traffic starvation triggers NETDEV WATCHDOG timeout for all TX DMA Channels
> that haven't been serviced due to the presence of traffic on the higher
> priority TX DMA Channels.

I get it right that the starving is caused by HW/DMA (not SW)

> 
> Fix this by defaulting to Round-Robin dequeuing at the Host Port, which
> shall ensure that traffic is dequeued from all TX DMA Channels irrespective
> of the traffic profile. This will address the NETDEV WATCHDOG timeouts.
> At the same time, users can still switch from Round-Robin to Priority
> based dequeuing at the Host Port with the help of the "p0-rx-ptype-rrobin"

why the flag has rx in the name?

> private flag of "ethtool". Users are expected to setup an appropriate
> "tc mapping" that suits their traffic profile when switching to priority
> based dequeuing at the Host Port.
> 
> [0] commit be397ea3473d ("net: ethernet: am65-cpsw: Set default TX channels to maximum")
> Fixes: be397ea3473d ("net: ethernet: am65-cpsw: Set default TX channels to maximum")
> Cc: <stable@vger.kernel.org>
> Signed-off-by: Siddharth Vadapalli <s-vadapalli@ti.com>
> ---
> 
> Hello,
> 
> This patch is based on commit
> 8faabc041a00 Merge tag 'net-6.13-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net
> of Mainline Linux.
> 
> Regards,
> Siddharth.
> 
>   drivers/net/ethernet/ti/am65-cpsw-nuss.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.c b/drivers/net/ethernet/ti/am65-cpsw-nuss.c
> index 14e1df721f2e..5465bf872734 100644
> --- a/drivers/net/ethernet/ti/am65-cpsw-nuss.c
> +++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.c
> @@ -3551,7 +3551,7 @@ static int am65_cpsw_nuss_probe(struct platform_device *pdev)
>   	init_completion(&common->tdown_complete);
>   	common->tx_ch_num = AM65_CPSW_DEFAULT_TX_CHNS;
>   	common->rx_ch_num_flows = AM65_CPSW_DEFAULT_RX_CHN_FLOWS;
> -	common->pf_p0_rx_ptype_rrobin = false;
> +	common->pf_p0_rx_ptype_rrobin = true;
>   	common->default_vlan = 1;
>   
>   	common->ports = devm_kcalloc(dev, common->port_num,

Re: [PATCH net] net: ethernet: ti: am65-cpsw: default to round-robin for host port receive

[Siddharth Vadapalli]

On Fri, Dec 20, 2024 at 11:41:26AM +0100, Przemek Kitszel wrote:
> On 12/20/24 08:56, Siddharth Vadapalli wrote:
> > The Host Port (i.e. CPU facing port) of CPSW receives traffic from Linux
> > via TX DMA Channels which are Hardware Queues consisting of traffic
> > categorized according to their priority. The Host Port is configured to
> > dequeue traffic from these Hardware Queues on the basis of priority i.e.
> > as long as traffic exists on a Hardware Queue of a higher priority, the
> > traffic on Hardware Queues of lower priority isn't dequeued. An alternate
> > operation is also supported wherein traffic can be dequeued by the Host
> > Port in a Round-Robin manner.
> > 
> > Until [0], the am65-cpsw driver enabled a single TX DMA Channel, due to
> > which, unless modified by user via "ethtool", all traffic from Linux is
> > transmitted on DMA Channel 0. Therefore, configuring the Host Port for
> > priority based dequeuing or Round-Robin operation is identical since
> > there is a single DMA Channel.
> > 
> > Since [0], all 8 TX DMA Channels are enabled by default. Additionally,
> > the default "tc mapping" doesn't take into account the possibility of
> > different traffic profiles which various users might have. This results
> > in traffic starvation at the Host Port due to the priority based dequeuing
> > which has been enabled by default since the inception of the driver. The
> > traffic starvation triggers NETDEV WATCHDOG timeout for all TX DMA Channels
> > that haven't been serviced due to the presence of traffic on the higher
> > priority TX DMA Channels.
> 
> I get it right that the starving is caused by HW/DMA (not SW)

Yes. The CPSW Host Port in Hardware can be configured to dequeue packets
queued onto the DMA Channels by Linux in either a Priority-based manner
or Round-Robin manner. Linux is submitting the packets onto the TX DMA
Channels to be received by CPSW, with CPSW's Host Port dequeuing them
in one of the two modes (fixed priority or round-robin) that has been
programmed.

> 
> > 
> > Fix this by defaulting to Round-Robin dequeuing at the Host Port, which
> > shall ensure that traffic is dequeued from all TX DMA Channels irrespective
> > of the traffic profile. This will address the NETDEV WATCHDOG timeouts.
> > At the same time, users can still switch from Round-Robin to Priority
> > based dequeuing at the Host Port with the help of the "p0-rx-ptype-rrobin"
> 
> why the flag has rx in the name?

The "rx" corresponds to "receive" w.r.t. the Host Port (P0). As indicated
in the $subject of this patch, we are defaulting to "Round-Robin" (rrobin)
mode of operation of Host Port dequeuing i.e. packets received by the Host
Port from Linux are to be dequeued in a Round-Robin manner from each of the
TX DMA Channels (TX from Linux's perspective). As for the naming convention
being followed, it is based on the Techincal Reference Manual.

[...]

Regards,
Siddharth.

Re: [PATCH net] net: ethernet: ti: am65-cpsw: default to round-robin for host port receive

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to netdev/net.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Fri, 20 Dec 2024 13:26:14 +0530 you wrote:
> The Host Port (i.e. CPU facing port) of CPSW receives traffic from Linux
> via TX DMA Channels which are Hardware Queues consisting of traffic
> categorized according to their priority. The Host Port is configured to
> dequeue traffic from these Hardware Queues on the basis of priority i.e.
> as long as traffic exists on a Hardware Queue of a higher priority, the
> traffic on Hardware Queues of lower priority isn't dequeued. An alternate
> operation is also supported wherein traffic can be dequeued by the Host
> Port in a Round-Robin manner.
> 
> [...]

Here is the summary with links:
  - [net] net: ethernet: ti: am65-cpsw: default to round-robin for host port receive
    https://git.kernel.org/netdev/net/c/4a4d38ace1fb

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html