[PATCH] arm64: dts: rockchip: set TX PBL for rk3328-roc-cc gmac

[PATCH] arm64: dts: rockchip: set TX PBL for rk3328-roc-cc gmac

[Leonidas P. Papadakos]

Some rockchip boards have issues with tx checksumming for packets
larger than 1498 B.

An iperf test will tell you that transmissions stop almost immediately.

A workaround to this issue is disabling TX checksumming offload,
and one of the ways to do that is snps,force_thresh_dma_mode.

But after some discussion here it seems there is another path:
https://lkml.org/lkml/2019/4/5/148

Setting txpbl is a better-performing (maybe temporary) solution
compared to disabling T offload.

In my testing with a Libre Computer Renegade, this was the best
all-around option, and increasing the MTU doesn't cause a queue timeout anymore.

Signed-off-by: Leonidas P. Papadakos <papadakospan@gmail.com>
---
 arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts b/arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts
index 5d499c908..974f2dcdf 100644
--- a/arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts
@@ -141,10 +141,10 @@
 	phy-mode = "rgmii";
 	pinctrl-names = "default";
 	pinctrl-0 = <&rgmiim1_pins>;
-	snps,force_thresh_dma_mode;
 	snps,reset-gpio = <&gpio1 RK_PC2 GPIO_ACTIVE_LOW>;
 	snps,reset-active-low;
 	snps,reset-delays-us = <0 10000 50000>;
+	snps,txpbl = <0x4>;
 	tx_delay = <0x24>;
 	rx_delay = <0x18>;
 	status = "okay";
-- 
2.21.0

Re: [PATCH] arm64: dts: rockchip: set TX PBL for rk3328-roc-cc gmac

[Peter Geis]

Thanks for sharing that thread, it's nice to finally have an answer to 
why we have these issues.
I've tested this on *my* Firefly rk3328-roc-cc, and while this patch 
succeeds in preventing the tx lockup, iperf3 drops to a delightful 15mbps.

It would seem we have significant differences between manufacturing runs 
of the various boards.
The auto adjustment feature that was discussed in that thread is 
probably the only thing that will solve this for all boards.
Until then I don't think we are going to get a solution that works for all.

On Wed, Apr 17, 2019 at 5:01 PM Leonidas P. Papadakos 
<papadakospan-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
> Some rockchip boards have issues with tx checksumming for packets
> larger than 1498 B.
>
> An iperf test will tell you that transmissions stop almost immediately.
>
> A workaround to this issue is disabling TX checksumming offload,
> and one of the ways to do that is snps,force_thresh_dma_mode.
>
> But after some discussion here it seems there is another path:
> https://lkml.org/lkml/2019/4/5/148
>
> Setting txpbl is a better-performing (maybe temporary) solution
> compared to disabling T offload.
>
> In my testing with a Libre Computer Renegade, this was the best
> all-around option, and increasing the MTU doesn't cause a queue timeout anymore.
>
> Signed-off-by: Leonidas P. Papadakos <papadakospan-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> ---
>  arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts b/arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts
> index 5d499c908..974f2dcdf 100644
> --- a/arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts
> +++ b/arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts
> @@ -141,10 +141,10 @@
>         phy-mode = "rgmii";
>         pinctrl-names = "default";
>         pinctrl-0 = <&rgmiim1_pins>;
> -       snps,force_thresh_dma_mode;
>         snps,reset-gpio = <&gpio1 RK_PC2 GPIO_ACTIVE_LOW>;
>         snps,reset-active-low;
>         snps,reset-delays-us = <0 10000 50000>;
> +       snps,txpbl = <0x4>;
>         tx_delay = <0x24>;
>         rx_delay = <0x18>;
>         status = "okay";
> --
> 2.21.0
>
>
> _______________________________________________
> Linux-rockchip mailing list
> Linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
> http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH] arm64: dts: rockchip: set TX PBL for rk3328-roc-cc gmac

[Leonidas P. Papadakos]

[-- Attachment #1: Type: text/plain, Size: 1028 bytes --]



Στις Κυρ, 21 Απρ, 2019 at 7:00 ΜΜ, ο/η Peter Geis 
<pgwipeout@gmail.com> έγραψε:
> Thanks for sharing that thread, it's nice to finally have an answer 
> to why we have these issues.
> I've tested this on *my* Firefly rk3328-roc-cc, and while this patch 
> succeeds in preventing the tx lockup, iperf3 drops to a delightful 
> 15mbps.
> 
> It would seem we have significant differences between manufacturing 
> runs of the various boards.
> The auto adjustment feature that was discussed in that thread is 
> probably the only thing that will solve this for all boards.
> Until then I don't think we are going to get a solution that works 
> for all.

Indeed that feature would be optimal. Just to confirm, you have 
replaced force_thresh_dma_mode with this txpbl thing, right?
That is the only ethernet modification I have in the dts, done though 
u-boot. That is on linux 5.1

> fdt rm /ethernet@ff540000 snps,force_thresh_dma_mode
> fdt set /ethernet@ff540000 snps,txpbl <0x4>


[-- Attachment #2: Type: text/html, Size: 1559 bytes --]

Re: [PATCH] arm64: dts: rockchip: set TX PBL for rk3328-roc-cc gmac

[Leonidas P. Papadakos]

Indeed that feature would be optimal. Just to confirm, you have 
replaced force_thresh_dma_mode with this txpbl thing, right?
That is the only ethernet modification I have in the dts, done though 
u-boot. That is on linux 5.1

> fdt rm /ethernet@ff540000 snps,force_thresh_dma_mode
> fdt set /ethernet@ff540000 snps,txpbl <0x4>

Re: [PATCH] arm64: dts: rockchip: set TX PBL for rk3328-roc-cc gmac

[Peter Geis]

Yes, I ensured force_thresh_dma_mode was disabled.
I submitted the force_thresh_dma_mode patch so that we could at least 
get functional tx.

I'll mess about with tweaking that value to see if I can get anything 
better.

On 4/22/2019 2:43 AM, Leonidas P. Papadakos wrote:
> 
> Indeed that feature would be optimal. Just to confirm, you have replaced 
> force_thresh_dma_mode with this txpbl thing, right?
> That is the only ethernet modification I have in the dts, done though 
> u-boot. That is on linux 5.1
> 
>> fdt rm /ethernet@ff540000 snps,force_thresh_dma_mode
>> fdt set /ethernet@ff540000 snps,txpbl <0x4>
> 
> 

Re: [PATCH] arm64: dts: rockchip: set TX PBL for rk3328-roc-cc gmac

[Leonidas P. Papadakos]

Indeed, at least with that first patch users won't have a crappy 
experience by default (ssh lags and the like)

Re: [PATCH] arm64: dts: rockchip: set TX PBL for rk3328-roc-cc gmac

[Peter Geis]

On 4/22/2019 9:13 AM, Leonidas P. Papadakos wrote:
> 
> Indeed, at least with that first patch users won't have a crappy 
> experience by default (ssh lags and the like)
> 
> 

I apologize for taking so long at this, but I wanted to make sure I had 
it right.

So far I've found a few things that seem to help, and would like y'all 
to test them when you get the chance.

Adding snps,aal (configures DMA for address alignment) appears to make a 
massive improvement to stability, although the the snps,txpbl = <0x4> is 
still necessary for this to work.

Second, I added snps,rxpbl = <0x4> as well, which seems to help with my 
RX issues a bit.
The biggest improvement with these two settings is more stable transmit 
and receive.
Currently I'm getting about 800mbps on tx and 400 mbps on rx.

Please test when you get the chance and provide feedback.

Re: [PATCH] arm64: dts: rockchip: set TX PBL for rk3328-roc-cc gmac

[Heiko Stuebner]

Hi,

Am Donnerstag, 23. Mai 2019, 02:00:27 CEST schrieb Peter Geis:
> 
> On 4/22/2019 9:13 AM, Leonidas P. Papadakos wrote:
> > 
> > Indeed, at least with that first patch users won't have a crappy 
> > experience by default (ssh lags and the like)
> > 
> > 
> 
> I apologize for taking so long at this, but I wanted to make sure I had 
> it right.
> 
> So far I've found a few things that seem to help, and would like y'all 
> to test them when you get the chance.
> 
> Adding snps,aal (configures DMA for address alignment) appears to make a 
> massive improvement to stability, although the the snps,txpbl = <0x4> is 
> still necessary for this to work.
> 
> Second, I added snps,rxpbl = <0x4> as well, which seems to help with my 
> RX issues a bit.
> The biggest improvement with these two settings is more stable transmit 
> and receive.
> Currently I'm getting about 800mbps on tx and 400 mbps on rx.
> 
> Please test when you get the chance and provide feedback.

I have to confess I lost track of the multitude of patches touching
the gmac on rk3328 and their long mail threads in terms of which
to pick up and which are considered superseeded.

So I'd really appreciate if you could resend the patches that matter
and provide relevant Tested-by, Reviewed-by, etc tags from involved
people that actually were able to test patches

Thanks
Heiko

Re: [PATCH] arm64: dts: rockchip: set TX PBL for rk3328-roc-cc gmac

[Peter Geis]

On 6/14/2019 5:26 AM, Heiko Stuebner wrote:
> Hi,
> 
> Am Donnerstag, 23. Mai 2019, 02:00:27 CEST schrieb Peter Geis:
>>
>> On 4/22/2019 9:13 AM, Leonidas P. Papadakos wrote:
>>>
>>> Indeed, at least with that first patch users won't have a crappy
>>> experience by default (ssh lags and the like)
>>>
>>>
>>
>> I apologize for taking so long at this, but I wanted to make sure I had
>> it right.
>>
>> So far I've found a few things that seem to help, and would like y'all
>> to test them when you get the chance.
>>
>> Adding snps,aal (configures DMA for address alignment) appears to make a
>> massive improvement to stability, although the the snps,txpbl = <0x4> is
>> still necessary for this to work.
>>
>> Second, I added snps,rxpbl = <0x4> as well, which seems to help with my
>> RX issues a bit.
>> The biggest improvement with these two settings is more stable transmit
>> and receive.
>> Currently I'm getting about 800mbps on tx and 400 mbps on rx.
>>
>> Please test when you get the chance and provide feedback.
> 
> I have to confess I lost track of the multitude of patches touching
> the gmac on rk3328 and their long mail threads in terms of which
> to pick up and which are considered superseeded.
> 
> So I'd really appreciate if you could resend the patches that matter
> and provide relevant Tested-by, Reviewed-by, etc tags from involved
> people that actually were able to test patches
> 
> Thanks
> Heiko
> 
> 

Good Morning Heiko,

Here's a quick catch-up for you:
Currently we use thresh dma mode to allow rgmii to work, but this has 
performance limitations.
Leonidas submitted the txpbl 0x4 patch, which works well for his board 
but my board has very poor performance with.
I submitted a new patch that combined txpbl 0x4 with aal dma mode, which 
fixes my issue and seems to work well for Leonidas.

That patch is under [PATCH] arm64: dts: rockchip: improve rk3328-roc-cc 
rgmii performance.
I'd like more people to test it prior to submitting it to replace thresh 
dma mode.

If we get a few more tests without regression, I'll resubmit it with 
Tested-by tags for inclusion.

Thanks,
Peter