Re: Ethernet issue on imx6

[Miquel Raynal <miquel.raynal@bootlin.com>]

Hi Alexander,

alexander.stein@ew.tq-group.com wrote on Mon, 16 Oct 2023 16:41:50
+0200:

> Hi Miquel,
> 
> Am Montag, 16. Oktober 2023, 15:31:54 CEST schrieb Miquel Raynal:
> > Hi Alexander,
> > 
> > Thanks a lot for your feedback.
> >   
> > > > switch to partitions #0, OK
> > > > mmc1 is current device
> > > > reading boot.scr
> > > > 444 bytes read in 10 ms (43 KiB/s)
> > > > ## Executing script at 20000000
> > > > Booting from mmc ...
> > > > reading zImage
> > > > 9160016 bytes read in 462 ms (18.9 MiB/s)
> > > > reading <board>.dtb  
> > > 
> > > Which device tree is that?
> > >   
> > > > 40052 bytes read in 22 ms (1.7 MiB/s)
> > > > boot device tree kernel ...
> > > > Kernel image @ 0x12000000 [ 0x000000 - 0x8bc550 ]
> > > > ## Flattened Device Tree blob at 18000000
> > > > 
> > > >    Booting using the fdt blob at 0x18000000
> > > >    Using Device Tree in place at 18000000, end 1800cc73
> > > > 
> > > > Starting kernel ...
> > > > 
> > > > [    0.000000] Booting Linux on physical CPU 0x0
> > > > [    0.000000] Linux version 6.5.0 (mraynal@xps-13)
> > > > (arm-linux-gcc.br_real
> > > > (Buildroot 2 020.08-14-ge5a2a90) 10.2.0, GNU ld (GNU Binutils) 2.34)
> > > > #120
> > > > SMP Thu Oct 12 18:10:20 CE ST 2023
> > > > [    0.000000] CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7),
> > > > cr=10c5387d [    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT
> > > > aliasing instruction cache
> > > > [    0.000000] OF: fdt: Machine model: TQ TQMa6Q
> > > > on MBa6x  
> > > 
> > > Your first mail mentions a custom board, but this indicates "TQMa6Q
> > > on MBa6x", so which is it?  
> > 
> > It's a custom carrier board with a TQMA6Q-AA module.  
> 
> Could you please adjust the machine model to your mainboard if it is not a 
> MBa6x? Thanks.
> Which HW revision is this module? It should be printed in u-boot during start. 
> Can you provide a full log?

The full kernel log is at the bottom of this e-mail:
https://lore.kernel.org/netdev/20231013102718.6b3a2dfe@xps-13/

On the module I read on a white sticker:
	TQMA6Q-AA
	RK.0203
And on one side of the PCB:
	TQMa6x.0201

Do you know if this module has the hardware workaround discussed below?
(I don't have the schematics of the module)

Here is also the U-Boot log:

U-Boot 2017.11 (Aug 11 2023 - 19:35:47 +0200)

CPU:   Freescale i.MX6Q rev1.5 at 792 MHz
Reset cause: POR
Board: TQMa6Q on a MBa6x
I2C:   ready
DRAM:  1 GiB
PMIC: PFUZE100 ID=0x10 REV=0x21
MMC:   FSL_SDHC: 0, FSL_SDHC: 1
reading uboot.env
In:    serial
Out:   serial
Err:   serial
Net:   FEC [PRIME]
Warning: FEC MAC addresses don't match:
Address in SROM is         00:d0:93:44:a4:c0
Address in environment is  fc:c2:3d:18:5f:91

starting USB...
USB0:   Port not available.
USB1:   USB EHCI 1.00
scanning bus 1 for devices... 3 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
       scanning usb for ethernet devices... 1 Ethernet Device(s) found
Hit any key to stop autoboot:  0 
switch to partitions #0, OK
mmc1 is current device
reading boot.scr
444 bytes read in 10 ms (43 KiB/s)
## Executing script at 20000000
Booting from mmc ...
reading zImage
7354128 bytes read in 368 ms (19.1 MiB/s)
reading stephan_Stephanie_ControlUnit_A809_60_408.dtb
40002 bytes read in 25 ms (1.5 MiB/s)
boot device tree kernel ...
Kernel image @ 0x12000000 [ 0x000000 - 0x703710 ]
## Flattened Device Tree blob at 18000000
   Booting using the fdt blob at 0x18000000
   Using Device Tree in place at 18000000, end 1800cc41

Starting kernel ...

> > > Please note that there are two different module variants,
> > > imx6qdl-tqma6a.dtsi and imx6qdl-tqma6b.dtsi. They deal with i.MX6's
> > > ERR006687 differently. Package drop without any load somewhat indicates
> > > this issue.  
> > 
> > I've tried with and without the fsl,err006687-workaround-present DT
> > property. It gets successfully parsed an I see the lower idle state
> > being disabled under mach-imx. I've also tried just commenting out the
> > registration of the cpuidle driver, just to be sure. I saw no
> > difference.  
> 
> fsl,err006687-workaround-present requires a specific HW workaround, see [1]. 
> So this is not applicable on every module.

Based on the information provided above, do you think I can rely on the
HW workaround?

I've tried disabling the registration of both the CPUidle and CPUfreq
drivers in the machine code and I see a real difference. The transfers
are still not perfect though, but I believe this is related to the ~1%
drop of the RGMII lines (timings are not perfect, but I could not
extend them more).

I believe if the hardware workaround is not available on this module I
can still disable CPUidle and CPUfreq as a workaround of the
workaround...?

> > By the way, we tried with a TQ eval board with this SoM and saw the same
> > issue (not me, I don't have this board in hands). Don't you experience
> > something similar? I went across a couple of people reporting similar
> > issues with these modules but none of them reported how they fixed it
> > (if they did). I tried two different images based on TQ's Github using
> > v4.14.69 and v5.10 kernels.  
> 
> Personally I've heard the first time about this issue. I never noticed 
> something like this. Does this issue also appear when using TCP? Or is it an 
> UDP only issue?

With a mainline kernel:
* With UDP I get a high drop rate.
* With TCP I get slow/bumpy throughputs.

> [1] https://github.com/tq-systems/linux-tqmaxx/blob/TQMa8-fslc-5.10-2.1.x-imx/
> arch/arm/boot/dts/imx6qdl-tqma6a.dtsi#L36-L48

Thanks,
Miquèl