From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BABDC61DA4 for ; Sat, 18 Feb 2023 13:21:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229821AbjBRNVT convert rfc822-to-8bit (ORCPT ); Sat, 18 Feb 2023 08:21:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45296 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229820AbjBRNVI (ORCPT ); Sat, 18 Feb 2023 08:21:08 -0500 Received: from lithops.sigma-star.at (lithops.sigma-star.at [195.201.40.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 03B2D18AB2 for ; Sat, 18 Feb 2023 05:20:57 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by lithops.sigma-star.at (Postfix) with ESMTP id 5FD01605DED8; Sat, 18 Feb 2023 14:20:54 +0100 (CET) Received: from lithops.sigma-star.at ([127.0.0.1]) by localhost (lithops.sigma-star.at [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id Dozcj5lHcvDY; Sat, 18 Feb 2023 14:20:53 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by lithops.sigma-star.at (Postfix) with ESMTP id DC3D4622624B; Sat, 18 Feb 2023 14:20:53 +0100 (CET) Received: from lithops.sigma-star.at ([127.0.0.1]) by localhost (lithops.sigma-star.at [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id zHeRw7gcFnck; Sat, 18 Feb 2023 14:20:53 +0100 (CET) Received: from lithops.sigma-star.at (lithops.sigma-star.at [195.201.40.130]) by lithops.sigma-star.at (Postfix) with ESMTP id BB4BB605DED8; Sat, 18 Feb 2023 14:20:53 +0100 (CET) Date: Sat, 18 Feb 2023 14:20:53 +0100 (CET) From: Richard Weinberger To: wei fang Cc: Andrew Lunn , David Laight , netdev , shenwei wang , xiaoning wang , linux-imx Message-ID: <2015643728.147543.1676726453615.JavaMail.zimbra@nod.at> In-Reply-To: References: <1422776754.146013.1676652774408.JavaMail.zimbra@nod.at> <130183416.146934.1676713353800.JavaMail.zimbra@nod.at> <2030061857.147332.1676721783879.JavaMail.zimbra@nod.at> Subject: Re: high latency with imx8mm compared to imx6q MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8BIT X-Originating-IP: [195.201.40.130] X-Mailer: Zimbra 8.8.12_GA_3807 (ZimbraWebClient - FF97 (Linux)/8.8.12_GA_3809) Thread-Topic: high latency with imx8mm compared to imx6q Thread-Index: KX6nItb3xzzXXgLsKrBE7F/dfPfNJbU33POAgABHTYCAAAZaMMtWSb1ctKpkzJDnhT13z5h6zK7wR4xj2cE= Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org ----- Ursprüngliche Mail ----- > Von: "wei fang" > If you use the ethtool cmd, the minimum can only be set to 1. > But you can set the coalescing registers directly on your console, > ENET_RXICn[ICEN] (addr: base + F0h offset + (4d × n) where n=0,1,2) and > ENET_TXICn[ICEN] (addr: base + 100h offset + (4d × n), where n=0d to 2d) > set the ICEN bit (bit 31) to 0: > 0 disable Interrupt coalescing. > 1 disable Interrupt coalescing. > or modify you fec driver, but remember, the interrupt coalescing feature > can only be disable by setting the ICEN bit to 0, do not set the tx/rx > usecs/frames > to 0. Disabling interrupt coalescing seems to make things much better. :-) >> >> >> >> But I noticed something interesting this morning. When I set >> >> rx-usecs, tx-usecs, rx-frames and tx-frames to 1, *sometimes* the RTT is >> good. >> >> >> >> PING 192.168.0.52 (192.168.0.52) 56(84) bytes of data. >> >> 64 bytes from 192.168.0.52: icmp_seq=1 ttl=64 time=0.730 ms >> >> 64 bytes from 192.168.0.52: icmp_seq=2 ttl=64 time=0.356 ms >> >> 64 bytes from 192.168.0.52: icmp_seq=3 ttl=64 time=0.303 ms >> >> 64 bytes from 192.168.0.52: icmp_seq=4 ttl=64 time=2.22 ms >> >> 64 bytes from 192.168.0.52: icmp_seq=5 ttl=64 time=2.54 ms >> >> 64 bytes from 192.168.0.52: icmp_seq=6 ttl=64 time=0.354 ms >> >> 64 bytes from 192.168.0.52: icmp_seq=7 ttl=64 time=2.22 ms >> >> 64 bytes from 192.168.0.52: icmp_seq=8 ttl=64 time=2.54 ms >> >> 64 bytes from 192.168.0.52: icmp_seq=9 ttl=64 time=2.53 ms >> >> >> >> So coalescing plays a role but it looks like the ethernet controller >> >> does not always obey my settings. >> >> I didn't look into the configured registers so far, maybe ethtool >> >> does not set them correctly. >> >> >> > It look a bit weird. I did the same setting with my i.MX8ULP and >> > didn't have this issue. I'm not sure whether you network is stable or >> > network node devices also enable interrupt coalescing and the relevant >> > parameters are set to a bit high. >> >> I'm pretty sure my network is good, I've tested also different locations. >> And as I said, with the imx6q on the very same network everything works as >> expected. >> >> So, with rx-usecs/rx-frames/tx-usecs/tx-frames set to 1, you see a RTT smaller >> than 1ms? >> > Yes, but my platform is i.MX8ULP not i.MX8MM, I'll check i.MX8MM next Monday. Now I don't see the outlines anymore. Maybe the test from before was really wonky. :-S Next week I'll do a bigger test on the testbed with interrupt coalescing disabled at driver level. Thanks a lot for all the great input so far! //richard