From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B79B0F419A2 for ; Wed, 15 Apr 2026 12:44:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To: Content-Transfer-Encoding:Content-Type:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=UyQKCDTxnGTJw71Mu2rprj6cyd6GgdGyAa8DuKAW8c4=; b=43Sz8BCERINwEWUuUO1lJB6BCF 9sCSO77Ex+pdW0wDz52oH+u8l+5Xpf5eNWRWnnMmXV+FSTd+nmxPF40OmaOixERkf2YOcHKMGkRBH I9bWGtxOkaev9JKVWXZoQd7HuXB65VvJoRKkHfG4dMtvYD72XrqIKNPShFLFzEJ7wq6SdgFbHJ6Hx GoF93CVegJ+SWNXUSyjYP4nIGolUfE2dv8fBXObMvbpLm8pcaYH4Tp8cFKWImupq3tUgZmRAKLYzS 5QSU2y7TU7kH3eqzV5tiEKUZbJPWSebu1MqhRMrUeHaH2mtC3qc1Rr1fZtLwC/m25vgXKe4Xci7PS kTcqk29w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1wCzbK-000000018p4-1Zic; Wed, 15 Apr 2026 12:44:14 +0000 Received: from pandora.armlinux.org.uk ([2001:4d48:ad52:32c8:5054:ff:fe00:142]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1wCzbH-000000018ok-2NHw for linux-arm-kernel@lists.infradead.org; Wed, 15 Apr 2026 12:44:13 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=armlinux.org.uk; s=pandora-2019; h=Sender:In-Reply-To: Content-Transfer-Encoding:Content-Type:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=UyQKCDTxnGTJw71Mu2rprj6cyd6GgdGyAa8DuKAW8c4=; b=Cswnx1KXs9lvtqeyPXB2GfbbT+ 3gXcHhOdEPK2RGQNkTR97u+pDvuccO8W39CMIXcn83fgqWIwZsUcJCsev36tvt4tjFScphSivRC+n tigbE7SrhL54KMcqh2FLg6EFvK1oQo189TZ8zw66E3PrluqxvTdg1LEx1YIQXbwq2r/j6qw8gKdwT MwkfaRT3TjbM4SSls8cXz6s1GhSD3uxsCnv9v1YT0DC+fU9sYYPYAvRWHfmCKANjvdvpoyuWh9BLw 5Zt6Pd8gpcqiQUTwdWp0vB2oxEqDEP9d2aDPH88PVMaM0PipMEspk0VWKEfeq1XW0vdkJoU05a9Pl Y/k36gEg==; Received: from shell.armlinux.org.uk ([fd8f:7570:feb6:1:5054:ff:fe00:4ec]:38004) by pandora.armlinux.org.uk with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1wCzb9-0000000025E-0kHQ; Wed, 15 Apr 2026 13:44:03 +0100 Received: from linux by shell.armlinux.org.uk with local (Exim 4.98.2) (envelope-from ) id 1wCzb5-000000002E2-1vhk; Wed, 15 Apr 2026 13:43:59 +0100 Date: Wed, 15 Apr 2026 13:43:59 +0100 From: "Russell King (Oracle)" To: Sam Edwards Cc: Jakub Kicinski , Andrew Lunn , Alexandre Torgue , Andrew Lunn , "David S. Miller" , Eric Dumazet , "moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE" , linux-stm32@st-md-mailman.stormreply.com, Linux Network Development Mailing List , Paolo Abeni Subject: Re: [PATCH net-next] net: stmmac: enable RPS and RBU interrupts Message-ID: References: <20260413110222.49fc3759@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260415_054411_630578_D4A10590 X-CRM114-Status: GOOD ( 28.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Apr 14, 2026 at 07:12:34PM -0700, Sam Edwards wrote: > On Tue, Apr 14, 2026 at 6:19 PM Russell King (Oracle) > wrote: > > Okay, just a quick note to say that nvidia's 5.10.216-tegra kernel > > survives iperf3 -c -R to the imx6. > > Hi Russell, > > Aw, you beat me to it! I was about to report that 5.10.104-tegra is > unaffected. And my iperf3 server is a multi-GbE amd64 machine. > > > Dumping the registers and comparing, and then forcing the RQS and TQS > > values to 0x23 (+1 = 36, *256 = 9216 bytes) and 0x8f (+1 = 144, > > *256 = 36864 ytes) respectively seems to solve the problem. Under > > net-next, these both end up being 0xff (+1 = 256, *256 = 65536 bytes.) > > Suspiciously, 36 * 4 = 144, and I also see that this kernel programs > > all four of the MTL receive operation mode registers, but only the > > first MTL transmit operation mode register. However, DMA channels 1-3 > > aren't initialised. > > Wow, great! I wonder if the problem is that the MTL FIFOs are smaller > than that, so when the DMA suffers a momentary hiccup, the FIFOs are > allowed to overflow, putting the hardware in a bad state. > > Though I suspect this is only half of the problem: do you still see > RBUs? Everything you've shared so far suggests the DMA failures are > _not_ because the rx ring is drying up. Yes. Note that RBUs will happen not because of DMA failures, but if the kernel fails to keep up with the packet rate. RBU means "we read the next descriptor, and it wasn't owned by hardware". > > Looking back at 5.10, I don't see any code that would account for these > > values being programmed for TQS and RQS, it looks like the calculations > > are basically the same as we have today. > > Note that Nvidia have their own "nvethernet" driver for their vendor > kernel, which appears to pick the FIFO sizes from hardcoded tables in > its eqos_configure_mtl_queue() [1] function. That has: const nveu32_t rx_fifo_sz[2U][OSI_EQOS_MAX_NUM_QUEUES] = { { FIFO_SZ(9U), FIFO_SZ(9U), FIFO_SZ(9U), FIFO_SZ(9U), FIFO_SZ(1U), FIFO_SZ(1U), FIFO_SZ(1U), FIFO_SZ(1U) }, { FIFO_SZ(36U), FIFO_SZ(2U), FIFO_SZ(2U), FIFO_SZ(2U), FIFO_SZ(2U), FIFO_SZ(2U), FIFO_SZ(2U), FIFO_SZ(16U) }, }; const nveu32_t tx_fifo_sz[2U][OSI_EQOS_MAX_NUM_QUEUES] = { { FIFO_SZ(9U), FIFO_SZ(9U), FIFO_SZ(9U), FIFO_SZ(9U), FIFO_SZ(1U), FIFO_SZ(1U), FIFO_SZ(1U), FIFO_SZ(1U) }, { FIFO_SZ(8U), FIFO_SZ(8U), FIFO_SZ(8U), FIFO_SZ(8U), FIFO_SZ(8U), FIFO_SZ(8U), FIFO_SZ(8U), FIFO_SZ(8U) }, }; where each of those values is the RQS/TQS value to use in KiB: #define FIFO_SZ(x) ((((x) * 1024U) / 256U) - 1U) This doesn't correspond with the values I'm seeing programmed into the hardware under the 5.10.216-tegra kernel. I'm seeing TQS = 143 (36KiB), and RQS = 35 (9KiB). Yes, these values exist in the tables above from a quick look, but they're not in the right place! For example, tx_fifo_sz[] doesn't contain an entry for 36KiB. rx_fifo_sz[0][0..3] looks plausible. It's certainly not a case of misreading the register values, this is what devmem2 said: Value at address 0x02490d00: 0x008f000a Value at address 0x02490d30: 0x02379eb0 where TQS is bits 24:16 of the register at offset 0xd00 - which is 0x8f, and RQS is bits 29:20 of the register at 0xd30, which is 0x23. Now, as for FIFO sizes, if we sum up all the entries, then we get: SUM(rx_fifo_size[0][]) = 60KiB SUM(rx_fifo_size[1][]) = 64KiB SUM(tx_fifo_size[0][]) = 60KiB SUM(tx_fifo_size[1][]) = 64KiB >From what I gather in core_local.h, l_mac_ver contains one of three values - 0 = Legacy EQOS, 1 = Orin EQOS, 2 = Orin MGBE, and which set of values is selected by bit 0 of that. Decoding this further, Legacy EQOS is IP version v5.0, Orin EQOS is v5.3, and Orin MGBE is v3.1 and v4.0. So, I wonder whether there's something in "Legacy EQOS" that consumes 4KiB of FIFO that isn't documented in iMX8M (IP v5.1). Is anyone aware of public SoC documentation that covers the v5.0 IP version? -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!