From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qt1-f180.google.com (mail-qt1-f180.google.com [209.85.160.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 536795B5AB for ; Thu, 5 Mar 2026 14:33:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.180 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772721220; cv=none; b=j/X6rLfg8y7dgoDFta5BzP2VL79vX3z+3+qcQ/usjIwAiMRfahTlsYoKK6HApEAbOy98dk+/yE8ptsR4d67sU7dRab+teFPt9sp8M7P76OxzyQXeDYFFlEN+IIoDBVl+lGIcn8lsGgJE4A/LD+VnoRXGIsLWEF4fvliExJaSjJ8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772721220; c=relaxed/simple; bh=4eT3D3g80FssQI0tnFYVQ3K8PF/PCF8oq0EsRcjm9Vk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:Cc; b=fa0DtvZabc/yd9mtE3GDsLie42p4z0QSwL6d73HIOemVvt4/UfWQpCPNNdCbh1/1AT5FtrFHmbluLEKdn2ou2H5Fmhjgjcb+DF8Jrel/EN10IbLyLq8CJ/W2dggITCoLSxj4rB+nnZFX6TNeviq1mWBR6D3BrjLz/gnTy+qDCZU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=eCBZ3zRu; arc=none smtp.client-ip=209.85.160.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eCBZ3zRu" Received: by mail-qt1-f180.google.com with SMTP id d75a77b69052e-503347e8715so97090271cf.2 for ; Thu, 05 Mar 2026 06:33:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1772721218; x=1773326018; darn=vger.kernel.org; h=cc:to:message-id:content-transfer-encoding:mime-version:subject :date:from:from:to:cc:subject:date:message-id:reply-to; bh=Ar9RsVFxBZUWUg9Rgu1+EQpRr9zNJUYue1v36pHePqE=; b=eCBZ3zRuBRDAwwjOr9SF6tlg/4ISvqp+ELeKSlP/qhznCz46G+zO8jPNj2Y4bqyEi3 NobWUsmHNo9dE8j1VnrfeRmdf4ibuFA+gpx7X/1b7LoUCiAFDMsxeEEl2lRp1P4k2Wtw mV5OOs9yOx71ysXONl8XjD9caJ1nrygHxKvpRe4iMzkH4jxu5tJmfNSPhFfmZFQGYcDo oqBfNgE32xjTi3my7SmU1J9U0Ec/5pHZX7zlqqv9G7JRBvS0w9183LA0nrJuQ43V9JXP Eg8hirdOZjoK2c/ayBAQfAV3cHhdan5eVoha14ZEgsq2PcEjATm2z7+Oncm5am4xvbUC +CPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772721218; x=1773326018; h=cc:to:message-id:content-transfer-encoding:mime-version:subject :date:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Ar9RsVFxBZUWUg9Rgu1+EQpRr9zNJUYue1v36pHePqE=; b=ijqL9Q8C5pgjjdE3atSLq4Xc10P0LyQWzeR+Kf8/gMYt/cYwKgC4AIHRqDlYCqyrfO Qn+hIIOBCRTI/siQatHe8i/XkPdPvsRFvNJqNEKC/H5pXCDqKz50dkejTqui/JwML3Jn MJKW1ZjYZZnQpVlVhdWf9ExIW2sv1HEosKcxnBdQLl3BtWwpOFJe25YNJ1Fk9K7LkvWs 3qA5Ty1p5F26RBDa1j9wg4/ZXoM9+Y8dIktS7TQfTLS88YpPq+OMoSFmK8Md2Rfn8UG7 mmVP0lDciARowjtuFkoe9QGJ9bEIn9x2hG1uLNcmx1zkyc2JNDl6ohw5f637llp87yDY vzMA== X-Gm-Message-State: AOJu0Ywg1rAZtV7FUulFWQj6Co+n+0Xj4rcv8ka6+UOa/fMsd/FuUYyB FtkA6tAzXZYbrT3Iv2Wopd+VKXfyiZbqDNGGOTjEpCC98zveHyQPEW/BPp1wY9NeKKA= X-Gm-Gg: ATEYQzwsJoHereJV8bm4uPuXTewEle9gL7t8CBfOXtDVZAyuVJQKnjl3BE7xsETwHAi VLiq1xALEvOMmwowj0t/B7TwAHXI4N9JkybN0P/WlpIbGMdDpng4kX3epSgYon2+4kGineehjxk +y/axDsNdy8bCmzTsudTg4G9VKtO4ot2ZftcnXVcc6xB/RI3Ry94jd+Sgcfk1UlFlKJr6hJroEj JM1496xzqmDFjtPOQT2DAu9zKi8eR4QAfa0BXF8STlWYevddDUA4Z/rEstgi3/LMW7Nm+NP23Lf mzgfdvNMDjqJQS6wS3xC1lpbEhmVUq9Xoc4zUMRj/XObkyUO/FLUraN3LOEZ9DsQYnR3yMVTrrm 5rAGSvuxxNx7bXr6zhdUFMSc9B/Tyw3tse6+j3xPI4ilZ9Bq8hkarOuR2L8ojvEGSZFVHa5sdIy pMBykSozIuhJmHYKUqUjrMMj7pTSciUuwpdA0N5Pnxd0E= X-Received: by 2002:ac8:590d:0:b0:506:9d3e:67e3 with SMTP id d75a77b69052e-508db3ad17cmr73909121cf.70.1772721218001; Thu, 05 Mar 2026 06:33:38 -0800 (PST) Received: from localhost.localdomain ([128.224.253.2]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-507512559b2sm155794251cf.17.2026.03.05.06.33.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Mar 2026 06:33:37 -0800 (PST) From: Kevin Hao Date: Thu, 05 Mar 2026 22:32:29 +0800 Subject: [PATCH net] net: macb: Shuffle the tx ring before enabling tx Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260305-zynqmp-v1-1-5de72254d56b@gmail.com> X-B4-Tracking: v=1; b=H4sIAPyTqWkC/6tWKk4tykwtVrJSqFYqSi3LLM7MzwNyDHUUlJIzE vPSU3UzU4B8JSMDIzMDI0MT3arKvMLcAl1zkxRj0zRjCzOjNAsloOKCotS0zAqwQdFKeaklSrG 1tQBN2duEXQAAAA== X-Change-ID: 20260214-zynqmp-74d35f3862f8 To: netdev@vger.kernel.org Cc: Kevin Hao , Quanyang Wang , stable@vger.kernel.org, Nicolas Ferre , Claudiu Beznea , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Russell King X-Mailer: b4 0.14.2 Quanyang observed that when using an NFS rootfs on an AMD ZynqMp board, the rootfs may take an extended time to recover after a suspend. Upon investigation, it was determined that the issue originates from a problem in the macb driver. According to the Zynq UltraScale TRM [1], when transmit is disabled, the transmit buffer queue pointer resets to point to the address specified by the transmit buffer queue base address register. In the current implementation, the code merely resets `queue->tx_head` and `queue->tx_tail` to '0'. This approach presents several issues: - Packets already queued in the tx ring are silently lost, leading to memory leaks since the associated skbs cannot be released. - Concurrent write access to `queue->tx_head` and `queue->tx_tail` may occur from `macb_tx_poll()` or `macb_start_xmit()` when these values are reset to '0'. - The transmission may become stuck on a packet that has already been sent out, with its 'TX_USED' bit set, but has not yet been processed. However, due to the manipulation of 'queue->tx_head' and 'queue->tx_tail', `macb_tx_poll()` incorrectly assumes there are no packets to handle because `queue->tx_head == queue->tx_tail`. This issue is only resolved when a new packet is placed at this position. This is the root cause of the prolonged recovery time observed for the NFS root filesystem. To resolve this issue, shuffle the tx ring and tx skb array so that the first unsent packet is positioned at the start of the tx ring. Additionally, ensure that updates to `queue->tx_head` and `queue->tx_tail` are properly protected with the appropriate lock. [1] https://docs.amd.com/v/u/en-US/ug1085-zynq-ultrascale-trm Fixes: bf9cf80cab81 ("net: macb: Fix tx/rx malfunction after phy link down and up") Reported-by: Quanyang Wang Signed-off-by: Kevin Hao Cc: stable@vger.kernel.org --- Cc: Nicolas Ferre Cc: Claudiu Beznea Cc: Andrew Lunn Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: Russell King --- drivers/net/ethernet/cadence/macb_main.c | 89 ++++++++++++++++++++++++++++++-- 1 file changed, 86 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c index 17f0ad3d7a0924a7dc2fc0a13505aff7d2499ffa..fce144a1830823da9821ad3245784e62fed97e33 100644 --- a/drivers/net/ethernet/cadence/macb_main.c +++ b/drivers/net/ethernet/cadence/macb_main.c @@ -36,6 +36,7 @@ #include #include #include +#include #include #include "macb.h" @@ -684,6 +685,88 @@ static void macb_mac_link_down(struct phylink_config *config, unsigned int mode, netif_tx_stop_all_queues(ndev); } +/* Use juggling algorithm to left rotate tx ring and tx skb array */ +static void gem_shuffle_tx_one_ring(struct macb_queue *queue) +{ + unsigned int i, head, tail, count, size, cycles, shift, curr, next; + struct macb_dma_desc desc, *desc_curr, *desc_next; + struct macb_tx_skb tx_skb, *skb_curr, *skb_next; + struct macb *bp = queue->bp; + unsigned long flags; + + spin_lock_irqsave(&queue->tx_ptr_lock, flags); + head = queue->tx_head; + tail = queue->tx_tail; + size = bp->tx_ring_size; + count = CIRC_CNT(head, tail, size); + + if (!(tail % size)) + goto unlock; + + if (!count) { + queue->tx_head = 0; + queue->tx_tail = 0; + goto unlock; + } + + shift = tail % size; + cycles = gcd(size, shift); + + for (i = 0; i < cycles; i++) { + memcpy(&desc, macb_tx_desc(queue, i), sizeof(struct macb_dma_desc)); + memcpy(&tx_skb, macb_tx_skb(queue, i), sizeof(struct macb_tx_skb)); + + curr = i; + next = (curr + shift) % size; + + while (next != i) { + desc_curr = macb_tx_desc(queue, curr); + desc_next = macb_tx_desc(queue, next); + + memcpy(desc_curr, desc_next, sizeof(struct macb_dma_desc)); + + if (next == bp->tx_ring_size - 1) + desc_curr->ctrl &= ~MACB_BIT(TX_WRAP); + if (curr == bp->tx_ring_size - 1) + desc_curr->ctrl |= MACB_BIT(TX_WRAP); + + skb_curr = macb_tx_skb(queue, curr); + skb_next = macb_tx_skb(queue, next); + memcpy(skb_curr, skb_next, sizeof(struct macb_tx_skb)); + + curr = next; + next = (curr + shift) % size; + } + + desc_curr = macb_tx_desc(queue, curr); + memcpy(desc_curr, &desc, sizeof(struct macb_dma_desc)); + if (i == bp->tx_ring_size - 1) + desc_curr->ctrl &= ~MACB_BIT(TX_WRAP); + if (curr == bp->tx_ring_size - 1) + desc_curr->ctrl |= MACB_BIT(TX_WRAP); + memcpy(macb_tx_skb(queue, curr), &tx_skb, sizeof(struct macb_tx_skb)); + } + + queue->tx_head = count; + queue->tx_tail = 0; + + /* Make descriptor updates visible to hardware */ + wmb(); + +unlock: + spin_unlock_irqrestore(&queue->tx_ptr_lock, flags); +} + +/* Rotate the queue so that the tail is at index 0 */ +static void gem_shuffle_tx_rings(struct macb *bp) +{ + struct macb_queue *queue; + int q; + + for (q = 0, queue = bp->queues; q < bp->num_queues; q++, queue++) + gem_shuffle_tx_one_ring(queue); +} + static void macb_mac_link_up(struct phylink_config *config, struct phy_device *phy, unsigned int mode, phy_interface_t interface, @@ -722,8 +805,6 @@ static void macb_mac_link_up(struct phylink_config *config, ctrl |= MACB_BIT(PAE); for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { - queue->tx_head = 0; - queue->tx_tail = 0; queue_writel(queue, IER, bp->rx_intr_mask | MACB_TX_INT_FLAGS | MACB_BIT(HRESP)); } @@ -737,8 +818,10 @@ static void macb_mac_link_up(struct phylink_config *config, spin_unlock_irqrestore(&bp->lock, flags); - if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC)) + if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC)) { macb_set_tx_clk(bp, speed); + gem_shuffle_tx_rings(bp); + } /* Enable Rx and Tx; Enable PTP unicast */ ctrl = macb_readl(bp, NCR); --- base-commit: fc7b1a72c6cd5cbbd989c6c32a6486e3e4e3594d change-id: 20260214-zynqmp-74d35f3862f8 Best regards, -- Kevin Hao