From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from esa.microchip.iphmx.com (esa.microchip.iphmx.com [68.232.154.123]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB6BE5616A8; Wed, 9 Sep 2026 13:01:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=68.232.154.123 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788958871; cv=none; b=OalHTJZk2d8A5tEnq2bgVdfKKtlut2WTx5v8zsw/jHYUw+svlSAqW/AU7c72zFNmdBClv7jUQ1B0IQQEIBlKbDudS/7uZGFE1GROt8OSwm8MUDkB8uk3DhtV3UqvzcnKfjTBX9/zAlRyLOm+Kv/0VmN6yw9u1MzMunUlrRf006A= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788958871; c=relaxed/simple; bh=b2dGVJTd0yHdTgyW6Eaws6C1vM3i3wCLqxj+jxHi9Vw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-ID:References: In-Reply-To:To:CC; b=ebPEjjlyowE9AOy1ydh5TOiINXJE15qpix06HT3PJS6eusttzUPHxec9Vw636NG/kVkW7pq4RO2OZve/WZewP3cLWNB+K9nAQSvxn8qkIAhKIaXL31QVR8oGfmpKPs4hnbVX1hZR6iaGp2FiQMdtXNmET/EP5DFOI3kf0UIHFHU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=microchip.com; spf=pass smtp.mailfrom=microchip.com; dkim=pass (2048-bit key) header.d=microchip.com header.i=@microchip.com header.b=pny5lrGw; arc=none smtp.client-ip=68.232.154.123 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=microchip.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=microchip.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=microchip.com header.i=@microchip.com header.b="pny5lrGw" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=microchip.com; i=@microchip.com; q=dns/txt; s=mchp; t=1788958869; x=1820494869; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=b2dGVJTd0yHdTgyW6Eaws6C1vM3i3wCLqxj+jxHi9Vw=; b=pny5lrGw4nbI9Oza9DVsNdTk++MDWOourECVZWSuBCQDeBLoljIx61Fj QVdZxEOb20mQBbyFywF8zL492v2bNYxIsdbaefSCLj+AtpbBAKAIjI7iR N0mcVkrfNmexN0o1sAa55QVhyRjVgKBptJaVhpSDLkQ3HPZ/SKgziPDEL SmZ/yzdFeIQ7gUevUjcbzFoFfkem3eMEYsXbZRwm1K3U/4NXEFWX/EYaL JAOgQ7jZcmqYpcb1yAmtwVK7rUZoenRaKcBpn8W8dLm15ofJ6S+Go9Z46 HRLNqEcbEAMgBd2biTT8jl2Jn2hfR7yW2I0V16VHAAPdX78oCYYE/7S26 A==; X-CSE-ConnectionGUID: hVe6ykYaTLaF3yn7jHyvvA== X-CSE-MsgGUID: W1Z8YWMeStGxhRQODZM2NQ== X-IronPort-AV: E=Sophos;i="6.25,270,1779174000"; d="scan'208";a="230464594" X-Amp-Result: SKIPPED(no attachment in message) Received: from unknown (HELO email.microchip.com) ([170.129.1.10]) by esa6.microchip.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2026 06:01:08 -0700 Received: from chn-vm-ex02.mchp-main.com (10.10.87.72) by chn-vm-ex4.mchp-main.com (10.10.87.33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.2562.45; Wed, 9 Sep 2026 06:01:08 -0700 Received: from DEN-DL-M70577.microsemi.net (10.10.85.11) by chn-vm-ex02.mchp-main.com (10.10.85.144) with Microsoft SMTP Server id 15.1.2507.58 via Frontend Transport; Wed, 9 Sep 2026 06:01:03 -0700 From: Daniel Machon Date: Wed, 9 Sep 2026 15:00:13 +0200 Subject: [PATCH net-next v6 11/14] net: lan966x: add PCIe FDMA MTU change support Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-ID: <20260909-lan966x-pci-fdma-v6-11-6f48dab9d671@microchip.com> References: <20260909-lan966x-pci-fdma-v6-0-6f48dab9d671@microchip.com> In-Reply-To: <20260909-lan966x-pci-fdma-v6-0-6f48dab9d671@microchip.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Horatiu Vultur , Steen Hegelund , , "Alexei Starovoitov" , Daniel Borkmann , "Jesper Dangaard Brouer" , John Fastabend , Stanislav Fomichev , Herve Codina , Arnd Bergmann , Greg Kroah-Hartman , Mohsin Bashir CC: Richard Cochran , , , , X-Mailer: b4 0.14.3 Add MTU change support for the PCIe FDMA path. When the MTU changes, the contiguous ATU-mapped RX and TX buffers are reallocated with the new size. On allocation failure, the existing buffers are reused after being reset. Cap the PCIe DCB ring at 256 (FDMA_PCI_DCB_MAX) to keep the entire contiguous allocation under MAX_PAGE_ORDER at jumbo MTU, which 512 DCBs would overflow. Tested-by: Herve Codina Signed-off-by: Daniel Machon --- .../ethernet/microchip/lan966x/lan966x_fdma_pci.c | 168 ++++++++++++++++++++- 1 file changed, 165 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_fdma_pci.c b/drivers/net/ethernet/microchip/lan966x/lan966x_fdma_pci.c index f1f3c789d3a6..6cabbb8b47f2 100644 --- a/drivers/net/ethernet/microchip/lan966x/lan966x_fdma_pci.c +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_fdma_pci.c @@ -1,8 +1,15 @@ // SPDX-License-Identifier: GPL-2.0+ +#include + #include "fdma_api.h" #include "lan966x_main.h" +/* Ring must fit in one MAX_PAGE_ORDER DMA block; 512 DCBs overflows + * at jumbo MTU. + */ +#define FDMA_PCI_DCB_MAX 256 + static int lan966x_fdma_pci_dataptr_cb(struct fdma *fdma, int dcb, int db, u64 *dataptr) { @@ -341,7 +348,7 @@ static int lan966x_fdma_pci_init(struct lan966x *lan966x) lan966x->rx.lan966x = lan966x; lan966x->rx.max_mtu = lan966x_fdma_get_max_frame(lan966x); rx_fdma->channel_id = FDMA_XTR_CHANNEL; - rx_fdma->n_dcbs = FDMA_DCB_MAX; + rx_fdma->n_dcbs = FDMA_PCI_DCB_MAX; rx_fdma->n_dbs = FDMA_RX_DCB_MAX_DBS; rx_fdma->priv = lan966x; rx_fdma->db_size = FDMA_PCI_DB_SIZE(lan966x->rx.max_mtu); @@ -351,7 +358,7 @@ static int lan966x_fdma_pci_init(struct lan966x *lan966x) lan966x->tx.lan966x = lan966x; tx_fdma->channel_id = FDMA_INJ_CHANNEL; - tx_fdma->n_dcbs = FDMA_DCB_MAX; + tx_fdma->n_dcbs = FDMA_PCI_DCB_MAX; tx_fdma->n_dbs = FDMA_TX_DCB_MAX_DBS; tx_fdma->priv = lan966x; tx_fdma->db_size = FDMA_PCI_DB_SIZE(lan966x->rx.max_mtu); @@ -374,9 +381,164 @@ static int lan966x_fdma_pci_init(struct lan966x *lan966x) return 0; } +/* Reset existing rx and tx buffers. */ +static void lan966x_fdma_pci_reset_mem(struct lan966x *lan966x) +{ + struct lan966x_rx *rx = &lan966x->rx; + struct lan966x_tx *tx = &lan966x->tx; + + memset(rx->fdma.dcbs, 0, rx->fdma.size); + memset(tx->fdma.dcbs, 0, tx->fdma.size); + + fdma_dcbs_init(&rx->fdma, + FDMA_DCB_INFO_DATAL(rx->fdma.db_size - XDP_PACKET_HEADROOM), + FDMA_DCB_STATUS_INTR); + + fdma_dcbs_init(&tx->fdma, + FDMA_DCB_INFO_DATAL(tx->fdma.db_size), + FDMA_DCB_STATUS_DONE); + + lan966x_fdma_llp_configure(lan966x, + tx->fdma.atu_region->base_addr, + tx->fdma.channel_id); + lan966x_fdma_llp_configure(lan966x, + rx->fdma.atu_region->base_addr, + rx->fdma.channel_id); +} + +/* Wake all TX queues on every port (undoes lan966x_fdma_tx_disable_netdev). */ +static void lan966x_fdma_pci_wakeup_netdev(struct lan966x *lan966x) +{ + for (int i = 0; i < lan966x->num_phys_ports; ++i) { + struct lan966x_port *port = lan966x->ports[i]; + + if (port) + netif_tx_wake_all_queues(port->dev); + } +} + +static int lan966x_fdma_pci_reload(struct lan966x *lan966x, int new_mtu) +{ + struct fdma tx_fdma_old = lan966x->tx.fdma; + struct fdma rx_fdma_old = lan966x->rx.fdma; + u32 old_mtu = lan966x->rx.max_mtu; + int err; + + napi_disable(&lan966x->napi); + lan966x_fdma_tx_disable_netdev(lan966x); + lan966x_fdma_rx_disable(&lan966x->rx); + lan966x_fdma_tx_disable(&lan966x->tx); + + lan966x->rx.max_mtu = new_mtu; + + /* Must be NULL'ed in order to realloc them. */ + lan966x->rx.fdma.atu_region = NULL; + lan966x->tx.fdma.atu_region = NULL; + + lan966x->tx.fdma.db_size = FDMA_PCI_DB_SIZE(lan966x->rx.max_mtu); + lan966x->tx.fdma.size = fdma_get_size_contiguous(&lan966x->tx.fdma); + lan966x->rx.fdma.db_size = FDMA_PCI_DB_SIZE(lan966x->rx.max_mtu); + lan966x->rx.fdma.size = fdma_get_size_contiguous(&lan966x->rx.fdma); + + err = lan966x_fdma_pci_rx_alloc(&lan966x->rx); + if (err) + goto restore; + + err = lan966x_fdma_pci_tx_alloc(&lan966x->tx); + if (err) { + fdma_free_coherent_and_unmap(lan966x->dma_dev, + &lan966x->rx.fdma); + goto restore; + } + + /* Free and unmap old memory. */ + fdma_free_coherent_and_unmap(lan966x->dma_dev, &rx_fdma_old); + fdma_free_coherent_and_unmap(lan966x->dma_dev, &tx_fdma_old); + + napi_enable(&lan966x->napi); + lan966x_fdma_rx_start(&lan966x->rx); + lan966x_fdma_pci_wakeup_netdev(lan966x); + + return err; +restore: + + /* No new buffers are allocated at this point. Use the old buffers, + * but reset them before starting the FDMA again. + */ + + memcpy(&lan966x->tx.fdma, &tx_fdma_old, sizeof(struct fdma)); + memcpy(&lan966x->rx.fdma, &rx_fdma_old, sizeof(struct fdma)); + + lan966x->rx.max_mtu = old_mtu; + + lan966x_fdma_pci_reset_mem(lan966x); + + napi_enable(&lan966x->napi); + lan966x_fdma_rx_start(&lan966x->rx); + lan966x_fdma_pci_wakeup_netdev(lan966x); + + return err; +} + +static int __lan966x_fdma_pci_reload(struct lan966x *lan966x, int max_mtu) +{ + int err; + u32 val; + + /* Disable the CPU port. */ + lan_rmw(QSYS_SW_PORT_MODE_PORT_ENA_SET(0), + QSYS_SW_PORT_MODE_PORT_ENA, + lan966x, QSYS_SW_PORT_MODE(CPU_PORT)); + + /* Flush the CPU queues. */ + readx_poll_timeout(lan966x_qsys_sw_status, + lan966x, + val, + !(QSYS_SW_STATUS_EQ_AVAIL_GET(val)), + READL_SLEEP_US, READL_TIMEOUT_US); + + /* Add a sleep in case there are frames between the queues and the CPU + * port + */ + usleep_range(USEC_PER_MSEC, 2 * USEC_PER_MSEC); + + err = lan966x_fdma_pci_reload(lan966x, max_mtu); + + /* Enable back the CPU port. */ + lan_rmw(QSYS_SW_PORT_MODE_PORT_ENA_SET(1), + QSYS_SW_PORT_MODE_PORT_ENA, + lan966x, QSYS_SW_PORT_MODE(CPU_PORT)); + + return err; +} + static int lan966x_fdma_pci_resize(struct lan966x *lan966x) { - return -EOPNOTSUPP; + struct fdma rx_fdma; + int max_mtu; + + max_mtu = lan966x_fdma_get_max_frame(lan966x); + if (max_mtu == lan966x->rx.max_mtu) + return 0; + + /* rx and tx have n_dbs == 1, so both rings need the same contiguous + * dma_alloc_coherent() block, which can't exceed MAX_PAGE_ORDER. The + * allocation is padded to the ATU region granularity, so test the + * padded size. + */ + rx_fdma = lan966x->rx.fdma; + rx_fdma.db_size = FDMA_PCI_DB_SIZE(max_mtu); + if (ALIGN(fdma_get_size_contiguous(&rx_fdma), + FDMA_PCI_ATU_REGION_ALIGN) > (PAGE_SIZE << MAX_PAGE_ORDER)) + return -ERANGE; + + /* db_size is also handed to the FDMA in the 16-bit DCB DATAL field, + * where a larger value would be silently truncated. + */ + if (rx_fdma.db_size > GENMASK(15, 0)) + return -ERANGE; + + return __lan966x_fdma_pci_reload(lan966x, max_mtu); } static void lan966x_fdma_pci_deinit(struct lan966x *lan966x) -- 2.34.1