* [PATCH] net/stmmac: add PMD for Synopsys DWC EQoS (GMAC4)
@ 2026-11-14 14:55 Kiran Patchala
2026-11-17 18:42 ` Stephen Hemminger
0 siblings, 1 reply; 2+ messages in thread
From: Kiran Patchala @ 2026-11-14 14:55 UTC (permalink / raw)
To: dev; +Cc: Kiran Patchala, Mohan Degalwade
From: Kiran Patchala <kiran.patchala@oss.qualcomm.com>
Add a new poll-mode driver for the Synopsys DesignWare Cores Enhanced
Quality-of-Service (DWC EQoS / GMAC4, "stmmac") Ethernet MAC present.
The device is enumerated by the platform bus and bound to vfio-platform.
The kernel recovery driver handles clock and PHY reset before the PMD
takes over.
Driver features:
- One TX queue and one RX queue on DMA channel 0; single-segment mbufs
only; no TSO, PTP, or secondary process support.
- Non-coherent DMA: DC CVAC/CIVAC cache maintenance on arm64 for
descriptor and payload lines; no-op stubs on other architectures.
- Ethernet ops: configure, start, stop, close, link_update,
promiscuous enable/disable, mac_addr_set, stats_get, stats_reset,
dev_infos_get, rx/tx_queue_setup, rxq/txq_info_get.
Limitations:
- Single DMA channel (queue 0 only); no multi-queue support.
- Single-segment mbufs; scatter-gather, TSO and jumbo frames are not
supported.
- Tested on arm64 only.
Testing:
dpdk-testpmd: txonly / rxonly / io forward at 1 Gb/s link
Co-developed-by: Mohan Degalwade <mohan.degalwade@oss.qualcomm.com>
Signed-off-by: Mohan Degalwade <mohan.degalwade@oss.qualcomm.com>
Signed-off-by: Kiran Patchala <kiran.patchala@oss.qualcomm.com>
---
doc/guides/nics/index.rst | 1 +
doc/guides/nics/stmmac.rst | 304 +++++++++
drivers/net/meson.build | 1 +
drivers/net/stmmac/meson.build | 15 +
drivers/net/stmmac/stmmac.h | 95 +++
drivers/net/stmmac/stmmac_ethdev.c | 1007 ++++++++++++++++++++++++++++
drivers/net/stmmac/stmmac_logs.h | 25 +
drivers/net/stmmac/stmmac_regs.h | 302 +++++++++
drivers/net/stmmac/stmmac_rxtx.c | 416 ++++++++++++
9 files changed, 2166 insertions(+)
create mode 100644 doc/guides/nics/stmmac.rst
create mode 100644 drivers/net/stmmac/meson.build
create mode 100644 drivers/net/stmmac/stmmac.h
create mode 100644 drivers/net/stmmac/stmmac_ethdev.c
create mode 100644 drivers/net/stmmac/stmmac_logs.h
create mode 100644 drivers/net/stmmac/stmmac_regs.h
create mode 100644 drivers/net/stmmac/stmmac_rxtx.c
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index e20be478f8..b5ca9bddb5 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -68,6 +68,7 @@ Network Interface Controller Drivers
rnp
sfc_efx
softnic
+ stmmac
sxe2
tap
thunderx
diff --git a/doc/guides/nics/stmmac.rst b/doc/guides/nics/stmmac.rst
new file mode 100644
index 0000000000..3cb9cc5af0
--- /dev/null
+++ b/doc/guides/nics/stmmac.rst
@@ -0,0 +1,304 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright 2026 Qualcomm Technologies, Inc.
+
+Stmmac Poll Mode Driver
+=======================
+
+The stmmac PMD (**librte_net_stmmac**) provides poll mode driver
+support for the Synopsys DesignWare Cores Enhanced Quality-of-Service
+(DWC EQoS / GMAC4) Ethernet MAC present on Qualcomm SoCs such as the
+IQ9075 (QCS9075) found on the Shikra EVK platform.
+
+More information on the Qualcomm Shikra platform can be found at
+`Qualcomm Developer Network <https://www.qualcomm.com/developer>`_.
+
+Synopsys DWC EQoS (GMAC4, "stmmac")
+-------------------------------------
+
+This section provides an overview of the Synopsys DWC EQoS MAC
+architecture and how it is integrated into DPDK.
+
+Contents summary
+
+- DWC EQoS overview
+- Driver architecture overview
+- Device configuration and devargs
+
+.. _stmmac_overview:
+
+DWC EQoS Overview
+~~~~~~~~~~~~~~~~~~
+
+The Synopsys DesignWare Cores Enhanced Quality-of-Service (DWC EQoS)
+is a fully integrated 10/100/1000 Mb/s Ethernet MAC core. The "stmmac"
+name follows the Linux kernel driver family
+(``drivers/net/ethernet/stmicro/stmmac``).
+
+The MAC core revision used on the Qualcomm Shikra EVK is GMAC4 v5.30a.
+
+DWC EQoS includes:
+
+- MAC core (GMAC4): frame transmission, reception, MDIO management,
+ address filtering, and hardware checksum offload
+- MTL (MAC Transaction Layer): configurable TX/RX FIFOs with
+ store-and-forward mode; FIFO sizes are read from HW_FEATURE1 at probe
+- DMA engine: AXI system bus with fixed burst, mixed burst, AAL and
+ EAME (>4 GiB IOVA support); per-channel burst (PBL 32, PBLx8, OSP)
+- RGMII IO macro: Qualcomm-specific SerDes block for RGMII-ID timing
+
+The Network and packet I/O component:
+
+- One DMA channel (channel 0) with one TX queue and one RX queue.
+ The PMD operates DMA channel 0 exclusively.
+- The DMA is not cache-coherent on the Shikra platform. The PMD
+ performs explicit DC CVAC/CIVAC cache maintenance on arm64 for both
+ descriptor lines and packet payload. No-op stubs are provided for
+ other architectures.
+
+Stmmac DPDK - Poll Mode Driver Overview
+-----------------------------------------
+
+This section provides an overview of the stmmac PMD for DPDK.
+
+The PMD is probed by the DPDK platform bus and uses ``vfio-platform``
+for MMIO access and DMA mapping. The kernel recovery driver must be
+loaded first to perform clock enable and PHY reset; the PMD then takes
+over and must not be unloaded while the PMD is running.
+
+.. code-block:: console
+
+ +-------------------+
+ | DPDK Application |
+ +--------+----------+
+ |
+ +--------+----------+
+ | stmmac PMD |
+ | (librte_net_stmmac)|
+ +----+----------+---+
+ | |
+ +---------+--+ +---+---------+
+ | TX Burst | | RX Burst |
+ | (DMA ch 0) | | (DMA ch 0) |
+ +---------+--+ +---+---------+
+ | |
+ ===================|==========|================ HARDWARE
+ +----+----------+---+
+ | DWC EQoS (GMAC4) |
+ | MAC / MTL / DMA |
+ +--------+----------+
+ |
+ +--------+----------+
+ | RGMII IO Macro |
+ +--------+----------+
+ |
+ +--------+----------+
+ | DP83867 PHY |
+ +-------------------+
+
+Platform Bus Driver
+~~~~~~~~~~~~~~~~~~~
+
+The stmmac PMD registers as an ``rte_platform_driver``. The platform bus
+scans the kernel sysfs tree for a device whose driver alias matches
+``vfio-platform``. Two MMIO resources are consumed:
+
+- ``resource[0]``: MAC/MTL/DMA register block (``stmmaceth`` in DT)
+- ``resource[1]``: RGMII IO macro register block (``rgmii`` in DT)
+
+The kernel ``vfio-platform`` driver must be bound to the stmmac device
+before launching a DPDK application.
+
+Stmmac NIC Driver (PMD)
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The stmmac PMD implements the standard DPDK ethdev API for the DWC EQoS
+MAC. It directly programs the MAC, MTL and DMA register blocks over
+the MMIO mapping provided by vfio-platform.
+
+- On probe, the PMD reads devargs, maps MMIO resources, sets RGMII IO
+ macro timing, programs DP83867 RGMII internal delays via MMD indirect
+ access, and registers the ethdev.
+- On ``dev_start``, the PMD enables the RGMII functional clock,
+ performs a DMA soft reset, programs AXI sysbus and per-channel burst
+ settings, sets up TX and RX descriptor rings, initialises the MAC
+ and MTL, and starts DMA.
+- Link state is polled from the DP83867 PHY using IEEE 802.3 clause-22
+ MDIO. BMSR is double-read to clear the latched link bit; live speed
+ and duplex come from PHYSTS.
+
+Features
+^^^^^^^^
+
+Features of the stmmac PMD are:
+
+- Single TX queue and single RX queue on DMA channel 0
+- IPv4, TCP and UDP hardware checksum offload (TX and RX)
+- Promiscuous mode
+- MAC address set
+- Link state update via DP83867 PHY MDIO (clause-22)
+- Per-queue packet and byte counters (stats_get / stats_reset)
+- Configurable ring sizes (64..1024 descriptors, multiples of 4)
+- Non-coherent DMA support (arm64 DC CVAC/CIVAC cache maintenance)
+- 36-bit DMA addressing (EAME enabled, mbufs above 4 GiB supported)
+
+Allowing & Blocking
+-------------------
+
+To block a stmmac device from being used by DPDK, the following EAL
+argument can be used:
+
+.. code-block:: console
+
+ <dpdk app> <EAL args> -b "platform:<device_name>" -- ...
+ e.g. -b "platform:a000000.ethernet"
+
+Supported Platforms
+-------------------
+
+- Qualcomm IQ9075 (QCS9075) — Shikra EVK (arm64)
+
+Prerequisites
+-------------
+
+- Follow the DPDK :doc:`/linux_gsg/index` to set up the basic DPDK
+ environment.
+- An arm64 cross-compilation toolchain is required to build for the
+ Shikra EVK target from an x86 host.
+- The kernel ``vfio-platform`` and ``vfio_iommu_type1`` modules must be
+ loaded on the target before launching a DPDK application.
+- The Qualcomm stmmac recovery kernel driver must be loaded first to
+ enable clocks and reset the PHY. The PMD does not perform these steps.
+
+.. note::
+
+ The stmmac PMD is supported on Linux only. It requires the platform
+ bus (``rte_bus_platform``) and ``vfio-platform`` kernel support.
+
+Configuration
+-------------
+
+Devargs
+~~~~~~~
+
+The stmmac PMD accepts the following device arguments:
+
+.. list-table:: stmmac devargs
+ :header-rows: 1
+ :widths: 20 10 70
+
+ * - Parameter
+ - Default
+ - Description
+ * - ``has_emac_ge_3``
+ - ``1``
+ - DMA/MTL channel stride override. Set to ``1`` for EMAC >= 3.0
+ platforms (Shikra IQ9075), which use a 0x1000 channel stride
+ (base 0x8100 for DMA, 0x8000 for MTL). Set to ``0`` for
+ generic dwmac4 layout (base 0x1100, stride 0x80).
+ * - ``mdio_addr``
+ - ``7``
+ - PHY MDIO address. The DP83867 on the Shikra EVK is at address 7.
+ Override if the PHY is at a different address.
+
+Example devarg usage:
+
+.. code-block:: console
+
+ --dev "platform:a000000.ethernet,has_emac_ge_3=1,mdio_addr=7"
+
+Driver Compilation and Testing
+-------------------------------
+
+Refer to the document :doc:`build_and_test` for details.
+
+#. Cross-compile for arm64 (from an x86 host):
+
+ .. code-block:: console
+
+ meson setup build-arm64 \
+ --cross-file config/arm/arm64_armv8_linux_gcc \
+ -Dplatform=generic
+ ninja -C build-arm64
+
+#. Bind the device to ``vfio-platform`` on the target:
+
+ .. code-block:: console
+
+ # Load required kernel modules
+ modprobe vfio
+ modprobe vfio-platform
+ modprobe vfio_iommu_type1
+
+ # Identify the device name from sysfs
+ ls /sys/bus/platform/devices/ | grep ethernet
+ # e.g. a000000.ethernet
+
+ # Bind to vfio-platform
+ echo vfio-platform > /sys/bus/platform/devices/a000000.ethernet/driver_override
+ echo a000000.ethernet > /sys/bus/platform/drivers/vfio-platform/bind
+
+ # Verify the bind succeeded
+ ls /sys/bus/platform/devices/a000000.ethernet/driver
+ # Should show: .../vfio-platform
+
+#. Running testpmd on the Shikra EVK:
+
+ .. code-block:: console
+
+ ./<build_dir>/app/dpdk-testpmd \
+ -l 0-3 -n 1 --vdev="" \
+ -- -i --portmask=0x1 --nb-cores=1 --no-flush-rx
+ .....
+ EAL: Registered [platform] bus.
+ EAL: Detected 8 lcore(s)
+ .....
+ Configuring Port 0 (socket 0)
+ Port 0: 02:02:02:02:02:02
+ .....
+ Checking link statuses...
+ Port 0 Link Up - speed 1000 Mbps - full-duplex
+ Done
+ testpmd>
+
+ * Use devarg ``has_emac_ge_3=0`` for non-Shikra platforms with the
+ generic dwmac4 DMA channel layout.
+ * Use devarg ``mdio_addr=<n>`` if the PHY MDIO address differs from
+ the Shikra default of 7.
+
+Limitations
+-----------
+
+Single Queue
+~~~~~~~~~~~~
+
+The stmmac PMD supports one TX queue and one RX queue only (DMA
+channel 0). Multi-queue configurations are not supported in this
+release.
+
+Single-segment Mbufs
+~~~~~~~~~~~~~~~~~~~~~
+
+Only single-segment mbufs are supported. Scatter-gather, TSO, jumbo
+frames, and multi-segment packet transmission are not supported. The
+application must ensure that ``nb_segs == 1`` for every transmitted
+mbuf.
+
+No Interrupt Support
+~~~~~~~~~~~~~~~~~~~~~
+
+The PMD is polling-only. No interrupt-driven TX completion or RX
+notification is supported. The platform bus does not expose an interrupt
+to the PMD.
+
+No Secondary Process Support
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Multi-process operation where I/O is performed by a secondary DPDK
+process is not supported.
+
+Platform Requirement
+~~~~~~~~~~~~~~~~~~~~~
+
+The stmmac PMD is supported on Linux on arm64 only. It has been tested
+on the Qualcomm Shikra EVK (IQ9075 / QCS9075 SoC). Operation on other
+platforms is not guaranteed.
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 4e8ccb945f..4d8549c4f9 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -58,6 +58,7 @@ drivers = [
'rnp',
'sfc',
'softnic',
+ 'stmmac',
'sxe2',
'tap',
'thunderx',
diff --git a/drivers/net/stmmac/meson.build b/drivers/net/stmmac/meson.build
new file mode 100644
index 0000000000..fb3ed0e162
--- /dev/null
+++ b/drivers/net/stmmac/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2026 Qualcomm Technologies, Inc.
+
+if not is_linux
+ build = false
+ reason = 'only supported on linux'
+ subdir_done()
+endif
+
+deps += ['bus_platform']
+
+sources = files(
+ 'stmmac_ethdev.c',
+ 'stmmac_rxtx.c',
+)
diff --git a/drivers/net/stmmac/stmmac.h b/drivers/net/stmmac/stmmac.h
new file mode 100644
index 0000000000..ccfc9309c3
--- /dev/null
+++ b/drivers/net/stmmac/stmmac.h
@@ -0,0 +1,95 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2026 Qualcomm Technologies, Inc.
+ */
+
+#ifndef _STMMAC_H_
+#define _STMMAC_H_
+
+#include <stdint.h>
+
+#include <rte_ether.h>
+#include <rte_ethdev.h>
+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+
+#include "stmmac_regs.h"
+
+#define STMMAC_MAX_Q 1
+
+#define STMMAC_DEFAULT_RING_SIZE 256
+#define STMMAC_MAX_RING_SIZE 1024 /* Linux DMA_MAX_RX/TX_SIZE */
+
+/*
+ * Descriptor overlay: raw 32-bit words, interpreted through the bit macros
+ * in stmmac_regs.h, as the kernel driver does for the GMAC4 normal descriptor.
+ */
+struct stmmac_desc {
+ uint32_t des[4];
+};
+
+struct stmmac_tx_queue {
+ struct stmmac_desc *tx_ring;
+ uint64_t tx_ring_iova;
+ struct rte_mbuf **sw_ring;
+ uint16_t nb_desc;
+ uint16_t tx_tail; /* next slot to fill */
+ uint16_t tx_free; /* next slot to reclaim */
+ uint16_t queue_id;
+ uint16_t port_id;
+ void *chan_base; /* mac_base + DMA chan(0) offset */
+ const struct rte_memzone *mz;
+ uint64_t tx_packets;
+ uint64_t tx_bytes;
+ uint64_t tx_errors; /* TDES3 ES seen on reclaim */
+ uint64_t tx_ring_full; /* burst stopped: no free slot */
+};
+
+struct stmmac_rx_queue {
+ struct stmmac_desc *rx_ring;
+ uint64_t rx_ring_iova;
+ struct rte_mbuf **sw_ring;
+ uint16_t nb_desc;
+ uint16_t rx_tail; /* next slot to check */
+ uint16_t rx_refill; /* next slot to refill */
+ uint16_t queue_id;
+ uint16_t port_id;
+ struct rte_mempool *mb_pool;
+ void *chan_base; /* mac_base + DMA chan(0) offset */
+ const struct rte_memzone *mz;
+ uint64_t rx_packets;
+ uint64_t rx_bytes;
+ uint64_t rx_errors;
+ uint64_t rx_nombuf;
+};
+
+struct stmmac_priv {
+ void *mac_base; /* resource[0]: MAC/MTL/DMA block */
+ void *rgmii_base; /* resource[1]: RGMII IO macro (NULL if absent) */
+ struct stmmac_chan_addrs chan_addrs;
+
+ struct stmmac_tx_queue *txq[STMMAC_MAX_Q];
+ struct stmmac_rx_queue *rxq[STMMAC_MAX_Q];
+
+ struct rte_ether_addr mac_addr;
+
+ uint16_t mdio_addr; /* PHY MDIO address, from devarg */
+ int has_emac_ge_3; /* DMA/MTL channel stride override */
+
+ int link_up;
+ uint32_t link_speed;
+
+ struct rte_eth_stats stats;
+};
+
+/* stmmac_rxtx.c */
+uint16_t stmmac_rx_pkt_burst_nc(void *rxq, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t stmmac_tx_pkt_burst_nc(void *txq, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+
+uint16_t stmmac_mdio_read(void *mac_base, uint16_t mdio_addr,
+ uint16_t phyreg);
+void stmmac_mdio_write(void *mac_base, uint16_t mdio_addr,
+ uint16_t phyreg, uint16_t phydata);
+
+#endif /* _STMMAC_H_ */
diff --git a/drivers/net/stmmac/stmmac_ethdev.c b/drivers/net/stmmac/stmmac_ethdev.c
new file mode 100644
index 0000000000..ecbca764a5
--- /dev/null
+++ b/drivers/net/stmmac/stmmac_ethdev.c
@@ -0,0 +1,1007 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2026 Qualcomm Technologies, Inc.
+ *
+ * DPDK poll-mode driver for the Synopsys DWC EQoS (GMAC4, "stmmac") MAC.
+ *
+ * Scope:
+ * - DMA channel 0 with one TX and one RX queue.
+ * - Polling only: the platform bus has no interrupt support, so TX and RX
+ * completion and link state are polled.
+ * - Single-segment mbufs only; no TSO, PTP, or secondary process support.
+ * - Link state from the DP83867 PHY (BMSR and PHYSTS), using generic
+ * IEEE 802.3 clause-22 MDIO.
+ * - The RGMII IO macro and the DP83867 RGMII delays are configured here.
+ * The kernel recovery driver handles the clocks and the PHY reset.
+ */
+
+#include <stdlib.h>
+#include <string.h>
+
+#include <bus_platform_driver.h>
+#include <ethdev_driver.h>
+#include <rte_common.h>
+#include <rte_cycles.h>
+#include <rte_ethdev.h>
+#include <rte_io.h>
+#include <rte_kvargs.h>
+#include <rte_malloc.h>
+
+#include "stmmac.h"
+#include "stmmac_regs.h"
+#include "stmmac_logs.h"
+
+RTE_LOG_REGISTER_DEFAULT(stmmac_logtype_pmd, NOTICE);
+
+#define STMMAC_DRIVER_NAME net_stmmac
+
+#define STMMAC_DEVARG_HAS_EMAC_GE_3 "has_emac_ge_3"
+#define STMMAC_DEVARG_MDIO_ADDR "mdio_addr"
+
+/* Shikra EVK PHY (DP83867) MDIO address */
+#define STMMAC_DEFAULT_MDIO_ADDR 7
+
+#define STMMAC_DMA_RESET_POLL_US 1000
+#define STMMAC_DMA_RESET_POLLS 1000 /* 1 s in total, as dwmac4_dma_reset() */
+
+#define STMMAC_DMA_PBL 32
+#define STMMAC_DMA_PBL_SHIFT 16
+#define STMMAC_DMA_PBL_MASK (0x3fu << STMMAC_DMA_PBL_SHIFT)
+
+/* MTL RX/TX queue FIFO size fields (see stmmac_mtl_init) */
+#define STMMAC_ACTIVE_QUEUES 1
+
+/* Qualcomm RGMII IO macro and SDCC registers (offsets within the rgmii resource) */
+#define RGMII_IO_MACRO_CONFIG 0x00
+#define RGMII_IO_MACRO_CONFIG2 0x1c
+#define SDCC_HC_REG_DLL_CONFIG 0x04
+#define SDCC_USR_CTL 0x18
+
+#define RGMII_CONFIG_FUNC_CLK_EN RTE_BIT32(30)
+#define RGMII_CFG_DDR_MODE RTE_BIT32(0)
+#define RGMII_CFG_PROG_SWAP RTE_BIT32(1)
+#define RGMII_CFG_LOOPBACK_EN RTE_BIT32(2)
+#define RGMII_CFG_BYPASS_TX_ID_EN RTE_BIT32(3)
+#define RGMII_CFG_POS_NEG_DATA_SEL RTE_BIT32(23)
+#define RGMII_CFG2_TX_TO_RX_LOOPBACK RTE_BIT32(13)
+#define RGMII_CFG2_RSVD_CONFIG15 (0x7fffu << 17)
+#define RGMII_CFG2_RX_PROG_SWAP RTE_BIT32(7)
+#define RGMII_CFG2_DATA_DIV_CLK_SEL RTE_BIT32(6)
+#define RGMII_CFG2_TX_PHASE_SHIFT_EN RTE_BIT32(5)
+#define SDCC_DLL_CONFIG_PDN RTE_BIT32(29)
+#define SDCC_USR_CTL_DDR_BYPASS RTE_BIT32(30)
+
+/* DP83867 RGMII internal delays, set through the MMD indirect registers */
+#define MII_PHYSID1 0x02
+#define MII_PHYSID2 0x03
+#define MII_REGCR 0x0d
+#define MII_ADDAR 0x0e
+#define MII_REGCR_ADDR 0x001f
+#define MII_REGCR_DATA 0x401f
+#define DP83867_RGMIICTL 0x0032
+#define DP83867_RGMIIDCTL 0x0086
+#define DP83867_RGMII_TX_CLK_DELAY_EN RTE_BIT32(1)
+#define DP83867_RGMII_RX_CLK_DELAY_EN RTE_BIT32(0)
+#define DP83867_RGMIIDCTL_2_00_NS 0x7
+#define DP83867_PHYCTRL 0x10
+#define DP83867_PHY_ID 0x2000a231
+
+static const char * const stmmac_valid_args[] = {
+ STMMAC_DEVARG_HAS_EMAC_GE_3,
+ STMMAC_DEVARG_MDIO_ADDR,
+ NULL,
+};
+
+static uint64_t dev_rx_offloads_sup = RTE_ETH_RX_OFFLOAD_IPV4_CKSUM;
+static uint64_t dev_tx_offloads_sup = RTE_ETH_TX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_TX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_UDP_CKSUM;
+
+static int
+stmmac_kvarg_handler(const char *key, const char *value, void *opaque)
+{
+ unsigned long v;
+ char *end;
+
+ RTE_SET_USED(key);
+
+ v = strtoul(value, &end, 0);
+ if (*end != '\0')
+ return -EINVAL;
+
+ *(uint32_t *)opaque = (uint32_t)v;
+ return 0;
+}
+
+static void
+stmmac_parse_devargs(struct rte_platform_device *pdev, struct stmmac_priv *priv)
+{
+ struct rte_kvargs *kvlist;
+ uint32_t v;
+
+ priv->has_emac_ge_3 = 1;
+ priv->mdio_addr = STMMAC_DEFAULT_MDIO_ADDR;
+
+ if (pdev->device.devargs == NULL ||
+ pdev->device.devargs->args == NULL)
+ return;
+
+ kvlist = rte_kvargs_parse(pdev->device.devargs->args,
+ stmmac_valid_args);
+ if (kvlist == NULL)
+ return;
+
+ if (rte_kvargs_count(kvlist, STMMAC_DEVARG_HAS_EMAC_GE_3) > 0) {
+ v = 0;
+ if (rte_kvargs_process(kvlist, STMMAC_DEVARG_HAS_EMAC_GE_3,
+ stmmac_kvarg_handler, &v) == 0)
+ priv->has_emac_ge_3 = (int)v;
+ }
+
+ if (rte_kvargs_count(kvlist, STMMAC_DEVARG_MDIO_ADDR) > 0) {
+ v = 0;
+ if (rte_kvargs_process(kvlist, STMMAC_DEVARG_MDIO_ADDR,
+ stmmac_kvarg_handler, &v) == 0)
+ priv->mdio_addr = (uint16_t)v;
+ }
+
+ rte_kvargs_free(kvlist);
+}
+
+/*
+ * Keep the RGMII IO macro clock enabled. Without FUNC_CLK_EN the DMA reset
+ * can time out because the PHY clock is missing.
+ */
+static void
+stmmac_rgmii_func_clk_en(struct stmmac_priv *priv)
+{
+ void *reg;
+
+ if (priv->rgmii_base == NULL) {
+ STMMAC_PMD_WARN("RGMII region not mapped, FUNC_CLK_EN not set");
+ return;
+ }
+ reg = (uint8_t *)priv->rgmii_base + RGMII_IO_MACRO_CONFIG;
+ rte_write32(rte_read32(reg) | RGMII_CONFIG_FUNC_CLK_EN, reg);
+}
+
+static void
+stmmac_rgmii_rmw(struct stmmac_priv *priv, uint32_t off, uint32_t clr, uint32_t set)
+{
+ void *reg = (uint8_t *)priv->rgmii_base + off;
+
+ rte_write32((rte_read32(reg) & ~clr) | set, reg);
+}
+
+/*
+ * RGMII IO macro setup for rgmii-id at 1 Gb/s, as the kernel does at link-up.
+ * The PHY adds the RGMII delays, so the MAC-side DLL is powered down and the
+ * DDR path is bypassed. TX-to-RX loopback is disabled so RX data comes from
+ * the PHY.
+ */
+static void
+stmmac_rgmii_id_1g_init(struct stmmac_priv *priv)
+{
+ if (priv->rgmii_base == NULL)
+ return;
+
+ stmmac_rgmii_rmw(priv, SDCC_HC_REG_DLL_CONFIG, 0, SDCC_DLL_CONFIG_PDN);
+ stmmac_rgmii_rmw(priv, SDCC_USR_CTL, 0, SDCC_USR_CTL_DDR_BYPASS);
+
+ stmmac_rgmii_rmw(priv, RGMII_IO_MACRO_CONFIG2, RGMII_CFG2_TX_TO_RX_LOOPBACK, 0);
+ stmmac_rgmii_rmw(priv, RGMII_IO_MACRO_CONFIG,
+ RGMII_CFG_POS_NEG_DATA_SEL | RGMII_CFG_PROG_SWAP |
+ RGMII_CFG_LOOPBACK_EN,
+ RGMII_CFG_DDR_MODE | RGMII_CFG_BYPASS_TX_ID_EN);
+ stmmac_rgmii_rmw(priv, RGMII_IO_MACRO_CONFIG2,
+ RGMII_CFG2_DATA_DIV_CLK_SEL | RGMII_CFG2_TX_PHASE_SHIFT_EN |
+ RGMII_CFG2_RSVD_CONFIG15,
+ RGMII_CFG2_RX_PROG_SWAP);
+}
+
+static uint16_t
+stmmac_phy_mmd_read(struct stmmac_priv *priv, uint16_t reg)
+{
+ stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_REGCR, MII_REGCR_ADDR);
+ stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_ADDAR, reg);
+ stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_REGCR, MII_REGCR_DATA);
+ return stmmac_mdio_read(priv->mac_base, priv->mdio_addr, MII_ADDAR);
+}
+
+static void
+stmmac_phy_mmd_write(struct stmmac_priv *priv, uint16_t reg, uint16_t val)
+{
+ stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_REGCR, MII_REGCR_ADDR);
+ stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_ADDAR, reg);
+ stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_REGCR, MII_REGCR_DATA);
+ stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_ADDAR, val);
+}
+
+/*
+ * Set the DP83867 RGMII internal delays to 2.00 ns on both TX and RX. The
+ * kernel does the same for phy-mode "rgmii-id". The MAC does not add these
+ * delays, so without them RX frames are sampled off the data eye.
+ */
+static void
+stmmac_phy_rgmii_id_init(struct stmmac_priv *priv)
+{
+ uint16_t id1, id2, ctl;
+
+ id1 = stmmac_mdio_read(priv->mac_base, priv->mdio_addr, MII_PHYSID1);
+ id2 = stmmac_mdio_read(priv->mac_base, priv->mdio_addr, MII_PHYSID2);
+ if (id1 == 0xffff || id2 == 0xffff) {
+ STMMAC_PMD_WARN("PHY not responding on MDIO, RGMII delays not set");
+ return;
+ }
+ if (((uint32_t)id1 << 16 | id2) != DP83867_PHY_ID)
+ STMMAC_PMD_WARN("unexpected PHY ID 0x%04x%04x", id1, id2);
+
+ ctl = stmmac_phy_mmd_read(priv, DP83867_RGMIICTL);
+ stmmac_phy_mmd_write(priv, DP83867_RGMIICTL,
+ ctl | DP83867_RGMII_TX_CLK_DELAY_EN |
+ DP83867_RGMII_RX_CLK_DELAY_EN);
+ stmmac_phy_mmd_write(priv, DP83867_RGMIIDCTL,
+ DP83867_RGMIIDCTL_2_00_NS |
+ (DP83867_RGMIIDCTL_2_00_NS << 4));
+}
+
+/*
+ * DMA soft reset, as dwmac4_dma_reset(): poll the SWR bit for up to 1 s.
+ */
+static int
+stmmac_dma_reset(void *mac_base)
+{
+ void *reg = (uint8_t *)mac_base + DMA_BUS_MODE;
+ unsigned int i;
+
+ rte_write32(rte_read32(reg) | DMA_BUS_MODE_SFT_RESET, reg);
+
+ for (i = 0; i < STMMAC_DMA_RESET_POLLS; i++) {
+ if (!(rte_read32(reg) & DMA_BUS_MODE_SFT_RESET))
+ return 0;
+ rte_delay_us(STMMAC_DMA_RESET_POLL_US);
+ }
+
+ STMMAC_PMD_ERR("DMA soft reset timed out");
+ return -ETIMEDOUT;
+}
+
+/*
+ * DMA system and AXI bus configuration. Fixed burst, mixed burst and AAL are
+ * enabled, with BLEN16 and the maximum outstanding requests. EAME is set
+ * because the rings and mbufs are above 4 GiB.
+ */
+static void
+stmmac_dma_sysbus_init(void *mac_base)
+{
+ void *sysbus_reg = (uint8_t *)mac_base + DMA_SYS_BUS_MODE;
+ void *axi_reg = (uint8_t *)mac_base + DMA_AXI_BUS_MODE;
+ uint32_t value;
+
+ value = rte_read32(sysbus_reg);
+ value |= DMA_SYS_BUS_FB | DMA_SYS_BUS_MB | DMA_SYS_BUS_AAL;
+ value |= DMA_SYS_BUS_EAME;
+ rte_write32(value, sysbus_reg);
+
+ value = rte_read32(axi_reg);
+ value &= ~(DMA_AXI_WR_OSR_LMT | DMA_AXI_RD_OSR_LMT);
+ value |= (DMA_AXI_OSR_MAX << DMA_AXI_WR_OSR_LMT_SHIFT) & DMA_AXI_WR_OSR_LMT;
+ value |= (DMA_AXI_OSR_MAX << DMA_AXI_RD_OSR_LMT_SHIFT) & DMA_AXI_RD_OSR_LMT;
+ value |= DMA_AXI_BLEN16;
+ rte_write32(value, axi_reg);
+}
+
+/*
+ * Per-channel DMA burst settings: PBL 32, PBLx8 on, and OSP on TX. The DMA is
+ * too slow to keep up with the wire if these stay at the reset values.
+ */
+static void
+stmmac_dma_chan_cfg(void *chan_base)
+{
+ void *ch_ctrl = (uint8_t *)chan_base + DMA_CHAN_OFF_CONTROL;
+ void *tx_ctrl = (uint8_t *)chan_base + DMA_CHAN_OFF_TX_CONTROL;
+ void *rx_ctrl = (uint8_t *)chan_base + DMA_CHAN_OFF_RX_CONTROL;
+ uint32_t v;
+
+ v = rte_read32(ch_ctrl);
+ rte_write32(v | DMA_CONTROL_PBLX8, ch_ctrl);
+
+ v = rte_read32(tx_ctrl);
+ v &= ~STMMAC_DMA_PBL_MASK;
+ v |= ((uint32_t)STMMAC_DMA_PBL << STMMAC_DMA_PBL_SHIFT) | DMA_CHAN_TX_CONTROL_OSP;
+ rte_write32(v, tx_ctrl);
+
+ v = rte_read32(rx_ctrl);
+ v &= ~STMMAC_DMA_PBL_MASK;
+ v |= (uint32_t)STMMAC_DMA_PBL << STMMAC_DMA_PBL_SHIFT;
+ rte_write32(v, rx_ctrl);
+}
+
+static int
+stmmac_tx_ring_setup(struct stmmac_tx_queue *txq)
+{
+ uint8_t *chan = txq->chan_base;
+ unsigned int i;
+
+ for (i = 0; i < txq->nb_desc; i++)
+ memset(&txq->tx_ring[i], 0, sizeof(struct stmmac_desc));
+ /* the non-coherent DMA must see the zeroed ring */
+ stmmac_dcache_clean(txq->tx_ring, (size_t)txq->nb_desc * sizeof(struct stmmac_desc));
+
+ rte_write32((uint32_t)(txq->tx_ring_iova >> 32), chan + DMA_CHAN_OFF_TX_BASE_ADDR_HI);
+ rte_write32((uint32_t)txq->tx_ring_iova, chan + DMA_CHAN_OFF_TX_BASE_ADDR);
+ /* the ring length register holds nb_desc - 1 */
+ rte_write32(txq->nb_desc - 1, chan + DMA_CHAN_OFF_TX_RING_LEN);
+ rte_write32((uint32_t)txq->tx_ring_iova, chan + DMA_CHAN_OFF_TX_END_ADDR);
+
+ return 0;
+}
+
+static int
+stmmac_rx_ring_setup(struct stmmac_rx_queue *rxq)
+{
+ uint8_t *chan = rxq->chan_base;
+ unsigned int i;
+ uint64_t tail;
+ uint32_t bufsz, v;
+
+ for (i = 0; i < rxq->nb_desc; i++) {
+ struct rte_mbuf *mb = rte_pktmbuf_alloc(rxq->mb_pool);
+ uint64_t iova;
+
+ if (mb == NULL) {
+ STMMAC_PMD_ERR("failed to allocate RX mbuf %u", i);
+ return -ENOMEM;
+ }
+
+ iova = rte_pktmbuf_iova(mb);
+ rxq->sw_ring[i] = mb;
+ /* drop any dirty lines over the buffer before the DMA writes it */
+ stmmac_dcache_flush(rte_pktmbuf_mtod(mb, void *),
+ rte_pktmbuf_data_room_size(rxq->mb_pool) -
+ RTE_PKTMBUF_HEADROOM);
+ rxq->rx_ring[i].des[0] = (uint32_t)iova;
+ rxq->rx_ring[i].des[1] = (uint32_t)(iova >> 32);
+ rxq->rx_ring[i].des[2] = 0;
+ rxq->rx_ring[i].des[3] = RDES3_OWN | RDES3_BUFFER1_VALID_ADDR |
+ RDES3_INT_ON_COMPLETION_EN;
+ }
+ /* the non-coherent DMA must see every OWN=1 descriptor before RXTAIL is set */
+ stmmac_dcache_clean(rxq->rx_ring, (size_t)rxq->nb_desc * sizeof(struct stmmac_desc));
+
+ /* RX buffer size, which must be non-zero and a multiple of 4 */
+ bufsz = (rte_pktmbuf_data_room_size(rxq->mb_pool) - RTE_PKTMBUF_HEADROOM) & ~3u;
+ v = rte_read32(chan + DMA_CHAN_OFF_RX_CONTROL);
+ v = (v & ~DMA_CONTROL_RBSZ_MASK) | ((bufsz << DMA_CONTROL_RBSZ_SHIFT) &
+ DMA_CONTROL_RBSZ_MASK);
+ rte_write32(v, chan + DMA_CHAN_OFF_RX_CONTROL);
+
+ rte_write32((uint32_t)(rxq->rx_ring_iova >> 32), chan + DMA_CHAN_OFF_RX_BASE_ADDR_HI);
+ rte_write32((uint32_t)rxq->rx_ring_iova, chan + DMA_CHAN_OFF_RX_BASE_ADDR);
+ /* the ring length register holds nb_desc - 1 */
+ rte_write32(rxq->nb_desc - 1, chan + DMA_CHAN_OFF_RX_RING_LEN);
+ tail = rxq->rx_ring_iova + (uint64_t)rxq->nb_desc * sizeof(struct stmmac_desc);
+ rte_write32((uint32_t)tail, chan + DMA_CHAN_OFF_RX_END_ADDR);
+
+ return 0;
+}
+
+/*
+ * MTL operating mode for the single active queue. Store-and-forward is
+ * enabled, and the queue FIFO sizes (RQS/TQS) are programmed from
+ * HW_FEATURE1. The reset value of 256 bytes is too small for store-and-forward.
+ * Queue 0 owns the whole FIFO and maps to DMA channel 0. Flow control is not
+ * enabled.
+ */
+static void
+stmmac_mtl_init(void *mac_base, const struct stmmac_chan_addrs *addrs)
+{
+ void *tx_op = (uint8_t *)mac_base + MTL_CHAN_TX_OP_MODE(addrs, 0);
+ void *rx_op = (uint8_t *)mac_base + MTL_CHAN_RX_OP_MODE(addrs, 0);
+ void *rxq_map = (uint8_t *)mac_base + MTL_RXQ_DMA_MAP0;
+ uint32_t hwf1 = rte_read32((uint8_t *)mac_base + GMAC_HW_FEATURE1);
+ uint32_t rx_fifo = 128u << (hwf1 & GMAC_HWF1_RXFIFOSIZE_MASK);
+ uint32_t tx_fifo = 128u << ((hwf1 >> GMAC_HWF1_TXFIFOSIZE_SHIFT) &
+ GMAC_HWF1_TXFIFOSIZE_MASK);
+ uint32_t rqs = rx_fifo / STMMAC_ACTIVE_QUEUES >= 256 ?
+ rx_fifo / STMMAC_ACTIVE_QUEUES / 256 - 1 : 0;
+ uint32_t tqs = tx_fifo / STMMAC_ACTIVE_QUEUES >= 256 ?
+ tx_fifo / STMMAC_ACTIVE_QUEUES / 256 - 1 : 0;
+ uint32_t value;
+
+ if (rqs > (MTL_OP_MODE_RQS_MASK >> MTL_OP_MODE_RQS_SHIFT))
+ rqs = MTL_OP_MODE_RQS_MASK >> MTL_OP_MODE_RQS_SHIFT;
+ if (tqs > (MTL_OP_MODE_TQS_MASK >> MTL_OP_MODE_TQS_SHIFT))
+ tqs = MTL_OP_MODE_TQS_MASK >> MTL_OP_MODE_TQS_SHIFT;
+
+ value = rte_read32(tx_op);
+ value &= ~MTL_OP_MODE_TQS_MASK;
+ value |= MTL_OP_MODE_TSF | MTL_OP_MODE_TXQEN | (tqs << MTL_OP_MODE_TQS_SHIFT);
+ rte_write32(value, tx_op);
+
+ value = rte_read32(rx_op);
+ value &= ~MTL_OP_MODE_RQS_MASK;
+ value |= MTL_OP_MODE_RSF | (rqs << MTL_OP_MODE_RQS_SHIFT);
+ rte_write32(value, rx_op);
+
+ /* RX queue 0 maps to DMA channel 0 */
+ value = rte_read32(rxq_map);
+ value &= ~0xfu;
+ rte_write32(value, rxq_map);
+}
+
+static void
+stmmac_mac_init(void *mac_base, struct rte_ether_addr *mac_addr)
+{
+ void *config_reg = (uint8_t *)mac_base + GMAC_CONFIG;
+ uint32_t value;
+ uint32_t addr_lo, addr_hi;
+
+ addr_lo = mac_addr->addr_bytes[0] | (mac_addr->addr_bytes[1] << 8) |
+ (mac_addr->addr_bytes[2] << 16) | (mac_addr->addr_bytes[3] << 24);
+ addr_hi = mac_addr->addr_bytes[4] | (mac_addr->addr_bytes[5] << 8);
+
+ rte_write32(addr_lo, (uint8_t *)mac_base + GMAC_ADDR_LOW(0));
+ rte_write32(addr_hi, (uint8_t *)mac_base + GMAC_ADDR_HIGH(0));
+
+ value = rte_read32(config_reg);
+ value |= GMAC_CONFIG_IPC | GMAC_CONFIG_ACS;
+ value |= GMAC_CONFIG_TE | GMAC_CONFIG_RE;
+ rte_write32(value, config_reg);
+}
+
+static void
+stmmac_dma_start(struct stmmac_priv *priv)
+{
+ uint8_t *chan = (uint8_t *)priv->mac_base + stmmac_dma_chan_base(&priv->chan_addrs, 0);
+ void *tx_ctrl = chan + DMA_CHAN_OFF_TX_CONTROL;
+ void *rx_ctrl = chan + DMA_CHAN_OFF_RX_CONTROL;
+ uint32_t value;
+
+ value = rte_read32(tx_ctrl);
+ value |= DMA_CONTROL_ST;
+ rte_write32(value, tx_ctrl);
+
+ value = rte_read32(rx_ctrl);
+ value &= ~DMA_CONTROL_RPF;
+ value |= DMA_CONTROL_SR;
+ rte_write32(value, rx_ctrl);
+}
+
+static void
+stmmac_dma_stop(struct stmmac_priv *priv)
+{
+ uint8_t *chan = (uint8_t *)priv->mac_base + stmmac_dma_chan_base(&priv->chan_addrs, 0);
+ void *tx_ctrl = chan + DMA_CHAN_OFF_TX_CONTROL;
+ void *rx_ctrl = chan + DMA_CHAN_OFF_RX_CONTROL;
+ uint32_t value;
+
+ value = rte_read32(tx_ctrl);
+ value &= ~DMA_CONTROL_ST;
+ rte_write32(value, tx_ctrl);
+
+ value = rte_read32(rx_ctrl);
+ value |= DMA_CONTROL_RPF;
+ value &= ~DMA_CONTROL_SR;
+ rte_write32(value, rx_ctrl);
+}
+
+static int
+stmmac_eth_configure(struct rte_eth_dev *dev)
+{
+ RTE_SET_USED(dev);
+ return 0;
+}
+
+static int
+stmmac_eth_start(struct rte_eth_dev *dev)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+ int ret;
+
+ if (priv->rxq[0] == NULL || priv->txq[0] == NULL) {
+ STMMAC_PMD_ERR("queues not set up");
+ return -EINVAL;
+ }
+
+ /* the RGMII clock must run before the DMA reset */
+ stmmac_rgmii_func_clk_en(priv);
+ stmmac_rgmii_id_1g_init(priv);
+ stmmac_phy_rgmii_id_init(priv);
+
+ ret = stmmac_dma_reset(priv->mac_base);
+ if (ret != 0)
+ return ret;
+
+ stmmac_dma_sysbus_init(priv->mac_base);
+ stmmac_dma_chan_cfg((uint8_t *)priv->mac_base +
+ stmmac_dma_chan_base(&priv->chan_addrs, 0));
+
+ ret = stmmac_tx_ring_setup(priv->txq[0]);
+ if (ret != 0)
+ return ret;
+
+ ret = stmmac_rx_ring_setup(priv->rxq[0]);
+ if (ret != 0)
+ return ret;
+
+ stmmac_mac_init(priv->mac_base, &priv->mac_addr);
+
+ /* enable RX queue 0 in generic (non-DCB) mode */
+ {
+ void *rxq_ctrl = (uint8_t *)priv->mac_base + GMAC_RXQ_CTRL0;
+ uint32_t v = rte_read32(rxq_ctrl);
+
+ rte_write32((v & ~0x3u) | 0x2u, rxq_ctrl);
+ }
+
+ stmmac_mtl_init(priv->mac_base, &priv->chan_addrs);
+ stmmac_dma_start(priv);
+
+ dev->rx_pkt_burst = stmmac_rx_pkt_burst_nc;
+ dev->tx_pkt_burst = stmmac_tx_pkt_burst_nc;
+
+ dev->data->dev_started = 1;
+ dev->data->rx_queue_state[0] = RTE_ETH_QUEUE_STATE_STARTED;
+ dev->data->tx_queue_state[0] = RTE_ETH_QUEUE_STATE_STARTED;
+
+ return 0;
+}
+
+static int
+stmmac_eth_stop(struct rte_eth_dev *dev)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+
+ dev->data->dev_started = 0;
+ stmmac_dma_stop(priv);
+ dev->data->rx_queue_state[0] = RTE_ETH_QUEUE_STATE_STOPPED;
+ dev->data->tx_queue_state[0] = RTE_ETH_QUEUE_STATE_STOPPED;
+ return 0;
+}
+
+static void
+stmmac_queue_release(struct stmmac_priv *priv)
+{
+ if (priv->txq[0] != NULL) {
+ rte_free(priv->txq[0]->sw_ring);
+ rte_free(priv->txq[0]);
+ priv->txq[0] = NULL;
+ }
+ if (priv->rxq[0] != NULL) {
+ unsigned int i;
+
+ for (i = 0; i < priv->rxq[0]->nb_desc; i++)
+ rte_pktmbuf_free(priv->rxq[0]->sw_ring[i]);
+ rte_free(priv->rxq[0]->sw_ring);
+ rte_free(priv->rxq[0]);
+ priv->rxq[0] = NULL;
+ }
+}
+
+static int
+stmmac_eth_close(struct rte_eth_dev *dev)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+
+ stmmac_queue_release(priv);
+
+ return 0;
+}
+
+/*
+ * Link state. The DP83867 BMSR link bit is latched low, so BMSR is read twice
+ * to clear the latch. The live link state and the speed and duplex come from
+ * PHYSTS, which is not latched. BMSR is used only if PHYSTS does not respond.
+ */
+static int
+stmmac_eth_link_update(struct rte_eth_dev *dev, int wait_to_complete)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+ struct rte_eth_link link;
+ uint16_t bmsr, physts;
+
+ RTE_SET_USED(wait_to_complete);
+
+ memset(&link, 0, sizeof(link));
+
+ /* the first read clears the latched link bit */
+ (void)stmmac_mdio_read(priv->mac_base, priv->mdio_addr, STMMAC_MII_BMSR);
+ bmsr = stmmac_mdio_read(priv->mac_base, priv->mdio_addr, STMMAC_MII_BMSR);
+ physts = stmmac_mdio_read(priv->mac_base, priv->mdio_addr, DP83867_PHYSTS);
+
+ if (physts != 0xffff)
+ priv->link_up = !!(physts & DP83867_PHYSTS_LINK_BIT);
+ else
+ priv->link_up = !!(bmsr & STMMAC_MII_BMSR_LSTATUS);
+ link.link_status = priv->link_up;
+
+ if (physts != 0xffff && priv->link_up) {
+ uint16_t spd = physts & DP83867_PHYSTS_SPEED_MASK;
+
+ if (spd == DP83867_PHYSTS_SPEED_1G)
+ link.link_speed = RTE_ETH_SPEED_NUM_1G;
+ else if (spd == DP83867_PHYSTS_SPEED_100M)
+ link.link_speed = RTE_ETH_SPEED_NUM_100M;
+ else
+ link.link_speed = RTE_ETH_SPEED_NUM_10M;
+
+ link.link_duplex = (physts & DP83867_PHYSTS_DUPLEX_BIT) ?
+ RTE_ETH_LINK_FULL_DUPLEX : RTE_ETH_LINK_HALF_DUPLEX;
+ } else {
+ link.link_speed = RTE_ETH_SPEED_NUM_1G;
+ link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ }
+
+ link.link_autoneg = RTE_ETH_LINK_AUTONEG;
+
+ return rte_eth_linkstatus_set(dev, &link);
+}
+
+static int
+stmmac_promiscuous_enable(struct rte_eth_dev *dev)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+ void *filter_reg = (uint8_t *)priv->mac_base + GMAC_PACKET_FILTER;
+
+ rte_write32(rte_read32(filter_reg) | GMAC_PACKET_FILTER_PR, filter_reg);
+
+ return 0;
+}
+
+static int
+stmmac_promiscuous_disable(struct rte_eth_dev *dev)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+ void *filter_reg = (uint8_t *)priv->mac_base + GMAC_PACKET_FILTER;
+
+ rte_write32(rte_read32(filter_reg) & ~GMAC_PACKET_FILTER_PR, filter_reg);
+
+ return 0;
+}
+
+static int
+stmmac_mac_addr_set(struct rte_eth_dev *dev, struct rte_ether_addr *addr)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+
+ rte_ether_addr_copy(addr, &priv->mac_addr);
+
+ rte_write32(addr->addr_bytes[0] | (addr->addr_bytes[1] << 8) |
+ (addr->addr_bytes[2] << 16) | (addr->addr_bytes[3] << 24),
+ (uint8_t *)priv->mac_base + GMAC_ADDR_LOW(0));
+ rte_write32(addr->addr_bytes[4] | (addr->addr_bytes[5] << 8),
+ (uint8_t *)priv->mac_base + GMAC_ADDR_HIGH(0));
+
+ return 0;
+}
+
+static int
+stmmac_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats,
+ struct eth_queue_stats *qstats)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+
+ RTE_SET_USED(qstats);
+ memcpy(stats, &priv->stats, sizeof(*stats));
+
+ /* the queue counters are updated by the burst functions */
+ if (priv->rxq[0] != NULL) {
+ stats->ipackets = priv->rxq[0]->rx_packets;
+ stats->ibytes = priv->rxq[0]->rx_bytes;
+ stats->ierrors = priv->rxq[0]->rx_errors;
+ stats->rx_nombuf = priv->rxq[0]->rx_nombuf;
+ }
+ if (priv->txq[0] != NULL) {
+ stats->opackets = priv->txq[0]->tx_packets;
+ stats->obytes = priv->txq[0]->tx_bytes;
+ stats->oerrors = priv->txq[0]->tx_errors;
+ }
+
+ return 0;
+}
+
+static int
+stmmac_stats_reset(struct rte_eth_dev *dev)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+
+ memset(&priv->stats, 0, sizeof(priv->stats));
+ if (priv->rxq[0] != NULL) {
+ priv->rxq[0]->rx_packets = 0;
+ priv->rxq[0]->rx_bytes = 0;
+ priv->rxq[0]->rx_errors = 0;
+ priv->rxq[0]->rx_nombuf = 0;
+ }
+ if (priv->txq[0] != NULL) {
+ priv->txq[0]->tx_packets = 0;
+ priv->txq[0]->tx_bytes = 0;
+ priv->txq[0]->tx_errors = 0;
+ priv->txq[0]->tx_ring_full = 0;
+ }
+
+ return 0;
+}
+
+static int
+stmmac_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ RTE_SET_USED(dev);
+
+ dev_info->max_rx_queues = STMMAC_MAX_Q;
+ dev_info->max_tx_queues = STMMAC_MAX_Q;
+ dev_info->min_rx_bufsize = RTE_ETHER_MIN_LEN;
+ dev_info->max_rx_pktlen = RTE_ETHER_MAX_LEN;
+ dev_info->rx_offload_capa = dev_rx_offloads_sup;
+ dev_info->tx_offload_capa = dev_tx_offloads_sup;
+
+ /*
+ * Ring sizes follow the Linux limits (64..1024). Sizes must be a
+ * multiple of 4, one descriptor cache line. One segment per packet.
+ */
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = STMMAC_MAX_RING_SIZE,
+ .nb_min = 64,
+ .nb_align = 4,
+ .nb_seg_max = 1,
+ .nb_mtu_seg_max = 1,
+ };
+ dev_info->tx_desc_lim = dev_info->rx_desc_lim;
+ dev_info->default_rxportconf.ring_size = STMMAC_MAX_RING_SIZE;
+ dev_info->default_txportconf.ring_size = STMMAC_MAX_RING_SIZE;
+
+ return 0;
+}
+
+static int
+stmmac_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
+ uint16_t nb_rx_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mb_pool)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+ struct stmmac_rx_queue *rxq;
+ const struct rte_memzone *mz;
+ size_t ring_size;
+
+ RTE_SET_USED(rx_conf);
+
+ if (rx_queue_id >= STMMAC_MAX_Q) {
+ STMMAC_PMD_ERR("invalid rx queue id %u, max %u",
+ rx_queue_id, STMMAC_MAX_Q);
+ return -EINVAL;
+ }
+
+ if (nb_rx_desc == 0)
+ nb_rx_desc = STMMAC_DEFAULT_RING_SIZE;
+
+ rxq = rte_zmalloc_socket("stmmac_rxq", sizeof(*rxq),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (rxq == NULL)
+ return -ENOMEM;
+
+ ring_size = nb_rx_desc * sizeof(struct stmmac_desc);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_ring", rx_queue_id, ring_size,
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (mz == NULL) {
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ rxq->sw_ring = rte_zmalloc_socket("stmmac_rx_sw_ring",
+ nb_rx_desc * sizeof(struct rte_mbuf *),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (rxq->sw_ring == NULL) {
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ rxq->rx_ring = mz->addr;
+ rxq->rx_ring_iova = mz->iova;
+ rxq->nb_desc = nb_rx_desc;
+ rxq->rx_tail = 0;
+ rxq->rx_refill = 0;
+ rxq->queue_id = rx_queue_id;
+ rxq->port_id = dev->data->port_id;
+ rxq->mb_pool = mb_pool;
+ rxq->chan_base = (uint8_t *)priv->mac_base +
+ stmmac_dma_chan_base(&priv->chan_addrs, 0);
+ rxq->mz = mz;
+
+ priv->rxq[rx_queue_id] = rxq;
+ dev->data->rx_queues[rx_queue_id] = rxq;
+ return 0;
+}
+
+static int
+stmmac_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id,
+ uint16_t nb_tx_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+ struct stmmac_tx_queue *txq;
+ const struct rte_memzone *mz;
+ size_t ring_size;
+
+ RTE_SET_USED(tx_conf);
+
+ if (tx_queue_id >= STMMAC_MAX_Q) {
+ STMMAC_PMD_ERR("invalid tx queue id %u, max %u",
+ tx_queue_id, STMMAC_MAX_Q);
+ return -EINVAL;
+ }
+
+ if (nb_tx_desc == 0)
+ nb_tx_desc = STMMAC_DEFAULT_RING_SIZE;
+ /* the TX burst writes whole descriptor cache lines: the ring must be a multiple of 4 */
+ if (nb_tx_desc % STMMAC_DESCS_PER_LINE) {
+ STMMAC_PMD_ERR("nb_tx_desc %u must be a multiple of %u",
+ nb_tx_desc, STMMAC_DESCS_PER_LINE);
+ return -EINVAL;
+ }
+
+ txq = rte_zmalloc_socket("stmmac_txq", sizeof(*txq),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (txq == NULL)
+ return -ENOMEM;
+
+ ring_size = nb_tx_desc * sizeof(struct stmmac_desc);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", tx_queue_id, ring_size,
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (mz == NULL) {
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->sw_ring = rte_zmalloc_socket("stmmac_tx_sw_ring",
+ nb_tx_desc * sizeof(struct rte_mbuf *),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (txq->sw_ring == NULL) {
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring = mz->addr;
+ txq->tx_ring_iova = mz->iova;
+ txq->nb_desc = nb_tx_desc;
+ txq->tx_tail = 0;
+ txq->tx_free = 0;
+ txq->queue_id = tx_queue_id;
+ txq->port_id = dev->data->port_id;
+ txq->chan_base = (uint8_t *)priv->mac_base +
+ stmmac_dma_chan_base(&priv->chan_addrs, 0);
+ txq->mz = mz;
+
+ priv->txq[tx_queue_id] = txq;
+ dev->data->tx_queues[tx_queue_id] = txq;
+ return 0;
+}
+
+static void
+stmmac_rxq_info_get(struct rte_eth_dev *dev, uint16_t qid,
+ struct rte_eth_rxq_info *qinfo)
+{
+ struct stmmac_rx_queue *rxq = dev->data->rx_queues[qid];
+
+ qinfo->mp = rxq->mb_pool;
+ qinfo->nb_desc = rxq->nb_desc;
+}
+
+static void
+stmmac_txq_info_get(struct rte_eth_dev *dev, uint16_t qid,
+ struct rte_eth_txq_info *qinfo)
+{
+ struct stmmac_tx_queue *txq = dev->data->tx_queues[qid];
+
+ qinfo->nb_desc = txq->nb_desc;
+}
+
+static const struct eth_dev_ops stmmac_eth_ops = {
+ .dev_configure = stmmac_eth_configure,
+ .dev_start = stmmac_eth_start,
+ .dev_stop = stmmac_eth_stop,
+ .dev_close = stmmac_eth_close,
+ .link_update = stmmac_eth_link_update,
+ .promiscuous_enable = stmmac_promiscuous_enable,
+ .promiscuous_disable = stmmac_promiscuous_disable,
+ .mac_addr_set = stmmac_mac_addr_set,
+ .stats_get = stmmac_stats_get,
+ .stats_reset = stmmac_stats_reset,
+ .dev_infos_get = stmmac_dev_infos_get,
+ .rx_queue_setup = stmmac_rx_queue_setup,
+ .tx_queue_setup = stmmac_tx_queue_setup,
+ .rxq_info_get = stmmac_rxq_info_get,
+ .txq_info_get = stmmac_txq_info_get,
+};
+
+static int
+stmmac_platform_probe(struct rte_platform_device *pdev)
+{
+ struct rte_eth_dev *dev;
+ struct stmmac_priv *priv;
+ unsigned int i;
+
+ if (pdev->num_resource < 1 || pdev->resource[0].mem.addr == NULL) {
+ STMMAC_PMD_ERR("probe failed: no MMIO resource[0] (MAC/DMA/MTL block)");
+ return -EINVAL;
+ }
+
+ dev = rte_eth_dev_allocate(pdev->device.name);
+ if (dev == NULL) {
+ STMMAC_PMD_ERR("probe failed: rte_eth_dev_allocate() returned NULL for %s",
+ pdev->device.name);
+ return -ENOMEM;
+ }
+ dev->device = &pdev->device;
+
+ priv = rte_zmalloc_socket(NULL, sizeof(*priv), RTE_CACHE_LINE_SIZE,
+ dev->data->numa_node);
+ if (priv == NULL) {
+ STMMAC_PMD_ERR("probe failed: unable to allocate stmmac_priv for %s",
+ pdev->device.name);
+ rte_eth_dev_release_port(dev);
+ return -ENOMEM;
+ }
+ dev->data->dev_private = priv;
+
+ priv->mac_base = pdev->resource[0].mem.addr;
+ /* DT reg-names = "stmmaceth", "rgmii": resource[1] is the RGMII IO macro */
+ if (pdev->num_resource > 1)
+ priv->rgmii_base = pdev->resource[1].mem.addr;
+
+ stmmac_parse_devargs(pdev, priv);
+
+ if (priv->has_emac_ge_3) {
+ struct stmmac_chan_addrs ge3 = STMMAC_CHAN_ADDRS_EMAC_GE3_INIT;
+
+ priv->chan_addrs = ge3;
+ } else {
+ struct stmmac_chan_addrs generic = STMMAC_CHAN_ADDRS_GENERIC_INIT;
+
+ priv->chan_addrs = generic;
+ }
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ STMMAC_PMD_ERR("probe failed: unable to allocate mac_addrs for %s",
+ pdev->device.name);
+ rte_free(priv);
+ rte_eth_dev_release_port(dev);
+ return -ENOMEM;
+ }
+ for (i = 0; i < RTE_ETHER_ADDR_LEN; i++)
+ dev->data->mac_addrs[0].addr_bytes[i] = 0x02;
+ rte_ether_addr_copy(&dev->data->mac_addrs[0], &priv->mac_addr);
+
+ dev->dev_ops = &stmmac_eth_ops;
+
+ rte_eth_dev_probing_finish(dev);
+
+ return 0;
+}
+
+static int
+stmmac_platform_remove(struct rte_platform_device *pdev)
+{
+ struct rte_eth_dev *dev;
+
+ dev = rte_eth_dev_allocated(pdev->device.name);
+ if (dev == NULL)
+ return 0;
+
+ stmmac_eth_close(dev);
+ rte_free(dev->data->dev_private);
+ rte_eth_dev_release_port(dev);
+
+ return 0;
+}
+
+static struct rte_platform_driver stmmac_platform_drv = {
+ .probe = stmmac_platform_probe,
+ .remove = stmmac_platform_remove,
+};
+
+RTE_PMD_REGISTER_PLATFORM(STMMAC_DRIVER_NAME, stmmac_platform_drv);
+RTE_PMD_REGISTER_ALIAS(STMMAC_DRIVER_NAME, vfio-platform);
+RTE_PMD_REGISTER_PARAM_STRING(STMMAC_DRIVER_NAME,
+ STMMAC_DEVARG_HAS_EMAC_GE_3 "=<0|1> "
+ STMMAC_DEVARG_MDIO_ADDR "=<uint16>");
diff --git a/drivers/net/stmmac/stmmac_logs.h b/drivers/net/stmmac/stmmac_logs.h
new file mode 100644
index 0000000000..37db8e3ea9
--- /dev/null
+++ b/drivers/net/stmmac/stmmac_logs.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2026 Qualcomm Technologies, Inc.
+ */
+
+#ifndef _STMMAC_LOGS_H_
+#define _STMMAC_LOGS_H_
+
+#include <rte_log.h>
+
+extern int stmmac_logtype_pmd;
+#define RTE_LOGTYPE_STMMAC_NET stmmac_logtype_pmd
+
+#define STMMAC_PMD_LOG(level, ...) \
+ RTE_LOG_LINE_PREFIX(level, STMMAC_NET, "%s()", __func__, __VA_ARGS__)
+
+#define STMMAC_PMD_DEBUG(fmt, ...) \
+ STMMAC_PMD_LOG(DEBUG, fmt, ## __VA_ARGS__)
+#define STMMAC_PMD_ERR(fmt, ...) \
+ STMMAC_PMD_LOG(ERR, fmt, ## __VA_ARGS__)
+#define STMMAC_PMD_INFO(fmt, ...) \
+ STMMAC_PMD_LOG(INFO, fmt, ## __VA_ARGS__)
+#define STMMAC_PMD_WARN(fmt, ...) \
+ STMMAC_PMD_LOG(WARNING, fmt, ## __VA_ARGS__)
+
+#endif /* _STMMAC_LOGS_H_ */
diff --git a/drivers/net/stmmac/stmmac_regs.h b/drivers/net/stmmac/stmmac_regs.h
new file mode 100644
index 0000000000..2fac1d9f21
--- /dev/null
+++ b/drivers/net/stmmac/stmmac_regs.h
@@ -0,0 +1,302 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2026 Qualcomm Technologies, Inc.
+ */
+
+#ifndef _STMMAC_REGS_H_
+#define _STMMAC_REGS_H_
+
+#include <stdint.h>
+#include <rte_bitops.h>
+#include <rte_common.h>
+#include <rte_io.h>
+
+/*
+ * Register and bitfield definitions for the Synopsys DWC EQoS GMAC4 core
+ * ("stmmac" / "dwmac4" in the Linux driver family). Offsets and bit positions
+ * follow drivers/net/ethernet/stmicro/stmmac/{dwmac4.h, dwmac4_dma.h,
+ * dwmac4_descs.h, descs.h, stmmac_mdio.c, dwmac4_core.c}.
+ */
+
+/*
+ * Per-channel DMA/MTL register stride. The generic dwmac4 layout is used by
+ * default; platforms with has_emac_ge_3 use a 0x1000 stride.
+ */
+struct stmmac_chan_addrs {
+ uint32_t dma_chan_base;
+ uint32_t dma_chan_stride;
+ uint32_t mtl_chan_base;
+ uint32_t mtl_chan_stride;
+};
+
+/* ---------------------------------------------------------------- */
+/* MAC registers */
+/* ---------------------------------------------------------------- */
+
+#define GMAC_CONFIG 0x00000000
+#define GMAC_PACKET_FILTER 0x00000008
+#define GMAC_HW_FEATURE1 0x00000120
+#define GMAC_MDIO_ADDR 0x00000200
+#define GMAC_MDIO_DATA 0x00000204
+#define GMAC_RXQ_CTRL0 0x000000a0
+#define GMAC_ADDR_HIGH(reg) (0x300 + (reg) * 8)
+#define GMAC_ADDR_LOW(reg) (0x304 + (reg) * 8)
+
+/* GMAC_CONFIG bits */
+#define GMAC_CONFIG_IPC RTE_BIT32(27)
+#define GMAC_CONFIG_ACS RTE_BIT32(20)
+#define GMAC_CONFIG_TE RTE_BIT32(1)
+#define GMAC_CONFIG_RE RTE_BIT32(0)
+
+/* GMAC_PACKET_FILTER bits */
+#define GMAC_PACKET_FILTER_PR RTE_BIT32(0)
+
+/* GMAC_HW_FEATURE1: RX/TX FIFO size encodings */
+#define GMAC_HWF1_RXFIFOSIZE_MASK 0x1fu /* [4:0] */
+#define GMAC_HWF1_TXFIFOSIZE_SHIFT 6 /* [10:6] */
+#define GMAC_HWF1_TXFIFOSIZE_MASK 0x1fu
+
+/* ---------------------------------------------------------------- */
+/* MTL registers */
+/* ---------------------------------------------------------------- */
+
+#define MTL_CHAN_BASE_ADDR 0x00000d00
+#define MTL_CHAN_BASE_OFFSET 0x40
+#define MTL_RXQ_DMA_MAP0 0x00000c30
+
+static inline uint32_t
+stmmac_mtl_chan_base(const struct stmmac_chan_addrs *addrs, uint32_t x)
+{
+ if (addrs != NULL)
+ return addrs->mtl_chan_base + x * addrs->mtl_chan_stride;
+ return MTL_CHAN_BASE_ADDR + x * MTL_CHAN_BASE_OFFSET;
+}
+
+#define MTL_CHAN_TX_OP_MODE(addrs, x) \
+ (stmmac_mtl_chan_base((addrs), (x)))
+#define MTL_CHAN_RX_OP_MODE(addrs, x) \
+ (stmmac_mtl_chan_base((addrs), (x)) + 0x30)
+
+#define MTL_OP_MODE_TSF RTE_BIT32(1)
+#define MTL_OP_MODE_TXQEN RTE_BIT32(3)
+#define MTL_OP_MODE_TXQEN_MASK (0x3u << 2)
+#define MTL_OP_MODE_RSF RTE_BIT32(5)
+#define MTL_OP_MODE_TQS_MASK (0x1ffu << 16)
+#define MTL_OP_MODE_TQS_SHIFT 16
+#define MTL_OP_MODE_RQS_MASK (0x3ffu << 20)
+#define MTL_OP_MODE_RQS_SHIFT 20
+
+/* ---------------------------------------------------------------- */
+/* DMA registers */
+/* ---------------------------------------------------------------- */
+
+#define DMA_BUS_MODE 0x00001000
+#define DMA_SYS_BUS_MODE 0x00001004
+#define DMA_AXI_BUS_MODE 0x00001028
+
+#define DMA_BUS_MODE_SFT_RESET RTE_BIT32(0)
+
+/* DMA_SYS_BUS_MODE bits */
+#define DMA_SYS_BUS_MB RTE_BIT32(14)
+#define DMA_SYS_BUS_AAL RTE_BIT32(12)
+#define DMA_SYS_BUS_EAME RTE_BIT32(11)
+#define DMA_SYS_BUS_FB RTE_BIT32(0)
+
+/* DMA_AXI_BUS_MODE bits */
+#define DMA_AXI_WR_OSR_LMT (0xfu << 24)
+#define DMA_AXI_WR_OSR_LMT_SHIFT 24
+#define DMA_AXI_RD_OSR_LMT (0xfu << 16)
+#define DMA_AXI_RD_OSR_LMT_SHIFT 16
+#define DMA_AXI_BLEN16 RTE_BIT32(3)
+#define DMA_AXI_OSR_MAX 0xf
+
+/* Per-channel DMA registers */
+#define DMA_CHAN_BASE_ADDR 0x00001100
+#define DMA_CHAN_BASE_OFFSET 0x80
+
+static inline uint32_t
+stmmac_dma_chan_base(const struct stmmac_chan_addrs *addrs, uint32_t x)
+{
+ if (addrs != NULL)
+ return addrs->dma_chan_base + x * addrs->dma_chan_stride;
+ return DMA_CHAN_BASE_ADDR + x * DMA_CHAN_BASE_OFFSET;
+}
+
+/*
+ * Offsets relative to a resolved per-channel base (chan_base already holds
+ * mac_base + stmmac_dma_chan_base(addrs, 0)).
+ */
+#define DMA_CHAN_OFF_CONTROL 0x00
+#define DMA_CHAN_OFF_TX_CONTROL 0x04
+#define DMA_CHAN_OFF_RX_CONTROL 0x08
+#define DMA_CHAN_OFF_TX_BASE_ADDR_HI 0x10
+#define DMA_CHAN_OFF_TX_BASE_ADDR 0x14
+#define DMA_CHAN_OFF_RX_BASE_ADDR_HI 0x18
+#define DMA_CHAN_OFF_RX_BASE_ADDR 0x1c
+#define DMA_CHAN_OFF_RX_END_ADDR_HI 0x24
+#define DMA_CHAN_OFF_TX_END_ADDR 0x20
+#define DMA_CHAN_OFF_RX_END_ADDR 0x28
+#define DMA_CHAN_OFF_TX_RING_LEN 0x2c
+#define DMA_CHAN_OFF_RX_RING_LEN 0x30
+#define DMA_CHAN_OFF_CURTXD 0x44
+#define DMA_CHAN_OFF_STATUS 0x60
+
+#define DMA_CHAN_TX_CONTROL_OSP RTE_BIT32(4)
+
+/* DMA_CHAN_CONTROL / TX / RX control bits */
+#define DMA_CONTROL_PBLX8 RTE_BIT32(16)
+#define DMA_CONTROL_ST RTE_BIT32(0)
+#define DMA_CONTROL_RPF RTE_BIT32(31)
+#define DMA_CONTROL_SR RTE_BIT32(0)
+#define DMA_CONTROL_PBL_SHIFT 16
+#define DMA_CONTROL_PBL_MASK (0x3fu << DMA_CONTROL_PBL_SHIFT)
+#define DMA_CONTROL_RBSZ_SHIFT 1
+#define DMA_CONTROL_RBSZ_MASK (0x3fffu << DMA_CONTROL_RBSZ_SHIFT)
+
+/* DMA_CHAN_STATUS bits */
+#define DMA_CHAN_STATUS_TI RTE_BIT32(0)
+#define DMA_CHAN_STATUS_RI RTE_BIT32(6)
+#define DMA_CHAN_STATUS_RBU RTE_BIT32(7)
+
+/*
+ * The Shikra EMAC is not DMA-coherent: CPU writes to descriptors must be
+ * cleaned to the point of coherency before the DMA reads them, and lines must
+ * be invalidated before the CPU reads DMA write-back.
+ */
+#define STMMAC_CACHE_LINE 64
+/* 16-byte descriptors, four per cache line; the ring base is 64-byte aligned */
+#define STMMAC_DESCS_PER_LINE (STMMAC_CACHE_LINE / 16)
+
+/*
+ * Cache maintenance for a non-coherent DMA (arm64). Clean writes CPU-written
+ * lines back so the DMA reads them; flush also discards CPU copies before the
+ * CPU reads DMA write-back. dsb orders the maintenance before the DMA sees it.
+ */
+static inline void
+stmmac_dcache_clean(const void *addr, size_t len)
+{
+ uintptr_t p = (uintptr_t)addr & ~(uintptr_t)(STMMAC_CACHE_LINE - 1);
+ uintptr_t end = (uintptr_t)addr + len;
+
+ for (; p < end; p += STMMAC_CACHE_LINE)
+ asm volatile("dc cvac, %0" : : "r"(p) : "memory");
+ asm volatile("dsb sy" : : : "memory");
+}
+
+static inline void
+stmmac_dcache_flush(const void *addr, size_t len)
+{
+ uintptr_t p = (uintptr_t)addr & ~(uintptr_t)(STMMAC_CACHE_LINE - 1);
+ uintptr_t end = (uintptr_t)addr + len;
+
+ for (; p < end; p += STMMAC_CACHE_LINE)
+ asm volatile("dc civac, %0" : : "r"(p) : "memory");
+ asm volatile("dsb sy" : : : "memory");
+}
+
+static inline void
+stmmac_dcache_inval_line(const void *addr)
+{
+ asm volatile("dc civac, %0" : : "r"(addr) : "memory");
+ rte_io_rmb();
+}
+
+/* ---------------------------------------------------------------- */
+/* TX descriptor bits */
+/* ---------------------------------------------------------------- */
+
+/* TDES2 (read format) */
+#define TDES2_BUFFER1_SIZE_MASK 0x3fffu
+#define TDES2_INTERRUPT_ON_COMPLETION RTE_BIT32(31)
+
+/* TDES3 (read format) */
+#define TDES3_PACKET_SIZE_MASK 0x7fffu
+#define TDES3_CHECKSUM_INSERTION_MASK (0x3u << 16)
+#define TDES3_CHECKSUM_INSERTION_SHIFT 16
+
+/* Insert IP header and pseudo-header checksums */
+#define TX_CIC_FULL 3
+
+/* TDES3 (write-back and common) */
+#define TDES3_ERROR_SUMMARY RTE_BIT32(15)
+#define TDES3_LAST_DESCRIPTOR RTE_BIT32(28)
+#define TDES3_FIRST_DESCRIPTOR RTE_BIT32(29)
+#define TDES3_CONTEXT_TYPE RTE_BIT32(30)
+#define TDES3_OWN RTE_BIT32(31)
+
+/* ---------------------------------------------------------------- */
+/* RX descriptor bits */
+/* ---------------------------------------------------------------- */
+
+/* RDES1 (write-back format) */
+#define RDES1_IP_HDR_ERROR RTE_BIT32(3)
+#define RDES1_IPV4_HEADER RTE_BIT32(4)
+#define RDES1_IPV6_HEADER RTE_BIT32(5)
+#define RDES1_IP_CSUM_BYPASSED RTE_BIT32(6)
+#define RDES1_IP_CSUM_ERROR RTE_BIT32(7)
+
+/* RDES3 (write-back format) */
+#define RDES3_PACKET_SIZE_MASK 0x7fffu
+#define RDES3_ERROR_SUMMARY RTE_BIT32(15)
+#define RDES3_OVERFLOW_ERROR RTE_BIT32(21)
+#define RDES3_CRC_ERROR RTE_BIT32(24)
+#define RDES3_LAST_DESCRIPTOR RTE_BIT32(28)
+#define RDES3_FIRST_DESCRIPTOR RTE_BIT32(29)
+#define RDES3_CONTEXT_DESCRIPTOR RTE_BIT32(30)
+
+/* RDES3 (read format) */
+#define RDES3_BUFFER1_VALID_ADDR RTE_BIT32(24)
+#define RDES3_INT_ON_COMPLETION_EN RTE_BIT32(30)
+#define RDES3_OWN RTE_BIT32(31)
+
+/* ---------------------------------------------------------------- */
+/* MDIO */
+/* ---------------------------------------------------------------- */
+
+#define MII_BUSY RTE_BIT32(0)
+#define MII_DATA_MASK 0xffffu
+
+#define MII_GMAC4_GOC_SHIFT 2
+#define MII_GMAC4_WRITE (1 << MII_GMAC4_GOC_SHIFT)
+#define MII_GMAC4_READ (3 << MII_GMAC4_GOC_SHIFT)
+
+#define MII_ADDR_SHIFT 21
+#define MII_ADDR_MASK (0x1fu << MII_ADDR_SHIFT)
+#define MII_REG_SHIFT 16
+#define MII_REG_MASK (0x1fu << MII_REG_SHIFT)
+
+/* IEEE 802.3 clause-22 registers */
+#define STMMAC_MII_BMSR 0x01
+#define STMMAC_MII_BMSR_LSTATUS RTE_BIT32(2)
+
+/*
+ * DP83867 PHY registers (clause-22 direct access).
+ * PHYSTS bits: 15:14 speed (11 = 1000, 10 = 100, 01 = 10 Mb/s),
+ * 13 duplex (1 = full), 10 link (not latched).
+ */
+#define DP83867_PHYSTS 0x11
+#define DP83867_PHYSTS_LINK_BIT RTE_BIT32(10)
+#define DP83867_PHYSTS_SPEED_MASK (RTE_BIT32(15) | RTE_BIT32(14))
+#define DP83867_PHYSTS_SPEED_1G RTE_BIT32(15)
+#define DP83867_PHYSTS_SPEED_100M RTE_BIT32(14)
+#define DP83867_PHYSTS_DUPLEX_BIT RTE_BIT32(13)
+
+/* ---------------------------------------------------------------- */
+/* Channel stride presets */
+/* ---------------------------------------------------------------- */
+
+#define STMMAC_CHAN_ADDRS_GENERIC_INIT { \
+ .dma_chan_base = DMA_CHAN_BASE_ADDR, \
+ .dma_chan_stride = DMA_CHAN_BASE_OFFSET, \
+ .mtl_chan_base = MTL_CHAN_BASE_ADDR, \
+ .mtl_chan_stride = MTL_CHAN_BASE_OFFSET, \
+}
+
+/* has_emac_ge_3 layout (dwmac-qcom-ethqos.c emac_v4_0_0_data) */
+#define STMMAC_CHAN_ADDRS_EMAC_GE3_INIT { \
+ .dma_chan_base = 0x8100, \
+ .dma_chan_stride = 0x1000, \
+ .mtl_chan_base = 0x8000, \
+ .mtl_chan_stride = 0x1000, \
+}
+
+#endif /* _STMMAC_REGS_H_ */
diff --git a/drivers/net/stmmac/stmmac_rxtx.c b/drivers/net/stmmac/stmmac_rxtx.c
new file mode 100644
index 0000000000..61f7249759
--- /dev/null
+++ b/drivers/net/stmmac/stmmac_rxtx.c
@@ -0,0 +1,416 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2026 Qualcomm Technologies, Inc.
+ */
+
+#include <rte_branch_prediction.h>
+#include <rte_common.h>
+#include <rte_cycles.h>
+#include <rte_io.h>
+#include <rte_mbuf.h>
+
+#include "stmmac.h"
+#include "stmmac_regs.h"
+#include "stmmac_logs.h"
+
+#define STMMAC_RX_BUNDLE 32
+#define STMMAC_TX_BUNDLE 32
+
+/* MDIO busy-poll bound, matching stmmac_mdio.c (10000 x 1 us) */
+#define STMMAC_MDIO_TIMEOUT_US 10000
+
+/* Index of the TX descriptor the DMA is working on: everything before it is sent. */
+static inline uint16_t
+stmmac_tx_dma_slot(struct stmmac_tx_queue *txq)
+{
+ uint32_t cur = rte_read32((uint8_t *)txq->chan_base + DMA_CHAN_OFF_CURTXD);
+
+ return (uint16_t)(((cur - (uint32_t)txq->tx_ring_iova) /
+ sizeof(struct stmmac_desc)) % txq->nb_desc);
+}
+
+/*
+ * MDIO clause-22 read. Uses the GMAC4 opcode and the address/register field
+ * layout from dwmac4_core.c. The CSR clock divider is left at zero.
+ */
+uint16_t
+stmmac_mdio_read(void *mac_base, uint16_t mdio_addr, uint16_t phyreg)
+{
+ void *addr_reg = (uint8_t *)mac_base + GMAC_MDIO_ADDR;
+ void *data_reg = (uint8_t *)mac_base + GMAC_MDIO_DATA;
+ uint32_t value;
+ unsigned int timeout = STMMAC_MDIO_TIMEOUT_US;
+
+ while ((rte_read32(addr_reg) & MII_BUSY) && timeout > 0) {
+ rte_delay_us(1);
+ timeout--;
+ }
+ if (timeout == 0) {
+ STMMAC_PMD_ERR("MDIO busy timeout before read");
+ return 0xffff;
+ }
+
+ value = MII_GMAC4_READ | MII_BUSY;
+ value |= ((uint32_t)mdio_addr << MII_ADDR_SHIFT) & MII_ADDR_MASK;
+ value |= ((uint32_t)phyreg << MII_REG_SHIFT) & MII_REG_MASK;
+
+ rte_write32(value, addr_reg);
+
+ timeout = STMMAC_MDIO_TIMEOUT_US;
+ while ((rte_read32(addr_reg) & MII_BUSY) && timeout > 0) {
+ rte_delay_us(1);
+ timeout--;
+ }
+ if (timeout == 0) {
+ STMMAC_PMD_ERR("MDIO busy timeout after read");
+ return 0xffff;
+ }
+
+ return (uint16_t)(rte_read32(data_reg) & MII_DATA_MASK);
+}
+
+void
+stmmac_mdio_write(void *mac_base, uint16_t mdio_addr, uint16_t phyreg,
+ uint16_t phydata)
+{
+ void *addr_reg = (uint8_t *)mac_base + GMAC_MDIO_ADDR;
+ void *data_reg = (uint8_t *)mac_base + GMAC_MDIO_DATA;
+ uint32_t value;
+ unsigned int timeout = STMMAC_MDIO_TIMEOUT_US;
+
+ while ((rte_read32(addr_reg) & MII_BUSY) && timeout > 0) {
+ rte_delay_us(1);
+ timeout--;
+ }
+ if (timeout == 0) {
+ STMMAC_PMD_ERR("MDIO busy timeout before write");
+ return;
+ }
+
+ rte_write32(phydata, data_reg);
+
+ value = MII_GMAC4_WRITE | MII_BUSY;
+ value |= ((uint32_t)mdio_addr << MII_ADDR_SHIFT) & MII_ADDR_MASK;
+ value |= ((uint32_t)phyreg << MII_REG_SHIFT) & MII_REG_MASK;
+
+ rte_write32(value, addr_reg);
+
+ timeout = STMMAC_MDIO_TIMEOUT_US;
+ while ((rte_read32(addr_reg) & MII_BUSY) && timeout > 0) {
+ rte_delay_us(1);
+ timeout--;
+ }
+ if (timeout == 0)
+ STMMAC_PMD_ERR("MDIO busy timeout after write");
+}
+
+/*
+ * Refill 'cleaned_cnt' RX descriptors starting at rx_refill. Mbufs are
+ * allocated in bulk, descriptors are written, and each cache line of
+ * descriptors is cleaned once. The DMA doorbell moves to the last descriptor
+ * handed back. Returns the number refilled; a short count means the mbuf pool
+ * ran out, and the remaining slots are retried on the next burst.
+ */
+static uint16_t
+stmmac_refill_rx_ring(struct stmmac_rx_queue *rxq, uint16_t cleaned_cnt)
+{
+ struct rte_mbuf *m[STMMAC_RX_BUNDLE];
+ struct stmmac_desc *grp_start = NULL;
+ uint16_t i = rxq->rx_refill;
+ uint16_t j;
+ uint16_t k = STMMAC_RX_BUNDLE;
+ uint16_t grp_cnt = 0;
+
+ for (j = 0; j < cleaned_cnt; j++) {
+ struct stmmac_desc *desc = &rxq->rx_ring[i];
+ struct rte_mbuf *mb;
+ uint64_t iova;
+
+ /* bulk allocation, sized to the descriptors still to refill */
+ if (k == STMMAC_RX_BUNDLE) {
+ uint16_t m_cnt = RTE_MIN(cleaned_cnt - j, STMMAC_RX_BUNDLE);
+
+ if (rte_pktmbuf_alloc_bulk(rxq->mb_pool, m, m_cnt)) {
+ rxq->rx_nombuf++;
+ break;
+ }
+ k = 0;
+ }
+ mb = m[k++];
+
+ rxq->sw_ring[i] = mb;
+ /* drop stale CPU lines over the buffer the DMA is about to write */
+ stmmac_dcache_flush(rte_pktmbuf_mtod(mb, void *),
+ rte_pktmbuf_data_room_size(rxq->mb_pool) -
+ RTE_PKTMBUF_HEADROOM);
+
+ /* full 36-bit buffer address: the mbufs live above 4 GiB */
+ iova = rte_pktmbuf_iova(mb);
+ desc->des[0] = (uint32_t)iova;
+ desc->des[1] = (uint32_t)(iova >> 32);
+ desc->des[2] = 0;
+ rte_io_wmb();
+ desc->des[3] = RDES3_OWN | RDES3_BUFFER1_VALID_ADDR |
+ RDES3_INT_ON_COMPLETION_EN;
+
+ /* clean the descriptors one cache line at a time */
+ if (grp_cnt == 0)
+ grp_start = desc;
+ grp_cnt++;
+ if (grp_cnt == STMMAC_DESCS_PER_LINE || i == rxq->nb_desc - 1) {
+ stmmac_dcache_clean(grp_start, grp_cnt * sizeof(*desc));
+ grp_cnt = 0;
+ }
+
+ i = (i + 1) % rxq->nb_desc;
+ }
+
+ /* clean the partial group left by the last descriptor */
+ if (grp_cnt)
+ stmmac_dcache_clean(grp_start, grp_cnt * sizeof(struct stmmac_desc));
+
+ rxq->rx_refill = i;
+
+ if (j > 0) {
+ uint16_t last = (i + rxq->nb_desc - 1) % rxq->nb_desc;
+ uint64_t tail_iova = rxq->rx_ring_iova +
+ (uint64_t)last * sizeof(struct stmmac_desc);
+
+ /*
+ * Clear RBU if the DMA suspended on an unavailable descriptor. This
+ * is a write, not a status read: a register read stalls the core.
+ * The DMA only suspends after it has used every slot refilled so
+ * far, so the burst that refills again is where RBU is cleared.
+ */
+ rte_write32(DMA_CHAN_STATUS_RBU,
+ (uint8_t *)rxq->chan_base + DMA_CHAN_OFF_STATUS);
+
+ /*
+ * The doorbell is the last descriptor handed back. Writing the
+ * refill index instead would make RXTAIL equal CURRXD, which the
+ * DMA treats as an empty ring. The 36-bit tail is written HI first.
+ */
+ rte_write32((uint32_t)(tail_iova >> 32),
+ (uint8_t *)rxq->chan_base + DMA_CHAN_OFF_RX_END_ADDR_HI);
+ rte_io_wmb();
+ rte_write32((uint32_t)tail_iova,
+ (uint8_t *)rxq->chan_base + DMA_CHAN_OFF_RX_END_ADDR);
+ }
+
+ return j;
+}
+
+uint16_t
+stmmac_rx_pkt_burst_nc(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ struct stmmac_rx_queue *rxq = rx_queue;
+ uint16_t nb_rx = 0;
+ uint16_t start = rxq->rx_tail;
+ uint16_t tail = start;
+ uint16_t pending;
+
+ while (nb_rx < nb_pkts) {
+ struct stmmac_desc *desc = &rxq->rx_ring[tail];
+ struct rte_mbuf *rxm;
+ uint32_t des3;
+ uint32_t pkt_len;
+
+ /* slot is waiting for a refill (mbuf allocation failed) */
+ if (rxq->sw_ring[tail] == NULL)
+ break;
+
+ /*
+ * Shikra is not DMA-coherent: invalidate the descriptor line once
+ * per cache line, at the start of the burst and at each line
+ * boundary. A stale OWN bit on a later slot only defers that frame
+ * to the next burst.
+ */
+ if (tail == start || tail % STMMAC_DESCS_PER_LINE == 0)
+ stmmac_dcache_inval_line(desc);
+ des3 = rte_read32_relaxed(&desc->des[3]);
+
+ if (des3 & RDES3_OWN)
+ break;
+
+ if (des3 & (RDES3_CONTEXT_DESCRIPTOR | RDES3_ERROR_SUMMARY)) {
+ /* not a frame to deliver: free its buffer, refill allocates a new one */
+ if (des3 & RDES3_ERROR_SUMMARY)
+ rxq->rx_errors++;
+ rte_pktmbuf_free(rxq->sw_ring[tail]);
+ rxq->sw_ring[tail] = NULL;
+ tail = (tail + 1) % rxq->nb_desc;
+ continue;
+ }
+
+ rxm = rxq->sw_ring[tail];
+ rxq->sw_ring[tail] = NULL;
+ pkt_len = des3 & RDES3_PACKET_SIZE_MASK;
+
+ /* drop stale CPU lines over the received payload */
+ stmmac_dcache_flush((uint8_t *)rxm->buf_addr + RTE_PKTMBUF_HEADROOM,
+ pkt_len);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->port = rxq->port_id;
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->ol_flags = 0;
+
+ if (desc->des[1] & (RDES1_IPV4_HEADER | RDES1_IPV6_HEADER)) {
+ if (desc->des[1] & RDES1_IP_CSUM_ERROR)
+ rxm->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ rxm->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+ }
+
+ rx_pkts[nb_rx++] = rxm;
+ rxq->rx_packets++;
+ rxq->rx_bytes += pkt_len;
+
+ tail = (tail + 1) % rxq->nb_desc;
+ }
+
+ rxq->rx_tail = tail;
+
+ /* refill what was consumed, plus any slots a failed allocation left pending */
+ pending = (tail + rxq->nb_desc - rxq->rx_refill) % rxq->nb_desc;
+ if (pending)
+ stmmac_refill_rx_ring(rxq, pending);
+
+ return nb_rx;
+}
+
+/*
+ * Reclaim sent TX frames in bulk, up to the descriptor the DMA is working on.
+ * Everything before that slot has been transmitted.
+ */
+static uint16_t
+stmmac_clean_tx_ring(struct stmmac_tx_queue *txq)
+{
+ struct rte_mbuf *m[STMMAC_TX_BUNDLE];
+ uint16_t hwci = stmmac_tx_dma_slot(txq);
+ uint16_t i = txq->tx_free;
+ uint16_t n = 0;
+ uint16_t total = 0;
+
+ while (i != hwci && txq->sw_ring[i] != NULL) {
+ if (i == txq->tx_free || i % STMMAC_DESCS_PER_LINE == 0)
+ stmmac_dcache_inval_line(&txq->tx_ring[i]);
+ if (rte_read32_relaxed(&txq->tx_ring[i].des[3]) & TDES3_ERROR_SUMMARY)
+ txq->tx_errors++;
+
+ m[n++] = txq->sw_ring[i];
+ txq->sw_ring[i] = NULL;
+ if (n == STMMAC_TX_BUNDLE) {
+ rte_pktmbuf_free_bulk(m, n);
+ total += n;
+ n = 0;
+ }
+ i = (i + 1) % txq->nb_desc;
+ }
+
+ if (n) {
+ rte_pktmbuf_free_bulk(m, n);
+ total += n;
+ }
+ txq->tx_free = i;
+ return total;
+}
+
+/* Clean the descriptor lines that hold 'cnt' slots from 'first', once, before the doorbell */
+static void
+stmmac_tx_clean_range(struct stmmac_tx_queue *txq, uint16_t first, uint16_t cnt)
+{
+ uint16_t s = first - first % STMMAC_DESCS_PER_LINE;
+ uint16_t left = cnt + (first - s);
+
+ left = (left + STMMAC_DESCS_PER_LINE - 1) / STMMAC_DESCS_PER_LINE *
+ STMMAC_DESCS_PER_LINE;
+ if (left > txq->nb_desc)
+ left = txq->nb_desc;
+
+ while (left) {
+ uint16_t run = RTE_MIN((uint16_t)(txq->nb_desc - s), left);
+
+ stmmac_dcache_clean(&txq->tx_ring[s], run * sizeof(struct stmmac_desc));
+ s = (s + run) % txq->nb_desc;
+ left -= run;
+ }
+}
+
+uint16_t
+stmmac_tx_pkt_burst_nc(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct stmmac_tx_queue *txq = tx_queue;
+ uint16_t nb_tx = 0;
+ uint16_t first = txq->tx_tail;
+ uint16_t tail = first;
+
+ stmmac_clean_tx_ring(txq);
+
+ /* drop the CPU copy of the first line before filling into it */
+ stmmac_dcache_inval_line(&txq->tx_ring[first - first % STMMAC_DESCS_PER_LINE]);
+
+ while (nb_tx < nb_pkts) {
+ struct stmmac_desc *desc = &txq->tx_ring[tail];
+ struct rte_mbuf *txm = tx_pkts[nb_tx];
+ uint32_t tdes3;
+ uint64_t iova;
+
+ if (unlikely(txm->nb_segs != 1)) {
+ STMMAC_PMD_ERR("multi-segment mbuf not supported");
+ break;
+ }
+
+ /* ring full: this slot still holds a frame the DMA has not finished */
+ if (txq->sw_ring[tail] != NULL) {
+ txq->tx_ring_full++;
+ break;
+ }
+
+ /* entering a new line: drop the CPU copy before writing into it */
+ if (tail != first && tail % STMMAC_DESCS_PER_LINE == 0)
+ stmmac_dcache_inval_line(desc);
+
+ /* the CPU-written payload must reach memory before the DMA reads it */
+ stmmac_dcache_clean(rte_pktmbuf_mtod(txm, void *),
+ rte_pktmbuf_data_len(txm));
+
+ iova = rte_pktmbuf_iova(txm);
+ desc->des[0] = (uint32_t)iova;
+ desc->des[1] = (uint32_t)(iova >> 32);
+ desc->des[2] = rte_pktmbuf_data_len(txm) & TDES2_BUFFER1_SIZE_MASK;
+
+ tdes3 = rte_pktmbuf_pkt_len(txm) & TDES3_PACKET_SIZE_MASK;
+ tdes3 |= TDES3_FIRST_DESCRIPTOR | TDES3_LAST_DESCRIPTOR;
+ if (txm->ol_flags & (RTE_MBUF_F_TX_IP_CKSUM | RTE_MBUF_F_TX_L4_MASK))
+ tdes3 |= (TX_CIC_FULL << TDES3_CHECKSUM_INSERTION_SHIFT);
+
+ txq->sw_ring[tail] = txm;
+
+ rte_io_wmb();
+ desc->des[3] = tdes3 | TDES3_OWN;
+
+ txq->tx_packets++;
+ txq->tx_bytes += rte_pktmbuf_pkt_len(txm);
+
+ tail = (tail + 1) % txq->nb_desc;
+ nb_tx++;
+ }
+
+ if (nb_tx > 0) {
+ uint64_t tail_iova = txq->tx_ring_iova +
+ (uint64_t)tail * sizeof(struct stmmac_desc);
+
+ /* the written descriptor lines are cleaned once, then the doorbell moves */
+ stmmac_tx_clean_range(txq, first, nb_tx);
+ txq->tx_tail = tail;
+ rte_write32((uint32_t)tail_iova,
+ (uint8_t *)txq->chan_base + DMA_CHAN_OFF_TX_END_ADDR);
+ }
+
+ return nb_tx;
+}
--
2.34.1
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [PATCH] net/stmmac: add PMD for Synopsys DWC EQoS (GMAC4)
2026-11-14 14:55 [PATCH] net/stmmac: add PMD for Synopsys DWC EQoS (GMAC4) Kiran Patchala
@ 2026-11-17 18:42 ` Stephen Hemminger
0 siblings, 0 replies; 2+ messages in thread
From: Stephen Hemminger @ 2026-11-17 18:42 UTC (permalink / raw)
To: Kiran Patchala; +Cc: dev, Mohan Degalwade
On Sat, 14 Nov 2026 20:25:39 +0530
Kiran Patchala <kiran.patchala@oss.qualcomm.com> wrote:
> From: Kiran Patchala <kiran.patchala@oss.qualcomm.com>
>
> Add a new poll-mode driver for the Synopsys DesignWare Cores Enhanced
> Quality-of-Service (DWC EQoS / GMAC4, "stmmac") Ethernet MAC present.
>
> The device is enumerated by the platform bus and bound to vfio-platform.
> The kernel recovery driver handles clock and PHY reset before the PMD
> takes over.
>
> Driver features:
> - One TX queue and one RX queue on DMA channel 0; single-segment mbufs
> only; no TSO, PTP, or secondary process support.
> - Non-coherent DMA: DC CVAC/CIVAC cache maintenance on arm64 for
> descriptor and payload lines; no-op stubs on other architectures.
> - Ethernet ops: configure, start, stop, close, link_update,
> promiscuous enable/disable, mac_addr_set, stats_get, stats_reset,
> dev_infos_get, rx/tx_queue_setup, rxq/txq_info_get.
>
> Limitations:
> - Single DMA channel (queue 0 only); no multi-queue support.
> - Single-segment mbufs; scatter-gather, TSO and jumbo frames are not
> supported.
> - Tested on arm64 only.
Lots of things still need fixing before this is ready.
See AI review.
It is also generally preferable to split new driver across multiple patches
to allow for easier human review. AI doesn't seem to get overloaded.
Subject: Re: [PATCH] net/stmmac: add PMD for Synopsys DWC EQoS (GMAC4)
Thanks for the driver. Several correctness problems need fixing before
this can go in; most are in the non-coherent DMA handling and the
start/stop/close lifecycle.
Series-level:
The driver is named after the generic Synopsys IP but hardcodes the
Qualcomm RGMII IO macro, the TI DP83867 PHY, and 1G rgmii-id timing.
Either name it for what it supports (qcom ethqos) or split the board
specifics out so another dwmac4 integration can use it. The
has_emac_ge_3 devarg defaulting to 1 is a symptom of this; the layout
should come from the device (compatible string / HW version register),
not from the user.
The docs say a "Qualcomm stmmac recovery kernel driver" must be loaded
and stay loaded, but the device is bound to vfio-platform. Explain how
both coexist, and point to where that kernel module lives. A PMD that
depends on an out-of-tree kernel module is a hard sell.
Missing: doc/guides/nics/features/stmmac.ini, release notes entry,
MAINTAINERS entry.
Error
-----
1. Build breaks on every non-arm64 target.
stmmac_regs.h uses "dc cvac" / "dc civac" / "dsb sy" inline asm
unconditionally, and meson.build only checks is_linux. The commit
message and docs claim "no-op stubs on other architectures"; there are
none. No-op stubs would be wrong anyway for a non-coherent device.
Restrict the build:
if not is_linux or arch_subdir != 'arm' or not dpdk_conf.get('RTE_ARCH_64')
build = false
reason = 'only supported on Linux arm64'
subdir_done()
endif
and fix the commit message and doc text.
2. Descriptor cache maintenance races with DMA write-back.
Descriptors are 16 bytes, four per 64-byte line. Ownership is per
descriptor but cache maintenance is per line. On RX, refill writes and
cleans (dc cvac) a line whose other descriptors may be DMA-owned and
being written back by the DMA at the same time. The clean writes the
CPU's stale copy of those neighbours (OWN=1, no length) over the DMA
write-back, losing received frames. TX has the same problem:
stmmac_tx_pkt_burst_nc() invalidates/cleans the line containing
'first', which can hold descriptors the DMA is still completing.
The kernel avoids this by putting rings in coherent (uncached) memory.
In DPDK the simplest fix is to make each descriptor occupy its own
cache line with the DSL (descriptor skip length) field in
DMA_CHAN_CONTROL bits 20:18, e.g. DSL=6 on a 64-bit bus gives a
64-byte stride, and index the ring in 64-byte steps. Alternatively,
only refill/post whole lines, but that does not help the TX side.
Also, stmmac_dcache_inval_line() follows dc civac with rte_io_rmb()
(dmb). The kernel's dcache_inval_poc uses dsb sy; use dsb here too.
3. TX ring can be filled completely, then the queue deadlocks.
The only full check is sw_ring[tail] != NULL, so the burst can post all
nb_desc slots. Then tail == tx_free and the doorbell value equals the
DMA's current descriptor, which the DMA treats as an empty ring.
stmmac_clean_tx_ring() stops at i == hwci, so nothing is ever
reclaimed. Keep one slot free:
uint16_t avail = (txq->tx_free + txq->nb_desc - txq->tx_tail - 1)
% txq->nb_desc;
nb_pkts = RTE_MIN(nb_pkts, avail);
4. Stop/start leaks mbufs and desyncs ring indices.
stmmac_rx_ring_setup() allocates nb_desc new mbufs on every dev_start
without freeing the ones from the previous start, and does not reset
rx_tail/rx_refill. stmmac_tx_ring_setup() does not free mbufs left in
sw_ring or reset tx_tail/tx_free, while the DMA soft reset puts the
hardware back at slot 0. "port stop 0; port start 0" in testpmd leaks
a ring's worth of mbufs and leaves TX/RX pointing at the wrong slots.
Free outstanding mbufs in dev_stop (after stopping the DMA), and reset
the indices in the ring setup functions. On allocation failure partway
through stmmac_rx_ring_setup() the mbufs already allocated also leak.
5. Double free of dev_private.
rte_eth_dev_release_port() frees dev->data->dev_private (and
mac_addrs) in the primary process. stmmac_platform_remove() and the
mac_addrs failure path in probe both rte_free(priv) first and then call
rte_eth_dev_release_port() with dev_private still set. Drop the
explicit rte_free(), or allocate with rte_eth_dev_create() which
handles this.
6. tx_burst returns short on a multi-segment mbuf.
A short return means "ring full, retry"; the application will resubmit
the same mbuf forever. Free it, count it in tx_errors, and continue.
Better, advertise no MULTI_SEGS offload and provide tx_pkt_prepare to
reject it there.
7. doc/guides/nics/index.rst uses a tab where the other entries use
spaces.
Warning
-------
8. RX checksum flags use the wrong bits.
In dwmac4 RDES1, bit 3 is the IP header error and bit 7 is the IP
payload (L4) checksum error. The driver ignores bit 3, reports bit 7 as
RTE_MBUF_F_RX_IP_CKSUM_BAD, and sets IP_CKSUM_GOOD for IPv6 which has
no header checksum. Suggested:
uint32_t des1 = desc->des[1];
if (des1 & RDES1_IPV4_HEADER)
rxm->ol_flags |= (des1 & RDES1_IP_HDR_ERROR) ?
RTE_MBUF_F_RX_IP_CKSUM_BAD :
RTE_MBUF_F_RX_IP_CKSUM_GOOD;
if ((des1 & (RDES1_IPV4_HEADER | RDES1_IPV6_HEADER)) &&
!(des1 & RDES1_IP_CSUM_BYPASSED))
rxm->ol_flags |= (des1 & RDES1_IP_CSUM_ERROR) ?
RTE_MBUF_F_RX_L4_CKSUM_BAD :
RTE_MBUF_F_RX_L4_CKSUM_GOOD;
and advertise the L4 RX offloads if you report them. Also add
rte_io_rmb() after the OWN check so des[1] is not read ahead of des[3].
9. No check that a frame fits in one RX buffer.
If data_room - headroom is less than the max frame, the DMA splits the
frame across descriptors and each piece is delivered as a separate
packet; FD/LD are never checked. Reject such mempools in
rx_queue_setup, and drop anything without both FD and LD set.
10. MAC duplex and speed are never programmed.
The DMA soft reset clears GMAC_CONFIG, and nothing sets GMAC_CONFIG_DM
(full duplex) or PS/FES. The MAC runs half duplex on a full duplex
link. Set them from PHYSTS. The RGMII IO macro is likewise programmed
for 1G only; either handle 10/100 or document that only 1G works.
11. Link down reports 1G full duplex.
Use RTE_ETH_SPEED_NUM_NONE and leave duplex at half when link is down.
12. Every port gets MAC 02:02:02:02:02:02.
Read the address left in GMAC_ADDR_HIGH/LOW(0) by firmware, and fall
back to rte_eth_random_addr() if it is invalid.
13. No queue release ops; memzones never freed.
Without rx_queue_release/tx_queue_release, a second queue_setup (port
reconfigure) leaks the previous queue, its sw_ring and its mbufs. The
error paths in queue_setup leak the memzone, and close never frees
either memzone. Add the release ops and rte_memzone_free().
14. DMA addressing limit is not checked.
The descriptor and EAME setup assume 36-bit IOVAs, but with
vfio-platform in IOVA-as-VA mode addresses can be wider. Call
rte_mem_check_dma_mask() with the width from HW_FEATURE1 (ADDR64) at
probe.
15. Bad devargs are silently ignored.
If rte_kvargs_parse() or a handler fails, probe continues with
defaults. Fail the probe instead. mdio_addr > 31 is masked to a
different PHY address without warning.
16. RX tail register DMA_CHAN_OFF_RX_END_ADDR_HI (0x24) is not in the
dwmac4 register map; the kernel never writes it, and the TX path does
not use an equivalent. Drop it unless the databook documents it for
this core.
17. Secondary process: probe will fail in rte_eth_dev_allocate() with
a misleading -ENOMEM. Check rte_eal_process_type() and return a clear
-ENOTSUP.
Info
----
18. Leftovers: priv->stats (always zero), priv->link_speed,
txq/rxq->queue_id, tx_ring_full (counted, never reported),
STMMAC_ACTIVE_QUEUES, DMA_CONTROL_PBL_SHIFT/MASK (duplicated by
STMMAC_DMA_PBL_*), MTL_OP_MODE_TXQEN_MASK, DMA_CHAN_STATUS_TI/RI,
TDES2_INTERRUPT_ON_COMPLETION, TDES3_CONTEXT_TYPE,
RDES3_OVERFLOW_ERROR, RDES3_CRC_ERROR, DP83867_PHYCTRL, and the NULL
branches of stmmac_dma_chan_base()/stmmac_mtl_chan_base().
19. Redundant with ethdev: nb_desc == 0 defaulting and the TX % 4 check
(nb_align already enforced for both RX and TX), and setting
dev_started in start/stop.
20. default_rxportconf/default_txportconf use STMMAC_MAX_RING_SIZE
while STMMAC_DEFAULT_RING_SIZE (256) exists and the docs imply 256.
21. Count allocation failures in dev->data->rx_mbuf_alloc_failed (per
mbuf) rather than a private per-burst counter.
22. Reading CURTXD on every TX burst is an MMIO round trip; with
per-line descriptors (item 2) you can reclaim on the OWN bit instead.
23. GMAC_RXQ_CTRL0 uses magic 0x3/0x2; add defines. dev_rx/tx_offloads_sup
should be const.
24. Commit message: "Ethernet MAC present." is missing its object.
Docs: drop the "Contents summary" boilerplate, the em-dash in
Supported Platforms, and the --vdev="" in the testpmd example.
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2026-11-17 18:42 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-14 14:55 [PATCH] net/stmmac: add PMD for Synopsys DWC EQoS (GMAC4) Kiran Patchala
2026-11-17 18:42 ` Stephen Hemminger
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox