[PATCH net-next v2 0/2] Add the capability to allocate hw buffers in SRAM for EN7581 SoC

[PATCH net-next v2 0/2] Add the capability to allocate hw buffers in SRAM for EN7581 SoC

[Lorenzo Bianconi]

In order to improve packet processing and packet forwarding
performances, EN7581 SoC supports allocating buffers for hw forwarding
queues in SRAM instead of DRAM if available on the system.
Rely on SRAM for buffers allocation if available on the system and use
DRAM as fallback.

---
Changes in v2:
- fix sparse warnings
- Link to v1: https://lore.kernel.org/r/20250507-airopha-desc-sram-v1-0-d42037431bfa@kernel.org

---
Lorenzo Bianconi (2):
      dt-bindings: net: airoha: Add EN7581 memory-region property
      net: airoha: Add the capability to allocate hw buffers in SRAM

 .../devicetree/bindings/net/airoha,en7581-eth.yaml | 13 +++++
 drivers/net/ethernet/airoha/airoha_eth.c           | 57 ++++++++++++++++++----
 2 files changed, 61 insertions(+), 9 deletions(-)
---
base-commit: a9ce2ce1800e04267e6d99016ed0fe132d6049a9
change-id: 20250507-airopha-desc-sram-faeb3ea6cbc5

Best regards,
-- 
Lorenzo Bianconi <lorenzo@kernel.org>

[PATCH net-next v2 1/2] dt-bindings: net: airoha: Add EN7581 memory-region property

[Lorenzo Bianconi]

Introduce the memory-region and memory-region-names properties for the
ethernet node available on EN7581 SoC. In order to improve performances,
EN7581 SoC supports allocating buffers for hw forwarding queues in SRAM
instead of DRAM if available on the system.

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
---
 .../devicetree/bindings/net/airoha,en7581-eth.yaml          | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml b/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml
index 0fdd1126541774acacc783d98e4c089b2d2b85e2..6d22131ac2f9e28390b9e785ce33e8d983eafd0f 100644
--- a/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml
+++ b/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml
@@ -57,6 +57,16 @@ properties:
       - const: hsi-mac
       - const: xfp-mac
 
+  memory-region:
+    items:
+      - description: QDMA0 buffer memory
+      - description: QDMA1 buffer memory
+
+  memory-region-names:
+    items:
+      - const: qdma0-buf
+      - const: qdma1-buf
+
   "#address-cells":
     const: 1
 
@@ -140,6 +150,9 @@ examples:
                      <GIC_SPI 49 IRQ_TYPE_LEVEL_HIGH>,
                      <GIC_SPI 64 IRQ_TYPE_LEVEL_HIGH>;
 
+        memory-region = <&qdma0_buf>, <&qdma1_buf>;
+        memory-region-names = "qdma0-buf", "qdma1-buf";
+
         airoha,npu = <&npu>;
 
         #address-cells = <1>;

-- 
2.49.0

Re: [PATCH net-next v2 1/2] dt-bindings: net: airoha: Add EN7581 memory-region property

[Rob Herring]

On Fri, May 09, 2025 at 04:51:33PM +0200, Lorenzo Bianconi wrote:
> Introduce the memory-region and memory-region-names properties for the
> ethernet node available on EN7581 SoC. In order to improve performances,
> EN7581 SoC supports allocating buffers for hw forwarding queues in SRAM
> instead of DRAM if available on the system.

But 'reserved-memory' is generally for system memory which is DRAM 
though we unfortunately don't enforce that. For small onchip SRAM, you 
should be using the mmio-sram binding and the 'sram' property.

> 
> Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
> ---
>  .../devicetree/bindings/net/airoha,en7581-eth.yaml          | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml b/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml
> index 0fdd1126541774acacc783d98e4c089b2d2b85e2..6d22131ac2f9e28390b9e785ce33e8d983eafd0f 100644
> --- a/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml
> +++ b/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml
> @@ -57,6 +57,16 @@ properties:
>        - const: hsi-mac
>        - const: xfp-mac
>  
> +  memory-region:
> +    items:
> +      - description: QDMA0 buffer memory
> +      - description: QDMA1 buffer memory
> +
> +  memory-region-names:
> +    items:
> +      - const: qdma0-buf
> +      - const: qdma1-buf
> +
>    "#address-cells":
>      const: 1
>  
> @@ -140,6 +150,9 @@ examples:
>                       <GIC_SPI 49 IRQ_TYPE_LEVEL_HIGH>,
>                       <GIC_SPI 64 IRQ_TYPE_LEVEL_HIGH>;
>  
> +        memory-region = <&qdma0_buf>, <&qdma1_buf>;
> +        memory-region-names = "qdma0-buf", "qdma1-buf";
> +
>          airoha,npu = <&npu>;
>  
>          #address-cells = <1>;
> 
> -- 
> 2.49.0
> 

Re: [PATCH net-next v2 1/2] dt-bindings: net: airoha: Add EN7581 memory-region property

[Lorenzo Bianconi]

[-- Attachment #1: Type: text/plain, Size: 2049 bytes --]

> On Fri, May 09, 2025 at 04:51:33PM +0200, Lorenzo Bianconi wrote:
> > Introduce the memory-region and memory-region-names properties for the
> > ethernet node available on EN7581 SoC. In order to improve performances,
> > EN7581 SoC supports allocating buffers for hw forwarding queues in SRAM
> > instead of DRAM if available on the system.
> 
> But 'reserved-memory' is generally for system memory which is DRAM 
> though we unfortunately don't enforce that. For small onchip SRAM, you 
> should be using the mmio-sram binding and the 'sram' property.

ack, thx for pointing this out, I will look into it.

Regards,
Lorenzo

> 
> > 
> > Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
> > ---
> >  .../devicetree/bindings/net/airoha,en7581-eth.yaml          | 13 +++++++++++++
> >  1 file changed, 13 insertions(+)
> > 
> > diff --git a/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml b/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml
> > index 0fdd1126541774acacc783d98e4c089b2d2b85e2..6d22131ac2f9e28390b9e785ce33e8d983eafd0f 100644
> > --- a/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml
> > +++ b/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml
> > @@ -57,6 +57,16 @@ properties:
> >        - const: hsi-mac
> >        - const: xfp-mac
> >  
> > +  memory-region:
> > +    items:
> > +      - description: QDMA0 buffer memory
> > +      - description: QDMA1 buffer memory
> > +
> > +  memory-region-names:
> > +    items:
> > +      - const: qdma0-buf
> > +      - const: qdma1-buf
> > +
> >    "#address-cells":
> >      const: 1
> >  
> > @@ -140,6 +150,9 @@ examples:
> >                       <GIC_SPI 49 IRQ_TYPE_LEVEL_HIGH>,
> >                       <GIC_SPI 64 IRQ_TYPE_LEVEL_HIGH>;
> >  
> > +        memory-region = <&qdma0_buf>, <&qdma1_buf>;
> > +        memory-region-names = "qdma0-buf", "qdma1-buf";
> > +
> >          airoha,npu = <&npu>;
> >  
> >          #address-cells = <1>;
> > 
> > -- 
> > 2.49.0
> > 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH net-next v2 1/2] dt-bindings: net: airoha: Add EN7581 memory-region property

[Lorenzo Bianconi]

[-- Attachment #1: Type: text/plain, Size: 2552 bytes --]

> On Fri, May 09, 2025 at 04:51:33PM +0200, Lorenzo Bianconi wrote:
> > Introduce the memory-region and memory-region-names properties for the
> > ethernet node available on EN7581 SoC. In order to improve performances,
> > EN7581 SoC supports allocating buffers for hw forwarding queues in SRAM
> > instead of DRAM if available on the system.
> 
> But 'reserved-memory' is generally for system memory which is DRAM 
> though we unfortunately don't enforce that. For small onchip SRAM, you 
> should be using the mmio-sram binding and the 'sram' property.

Reviewing the vendor sdk my understanding was wrong (sorry for the noise).
Here we just want to add the capability to allocate hw forwarding buffers queue
defining the memory region in the DTS instead of using dmam_alloc_coherent()
since in some configurations QDMA blocks require a contiguous block of
system memory for hwfd buffers queue.
Moreover, EN7581 SoC supports consuming SRAM instead of DRAM for hw forwarding
descriptors queue. This is managed in hw, we just need to request it in the
flowtable entry configuration.
I will fix it in v2.

Regards,
Lorenzo

> 
> > 
> > Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
> > ---
> >  .../devicetree/bindings/net/airoha,en7581-eth.yaml          | 13 +++++++++++++
> >  1 file changed, 13 insertions(+)
> > 
> > diff --git a/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml b/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml
> > index 0fdd1126541774acacc783d98e4c089b2d2b85e2..6d22131ac2f9e28390b9e785ce33e8d983eafd0f 100644
> > --- a/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml
> > +++ b/Documentation/devicetree/bindings/net/airoha,en7581-eth.yaml
> > @@ -57,6 +57,16 @@ properties:
> >        - const: hsi-mac
> >        - const: xfp-mac
> >  
> > +  memory-region:
> > +    items:
> > +      - description: QDMA0 buffer memory
> > +      - description: QDMA1 buffer memory
> > +
> > +  memory-region-names:
> > +    items:
> > +      - const: qdma0-buf
> > +      - const: qdma1-buf
> > +
> >    "#address-cells":
> >      const: 1
> >  
> > @@ -140,6 +150,9 @@ examples:
> >                       <GIC_SPI 49 IRQ_TYPE_LEVEL_HIGH>,
> >                       <GIC_SPI 64 IRQ_TYPE_LEVEL_HIGH>;
> >  
> > +        memory-region = <&qdma0_buf>, <&qdma1_buf>;
> > +        memory-region-names = "qdma0-buf", "qdma1-buf";
> > +
> >          airoha,npu = <&npu>;
> >  
> >          #address-cells = <1>;
> > 
> > -- 
> > 2.49.0
> > 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[PATCH net-next v2 2/2] net: airoha: Add the capability to allocate hw buffers in SRAM

[Lorenzo Bianconi]

In order to improve packet processing and packet forwarding
performances, EN7581 SoC supports allocating buffers for hw forwarding
queues in SRAM instead of DRAM if available on the system.
Rely on SRAM for buffers allocation if available on the system and use
DRAM as fallback.

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
---
 drivers/net/ethernet/airoha/airoha_eth.c | 57 +++++++++++++++++++++++++++-----
 1 file changed, 48 insertions(+), 9 deletions(-)

diff --git a/drivers/net/ethernet/airoha/airoha_eth.c b/drivers/net/ethernet/airoha/airoha_eth.c
index 16c7896f931fd9532aa3b8cc78f41afc676aa117..b11c25442eb8c8c2b5aa45ded7a9d61e24aa2e4a 100644
--- a/drivers/net/ethernet/airoha/airoha_eth.c
+++ b/drivers/net/ethernet/airoha/airoha_eth.c
@@ -5,6 +5,7 @@
  */
 #include <linux/of.h>
 #include <linux/of_net.h>
+#include <linux/of_reserved_mem.h>
 #include <linux/platform_device.h>
 #include <linux/tcp.h>
 #include <linux/u64_stats_sync.h>
@@ -1076,9 +1077,11 @@ static void airoha_qdma_cleanup_tx_queue(struct airoha_queue *q)
 static int airoha_qdma_init_hfwd_queues(struct airoha_qdma *qdma)
 {
 	struct airoha_eth *eth = qdma->eth;
+	int id = qdma - &eth->qdma[0];
 	dma_addr_t dma_addr;
-	u32 status;
-	int size;
+	const char *name;
+	int size, index;
+	u32 status, val;
 
 	size = HW_DSCP_NUM * sizeof(struct airoha_qdma_fwd_desc);
 	qdma->hfwd.desc = dmam_alloc_coherent(eth->dev, size, &dma_addr,
@@ -1088,12 +1091,45 @@ static int airoha_qdma_init_hfwd_queues(struct airoha_qdma *qdma)
 
 	airoha_qdma_wr(qdma, REG_FWD_DSCP_BASE, dma_addr);
 
-	size = AIROHA_MAX_PACKET_SIZE * HW_DSCP_NUM;
-	qdma->hfwd.q = dmam_alloc_coherent(eth->dev, size, &dma_addr,
-					   GFP_KERNEL);
-	if (!qdma->hfwd.q)
+	name = devm_kasprintf(eth->dev, GFP_KERNEL, "qdma%d-buf", id);
+	if (!name)
 		return -ENOMEM;
 
+	index = of_property_match_string(eth->dev->of_node,
+					 "memory-region-names", name);
+	if (index >= 0) { /* buffers in sram */
+		struct reserved_mem *rmem;
+		struct device_node *np;
+		void *q;
+
+		np = of_parse_phandle(eth->dev->of_node, "memory-region",
+				      index);
+		if (!np)
+			return -ENODEV;
+
+		rmem = of_reserved_mem_lookup(np);
+		of_node_put(np);
+
+		/* SRAM is actual memory and supports transparent access just
+		 * like DRAM. Hence we don't require __iomem being set and
+		 * we don't need to use accessor routines to read from or write
+		 * to SRAM.
+		 */
+		q = (void __force *)devm_ioremap(eth->dev, rmem->base,
+						 rmem->size);
+		if (!q)
+			return -ENOMEM;
+
+		qdma->hfwd.q = q;
+		dma_addr = rmem->base;
+	} else {
+		size = AIROHA_MAX_PACKET_SIZE * HW_DSCP_NUM;
+		qdma->hfwd.q = dmam_alloc_coherent(eth->dev, size, &dma_addr,
+						   GFP_KERNEL);
+		if (!qdma->hfwd.q)
+			return -ENOMEM;
+	}
+
 	airoha_qdma_wr(qdma, REG_FWD_BUF_BASE, dma_addr);
 
 	airoha_qdma_rmw(qdma, REG_HW_FWD_DSCP_CFG,
@@ -1101,11 +1137,14 @@ static int airoha_qdma_init_hfwd_queues(struct airoha_qdma *qdma)
 			FIELD_PREP(HW_FWD_DSCP_PAYLOAD_SIZE_MASK, 0));
 	airoha_qdma_rmw(qdma, REG_FWD_DSCP_LOW_THR, FWD_DSCP_LOW_THR_MASK,
 			FIELD_PREP(FWD_DSCP_LOW_THR_MASK, 128));
+
+	val = FIELD_PREP(HW_FWD_DESC_NUM_MASK, HW_DSCP_NUM) |
+	      LMGR_INIT_START;
+	if (index >= 0)
+		val |= LMGR_SRAM_MODE_MASK;
 	airoha_qdma_rmw(qdma, REG_LMGR_INIT_CFG,
 			LMGR_INIT_START | LMGR_SRAM_MODE_MASK |
-			HW_FWD_DESC_NUM_MASK,
-			FIELD_PREP(HW_FWD_DESC_NUM_MASK, HW_DSCP_NUM) |
-			LMGR_INIT_START);
+			HW_FWD_DESC_NUM_MASK, val);
 
 	return read_poll_timeout(airoha_qdma_rr, status,
 				 !(status & LMGR_INIT_START), USEC_PER_MSEC,

-- 
2.49.0

Re: [PATCH net-next v2 2/2] net: airoha: Add the capability to allocate hw buffers in SRAM

[Simon Horman]

On Fri, May 09, 2025 at 04:51:34PM +0200, Lorenzo Bianconi wrote:
> In order to improve packet processing and packet forwarding
> performances, EN7581 SoC supports allocating buffers for hw forwarding
> queues in SRAM instead of DRAM if available on the system.
> Rely on SRAM for buffers allocation if available on the system and use
> DRAM as fallback.
> 
> Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>

Reviewed-by: Simon Horman <horms@kernel.org>

> ---
>  drivers/net/ethernet/airoha/airoha_eth.c | 57 +++++++++++++++++++++++++++-----
>  1 file changed, 48 insertions(+), 9 deletions(-)
> 
> diff --git a/drivers/net/ethernet/airoha/airoha_eth.c b/drivers/net/ethernet/airoha/airoha_eth.c

...

> @@ -1088,12 +1091,45 @@ static int airoha_qdma_init_hfwd_queues(struct airoha_qdma *qdma)
>  
>  	airoha_qdma_wr(qdma, REG_FWD_DSCP_BASE, dma_addr);
>  
> -	size = AIROHA_MAX_PACKET_SIZE * HW_DSCP_NUM;
> -	qdma->hfwd.q = dmam_alloc_coherent(eth->dev, size, &dma_addr,
> -					   GFP_KERNEL);
> -	if (!qdma->hfwd.q)
> +	name = devm_kasprintf(eth->dev, GFP_KERNEL, "qdma%d-buf", id);
> +	if (!name)
>  		return -ENOMEM;
>  
> +	index = of_property_match_string(eth->dev->of_node,
> +					 "memory-region-names", name);
> +	if (index >= 0) { /* buffers in sram */
> +		struct reserved_mem *rmem;
> +		struct device_node *np;
> +		void *q;
> +
> +		np = of_parse_phandle(eth->dev->of_node, "memory-region",
> +				      index);
> +		if (!np)
> +			return -ENODEV;
> +
> +		rmem = of_reserved_mem_lookup(np);
> +		of_node_put(np);
> +
> +		/* SRAM is actual memory and supports transparent access just
> +		 * like DRAM. Hence we don't require __iomem being set and
> +		 * we don't need to use accessor routines to read from or write
> +		 * to SRAM.
> +		 */

Thanks for this comment. IMHO It is really useful.

> +		q = (void __force *)devm_ioremap(eth->dev, rmem->base,
> +						 rmem->size);
> +		if (!q)
> +			return -ENOMEM;
> +
> +		qdma->hfwd.q = q;
> +		dma_addr = rmem->base;
> +	} else {
> +		size = AIROHA_MAX_PACKET_SIZE * HW_DSCP_NUM;
> +		qdma->hfwd.q = dmam_alloc_coherent(eth->dev, size, &dma_addr,
> +						   GFP_KERNEL);
> +		if (!qdma->hfwd.q)
> +			return -ENOMEM;
> +	}
> +
>  	airoha_qdma_wr(qdma, REG_FWD_BUF_BASE, dma_addr);
>  
>  	airoha_qdma_rmw(qdma, REG_HW_FWD_DSCP_CFG,

...