[PATCH] stmmac: reduce queue lengths if MTU is very large

[PATCH] stmmac: reduce queue lengths if MTU is very large

[Giuseppe CAVALLARO]

From: Alessandro Rubini <rubini@gnudd.com>

In my use case (STA2X11 IO-Hub), the alloc_coherent, map_single and
friends map to a reserved area set up in swiotlb.  With the default
MTU each packet takes 2kB, but with jumbo frames it increases to 16kB
and consumes up my area (it takes 8MB instead of 1MB).

Instead of increasing the reserved (== wasted) preallocated area in
swiotlb, this patch rescales the user-selected queue lengths if the MTU
is modified at runtime to be larger than the default. User choices
are still obeyed (compiled default, module parameter, command line)

Signed-off-by: Alessandro Rubini <rubini@gnudd.com>
Acked-by: Giancarlo Asnaghi <giancarlo.asnaghi@st.com>
Acked-by: Giuseppe Cavallaro <peppe.cavallaro@st.com>
---
 drivers/net/ethernet/stmicro/stmmac/stmmac_main.c |   19 ++++++++++++++++---
 1 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index 72cd190..e8ebb6e 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -413,9 +413,8 @@ static void init_dma_desc_rings(struct net_device *dev)
 	int i;
 	struct stmmac_priv *priv = netdev_priv(dev);
 	struct sk_buff *skb;
-	unsigned int txsize = priv->dma_tx_size;
-	unsigned int rxsize = priv->dma_rx_size;
-	unsigned int bfsize;
+	unsigned int txsize, rxsize;
+	unsigned int bfsize = priv->dma_buf_sz;
 	int dis_ic = 0;
 	int des3_as_data_buf = 0;
 
@@ -428,6 +427,20 @@ static void init_dma_desc_rings(struct net_device *dev)
 	else
 		bfsize = stmmac_set_bfsize(dev->mtu, priv->dma_buf_sz);
 
+	/*
+	 * If we use jumbo frames, we end up consuming too much memory
+	 * in the coherent buffer, so reduce the queue lenght according
+	 * to the increase in frame size over the default.
+	 */
+	i = bfsize / BUF_SIZE_2KiB;
+	if (i > 1) {
+		pr_warning("%s: reducing queue length by %i\n", __func__, i);
+		priv->dma_tx_size /= i;
+		priv->dma_rx_size /= i;
+	}
+	txsize = priv->dma_tx_size;
+	rxsize = priv->dma_rx_size;
+
 #ifdef CONFIG_STMMAC_TIMER
 	/* Disable interrupts on completion for the reception if timer is on */
 	if (likely(priv->tm->enable))
-- 
1.7.4.4

Re: [PATCH] stmmac: reduce queue lengths if MTU is very large

[David Miller]

From: Giuseppe CAVALLARO <peppe.cavallaro@st.com>
Date: Mon, 19 Dec 2011 06:45:30 +0100

> From: Alessandro Rubini <rubini@gnudd.com>
> 
> In my use case (STA2X11 IO-Hub), the alloc_coherent, map_single and
> friends map to a reserved area set up in swiotlb.  With the default
> MTU each packet takes 2kB, but with jumbo frames it increases to 16kB
> and consumes up my area (it takes 8MB instead of 1MB).
> 
> Instead of increasing the reserved (== wasted) preallocated area in
> swiotlb, this patch rescales the user-selected queue lengths if the MTU
> is modified at runtime to be larger than the default. User choices
> are still obeyed (compiled default, module parameter, command line)
> 
> Signed-off-by: Alessandro Rubini <rubini@gnudd.com>
> Acked-by: Giancarlo Asnaghi <giancarlo.asnaghi@st.com>
> Acked-by: Giuseppe Cavallaro <peppe.cavallaro@st.com>

This change doesn't make sense.

If the user is only sending/receiving small frames, then this patch
will hurt them.

I'm not applying this, it just points out that the amount of reserved
space in your SWIOMMU setup might be insufficient for this situation
rather than something that should be tweaked in this driver.

Re: [PATCH] stmmac: reduce queue lengths if MTU is very large

[Alessandro Rubini]

> This change doesn't make sense.

I disagree, but you are the boss here, so I accept that.

> I'm not applying this, it just points out that the amount of reserved
> space in your SWIOMMU setup might be insufficient for this situation
> rather than something that should be tweaked in this driver.

Allocation of the queues is based on a parameter. If the parameter was
a size, instead of a lenght, all would be fine, wouldn't it? If you
agree, then this patch is turning the lenght into a size, in a way.

Now, how could we deal with the fact that the size explodes if the MTU
is increased?  I don't want to reserve and waste 16MB of RAM for a
single device: the device won't even use them for non-jumbo setups.  I
can use the parameter to shorten the queue in all cases, but this is
not good either.

What's your suggestion?

thanks
/alessandro

Re: [PATCH] stmmac: reduce queue lengths if MTU is very large

[David Miller]

From: Alessandro Rubini <rubini@gnudd.com>
Date: Tue, 20 Dec 2011 08:30:50 +0100

> Allocation of the queues is based on a parameter. If the parameter was
> a size, instead of a lenght, all would be fine, wouldn't it? If you
> agree, then this patch is turning the lenght into a size, in a way.

That doesn't make any sense either, queues are sized by TX descriptor
slots not bytes.

> Now, how could we deal with the fact that the size explodes if the MTU
> is increased?

Maybe... set the initial parameter appropriately?  This is enormously
better than fixing it up after the fact.

Maybe I set the number of TX entries based upon my expected packet
load of small packets.  If I happen to set my MTU to 9000 that's
irrelevant to the setting.