[PATCH] solos-pci: Increase headroom on received packets

[PATCH] solos-pci: Increase headroom on received packets

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 2283 bytes --]

A comment in include/linux/skbuff.h says that:

 * Various parts of the networking layer expect at least 32 bytes of
 * headroom, you should not reduce this.

This was demonstrated by a panic when handling fragmented IPv6 packets:
http://marc.info/?l=linux-netdev&m=144236093519172&w=2

It's not entirely clear if that comment is still valid — and if it is,
perhaps netif_rx() ought to be enforcing it with a warning.

But either way, it is rather stupid from a performance point of view
for us to be receiving packets into a buffer which doesn't have enough
room to prepend an Ethernet header — it means that *every* incoming
packet is going to be need to be reallocated. So let's fix that.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
--- 
Tested in the DMA code path; I don't believe the DMA-capable devices
can still be used in MMIO mode. Simon, Guy, would you be able to test
the MMIO version?

diff --git a/drivers/atm/solos-pci.c b/drivers/atm/solos-pci.c
index 74e18b0..be8225e 100644
--- a/drivers/atm/solos-pci.c
+++ b/drivers/atm/solos-pci.c
@@ -805,13 +805,13 @@ static void solos_bh(unsigned long card_arg)
 					continue;
 				}
 
-				skb = alloc_skb(size + 1, GFP_ATOMIC);
+				skb = alloc_skb(size + NET_SKB_PAD + 1, GFP_ATOMIC);
 				if (!skb) {
 					if (net_ratelimit())
 						dev_warn(&card->dev->dev, "Failed to allocate sk_buff for RX\n");
 					continue;
 				}
-
+				skb_reserve(skb, NET_SKB_PAD);
 				memcpy_fromio(skb_put(skb, size),
 					      RX_BUF(card, port) + sizeof(*header),
 					      size);
@@ -869,8 +869,10 @@ static void solos_bh(unsigned long card_arg)
 		/* Allocate RX skbs for any ports which need them */
 		if (card->using_dma && card->atmdev[port] &&
 		    !card->rx_skb[port]) {
-			struct sk_buff *skb = alloc_skb(RX_DMA_SIZE, GFP_ATOMIC);
+			struct sk_buff *skb = alloc_skb(RX_DMA_SIZE + NET_SKB_PAD,
+							GFP_ATOMIC);
 			if (skb) {
+				skb_reserve(skb, NET_SKB_PAD);
 				SKB_CB(skb)->dma_addr =
 					dma_map_single(&card->dev->dev, skb->data,
 						       RX_DMA_SIZE, DMA_FROM_DEVICE);

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation


[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5691 bytes --]

Re: [PATCH] solos-pci: Increase headroom on received packets

[Eric Dumazet]

On Wed, 2015-09-16 at 11:25 +0100, David Woodhouse wrote:
> A comment in include/linux/skbuff.h says that:
> 
>  * Various parts of the networking layer expect at least 32 bytes of
>  * headroom, you should not reduce this.
> 
> This was demonstrated by a panic when handling fragmented IPv6 packets:
> http://marc.info/?l=linux-netdev&m=144236093519172&w=2
> 
> It's not entirely clear if that comment is still valid — and if it is,
> perhaps netif_rx() ought to be enforcing it with a warning.
> 
> But either way, it is rather stupid from a performance point of view
> for us to be receiving packets into a buffer which doesn't have enough
> room to prepend an Ethernet header — it means that *every* incoming
> packet is going to be need to be reallocated. So let's fix that.
> 
> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
> --- 
> Tested in the DMA code path; I don't believe the DMA-capable devices
> can still be used in MMIO mode. Simon, Guy, would you be able to test
> the MMIO version?

You should use netdev_alloc_skb() : This helper is better for rx skbs,
as it allows for better packing of frames in GRO or TCP stack.

Also netdev_alloc_skb_ip_align() might handle the NET_IP_ALIGN stuff
for arches that care.

Re: [PATCH] solos-pci: Increase headroom on received packets

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 964 bytes --]

On Wed, 2015-09-16 at 03:53 -0700, Eric Dumazet wrote:
> You should use netdev_alloc_skb() : This helper is better for rx skbs,
> as it allows for better packing of frames in GRO or TCP stack.

OK, thanks. I don't have a netdev (this is an ATM device) but I can use
dev_alloc_skb().

> Also netdev_alloc_skb_ip_align() might handle the NET_IP_ALIGN stuff
> for arches that care.

I'd briefly considered NET_IP_ALIGN but decided against it because this
isn't Ethernet and my hardware header is a nice sane 8 bytes, not 14.

But actually, the primary use cases for this are PPPoATM — with 2 bytes
of PPP frame type, and PPPoE over BR2684 — with 14 bytes of Ethernet
header. So NET_IP_ALIGN would actually make sense.

Unfortunately the FPGA can't do DMA to unaligned addresses, so I can't
do it in the DMA case. I can do it for the MMIO code path though (which
I still haven't tested).

I'll send a new patch in a moment...

-- 
dwmw2


[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5691 bytes --]

[PATCH v2] solos-pci: Increase headroom on received packets

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 2287 bytes --]

A comment in include/linux/skbuff.h says that:

 * Various parts of the networking layer expect at least 32 bytes of
 * headroom, you should not reduce this.

This was demonstrated by a panic when handling fragmented IPv6 packets:
http://marc.info/?l=linux-netdev&m=144236093519172&w=2

It's not entirely clear if that comment is still valid — and if it is,
perhaps netif_rx() ought to be enforcing it with a warning.

But either way, it is rather stupid from a performance point of view
for us to be receiving packets into a buffer which doesn't have enough
room to prepend an Ethernet header — it means that *every* incoming
packet is going to be need to be reallocated. So let's fix that.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
---
diff --git a/drivers/atm/solos-pci.c b/drivers/atm/solos-pci.c
index 74e18b0..3d7fb65 100644
--- a/drivers/atm/solos-pci.c
+++ b/drivers/atm/solos-pci.c
@@ -805,7 +805,12 @@ static void solos_bh(unsigned long card_arg)
 					continue;
 				}
 
-				skb = alloc_skb(size + 1, GFP_ATOMIC);
+				/* Use netdev_alloc_skb() because it adds NET_SKB_PAD of
+				 * headroom, and ensures we can route packets back out an
+				 * Ethernet interface (for example) without having to
+				 * reallocate. Adding NET_IP_ALIGN also ensures that both
+				 * PPPoATM and PPPoEoBR2684 packets end up aligned. */
+				skb = netdev_alloc_skb_ip_align(NULL, size + 1);
 				if (!skb) {
 					if (net_ratelimit())
 						dev_warn(&card->dev->dev, "Failed to allocate sk_buff for RX\n");
@@ -869,7 +874,10 @@ static void solos_bh(unsigned long card_arg)
 		/* Allocate RX skbs for any ports which need them */
 		if (card->using_dma && card->atmdev[port] &&
 		    !card->rx_skb[port]) {
-			struct sk_buff *skb = alloc_skb(RX_DMA_SIZE, GFP_ATOMIC);
+			/* Unlike the MMIO case (qv) we can't add NET_IP_ALIGN
+			 * here; the FPGA can only DMA to addresses which are
+			 * aligned to 4 bytes. */
+			struct sk_buff *skb = dev_alloc_skb(RX_DMA_SIZE);
 			if (skb) {
 				SKB_CB(skb)->dma_addr =
 					dma_map_single(&card->dev->dev, skb->data,

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation


[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5691 bytes --]

Re: [PATCH v2] solos-pci: Increase headroom on received packets

[Eric Dumazet]

On Wed, 2015-09-16 at 12:35 +0100, David Woodhouse wrote:
> A comment in include/linux/skbuff.h says that:
> 
>  * Various parts of the networking layer expect at least 32 bytes of
>  * headroom, you should not reduce this.
> 
> This was demonstrated by a panic when handling fragmented IPv6 packets:
> http://marc.info/?l=linux-netdev&m=144236093519172&w=2
> 
> It's not entirely clear if that comment is still valid — and if it is,
> perhaps netif_rx() ought to be enforcing it with a warning.
> 
> But either way, it is rather stupid from a performance point of view
> for us to be receiving packets into a buffer which doesn't have enough
> room to prepend an Ethernet header — it means that *every* incoming
> packet is going to be need to be reallocated. So let's fix that.
> 
> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
> ---

LGTM

Acked-by: Eric Dumazet <edumazet@google.com>

Re: [PATCH v2] solos-pci: Increase headroom on received packets

[David Miller]

From: David Woodhouse <dwmw2@infradead.org>
Date: Wed, 16 Sep 2015 12:35:00 +0100

> A comment in include/linux/skbuff.h says that:
> 
>  * Various parts of the networking layer expect at least 32 bytes of
>  * headroom, you should not reduce this.
> 
> This was demonstrated by a panic when handling fragmented IPv6 packets:
> http://marc.info/?l=linux-netdev&m=144236093519172&w=2
> 
> It's not entirely clear if that comment is still valid ― and if it is,
> perhaps netif_rx() ought to be enforcing it with a warning.
> 
> But either way, it is rather stupid from a performance point of view
> for us to be receiving packets into a buffer which doesn't have enough
> room to prepend an Ethernet header ― it means that *every* incoming
> packet is going to be need to be reallocated. So let's fix that.
> 
> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>

Applied, thanks David.