2.6.18-rc6 memory mapped pcap truncates outgoing TCP packets, but not icmp

2.6.18-rc6 memory mapped pcap truncates outgoing TCP packets, but not icmp

[bert hubert]

Hi people,

I like to use memory mapped pcap (PACKET_MMAP) since off the shelf, linux is
a tad prone to drop packets while capturing these days. It used to be lots
better at it, but right now memory mapped pcap is the only way to get things
working a bit. I've noticed this on many machines.

However, memory mapped pcap has started to truncate outgoing packets for me
recently, and interestingly, I only see this with locally generated TCP
packets, not with locally generated ICMP packets. I haven't yet tried UDP,
nor actual sniffing, this is all locally generated packets going out on
eth0.

Incoming packets are not truncated.

My commandline:
# PCAP_VERBOSE=1 PCAP_FRAMES=15000 tcpdump  -i eth0 -s 1600 -p -w test-dump
libpcap version: 0.9
Kernel filter, Protocol 0300, MMAP mode (12188 frames, snapshot 1600), socket type: Raw
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 1600 bytes


Within this dump we find the following outgoing TCP packet:
Frame 289 (553 bytes on wire, 66 bytes captured)
    Arrival Time: Sep 13, 2006 13:18:31.960250000
    Time delta from previous packet: 0.000072000 seconds
    Time since reference or first frame: 42.738582000 seconds
    Frame Number: 289
    Packet Length: 553 bytes
    Capture Length: 66 bytes
    Protocols in frame: eth:ip:tcp
    Type: IP (0x0800)
Internet Protocol, Src: 10.0.3.146 (10.0.3.146), Dst: 82.165.25.125 (82.165.25.125)

Which is truncated!

However, we also find this incoming packet:
Frame 290 (1508 bytes on wire, 1508 bytes captured)
    Arrival Time: Sep 13, 2006 13:18:32.036536000
    Time delta from previous packet: 0.076286000 seconds
    Time since reference or first frame: 42.814868000 seconds
    Frame Number: 290
    Packet Length: 1508 bytes
    Capture Length: 1508 bytes
    Protocols in frame: eth:ip:tcp:http
Internet Protocol, Src: 82.165.25.125 (82.165.25.125), Dst: 10.0.3.146 (10.0.3.146)

Which looks just fine.

Downgrading to 'normal' mode fixes this problem, but suffers from too much
packet loss to be useful.

My tcpdump is built with:
http://public.lanl.gov/cpw/libpcap-0.9.20060417.tar.gz

It used to work just fine, but I haven't been able to find when it broke.

Please let me know how I can help solve this!

	Bert

-- 
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services

Re: 2.6.18-rc6 memory mapped pcap truncates outgoing TCP packets, but not icmp

[Patrick McHardy]

bert hubert wrote:
> Hi people,
> 
> I like to use memory mapped pcap (PACKET_MMAP) since off the shelf, linux is
> a tad prone to drop packets while capturing these days. It used to be lots
> better at it, but right now memory mapped pcap is the only way to get things
> working a bit. I've noticed this on many machines.
> 
> However, memory mapped pcap has started to truncate outgoing packets for me
> recently, and interestingly, I only see this with locally generated TCP
> packets, not with locally generated ICMP packets. I haven't yet tried UDP,
> nor actual sniffing, this is all locally generated packets going out on
> eth0.

Are you using TSO on the outgoing device? If so please try to log the
packet using iptables to see if it really is a TSO packet.

Re: 2.6.18-rc6 memory mapped pcap truncates outgoing TCP packets, but not icmp

[bert hubert]

On Wed, Sep 13, 2006 at 08:44:21PM +0200, Patrick McHardy wrote:

> Are you using TSO on the outgoing device? If so please try to log the
> packet using iptables to see if it really is a TSO packet.

Good catch! I turned off TSO and things are working fine again.

Is this a known problem, should it be documented or fixed? I'm more than
willing to write up some warnings should this be a good idea.

Thanks Patrick! I can do without TSO but not without mmapped pcap!

	Bert


-- 
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services

Re: 2.6.18-rc6 memory mapped pcap truncates outgoing TCP packets, but not icmp

[Patrick McHardy]

[-- Attachment #1: Type: text/plain, Size: 559 bytes --]

bert hubert wrote:
> On Wed, Sep 13, 2006 at 08:44:21PM +0200, Patrick McHardy wrote:
> 
> 
>>Are you using TSO on the outgoing device? If so please try to log the
>>packet using iptables to see if it really is a TSO packet.
> 
> 
> Good catch! I turned off TSO and things are working fine again.
> 
> Is this a known problem, should it be documented or fixed? I'm more than
> willing to write up some warnings should this be a good idea.

It appears to be intentionally, but I don't see a reason for it.
Can you try if this patch makes it work as expected?


[-- Attachment #2: x --]
[-- Type: text/plain, Size: 1332 bytes --]

[PACKET]: Don't truncate non-linear skbs with mmaped IO

Non-linear skbs are truncated to their linear part with mmaped IO.
Fix by using skb_copy_bits instead of memcpy.

Signed-off-by: Patrick McHardy <kaber@trash.net>

---
commit 6e184976552a407c331f9b4e52b2c26fcae46ee7
tree 1bb4fc2b65179a51e795dc07908277b83cc0921a
parent 9f737633e6ee54fc174282d49b2559bd2208391d
author Patrick McHardy <kaber@trash.net> Thu, 14 Sep 2006 11:59:09 +0200
committer Patrick McHardy <kaber@trash.net> Thu, 14 Sep 2006 11:59:09 +0200

 net/packet/af_packet.c |    4 +---
 1 files changed, 1 insertions(+), 3 deletions(-)

diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
index f9cef36..4172a52 100644
--- a/net/packet/af_packet.c
+++ b/net/packet/af_packet.c
@@ -626,8 +626,6 @@ static int tpacket_rcv(struct sk_buff *s
 		if ((int)snaplen < 0)
 			snaplen = 0;
 	}
-	if (snaplen > skb->len-skb->data_len)
-		snaplen = skb->len-skb->data_len;
 
 	spin_lock(&sk->sk_receive_queue.lock);
 	h = (struct tpacket_hdr *)packet_lookup_frame(po, po->head);
@@ -644,7 +642,7 @@ static int tpacket_rcv(struct sk_buff *s
 		status &= ~TP_STATUS_LOSING;
 	spin_unlock(&sk->sk_receive_queue.lock);
 
-	memcpy((u8*)h + macoff, skb->data, snaplen);
+	skb_copy_bits(skb, 0, (u8*)h + macoff, snaplen);
 
 	h->tp_len = skb->len;
 	h->tp_snaplen = snaplen;

Re: 2.6.18-rc6 memory mapped pcap truncates outgoing TCP packets, but not icmp

[Alexey Kuznetsov]

Hello!

> [PACKET]: Don't truncate non-linear skbs with mmaped IO
> 
> Non-linear skbs are truncated to their linear part with mmaped IO.
> Fix by using skb_copy_bits instead of memcpy.

Ack.

I remember this trick. The "idea" was that I needed only TCP header in any
case and it was perfect cutoff. This hack was not supposed to survive. :-)

Alexey

Re: 2.6.18-rc6 memory mapped pcap truncates outgoing TCP packets, but not icmp

[bert hubert]

> It appears to be intentionally, but I don't see a reason for it.
> Can you try if this patch makes it work as expected?

> [PACKET]: Don't truncate non-linear skbs with mmaped IO
> 
> Non-linear skbs are truncated to their linear part with mmaped IO.
> Fix by using skb_copy_bits instead of memcpy.

Works very well for me! I hope this can make it into 2.6.18.
Thanks everybody.



-- 
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services

Re: 2.6.18-rc6 memory mapped pcap truncates outgoing TCP packets, but not icmp

[Patrick McHardy]

[-- Attachment #1: Type: text/plain, Size: 452 bytes --]

bert hubert wrote:
>>It appears to be intentionally, but I don't see a reason for it.
>>Can you try if this patch makes it work as expected?
> 
> 
>>[PACKET]: Don't truncate non-linear skbs with mmaped IO
>>
>>Non-linear skbs are truncated to their linear part with mmaped IO.
>>Fix by using skb_copy_bits instead of memcpy.
> 
> 
> Works very well for me! I hope this can make it into 2.6.18.


That would be fine with me, lets see what Dave thinks.


[-- Attachment #2: x --]
[-- Type: text/plain, Size: 1332 bytes --]

[PACKET]: Don't truncate non-linear skbs with mmaped IO

Non-linear skbs are truncated to their linear part with mmaped IO.
Fix by using skb_copy_bits instead of memcpy.

Signed-off-by: Patrick McHardy <kaber@trash.net>

---
commit 6e184976552a407c331f9b4e52b2c26fcae46ee7
tree 1bb4fc2b65179a51e795dc07908277b83cc0921a
parent 9f737633e6ee54fc174282d49b2559bd2208391d
author Patrick McHardy <kaber@trash.net> Thu, 14 Sep 2006 11:59:09 +0200
committer Patrick McHardy <kaber@trash.net> Thu, 14 Sep 2006 11:59:09 +0200

 net/packet/af_packet.c |    4 +---
 1 files changed, 1 insertions(+), 3 deletions(-)

diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
index f9cef36..4172a52 100644
--- a/net/packet/af_packet.c
+++ b/net/packet/af_packet.c
@@ -626,8 +626,6 @@ static int tpacket_rcv(struct sk_buff *s
 		if ((int)snaplen < 0)
 			snaplen = 0;
 	}
-	if (snaplen > skb->len-skb->data_len)
-		snaplen = skb->len-skb->data_len;
 
 	spin_lock(&sk->sk_receive_queue.lock);
 	h = (struct tpacket_hdr *)packet_lookup_frame(po, po->head);
@@ -644,7 +642,7 @@ static int tpacket_rcv(struct sk_buff *s
 		status &= ~TP_STATUS_LOSING;
 	spin_unlock(&sk->sk_receive_queue.lock);
 
-	memcpy((u8*)h + macoff, skb->data, snaplen);
+	skb_copy_bits(skb, 0, (u8*)h + macoff, snaplen);
 
 	h->tp_len = skb->len;
 	h->tp_snaplen = snaplen;

Re: 2.6.18-rc6 memory mapped pcap truncates outgoing TCP packets, but not icmp

[David Miller]

From: Patrick McHardy <kaber@trash.net>
Date: Fri, 15 Sep 2006 22:16:17 +0200

> bert hubert wrote:
> >>It appears to be intentionally, but I don't see a reason for it.
> >>Can you try if this patch makes it work as expected?
> > 
> > 
> >>[PACKET]: Don't truncate non-linear skbs with mmaped IO
> >>
> >>Non-linear skbs are truncated to their linear part with mmaped IO.
> >>Fix by using skb_copy_bits instead of memcpy.
> > 
> > 
> > Works very well for me! I hope this can make it into 2.6.18.
> 
> 
> That would be fine with me, lets see what Dave thinks.

Applied to net-2.6, thanks a lot.