[PATCH] kvm: qemu: increase the size of the tap buffer [was Re: TAP MTU >= 4055 problem?]

[Mark McLoughlin <markmc@redhat.com>]

Hi Matthew,

On Thu, 2008-10-30 at 19:43 +0000, Matthew Faulkner wrote:
> Hey all,
> 
> I've solved the problem. Turns out the tap buffers were set at the
> size 4096. When pulling a packet off that is greater than 4096 the 2nd
> packet (for some reason) is thrown aaway or ignored.
> 
> I've done a hacky solutoin by simply increasing this number of 65536.

Thanks for debugging - that makes a whole lot of sense.

To explain in more detail - if you set the tap MTU size to 9000 and do
ping -s 4055

  1) the host's networking stack will construct an ethernet frame of
     4097 (4055 bytes ICMP data, 8 bytes ICMP header, 20 bytes TCP
     header and 14 bytes ethernet header)

  2) qemu will read the frame from the tap device into it's 4096 byte 
     buffer causing the kernel to truncate it to 4096 bytes

  3) qemu passes that on to the guest and the guest's IP stack will see 
     that there is less data in the buffer than is specified in the IP 
     header. It then silently drops the packet (see InTruncatedPkts 
     in /proc/net/netstat)

It sucks that the kernel just silently truncates and doesn't give
userspace a chance to retry with a larger buffer. We could add that as
an extension, I guess.

Best short term thing to do is just increase the tap buffer size. Patch
below. A similar patch should go into qemu upstream.

Cheers,
Mark.

From: Mark McLoughlin <markmc@redhat.com>
Subject: [PATCH] kvm: qemu: increase size of tun/tap buffer

As debugged by Matthew Faulkner, with a 4k byte tap buffer if you
increase the MTU on the tap device to greater than 4k, the packets
read by qemu into the tap buffer will be truncated and the guest
will discard the packet.

With GSO enabled, we use a 64k tap buffer, so let's just use a 64k
buffer in all cases.

Also, remove the obtuse logic for figuring out the max GSO buffer
size. We shouldn't receive IP packets larger than 64k, so let's just
use 17 pages to make sure we've enough room for headers.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
---
 qemu/vl.c |   14 ++++++--------
 1 files changed, 6 insertions(+), 8 deletions(-)

diff --git a/qemu/vl.c b/qemu/vl.c
index da92785..4dccb21 100644
--- a/qemu/vl.c
+++ b/qemu/vl.c
@@ -4413,15 +4413,13 @@ void tap_using_vnet_hdr(void *opaque, int using_vnet_hdr)
 
 #else /* !defined(_WIN32) */
 
-#ifndef IFF_VNET_HDR
-#define TAP_BUFSIZE 4096
-#else
+/* Maximum GSO packet size (64k) plus plenty of room for
+ * the ethernet and virtio_net headers
+ */
+#define TAP_BUFSIZE (4096 + 65536)
+
+#ifdef IFF_VNET_HDR
 #include <linux/virtio_net.h>
-#define ETH_HLEN 14
-#define ETH_DATA_LEN 1500
-#define MAX_PACKET_LEN (ETH_HLEN + ETH_DATA_LEN)
-#define MAX_SKB_FRAGS ((65536/TARGET_PAGE_SIZE) + 2)
-#define TAP_BUFSIZE (sizeof(struct virtio_net_hdr) + MAX_PACKET_LEN + (MAX_SKB_FRAGS*TARGET_PAGE_SIZE))
 #endif
 
 typedef struct TAPState {
-- 
1.6.0.1