[Qemu-devel] [PATCH v4 0/6] Add netmap backend offloadings support

[Qemu-devel] [PATCH v4 0/6] Add netmap backend offloadings support

[Vincenzo Maffione]

The purpose of this patch series is to add offloadings support
(TSO/UFO/CSUM) to the netmap network backend, and make it possible
for the paravirtual network frontends (virtio-net and vmxnet3) to
use it.
In order to achieve this, these patches extend the existing
net.h interface to add abstract operations through which a network
frontend can manipulate backend offloading features, instead of
directly calling TAP-specific functions.

Guest-to-guest performance before this patches for virtio-net + netmap:

    TCP_STREAM                  5.0 Gbps
    TCP_RR                      12.7 Gbps
    UDP_STREAM (64-bytes)       790 Kpps

Guest-to-guest performance after this patches for virtio-net + netmap:

    TCP_STREAM                  21.4 Gbps
    TCP_RR                      12.7 Gbps
    UDP_STREAM (64-bytes)       790 Kpps

Experiment details:
    - Processor: Intel i7-3770K CPU @ 3.50GHz (8 cores)
    - Memory @ 1333 MHz
    - Host O.S.: Archlinux with Linux 3.11
    - Guest O.S.: Archlinux with Linux 3.11

    - QEMU command line:
        qemu-system-x86_64 archdisk.qcow -snapshot -enable-kvm -device virtio-net-pci,ioeventfd=on,mac=00:AA:BB:CC:DD:01,netdev=mynet -netdev netmap,ifname=vale0:01,id=mynet -smp 2 -vga std -m 3G


******** Changes against the previous version ***********
  (1) make tap offloading callbacks static
  (2) tap-win32.c modified to initialize offloading callback pointers
  (3) fix a bug when calling qemu_peer_using_vnet_header

Vincenzo Maffione (6):
  net: change vnet-hdr TAP prototypes
  net: extend NetClientInfo for offloading
  net: TAP uses NetClientInfo offloading callbacks
  net: virtio-net and vmxnet3 use offloading API
  net: make tap offloading callbacks static
  net: add offloading support to netmap backend

 hw/net/vhost_net.c  |  4 +--
 hw/net/virtio-net.c | 16 ++++------
 hw/net/vmxnet3.c    | 12 +++----
 include/net/net.h   | 19 +++++++++++
 include/net/tap.h   |  6 ----
 net/net.c           | 55 ++++++++++++++++++++++++++++++++
 net/netmap.c        | 68 ++++++++++++++++++++++++++++++++++++++-
 net/tap-win32.c     | 92 ++++++++++++++++++++++++++++-------------------------
 net/tap.c           | 20 ++++++++----
 9 files changed, 216 insertions(+), 76 deletions(-)

-- 
1.8.5.3

[Qemu-devel] [PATCH v4 1/6] net: change vnet-hdr TAP prototypes

[Vincenzo Maffione]

The tap_has_vnet_hdr() and tap_has_vnet_hdr_len() functions used
to return int, even though they only return true/false values.
This patch changes the prototypes to return bool.

Signed-off-by: Vincenzo Maffione <v.maffione@gmail.com>
---
 include/net/tap.h | 4 ++--
 net/tap-win32.c   | 8 ++++----
 net/tap.c         | 6 +++---
 3 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/include/net/tap.h b/include/net/tap.h
index a994f20..a3490a9 100644
--- a/include/net/tap.h
+++ b/include/net/tap.h
@@ -30,8 +30,8 @@
 #include "qapi-types.h"
 
 bool tap_has_ufo(NetClientState *nc);
-int tap_has_vnet_hdr(NetClientState *nc);
-int tap_has_vnet_hdr_len(NetClientState *nc, int len);
+bool tap_has_vnet_hdr(NetClientState *nc);
+bool tap_has_vnet_hdr_len(NetClientState *nc, int len);
 void tap_using_vnet_hdr(NetClientState *nc, bool using_vnet_hdr);
 void tap_set_offload(NetClientState *nc, int csum, int tso4, int tso6, int ecn, int ufo);
 void tap_set_vnet_hdr_len(NetClientState *nc, int len);
diff --git a/net/tap-win32.c b/net/tap-win32.c
index 91e9e84..edf26c4 100644
--- a/net/tap-win32.c
+++ b/net/tap-win32.c
@@ -727,9 +727,9 @@ bool tap_has_ufo(NetClientState *nc)
     return false;
 }
 
-int tap_has_vnet_hdr(NetClientState *nc)
+bool tap_has_vnet_hdr(NetClientState *nc)
 {
-    return 0;
+    return false;
 }
 
 int tap_probe_vnet_hdr_len(int fd, int len)
@@ -755,9 +755,9 @@ struct vhost_net *tap_get_vhost_net(NetClientState *nc)
     return NULL;
 }
 
-int tap_has_vnet_hdr_len(NetClientState *nc, int len)
+bool tap_has_vnet_hdr_len(NetClientState *nc, int len)
 {
-    return 0;
+    return false;
 }
 
 void tap_set_vnet_hdr_len(NetClientState *nc, int len)
diff --git a/net/tap.c b/net/tap.c
index 39c1cda..c805f3c 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -219,7 +219,7 @@ bool tap_has_ufo(NetClientState *nc)
     return s->has_ufo;
 }
 
-int tap_has_vnet_hdr(NetClientState *nc)
+bool tap_has_vnet_hdr(NetClientState *nc)
 {
     TAPState *s = DO_UPCAST(TAPState, nc, nc);
 
@@ -228,13 +228,13 @@ int tap_has_vnet_hdr(NetClientState *nc)
     return !!s->host_vnet_hdr_len;
 }
 
-int tap_has_vnet_hdr_len(NetClientState *nc, int len)
+bool tap_has_vnet_hdr_len(NetClientState *nc, int len)
 {
     TAPState *s = DO_UPCAST(TAPState, nc, nc);
 
     assert(nc->info->type == NET_CLIENT_OPTIONS_KIND_TAP);
 
-    return tap_probe_vnet_hdr_len(s->fd, len);
+    return !!tap_probe_vnet_hdr_len(s->fd, len);
 }
 
 void tap_set_vnet_hdr_len(NetClientState *nc, int len)
-- 
1.8.5.3

[Qemu-devel] [PATCH v4 2/6] net: extend NetClientInfo for offloading

[Vincenzo Maffione]

Some new callbacks have been added to generalize the operations done
by virtio-net and vmxnet3 frontends to manipulate TAP offloadings.

Signed-off-by: Vincenzo Maffione <v.maffione@gmail.com>
---
 include/net/net.h | 19 +++++++++++++++++++
 net/net.c         | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 74 insertions(+)

diff --git a/include/net/net.h b/include/net/net.h
index 11e1468..7b25394 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -50,6 +50,12 @@ typedef void (NetCleanup) (NetClientState *);
 typedef void (LinkStatusChanged)(NetClientState *);
 typedef void (NetClientDestructor)(NetClientState *);
 typedef RxFilterInfo *(QueryRxFilter)(NetClientState *);
+typedef bool (HasUfo)(NetClientState *);
+typedef bool (HasVnetHdr)(NetClientState *);
+typedef bool (HasVnetHdrLen)(NetClientState *, int);
+typedef void (UsingVnetHdr)(NetClientState *, bool);
+typedef void (SetOffload)(NetClientState *, int, int, int, int, int);
+typedef void (SetVnetHdrLen)(NetClientState *, int);
 
 typedef struct NetClientInfo {
     NetClientOptionsKind type;
@@ -62,6 +68,12 @@ typedef struct NetClientInfo {
     LinkStatusChanged *link_status_changed;
     QueryRxFilter *query_rx_filter;
     NetPoll *poll;
+    HasUfo *has_ufo;
+    HasVnetHdr *has_vnet_hdr;
+    HasVnetHdrLen *has_vnet_hdr_len;
+    UsingVnetHdr *using_vnet_hdr;
+    SetOffload *set_offload;
+    SetVnetHdrLen *set_vnet_hdr_len;
 } NetClientInfo;
 
 struct NetClientState {
@@ -120,6 +132,13 @@ ssize_t qemu_send_packet_async(NetClientState *nc, const uint8_t *buf,
 void qemu_purge_queued_packets(NetClientState *nc);
 void qemu_flush_queued_packets(NetClientState *nc);
 void qemu_format_nic_info_str(NetClientState *nc, uint8_t macaddr[6]);
+bool qemu_peer_has_ufo(NetClientState *nc);
+bool qemu_peer_has_vnet_hdr(NetClientState *nc);
+bool qemu_peer_has_vnet_hdr_len(NetClientState *nc, int len);
+void qemu_peer_using_vnet_hdr(NetClientState *nc, bool enable);
+void qemu_peer_set_offload(NetClientState *nc, int csum, int tso4, int tso6,
+                           int ecn, int ufo);
+void qemu_peer_set_vnet_hdr_len(NetClientState *nc, int len);
 void qemu_macaddr_default_if_unset(MACAddr *macaddr);
 int qemu_show_nic_models(const char *arg, const char *const *models);
 void qemu_check_nic_model(NICInfo *nd, const char *model);
diff --git a/net/net.c b/net/net.c
index 2c3af20..173673c 100644
--- a/net/net.c
+++ b/net/net.c
@@ -378,6 +378,61 @@ void qemu_foreach_nic(qemu_nic_foreach func, void *opaque)
     }
 }
 
+bool qemu_peer_has_ufo(NetClientState *nc)
+{
+    if (!nc->peer || !nc->peer->info->has_ufo) {
+        return false;
+    }
+
+    return nc->peer->info->has_ufo(nc->peer);
+}
+
+bool qemu_peer_has_vnet_hdr(NetClientState *nc)
+{
+    if (!nc->peer || !nc->peer->info->has_vnet_hdr) {
+        return false;
+    }
+
+    return nc->peer->info->has_vnet_hdr(nc->peer);
+}
+
+bool qemu_peer_has_vnet_hdr_len(NetClientState *nc, int len)
+{
+    if (!nc->peer || !nc->peer->info->has_vnet_hdr_len) {
+        return false;
+    }
+
+    return nc->peer->info->has_vnet_hdr_len(nc->peer, len);
+}
+
+void qemu_peer_using_vnet_hdr(NetClientState *nc, bool enable)
+{
+    if (!nc->peer || !nc->peer->info->using_vnet_hdr) {
+        return;
+    }
+
+    nc->peer->info->using_vnet_hdr(nc->peer, enable);
+}
+
+void qemu_peer_set_offload(NetClientState *nc, int csum, int tso4, int tso6,
+                          int ecn, int ufo)
+{
+    if (!nc->peer || !nc->peer->info->set_offload) {
+        return;
+    }
+
+    nc->peer->info->set_offload(nc->peer, csum, tso4, tso6, ecn, ufo);
+}
+
+void qemu_peer_set_vnet_hdr_len(NetClientState *nc, int len)
+{
+    if (!nc->peer || !nc->peer->info->set_vnet_hdr_len) {
+        return;
+    }
+
+    nc->peer->info->set_vnet_hdr_len(nc->peer, len);
+}
+
 int qemu_can_send_packet(NetClientState *sender)
 {
     if (!sender->peer) {
-- 
1.8.5.3

[Qemu-devel] [PATCH v4 3/6] net: TAP uses NetClientInfo offloading callbacks

[Vincenzo Maffione]

The TAP NetClientInfo structure is inizialized with the TAP-specific
functions that manipulates offloading features.

Signed-off-by: Vincenzo Maffione <v.maffione@gmail.com>
---
 net/tap-win32.c | 92 ++++++++++++++++++++++++++++++---------------------------
 net/tap.c       |  6 ++++
 2 files changed, 55 insertions(+), 43 deletions(-)

diff --git a/net/tap-win32.c b/net/tap-win32.c
index edf26c4..924ca55 100644
--- a/net/tap-win32.c
+++ b/net/tap-win32.c
@@ -669,11 +669,60 @@ static void tap_win32_send(void *opaque)
     }
 }
 
+bool tap_has_ufo(NetClientState *nc)
+{
+    return false;
+}
+
+bool tap_has_vnet_hdr(NetClientState *nc)
+{
+    return false;
+}
+
+int tap_probe_vnet_hdr_len(int fd, int len)
+{
+    return 0;
+}
+
+void tap_fd_set_vnet_hdr_len(int fd, int len)
+{
+}
+
+void tap_using_vnet_hdr(NetClientState *nc, bool using_vnet_hdr)
+{
+}
+
+void tap_set_offload(NetClientState *nc, int csum, int tso4,
+                     int tso6, int ecn, int ufo)
+{
+}
+
+struct vhost_net *tap_get_vhost_net(NetClientState *nc)
+{
+    return NULL;
+}
+
+bool tap_has_vnet_hdr_len(NetClientState *nc, int len)
+{
+    return false;
+}
+
+void tap_set_vnet_hdr_len(NetClientState *nc, int len)
+{
+    abort();
+}
+
 static NetClientInfo net_tap_win32_info = {
     .type = NET_CLIENT_OPTIONS_KIND_TAP,
     .size = sizeof(TAPState),
     .receive = tap_receive,
     .cleanup = tap_cleanup,
+    .has_ufo = tap_has_ufo,
+    .has_vnet_hdr = tap_has_vnet_hdr,
+    .has_vnet_hdr_len = tap_has_vnet_hdr_len,
+    .using_vnet_hdr = tap_using_vnet_hdr,
+    .set_offload = tap_set_offload,
+    .set_vnet_hdr_len = tap_set_vnet_hdr_len,
 };
 
 static int tap_win32_init(NetClientState *peer, const char *model,
@@ -722,49 +771,6 @@ int net_init_tap(const NetClientOptions *opts, const char *name,
     return 0;
 }
 
-bool tap_has_ufo(NetClientState *nc)
-{
-    return false;
-}
-
-bool tap_has_vnet_hdr(NetClientState *nc)
-{
-    return false;
-}
-
-int tap_probe_vnet_hdr_len(int fd, int len)
-{
-    return 0;
-}
-
-void tap_fd_set_vnet_hdr_len(int fd, int len)
-{
-}
-
-void tap_using_vnet_hdr(NetClientState *nc, bool using_vnet_hdr)
-{
-}
-
-void tap_set_offload(NetClientState *nc, int csum, int tso4,
-                     int tso6, int ecn, int ufo)
-{
-}
-
-struct vhost_net *tap_get_vhost_net(NetClientState *nc)
-{
-    return NULL;
-}
-
-bool tap_has_vnet_hdr_len(NetClientState *nc, int len)
-{
-    return false;
-}
-
-void tap_set_vnet_hdr_len(NetClientState *nc, int len)
-{
-    abort();
-}
-
 int tap_enable(NetClientState *nc)
 {
     abort();
diff --git a/net/tap.c b/net/tap.c
index c805f3c..d34ec88 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -314,6 +314,12 @@ static NetClientInfo net_tap_info = {
     .receive_iov = tap_receive_iov,
     .poll = tap_poll,
     .cleanup = tap_cleanup,
+    .has_ufo = tap_has_ufo,
+    .has_vnet_hdr = tap_has_vnet_hdr,
+    .has_vnet_hdr_len = tap_has_vnet_hdr_len,
+    .using_vnet_hdr = tap_using_vnet_hdr,
+    .set_offload = tap_set_offload,
+    .set_vnet_hdr_len = tap_set_vnet_hdr_len,
 };
 
 static TAPState *net_tap_fd_init(NetClientState *peer,
-- 
1.8.5.3

[Qemu-devel] [PATCH v4 4/6] net: virtio-net and vmxnet3 use offloading API

[Vincenzo Maffione]

With this patch, virtio-net and vmxnet3 frontends make
use of the qemu_peer_* API for backend offloadings manipulations,
instead of calling TAP-specific functions directly.
We also remove the existing checks which prevent those frontends
from using offloadings with backends different from TAP (e.g. netmap).

Signed-off-by: Vincenzo Maffione <v.maffione@gmail.com>
---
 hw/net/vhost_net.c  |  4 ++--
 hw/net/virtio-net.c | 16 ++++++----------
 hw/net/vmxnet3.c    | 12 +++++-------
 3 files changed, 13 insertions(+), 19 deletions(-)

diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 854997d..c90b9ec 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -106,7 +106,7 @@ struct vhost_net *vhost_net_init(NetClientState *backend, int devfd,
         goto fail;
     }
     net->nc = backend;
-    net->dev.backend_features = tap_has_vnet_hdr(backend) ? 0 :
+    net->dev.backend_features = backend->info->has_vnet_hdr(backend) ? 0 :
         (1 << VHOST_NET_F_VIRTIO_NET_HDR);
     net->backend = r;
 
@@ -117,7 +117,7 @@ struct vhost_net *vhost_net_init(NetClientState *backend, int devfd,
     if (r < 0) {
         goto fail;
     }
-    if (!tap_has_vnet_hdr_len(backend,
+    if (!backend->info->has_vnet_hdr_len(backend,
                               sizeof(struct virtio_net_hdr_mrg_rxbuf))) {
         net->dev.features &= ~(1 << VIRTIO_NET_F_MRG_RXBUF);
     }
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 3626608..cda8c75 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -325,11 +325,7 @@ static void peer_test_vnet_hdr(VirtIONet *n)
         return;
     }
 
-    if (nc->peer->info->type != NET_CLIENT_OPTIONS_KIND_TAP) {
-        return;
-    }
-
-    n->has_vnet_hdr = tap_has_vnet_hdr(nc->peer);
+    n->has_vnet_hdr = qemu_peer_has_vnet_hdr(nc);
 }
 
 static int peer_has_vnet_hdr(VirtIONet *n)
@@ -342,7 +338,7 @@ static int peer_has_ufo(VirtIONet *n)
     if (!peer_has_vnet_hdr(n))
         return 0;
 
-    n->has_ufo = tap_has_ufo(qemu_get_queue(n->nic)->peer);
+    n->has_ufo = qemu_peer_has_ufo(qemu_get_queue(n->nic));
 
     return n->has_ufo;
 }
@@ -361,8 +357,8 @@ static void virtio_net_set_mrg_rx_bufs(VirtIONet *n, int mergeable_rx_bufs)
         nc = qemu_get_subqueue(n->nic, i);
 
         if (peer_has_vnet_hdr(n) &&
-            tap_has_vnet_hdr_len(nc->peer, n->guest_hdr_len)) {
-            tap_set_vnet_hdr_len(nc->peer, n->guest_hdr_len);
+            qemu_peer_has_vnet_hdr_len(nc, n->guest_hdr_len)) {
+            qemu_peer_set_vnet_hdr_len(nc, n->guest_hdr_len);
             n->host_hdr_len = n->guest_hdr_len;
         }
     }
@@ -463,7 +459,7 @@ static uint32_t virtio_net_bad_features(VirtIODevice *vdev)
 
 static void virtio_net_apply_guest_offloads(VirtIONet *n)
 {
-    tap_set_offload(qemu_get_subqueue(n->nic, 0)->peer,
+    qemu_peer_set_offload(qemu_get_subqueue(n->nic, 0),
             !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_CSUM)),
             !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_TSO4)),
             !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_TSO6)),
@@ -1544,7 +1540,7 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
     peer_test_vnet_hdr(n);
     if (peer_has_vnet_hdr(n)) {
         for (i = 0; i < n->max_queues; i++) {
-            tap_using_vnet_hdr(qemu_get_subqueue(n->nic, i)->peer, true);
+            qemu_peer_using_vnet_hdr(qemu_get_subqueue(n->nic, i), true);
         }
         n->host_hdr_len = sizeof(struct virtio_net_hdr);
     } else {
diff --git a/hw/net/vmxnet3.c b/hw/net/vmxnet3.c
index 19687aa..0524684 100644
--- a/hw/net/vmxnet3.c
+++ b/hw/net/vmxnet3.c
@@ -1290,7 +1290,7 @@ static void vmxnet3_update_features(VMXNET3State *s)
               s->lro_supported, rxcso_supported,
               s->rx_vlan_stripping);
     if (s->peer_has_vhdr) {
-        tap_set_offload(qemu_get_queue(s->nic)->peer,
+        qemu_peer_set_offload(qemu_get_queue(s->nic),
                         rxcso_supported,
                         s->lro_supported,
                         s->lro_supported,
@@ -1883,11 +1883,9 @@ static NetClientInfo net_vmxnet3_info = {
 
 static bool vmxnet3_peer_has_vnet_hdr(VMXNET3State *s)
 {
-    NetClientState *peer = qemu_get_queue(s->nic)->peer;
+    NetClientState *nc = qemu_get_queue(s->nic);
 
-    if ((NULL != peer)                              &&
-        (peer->info->type == NET_CLIENT_OPTIONS_KIND_TAP)   &&
-        tap_has_vnet_hdr(peer)) {
+    if (qemu_peer_has_vnet_hdr(nc)) {
         return true;
     }
 
@@ -1935,10 +1933,10 @@ static void vmxnet3_net_init(VMXNET3State *s)
     s->lro_supported = false;
 
     if (s->peer_has_vhdr) {
-        tap_set_vnet_hdr_len(qemu_get_queue(s->nic)->peer,
+        qemu_peer_set_vnet_hdr_len(qemu_get_queue(s->nic),
             sizeof(struct virtio_net_hdr));
 
-        tap_using_vnet_hdr(qemu_get_queue(s->nic)->peer, 1);
+        qemu_peer_using_vnet_hdr(qemu_get_queue(s->nic), 1);
     }
 
     qemu_format_nic_info_str(qemu_get_queue(s->nic), s->conf.macaddr.a);
-- 
1.8.5.3

[Qemu-devel] [PATCH v4 5/6] net: make tap offloading callbacks static

[Vincenzo Maffione]

Since TAP offloadings are manipulated through a new API, it's
not necessary to export them in include/net/tap.h anymore.

Signed-off-by: Vincenzo Maffione <v.maffione@gmail.com>
---
 include/net/tap.h |  6 ------
 net/tap-win32.c   | 12 ++++++------
 net/tap.c         | 12 ++++++------
 3 files changed, 12 insertions(+), 18 deletions(-)

diff --git a/include/net/tap.h b/include/net/tap.h
index a3490a9..6daeb42 100644
--- a/include/net/tap.h
+++ b/include/net/tap.h
@@ -29,12 +29,6 @@
 #include "qemu-common.h"
 #include "qapi-types.h"
 
-bool tap_has_ufo(NetClientState *nc);
-bool tap_has_vnet_hdr(NetClientState *nc);
-bool tap_has_vnet_hdr_len(NetClientState *nc, int len);
-void tap_using_vnet_hdr(NetClientState *nc, bool using_vnet_hdr);
-void tap_set_offload(NetClientState *nc, int csum, int tso4, int tso6, int ecn, int ufo);
-void tap_set_vnet_hdr_len(NetClientState *nc, int len);
 int tap_enable(NetClientState *nc);
 int tap_disable(NetClientState *nc);
 
diff --git a/net/tap-win32.c b/net/tap-win32.c
index 924ca55..8aee611 100644
--- a/net/tap-win32.c
+++ b/net/tap-win32.c
@@ -669,12 +669,12 @@ static void tap_win32_send(void *opaque)
     }
 }
 
-bool tap_has_ufo(NetClientState *nc)
+static bool tap_has_ufo(NetClientState *nc)
 {
     return false;
 }
 
-bool tap_has_vnet_hdr(NetClientState *nc)
+static bool tap_has_vnet_hdr(NetClientState *nc)
 {
     return false;
 }
@@ -688,11 +688,11 @@ void tap_fd_set_vnet_hdr_len(int fd, int len)
 {
 }
 
-void tap_using_vnet_hdr(NetClientState *nc, bool using_vnet_hdr)
+static void tap_using_vnet_hdr(NetClientState *nc, bool using_vnet_hdr)
 {
 }
 
-void tap_set_offload(NetClientState *nc, int csum, int tso4,
+static void tap_set_offload(NetClientState *nc, int csum, int tso4,
                      int tso6, int ecn, int ufo)
 {
 }
@@ -702,12 +702,12 @@ struct vhost_net *tap_get_vhost_net(NetClientState *nc)
     return NULL;
 }
 
-bool tap_has_vnet_hdr_len(NetClientState *nc, int len)
+static bool tap_has_vnet_hdr_len(NetClientState *nc, int len)
 {
     return false;
 }
 
-void tap_set_vnet_hdr_len(NetClientState *nc, int len)
+static void tap_set_vnet_hdr_len(NetClientState *nc, int len)
 {
     abort();
 }
diff --git a/net/tap.c b/net/tap.c
index d34ec88..2d5099b 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -210,7 +210,7 @@ static void tap_send(void *opaque)
     } while (size > 0 && qemu_can_send_packet(&s->nc));
 }
 
-bool tap_has_ufo(NetClientState *nc)
+static bool tap_has_ufo(NetClientState *nc)
 {
     TAPState *s = DO_UPCAST(TAPState, nc, nc);
 
@@ -219,7 +219,7 @@ bool tap_has_ufo(NetClientState *nc)
     return s->has_ufo;
 }
 
-bool tap_has_vnet_hdr(NetClientState *nc)
+static bool tap_has_vnet_hdr(NetClientState *nc)
 {
     TAPState *s = DO_UPCAST(TAPState, nc, nc);
 
@@ -228,7 +228,7 @@ bool tap_has_vnet_hdr(NetClientState *nc)
     return !!s->host_vnet_hdr_len;
 }
 
-bool tap_has_vnet_hdr_len(NetClientState *nc, int len)
+static bool tap_has_vnet_hdr_len(NetClientState *nc, int len)
 {
     TAPState *s = DO_UPCAST(TAPState, nc, nc);
 
@@ -237,7 +237,7 @@ bool tap_has_vnet_hdr_len(NetClientState *nc, int len)
     return !!tap_probe_vnet_hdr_len(s->fd, len);
 }
 
-void tap_set_vnet_hdr_len(NetClientState *nc, int len)
+static void tap_set_vnet_hdr_len(NetClientState *nc, int len)
 {
     TAPState *s = DO_UPCAST(TAPState, nc, nc);
 
@@ -249,7 +249,7 @@ void tap_set_vnet_hdr_len(NetClientState *nc, int len)
     s->host_vnet_hdr_len = len;
 }
 
-void tap_using_vnet_hdr(NetClientState *nc, bool using_vnet_hdr)
+static void tap_using_vnet_hdr(NetClientState *nc, bool using_vnet_hdr)
 {
     TAPState *s = DO_UPCAST(TAPState, nc, nc);
 
@@ -259,7 +259,7 @@ void tap_using_vnet_hdr(NetClientState *nc, bool using_vnet_hdr)
     s->using_vnet_hdr = using_vnet_hdr;
 }
 
-void tap_set_offload(NetClientState *nc, int csum, int tso4,
+static void tap_set_offload(NetClientState *nc, int csum, int tso4,
                      int tso6, int ecn, int ufo)
 {
     TAPState *s = DO_UPCAST(TAPState, nc, nc);
-- 
1.8.5.3

[Qemu-devel] [PATCH v4 6/6] net: add offloading support to netmap backend

[Vincenzo Maffione]

Whit this patch, the netmap backend supports TSO/UFO/CSUM
offloadings, and accepts the virtio-net header, similarly to what
happens with TAP. The offloading callbacks in the NetClientInfo
interface have been implemented.

Signed-off-by: Vincenzo Maffione <v.maffione@gmail.com>
---
 net/netmap.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 67 insertions(+), 1 deletion(-)

diff --git a/net/netmap.c b/net/netmap.c
index 0ccc497..73f6d7a 100644
--- a/net/netmap.c
+++ b/net/netmap.c
@@ -31,6 +31,7 @@
 #include <net/netmap_user.h>
 
 #include "net/net.h"
+#include "net/tap.h"
 #include "clients.h"
 #include "sysemu/sysemu.h"
 #include "qemu/error-report.h"
@@ -54,6 +55,7 @@ typedef struct NetmapState {
     bool                read_poll;
     bool                write_poll;
     struct iovec        iov[IOV_MAX];
+    int                 vnet_hdr_len;  /* Current virtio-net header length. */
 } NetmapState;
 
 #define D(format, ...)                                          \
@@ -274,7 +276,7 @@ static ssize_t netmap_receive_iov(NetClientState *nc,
         return iov_size(iov, iovcnt);
     }
 
-    i = ring->cur;
+    last = i = ring->cur;
     avail = ring->avail;
 
     if (avail < iovcnt) {
@@ -394,6 +396,63 @@ static void netmap_cleanup(NetClientState *nc)
     s->me.fd = -1;
 }
 
+/* Offloading manipulation support callbacks. */
+static bool netmap_has_ufo(NetClientState *nc)
+{
+    return true;
+}
+
+static bool netmap_has_vnet_hdr(NetClientState *nc)
+{
+    return true;
+}
+
+static bool netmap_has_vnet_hdr_len(NetClientState *nc, int len)
+{
+    return len == 0 || len == sizeof(struct virtio_net_hdr) ||
+                len == sizeof(struct virtio_net_hdr_mrg_rxbuf);
+}
+
+static void netmap_using_vnet_hdr(NetClientState *nc, bool enable)
+{
+}
+
+static void netmap_set_vnet_hdr_len(NetClientState *nc, int len)
+{
+    NetmapState *s = DO_UPCAST(NetmapState, nc, nc);
+    int err;
+    struct nmreq req;
+
+    /* Issue a NETMAP_BDG_VNET_HDR command to change the virtio-net header
+     * length for the netmap adapter associated to 'me->ifname'.
+     */
+    memset(&req, 0, sizeof(req));
+    pstrcpy(req.nr_name, sizeof(req.nr_name), s->me.ifname);
+    req.nr_version = NETMAP_API;
+    req.nr_cmd = NETMAP_BDG_VNET_HDR;
+    req.nr_arg1 = len;
+    err = ioctl(s->me.fd, NIOCREGIF, &req);
+    if (err) {
+        error_report("Unable to execute NETMAP_BDG_VNET_HDR on %s: %s",
+                     s->me.ifname, strerror(errno));
+    } else {
+        /* Keep track of the current length. */
+        s->vnet_hdr_len = len;
+    }
+}
+
+static void netmap_set_offload(NetClientState *nc, int csum, int tso4, int tso6,
+                               int ecn, int ufo)
+{
+    NetmapState *s = DO_UPCAST(NetmapState, nc, nc);
+
+    /* Setting a virtio-net header length greater than zero automatically
+     * enables the offloadings.
+     */
+    if (!s->vnet_hdr_len) {
+        netmap_set_vnet_hdr_len(nc, sizeof(struct virtio_net_hdr));
+    }
+}
 
 /* NetClientInfo methods */
 static NetClientInfo net_netmap_info = {
@@ -403,6 +462,12 @@ static NetClientInfo net_netmap_info = {
     .receive_iov = netmap_receive_iov,
     .poll = netmap_poll,
     .cleanup = netmap_cleanup,
+    .has_ufo = netmap_has_ufo,
+    .has_vnet_hdr = netmap_has_vnet_hdr,
+    .has_vnet_hdr_len = netmap_has_vnet_hdr_len,
+    .using_vnet_hdr = netmap_using_vnet_hdr,
+    .set_offload = netmap_set_offload,
+    .set_vnet_hdr_len = netmap_set_vnet_hdr_len,
 };
 
 /* The exported init function
@@ -428,6 +493,7 @@ int net_init_netmap(const NetClientOptions *opts,
     nc = qemu_new_net_client(&net_netmap_info, peer, "netmap", name);
     s = DO_UPCAST(NetmapState, nc, nc);
     s->me = me;
+    s->vnet_hdr_len = 0;
     netmap_read_poll(s, true); /* Initially only poll for reads. */
 
     return 0;
-- 
1.8.5.3

Re: [Qemu-devel] [PATCH v4 0/6] Add netmap backend offloadings support

[Stefan Hajnoczi]

On Thu, Feb 06, 2014 at 05:02:14PM +0100, Vincenzo Maffione wrote:
> The purpose of this patch series is to add offloadings support
> (TSO/UFO/CSUM) to the netmap network backend, and make it possible
> for the paravirtual network frontends (virtio-net and vmxnet3) to
> use it.
> In order to achieve this, these patches extend the existing
> net.h interface to add abstract operations through which a network
> frontend can manipulate backend offloading features, instead of
> directly calling TAP-specific functions.
> 
> Guest-to-guest performance before this patches for virtio-net + netmap:
> 
>     TCP_STREAM                  5.0 Gbps
>     TCP_RR                      12.7 Gbps
>     UDP_STREAM (64-bytes)       790 Kpps
> 
> Guest-to-guest performance after this patches for virtio-net + netmap:
> 
>     TCP_STREAM                  21.4 Gbps
>     TCP_RR                      12.7 Gbps
>     UDP_STREAM (64-bytes)       790 Kpps
> 
> Experiment details:
>     - Processor: Intel i7-3770K CPU @ 3.50GHz (8 cores)
>     - Memory @ 1333 MHz
>     - Host O.S.: Archlinux with Linux 3.11
>     - Guest O.S.: Archlinux with Linux 3.11
> 
>     - QEMU command line:
>         qemu-system-x86_64 archdisk.qcow -snapshot -enable-kvm -device virtio-net-pci,ioeventfd=on,mac=00:AA:BB:CC:DD:01,netdev=mynet -netdev netmap,ifname=vale0:01,id=mynet -smp 2 -vga std -m 3G
> 
> 
> ******** Changes against the previous version ***********
>   (1) make tap offloading callbacks static
>   (2) tap-win32.c modified to initialize offloading callback pointers
>   (3) fix a bug when calling qemu_peer_using_vnet_header
> 
> Vincenzo Maffione (6):
>   net: change vnet-hdr TAP prototypes
>   net: extend NetClientInfo for offloading
>   net: TAP uses NetClientInfo offloading callbacks
>   net: virtio-net and vmxnet3 use offloading API
>   net: make tap offloading callbacks static
>   net: add offloading support to netmap backend
> 
>  hw/net/vhost_net.c  |  4 +--
>  hw/net/virtio-net.c | 16 ++++------
>  hw/net/vmxnet3.c    | 12 +++----
>  include/net/net.h   | 19 +++++++++++
>  include/net/tap.h   |  6 ----
>  net/net.c           | 55 ++++++++++++++++++++++++++++++++
>  net/netmap.c        | 68 ++++++++++++++++++++++++++++++++++++++-
>  net/tap-win32.c     | 92 ++++++++++++++++++++++++++++-------------------------
>  net/tap.c           | 20 ++++++++----
>  9 files changed, 216 insertions(+), 76 deletions(-)
> 
> -- 
> 1.8.5.3
> 
> 

Thanks, applied to my net tree:
https://github.com/stefanha/qemu/commits/net

Stefan