[Qemu-devel] [PATCH v11 00/12] Add a netfilter object and netbuffer filter

[Qemu-devel] [PATCH v11 00/12] Add a netfilter object and netbuffer filter

[Yang Hongyang]

This patch add an netfilter abstract object, captures all network packets
on associated netdev. Also implement a concrete filter buffer based on
this abstract object. the "buffer" netfilter could be used by VM FT solutions
like MicroCheckpointing, to buffer/release packets. Or to simulate
packet delay.

You can also get the series from:
https://github.com/macrosheep/qemu/tree/netfilter-v11

Usage:
 -netdev tap,id=bn0
 -device e1000,netdev=bn0
 -object filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000

dynamically add/remove netfilters:
 object_add filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000
 object_del f0

NOTE:
 interval's scale is microsecond.
 chain is optional, and is one of in|out|all, default is "all".
       "in" means this filter will receive packets sent to the @netdev
       "out" means this filter will receive packets sent from the @netdev
       "all" means this filter will receive packets both sent to/from
             the @netdev

v11:
 - address Jason&Daniel's comments
 - add multiqueue support, the last 2 patches
 - rebased to the latest master

v10:
 - Reimplemented using QOM (suggested by stefan)
 - Do not export NetQueue internals (suggested by stefan)
 - see individual patch for detail

v9:
 - squash command description and help to patch 1&3
 - qapi changes according to Markus&Eric's comments
 - see individual patch for detail

v8:
 - some minor fixes according to Thomas's comments
 - rebased to the latest master branch

v7:
 - print filter info when execute 'info network'
 - addressed Jason's comments

v6:
 - add multiqueue support, please see individual patch for detail

v5:
 - add a sent_cb param to filter receive_iov api
 - squash the 4th patch into patch 3
 - remove dummy sent_cb (buffer filter)
 - addressed Jason's other comments, see individual patches for detail

v4:
 - get rid of struct Filter
 - squash the 4th patch into patch 2
 - fix qemu_netfilter_pass_to_next_iov
 - get rid of bh (buffer filter)
 - release the packet to next filter instead of to receiver (buffer filter)

v3:
 - add an api to pass the packet to next filter
 - remove netfilters when delete netdev
 - add qtest testcases for netfilter
 - addressed comments from Jason

v2:
 - add a chain option to netfilter object
 - move the hook place earlier, before net_queue_send
 - drop the unused api in buffer filter
 - squash buffer filter patches into one
 - remove receive() api from netfilter, only receive_iov() is enough
 - addressed comments from Jason&Thomas

v1:
 initial patch.

Yang Hongyang (12):
  qmp: delete qemu opts when delete an object
  init/cleanup of netfilter object
  netfilter: hook packets before net queue send
  net: merge qemu_deliver_packet and qemu_deliver_packet_iov
  net/queue: introduce NetQueueDeliverFunc
  netfilter: add an API to pass the packet to next filter
  netfilter: print filter info associate with the netdev
  net/queue: export qemu_net_queue_append_iov
  netfilter: add a netbuffer filter
  tests: add test cases for netfilter object
  netfilter/multiqueue: introduce netfilter name
  netfilter: add multiqueue support

 include/net/filter.h    |  76 ++++++++++++
 include/net/net.h       |   6 +-
 include/net/queue.h     |  20 +++-
 include/qemu/typedefs.h |   1 +
 net/Makefile.objs       |   2 +
 net/filter-buffer.c     | 170 +++++++++++++++++++++++++++
 net/filter.c            | 306 ++++++++++++++++++++++++++++++++++++++++++++++++
 net/net.c               | 108 +++++++++++++----
 net/queue.c             |  24 ++--
 qapi-schema.json        |  18 +++
 qemu-options.hx         |  18 +++
 qmp.c                   |   4 +
 tests/.gitignore        |   1 +
 tests/Makefile          |   2 +
 tests/test-netfilter.c  | 200 +++++++++++++++++++++++++++++++
 vl.c                    |  18 ++-
 16 files changed, 927 insertions(+), 47 deletions(-)
 create mode 100644 include/net/filter.h
 create mode 100644 net/filter-buffer.c
 create mode 100644 net/filter.c
 create mode 100644 tests/test-netfilter.c

-- 
1.9.1

[Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Yang Hongyang]

When delete an object, we need to delete the associated qemu opts,
otherwise, we can not add another object with the same name using
object_add.
The case happens when we start qemu with:
-object xxx,id=aa
then we delete this object with:
object_del aa
then add the object with:
object_add xxx,id=aa

QEMU will return error with Duplicate ID error.

Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
---
 qmp.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/qmp.c b/qmp.c
index 9623c80..4bd44c3 100644
--- a/qmp.c
+++ b/qmp.c
@@ -686,6 +686,7 @@ void qmp_object_del(const char *id, Error **errp)
 {
     Object *container;
     Object *obj;
+    QemuOpts *opts;
 
     container = object_get_objects_root();
     obj = object_resolve_path_component(container, id);
@@ -699,6 +700,9 @@ void qmp_object_del(const char *id, Error **errp)
         return;
     }
     object_unparent(obj);
+
+    opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
+    qemu_opts_del(opts);
 }
 
 MemoryDeviceInfoList *qmp_query_memory_devices(Error **errp)
-- 
1.9.1

[Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Yang Hongyang]

Add a netfilter object based on QOM.

A netfilter is attached to a netdev, captures all network packets
that pass through the netdev. When we delete the netdev, we also
delete the netfilter object attached to it, because if the netdev is
removed, the filter which attached to it is useless.

QTAILQ_ENTRY next used by netdev, filter belongs to the specific netdev is
in this queue.

Also init delayed object after net_init_clients, because netfilters need
to be initialized after net clients initialized.

Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
---
v11: no need to free nf->netdev_id, it will be auto freeed while object deleted
     remove global_list net_filters, will add back when needed
v10: use QOM for netfilter
v9: use flat union instead of simple union in QAPI schema
v8: include vhost_net header
v7: add check for vhost
    fix error propagate bug
v6: add multiqueue support (net_filter_init1)
v5: remove model from NetFilterState
    add a sent_cb param to receive_iov API
---
 include/net/filter.h    |  60 +++++++++++++++++++++
 include/net/net.h       |   1 +
 include/qemu/typedefs.h |   1 +
 net/Makefile.objs       |   1 +
 net/filter.c            | 138 ++++++++++++++++++++++++++++++++++++++++++++++++
 net/net.c               |   7 +++
 qapi-schema.json        |  18 +++++++
 vl.c                    |  13 ++---
 8 files changed, 233 insertions(+), 6 deletions(-)
 create mode 100644 include/net/filter.h
 create mode 100644 net/filter.c

diff --git a/include/net/filter.h b/include/net/filter.h
new file mode 100644
index 0000000..226f2f7
--- /dev/null
+++ b/include/net/filter.h
@@ -0,0 +1,60 @@
+/*
+ * Copyright (c) 2015 FUJITSU LIMITED
+ * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.
+ */
+
+#ifndef QEMU_NET_FILTER_H
+#define QEMU_NET_FILTER_H
+
+#include "qom/object.h"
+#include "qemu-common.h"
+#include "qemu/typedefs.h"
+#include "net/queue.h"
+
+#define TYPE_NETFILTER "netfilter"
+#define NETFILTER(obj) \
+    OBJECT_CHECK(NetFilterState, (obj), TYPE_NETFILTER)
+#define NETFILTER_GET_CLASS(obj) \
+    OBJECT_GET_CLASS(NetFilterClass, (obj), TYPE_NETFILTER)
+#define NETFILTER_CLASS(klass) \
+    OBJECT_CLASS_CHECK(NetFilterClass, (klass), TYPE_NETFILTER)
+
+typedef void (FilterSetup) (NetFilterState *nf, Error **errp);
+typedef void (FilterCleanup) (NetFilterState *nf);
+/*
+ * Return:
+ *   0: finished handling the packet, we should continue
+ *   size: filter stolen this packet, we stop pass this packet further
+ */
+typedef ssize_t (FilterReceiveIOV)(NetFilterState *nc,
+                                   NetClientState *sender,
+                                   unsigned flags,
+                                   const struct iovec *iov,
+                                   int iovcnt,
+                                   NetPacketSent *sent_cb);
+
+struct NetFilterClass {
+    ObjectClass parent_class;
+
+    FilterSetup *setup;
+    FilterCleanup *cleanup;
+    FilterReceiveIOV *receive_iov;
+};
+typedef struct NetFilterClass NetFilterClass;
+
+
+struct NetFilterState {
+    /* private */
+    Object parent;
+
+    /* protected */
+    char *netdev_id;
+    NetClientState *netdev;
+    NetFilterChain chain;
+    QTAILQ_ENTRY(NetFilterState) next;
+};
+
+#endif /* QEMU_NET_FILTER_H */
diff --git a/include/net/net.h b/include/net/net.h
index 6a6cbef..36e5fab 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -92,6 +92,7 @@ struct NetClientState {
     NetClientDestructor *destructor;
     unsigned int queue_index;
     unsigned rxfilter_notify_enabled:1;
+    QTAILQ_HEAD(, NetFilterState) filters;
 };
 
 typedef struct NICState {
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index f8a9dd6..2c0648f 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -45,6 +45,7 @@ typedef struct Monitor Monitor;
 typedef struct MouseTransformInfo MouseTransformInfo;
 typedef struct MSIMessage MSIMessage;
 typedef struct NetClientState NetClientState;
+typedef struct NetFilterState NetFilterState;
 typedef struct NICInfo NICInfo;
 typedef struct PcGuestInfo PcGuestInfo;
 typedef struct PCIBridge PCIBridge;
diff --git a/net/Makefile.objs b/net/Makefile.objs
index ec19cb3..914aec0 100644
--- a/net/Makefile.objs
+++ b/net/Makefile.objs
@@ -13,3 +13,4 @@ common-obj-$(CONFIG_HAIKU) += tap-haiku.o
 common-obj-$(CONFIG_SLIRP) += slirp.o
 common-obj-$(CONFIG_VDE) += vde.o
 common-obj-$(CONFIG_NETMAP) += netmap.o
+common-obj-y += filter.o
diff --git a/net/filter.c b/net/filter.c
new file mode 100644
index 0000000..3b810c8
--- /dev/null
+++ b/net/filter.c
@@ -0,0 +1,138 @@
+/*
+ * Copyright (c) 2015 FUJITSU LIMITED
+ * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.
+ */
+
+#include "qemu-common.h"
+#include "qapi/qmp/qerror.h"
+#include "qemu/error-report.h"
+
+#include "net/filter.h"
+#include "net/net.h"
+#include "net/vhost_net.h"
+#include "qom/object_interfaces.h"
+
+static char *netfilter_get_netdev_id(Object *obj, Error **errp)
+{
+    NetFilterState *nf = NETFILTER(obj);
+
+    return g_strdup(nf->netdev_id);
+}
+
+static void netfilter_set_netdev_id(Object *obj, const char *str, Error **errp)
+{
+    NetFilterState *nf = NETFILTER(obj);
+
+    nf->netdev_id = g_strdup(str);
+}
+
+static int netfilter_get_chain(Object *obj, Error **errp G_GNUC_UNUSED)
+{
+    NetFilterState *nf = NETFILTER(obj);
+    return nf->chain;
+}
+
+static void netfilter_set_chain(Object *obj, int chain, Error **errp)
+{
+    NetFilterState *nf = NETFILTER(obj);
+    nf->chain = chain;
+}
+
+static void netfilter_init(Object *obj)
+{
+    object_property_add_str(obj, "netdev",
+                            netfilter_get_netdev_id, netfilter_set_netdev_id,
+                            NULL);
+    object_property_add_enum(obj, "chain", "NetFilterChain",
+                             NetFilterChain_lookup,
+                             netfilter_get_chain, netfilter_set_chain,
+                             NULL);
+}
+
+static void netfilter_finalize(Object *obj)
+{
+    NetFilterState *nf = NETFILTER(obj);
+    NetFilterClass *nfc = NETFILTER_GET_CLASS(obj);
+
+    if (nfc->cleanup) {
+        nfc->cleanup(nf);
+    }
+
+    if (nf->netdev && !QTAILQ_EMPTY(&nf->netdev->filters)) {
+        QTAILQ_REMOVE(&nf->netdev->filters, nf, next);
+    }
+}
+
+static void netfilter_complete(UserCreatable *uc, Error **errp)
+{
+    NetFilterState *nf = NETFILTER(uc);
+    NetClientState *ncs[MAX_QUEUE_NUM];
+    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc);
+    int queues;
+    Error *local_err = NULL;
+
+    if (!nf->netdev_id) {
+        error_setg(errp, "Parameter 'netdev' is required");
+        return;
+    }
+
+    queues = qemu_find_net_clients_except(nf->netdev_id, ncs,
+                                          NET_CLIENT_OPTIONS_KIND_NIC,
+                                          MAX_QUEUE_NUM);
+    if (queues < 1) {
+        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "netdev",
+                   "a network backend id");
+        return;
+    } else if (queues > 1) {
+        error_setg(errp, "Multi queue is not supported");
+        return;
+    }
+
+    if (get_vhost_net(ncs[0])) {
+        error_setg(errp, "Vhost is not supported");
+        return;
+    }
+
+    nf->netdev = ncs[0];
+
+    if (nfc->setup) {
+        nfc->setup(nf, &local_err);
+        if (local_err) {
+            error_propagate(errp, local_err);
+            return;
+        }
+    }
+    QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);
+}
+
+static void netfilter_class_init(ObjectClass *oc, void *data)
+{
+    UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
+
+    ucc->complete = netfilter_complete;
+}
+
+static const TypeInfo netfilter_info = {
+    .name = TYPE_NETFILTER,
+    .parent = TYPE_OBJECT,
+    .abstract = true,
+    .class_size = sizeof(NetFilterClass),
+    .class_init = netfilter_class_init,
+    .instance_size = sizeof(NetFilterState),
+    .instance_init = netfilter_init,
+    .instance_finalize = netfilter_finalize,
+    .interfaces = (InterfaceInfo[]) {
+        { TYPE_USER_CREATABLE },
+        { }
+    }
+};
+
+static void register_types(void)
+{
+    type_register_static(&netfilter_info);
+}
+
+type_init(register_types);
diff --git a/net/net.c b/net/net.c
index 28a5597..033f4f3 100644
--- a/net/net.c
+++ b/net/net.c
@@ -44,6 +44,7 @@
 #include "qapi/opts-visitor.h"
 #include "qapi/dealloc-visitor.h"
 #include "sysemu/sysemu.h"
+#include "net/filter.h"
 
 /* Net bridge is currently not supported for W32. */
 #if !defined(_WIN32)
@@ -287,6 +288,7 @@ static void qemu_net_client_setup(NetClientState *nc,
 
     nc->incoming_queue = qemu_new_net_queue(nc);
     nc->destructor = destructor;
+    QTAILQ_INIT(&nc->filters);
 }
 
 NetClientState *qemu_new_net_client(NetClientInfo *info,
@@ -384,6 +386,7 @@ void qemu_del_net_client(NetClientState *nc)
 {
     NetClientState *ncs[MAX_QUEUE_NUM];
     int queues, i;
+    NetFilterState *nf, *next;
 
     assert(nc->info->type != NET_CLIENT_OPTIONS_KIND_NIC);
 
@@ -395,6 +398,10 @@ void qemu_del_net_client(NetClientState *nc)
                                           MAX_QUEUE_NUM);
     assert(queues != 0);
 
+    QTAILQ_FOREACH_SAFE(nf, &nc->filters, next, next) {
+        object_unparent(OBJECT(nf));
+    }
+
     /* If there is a peer NIC, delete and cleanup client, but do not free. */
     if (nc->peer && nc->peer->info->type == NET_CLIENT_OPTIONS_KIND_NIC) {
         NICState *nic = qemu_get_nic(nc->peer);
diff --git a/qapi-schema.json b/qapi-schema.json
index 2bada60..546500a 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2551,6 +2551,24 @@
     'opts': 'NetClientOptions' } }
 
 ##
+# @NetFilterChain
+#
+# netfilter chain, a netfilter is attached to a netdev, captures the
+# network packets of the netdev.
+#
+# @all: the filter will receive packets both sent to/from the netdev, this
+#       is the default chain.
+#
+# @in: the filter will receive packets sent to the netdev.
+#
+# @out: the filter will receive packets sent from the netdev.
+#
+# Since 2.5
+##
+{ 'enum': 'NetFilterChain',
+  'data': [ 'all', 'in', 'out' ] }
+
+##
 # @InetSocketAddress
 #
 # Captures a socket address or address range in the Internet namespace.
diff --git a/vl.c b/vl.c
index 066a080..ec589e2 100644
--- a/vl.c
+++ b/vl.c
@@ -2794,6 +2794,7 @@ static bool object_create_initial(const char *type)
     if (g_str_equal(type, "rng-egd")) {
         return false;
     }
+    /* TODO: return false for concrete netfilters */
     return true;
 }
 
@@ -4313,12 +4314,6 @@ int main(int argc, char **argv, char **envp)
         exit(0);
     }
 
-    if (qemu_opts_foreach(qemu_find_opts("object"),
-                          object_create,
-                          object_create_delayed, NULL)) {
-        exit(1);
-    }
-
     machine_opts = qemu_get_machine_opts();
     if (qemu_opt_foreach(machine_opts, machine_set_property, current_machine,
                          NULL)) {
@@ -4424,6 +4419,12 @@ int main(int argc, char **argv, char **envp)
         exit(1);
     }
 
+    if (qemu_opts_foreach(qemu_find_opts("object"),
+                          object_create,
+                          object_create_delayed, NULL)) {
+        exit(1);
+    }
+
 #ifdef CONFIG_TPM
     if (tpm_init() < 0) {
         exit(1);
-- 
1.9.1

[Qemu-devel] [PATCH v11 03/12] netfilter: hook packets before net queue send

[Yang Hongyang]

Capture packets that will be sent.

Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
v11: introduce qemu_netfilter_receive() a avoid dup code
v10: adjust due to QOM
v5: do not check ret against iov_size
    pass sent_cb to filters
---
 include/net/filter.h |  7 ++++++
 net/filter.c         | 15 ++++++++++++
 net/net.c            | 64 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 86 insertions(+)

diff --git a/include/net/filter.h b/include/net/filter.h
index 226f2f7..69902ea 100644
--- a/include/net/filter.h
+++ b/include/net/filter.h
@@ -57,4 +57,11 @@ struct NetFilterState {
     QTAILQ_ENTRY(NetFilterState) next;
 };
 
+ssize_t qemu_netfilter_receive(NetFilterState *nf, NetFilterChain chain,
+                               NetClientState *sender,
+                               unsigned flags,
+                               const struct iovec *iov,
+                               int iovcnt,
+                               NetPacketSent *sent_cb);
+
 #endif /* QEMU_NET_FILTER_H */
diff --git a/net/filter.c b/net/filter.c
index 3b810c8..905817d 100644
--- a/net/filter.c
+++ b/net/filter.c
@@ -15,6 +15,21 @@
 #include "net/vhost_net.h"
 #include "qom/object_interfaces.h"
 
+ssize_t qemu_netfilter_receive(NetFilterState *nf, NetFilterChain chain,
+                               NetClientState *sender,
+                               unsigned flags,
+                               const struct iovec *iov,
+                               int iovcnt,
+                               NetPacketSent *sent_cb)
+{
+    if (nf->chain == chain || nf->chain == NET_FILTER_CHAIN_ALL) {
+        return NETFILTER_GET_CLASS(OBJECT(nf))->receive_iov(
+                                   nf, sender, flags, iov, iovcnt, sent_cb);
+    }
+
+    return 0;
+}
+
 static char *netfilter_get_netdev_id(Object *obj, Error **errp)
 {
     NetFilterState *nf = NETFILTER(obj);
diff --git a/net/net.c b/net/net.c
index 033f4f3..ad37419 100644
--- a/net/net.c
+++ b/net/net.c
@@ -561,6 +561,42 @@ int qemu_can_send_packet(NetClientState *sender)
     return 1;
 }
 
+static ssize_t filter_receive_iov(NetClientState *nc, NetFilterChain chain,
+                                  NetClientState *sender,
+                                  unsigned flags,
+                                  const struct iovec *iov,
+                                  int iovcnt,
+                                  NetPacketSent *sent_cb)
+{
+    ssize_t ret = 0;
+    NetFilterState *nf = NULL;
+
+    QTAILQ_FOREACH(nf, &nc->filters, next) {
+        ret = qemu_netfilter_receive(nf, chain, sender, flags, iov,
+                                     iovcnt, sent_cb);
+        if (ret) {
+            return ret;
+        }
+    }
+
+    return ret;
+}
+
+static ssize_t filter_receive(NetClientState *nc, NetFilterChain chain,
+                              NetClientState *sender,
+                              unsigned flags,
+                              const uint8_t *data,
+                              size_t size,
+                              NetPacketSent *sent_cb)
+{
+    struct iovec iov = {
+        .iov_base = (void *)data,
+        .iov_len = size
+    };
+
+    return filter_receive_iov(nc, chain, sender, flags, &iov, 1, sent_cb);
+}
+
 ssize_t qemu_deliver_packet(NetClientState *sender,
                             unsigned flags,
                             const uint8_t *data,
@@ -632,6 +668,7 @@ static ssize_t qemu_send_packet_async_with_flags(NetClientState *sender,
                                                  NetPacketSent *sent_cb)
 {
     NetQueue *queue;
+    int ret;
 
 #ifdef DEBUG_NET
     printf("qemu_send_packet_async:\n");
@@ -642,6 +679,19 @@ static ssize_t qemu_send_packet_async_with_flags(NetClientState *sender,
         return size;
     }
 
+    /* Let filters handle the packet first */
+    ret = filter_receive(sender, NET_FILTER_CHAIN_OUT,
+                         sender, flags, buf, size, sent_cb);
+    if (ret) {
+        return ret;
+    }
+
+    ret = filter_receive(sender->peer, NET_FILTER_CHAIN_IN,
+                         sender, flags, buf, size, sent_cb);
+    if (ret) {
+        return ret;
+    }
+
     queue = sender->peer->incoming_queue;
 
     return qemu_net_queue_send(queue, sender, flags, buf, size, sent_cb);
@@ -712,11 +762,25 @@ ssize_t qemu_sendv_packet_async(NetClientState *sender,
                                 NetPacketSent *sent_cb)
 {
     NetQueue *queue;
+    int ret;
 
     if (sender->link_down || !sender->peer) {
         return iov_size(iov, iovcnt);
     }
 
+    /* Let filters handle the packet first */
+    ret = filter_receive_iov(sender, NET_FILTER_CHAIN_OUT, sender,
+                             QEMU_NET_PACKET_FLAG_NONE, iov, iovcnt, sent_cb);
+    if (ret) {
+        return ret;
+    }
+
+    ret = filter_receive_iov(sender->peer, NET_FILTER_CHAIN_IN, sender,
+                             QEMU_NET_PACKET_FLAG_NONE, iov, iovcnt, sent_cb);
+    if (ret) {
+        return ret;
+    }
+
     queue = sender->peer->incoming_queue;
 
     return qemu_net_queue_send_iov(queue, sender,
-- 
1.9.1

[Qemu-devel] [PATCH v11 04/12] net: merge qemu_deliver_packet and qemu_deliver_packet_iov

[Yang Hongyang]

qemu_deliver_packet_iov already have the compat delivery, we
can drop qemu_deliver_packet.

Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
---
 include/net/net.h |  5 -----
 net/net.c         | 40 +++++++---------------------------------
 net/queue.c       |  6 +++++-
 3 files changed, 12 insertions(+), 39 deletions(-)

diff --git a/include/net/net.h b/include/net/net.h
index 36e5fab..7af3e15 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -152,11 +152,6 @@ void qemu_check_nic_model(NICInfo *nd, const char *model);
 int qemu_find_nic_model(NICInfo *nd, const char * const *models,
                         const char *default_model);
 
-ssize_t qemu_deliver_packet(NetClientState *sender,
-                            unsigned flags,
-                            const uint8_t *data,
-                            size_t size,
-                            void *opaque);
 ssize_t qemu_deliver_packet_iov(NetClientState *sender,
                             unsigned flags,
                             const struct iovec *iov,
diff --git a/net/net.c b/net/net.c
index ad37419..ca35b27 100644
--- a/net/net.c
+++ b/net/net.c
@@ -597,36 +597,6 @@ static ssize_t filter_receive(NetClientState *nc, NetFilterChain chain,
     return filter_receive_iov(nc, chain, sender, flags, &iov, 1, sent_cb);
 }
 
-ssize_t qemu_deliver_packet(NetClientState *sender,
-                            unsigned flags,
-                            const uint8_t *data,
-                            size_t size,
-                            void *opaque)
-{
-    NetClientState *nc = opaque;
-    ssize_t ret;
-
-    if (nc->link_down) {
-        return size;
-    }
-
-    if (nc->receive_disabled) {
-        return 0;
-    }
-
-    if (flags & QEMU_NET_PACKET_FLAG_RAW && nc->info->receive_raw) {
-        ret = nc->info->receive_raw(nc, data, size);
-    } else {
-        ret = nc->info->receive(nc, data, size);
-    }
-
-    if (ret == 0) {
-        nc->receive_disabled = 1;
-    }
-
-    return ret;
-}
-
 void qemu_purge_queued_packets(NetClientState *nc)
 {
     if (!nc->peer) {
@@ -717,14 +687,18 @@ ssize_t qemu_send_packet_raw(NetClientState *nc, const uint8_t *buf, int size)
 }
 
 static ssize_t nc_sendv_compat(NetClientState *nc, const struct iovec *iov,
-                               int iovcnt)
+                               int iovcnt, unsigned flags)
 {
     uint8_t buffer[NET_BUFSIZE];
     size_t offset;
 
     offset = iov_to_buf(iov, iovcnt, 0, buffer, sizeof(buffer));
 
-    return nc->info->receive(nc, buffer, offset);
+    if (flags & QEMU_NET_PACKET_FLAG_RAW && nc->info->receive_raw) {
+        return nc->info->receive_raw(nc, buffer, offset);
+    } else {
+        return nc->info->receive(nc, buffer, offset);
+    }
 }
 
 ssize_t qemu_deliver_packet_iov(NetClientState *sender,
@@ -747,7 +721,7 @@ ssize_t qemu_deliver_packet_iov(NetClientState *sender,
     if (nc->info->receive_iov) {
         ret = nc->info->receive_iov(nc, iov, iovcnt);
     } else {
-        ret = nc_sendv_compat(nc, iov, iovcnt);
+        ret = nc_sendv_compat(nc, iov, iovcnt, flags);
     }
 
     if (ret == 0) {
diff --git a/net/queue.c b/net/queue.c
index ebbe2bb..cf8db3a 100644
--- a/net/queue.c
+++ b/net/queue.c
@@ -152,9 +152,13 @@ static ssize_t qemu_net_queue_deliver(NetQueue *queue,
                                       size_t size)
 {
     ssize_t ret = -1;
+    struct iovec iov = {
+        .iov_base = (void *)data,
+        .iov_len = size
+    };
 
     queue->delivering = 1;
-    ret = qemu_deliver_packet(sender, flags, data, size, queue->opaque);
+    ret = qemu_deliver_packet_iov(sender, flags, &iov, 1, queue->opaque);
     queue->delivering = 0;
 
     return ret;
-- 
1.9.1

[Qemu-devel] [PATCH v11 05/12] net/queue: introduce NetQueueDeliverFunc

[Yang Hongyang]

net/queue.c has logic to send/queue/flush packets but a
qemu_deliver_packet_iov() call is hardcoded. Abstract this
func so that we can use our own deliver function in netfilter.

Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/net/queue.h | 13 ++++++++++++-
 net/net.c           |  2 +-
 net/queue.c         |  8 +++++---
 3 files changed, 18 insertions(+), 5 deletions(-)

diff --git a/include/net/queue.h b/include/net/queue.h
index fc02b33..b4a7183 100644
--- a/include/net/queue.h
+++ b/include/net/queue.h
@@ -34,7 +34,18 @@ typedef void (NetPacketSent) (NetClientState *sender, ssize_t ret);
 #define QEMU_NET_PACKET_FLAG_NONE  0
 #define QEMU_NET_PACKET_FLAG_RAW  (1<<0)
 
-NetQueue *qemu_new_net_queue(void *opaque);
+/* Returns:
+ *   >0 - success
+ *    0 - queue packet for future redelivery
+ *   <0 - failure (discard packet)
+ */
+typedef ssize_t (NetQueueDeliverFunc)(NetClientState *sender,
+                                      unsigned flags,
+                                      const struct iovec *iov,
+                                      int iovcnt,
+                                      void *opaque);
+
+NetQueue *qemu_new_net_queue(NetQueueDeliverFunc *deliver, void *opaque);
 
 void qemu_del_net_queue(NetQueue *queue);
 
diff --git a/net/net.c b/net/net.c
index ca35b27..845daf2 100644
--- a/net/net.c
+++ b/net/net.c
@@ -286,7 +286,7 @@ static void qemu_net_client_setup(NetClientState *nc,
     }
     QTAILQ_INSERT_TAIL(&net_clients, nc, next);
 
-    nc->incoming_queue = qemu_new_net_queue(nc);
+    nc->incoming_queue = qemu_new_net_queue(qemu_deliver_packet_iov, nc);
     nc->destructor = destructor;
     QTAILQ_INIT(&nc->filters);
 }
diff --git a/net/queue.c b/net/queue.c
index cf8db3a..16dddf0 100644
--- a/net/queue.c
+++ b/net/queue.c
@@ -52,13 +52,14 @@ struct NetQueue {
     void *opaque;
     uint32_t nq_maxlen;
     uint32_t nq_count;
+    NetQueueDeliverFunc *deliver;
 
     QTAILQ_HEAD(packets, NetPacket) packets;
 
     unsigned delivering : 1;
 };
 
-NetQueue *qemu_new_net_queue(void *opaque)
+NetQueue *qemu_new_net_queue(NetQueueDeliverFunc *deliver, void *opaque)
 {
     NetQueue *queue;
 
@@ -67,6 +68,7 @@ NetQueue *qemu_new_net_queue(void *opaque)
     queue->opaque = opaque;
     queue->nq_maxlen = 10000;
     queue->nq_count = 0;
+    queue->deliver = deliver;
 
     QTAILQ_INIT(&queue->packets);
 
@@ -158,7 +160,7 @@ static ssize_t qemu_net_queue_deliver(NetQueue *queue,
     };
 
     queue->delivering = 1;
-    ret = qemu_deliver_packet_iov(sender, flags, &iov, 1, queue->opaque);
+    ret = queue->deliver(sender, flags, &iov, 1, queue->opaque);
     queue->delivering = 0;
 
     return ret;
@@ -173,7 +175,7 @@ static ssize_t qemu_net_queue_deliver_iov(NetQueue *queue,
     ssize_t ret = -1;
 
     queue->delivering = 1;
-    ret = qemu_deliver_packet_iov(sender, flags, iov, iovcnt, queue->opaque);
+    ret = queue->deliver(sender, flags, iov, iovcnt, queue->opaque);
     queue->delivering = 0;
 
     return ret;
-- 
1.9.1

[Qemu-devel] [PATCH v11 06/12] netfilter: add an API to pass the packet to next filter

[Yang Hongyang]

add an API qemu_netfilter_pass_to_next() to pass the packet
to next filter.

Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
v11: use qemu_netfilter_receive to avoid dup code
v10: adjust as a NetQueueDeliverFunc
v9: fix a bug when curr filter chain is all
v5: fold params to NetPacket struct
---
 include/net/filter.h |  7 +++++++
 net/filter.c         | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)

diff --git a/include/net/filter.h b/include/net/filter.h
index 69902ea..90ab101 100644
--- a/include/net/filter.h
+++ b/include/net/filter.h
@@ -64,4 +64,11 @@ ssize_t qemu_netfilter_receive(NetFilterState *nf, NetFilterChain chain,
                                int iovcnt,
                                NetPacketSent *sent_cb);
 
+/* pass the packet to the next filter */
+ssize_t qemu_netfilter_pass_to_next(NetClientState *sender,
+                                    unsigned flags,
+                                    const struct iovec *iov,
+                                    int iovcnt,
+                                    void *opaque);
+
 #endif /* QEMU_NET_FILTER_H */
diff --git a/net/filter.c b/net/filter.c
index 905817d..d6fa164 100644
--- a/net/filter.c
+++ b/net/filter.c
@@ -14,6 +14,7 @@
 #include "net/net.h"
 #include "net/vhost_net.h"
 #include "qom/object_interfaces.h"
+#include "qemu/iov.h"
 
 ssize_t qemu_netfilter_receive(NetFilterState *nf, NetFilterChain chain,
                                NetClientState *sender,
@@ -30,6 +31,63 @@ ssize_t qemu_netfilter_receive(NetFilterState *nf, NetFilterChain chain,
     return 0;
 }
 
+ssize_t qemu_netfilter_pass_to_next(NetClientState *sender,
+                                    unsigned flags,
+                                    const struct iovec *iov,
+                                    int iovcnt,
+                                    void *opaque)
+{
+    int ret = 0;
+    int chain;
+    NetFilterState *nf = opaque;
+    NetFilterState *next = QTAILQ_NEXT(nf, next);
+
+    if (!sender || !sender->peer) {
+        /* no receiver, or sender been deleted, no need to pass it further */
+        goto out;
+    }
+
+    if (nf->chain == NET_FILTER_CHAIN_ALL) {
+        if (sender == nf->netdev) {
+            /* This packet is sent by netdev itself */
+            chain = NET_FILTER_CHAIN_OUT;
+        } else {
+            chain = NET_FILTER_CHAIN_IN;
+        }
+    } else {
+        chain = nf->chain;
+    }
+
+    while (next) {
+        /*
+         * if qemu_netfilter_pass_to_next been called, means that
+         * the packet has been hold by filter and has already retured size
+         * to the sender, so sent_cb shouldn't be called later, just
+         * pass NULL to next.
+         */
+        ret = qemu_netfilter_receive(next, chain, sender, flags, iov,
+                                     iovcnt, NULL);
+        if (ret) {
+            return ret;
+        }
+        next = QTAILQ_NEXT(next, next);
+    }
+
+    /*
+     * We have gone through all filters, pass it to receiver.
+     * Do the valid check again incase sender or receiver been
+     * deleted while we go through filters.
+     */
+    if (sender && sender->peer) {
+        return qemu_net_queue_send_iov(sender->peer->incoming_queue,
+                                       sender, flags, iov, iovcnt, NULL);
+    }
+
+out:
+    /* no receiver, or sender been deleted */
+    return iov_size(iov, iovcnt);
+}
+
 static char *netfilter_get_netdev_id(Object *obj, Error **errp)
 {
     NetFilterState *nf = NETFILTER(obj);
-- 
1.9.1

[Qemu-devel] [PATCH v11 07/12] netfilter: print filter info associate with the netdev

[Yang Hongyang]

From: Yang Hongyang <burnef@gmail.com>

When execute "info network", print filter info also.
add a info_str member to NetFilterState, store specific filters
info.

Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
---
v11: generate info_str in filter layer
v10: add a info_str member to NetFilterState
v9: tiny cleanup according to Tomas's comment.
v7: initial patch
---
 include/net/filter.h |  1 +
 net/filter.c         | 20 ++++++++++++++++++++
 net/net.c            | 11 +++++++++++
 3 files changed, 32 insertions(+)

diff --git a/include/net/filter.h b/include/net/filter.h
index 90ab101..3fa80c9 100644
--- a/include/net/filter.h
+++ b/include/net/filter.h
@@ -54,6 +54,7 @@ struct NetFilterState {
     char *netdev_id;
     NetClientState *netdev;
     NetFilterChain chain;
+    char info_str[256];
     QTAILQ_ENTRY(NetFilterState) next;
 };
 
diff --git a/net/filter.c b/net/filter.c
index d6fa164..a8adc89 100644
--- a/net/filter.c
+++ b/net/filter.c
@@ -15,6 +15,7 @@
 #include "net/vhost_net.h"
 #include "qom/object_interfaces.h"
 #include "qemu/iov.h"
+#include "qapi/string-output-visitor.h"
 
 ssize_t qemu_netfilter_receive(NetFilterState *nf, NetFilterChain chain,
                                NetClientState *sender,
@@ -146,6 +147,9 @@ static void netfilter_complete(UserCreatable *uc, Error **errp)
     NetFilterClass *nfc = NETFILTER_GET_CLASS(uc);
     int queues;
     Error *local_err = NULL;
+    char *str, *info;
+    ObjectProperty *prop;
+    StringOutputVisitor *ov;
 
     if (!nf->netdev_id) {
         error_setg(errp, "Parameter 'netdev' is required");
@@ -179,6 +183,22 @@ static void netfilter_complete(UserCreatable *uc, Error **errp)
         }
     }
     QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);
+
+    /* generate info str */
+    QTAILQ_FOREACH(prop, &OBJECT(nf)->properties, node) {
+        if (!strcmp(prop->name, "type")) {
+            continue;
+        }
+        ov = string_output_visitor_new(false);
+        object_property_get(OBJECT(nf), string_output_get_visitor(ov),
+                            prop->name, errp);
+        str = string_output_get_string(ov);
+        string_output_visitor_cleanup(ov);
+        info = g_strdup_printf(",%s=%s", prop->name, str);
+        g_strlcat(nf->info_str, info, sizeof(nf->info_str));
+        g_free(str);
+        g_free(info);
+    }
 }
 
 static void netfilter_class_init(ObjectClass *oc, void *data)
diff --git a/net/net.c b/net/net.c
index 845daf2..e8f858d 100644
--- a/net/net.c
+++ b/net/net.c
@@ -1170,10 +1170,21 @@ void qmp_netdev_del(const char *id, Error **errp)
 
 void print_net_client(Monitor *mon, NetClientState *nc)
 {
+    NetFilterState *nf;
+
     monitor_printf(mon, "%s: index=%d,type=%s,%s\n", nc->name,
                    nc->queue_index,
                    NetClientOptionsKind_lookup[nc->info->type],
                    nc->info_str);
+    if (!QTAILQ_EMPTY(&nc->filters)) {
+        monitor_printf(mon, "filters:\n");
+    }
+    QTAILQ_FOREACH(nf, &nc->filters, next) {
+        monitor_printf(mon, "  - %s: type=%s%s\n",
+                       object_get_canonical_path_component(OBJECT(nf)),
+                       object_get_typename(OBJECT(nf)),
+                       nf->info_str);
+    }
 }
 
 RxFilterInfoList *qmp_query_rx_filter(bool has_name, const char *name,
-- 
1.9.1

[Qemu-devel] [PATCH v11 08/12] net/queue: export qemu_net_queue_append_iov

[Yang Hongyang]

This will be used by buffer filter implementation later to
queue packets.

Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
 include/net/queue.h |  7 +++++++
 net/queue.c         | 12 ++++++------
 2 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/include/net/queue.h b/include/net/queue.h
index b4a7183..5469fdb 100644
--- a/include/net/queue.h
+++ b/include/net/queue.h
@@ -47,6 +47,13 @@ typedef ssize_t (NetQueueDeliverFunc)(NetClientState *sender,
 
 NetQueue *qemu_new_net_queue(NetQueueDeliverFunc *deliver, void *opaque);
 
+void qemu_net_queue_append_iov(NetQueue *queue,
+                               NetClientState *sender,
+                               unsigned flags,
+                               const struct iovec *iov,
+                               int iovcnt,
+                               NetPacketSent *sent_cb);
+
 void qemu_del_net_queue(NetQueue *queue);
 
 ssize_t qemu_net_queue_send(NetQueue *queue,
diff --git a/net/queue.c b/net/queue.c
index 16dddf0..de8b9d3 100644
--- a/net/queue.c
+++ b/net/queue.c
@@ -112,12 +112,12 @@ static void qemu_net_queue_append(NetQueue *queue,
     QTAILQ_INSERT_TAIL(&queue->packets, packet, entry);
 }
 
-static void qemu_net_queue_append_iov(NetQueue *queue,
-                                      NetClientState *sender,
-                                      unsigned flags,
-                                      const struct iovec *iov,
-                                      int iovcnt,
-                                      NetPacketSent *sent_cb)
+void qemu_net_queue_append_iov(NetQueue *queue,
+                               NetClientState *sender,
+                               unsigned flags,
+                               const struct iovec *iov,
+                               int iovcnt,
+                               NetPacketSent *sent_cb)
 {
     NetPacket *packet;
     size_t max_len = 0;
-- 
1.9.1

[Qemu-devel] [PATCH v11 09/12] netfilter: add a netbuffer filter

[Yang Hongyang]

This filter is to buffer/release packets, this feature can be used
when using MicroCheckpointing, or other Remus like VM FT solutions, you
can also use it to simulate the network delay.
It has an interval option, if supplied, this filter will release
packets by interval.

Usage:
 -netdev tap,id=bn0
 -object filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000

NOTE:
 the scale of interval is microsecond.

Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
---
v11: add a fixme comment from Jason
v10: use NetQueue flush api to flush packets
     sent_cb can not be called when we already return size
v9: adjustment due to the qapi change
v7: use QTAILQ_FOREACH_SAFE() when flush packets
v6: move the interval check earlier and some comment adjust
v5: remove dummy sent_cb
    change interval type from int64 to uint32
    check interval!=0 when initialise
    rename FILTERBUFFERState to FilterBufferState
v4: remove bh
    pass the packet to next filter instead of receiver
v3: check packet's sender and sender->peer when flush it
---
 net/Makefile.objs   |   1 +
 net/filter-buffer.c | 170 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 qemu-options.hx     |  18 ++++++
 vl.c                |   7 ++-
 4 files changed, 195 insertions(+), 1 deletion(-)
 create mode 100644 net/filter-buffer.c

diff --git a/net/Makefile.objs b/net/Makefile.objs
index 914aec0..5fa2f97 100644
--- a/net/Makefile.objs
+++ b/net/Makefile.objs
@@ -14,3 +14,4 @@ common-obj-$(CONFIG_SLIRP) += slirp.o
 common-obj-$(CONFIG_VDE) += vde.o
 common-obj-$(CONFIG_NETMAP) += netmap.o
 common-obj-y += filter.o
+common-obj-y += filter-buffer.o
diff --git a/net/filter-buffer.c b/net/filter-buffer.c
new file mode 100644
index 0000000..ef94e91
--- /dev/null
+++ b/net/filter-buffer.c
@@ -0,0 +1,170 @@
+/*
+ * Copyright (c) 2015 FUJITSU LIMITED
+ * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.
+ */
+
+#include "net/filter.h"
+#include "net/queue.h"
+#include "qemu-common.h"
+#include "qemu/timer.h"
+#include "qemu/iov.h"
+#include "qapi/qmp/qerror.h"
+#include "qapi-visit.h"
+#include "qom/object.h"
+
+#define TYPE_FILTER_BUFFER "filter-buffer"
+
+#define FILTER_BUFFER(obj) \
+    OBJECT_CHECK(FilterBufferState, (obj), TYPE_FILTER_BUFFER)
+
+struct FilterBufferState {
+    NetFilterState parent_obj;
+
+    NetQueue *incoming_queue;
+    uint32_t interval;
+    QEMUTimer release_timer;
+};
+typedef struct FilterBufferState FilterBufferState;
+
+static void filter_buffer_flush(NetFilterState *nf)
+{
+    FilterBufferState *s = FILTER_BUFFER(nf);
+
+    if (!qemu_net_queue_flush(s->incoming_queue)) {
+        /* Unable to empty the queue, purge remaining packets */
+        qemu_net_queue_purge(s->incoming_queue, nf->netdev);
+    }
+}
+
+static void filter_buffer_release_timer(void *opaque)
+{
+    NetFilterState *nf = opaque;
+    FilterBufferState *s = FILTER_BUFFER(nf);
+    filter_buffer_flush(nf);
+    timer_mod(&s->release_timer,
+              qemu_clock_get_us(QEMU_CLOCK_VIRTUAL) + s->interval);
+}
+
+/* filter APIs */
+static ssize_t filter_buffer_receive_iov(NetFilterState *nf,
+                                         NetClientState *sender,
+                                         unsigned flags,
+                                         const struct iovec *iov,
+                                         int iovcnt,
+                                         NetPacketSent *sent_cb)
+{
+    FilterBufferState *s = FILTER_BUFFER(nf);
+
+    /*
+     * we return size when buffer a packet, the sender will take it as
+     * a already sent packet, so sent_cb should not be called later
+     * FIXME: even if guest can't receive packet for some reasons. Filter
+     * can still accept packet until its internal queue is full.
+     */
+    qemu_net_queue_append_iov(s->incoming_queue, sender, flags,
+                              iov, iovcnt, NULL);
+    return iov_size(iov, iovcnt);
+}
+
+static void filter_buffer_cleanup(NetFilterState *nf)
+{
+    FilterBufferState *s = FILTER_BUFFER(nf);
+
+    if (s->interval) {
+        timer_del(&s->release_timer);
+    }
+
+    /* flush packets */
+    if (s->incoming_queue) {
+        filter_buffer_flush(nf);
+        g_free(s->incoming_queue);
+    }
+}
+
+static void filter_buffer_setup(NetFilterState *nf, Error **errp)
+{
+    FilterBufferState *s = FILTER_BUFFER(nf);
+
+    /*
+     * this check should be dropped when there're VM FT solutions like MC
+     * or COLO use this filter to release packets on demand.
+     */
+    if (!s->interval) {
+        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "interval",
+                   "a non-zero interval");
+        return;
+    }
+
+    s->incoming_queue = qemu_new_net_queue(qemu_netfilter_pass_to_next, nf);
+    if (s->interval) {
+        timer_init_us(&s->release_timer, QEMU_CLOCK_VIRTUAL,
+                      filter_buffer_release_timer, nf);
+        timer_mod(&s->release_timer,
+                  qemu_clock_get_us(QEMU_CLOCK_VIRTUAL) + s->interval);
+    }
+}
+
+static void filter_buffer_class_init(ObjectClass *oc, void *data)
+{
+    NetFilterClass *nfc = NETFILTER_CLASS(oc);
+
+    nfc->setup = filter_buffer_setup;
+    nfc->cleanup = filter_buffer_cleanup;
+    nfc->receive_iov = filter_buffer_receive_iov;
+}
+
+static void filter_buffer_get_interval(Object *obj, Visitor *v, void *opaque,
+                                       const char *name, Error **errp)
+{
+    FilterBufferState *s = FILTER_BUFFER(obj);
+    uint32_t value = s->interval;
+
+    visit_type_uint32(v, &value, name, errp);
+}
+
+static void filter_buffer_set_interval(Object *obj, Visitor *v, void *opaque,
+                                       const char *name, Error **errp)
+{
+    FilterBufferState *s = FILTER_BUFFER(obj);
+    Error *local_err = NULL;
+    uint32_t value;
+
+    visit_type_uint32(v, &value, name, &local_err);
+    if (local_err) {
+        goto out;
+    }
+    if (!value) {
+        error_setg(&local_err, "Property '%s.%s' doesn't take value '%"
+                   PRIu32 "'", object_get_typename(obj), name, value);
+        goto out;
+    }
+    s->interval = value;
+
+out:
+    error_propagate(errp, local_err);
+}
+
+static void filter_buffer_init(Object *obj)
+{
+    object_property_add(obj, "interval", "int",
+                        filter_buffer_get_interval,
+                        filter_buffer_set_interval, NULL, NULL, NULL);
+}
+
+static const TypeInfo filter_buffer_info = {
+    .name = TYPE_FILTER_BUFFER,
+    .parent = TYPE_NETFILTER,
+    .class_init = filter_buffer_class_init,
+    .instance_init = filter_buffer_init,
+    .instance_size = sizeof(FilterBufferState),
+};
+
+static void register_types(void)
+{
+    type_register_static(&filter_buffer_info);
+}
+
+type_init(register_types);
diff --git a/qemu-options.hx b/qemu-options.hx
index 7e147b8..b09f97f 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -3642,6 +3642,24 @@ in PEM format, in filenames @var{ca-cert.pem}, @var{ca-crl.pem} (optional),
 @var{server-cert.pem} (only servers), @var{server-key.pem} (only servers),
 @var{client-cert.pem} (only clients), and @var{client-key.pem} (only clients).
 
+@item -object filter-buffer,id=@var{id},netdev=@var{netdevid}[,chain=@var{all|in|out}][,interval=@var{t}]
+
+Buffer network packets on netdev @var{netdevid}.
+If interval @var{t} provided, will release packets by interval.
+Interval scale: microsecond.
+
+If interval @var{t} not provided, you have to make sure the packets can be
+released, either by manually remove this filter or call the release buffer API,
+otherwise, the packets will be buffered forever. Use with caution.
+
+chain @var{all|in|out} is an option that can be applied to any netfilter, default is @option{all}.
+
+@option{all} means this filter will receive packets both sent to/from the netdev
+
+@option{in} means this filter will receive packets sent to the netdev
+
+@option{out} means this filter will receive packets sent from the netdev
+
 @end table
 
 ETEXI
diff --git a/vl.c b/vl.c
index ec589e2..3cf89d5 100644
--- a/vl.c
+++ b/vl.c
@@ -2794,7 +2794,12 @@ static bool object_create_initial(const char *type)
     if (g_str_equal(type, "rng-egd")) {
         return false;
     }
-    /* TODO: return false for concrete netfilters */
+
+    /* return false for concrete netfilters */
+    if (g_str_equal(type, "filter-buffer")) {
+        return false;
+    }
+
     return true;
 }
 
-- 
1.9.1

[Qemu-devel] [PATCH v11 10/12] tests: add test cases for netfilter object

[Yang Hongyang]

Using qtest qmp interface to implement following cases:
1) add/remove netfilter
2) add a netfilter then delete the netdev
3) add/remove more than one netfilters
4) add more than one netfilters and then delete the netdev

Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
---
v10: qmp command based on qom change
v9: qmp command change due to the qapi change
---
 tests/.gitignore       |   1 +
 tests/Makefile         |   2 +
 tests/test-netfilter.c | 200 +++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 203 insertions(+)
 create mode 100644 tests/test-netfilter.c

diff --git a/tests/.gitignore b/tests/.gitignore
index 2c5e2c3..81230c9 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -48,5 +48,6 @@ test-vmstate
 test-write-threshold
 test-x86-cpuid
 test-xbzrle
+test-netfilter
 *-test
 qapi-schema/*.test.*
diff --git a/tests/Makefile b/tests/Makefile
index 7c6025a..d369763 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -189,6 +189,7 @@ check-qtest-i386-y += tests/pc-cpu-test$(EXESUF)
 check-qtest-i386-y += tests/q35-test$(EXESUF)
 gcov-files-i386-y += hw/pci-host/q35.c
 check-qtest-i386-$(CONFIG_LINUX) += tests/vhost-user-test$(EXESUF)
+check-qtest-i386-y += tests/test-netfilter$(EXESUF)
 check-qtest-x86_64-y = $(check-qtest-i386-y)
 gcov-files-i386-y += i386-softmmu/hw/timer/mc146818rtc.c
 gcov-files-x86_64-y = $(subst i386-softmmu/,x86_64-softmmu/,$(gcov-files-i386-y))
@@ -428,6 +429,7 @@ tests/vhost-user-test$(EXESUF): tests/vhost-user-test.o qemu-char.o qemu-timer.o
 tests/qemu-iotests/socket_scm_helper$(EXESUF): tests/qemu-iotests/socket_scm_helper.o
 tests/test-qemu-opts$(EXESUF): tests/test-qemu-opts.o $(test-util-obj-y)
 tests/test-write-threshold$(EXESUF): tests/test-write-threshold.o $(test-block-obj-y)
+tests/test-netfilter$(EXESUF): tests/test-netfilter.o $(qtest-obj-y)
 
 ifeq ($(CONFIG_POSIX),y)
 LIBS += -lutil
diff --git a/tests/test-netfilter.c b/tests/test-netfilter.c
new file mode 100644
index 0000000..e114545
--- /dev/null
+++ b/tests/test-netfilter.c
@@ -0,0 +1,200 @@
+/*
+ * QTest testcase for netfilter
+ *
+ * Copyright (c) 2015 FUJITSU LIMITED
+ * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.
+ */
+
+#include <glib.h>
+#include "libqtest.h"
+
+/* add a netfilter to a netdev and then remove it */
+static void add_one_netfilter(void)
+{
+    QDict *response;
+
+    response = qmp("{'execute': 'object-add',"
+                   " 'arguments': {"
+                   "   'qom-type': 'filter-buffer',"
+                   "   'id': 'qtest-f0',"
+                   "   'props': {"
+                   "     'netdev': 'qtest-bn0',"
+                   "     'chain': 'in',"
+                   "     'interval': 1000"
+                   "}}}");
+
+    g_assert(response);
+    g_assert(!qdict_haskey(response, "error"));
+    QDECREF(response);
+
+    response = qmp("{'execute': 'object-del',"
+                   " 'arguments': {"
+                   "   'id': 'qtest-f0'"
+                   "}}");
+    g_assert(response);
+    g_assert(!qdict_haskey(response, "error"));
+    QDECREF(response);
+}
+
+/* add a netfilter to a netdev and then remove the netdev */
+static void remove_netdev_with_one_netfilter(void)
+{
+    QDict *response;
+
+    response = qmp("{'execute': 'object-add',"
+                   " 'arguments': {"
+                   "   'qom-type': 'filter-buffer',"
+                   "   'id': 'qtest-f0',"
+                   "   'props': {"
+                   "     'netdev': 'qtest-bn0',"
+                   "     'chain': 'in',"
+                   "     'interval': 1000"
+                   "}}}");
+
+    g_assert(response);
+    g_assert(!qdict_haskey(response, "error"));
+    QDECREF(response);
+
+    response = qmp("{'execute': 'netdev_del',"
+                   " 'arguments': {"
+                   "   'id': 'qtest-bn0'"
+                   "}}");
+    g_assert(response);
+    g_assert(!qdict_haskey(response, "error"));
+    QDECREF(response);
+
+    /* add back the netdev */
+    response = qmp("{'execute': 'netdev_add',"
+                   " 'arguments': {"
+                   "   'type': 'user',"
+                   "   'id': 'qtest-bn0'"
+                   "}}");
+    g_assert(response);
+    g_assert(!qdict_haskey(response, "error"));
+    QDECREF(response);
+}
+
+/* add multi(2) netfilters to a netdev and then remove them */
+static void add_multi_netfilter(void)
+{
+    QDict *response;
+
+    response = qmp("{'execute': 'object-add',"
+                   " 'arguments': {"
+                   "   'qom-type': 'filter-buffer',"
+                   "   'id': 'qtest-f0',"
+                   "   'props': {"
+                   "     'netdev': 'qtest-bn0',"
+                   "     'chain': 'in',"
+                   "     'interval': 1000"
+                   "}}}");
+
+    g_assert(response);
+    g_assert(!qdict_haskey(response, "error"));
+    QDECREF(response);
+
+    response = qmp("{'execute': 'object-add',"
+                   " 'arguments': {"
+                   "   'qom-type': 'filter-buffer',"
+                   "   'id': 'qtest-f1',"
+                   "   'props': {"
+                   "     'netdev': 'qtest-bn0',"
+                   "     'chain': 'in',"
+                   "     'interval': 1000"
+                   "}}}");
+
+    g_assert(response);
+    g_assert(!qdict_haskey(response, "error"));
+    QDECREF(response);
+
+    response = qmp("{'execute': 'object-del',"
+                   " 'arguments': {"
+                   "   'id': 'qtest-f0'"
+                   "}}");
+    g_assert(response);
+    g_assert(!qdict_haskey(response, "error"));
+    QDECREF(response);
+
+    response = qmp("{'execute': 'object-del',"
+                   " 'arguments': {"
+                   "   'id': 'qtest-f1'"
+                   "}}");
+    g_assert(response);
+    g_assert(!qdict_haskey(response, "error"));
+    QDECREF(response);
+}
+
+/* add multi(2) netfilters to a netdev and then remove the netdev */
+static void remove_netdev_with_multi_netfilter(void)
+{
+    QDict *response;
+
+    response = qmp("{'execute': 'object-add',"
+                   " 'arguments': {"
+                   "   'qom-type': 'filter-buffer',"
+                   "   'id': 'qtest-f0',"
+                   "   'props': {"
+                   "     'netdev': 'qtest-bn0',"
+                   "     'chain': 'in',"
+                   "     'interval': 1000"
+                   "}}}");
+
+    g_assert(response);
+    g_assert(!qdict_haskey(response, "error"));
+    QDECREF(response);
+
+    response = qmp("{'execute': 'object-add',"
+                   " 'arguments': {"
+                   "   'qom-type': 'filter-buffer',"
+                   "   'id': 'qtest-f1',"
+                   "   'props': {"
+                   "     'netdev': 'qtest-bn0',"
+                   "     'chain': 'in',"
+                   "     'interval': 1000"
+                   "}}}");
+
+    g_assert(response);
+    g_assert(!qdict_haskey(response, "error"));
+    QDECREF(response);
+
+    response = qmp("{'execute': 'netdev_del',"
+                   " 'arguments': {"
+                   "   'id': 'qtest-bn0'"
+                   "}}");
+    g_assert(response);
+    g_assert(!qdict_haskey(response, "error"));
+    QDECREF(response);
+
+    /* add back the netdev */
+    response = qmp("{'execute': 'netdev_add',"
+                   " 'arguments': {"
+                   "   'type': 'user',"
+                   "   'id': 'qtest-bn0'"
+                   "}}");
+    g_assert(response);
+    g_assert(!qdict_haskey(response, "error"));
+    QDECREF(response);
+}
+
+int main(int argc, char **argv)
+{
+    int ret;
+
+    g_test_init(&argc, &argv, NULL);
+    qtest_add_func("/netfilter/addremove_one", add_one_netfilter);
+    qtest_add_func("/netfilter/remove_netdev_one",
+                   remove_netdev_with_one_netfilter);
+    qtest_add_func("/netfilter/addremove_multi", add_multi_netfilter);
+    qtest_add_func("/netfilter/remove_netdev_multi",
+                   remove_netdev_with_multi_netfilter);
+
+    qtest_start("-netdev user,id=qtest-bn0 -device e1000,netdev=qtest-bn0");
+    ret = g_test_run();
+
+    qtest_end();
+
+    return ret;
+}
-- 
1.9.1

[Qemu-devel] [PATCH v11 11/12] netfilter/multiqueue: introduce netfilter name

[Yang Hongyang]

From: Yang Hongyang <burnef@gmail.com>

We will add multiple netfilter objects when using object_add id=xxx attach
a filter to a multiqueue enabled netdev. The multiple objects should have
different object id, but the filter name should be the same. Introduce
netfilter name to identify the netfilters.

Signed-off-by: Yang Hongyang <burnef@gmail.com>
---
v11: initial patch
---
 include/net/filter.h | 1 +
 net/filter.c         | 3 +++
 net/net.c            | 2 +-
 3 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/include/net/filter.h b/include/net/filter.h
index 3fa80c9..a3060c9 100644
--- a/include/net/filter.h
+++ b/include/net/filter.h
@@ -51,6 +51,7 @@ struct NetFilterState {
     Object parent;
 
     /* protected */
+    char *name;
     char *netdev_id;
     NetClientState *netdev;
     NetFilterChain chain;
diff --git a/net/filter.c b/net/filter.c
index a8adc89..aea619a 100644
--- a/net/filter.c
+++ b/net/filter.c
@@ -138,6 +138,8 @@ static void netfilter_finalize(Object *obj)
     if (nf->netdev && !QTAILQ_EMPTY(&nf->netdev->filters)) {
         QTAILQ_REMOVE(&nf->netdev->filters, nf, next);
     }
+
+    g_free(nf->name);
 }
 
 static void netfilter_complete(UserCreatable *uc, Error **errp)
@@ -173,6 +175,7 @@ static void netfilter_complete(UserCreatable *uc, Error **errp)
         return;
     }
 
+    nf->name = object_get_canonical_path_component(OBJECT(nf));
     nf->netdev = ncs[0];
 
     if (nfc->setup) {
diff --git a/net/net.c b/net/net.c
index e8f858d..33d4c99 100644
--- a/net/net.c
+++ b/net/net.c
@@ -1181,7 +1181,7 @@ void print_net_client(Monitor *mon, NetClientState *nc)
     }
     QTAILQ_FOREACH(nf, &nc->filters, next) {
         monitor_printf(mon, "  - %s: type=%s%s\n",
-                       object_get_canonical_path_component(OBJECT(nf)),
+                       nf->name,
                        object_get_typename(OBJECT(nf)),
                        nf->info_str);
     }
-- 
1.9.1

[Qemu-devel] [PATCH v11 12/12] netfilter: add multiqueue support

[Yang Hongyang]

From: Yang Hongyang <burnef@gmail.com>

add multiqueue support, if there's multiqueue, we add multi netfilter
objects, other netfilter objects is the child of the first added netfilter
object. So when we delete a netfilter, the other netfilter objects we
added will be automatically deleted.

Signed-off-by: Yang Hongyang <burnef@gmail.com>
---
v11: initial patch
---
 net/filter.c | 86 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 79 insertions(+), 7 deletions(-)

diff --git a/net/filter.c b/net/filter.c
index aea619a..cc27528 100644
--- a/net/filter.c
+++ b/net/filter.c
@@ -142,16 +142,25 @@ static void netfilter_finalize(Object *obj)
     g_free(nf->name);
 }
 
+static void proptb_free_val_func(gpointer data)
+{
+    g_free(data);
+}
+
 static void netfilter_complete(UserCreatable *uc, Error **errp)
 {
-    NetFilterState *nf = NETFILTER(uc);
+    NetFilterState *nf = NETFILTER(uc), *nfq = NULL;
     NetClientState *ncs[MAX_QUEUE_NUM];
-    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc);
-    int queues;
+    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc), *nfqc = NULL;
+    int queues, i;
     Error *local_err = NULL;
-    char *str, *info;
+    char *str, *info, *name;
     ObjectProperty *prop;
     StringOutputVisitor *ov;
+    Object *obj = NULL;
+    GHashTable *proptable = NULL;
+    GHashTableIter iter;
+    gpointer key, value;
 
     if (!nf->netdev_id) {
         error_setg(errp, "Parameter 'netdev' is required");
@@ -165,9 +174,6 @@ static void netfilter_complete(UserCreatable *uc, Error **errp)
         error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "netdev",
                    "a network backend id");
         return;
-    } else if (queues > 1) {
-        error_setg(errp, "Multi queue is not supported");
-        return;
     }
 
     if (get_vhost_net(ncs[0])) {
@@ -187,6 +193,17 @@ static void netfilter_complete(UserCreatable *uc, Error **errp)
     }
     QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);
 
+    if (queues > 1) {
+        /*
+         * Store the properties of the filter except "type" property.
+         * When there's multiqueue, we will create a new filter object
+         * of the same type and same properties. this hashtable is used
+         * to set newly created object properties.
+         */
+        proptable = g_hash_table_new_full(NULL, NULL, NULL,
+                                          proptb_free_val_func);
+    }
+
     /* generate info str */
     QTAILQ_FOREACH(prop, &OBJECT(nf)->properties, node) {
         if (!strcmp(prop->name, "type")) {
@@ -199,9 +216,64 @@ static void netfilter_complete(UserCreatable *uc, Error **errp)
         string_output_visitor_cleanup(ov);
         info = g_strdup_printf(",%s=%s", prop->name, str);
         g_strlcat(nf->info_str, info, sizeof(nf->info_str));
+        if (queues > 1) {
+            g_hash_table_insert(proptable, prop->name, g_strdup(str));
+        }
         g_free(str);
         g_free(info);
     }
+
+    for (i = 1; i < queues; i++) {
+        obj = object_new(object_get_typename(OBJECT(nf)));
+        /* set filter properties */
+        nfq = NETFILTER(obj);
+        nfq->name = g_strdup(nf->name);
+        nfq->netdev = ncs[i];
+        snprintf(nfq->info_str, sizeof(nfq->info_str), "%s", nf->info_str);
+
+        g_hash_table_iter_init(&iter, proptable);
+        while (g_hash_table_iter_next(&iter, &key, &value)) {
+            object_property_parse(obj, (char *)value, (char *)key, &local_err);
+            if (local_err) {
+                error_propagate(errp, local_err);
+                goto out;
+            }
+        }
+
+        /* add other queue's filter object as the first object's child */
+        name = g_strdup_printf("%s-queue-%d", nf->name, i);
+        object_property_add_child(OBJECT(nf), name, obj, &local_err);
+        g_free(name);
+        if (local_err) {
+            error_propagate(errp, local_err);
+            goto out;
+        }
+
+        /* setup filter */
+        nfqc = NETFILTER_GET_CLASS(obj);
+        if (nfqc->setup) {
+            nfqc->setup(nfq, &local_err);
+            if (local_err) {
+                error_propagate(errp, local_err);
+                goto out;
+            }
+        }
+        QTAILQ_INSERT_TAIL(&nfq->netdev->filters, nfq, next);
+        object_unref(obj);
+    }
+
+    if (proptable) {
+        g_hash_table_unref(proptable);
+    }
+    return;
+
+out:
+    if (proptable) {
+        g_hash_table_unref(proptable);
+    }
+    if (obj) {
+        object_unref(obj);
+    }
 }
 
 static void netfilter_class_init(ObjectClass *oc, void *data)
-- 
1.9.1

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1958 bytes --]

On 09/16/2015 06:15 AM, Yang Hongyang wrote:
> Add a netfilter object based on QOM.
> 
> A netfilter is attached to a netdev, captures all network packets
> that pass through the netdev. When we delete the netdev, we also
> delete the netfilter object attached to it, because if the netdev is
> removed, the filter which attached to it is useless.
> 
> QTAILQ_ENTRY next used by netdev, filter belongs to the specific netdev is
> in this queue.
> 
> Also init delayed object after net_init_clients, because netfilters need
> to be initialized after net clients initialized.
> 
> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
> ---

Interface review:

> +++ b/qapi-schema.json
> @@ -2551,6 +2551,24 @@
>      'opts': 'NetClientOptions' } }
>  
>  ##
> +# @NetFilterChain
> +#
> +# netfilter chain, a netfilter is attached to a netdev, captures the
> +# network packets of the netdev.

Grammar. Maybe:

This enum describes which packets are being tracked by a netfilter chain
attached as a filter to a netdev object.

> +#
> +# @all: the filter will receive packets both sent to/from the netdev, this
> +#       is the default chain.
> +#
> +# @in: the filter will receive packets sent to the netdev.
> +#
> +# @out: the filter will receive packets sent from the netdev.
> +#
> +# Since 2.5
> +##
> +{ 'enum': 'NetFilterChain',
> +  'data': [ 'all', 'in', 'out' ] }

I don't see any other QMP usage of this enum anywhere in the series. Are
you planning on supporting QMP?  If so, let's get that design discussion
started.  If not, why not?

In particular, you may want to base things on top of my work to make QMP
'netdev_add' a full-fledged introspectible command (still pending some
qapi commits landing upstream):

https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg02602.html

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Yang Hongyang]

Hi Eric,

On 09/17/2015 05:09 AM, Eric Blake wrote:
> On 09/16/2015 06:15 AM, Yang Hongyang wrote:
>> Add a netfilter object based on QOM.
>>
>> A netfilter is attached to a netdev, captures all network packets
>> that pass through the netdev. When we delete the netdev, we also
>> delete the netfilter object attached to it, because if the netdev is
>> removed, the filter which attached to it is useless.
>>
>> QTAILQ_ENTRY next used by netdev, filter belongs to the specific netdev is
>> in this queue.
>>
>> Also init delayed object after net_init_clients, because netfilters need
>> to be initialized after net clients initialized.
>>
>> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
>> ---
>
> Interface review:
>
>> +++ b/qapi-schema.json
>> @@ -2551,6 +2551,24 @@
>>       'opts': 'NetClientOptions' } }
>>
>>   ##
>> +# @NetFilterChain
>> +#
>> +# netfilter chain, a netfilter is attached to a netdev, captures the
>> +# network packets of the netdev.
>
> Grammar. Maybe:
>
> This enum describes which packets are being tracked by a netfilter chain
> attached as a filter to a netdev object.

Thanks!

>
>> +#
>> +# @all: the filter will receive packets both sent to/from the netdev, this
>> +#       is the default chain.
>> +#
>> +# @in: the filter will receive packets sent to the netdev.
>> +#
>> +# @out: the filter will receive packets sent from the netdev.
>> +#
>> +# Since 2.5
>> +##
>> +{ 'enum': 'NetFilterChain',
>> +  'data': [ 'all', 'in', 'out' ] }
>
> I don't see any other QMP usage of this enum anywhere in the series. Are
> you planning on supporting QMP?  If so, let's get that design discussion
> started.  If not, why not?

This series is based on QOM, so the QMP command for object_add
will use this enum, for example:
   1 { "execute": "qmp_capabilities" }
   2 { "execute": "object-add",
   3              "arguments": { "qom-type": "filter-buffer",
   4                             "id": "f0",
   5                             "props": { "netdev": "bn0",
   6                                        "chain": "in",
   7                                        "interval": 2000 } } }

for hmp:
object_add filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000

command options:
-object filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000

>
> In particular, you may want to base things on top of my work to make QMP
> 'netdev_add' a full-fledged introspectible command (still pending some
> qapi commits landing upstream):
>
> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg02602.html
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1235 bytes --]

On 09/16/2015 07:23 PM, Yang Hongyang wrote:

>>> +{ 'enum': 'NetFilterChain',
>>> +  'data': [ 'all', 'in', 'out' ] }
>>
>> I don't see any other QMP usage of this enum anywhere in the series. Are
>> you planning on supporting QMP?  If so, let's get that design discussion
>> started.  If not, why not?
> 
> This series is based on QOM, so the QMP command for object_add
> will use this enum, for example:
>   1 { "execute": "qmp_capabilities" }
>   2 { "execute": "object-add",
>   3              "arguments": { "qom-type": "filter-buffer",
>   4                             "id": "f0",
>   5                             "props": { "netdev": "bn0",
>   6                                        "chain": "in",
>   7                                        "interval": 2000 } } }
> 
> for hmp:
> object_add filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000
> 
> command options:
> -object filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000

Do these examples appear in the documentation anywhere?  If not, it is
worth considering (maybe under 'object-add' in qmp-commands.hx, for
example).

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Yang Hongyang]

On 09/18/2015 12:09 AM, Eric Blake wrote:
> On 09/16/2015 07:23 PM, Yang Hongyang wrote:
>
>>>> +{ 'enum': 'NetFilterChain',
>>>> +  'data': [ 'all', 'in', 'out' ] }
>>>
>>> I don't see any other QMP usage of this enum anywhere in the series. Are
>>> you planning on supporting QMP?  If so, let's get that design discussion
>>> started.  If not, why not?
>>
>> This series is based on QOM, so the QMP command for object_add
>> will use this enum, for example:
>>    1 { "execute": "qmp_capabilities" }
>>    2 { "execute": "object-add",
>>    3              "arguments": { "qom-type": "filter-buffer",
>>    4                             "id": "f0",
>>    5                             "props": { "netdev": "bn0",
>>    6                                        "chain": "in",
>>    7                                        "interval": 2000 } } }
>>
>> for hmp:
>> object_add filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000
>>
>> command options:
>> -object filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000
>
> Do these examples appear in the documentation anywhere?  If not, it is
> worth considering (maybe under 'object-add' in qmp-commands.hx, for
> example).

Yes, example is added in the 9th patch, under qemu-options.hx.
Thank you.

>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 04/12] net: merge qemu_deliver_packet and qemu_deliver_packet_iov

[Jason Wang]

On 09/16/2015 08:16 PM, Yang Hongyang wrote:
> qemu_deliver_packet_iov already have the compat delivery, we
> can drop qemu_deliver_packet.
>
> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
> ---
>  include/net/net.h |  5 -----
>  net/net.c         | 40 +++++++---------------------------------
>  net/queue.c       |  6 +++++-
>  3 files changed, 12 insertions(+), 39 deletions(-)
>
> diff --git a/include/net/net.h b/include/net/net.h
> index 36e5fab..7af3e15 100644
> --- a/include/net/net.h
> +++ b/include/net/net.h
> @@ -152,11 +152,6 @@ void qemu_check_nic_model(NICInfo *nd, const char *model);
>  int qemu_find_nic_model(NICInfo *nd, const char * const *models,
>                          const char *default_model);
>  
> -ssize_t qemu_deliver_packet(NetClientState *sender,
> -                            unsigned flags,
> -                            const uint8_t *data,
> -                            size_t size,
> -                            void *opaque);
>  ssize_t qemu_deliver_packet_iov(NetClientState *sender,
>                              unsigned flags,
>                              const struct iovec *iov,
> diff --git a/net/net.c b/net/net.c
> index ad37419..ca35b27 100644
> --- a/net/net.c
> +++ b/net/net.c
> @@ -597,36 +597,6 @@ static ssize_t filter_receive(NetClientState *nc, NetFilterChain chain,
>      return filter_receive_iov(nc, chain, sender, flags, &iov, 1, sent_cb);
>  }
>  
> -ssize_t qemu_deliver_packet(NetClientState *sender,
> -                            unsigned flags,
> -                            const uint8_t *data,
> -                            size_t size,
> -                            void *opaque)
> -{
> -    NetClientState *nc = opaque;
> -    ssize_t ret;
> -
> -    if (nc->link_down) {
> -        return size;
> -    }
> -
> -    if (nc->receive_disabled) {
> -        return 0;
> -    }
> -
> -    if (flags & QEMU_NET_PACKET_FLAG_RAW && nc->info->receive_raw) {
> -        ret = nc->info->receive_raw(nc, data, size);
> -    } else {
> -        ret = nc->info->receive(nc, data, size);
> -    }
> -
> -    if (ret == 0) {
> -        nc->receive_disabled = 1;
> -    }
> -
> -    return ret;
> -}
> -
>  void qemu_purge_queued_packets(NetClientState *nc)
>  {
>      if (!nc->peer) {
> @@ -717,14 +687,18 @@ ssize_t qemu_send_packet_raw(NetClientState *nc, const uint8_t *buf, int size)
>  }
>  
>  static ssize_t nc_sendv_compat(NetClientState *nc, const struct iovec *iov,
> -                               int iovcnt)
> +                               int iovcnt, unsigned flags)
>  {
>      uint8_t buffer[NET_BUFSIZE];
>      size_t offset;
>  
>      offset = iov_to_buf(iov, iovcnt, 0, buffer, sizeof(buffer));
>  
> -    return nc->info->receive(nc, buffer, offset);
> +    if (flags & QEMU_NET_PACKET_FLAG_RAW && nc->info->receive_raw) {
> +        return nc->info->receive_raw(nc, buffer, offset);
> +    } else {
> +        return nc->info->receive(nc, buffer, offset);
> +    }

Then for net clients that doesn't support receive_iov, an extra memcpy
is introduced. This seems unnecessary. And this patch looks independent,
so please drop this patch from the series (This can also help to reduce
the iteration). I think we may need this after all net clients support
receive_iov().

 
>  }
>  
>  ssize_t qemu_deliver_packet_iov(NetClientState *sender,
> @@ -747,7 +721,7 @@ ssize_t qemu_deliver_packet_iov(NetClientState *sender,
>      if (nc->info->receive_iov) {
>          ret = nc->info->receive_iov(nc, iov, iovcnt);
>      } else {
> -        ret = nc_sendv_compat(nc, iov, iovcnt);
> +        ret = nc_sendv_compat(nc, iov, iovcnt, flags);
>      }
>  
>      if (ret == 0) {
> diff --git a/net/queue.c b/net/queue.c
> index ebbe2bb..cf8db3a 100644
> --- a/net/queue.c
> +++ b/net/queue.c
> @@ -152,9 +152,13 @@ static ssize_t qemu_net_queue_deliver(NetQueue *queue,
>                                        size_t size)
>  {
>      ssize_t ret = -1;
> +    struct iovec iov = {
> +        .iov_base = (void *)data,
> +        .iov_len = size
> +    };
>  
>      queue->delivering = 1;
> -    ret = qemu_deliver_packet(sender, flags, data, size, queue->opaque);
> +    ret = qemu_deliver_packet_iov(sender, flags, &iov, 1, queue->opaque);
>      queue->delivering = 0;
>  
>      return ret;

Re: [Qemu-devel] [PATCH v11 12/12] netfilter: add multiqueue support

[Jason Wang]

On 09/16/2015 08:16 PM, Yang Hongyang wrote:
> From: Yang Hongyang <burnef@gmail.com>
>
> add multiqueue support, if there's multiqueue, we add multi netfilter
> objects, other netfilter objects is the child of the first added netfilter
> object. So when we delete a netfilter, the other netfilter objects we
> added will be automatically deleted.
>
> Signed-off-by: Yang Hongyang <burnef@gmail.com>
> ---
> v11: initial patch
> ---
>  net/filter.c | 86 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
>  1 file changed, 79 insertions(+), 7 deletions(-)
>
> diff --git a/net/filter.c b/net/filter.c
> index aea619a..cc27528 100644
> --- a/net/filter.c
> +++ b/net/filter.c
> @@ -142,16 +142,25 @@ static void netfilter_finalize(Object *obj)
>      g_free(nf->name);
>  }
>  
> +static void proptb_free_val_func(gpointer data)
> +{
> +    g_free(data);
> +}
> +
>  static void netfilter_complete(UserCreatable *uc, Error **errp)
>  {
> -    NetFilterState *nf = NETFILTER(uc);
> +    NetFilterState *nf = NETFILTER(uc), *nfq = NULL;
>      NetClientState *ncs[MAX_QUEUE_NUM];
> -    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc);
> -    int queues;
> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc), *nfqc = NULL;
> +    int queues, i;
>      Error *local_err = NULL;
> -    char *str, *info;
> +    char *str, *info, *name;
>      ObjectProperty *prop;
>      StringOutputVisitor *ov;
> +    Object *obj = NULL;
> +    GHashTable *proptable = NULL;
> +    GHashTableIter iter;
> +    gpointer key, value;
>  
>      if (!nf->netdev_id) {
>          error_setg(errp, "Parameter 'netdev' is required");
> @@ -165,9 +174,6 @@ static void netfilter_complete(UserCreatable *uc, Error **errp)
>          error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "netdev",
>                     "a network backend id");
>          return;
> -    } else if (queues > 1) {
> -        error_setg(errp, "Multi queue is not supported");
> -        return;
>      }
>  
>      if (get_vhost_net(ncs[0])) {
> @@ -187,6 +193,17 @@ static void netfilter_complete(UserCreatable *uc, Error **errp)
>      }
>      QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);
>  
> +    if (queues > 1) {
> +        /*
> +         * Store the properties of the filter except "type" property.
> +         * When there's multiqueue, we will create a new filter object
> +         * of the same type and same properties. this hashtable is used
> +         * to set newly created object properties.
> +         */
> +        proptable = g_hash_table_new_full(NULL, NULL, NULL,
> +                                          proptb_free_val_func);
> +    }

I'm thinking whether or not duplicate all the properties in each
netfilters is a good method. Maybe we can have a another ojbect with
array of pointers to NetFilter objects embedded? Another question is
whether or not we need to do this at this level. Maybe we can make the
necessary Netfilter multiqueue aware. E.g let buffer filter to have
multiqueue also? Then you may only need a single timer?

> +
>      /* generate info str */
>      QTAILQ_FOREACH(prop, &OBJECT(nf)->properties, node) {
>          if (!strcmp(prop->name, "type")) {
> @@ -199,9 +216,64 @@ static void netfilter_complete(UserCreatable *uc, Error **errp)
>          string_output_visitor_cleanup(ov);
>          info = g_strdup_printf(",%s=%s", prop->name, str);
>          g_strlcat(nf->info_str, info, sizeof(nf->info_str));
> +        if (queues > 1) {
> +            g_hash_table_insert(proptable, prop->name, g_strdup(str));
> +        }
>          g_free(str);
>          g_free(info);
>      }
> +
> +    for (i = 1; i < queues; i++) {
> +        obj = object_new(object_get_typename(OBJECT(nf)));
> +        /* set filter properties */
> +        nfq = NETFILTER(obj);
> +        nfq->name = g_strdup(nf->name);
> +        nfq->netdev = ncs[i];
> +        snprintf(nfq->info_str, sizeof(nfq->info_str), "%s", nf->info_str);
> +
> +        g_hash_table_iter_init(&iter, proptable);
> +        while (g_hash_table_iter_next(&iter, &key, &value)) {
> +            object_property_parse(obj, (char *)value, (char *)key, &local_err);
> +            if (local_err) {
> +                error_propagate(errp, local_err);
> +                goto out;
> +            }
> +        }
> +
> +        /* add other queue's filter object as the first object's child */
> +        name = g_strdup_printf("%s-queue-%d", nf->name, i);
> +        object_property_add_child(OBJECT(nf), name, obj, &local_err);
> +        g_free(name);
> +        if (local_err) {
> +            error_propagate(errp, local_err);
> +            goto out;
> +        }
> +
> +        /* setup filter */
> +        nfqc = NETFILTER_GET_CLASS(obj);
> +        if (nfqc->setup) {
> +            nfqc->setup(nfq, &local_err);
> +            if (local_err) {
> +                error_propagate(errp, local_err);
> +                goto out;
> +            }
> +        }
> +        QTAILQ_INSERT_TAIL(&nfq->netdev->filters, nfq, next);
> +        object_unref(obj);
> +    }
> +
> +    if (proptable) {
> +        g_hash_table_unref(proptable);
> +    }
> +    return;
> +
> +out:

We may leak objects here.

> +    if (proptable) {
> +        g_hash_table_unref(proptable);
> +    }
> +    if (obj) {
> +        object_unref(obj);
> +    }
>  }
>  
>  static void netfilter_class_init(ObjectClass *oc, void *data)

To reduce the review iterations, I suggest to drop the patch also.
Multiqueue support could be another series on top.

Thanks

Re: [Qemu-devel] [PATCH v11 00/12] Add a netfilter object and netbuffer filter

[Jason Wang]

On 09/16/2015 08:15 PM, Yang Hongyang wrote:
> This patch add an netfilter abstract object, captures all network packets
> on associated netdev. Also implement a concrete filter buffer based on
> this abstract object. the "buffer" netfilter could be used by VM FT solutions
> like MicroCheckpointing, to buffer/release packets. Or to simulate
> packet delay.
>
> You can also get the series from:
> https://github.com/macrosheep/qemu/tree/netfilter-v11
>
> Usage:
>  -netdev tap,id=bn0
>  -device e1000,netdev=bn0
>  -object filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000
>
> dynamically add/remove netfilters:
>  object_add filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000
>  object_del f0
>
> NOTE:
>  interval's scale is microsecond.
>  chain is optional, and is one of in|out|all, default is "all".
>        "in" means this filter will receive packets sent to the @netdev
>        "out" means this filter will receive packets sent from the @netdev
>        "all" means this filter will receive packets both sent to/from
>              the @netdev
>
> v11:
>  - address Jason&Daniel's comments
>  - add multiqueue support, the last 2 patches
>  - rebased to the latest master
>
> v10:
>  - Reimplemented using QOM (suggested by stefan)
>  - Do not export NetQueue internals (suggested by stefan)
>  - see individual patch for detail
>
> v9:
>  - squash command description and help to patch 1&3
>  - qapi changes according to Markus&Eric's comments
>  - see individual patch for detail
>
> v8:
>  - some minor fixes according to Thomas's comments
>  - rebased to the latest master branch
>
> v7:
>  - print filter info when execute 'info network'
>  - addressed Jason's comments
>
> v6:
>  - add multiqueue support, please see individual patch for detail
>
> v5:
>  - add a sent_cb param to filter receive_iov api
>  - squash the 4th patch into patch 3
>  - remove dummy sent_cb (buffer filter)
>  - addressed Jason's other comments, see individual patches for detail
>
> v4:
>  - get rid of struct Filter
>  - squash the 4th patch into patch 2
>  - fix qemu_netfilter_pass_to_next_iov
>  - get rid of bh (buffer filter)
>  - release the packet to next filter instead of to receiver (buffer filter)
>
> v3:
>  - add an api to pass the packet to next filter
>  - remove netfilters when delete netdev
>  - add qtest testcases for netfilter
>  - addressed comments from Jason
>
> v2:
>  - add a chain option to netfilter object
>  - move the hook place earlier, before net_queue_send
>  - drop the unused api in buffer filter
>  - squash buffer filter patches into one
>  - remove receive() api from netfilter, only receive_iov() is enough
>  - addressed comments from Jason&Thomas
>
> v1:
>  initial patch.
>
> Yang Hongyang (12):
>   qmp: delete qemu opts when delete an object
>   init/cleanup of netfilter object
>   netfilter: hook packets before net queue send
>   net: merge qemu_deliver_packet and qemu_deliver_packet_iov
>   net/queue: introduce NetQueueDeliverFunc
>   netfilter: add an API to pass the packet to next filter
>   netfilter: print filter info associate with the netdev
>   net/queue: export qemu_net_queue_append_iov
>   netfilter: add a netbuffer filter
>   tests: add test cases for netfilter object
>   netfilter/multiqueue: introduce netfilter name
>   netfilter: add multiqueue support
>
>  include/net/filter.h    |  76 ++++++++++++
>  include/net/net.h       |   6 +-
>  include/net/queue.h     |  20 +++-
>  include/qemu/typedefs.h |   1 +
>  net/Makefile.objs       |   2 +
>  net/filter-buffer.c     | 170 +++++++++++++++++++++++++++
>  net/filter.c            | 306 ++++++++++++++++++++++++++++++++++++++++++++++++
>  net/net.c               | 108 +++++++++++++----
>  net/queue.c             |  24 ++--
>  qapi-schema.json        |  18 +++
>  qemu-options.hx         |  18 +++
>  qmp.c                   |   4 +
>  tests/.gitignore        |   1 +
>  tests/Makefile          |   2 +
>  tests/test-netfilter.c  | 200 +++++++++++++++++++++++++++++++
>  vl.c                    |  18 ++-
>  16 files changed, 927 insertions(+), 47 deletions(-)
>  create mode 100644 include/net/filter.h
>  create mode 100644 net/filter-buffer.c
>  create mode 100644 net/filter.c
>  create mode 100644 tests/test-netfilter.c
>

Want to merge this. But have comments on patch 4 and multiqueue patches.
I suggest to drop them in next version so I can merge to reduce the
review iterations. Other optimizations and fixups could be done on top.

Thanks

Re: [Qemu-devel] [PATCH v11 04/12] net: merge qemu_deliver_packet and qemu_deliver_packet_iov

[Yang Hongyang]

Hi Jason,

   Thanks for the review,

On 09/22/2015 03:30 PM, Jason Wang wrote:
>
>
> On 09/16/2015 08:16 PM, Yang Hongyang wrote:
>> qemu_deliver_packet_iov already have the compat delivery, we
>> can drop qemu_deliver_packet.
>>
>> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
>> ---
>>   include/net/net.h |  5 -----
>>   net/net.c         | 40 +++++++---------------------------------
>>   net/queue.c       |  6 +++++-
>>   3 files changed, 12 insertions(+), 39 deletions(-)
>>
>> diff --git a/include/net/net.h b/include/net/net.h
>> index 36e5fab..7af3e15 100644
>> --- a/include/net/net.h
>> +++ b/include/net/net.h
>> @@ -152,11 +152,6 @@ void qemu_check_nic_model(NICInfo *nd, const char *model);
>>   int qemu_find_nic_model(NICInfo *nd, const char * const *models,
>>                           const char *default_model);
>>
>> -ssize_t qemu_deliver_packet(NetClientState *sender,
>> -                            unsigned flags,
>> -                            const uint8_t *data,
>> -                            size_t size,
>> -                            void *opaque);
>>   ssize_t qemu_deliver_packet_iov(NetClientState *sender,
>>                               unsigned flags,
>>                               const struct iovec *iov,
>> diff --git a/net/net.c b/net/net.c
>> index ad37419..ca35b27 100644
>> --- a/net/net.c
>> +++ b/net/net.c
>> @@ -597,36 +597,6 @@ static ssize_t filter_receive(NetClientState *nc, NetFilterChain chain,
>>       return filter_receive_iov(nc, chain, sender, flags, &iov, 1, sent_cb);
>>   }
>>
>> -ssize_t qemu_deliver_packet(NetClientState *sender,
>> -                            unsigned flags,
>> -                            const uint8_t *data,
>> -                            size_t size,
>> -                            void *opaque)
>> -{
>> -    NetClientState *nc = opaque;
>> -    ssize_t ret;
>> -
>> -    if (nc->link_down) {
>> -        return size;
>> -    }
>> -
>> -    if (nc->receive_disabled) {
>> -        return 0;
>> -    }
>> -
>> -    if (flags & QEMU_NET_PACKET_FLAG_RAW && nc->info->receive_raw) {
>> -        ret = nc->info->receive_raw(nc, data, size);
>> -    } else {
>> -        ret = nc->info->receive(nc, data, size);
>> -    }
>> -
>> -    if (ret == 0) {
>> -        nc->receive_disabled = 1;
>> -    }
>> -
>> -    return ret;
>> -}
>> -
>>   void qemu_purge_queued_packets(NetClientState *nc)
>>   {
>>       if (!nc->peer) {
>> @@ -717,14 +687,18 @@ ssize_t qemu_send_packet_raw(NetClientState *nc, const uint8_t *buf, int size)
>>   }
>>
>>   static ssize_t nc_sendv_compat(NetClientState *nc, const struct iovec *iov,
>> -                               int iovcnt)
>> +                               int iovcnt, unsigned flags)
>>   {
>>       uint8_t buffer[NET_BUFSIZE];
>>       size_t offset;
>>
>>       offset = iov_to_buf(iov, iovcnt, 0, buffer, sizeof(buffer));
>>
>> -    return nc->info->receive(nc, buffer, offset);
>> +    if (flags & QEMU_NET_PACKET_FLAG_RAW && nc->info->receive_raw) {
>> +        return nc->info->receive_raw(nc, buffer, offset);
>> +    } else {
>> +        return nc->info->receive(nc, buffer, offset);
>> +    }
>
> Then for net clients that doesn't support receive_iov, an extra memcpy
> is introduced. This seems unnecessary. And this patch looks independent,
> so please drop this patch from the series (This can also help to reduce
> the iteration). I think we may need this after all net clients support
> receive_iov().

This patch is required by the next patch which introduce a
+/* Returns:
+ *   >0 - success
+ *    0 - queue packet for future redelivery
+ *   <0 - failure (discard packet)
+ */
+typedef ssize_t (NetQueueDeliverFunc)(NetClientState *sender,
+                                      unsigned flags,
+                                      const struct iovec *iov,
+                                      int iovcnt,
+                                      void *opaque);

If we drop this patch, we will need to introduce 2 deliver func, which
seems not clean and simple.

>
>
>>   }
>>
>>   ssize_t qemu_deliver_packet_iov(NetClientState *sender,
>> @@ -747,7 +721,7 @@ ssize_t qemu_deliver_packet_iov(NetClientState *sender,
>>       if (nc->info->receive_iov) {
>>           ret = nc->info->receive_iov(nc, iov, iovcnt);
>>       } else {
>> -        ret = nc_sendv_compat(nc, iov, iovcnt);
>> +        ret = nc_sendv_compat(nc, iov, iovcnt, flags);
>>       }
>>
>>       if (ret == 0) {
>> diff --git a/net/queue.c b/net/queue.c
>> index ebbe2bb..cf8db3a 100644
>> --- a/net/queue.c
>> +++ b/net/queue.c
>> @@ -152,9 +152,13 @@ static ssize_t qemu_net_queue_deliver(NetQueue *queue,
>>                                         size_t size)
>>   {
>>       ssize_t ret = -1;
>> +    struct iovec iov = {
>> +        .iov_base = (void *)data,
>> +        .iov_len = size
>> +    };
>>
>>       queue->delivering = 1;
>> -    ret = qemu_deliver_packet(sender, flags, data, size, queue->opaque);
>> +    ret = qemu_deliver_packet_iov(sender, flags, &iov, 1, queue->opaque);
>>       queue->delivering = 0;
>>
>>       return ret;
>
>
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 12/12] netfilter: add multiqueue support

[Yang Hongyang]

On 09/22/2015 03:36 PM, Jason Wang wrote:
>
>
> On 09/16/2015 08:16 PM, Yang Hongyang wrote:
>> From: Yang Hongyang <burnef@gmail.com>
>>
>> add multiqueue support, if there's multiqueue, we add multi netfilter
>> objects, other netfilter objects is the child of the first added netfilter
>> object. So when we delete a netfilter, the other netfilter objects we
>> added will be automatically deleted.
>>
>> Signed-off-by: Yang Hongyang <burnef@gmail.com>
>> ---
>> v11: initial patch
>> ---
>>   net/filter.c | 86 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
>>   1 file changed, 79 insertions(+), 7 deletions(-)
>>
>> diff --git a/net/filter.c b/net/filter.c
>> index aea619a..cc27528 100644
>> --- a/net/filter.c
>> +++ b/net/filter.c
>> @@ -142,16 +142,25 @@ static void netfilter_finalize(Object *obj)
>>       g_free(nf->name);
>>   }
>>
>> +static void proptb_free_val_func(gpointer data)
>> +{
>> +    g_free(data);
>> +}
>> +
>>   static void netfilter_complete(UserCreatable *uc, Error **errp)
>>   {
>> -    NetFilterState *nf = NETFILTER(uc);
>> +    NetFilterState *nf = NETFILTER(uc), *nfq = NULL;
>>       NetClientState *ncs[MAX_QUEUE_NUM];
>> -    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc);
>> -    int queues;
>> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc), *nfqc = NULL;
>> +    int queues, i;
>>       Error *local_err = NULL;
>> -    char *str, *info;
>> +    char *str, *info, *name;
>>       ObjectProperty *prop;
>>       StringOutputVisitor *ov;
>> +    Object *obj = NULL;
>> +    GHashTable *proptable = NULL;
>> +    GHashTableIter iter;
>> +    gpointer key, value;
>>
>>       if (!nf->netdev_id) {
>>           error_setg(errp, "Parameter 'netdev' is required");
>> @@ -165,9 +174,6 @@ static void netfilter_complete(UserCreatable *uc, Error **errp)
>>           error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "netdev",
>>                      "a network backend id");
>>           return;
>> -    } else if (queues > 1) {
>> -        error_setg(errp, "Multi queue is not supported");
>> -        return;
>>       }
>>
>>       if (get_vhost_net(ncs[0])) {
>> @@ -187,6 +193,17 @@ static void netfilter_complete(UserCreatable *uc, Error **errp)
>>       }
>>       QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);
>>
>> +    if (queues > 1) {
>> +        /*
>> +         * Store the properties of the filter except "type" property.
>> +         * When there's multiqueue, we will create a new filter object
>> +         * of the same type and same properties. this hashtable is used
>> +         * to set newly created object properties.
>> +         */
>> +        proptable = g_hash_table_new_full(NULL, NULL, NULL,
>> +                                          proptb_free_val_func);
>> +    }
>
> I'm thinking whether or not duplicate all the properties in each
> netfilters is a good method. Maybe we can have a another ojbect with
> array of pointers to NetFilter objects embedded? Another question is
> whether or not we need to do this at this level. Maybe we can make the
> necessary Netfilter multiqueue aware. E.g let buffer filter to have
> multiqueue also? Then you may only need a single timer?

netfilter_complete() only called once when we add a netfilter. So in this
func, we can create the same filter object if there's multiqueue.

>
>> +
>>       /* generate info str */
>>       QTAILQ_FOREACH(prop, &OBJECT(nf)->properties, node) {
>>           if (!strcmp(prop->name, "type")) {
>> @@ -199,9 +216,64 @@ static void netfilter_complete(UserCreatable *uc, Error **errp)
>>           string_output_visitor_cleanup(ov);
>>           info = g_strdup_printf(",%s=%s", prop->name, str);
>>           g_strlcat(nf->info_str, info, sizeof(nf->info_str));
>> +        if (queues > 1) {
>> +            g_hash_table_insert(proptable, prop->name, g_strdup(str));
>> +        }
>>           g_free(str);
>>           g_free(info);
>>       }
>> +
>> +    for (i = 1; i < queues; i++) {
>> +        obj = object_new(object_get_typename(OBJECT(nf)));
>> +        /* set filter properties */
>> +        nfq = NETFILTER(obj);
>> +        nfq->name = g_strdup(nf->name);
>> +        nfq->netdev = ncs[i];
>> +        snprintf(nfq->info_str, sizeof(nfq->info_str), "%s", nf->info_str);
>> +
>> +        g_hash_table_iter_init(&iter, proptable);
>> +        while (g_hash_table_iter_next(&iter, &key, &value)) {
>> +            object_property_parse(obj, (char *)value, (char *)key, &local_err);
>> +            if (local_err) {
>> +                error_propagate(errp, local_err);
>> +                goto out;
>> +            }
>> +        }
>> +
>> +        /* add other queue's filter object as the first object's child */
>> +        name = g_strdup_printf("%s-queue-%d", nf->name, i);
>> +        object_property_add_child(OBJECT(nf), name, obj, &local_err);
>> +        g_free(name);
>> +        if (local_err) {
>> +            error_propagate(errp, local_err);
>> +            goto out;
>> +        }
>> +
>> +        /* setup filter */
>> +        nfqc = NETFILTER_GET_CLASS(obj);
>> +        if (nfqc->setup) {
>> +            nfqc->setup(nfq, &local_err);
>> +            if (local_err) {
>> +                error_propagate(errp, local_err);
>> +                goto out;
>> +            }
>> +        }
>> +        QTAILQ_INSERT_TAIL(&nfq->netdev->filters, nfq, next);
>> +        object_unref(obj);
             ^^^^^^^^^^^^

>> +    }
>> +
>> +    if (proptable) {
>> +        g_hash_table_unref(proptable);
>> +    }
>> +    return;
>> +
>> +out:
>
> We may leak objects here.

Seems not. see above object_unref(), if we come to
here, object create in previous iter will be unrefed.

>
>> +    if (proptable) {
>> +        g_hash_table_unref(proptable);
>> +    }
>> +    if (obj) {
>> +        object_unref(obj);
>> +    }
>>   }
>>
>>   static void netfilter_class_init(ObjectClass *oc, void *data)
>
> To reduce the review iterations, I suggest to drop the patch also.
> Multiqueue support could be another series on top.

I agree, thank you.

>
> Thanks
>
>
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 00/12] Add a netfilter object and netbuffer filter

[Yang Hongyang]

On 09/22/2015 03:39 PM, Jason Wang wrote:
>
[...]
>>
>
> Want to merge this. But have comments on patch 4 and multiqueue patches.
> I suggest to drop them in next version so I can merge to reduce the
> review iterations. Other optimizations and fixups could be done on top.

Thanks a lot! I agree that we could drop the last 2 patches
which introduce multiqueue support, but the patch 4 is required by the
following patch, so we can not simply drop it, please see my reply, thank you.

>
> Thanks
>
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 12/12] netfilter: add multiqueue support

[Yang Hongyang]

On 09/22/2015 03:36 PM, Jason Wang wrote:
>
>
> On 09/16/2015 08:16 PM, Yang Hongyang wrote:
>> From: Yang Hongyang <burnef@gmail.com>
>>
>> add multiqueue support, if there's multiqueue, we add multi netfilter
>> objects, other netfilter objects is the child of the first added netfilter
>> object. So when we delete a netfilter, the other netfilter objects we
>> added will be automatically deleted.
>>
>> Signed-off-by: Yang Hongyang <burnef@gmail.com>
>> ---
>> v11: initial patch
>> ---
>>   net/filter.c | 86 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
>>   1 file changed, 79 insertions(+), 7 deletions(-)
>>
>> diff --git a/net/filter.c b/net/filter.c
>> index aea619a..cc27528 100644
>> --- a/net/filter.c
>> +++ b/net/filter.c
>> @@ -142,16 +142,25 @@ static void netfilter_finalize(Object *obj)
>>       g_free(nf->name);
>>   }
>>
>> +static void proptb_free_val_func(gpointer data)
>> +{
>> +    g_free(data);
>> +}
>> +
>>   static void netfilter_complete(UserCreatable *uc, Error **errp)
>>   {
>> -    NetFilterState *nf = NETFILTER(uc);
>> +    NetFilterState *nf = NETFILTER(uc), *nfq = NULL;
>>       NetClientState *ncs[MAX_QUEUE_NUM];
>> -    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc);
>> -    int queues;
>> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc), *nfqc = NULL;
>> +    int queues, i;
>>       Error *local_err = NULL;
>> -    char *str, *info;
>> +    char *str, *info, *name;
>>       ObjectProperty *prop;
>>       StringOutputVisitor *ov;
>> +    Object *obj = NULL;
>> +    GHashTable *proptable = NULL;
>> +    GHashTableIter iter;
>> +    gpointer key, value;
>>
>>       if (!nf->netdev_id) {
>>           error_setg(errp, "Parameter 'netdev' is required");
>> @@ -165,9 +174,6 @@ static void netfilter_complete(UserCreatable *uc, Error **errp)
>>           error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "netdev",
>>                      "a network backend id");
>>           return;
>> -    } else if (queues > 1) {
>> -        error_setg(errp, "Multi queue is not supported");
>> -        return;
>>       }
>>
>>       if (get_vhost_net(ncs[0])) {
>> @@ -187,6 +193,17 @@ static void netfilter_complete(UserCreatable *uc, Error **errp)
>>       }
>>       QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);
>>
>> +    if (queues > 1) {
>> +        /*
>> +         * Store the properties of the filter except "type" property.
>> +         * When there's multiqueue, we will create a new filter object
>> +         * of the same type and same properties. this hashtable is used
>> +         * to set newly created object properties.
>> +         */
>> +        proptable = g_hash_table_new_full(NULL, NULL, NULL,
>> +                                          proptb_free_val_func);
>> +    }
>
> I'm thinking whether or not duplicate all the properties in each
> netfilters is a good method. Maybe we can have a another ojbect with
> array of pointers to NetFilter objects embedded? Another question is
> whether or not we need to do this at this level. Maybe we can make the
> necessary Netfilter multiqueue aware. E.g let buffer filter to have
> multiqueue also? Then you may only need a single timer?

Sorry I don't get it at first. I also thought about make the buffer filter to
have multiqueue, but there comes problem, how to distinguish which queue
we should go in when receive the packet, we need to add a mechanism to
distinguish which queue belongs to which net client's queue, that will
be more complex, while current multiqueue implementation of net clients
is multiple net clients with the same name, current solution is the simplest
solution I can think of...

>
>> +
>>       /* generate info str */
>>       QTAILQ_FOREACH(prop, &OBJECT(nf)->properties, node) {
>>           if (!strcmp(prop->name, "type")) {
>> @@ -199,9 +216,64 @@ static void netfilter_complete(UserCreatable *uc, Error **errp)
>>           string_output_visitor_cleanup(ov);
>>           info = g_strdup_printf(",%s=%s", prop->name, str);
>>           g_strlcat(nf->info_str, info, sizeof(nf->info_str));
>> +        if (queues > 1) {
>> +            g_hash_table_insert(proptable, prop->name, g_strdup(str));
>> +        }
>>           g_free(str);
>>           g_free(info);
>>       }
>> +
>> +    for (i = 1; i < queues; i++) {
>> +        obj = object_new(object_get_typename(OBJECT(nf)));
>> +        /* set filter properties */
>> +        nfq = NETFILTER(obj);
>> +        nfq->name = g_strdup(nf->name);
>> +        nfq->netdev = ncs[i];
>> +        snprintf(nfq->info_str, sizeof(nfq->info_str), "%s", nf->info_str);
>> +
>> +        g_hash_table_iter_init(&iter, proptable);
>> +        while (g_hash_table_iter_next(&iter, &key, &value)) {
>> +            object_property_parse(obj, (char *)value, (char *)key, &local_err);
>> +            if (local_err) {
>> +                error_propagate(errp, local_err);
>> +                goto out;
>> +            }
>> +        }
>> +
>> +        /* add other queue's filter object as the first object's child */
>> +        name = g_strdup_printf("%s-queue-%d", nf->name, i);
>> +        object_property_add_child(OBJECT(nf), name, obj, &local_err);
>> +        g_free(name);
>> +        if (local_err) {
>> +            error_propagate(errp, local_err);
>> +            goto out;
>> +        }
>> +
>> +        /* setup filter */
>> +        nfqc = NETFILTER_GET_CLASS(obj);
>> +        if (nfqc->setup) {
>> +            nfqc->setup(nfq, &local_err);
>> +            if (local_err) {
>> +                error_propagate(errp, local_err);
>> +                goto out;
>> +            }
>> +        }
>> +        QTAILQ_INSERT_TAIL(&nfq->netdev->filters, nfq, next);
>> +        object_unref(obj);
>> +    }
>> +
>> +    if (proptable) {
>> +        g_hash_table_unref(proptable);
>> +    }
>> +    return;
>> +
>> +out:
>
> We may leak objects here.
>
>> +    if (proptable) {
>> +        g_hash_table_unref(proptable);
>> +    }
>> +    if (obj) {
>> +        object_unref(obj);
>> +    }
>>   }
>>
>>   static void netfilter_class_init(ObjectClass *oc, void *data)
>
> To reduce the review iterations, I suggest to drop the patch also.
> Multiqueue support could be another series on top.
>
> Thanks
>
>
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 04/12] net: merge qemu_deliver_packet and qemu_deliver_packet_iov

[Jason Wang]

On 09/22/2015 03:44 PM, Yang Hongyang wrote:
> Hi Jason,
>
>   Thanks for the review,
>
> On 09/22/2015 03:30 PM, Jason Wang wrote:
>>
>>
>> On 09/16/2015 08:16 PM, Yang Hongyang wrote:
>>> qemu_deliver_packet_iov already have the compat delivery, we
>>> can drop qemu_deliver_packet.
>>>
>>> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
>>> ---
>>>   include/net/net.h |  5 -----
>>>   net/net.c         | 40 +++++++---------------------------------
>>>   net/queue.c       |  6 +++++-
>>>   3 files changed, 12 insertions(+), 39 deletions(-)
>>>
>>> diff --git a/include/net/net.h b/include/net/net.h
>>> index 36e5fab..7af3e15 100644
>>> --- a/include/net/net.h
>>> +++ b/include/net/net.h
>>> @@ -152,11 +152,6 @@ void qemu_check_nic_model(NICInfo *nd, const
>>> char *model);
>>>   int qemu_find_nic_model(NICInfo *nd, const char * const *models,
>>>                           const char *default_model);
>>>
>>> -ssize_t qemu_deliver_packet(NetClientState *sender,
>>> -                            unsigned flags,
>>> -                            const uint8_t *data,
>>> -                            size_t size,
>>> -                            void *opaque);
>>>   ssize_t qemu_deliver_packet_iov(NetClientState *sender,
>>>                               unsigned flags,
>>>                               const struct iovec *iov,
>>> diff --git a/net/net.c b/net/net.c
>>> index ad37419..ca35b27 100644
>>> --- a/net/net.c
>>> +++ b/net/net.c
>>> @@ -597,36 +597,6 @@ static ssize_t filter_receive(NetClientState
>>> *nc, NetFilterChain chain,
>>>       return filter_receive_iov(nc, chain, sender, flags, &iov, 1,
>>> sent_cb);
>>>   }
>>>
>>> -ssize_t qemu_deliver_packet(NetClientState *sender,
>>> -                            unsigned flags,
>>> -                            const uint8_t *data,
>>> -                            size_t size,
>>> -                            void *opaque)
>>> -{
>>> -    NetClientState *nc = opaque;
>>> -    ssize_t ret;
>>> -
>>> -    if (nc->link_down) {
>>> -        return size;
>>> -    }
>>> -
>>> -    if (nc->receive_disabled) {
>>> -        return 0;
>>> -    }
>>> -
>>> -    if (flags & QEMU_NET_PACKET_FLAG_RAW && nc->info->receive_raw) {
>>> -        ret = nc->info->receive_raw(nc, data, size);
>>> -    } else {
>>> -        ret = nc->info->receive(nc, data, size);
>>> -    }
>>> -
>>> -    if (ret == 0) {
>>> -        nc->receive_disabled = 1;
>>> -    }
>>> -
>>> -    return ret;
>>> -}
>>> -
>>>   void qemu_purge_queued_packets(NetClientState *nc)
>>>   {
>>>       if (!nc->peer) {
>>> @@ -717,14 +687,18 @@ ssize_t qemu_send_packet_raw(NetClientState
>>> *nc, const uint8_t *buf, int size)
>>>   }
>>>
>>>   static ssize_t nc_sendv_compat(NetClientState *nc, const struct
>>> iovec *iov,
>>> -                               int iovcnt)
>>> +                               int iovcnt, unsigned flags)
>>>   {
>>>       uint8_t buffer[NET_BUFSIZE];
>>>       size_t offset;
>>>
>>>       offset = iov_to_buf(iov, iovcnt, 0, buffer, sizeof(buffer));
>>>
>>> -    return nc->info->receive(nc, buffer, offset);
>>> +    if (flags & QEMU_NET_PACKET_FLAG_RAW && nc->info->receive_raw) {
>>> +        return nc->info->receive_raw(nc, buffer, offset);
>>> +    } else {
>>> +        return nc->info->receive(nc, buffer, offset);
>>> +    }
>>
>> Then for net clients that doesn't support receive_iov, an extra memcpy
>> is introduced. This seems unnecessary. And this patch looks independent,
>> so please drop this patch from the series (This can also help to reduce
>> the iteration). I think we may need this after all net clients support
>> receive_iov().
>
> This patch is required by the next patch which introduce a
> +/* Returns:
> + *   >0 - success
> + *    0 - queue packet for future redelivery
> + *   <0 - failure (discard packet)
> + */
> +typedef ssize_t (NetQueueDeliverFunc)(NetClientState *sender,
> +                                      unsigned flags,
> +                                      const struct iovec *iov,
> +                                      int iovcnt,
> +                                      void *opaque);
>
> If we drop this patch, we will need to introduce 2 deliver func, which
> seems not clean and simple.

Then you can probably skip the iov_to_buf() if iov_cnt is one in the
above code?

>
>>
>>
>>>   }
>>>
>>>   ssize_t qemu_deliver_packet_iov(NetClientState *sender,
>>> @@ -747,7 +721,7 @@ ssize_t qemu_deliver_packet_iov(NetClientState
>>> *sender,
>>>       if (nc->info->receive_iov) {
>>>           ret = nc->info->receive_iov(nc, iov, iovcnt);
>>>       } else {
>>> -        ret = nc_sendv_compat(nc, iov, iovcnt);
>>> +        ret = nc_sendv_compat(nc, iov, iovcnt, flags);
>>>       }
>>>
>>>       if (ret == 0) {
>>> diff --git a/net/queue.c b/net/queue.c
>>> index ebbe2bb..cf8db3a 100644
>>> --- a/net/queue.c
>>> +++ b/net/queue.c
>>> @@ -152,9 +152,13 @@ static ssize_t qemu_net_queue_deliver(NetQueue
>>> *queue,
>>>                                         size_t size)
>>>   {
>>>       ssize_t ret = -1;
>>> +    struct iovec iov = {
>>> +        .iov_base = (void *)data,
>>> +        .iov_len = size
>>> +    };
>>>
>>>       queue->delivering = 1;
>>> -    ret = qemu_deliver_packet(sender, flags, data, size,
>>> queue->opaque);
>>> +    ret = qemu_deliver_packet_iov(sender, flags, &iov, 1,
>>> queue->opaque);
>>>       queue->delivering = 0;
>>>
>>>       return ret;
>>
>>
>> .
>>
>

Re: [Qemu-devel] [PATCH v11 04/12] net: merge qemu_deliver_packet and qemu_deliver_packet_iov

[Yang Hongyang]

On 09/22/2015 04:14 PM, Jason Wang wrote:
>
>
> On 09/22/2015 03:44 PM, Yang Hongyang wrote:
>> Hi Jason,
>>
>>    Thanks for the review,
>>
>> On 09/22/2015 03:30 PM, Jason Wang wrote:
>>>
>>>
>>> On 09/16/2015 08:16 PM, Yang Hongyang wrote:
>>>> qemu_deliver_packet_iov already have the compat delivery, we
>>>> can drop qemu_deliver_packet.
>>>>
>>>> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
>>>> ---
>>>>    include/net/net.h |  5 -----
>>>>    net/net.c         | 40 +++++++---------------------------------
>>>>    net/queue.c       |  6 +++++-
>>>>    3 files changed, 12 insertions(+), 39 deletions(-)
>>>>
>>>> diff --git a/include/net/net.h b/include/net/net.h
>>>> index 36e5fab..7af3e15 100644
>>>> --- a/include/net/net.h
>>>> +++ b/include/net/net.h
>>>> @@ -152,11 +152,6 @@ void qemu_check_nic_model(NICInfo *nd, const
>>>> char *model);
>>>>    int qemu_find_nic_model(NICInfo *nd, const char * const *models,
>>>>                            const char *default_model);
>>>>
>>>> -ssize_t qemu_deliver_packet(NetClientState *sender,
>>>> -                            unsigned flags,
>>>> -                            const uint8_t *data,
>>>> -                            size_t size,
>>>> -                            void *opaque);
>>>>    ssize_t qemu_deliver_packet_iov(NetClientState *sender,
>>>>                                unsigned flags,
>>>>                                const struct iovec *iov,
>>>> diff --git a/net/net.c b/net/net.c
>>>> index ad37419..ca35b27 100644
>>>> --- a/net/net.c
>>>> +++ b/net/net.c
>>>> @@ -597,36 +597,6 @@ static ssize_t filter_receive(NetClientState
>>>> *nc, NetFilterChain chain,
>>>>        return filter_receive_iov(nc, chain, sender, flags, &iov, 1,
>>>> sent_cb);
>>>>    }
>>>>
>>>> -ssize_t qemu_deliver_packet(NetClientState *sender,
>>>> -                            unsigned flags,
>>>> -                            const uint8_t *data,
>>>> -                            size_t size,
>>>> -                            void *opaque)
>>>> -{
>>>> -    NetClientState *nc = opaque;
>>>> -    ssize_t ret;
>>>> -
>>>> -    if (nc->link_down) {
>>>> -        return size;
>>>> -    }
>>>> -
>>>> -    if (nc->receive_disabled) {
>>>> -        return 0;
>>>> -    }
>>>> -
>>>> -    if (flags & QEMU_NET_PACKET_FLAG_RAW && nc->info->receive_raw) {
>>>> -        ret = nc->info->receive_raw(nc, data, size);
>>>> -    } else {
>>>> -        ret = nc->info->receive(nc, data, size);
>>>> -    }
>>>> -
>>>> -    if (ret == 0) {
>>>> -        nc->receive_disabled = 1;
>>>> -    }
>>>> -
>>>> -    return ret;
>>>> -}
>>>> -
>>>>    void qemu_purge_queued_packets(NetClientState *nc)
>>>>    {
>>>>        if (!nc->peer) {
>>>> @@ -717,14 +687,18 @@ ssize_t qemu_send_packet_raw(NetClientState
>>>> *nc, const uint8_t *buf, int size)
>>>>    }
>>>>
>>>>    static ssize_t nc_sendv_compat(NetClientState *nc, const struct
>>>> iovec *iov,
>>>> -                               int iovcnt)
>>>> +                               int iovcnt, unsigned flags)
>>>>    {
>>>>        uint8_t buffer[NET_BUFSIZE];
>>>>        size_t offset;
>>>>
>>>>        offset = iov_to_buf(iov, iovcnt, 0, buffer, sizeof(buffer));
>>>>
>>>> -    return nc->info->receive(nc, buffer, offset);
>>>> +    if (flags & QEMU_NET_PACKET_FLAG_RAW && nc->info->receive_raw) {
>>>> +        return nc->info->receive_raw(nc, buffer, offset);
>>>> +    } else {
>>>> +        return nc->info->receive(nc, buffer, offset);
>>>> +    }
>>>
>>> Then for net clients that doesn't support receive_iov, an extra memcpy
>>> is introduced. This seems unnecessary. And this patch looks independent,
>>> so please drop this patch from the series (This can also help to reduce
>>> the iteration). I think we may need this after all net clients support
>>> receive_iov().
>>
>> This patch is required by the next patch which introduce a
>> +/* Returns:
>> + *   >0 - success
>> + *    0 - queue packet for future redelivery
>> + *   <0 - failure (discard packet)
>> + */
>> +typedef ssize_t (NetQueueDeliverFunc)(NetClientState *sender,
>> +                                      unsigned flags,
>> +                                      const struct iovec *iov,
>> +                                      int iovcnt,
>> +                                      void *opaque);
>>
>> If we drop this patch, we will need to introduce 2 deliver func, which
>> seems not clean and simple.
>
> Then you can probably skip the iov_to_buf() if iov_cnt is one in the
> above code?

Seems like a good idea to avoid the extra memcpy, thank you! BTW,
can I do this on top of the series if there's no other comments?
so that the first 10 patch can be merged as is. if not, I will send another
version v12 to fix this.

>
>>
>>>
>>>
>>>>    }
>>>>
>>>>    ssize_t qemu_deliver_packet_iov(NetClientState *sender,
>>>> @@ -747,7 +721,7 @@ ssize_t qemu_deliver_packet_iov(NetClientState
>>>> *sender,
>>>>        if (nc->info->receive_iov) {
>>>>            ret = nc->info->receive_iov(nc, iov, iovcnt);
>>>>        } else {
>>>> -        ret = nc_sendv_compat(nc, iov, iovcnt);
>>>> +        ret = nc_sendv_compat(nc, iov, iovcnt, flags);
>>>>        }
>>>>
>>>>        if (ret == 0) {
>>>> diff --git a/net/queue.c b/net/queue.c
>>>> index ebbe2bb..cf8db3a 100644
>>>> --- a/net/queue.c
>>>> +++ b/net/queue.c
>>>> @@ -152,9 +152,13 @@ static ssize_t qemu_net_queue_deliver(NetQueue
>>>> *queue,
>>>>                                          size_t size)
>>>>    {
>>>>        ssize_t ret = -1;
>>>> +    struct iovec iov = {
>>>> +        .iov_base = (void *)data,
>>>> +        .iov_len = size
>>>> +    };
>>>>
>>>>        queue->delivering = 1;
>>>> -    ret = qemu_deliver_packet(sender, flags, data, size,
>>>> queue->opaque);
>>>> +    ret = qemu_deliver_packet_iov(sender, flags, &iov, 1,
>>>> queue->opaque);
>>>>        queue->delivering = 0;
>>>>
>>>>        return ret;
>>>
>>>
>>> .
>>>
>>
>
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 12/12] netfilter: add multiqueue support

[Jason Wang]

On 09/22/2015 03:49 PM, Yang Hongyang wrote:
> On 09/22/2015 03:36 PM, Jason Wang wrote:
>>
>>
>> On 09/16/2015 08:16 PM, Yang Hongyang wrote:
>>> From: Yang Hongyang <burnef@gmail.com>
>>>
>>> add multiqueue support, if there's multiqueue, we add multi netfilter
>>> objects, other netfilter objects is the child of the first added
>>> netfilter
>>> object. So when we delete a netfilter, the other netfilter objects we
>>> added will be automatically deleted.
>>>
>>> Signed-off-by: Yang Hongyang <burnef@gmail.com>
>>> ---
>>> v11: initial patch
>>> ---
>>>   net/filter.c | 86
>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
>>>   1 file changed, 79 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/net/filter.c b/net/filter.c
>>> index aea619a..cc27528 100644
>>> --- a/net/filter.c
>>> +++ b/net/filter.c
>>> @@ -142,16 +142,25 @@ static void netfilter_finalize(Object *obj)
>>>       g_free(nf->name);
>>>   }
>>>
>>> +static void proptb_free_val_func(gpointer data)
>>> +{
>>> +    g_free(data);
>>> +}
>>> +
>>>   static void netfilter_complete(UserCreatable *uc, Error **errp)
>>>   {
>>> -    NetFilterState *nf = NETFILTER(uc);
>>> +    NetFilterState *nf = NETFILTER(uc), *nfq = NULL;
>>>       NetClientState *ncs[MAX_QUEUE_NUM];
>>> -    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc);
>>> -    int queues;
>>> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc), *nfqc = NULL;
>>> +    int queues, i;
>>>       Error *local_err = NULL;
>>> -    char *str, *info;
>>> +    char *str, *info, *name;
>>>       ObjectProperty *prop;
>>>       StringOutputVisitor *ov;
>>> +    Object *obj = NULL;
>>> +    GHashTable *proptable = NULL;
>>> +    GHashTableIter iter;
>>> +    gpointer key, value;
>>>
>>>       if (!nf->netdev_id) {
>>>           error_setg(errp, "Parameter 'netdev' is required");
>>> @@ -165,9 +174,6 @@ static void netfilter_complete(UserCreatable
>>> *uc, Error **errp)
>>>           error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "netdev",
>>>                      "a network backend id");
>>>           return;
>>> -    } else if (queues > 1) {
>>> -        error_setg(errp, "Multi queue is not supported");
>>> -        return;
>>>       }
>>>
>>>       if (get_vhost_net(ncs[0])) {
>>> @@ -187,6 +193,17 @@ static void netfilter_complete(UserCreatable
>>> *uc, Error **errp)
>>>       }
>>>       QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);
>>>
>>> +    if (queues > 1) {
>>> +        /*
>>> +         * Store the properties of the filter except "type" property.
>>> +         * When there's multiqueue, we will create a new filter object
>>> +         * of the same type and same properties. this hashtable is
>>> used
>>> +         * to set newly created object properties.
>>> +         */
>>> +        proptable = g_hash_table_new_full(NULL, NULL, NULL,
>>> +                                          proptb_free_val_func);
>>> +    }
>>
>> I'm thinking whether or not duplicate all the properties in each
>> netfilters is a good method. Maybe we can have a another ojbect with
>> array of pointers to NetFilter objects embedded? Another question is
>> whether or not we need to do this at this level. Maybe we can make the
>> necessary Netfilter multiqueue aware. E.g let buffer filter to have
>> multiqueue also? Then you may only need a single timer?
>
> netfilter_complete() only called once when we add a netfilter. So in this
> func, we can create the same filter object if there's multiqueue.
>
>>
>>> +
>>>       /* generate info str */
>>>       QTAILQ_FOREACH(prop, &OBJECT(nf)->properties, node) {
>>>           if (!strcmp(prop->name, "type")) {
>>> @@ -199,9 +216,64 @@ static void netfilter_complete(UserCreatable
>>> *uc, Error **errp)
>>>           string_output_visitor_cleanup(ov);
>>>           info = g_strdup_printf(",%s=%s", prop->name, str);
>>>           g_strlcat(nf->info_str, info, sizeof(nf->info_str));
>>> +        if (queues > 1) {
>>> +            g_hash_table_insert(proptable, prop->name, g_strdup(str));
>>> +        }
>>>           g_free(str);
>>>           g_free(info);
>>>       }
>>> +
>>> +    for (i = 1; i < queues; i++) {
>>> +        obj = object_new(object_get_typename(OBJECT(nf)));
>>> +        /* set filter properties */
>>> +        nfq = NETFILTER(obj);
>>> +        nfq->name = g_strdup(nf->name);
>>> +        nfq->netdev = ncs[i];
>>> +        snprintf(nfq->info_str, sizeof(nfq->info_str), "%s",
>>> nf->info_str);
>>> +
>>> +        g_hash_table_iter_init(&iter, proptable);
>>> +        while (g_hash_table_iter_next(&iter, &key, &value)) {
>>> +            object_property_parse(obj, (char *)value, (char *)key,
>>> &local_err);
>>> +            if (local_err) {
>>> +                error_propagate(errp, local_err);
>>> +                goto out;
>>> +            }
>>> +        }
>>> +
>>> +        /* add other queue's filter object as the first object's
>>> child */
>>> +        name = g_strdup_printf("%s-queue-%d", nf->name, i);
>>> +        object_property_add_child(OBJECT(nf), name, obj, &local_err);
>>> +        g_free(name);
>>> +        if (local_err) {
>>> +            error_propagate(errp, local_err);
>>> +            goto out;
>>> +        }
>>> +
>>> +        /* setup filter */
>>> +        nfqc = NETFILTER_GET_CLASS(obj);
>>> +        if (nfqc->setup) {
>>> +            nfqc->setup(nfq, &local_err);
>>> +            if (local_err) {
>>> +                error_propagate(errp, local_err);
>>> +                goto out;
>>> +            }
>>> +        }
>>> +        QTAILQ_INSERT_TAIL(&nfq->netdev->filters, nfq, next);
>>> +        object_unref(obj);
>             ^^^^^^^^^^^^
>
>>> +    }
>>> +
>>> +    if (proptable) {
>>> +        g_hash_table_unref(proptable);
>>> +    }
>>> +    return;
>>> +
>>> +out:
>>
>> We may leak objects here.
>
> Seems not. see above object_unref(), if we come to
> here, object create in previous iter will be unrefed.

Maybe I miss something. But there's still refcnt for objects created in
previous iters? (I mean object_property_add_child() will add another
refcnt?)

>
>>
>>> +    if (proptable) {
>>> +        g_hash_table_unref(proptable);
>>> +    }
>>> +    if (obj) {
>>> +        object_unref(obj);
>>> +    }
>>>   }
>>>
>>>   static void netfilter_class_init(ObjectClass *oc, void *data)
>>
>> To reduce the review iterations, I suggest to drop the patch also.
>> Multiqueue support could be another series on top.
>
> I agree, thank you.
>
>>
>> Thanks
>>
>>
>> .
>>
>

Re: [Qemu-devel] [PATCH v11 12/12] netfilter: add multiqueue support

[Jason Wang]

On 09/22/2015 04:07 PM, Yang Hongyang wrote:
> On 09/22/2015 03:36 PM, Jason Wang wrote:
>>
>>
>> On 09/16/2015 08:16 PM, Yang Hongyang wrote:
>>> From: Yang Hongyang <burnef@gmail.com>
>>>
>>> add multiqueue support, if there's multiqueue, we add multi netfilter
>>> objects, other netfilter objects is the child of the first added
>>> netfilter
>>> object. So when we delete a netfilter, the other netfilter objects we
>>> added will be automatically deleted.
>>>
>>> Signed-off-by: Yang Hongyang <burnef@gmail.com>
>>> ---
>>> v11: initial patch
>>> ---
>>>   net/filter.c | 86
>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
>>>   1 file changed, 79 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/net/filter.c b/net/filter.c
>>> index aea619a..cc27528 100644
>>> --- a/net/filter.c
>>> +++ b/net/filter.c
>>> @@ -142,16 +142,25 @@ static void netfilter_finalize(Object *obj)
>>>       g_free(nf->name);
>>>   }
>>>
>>> +static void proptb_free_val_func(gpointer data)
>>> +{
>>> +    g_free(data);
>>> +}
>>> +
>>>   static void netfilter_complete(UserCreatable *uc, Error **errp)
>>>   {
>>> -    NetFilterState *nf = NETFILTER(uc);
>>> +    NetFilterState *nf = NETFILTER(uc), *nfq = NULL;
>>>       NetClientState *ncs[MAX_QUEUE_NUM];
>>> -    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc);
>>> -    int queues;
>>> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc), *nfqc = NULL;
>>> +    int queues, i;
>>>       Error *local_err = NULL;
>>> -    char *str, *info;
>>> +    char *str, *info, *name;
>>>       ObjectProperty *prop;
>>>       StringOutputVisitor *ov;
>>> +    Object *obj = NULL;
>>> +    GHashTable *proptable = NULL;
>>> +    GHashTableIter iter;
>>> +    gpointer key, value;
>>>
>>>       if (!nf->netdev_id) {
>>>           error_setg(errp, "Parameter 'netdev' is required");
>>> @@ -165,9 +174,6 @@ static void netfilter_complete(UserCreatable
>>> *uc, Error **errp)
>>>           error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "netdev",
>>>                      "a network backend id");
>>>           return;
>>> -    } else if (queues > 1) {
>>> -        error_setg(errp, "Multi queue is not supported");
>>> -        return;
>>>       }
>>>
>>>       if (get_vhost_net(ncs[0])) {
>>> @@ -187,6 +193,17 @@ static void netfilter_complete(UserCreatable
>>> *uc, Error **errp)
>>>       }
>>>       QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);
>>>
>>> +    if (queues > 1) {
>>> +        /*
>>> +         * Store the properties of the filter except "type" property.
>>> +         * When there's multiqueue, we will create a new filter object
>>> +         * of the same type and same properties. this hashtable is
>>> used
>>> +         * to set newly created object properties.
>>> +         */
>>> +        proptable = g_hash_table_new_full(NULL, NULL, NULL,
>>> +                                          proptb_free_val_func);
>>> +    }
>>
>> I'm thinking whether or not duplicate all the properties in each
>> netfilters is a good method. Maybe we can have a another ojbect with
>> array of pointers to NetFilter objects embedded? Another question is
>> whether or not we need to do this at this level. Maybe we can make the
>> necessary Netfilter multiqueue aware. E.g let buffer filter to have
>> multiqueue also? Then you may only need a single timer?
>
> Sorry I don't get it at first. I also thought about make the buffer
> filter to
> have multiqueue, but there comes problem, how to distinguish which queue
> we should go in when receive the packet, we need to add a mechanism to
> distinguish which queue belongs to which net client's queue, that will
> be more complex, while current multiqueue implementation of net clients
> is multiple net clients with the same name, current solution is the
> simplest
> solution I can think of...

I'm not sure I get this. But there's a queue_index filed in each
NetClientState which may help in this case.

Re: [Qemu-devel] [PATCH v11 12/12] netfilter: add multiqueue support

[Yang Hongyang]

On 09/22/2015 04:31 PM, Jason Wang wrote:
[...]
>>>> +        /* setup filter */
>>>> +        nfqc = NETFILTER_GET_CLASS(obj);
>>>> +        if (nfqc->setup) {
>>>> +            nfqc->setup(nfq, &local_err);
>>>> +            if (local_err) {
>>>> +                error_propagate(errp, local_err);
>>>> +                goto out;
>>>> +            }
>>>> +        }
>>>> +        QTAILQ_INSERT_TAIL(&nfq->netdev->filters, nfq, next);
>>>> +        object_unref(obj);
>>              ^^^^^^^^^^^^
>>
>>>> +    }
>>>> +
>>>> +    if (proptable) {
>>>> +        g_hash_table_unref(proptable);
>>>> +    }
>>>> +    return;
>>>> +
>>>> +out:
>>>
>>> We may leak objects here.
>>
>> Seems not. see above object_unref(), if we come to
>> here, object create in previous iter will be unrefed.
>
> Maybe I miss something. But there's still refcnt for objects created in
> previous iters? (I mean object_property_add_child() will add another
> refcnt?)

This is intended, parent need to hold a refcnt of it's child. When we
delete the parent, it will release the child refcnt, and it's child obj will be
automatically deleted because the refcnt is decreased to 0.
The object_unref() release the refcnt hold by object_new().

>
>>
>>>
>>>> +    if (proptable) {
>>>> +        g_hash_table_unref(proptable);
>>>> +    }
>>>> +    if (obj) {
>>>> +        object_unref(obj);
>>>> +    }
>>>>    }
>>>>
>>>>    static void netfilter_class_init(ObjectClass *oc, void *data)
>>>
>>> To reduce the review iterations, I suggest to drop the patch also.
>>> Multiqueue support could be another series on top.
>>
>> I agree, thank you.
>>
>>>
>>> Thanks
>>>
>>>
>>> .
>>>
>>
>
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 12/12] netfilter: add multiqueue support

[Yang Hongyang]

On 09/22/2015 04:32 PM, Jason Wang wrote:
>
>
> On 09/22/2015 04:07 PM, Yang Hongyang wrote:
>> On 09/22/2015 03:36 PM, Jason Wang wrote:
>>>
>>>
>>> On 09/16/2015 08:16 PM, Yang Hongyang wrote:
>>>> From: Yang Hongyang <burnef@gmail.com>
>>>>
>>>> add multiqueue support, if there's multiqueue, we add multi netfilter
>>>> objects, other netfilter objects is the child of the first added
>>>> netfilter
>>>> object. So when we delete a netfilter, the other netfilter objects we
>>>> added will be automatically deleted.
>>>>
>>>> Signed-off-by: Yang Hongyang <burnef@gmail.com>
>>>> ---
>>>> v11: initial patch
>>>> ---
>>>>    net/filter.c | 86
>>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
>>>>    1 file changed, 79 insertions(+), 7 deletions(-)
>>>>
>>>> diff --git a/net/filter.c b/net/filter.c
>>>> index aea619a..cc27528 100644
>>>> --- a/net/filter.c
>>>> +++ b/net/filter.c
>>>> @@ -142,16 +142,25 @@ static void netfilter_finalize(Object *obj)
>>>>        g_free(nf->name);
>>>>    }
>>>>
>>>> +static void proptb_free_val_func(gpointer data)
>>>> +{
>>>> +    g_free(data);
>>>> +}
>>>> +
>>>>    static void netfilter_complete(UserCreatable *uc, Error **errp)
>>>>    {
>>>> -    NetFilterState *nf = NETFILTER(uc);
>>>> +    NetFilterState *nf = NETFILTER(uc), *nfq = NULL;
>>>>        NetClientState *ncs[MAX_QUEUE_NUM];
>>>> -    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc);
>>>> -    int queues;
>>>> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc), *nfqc = NULL;
>>>> +    int queues, i;
>>>>        Error *local_err = NULL;
>>>> -    char *str, *info;
>>>> +    char *str, *info, *name;
>>>>        ObjectProperty *prop;
>>>>        StringOutputVisitor *ov;
>>>> +    Object *obj = NULL;
>>>> +    GHashTable *proptable = NULL;
>>>> +    GHashTableIter iter;
>>>> +    gpointer key, value;
>>>>
>>>>        if (!nf->netdev_id) {
>>>>            error_setg(errp, "Parameter 'netdev' is required");
>>>> @@ -165,9 +174,6 @@ static void netfilter_complete(UserCreatable
>>>> *uc, Error **errp)
>>>>            error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "netdev",
>>>>                       "a network backend id");
>>>>            return;
>>>> -    } else if (queues > 1) {
>>>> -        error_setg(errp, "Multi queue is not supported");
>>>> -        return;
>>>>        }
>>>>
>>>>        if (get_vhost_net(ncs[0])) {
>>>> @@ -187,6 +193,17 @@ static void netfilter_complete(UserCreatable
>>>> *uc, Error **errp)
>>>>        }
>>>>        QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);
>>>>
>>>> +    if (queues > 1) {
>>>> +        /*
>>>> +         * Store the properties of the filter except "type" property.
>>>> +         * When there's multiqueue, we will create a new filter object
>>>> +         * of the same type and same properties. this hashtable is
>>>> used
>>>> +         * to set newly created object properties.
>>>> +         */
>>>> +        proptable = g_hash_table_new_full(NULL, NULL, NULL,
>>>> +                                          proptb_free_val_func);
>>>> +    }
>>>
>>> I'm thinking whether or not duplicate all the properties in each
>>> netfilters is a good method. Maybe we can have a another ojbect with
>>> array of pointers to NetFilter objects embedded? Another question is
>>> whether or not we need to do this at this level. Maybe we can make the
>>> necessary Netfilter multiqueue aware. E.g let buffer filter to have
>>> multiqueue also? Then you may only need a single timer?
>>
>> Sorry I don't get it at first. I also thought about make the buffer
>> filter to
>> have multiqueue, but there comes problem, how to distinguish which queue
>> we should go in when receive the packet, we need to add a mechanism to
>> distinguish which queue belongs to which net client's queue, that will
>> be more complex, while current multiqueue implementation of net clients
>> is multiple net clients with the same name, current solution is the
>> simplest
>> solution I can think of...
>
> I'm not sure I get this. But there's a queue_index filed in each
> NetClientState which may help in this case.

Ah, I see, thank you. There's another reason I do this in filter abstract
layer is that the concrete filter implementation do not need to deal with the
multiqueue, will simplify the filter implement a lot, otherwise, every
filter need to implement it's own multiqueue support.

>
>
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 04/12] net: merge qemu_deliver_packet and qemu_deliver_packet_iov

[Jason Wang]

On 09/22/2015 04:21 PM, Yang Hongyang wrote:
>
>
> On 09/22/2015 04:14 PM, Jason Wang wrote:
>>
>>
>> On 09/22/2015 03:44 PM, Yang Hongyang wrote:
>>> Hi Jason,
>>>
>>>    Thanks for the review,
>>>
>>> On 09/22/2015 03:30 PM, Jason Wang wrote:
>>>>
>>>>
>>>> On 09/16/2015 08:16 PM, Yang Hongyang wrote:
>>>>> qemu_deliver_packet_iov already have the compat delivery, we
>>>>> can drop qemu_deliver_packet.
>>>>>
>>>>> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
>>>>> ---
>>>>>    include/net/net.h |  5 -----
>>>>>    net/net.c         | 40 +++++++---------------------------------
>>>>>    net/queue.c       |  6 +++++-
>>>>>    3 files changed, 12 insertions(+), 39 deletions(-)
>>>>>
>>>>> diff --git a/include/net/net.h b/include/net/net.h
>>>>> index 36e5fab..7af3e15 100644
>>>>> --- a/include/net/net.h
>>>>> +++ b/include/net/net.h
>>>>> @@ -152,11 +152,6 @@ void qemu_check_nic_model(NICInfo *nd, const
>>>>> char *model);
>>>>>    int qemu_find_nic_model(NICInfo *nd, const char * const *models,
>>>>>                            const char *default_model);
>>>>>
>>>>> -ssize_t qemu_deliver_packet(NetClientState *sender,
>>>>> -                            unsigned flags,
>>>>> -                            const uint8_t *data,
>>>>> -                            size_t size,
>>>>> -                            void *opaque);
>>>>>    ssize_t qemu_deliver_packet_iov(NetClientState *sender,
>>>>>                                unsigned flags,
>>>>>                                const struct iovec *iov,
>>>>> diff --git a/net/net.c b/net/net.c
>>>>> index ad37419..ca35b27 100644
>>>>> --- a/net/net.c
>>>>> +++ b/net/net.c
>>>>> @@ -597,36 +597,6 @@ static ssize_t filter_receive(NetClientState
>>>>> *nc, NetFilterChain chain,
>>>>>        return filter_receive_iov(nc, chain, sender, flags, &iov, 1,
>>>>> sent_cb);
>>>>>    }
>>>>>
>>>>> -ssize_t qemu_deliver_packet(NetClientState *sender,
>>>>> -                            unsigned flags,
>>>>> -                            const uint8_t *data,
>>>>> -                            size_t size,
>>>>> -                            void *opaque)
>>>>> -{
>>>>> -    NetClientState *nc = opaque;
>>>>> -    ssize_t ret;
>>>>> -
>>>>> -    if (nc->link_down) {
>>>>> -        return size;
>>>>> -    }
>>>>> -
>>>>> -    if (nc->receive_disabled) {
>>>>> -        return 0;
>>>>> -    }
>>>>> -
>>>>> -    if (flags & QEMU_NET_PACKET_FLAG_RAW && nc->info->receive_raw) {
>>>>> -        ret = nc->info->receive_raw(nc, data, size);
>>>>> -    } else {
>>>>> -        ret = nc->info->receive(nc, data, size);
>>>>> -    }
>>>>> -
>>>>> -    if (ret == 0) {
>>>>> -        nc->receive_disabled = 1;
>>>>> -    }
>>>>> -
>>>>> -    return ret;
>>>>> -}
>>>>> -
>>>>>    void qemu_purge_queued_packets(NetClientState *nc)
>>>>>    {
>>>>>        if (!nc->peer) {
>>>>> @@ -717,14 +687,18 @@ ssize_t qemu_send_packet_raw(NetClientState
>>>>> *nc, const uint8_t *buf, int size)
>>>>>    }
>>>>>
>>>>>    static ssize_t nc_sendv_compat(NetClientState *nc, const struct
>>>>> iovec *iov,
>>>>> -                               int iovcnt)
>>>>> +                               int iovcnt, unsigned flags)
>>>>>    {
>>>>>        uint8_t buffer[NET_BUFSIZE];
>>>>>        size_t offset;
>>>>>
>>>>>        offset = iov_to_buf(iov, iovcnt, 0, buffer, sizeof(buffer));
>>>>>
>>>>> -    return nc->info->receive(nc, buffer, offset);
>>>>> +    if (flags & QEMU_NET_PACKET_FLAG_RAW && nc->info->receive_raw) {
>>>>> +        return nc->info->receive_raw(nc, buffer, offset);
>>>>> +    } else {
>>>>> +        return nc->info->receive(nc, buffer, offset);
>>>>> +    }
>>>>
>>>> Then for net clients that doesn't support receive_iov, an extra memcpy
>>>> is introduced. This seems unnecessary. And this patch looks
>>>> independent,
>>>> so please drop this patch from the series (This can also help to
>>>> reduce
>>>> the iteration). I think we may need this after all net clients support
>>>> receive_iov().
>>>
>>> This patch is required by the next patch which introduce a
>>> +/* Returns:
>>> + *   >0 - success
>>> + *    0 - queue packet for future redelivery
>>> + *   <0 - failure (discard packet)
>>> + */
>>> +typedef ssize_t (NetQueueDeliverFunc)(NetClientState *sender,
>>> +                                      unsigned flags,
>>> +                                      const struct iovec *iov,
>>> +                                      int iovcnt,
>>> +                                      void *opaque);
>>>
>>> If we drop this patch, we will need to introduce 2 deliver func, which
>>> seems not clean and simple.
>>
>> Then you can probably skip the iov_to_buf() if iov_cnt is one in the
>> above code?
>
> Seems like a good idea to avoid the extra memcpy, thank you! BTW,
> can I do this on top of the series if there's no other comments?

Better squash the changes in this patch or another independent patch
before this patch. (We don't want to break bisection).

> so that the first 10 patch can be merged as is. if not, I will send
> another
> version v12 to fix this.

Post V12 or if you wish, I can fix this myself :)

Thanks

Re: [Qemu-devel] [PATCH v11 12/12] netfilter: add multiqueue support

[Jason Wang]

On 09/22/2015 04:35 PM, Yang Hongyang wrote:
>
>
> On 09/22/2015 04:31 PM, Jason Wang wrote:
> [...]
>>>>> +        /* setup filter */
>>>>> +        nfqc = NETFILTER_GET_CLASS(obj);
>>>>> +        if (nfqc->setup) {
>>>>> +            nfqc->setup(nfq, &local_err);
>>>>> +            if (local_err) {
>>>>> +                error_propagate(errp, local_err);
>>>>> +                goto out;
>>>>> +            }
>>>>> +        }
>>>>> +        QTAILQ_INSERT_TAIL(&nfq->netdev->filters, nfq, next);
>>>>> +        object_unref(obj);
>>>              ^^^^^^^^^^^^
>>>
>>>>> +    }
>>>>> +
>>>>> +    if (proptable) {
>>>>> +        g_hash_table_unref(proptable);
>>>>> +    }
>>>>> +    return;
>>>>> +
>>>>> +out:
>>>>
>>>> We may leak objects here.
>>>
>>> Seems not. see above object_unref(), if we come to
>>> here, object create in previous iter will be unrefed.
>>
>> Maybe I miss something. But there's still refcnt for objects created in
>> previous iters? (I mean object_property_add_child() will add another
>> refcnt?)
>
> This is intended, parent need to hold a refcnt of it's child. When we
> delete the parent, it will release the child refcnt, and it's child
> obj will be
> automatically deleted because the refcnt is decreased to 0.
> The object_unref() release the refcnt hold by object_new(). 

I see. Thanks.

Re: [Qemu-devel] [PATCH v11 04/12] net: merge qemu_deliver_packet and qemu_deliver_packet_iov

[Yang Hongyang]

On 09/22/2015 05:19 PM, Jason Wang wrote:
[...]
>>>>
>>>> If we drop this patch, we will need to introduce 2 deliver func, which
>>>> seems not clean and simple.
>>>
>>> Then you can probably skip the iov_to_buf() if iov_cnt is one in the
>>> above code?
>>
>> Seems like a good idea to avoid the extra memcpy, thank you! BTW,
>> can I do this on top of the series if there's no other comments?
>
> Better squash the changes in this patch or another independent patch
> before this patch. (We don't want to break bisection).
>
>> so that the first 10 patch can be merged as is. if not, I will send
>> another
>> version v12 to fix this.
>
> Post V12 or if you wish, I can fix this myself :)

It's great if you can fix this, then I can focus on the multiqueue
support later, thanks a lot for the help! :)

>
> Thanks
>
>
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 12/12] netfilter: add multiqueue support

[Jason Wang]

On 09/22/2015 04:43 PM, Yang Hongyang wrote:
> On 09/22/2015 04:32 PM, Jason Wang wrote:
>>
>>
>> On 09/22/2015 04:07 PM, Yang Hongyang wrote:
>>> On 09/22/2015 03:36 PM, Jason Wang wrote:
>>>>
>>>>
>>>> On 09/16/2015 08:16 PM, Yang Hongyang wrote:
>>>>> From: Yang Hongyang <burnef@gmail.com>
>>>>>
>>>>> add multiqueue support, if there's multiqueue, we add multi netfilter
>>>>> objects, other netfilter objects is the child of the first added
>>>>> netfilter
>>>>> object. So when we delete a netfilter, the other netfilter objects we
>>>>> added will be automatically deleted.
>>>>>
>>>>> Signed-off-by: Yang Hongyang <burnef@gmail.com>
>>>>> ---
>>>>> v11: initial patch
>>>>> ---
>>>>>    net/filter.c | 86
>>>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
>>>>>    1 file changed, 79 insertions(+), 7 deletions(-)
>>>>>
>>>>> diff --git a/net/filter.c b/net/filter.c
>>>>> index aea619a..cc27528 100644
>>>>> --- a/net/filter.c
>>>>> +++ b/net/filter.c
>>>>> @@ -142,16 +142,25 @@ static void netfilter_finalize(Object *obj)
>>>>>        g_free(nf->name);
>>>>>    }
>>>>>
>>>>> +static void proptb_free_val_func(gpointer data)
>>>>> +{
>>>>> +    g_free(data);
>>>>> +}
>>>>> +
>>>>>    static void netfilter_complete(UserCreatable *uc, Error **errp)
>>>>>    {
>>>>> -    NetFilterState *nf = NETFILTER(uc);
>>>>> +    NetFilterState *nf = NETFILTER(uc), *nfq = NULL;
>>>>>        NetClientState *ncs[MAX_QUEUE_NUM];
>>>>> -    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc);
>>>>> -    int queues;
>>>>> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc), *nfqc = NULL;
>>>>> +    int queues, i;
>>>>>        Error *local_err = NULL;
>>>>> -    char *str, *info;
>>>>> +    char *str, *info, *name;
>>>>>        ObjectProperty *prop;
>>>>>        StringOutputVisitor *ov;
>>>>> +    Object *obj = NULL;
>>>>> +    GHashTable *proptable = NULL;
>>>>> +    GHashTableIter iter;
>>>>> +    gpointer key, value;
>>>>>
>>>>>        if (!nf->netdev_id) {
>>>>>            error_setg(errp, "Parameter 'netdev' is required");
>>>>> @@ -165,9 +174,6 @@ static void netfilter_complete(UserCreatable
>>>>> *uc, Error **errp)
>>>>>            error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "netdev",
>>>>>                       "a network backend id");
>>>>>            return;
>>>>> -    } else if (queues > 1) {
>>>>> -        error_setg(errp, "Multi queue is not supported");
>>>>> -        return;
>>>>>        }
>>>>>
>>>>>        if (get_vhost_net(ncs[0])) {
>>>>> @@ -187,6 +193,17 @@ static void netfilter_complete(UserCreatable
>>>>> *uc, Error **errp)
>>>>>        }
>>>>>        QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);
>>>>>
>>>>> +    if (queues > 1) {
>>>>> +        /*
>>>>> +         * Store the properties of the filter except "type"
>>>>> property.
>>>>> +         * When there's multiqueue, we will create a new filter
>>>>> object
>>>>> +         * of the same type and same properties. this hashtable is
>>>>> used
>>>>> +         * to set newly created object properties.
>>>>> +         */
>>>>> +        proptable = g_hash_table_new_full(NULL, NULL, NULL,
>>>>> +                                          proptb_free_val_func);
>>>>> +    }
>>>>
>>>> I'm thinking whether or not duplicate all the properties in each
>>>> netfilters is a good method. Maybe we can have a another ojbect with
>>>> array of pointers to NetFilter objects embedded? Another question is
>>>> whether or not we need to do this at this level. Maybe we can make the
>>>> necessary Netfilter multiqueue aware. E.g let buffer filter to have
>>>> multiqueue also? Then you may only need a single timer?
>>>
>>> Sorry I don't get it at first. I also thought about make the buffer
>>> filter to
>>> have multiqueue, but there comes problem, how to distinguish which
>>> queue
>>> we should go in when receive the packet, we need to add a mechanism to
>>> distinguish which queue belongs to which net client's queue, that will
>>> be more complex, while current multiqueue implementation of net clients
>>> is multiple net clients with the same name, current solution is the
>>> simplest
>>> solution I can think of...
>>
>> I'm not sure I get this. But there's a queue_index filed in each
>> NetClientState which may help in this case.
>
> Ah, I see, thank you. There's another reason I do this in filter abstract
> layer is that the concrete filter implementation do not need to deal
> with the
> multiqueue, will simplify the filter implement a lot, otherwise, every
> filter need to implement it's own multiqueue support.

Probably not, just pointing all queues to a single netfilter object may
be sufficient (e.g for dump). And it's much easier for some kind of
filter (e.g traffic throttling) if it was multiqueue aware.

Re: [Qemu-devel] [PATCH v11 04/12] net: merge qemu_deliver_packet and qemu_deliver_packet_iov

[Jason Wang]

On 09/22/2015 05:26 PM, Yang Hongyang wrote:
> On 09/22/2015 05:19 PM, Jason Wang wrote:
> [...]
>>>>>
>>>>> If we drop this patch, we will need to introduce 2 deliver func,
>>>>> which
>>>>> seems not clean and simple.
>>>>
>>>> Then you can probably skip the iov_to_buf() if iov_cnt is one in the
>>>> above code?
>>>
>>> Seems like a good idea to avoid the extra memcpy, thank you! BTW,
>>> can I do this on top of the series if there's no other comments?
>>
>> Better squash the changes in this patch or another independent patch
>> before this patch. (We don't want to break bisection).
>>
>>> so that the first 10 patch can be merged as is. if not, I will send
>>> another
>>> version v12 to fix this.
>>
>> Post V12 or if you wish, I can fix this myself :)
>
> It's great if you can fix this, then I can focus on the multiqueue
> support later, thanks a lot for the help! :)

Ok, will do.

Re: [Qemu-devel] [PATCH v11 12/12] netfilter: add multiqueue support

[Yang Hongyang]

On 09/22/2015 05:30 PM, Jason Wang wrote:
[...]
>>>>>>
>>>>>>         if (!nf->netdev_id) {
>>>>>>             error_setg(errp, "Parameter 'netdev' is required");
>>>>>> @@ -165,9 +174,6 @@ static void netfilter_complete(UserCreatable
>>>>>> *uc, Error **errp)
>>>>>>             error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "netdev",
>>>>>>                        "a network backend id");
>>>>>>             return;
>>>>>> -    } else if (queues > 1) {
>>>>>> -        error_setg(errp, "Multi queue is not supported");
>>>>>> -        return;
>>>>>>         }
>>>>>>
>>>>>>         if (get_vhost_net(ncs[0])) {
>>>>>> @@ -187,6 +193,17 @@ static void netfilter_complete(UserCreatable
>>>>>> *uc, Error **errp)
>>>>>>         }
>>>>>>         QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);
>>>>>>
>>>>>> +    if (queues > 1) {
>>>>>> +        /*
>>>>>> +         * Store the properties of the filter except "type"
>>>>>> property.
>>>>>> +         * When there's multiqueue, we will create a new filter
>>>>>> object
>>>>>> +         * of the same type and same properties. this hashtable is
>>>>>> used
>>>>>> +         * to set newly created object properties.
>>>>>> +         */
>>>>>> +        proptable = g_hash_table_new_full(NULL, NULL, NULL,
>>>>>> +                                          proptb_free_val_func);
>>>>>> +    }
>>>>>
>>>>> I'm thinking whether or not duplicate all the properties in each
>>>>> netfilters is a good method. Maybe we can have a another ojbect with
>>>>> array of pointers to NetFilter objects embedded? Another question is
>>>>> whether or not we need to do this at this level. Maybe we can make the
>>>>> necessary Netfilter multiqueue aware. E.g let buffer filter to have
>>>>> multiqueue also? Then you may only need a single timer?
>>>>
>>>> Sorry I don't get it at first. I also thought about make the buffer
>>>> filter to
>>>> have multiqueue, but there comes problem, how to distinguish which
>>>> queue
>>>> we should go in when receive the packet, we need to add a mechanism to
>>>> distinguish which queue belongs to which net client's queue, that will
>>>> be more complex, while current multiqueue implementation of net clients
>>>> is multiple net clients with the same name, current solution is the
>>>> simplest
>>>> solution I can think of...
>>>
>>> I'm not sure I get this. But there's a queue_index filed in each
>>> NetClientState which may help in this case.
>>
>> Ah, I see, thank you. There's another reason I do this in filter abstract
>> layer is that the concrete filter implementation do not need to deal
>> with the
>> multiqueue, will simplify the filter implement a lot, otherwise, every
>> filter need to implement it's own multiqueue support.
>
> Probably not, just pointing all queues to a single netfilter object may
> be sufficient (e.g for dump). And it's much easier for some kind of
> filter (e.g traffic throttling) if it was multiqueue aware.

I see, will think about doing this other way, thank you!

>
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 00/12] Add a netfilter object and netbuffer filter

[Jason Wang]

On 09/16/2015 08:15 PM, Yang Hongyang wrote:
> This patch add an netfilter abstract object, captures all network packets
> on associated netdev. Also implement a concrete filter buffer based on
> this abstract object. the "buffer" netfilter could be used by VM FT solutions
> like MicroCheckpointing, to buffer/release packets. Or to simulate
> packet delay.
>
> You can also get the series from:
> https://github.com/macrosheep/qemu/tree/netfilter-v11
>
> Usage:
>  -netdev tap,id=bn0
>  -device e1000,netdev=bn0
>  -object filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000
>
> dynamically add/remove netfilters:
>  object_add filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000
>  object_del f0
>
> NOTE:
>  interval's scale is microsecond.
>  chain is optional, and is one of in|out|all, default is "all".
>        "in" means this filter will receive packets sent to the @netdev
>        "out" means this filter will receive packets sent from the @netdev
>        "all" means this filter will receive packets both sent to/from
>              the @netdev
>
> v11:
>  - address Jason&Daniel's comments
>  - add multiqueue support, the last 2 patches
>  - rebased to the latest master
>
> v10:
>  - Reimplemented using QOM (suggested by stefan)
>  - Do not export NetQueue internals (suggested by stefan)
>  - see individual patch for detail
>
> v9:
>  - squash command description and help to patch 1&3
>  - qapi changes according to Markus&Eric's comments
>  - see individual patch for detail
>
> v8:
>  - some minor fixes according to Thomas's comments
>  - rebased to the latest master branch
>
> v7:
>  - print filter info when execute 'info network'
>  - addressed Jason's comments
>
> v6:
>  - add multiqueue support, please see individual patch for detail
>
> v5:
>  - add a sent_cb param to filter receive_iov api
>  - squash the 4th patch into patch 3
>  - remove dummy sent_cb (buffer filter)
>  - addressed Jason's other comments, see individual patches for detail
>
> v4:
>  - get rid of struct Filter
>  - squash the 4th patch into patch 2
>  - fix qemu_netfilter_pass_to_next_iov
>  - get rid of bh (buffer filter)
>  - release the packet to next filter instead of to receiver (buffer filter)
>
> v3:
>  - add an api to pass the packet to next filter
>  - remove netfilters when delete netdev
>  - add qtest testcases for netfilter
>  - addressed comments from Jason
>
> v2:
>  - add a chain option to netfilter object
>  - move the hook place earlier, before net_queue_send
>  - drop the unused api in buffer filter
>  - squash buffer filter patches into one
>  - remove receive() api from netfilter, only receive_iov() is enough
>  - addressed comments from Jason&Thomas
>
> v1:
>  initial patch.
>
> Yang Hongyang (12):
>   qmp: delete qemu opts when delete an object
>   init/cleanup of netfilter object
>   netfilter: hook packets before net queue send
>   net: merge qemu_deliver_packet and qemu_deliver_packet_iov
>   net/queue: introduce NetQueueDeliverFunc
>   netfilter: add an API to pass the packet to next filter
>   netfilter: print filter info associate with the netdev
>   net/queue: export qemu_net_queue_append_iov
>   netfilter: add a netbuffer filter
>   tests: add test cases for netfilter object
>   netfilter/multiqueue: introduce netfilter name
>   netfilter: add multiqueue support
>
>  include/net/filter.h    |  76 ++++++++++++
>  include/net/net.h       |   6 +-
>  include/net/queue.h     |  20 +++-
>  include/qemu/typedefs.h |   1 +
>  net/Makefile.objs       |   2 +
>  net/filter-buffer.c     | 170 +++++++++++++++++++++++++++
>  net/filter.c            | 306 ++++++++++++++++++++++++++++++++++++++++++++++++
>  net/net.c               | 108 +++++++++++++----
>  net/queue.c             |  24 ++--
>  qapi-schema.json        |  18 +++
>  qemu-options.hx         |  18 +++
>  qmp.c                   |   4 +
>  tests/.gitignore        |   1 +
>  tests/Makefile          |   2 +
>  tests/test-netfilter.c  | 200 +++++++++++++++++++++++++++++++
>  vl.c                    |  18 ++-
>  16 files changed, 927 insertions(+), 47 deletions(-)
>  create mode 100644 include/net/filter.h
>  create mode 100644 net/filter-buffer.c
>  create mode 100644 net/filter.c
>  create mode 100644 tests/test-netfilter.c
>

Apply patch 1 to patch 10 (with tweak on patch 4 for single iov).

https://github.com/jasowang/qemu/commits/net

Thanks a lot.

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Markus Armbruster]

This has finally reached the front of my review queue.  I apologize for
the loooong delay.

Copying Paolo for another pair of eyeballs (he wrote this code).

Yang Hongyang <yanghy@cn.fujitsu.com> writes:

> When delete an object, we need to delete the associated qemu opts,
> otherwise, we can not add another object with the same name using
> object_add.
> The case happens when we start qemu with:
> -object xxx,id=aa
> then we delete this object with:
> object_del aa
> then add the object with:
> object_add xxx,id=aa
>
> QEMU will return error with Duplicate ID error.
>
> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
> ---
>  qmp.c | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/qmp.c b/qmp.c
> index 9623c80..4bd44c3 100644
> --- a/qmp.c
> +++ b/qmp.c
> @@ -686,6 +686,7 @@ void qmp_object_del(const char *id, Error **errp)
>  {
>      Object *container;
>      Object *obj;
> +    QemuOpts *opts;
>  
>      container = object_get_objects_root();
>      obj = object_resolve_path_component(container, id);
> @@ -699,6 +700,9 @@ void qmp_object_del(const char *id, Error **errp)
>          return;
>      }
>      object_unparent(obj);
> +
> +    opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
> +    qemu_opts_del(opts);

qemu_find_opts_err("object", &error_abort) please, because when it
fails, we want to die right away, not when the null pointer it returns
gets dereferenced.

Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
your patch's fault.

Elsewhere, we store the QemuOpts in the object just so we can delete it:
DeviceState, DriveInfo.  Paolo, what do you think?

>  }
>  
>  MemoryDeviceInfoList *qmp_query_memory_devices(Error **errp)

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Yang Hongyang]

On 09/24/2015 03:43 PM, Markus Armbruster wrote:
> This has finally reached the front of my review queue.  I apologize for
> the loooong delay.
>
> Copying Paolo for another pair of eyeballs (he wrote this code).
>
[...]
>> +
>> +    opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
>> +    qemu_opts_del(opts);
>
> qemu_find_opts_err("object", &error_abort) please, because when it
> fails, we want to die right away, not when the null pointer it returns
> gets dereferenced.

Thanks for the review.
Jason, do you want me to propose a fix on top of this series or simply drop
this for now because this patch is an independent bug fix and won't affect the
other filter patch series.

>
> Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
> your patch's fault.
>
> Elsewhere, we store the QemuOpts in the object just so we can delete it:
> DeviceState, DriveInfo.  Paolo, what do you think?

I don't get it. Currently, only objects created at the beginning through
QEMU command line will be stored in the QemuOpts, objects that created
with object_add won't stored in QemuOpts. Do you mean for DeviceState,
DriveInfo they store there QemuOpts explicity so that they can delete it?
Why don't we just delete it from objects directly instead?

>
>>   }
>>
>>   MemoryDeviceInfoList *qmp_query_memory_devices(Error **errp)
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Markus Armbruster]

Yang Hongyang <yanghy@cn.fujitsu.com> writes:

> Add a netfilter object based on QOM.
>
> A netfilter is attached to a netdev, captures all network packets
> that pass through the netdev. When we delete the netdev, we also
> delete the netfilter object attached to it, because if the netdev is
> removed, the filter which attached to it is useless.
>
> QTAILQ_ENTRY next used by netdev, filter belongs to the specific netdev is
> in this queue.

I don't get this paragraph.  Not sure it's needed.

> Also init delayed object after net_init_clients, because netfilters need
> to be initialized after net clients initialized.

A paragraph starting with "Also" in a commit message is a pretty good
sign the patch should be split :)

>
> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
> ---
> v11: no need to free nf->netdev_id, it will be auto freeed while object deleted
>      remove global_list net_filters, will add back when needed
> v10: use QOM for netfilter
> v9: use flat union instead of simple union in QAPI schema
> v8: include vhost_net header
> v7: add check for vhost
>     fix error propagate bug
> v6: add multiqueue support (net_filter_init1)
> v5: remove model from NetFilterState
>     add a sent_cb param to receive_iov API
> ---
>  include/net/filter.h    |  60 +++++++++++++++++++++
>  include/net/net.h       |   1 +
>  include/qemu/typedefs.h |   1 +
>  net/Makefile.objs       |   1 +
>  net/filter.c            | 138 ++++++++++++++++++++++++++++++++++++++++++++++++
>  net/net.c               |   7 +++
>  qapi-schema.json        |  18 +++++++
>  vl.c                    |  13 ++---
>  8 files changed, 233 insertions(+), 6 deletions(-)
>  create mode 100644 include/net/filter.h
>  create mode 100644 net/filter.c
>
> diff --git a/include/net/filter.h b/include/net/filter.h
> new file mode 100644
> index 0000000..226f2f7
> --- /dev/null
> +++ b/include/net/filter.h
> @@ -0,0 +1,60 @@
> +/*
> + * Copyright (c) 2015 FUJITSU LIMITED
> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or
> + * later.  See the COPYING file in the top-level directory.
> + */
> +
> +#ifndef QEMU_NET_FILTER_H
> +#define QEMU_NET_FILTER_H
> +
> +#include "qom/object.h"
> +#include "qemu-common.h"
> +#include "qemu/typedefs.h"
> +#include "net/queue.h"
> +
> +#define TYPE_NETFILTER "netfilter"
> +#define NETFILTER(obj) \
> +    OBJECT_CHECK(NetFilterState, (obj), TYPE_NETFILTER)
> +#define NETFILTER_GET_CLASS(obj) \
> +    OBJECT_GET_CLASS(NetFilterClass, (obj), TYPE_NETFILTER)
> +#define NETFILTER_CLASS(klass) \
> +    OBJECT_CLASS_CHECK(NetFilterClass, (klass), TYPE_NETFILTER)
> +
> +typedef void (FilterSetup) (NetFilterState *nf, Error **errp);
> +typedef void (FilterCleanup) (NetFilterState *nf);
> +/*
> + * Return:
> + *   0: finished handling the packet, we should continue
> + *   size: filter stolen this packet, we stop pass this packet further
> + */
> +typedef ssize_t (FilterReceiveIOV)(NetFilterState *nc,
> +                                   NetClientState *sender,
> +                                   unsigned flags,
> +                                   const struct iovec *iov,
> +                                   int iovcnt,
> +                                   NetPacketSent *sent_cb);
> +
> +struct NetFilterClass {
> +    ObjectClass parent_class;
> +
> +    FilterSetup *setup;
> +    FilterCleanup *cleanup;
> +    FilterReceiveIOV *receive_iov;
> +};
> +typedef struct NetFilterClass NetFilterClass;

Not splitting the declaration is more concise:

    typedef struct {
        ObjectClass parent_class;
        FilterSetup *setup;
        FilterCleanup *cleanup;
        FilterReceiveIOV *receive_iov;
    } NetFilterClass;

Are any of the methods optional?  If yes, please add suitable comments.

> +
> +
> +struct NetFilterState {
> +    /* private */
> +    Object parent;
> +
> +    /* protected */
> +    char *netdev_id;
> +    NetClientState *netdev;
> +    NetFilterChain chain;
> +    QTAILQ_ENTRY(NetFilterState) next;
> +};
> +
> +#endif /* QEMU_NET_FILTER_H */
> diff --git a/include/net/net.h b/include/net/net.h
> index 6a6cbef..36e5fab 100644
> --- a/include/net/net.h
> +++ b/include/net/net.h
> @@ -92,6 +92,7 @@ struct NetClientState {
>      NetClientDestructor *destructor;
>      unsigned int queue_index;
>      unsigned rxfilter_notify_enabled:1;
> +    QTAILQ_HEAD(, NetFilterState) filters;
>  };
>  
>  typedef struct NICState {
> diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
> index f8a9dd6..2c0648f 100644
> --- a/include/qemu/typedefs.h
> +++ b/include/qemu/typedefs.h
> @@ -45,6 +45,7 @@ typedef struct Monitor Monitor;
>  typedef struct MouseTransformInfo MouseTransformInfo;
>  typedef struct MSIMessage MSIMessage;
>  typedef struct NetClientState NetClientState;
> +typedef struct NetFilterState NetFilterState;
>  typedef struct NICInfo NICInfo;
>  typedef struct PcGuestInfo PcGuestInfo;
>  typedef struct PCIBridge PCIBridge;
> diff --git a/net/Makefile.objs b/net/Makefile.objs
> index ec19cb3..914aec0 100644
> --- a/net/Makefile.objs
> +++ b/net/Makefile.objs
> @@ -13,3 +13,4 @@ common-obj-$(CONFIG_HAIKU) += tap-haiku.o
>  common-obj-$(CONFIG_SLIRP) += slirp.o
>  common-obj-$(CONFIG_VDE) += vde.o
>  common-obj-$(CONFIG_NETMAP) += netmap.o
> +common-obj-y += filter.o
> diff --git a/net/filter.c b/net/filter.c
> new file mode 100644
> index 0000000..3b810c8
> --- /dev/null
> +++ b/net/filter.c
> @@ -0,0 +1,138 @@
> +/*
> + * Copyright (c) 2015 FUJITSU LIMITED
> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or
> + * later.  See the COPYING file in the top-level directory.
> + */
> +
> +#include "qemu-common.h"
> +#include "qapi/qmp/qerror.h"
> +#include "qemu/error-report.h"
> +
> +#include "net/filter.h"
> +#include "net/net.h"
> +#include "net/vhost_net.h"
> +#include "qom/object_interfaces.h"
> +
> +static char *netfilter_get_netdev_id(Object *obj, Error **errp)
> +{
> +    NetFilterState *nf = NETFILTER(obj);
> +
> +    return g_strdup(nf->netdev_id);
> +}
> +
> +static void netfilter_set_netdev_id(Object *obj, const char *str, Error **errp)
> +{
> +    NetFilterState *nf = NETFILTER(obj);
> +
> +    nf->netdev_id = g_strdup(str);
> +}
> +
> +static int netfilter_get_chain(Object *obj, Error **errp G_GNUC_UNUSED)
> +{
> +    NetFilterState *nf = NETFILTER(obj);
> +    return nf->chain;
> +}
> +
> +static void netfilter_set_chain(Object *obj, int chain, Error **errp)
> +{
> +    NetFilterState *nf = NETFILTER(obj);
> +    nf->chain = chain;
> +}
> +
> +static void netfilter_init(Object *obj)
> +{
> +    object_property_add_str(obj, "netdev",
> +                            netfilter_get_netdev_id, netfilter_set_netdev_id,
> +                            NULL);
> +    object_property_add_enum(obj, "chain", "NetFilterChain",
> +                             NetFilterChain_lookup,
> +                             netfilter_get_chain, netfilter_set_chain,
> +                             NULL);
> +}
> +
> +static void netfilter_finalize(Object *obj)
> +{
> +    NetFilterState *nf = NETFILTER(obj);
> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(obj);
> +
> +    if (nfc->cleanup) {

Is ->cleanup optional?

> +        nfc->cleanup(nf);
> +    }
> +
> +    if (nf->netdev && !QTAILQ_EMPTY(&nf->netdev->filters)) {

How can nf->netdev be null?

How can nf->netdev->filters be empty?

> +        QTAILQ_REMOVE(&nf->netdev->filters, nf, next);
> +    }
> +}

Putting netfilter_finalize() after netfilter_complete() would be easier
to understand, because then destruction follows creation.

> +
> +static void netfilter_complete(UserCreatable *uc, Error **errp)
> +{
> +    NetFilterState *nf = NETFILTER(uc);
> +    NetClientState *ncs[MAX_QUEUE_NUM];
> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc);
> +    int queues;
> +    Error *local_err = NULL;
> +
> +    if (!nf->netdev_id) {
> +        error_setg(errp, "Parameter 'netdev' is required");
> +        return;
> +    }
> +
> +    queues = qemu_find_net_clients_except(nf->netdev_id, ncs,
> +                                          NET_CLIENT_OPTIONS_KIND_NIC,
> +                                          MAX_QUEUE_NUM);
> +    if (queues < 1) {
> +        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "netdev",
> +                   "a network backend id");
> +        return;
> +    } else if (queues > 1) {
> +        error_setg(errp, "Multi queue is not supported");

We spell this multiqueue elsewhere.

Since you're only interested in a single queue, you could save stack
space by making ncs[] just one element large.  Not worth it if you
intend to support multiqueue soon.

> +        return;
> +    }
> +
> +    if (get_vhost_net(ncs[0])) {
> +        error_setg(errp, "Vhost is not supported");
> +        return;
> +    }
> +
> +    nf->netdev = ncs[0];
> +
> +    if (nfc->setup) {

Is ->setup optional?

> +        nfc->setup(nf, &local_err);
> +        if (local_err) {
> +            error_propagate(errp, local_err);
> +            return;
> +        }
> +    }
> +    QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);

I think I'd create netdev functions to add and remove filters, so the
filter code doesn't have to modify the netdev state.  Right now,
NetClientState member filters is initialized in net.c, but modified
here.  But I'm not the net maintainer :)

> +}
> +
> +static void netfilter_class_init(ObjectClass *oc, void *data)
> +{
> +    UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
> +
> +    ucc->complete = netfilter_complete;
> +}
> +
> +static const TypeInfo netfilter_info = {
> +    .name = TYPE_NETFILTER,
> +    .parent = TYPE_OBJECT,
> +    .abstract = true,
> +    .class_size = sizeof(NetFilterClass),
> +    .class_init = netfilter_class_init,
> +    .instance_size = sizeof(NetFilterState),
> +    .instance_init = netfilter_init,
> +    .instance_finalize = netfilter_finalize,
> +    .interfaces = (InterfaceInfo[]) {
> +        { TYPE_USER_CREATABLE },
> +        { }
> +    }
> +};
> +
> +static void register_types(void)
> +{
> +    type_register_static(&netfilter_info);
> +}
> +
> +type_init(register_types);
> diff --git a/net/net.c b/net/net.c
> index 28a5597..033f4f3 100644
> --- a/net/net.c
> +++ b/net/net.c
> @@ -44,6 +44,7 @@
>  #include "qapi/opts-visitor.h"
>  #include "qapi/dealloc-visitor.h"
>  #include "sysemu/sysemu.h"
> +#include "net/filter.h"
>  
>  /* Net bridge is currently not supported for W32. */
>  #if !defined(_WIN32)
> @@ -287,6 +288,7 @@ static void qemu_net_client_setup(NetClientState *nc,
>  
>      nc->incoming_queue = qemu_new_net_queue(nc);
>      nc->destructor = destructor;
> +    QTAILQ_INIT(&nc->filters);
>  }
>  
>  NetClientState *qemu_new_net_client(NetClientInfo *info,
> @@ -384,6 +386,7 @@ void qemu_del_net_client(NetClientState *nc)
>  {
>      NetClientState *ncs[MAX_QUEUE_NUM];
>      int queues, i;
> +    NetFilterState *nf, *next;
>  
>      assert(nc->info->type != NET_CLIENT_OPTIONS_KIND_NIC);
>  
> @@ -395,6 +398,10 @@ void qemu_del_net_client(NetClientState *nc)
>                                            MAX_QUEUE_NUM);
>      assert(queues != 0);
>  
> +    QTAILQ_FOREACH_SAFE(nf, &nc->filters, next, next) {
> +        object_unparent(OBJECT(nf));
> +    }
> +
>      /* If there is a peer NIC, delete and cleanup client, but do not free. */
>      if (nc->peer && nc->peer->info->type == NET_CLIENT_OPTIONS_KIND_NIC) {
>          NICState *nic = qemu_get_nic(nc->peer);
> diff --git a/qapi-schema.json b/qapi-schema.json
> index 2bada60..546500a 100644
> --- a/qapi-schema.json
> +++ b/qapi-schema.json
> @@ -2551,6 +2551,24 @@
>      'opts': 'NetClientOptions' } }
>  
>  ##
> +# @NetFilterChain
> +#
> +# netfilter chain, a netfilter is attached to a netdev, captures the
> +# network packets of the netdev.
> +#
> +# @all: the filter will receive packets both sent to/from the netdev, this
> +#       is the default chain.
> +#
> +# @in: the filter will receive packets sent to the netdev.
> +#
> +# @out: the filter will receive packets sent from the netdev.

Uh, inhowfar is this a "chain"?  As far as I can tell, it specifies
whether a netfilter is attached to the transmit queue, the receive
queue, or both.

> +#
> +# Since 2.5
> +##
> +{ 'enum': 'NetFilterChain',
> +  'data': [ 'all', 'in', 'out' ] }
> +
> +##
>  # @InetSocketAddress
>  #
>  # Captures a socket address or address range in the Internet namespace.
> diff --git a/vl.c b/vl.c
> index 066a080..ec589e2 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -2794,6 +2794,7 @@ static bool object_create_initial(const char *type)
>      if (g_str_equal(type, "rng-egd")) {
>          return false;
>      }
> +    /* TODO: return false for concrete netfilters */

This confused me briefly, until I realized the context.  You mean
something like

    /* TODO implement netfilters */

>      return true;
>  }
>  
> @@ -4313,12 +4314,6 @@ int main(int argc, char **argv, char **envp)
>          exit(0);
>      }
>  
> -    if (qemu_opts_foreach(qemu_find_opts("object"),
> -                          object_create,
> -                          object_create_delayed, NULL)) {
> -        exit(1);
> -    }
> -
>      machine_opts = qemu_get_machine_opts();
>      if (qemu_opt_foreach(machine_opts, machine_set_property, current_machine,
>                           NULL)) {
> @@ -4424,6 +4419,12 @@ int main(int argc, char **argv, char **envp)
>          exit(1);
>      }
>  
> +    if (qemu_opts_foreach(qemu_find_opts("object"),
> +                          object_create,
> +                          object_create_delayed, NULL)) {
> +        exit(1);
> +    }
> +
>  #ifdef CONFIG_TPM
>      if (tpm_init() < 0) {
>          exit(1);

I'd make this code motion a separate patch, to make it more prominent in
git-log.

The rest of the patch is harmless: it creates a new abstact class.

The code motion isn't: it actually changes behavior.

Overall, patch is neatly done.

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Yang Hongyang]

On 09/24/2015 04:41 PM, Markus Armbruster wrote:
> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>
>> Add a netfilter object based on QOM.
>>
>> A netfilter is attached to a netdev, captures all network packets
>> that pass through the netdev. When we delete the netdev, we also
>> delete the netfilter object attached to it, because if the netdev is
>> removed, the filter which attached to it is useless.
>>
>> QTAILQ_ENTRY next used by netdev, filter belongs to the specific netdev is
>> in this queue.
>
> I don't get this paragraph.  Not sure it's needed.
>
>> Also init delayed object after net_init_clients, because netfilters need
>> to be initialized after net clients initialized.
>
> A paragraph starting with "Also" in a commit message is a pretty good
> sign the patch should be split :)
>
>>
>> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
>> ---
>> v11: no need to free nf->netdev_id, it will be auto freeed while object deleted
>>       remove global_list net_filters, will add back when needed
>> v10: use QOM for netfilter
>> v9: use flat union instead of simple union in QAPI schema
>> v8: include vhost_net header
>> v7: add check for vhost
>>      fix error propagate bug
>> v6: add multiqueue support (net_filter_init1)
>> v5: remove model from NetFilterState
>>      add a sent_cb param to receive_iov API
>> ---
>>   include/net/filter.h    |  60 +++++++++++++++++++++
>>   include/net/net.h       |   1 +
>>   include/qemu/typedefs.h |   1 +
>>   net/Makefile.objs       |   1 +
>>   net/filter.c            | 138 ++++++++++++++++++++++++++++++++++++++++++++++++
>>   net/net.c               |   7 +++
>>   qapi-schema.json        |  18 +++++++
>>   vl.c                    |  13 ++---
>>   8 files changed, 233 insertions(+), 6 deletions(-)
>>   create mode 100644 include/net/filter.h
>>   create mode 100644 net/filter.c
>>
>> diff --git a/include/net/filter.h b/include/net/filter.h
>> new file mode 100644
>> index 0000000..226f2f7
>> --- /dev/null
>> +++ b/include/net/filter.h
>> @@ -0,0 +1,60 @@
>> +/*
>> + * Copyright (c) 2015 FUJITSU LIMITED
>> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>> + * later.  See the COPYING file in the top-level directory.
>> + */
>> +
>> +#ifndef QEMU_NET_FILTER_H
>> +#define QEMU_NET_FILTER_H
>> +
>> +#include "qom/object.h"
>> +#include "qemu-common.h"
>> +#include "qemu/typedefs.h"
>> +#include "net/queue.h"
>> +
>> +#define TYPE_NETFILTER "netfilter"
>> +#define NETFILTER(obj) \
>> +    OBJECT_CHECK(NetFilterState, (obj), TYPE_NETFILTER)
>> +#define NETFILTER_GET_CLASS(obj) \
>> +    OBJECT_GET_CLASS(NetFilterClass, (obj), TYPE_NETFILTER)
>> +#define NETFILTER_CLASS(klass) \
>> +    OBJECT_CLASS_CHECK(NetFilterClass, (klass), TYPE_NETFILTER)
>> +
>> +typedef void (FilterSetup) (NetFilterState *nf, Error **errp);
>> +typedef void (FilterCleanup) (NetFilterState *nf);
>> +/*
>> + * Return:
>> + *   0: finished handling the packet, we should continue
>> + *   size: filter stolen this packet, we stop pass this packet further
>> + */
>> +typedef ssize_t (FilterReceiveIOV)(NetFilterState *nc,
>> +                                   NetClientState *sender,
>> +                                   unsigned flags,
>> +                                   const struct iovec *iov,
>> +                                   int iovcnt,
>> +                                   NetPacketSent *sent_cb);
>> +
>> +struct NetFilterClass {
>> +    ObjectClass parent_class;
>> +
>> +    FilterSetup *setup;
>> +    FilterCleanup *cleanup;
>> +    FilterReceiveIOV *receive_iov;
>> +};
>> +typedef struct NetFilterClass NetFilterClass;
>
> Not splitting the declaration is more concise:
>
>      typedef struct {
>          ObjectClass parent_class;
>          FilterSetup *setup;
>          FilterCleanup *cleanup;
>          FilterReceiveIOV *receive_iov;
>      } NetFilterClass;
>
> Are any of the methods optional?  If yes, please add suitable comments.

Hi Markus, I split it because the checkpatch.pl told me to do so...

>
>> +
>> +
>> +struct NetFilterState {
>> +    /* private */
>> +    Object parent;
>> +
>> +    /* protected */
>> +    char *netdev_id;
>> +    NetClientState *netdev;
>> +    NetFilterChain chain;
>> +    QTAILQ_ENTRY(NetFilterState) next;
>> +};
>> +
>> +#endif /* QEMU_NET_FILTER_H */
>> diff --git a/include/net/net.h b/include/net/net.h
>> index 6a6cbef..36e5fab 100644
>> --- a/include/net/net.h
>> +++ b/include/net/net.h
>> @@ -92,6 +92,7 @@ struct NetClientState {
>>       NetClientDestructor *destructor;
>>       unsigned int queue_index;
>>       unsigned rxfilter_notify_enabled:1;
>> +    QTAILQ_HEAD(, NetFilterState) filters;
>>   };
>>
>>   typedef struct NICState {
>> diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
>> index f8a9dd6..2c0648f 100644
>> --- a/include/qemu/typedefs.h
>> +++ b/include/qemu/typedefs.h
>> @@ -45,6 +45,7 @@ typedef struct Monitor Monitor;
>>   typedef struct MouseTransformInfo MouseTransformInfo;
>>   typedef struct MSIMessage MSIMessage;
>>   typedef struct NetClientState NetClientState;
>> +typedef struct NetFilterState NetFilterState;
>>   typedef struct NICInfo NICInfo;
>>   typedef struct PcGuestInfo PcGuestInfo;
>>   typedef struct PCIBridge PCIBridge;
>> diff --git a/net/Makefile.objs b/net/Makefile.objs
>> index ec19cb3..914aec0 100644
>> --- a/net/Makefile.objs
>> +++ b/net/Makefile.objs
>> @@ -13,3 +13,4 @@ common-obj-$(CONFIG_HAIKU) += tap-haiku.o
>>   common-obj-$(CONFIG_SLIRP) += slirp.o
>>   common-obj-$(CONFIG_VDE) += vde.o
>>   common-obj-$(CONFIG_NETMAP) += netmap.o
>> +common-obj-y += filter.o
>> diff --git a/net/filter.c b/net/filter.c
>> new file mode 100644
>> index 0000000..3b810c8
>> --- /dev/null
>> +++ b/net/filter.c
>> @@ -0,0 +1,138 @@
>> +/*
>> + * Copyright (c) 2015 FUJITSU LIMITED
>> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>> + * later.  See the COPYING file in the top-level directory.
>> + */
>> +
>> +#include "qemu-common.h"
>> +#include "qapi/qmp/qerror.h"
>> +#include "qemu/error-report.h"
>> +
>> +#include "net/filter.h"
>> +#include "net/net.h"
>> +#include "net/vhost_net.h"
>> +#include "qom/object_interfaces.h"
>> +
>> +static char *netfilter_get_netdev_id(Object *obj, Error **errp)
>> +{
>> +    NetFilterState *nf = NETFILTER(obj);
>> +
>> +    return g_strdup(nf->netdev_id);
>> +}
>> +
>> +static void netfilter_set_netdev_id(Object *obj, const char *str, Error **errp)
>> +{
>> +    NetFilterState *nf = NETFILTER(obj);
>> +
>> +    nf->netdev_id = g_strdup(str);
>> +}
>> +
>> +static int netfilter_get_chain(Object *obj, Error **errp G_GNUC_UNUSED)
>> +{
>> +    NetFilterState *nf = NETFILTER(obj);
>> +    return nf->chain;
>> +}
>> +
>> +static void netfilter_set_chain(Object *obj, int chain, Error **errp)
>> +{
>> +    NetFilterState *nf = NETFILTER(obj);
>> +    nf->chain = chain;
>> +}
>> +
>> +static void netfilter_init(Object *obj)
>> +{
>> +    object_property_add_str(obj, "netdev",
>> +                            netfilter_get_netdev_id, netfilter_set_netdev_id,
>> +                            NULL);
>> +    object_property_add_enum(obj, "chain", "NetFilterChain",
>> +                             NetFilterChain_lookup,
>> +                             netfilter_get_chain, netfilter_set_chain,
>> +                             NULL);
>> +}
>> +
>> +static void netfilter_finalize(Object *obj)
>> +{
>> +    NetFilterState *nf = NETFILTER(obj);
>> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(obj);
>> +
>> +    if (nfc->cleanup) {
>
> Is ->cleanup optional?
>
>> +        nfc->cleanup(nf);
>> +    }
>> +
>> +    if (nf->netdev && !QTAILQ_EMPTY(&nf->netdev->filters)) {
>
> How can nf->netdev be null?
>
> How can nf->netdev->filters be empty?
>
>> +        QTAILQ_REMOVE(&nf->netdev->filters, nf, next);
>> +    }
>> +}
>
> Putting netfilter_finalize() after netfilter_complete() would be easier
> to understand, because then destruction follows creation.
>
>> +
>> +static void netfilter_complete(UserCreatable *uc, Error **errp)
>> +{
>> +    NetFilterState *nf = NETFILTER(uc);
>> +    NetClientState *ncs[MAX_QUEUE_NUM];
>> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc);
>> +    int queues;
>> +    Error *local_err = NULL;
>> +
>> +    if (!nf->netdev_id) {
>> +        error_setg(errp, "Parameter 'netdev' is required");
>> +        return;
>> +    }
>> +
>> +    queues = qemu_find_net_clients_except(nf->netdev_id, ncs,
>> +                                          NET_CLIENT_OPTIONS_KIND_NIC,
>> +                                          MAX_QUEUE_NUM);
>> +    if (queues < 1) {
>> +        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "netdev",
>> +                   "a network backend id");
>> +        return;
>> +    } else if (queues > 1) {
>> +        error_setg(errp, "Multi queue is not supported");
>
> We spell this multiqueue elsewhere.
>
> Since you're only interested in a single queue, you could save stack
> space by making ncs[] just one element large.  Not worth it if you
> intend to support multiqueue soon.
>
>> +        return;
>> +    }
>> +
>> +    if (get_vhost_net(ncs[0])) {
>> +        error_setg(errp, "Vhost is not supported");
>> +        return;
>> +    }
>> +
>> +    nf->netdev = ncs[0];
>> +
>> +    if (nfc->setup) {
>
> Is ->setup optional?
>
>> +        nfc->setup(nf, &local_err);
>> +        if (local_err) {
>> +            error_propagate(errp, local_err);
>> +            return;
>> +        }
>> +    }
>> +    QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);
>
> I think I'd create netdev functions to add and remove filters, so the
> filter code doesn't have to modify the netdev state.  Right now,
> NetClientState member filters is initialized in net.c, but modified
> here.  But I'm not the net maintainer :)
>
>> +}
>> +
>> +static void netfilter_class_init(ObjectClass *oc, void *data)
>> +{
>> +    UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
>> +
>> +    ucc->complete = netfilter_complete;
>> +}
>> +
>> +static const TypeInfo netfilter_info = {
>> +    .name = TYPE_NETFILTER,
>> +    .parent = TYPE_OBJECT,
>> +    .abstract = true,
>> +    .class_size = sizeof(NetFilterClass),
>> +    .class_init = netfilter_class_init,
>> +    .instance_size = sizeof(NetFilterState),
>> +    .instance_init = netfilter_init,
>> +    .instance_finalize = netfilter_finalize,
>> +    .interfaces = (InterfaceInfo[]) {
>> +        { TYPE_USER_CREATABLE },
>> +        { }
>> +    }
>> +};
>> +
>> +static void register_types(void)
>> +{
>> +    type_register_static(&netfilter_info);
>> +}
>> +
>> +type_init(register_types);
>> diff --git a/net/net.c b/net/net.c
>> index 28a5597..033f4f3 100644
>> --- a/net/net.c
>> +++ b/net/net.c
>> @@ -44,6 +44,7 @@
>>   #include "qapi/opts-visitor.h"
>>   #include "qapi/dealloc-visitor.h"
>>   #include "sysemu/sysemu.h"
>> +#include "net/filter.h"
>>
>>   /* Net bridge is currently not supported for W32. */
>>   #if !defined(_WIN32)
>> @@ -287,6 +288,7 @@ static void qemu_net_client_setup(NetClientState *nc,
>>
>>       nc->incoming_queue = qemu_new_net_queue(nc);
>>       nc->destructor = destructor;
>> +    QTAILQ_INIT(&nc->filters);
>>   }
>>
>>   NetClientState *qemu_new_net_client(NetClientInfo *info,
>> @@ -384,6 +386,7 @@ void qemu_del_net_client(NetClientState *nc)
>>   {
>>       NetClientState *ncs[MAX_QUEUE_NUM];
>>       int queues, i;
>> +    NetFilterState *nf, *next;
>>
>>       assert(nc->info->type != NET_CLIENT_OPTIONS_KIND_NIC);
>>
>> @@ -395,6 +398,10 @@ void qemu_del_net_client(NetClientState *nc)
>>                                             MAX_QUEUE_NUM);
>>       assert(queues != 0);
>>
>> +    QTAILQ_FOREACH_SAFE(nf, &nc->filters, next, next) {
>> +        object_unparent(OBJECT(nf));
>> +    }
>> +
>>       /* If there is a peer NIC, delete and cleanup client, but do not free. */
>>       if (nc->peer && nc->peer->info->type == NET_CLIENT_OPTIONS_KIND_NIC) {
>>           NICState *nic = qemu_get_nic(nc->peer);
>> diff --git a/qapi-schema.json b/qapi-schema.json
>> index 2bada60..546500a 100644
>> --- a/qapi-schema.json
>> +++ b/qapi-schema.json
>> @@ -2551,6 +2551,24 @@
>>       'opts': 'NetClientOptions' } }
>>
>>   ##
>> +# @NetFilterChain
>> +#
>> +# netfilter chain, a netfilter is attached to a netdev, captures the
>> +# network packets of the netdev.
>> +#
>> +# @all: the filter will receive packets both sent to/from the netdev, this
>> +#       is the default chain.
>> +#
>> +# @in: the filter will receive packets sent to the netdev.
>> +#
>> +# @out: the filter will receive packets sent from the netdev.
>
> Uh, inhowfar is this a "chain"?  As far as I can tell, it specifies
> whether a netfilter is attached to the transmit queue, the receive
> queue, or both.
>
>> +#
>> +# Since 2.5
>> +##
>> +{ 'enum': 'NetFilterChain',
>> +  'data': [ 'all', 'in', 'out' ] }
>> +
>> +##
>>   # @InetSocketAddress
>>   #
>>   # Captures a socket address or address range in the Internet namespace.
>> diff --git a/vl.c b/vl.c
>> index 066a080..ec589e2 100644
>> --- a/vl.c
>> +++ b/vl.c
>> @@ -2794,6 +2794,7 @@ static bool object_create_initial(const char *type)
>>       if (g_str_equal(type, "rng-egd")) {
>>           return false;
>>       }
>> +    /* TODO: return false for concrete netfilters */
>
> This confused me briefly, until I realized the context.  You mean
> something like
>
>      /* TODO implement netfilters */
>
>>       return true;
>>   }
>>
>> @@ -4313,12 +4314,6 @@ int main(int argc, char **argv, char **envp)
>>           exit(0);
>>       }
>>
>> -    if (qemu_opts_foreach(qemu_find_opts("object"),
>> -                          object_create,
>> -                          object_create_delayed, NULL)) {
>> -        exit(1);
>> -    }
>> -
>>       machine_opts = qemu_get_machine_opts();
>>       if (qemu_opt_foreach(machine_opts, machine_set_property, current_machine,
>>                            NULL)) {
>> @@ -4424,6 +4419,12 @@ int main(int argc, char **argv, char **envp)
>>           exit(1);
>>       }
>>
>> +    if (qemu_opts_foreach(qemu_find_opts("object"),
>> +                          object_create,
>> +                          object_create_delayed, NULL)) {
>> +        exit(1);
>> +    }
>> +
>>   #ifdef CONFIG_TPM
>>       if (tpm_init() < 0) {
>>           exit(1);
>
> I'd make this code motion a separate patch, to make it more prominent in
> git-log.
>
> The rest of the patch is harmless: it creates a new abstact class.
>
> The code motion isn't: it actually changes behavior.
>
> Overall, patch is neatly done.
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Yang Hongyang]

On 09/24/2015 04:41 PM, Markus Armbruster wrote:
> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>
>> Add a netfilter object based on QOM.
>>
>> A netfilter is attached to a netdev, captures all network packets
>> that pass through the netdev. When we delete the netdev, we also
>> delete the netfilter object attached to it, because if the netdev is
>> removed, the filter which attached to it is useless.
>>
>> QTAILQ_ENTRY next used by netdev, filter belongs to the specific netdev is
>> in this queue.
>
> I don't get this paragraph.  Not sure it's needed.
>
>> Also init delayed object after net_init_clients, because netfilters need
>> to be initialized after net clients initialized.
>
> A paragraph starting with "Also" in a commit message is a pretty good
> sign the patch should be split :)
>
>>
>> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
>> ---
>> v11: no need to free nf->netdev_id, it will be auto freeed while object deleted
>>       remove global_list net_filters, will add back when needed
>> v10: use QOM for netfilter
>> v9: use flat union instead of simple union in QAPI schema
>> v8: include vhost_net header
>> v7: add check for vhost
>>      fix error propagate bug
>> v6: add multiqueue support (net_filter_init1)
>> v5: remove model from NetFilterState
>>      add a sent_cb param to receive_iov API
>> ---
>>   include/net/filter.h    |  60 +++++++++++++++++++++
>>   include/net/net.h       |   1 +
>>   include/qemu/typedefs.h |   1 +
>>   net/Makefile.objs       |   1 +
>>   net/filter.c            | 138 ++++++++++++++++++++++++++++++++++++++++++++++++
>>   net/net.c               |   7 +++
>>   qapi-schema.json        |  18 +++++++
>>   vl.c                    |  13 ++---
>>   8 files changed, 233 insertions(+), 6 deletions(-)
>>   create mode 100644 include/net/filter.h
>>   create mode 100644 net/filter.c
>>
>> diff --git a/include/net/filter.h b/include/net/filter.h
>> new file mode 100644
>> index 0000000..226f2f7
>> --- /dev/null
>> +++ b/include/net/filter.h
>> @@ -0,0 +1,60 @@
>> +/*
>> + * Copyright (c) 2015 FUJITSU LIMITED
>> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>> + * later.  See the COPYING file in the top-level directory.
>> + */
>> +
>> +#ifndef QEMU_NET_FILTER_H
>> +#define QEMU_NET_FILTER_H
>> +
>> +#include "qom/object.h"
>> +#include "qemu-common.h"
>> +#include "qemu/typedefs.h"
>> +#include "net/queue.h"
>> +
>> +#define TYPE_NETFILTER "netfilter"
>> +#define NETFILTER(obj) \
>> +    OBJECT_CHECK(NetFilterState, (obj), TYPE_NETFILTER)
>> +#define NETFILTER_GET_CLASS(obj) \
>> +    OBJECT_GET_CLASS(NetFilterClass, (obj), TYPE_NETFILTER)
>> +#define NETFILTER_CLASS(klass) \
>> +    OBJECT_CLASS_CHECK(NetFilterClass, (klass), TYPE_NETFILTER)
>> +
>> +typedef void (FilterSetup) (NetFilterState *nf, Error **errp);
>> +typedef void (FilterCleanup) (NetFilterState *nf);
>> +/*
>> + * Return:
>> + *   0: finished handling the packet, we should continue
>> + *   size: filter stolen this packet, we stop pass this packet further
>> + */
>> +typedef ssize_t (FilterReceiveIOV)(NetFilterState *nc,
>> +                                   NetClientState *sender,
>> +                                   unsigned flags,
>> +                                   const struct iovec *iov,
>> +                                   int iovcnt,
>> +                                   NetPacketSent *sent_cb);
>> +
>> +struct NetFilterClass {
>> +    ObjectClass parent_class;
>> +
>> +    FilterSetup *setup;
>> +    FilterCleanup *cleanup;
>> +    FilterReceiveIOV *receive_iov;
>> +};
>> +typedef struct NetFilterClass NetFilterClass;
>
> Not splitting the declaration is more concise:
>
>      typedef struct {
>          ObjectClass parent_class;
>          FilterSetup *setup;
>          FilterCleanup *cleanup;
>          FilterReceiveIOV *receive_iov;
>      } NetFilterClass;
>
> Are any of the methods optional?  If yes, please add suitable comments.
>
>> +
>> +
>> +struct NetFilterState {
>> +    /* private */
>> +    Object parent;
>> +
>> +    /* protected */
>> +    char *netdev_id;
>> +    NetClientState *netdev;
>> +    NetFilterChain chain;
>> +    QTAILQ_ENTRY(NetFilterState) next;
>> +};
>> +
>> +#endif /* QEMU_NET_FILTER_H */
>> diff --git a/include/net/net.h b/include/net/net.h
>> index 6a6cbef..36e5fab 100644
>> --- a/include/net/net.h
>> +++ b/include/net/net.h
>> @@ -92,6 +92,7 @@ struct NetClientState {
>>       NetClientDestructor *destructor;
>>       unsigned int queue_index;
>>       unsigned rxfilter_notify_enabled:1;
>> +    QTAILQ_HEAD(, NetFilterState) filters;
>>   };
>>
>>   typedef struct NICState {
>> diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
>> index f8a9dd6..2c0648f 100644
>> --- a/include/qemu/typedefs.h
>> +++ b/include/qemu/typedefs.h
>> @@ -45,6 +45,7 @@ typedef struct Monitor Monitor;
>>   typedef struct MouseTransformInfo MouseTransformInfo;
>>   typedef struct MSIMessage MSIMessage;
>>   typedef struct NetClientState NetClientState;
>> +typedef struct NetFilterState NetFilterState;
>>   typedef struct NICInfo NICInfo;
>>   typedef struct PcGuestInfo PcGuestInfo;
>>   typedef struct PCIBridge PCIBridge;
>> diff --git a/net/Makefile.objs b/net/Makefile.objs
>> index ec19cb3..914aec0 100644
>> --- a/net/Makefile.objs
>> +++ b/net/Makefile.objs
>> @@ -13,3 +13,4 @@ common-obj-$(CONFIG_HAIKU) += tap-haiku.o
>>   common-obj-$(CONFIG_SLIRP) += slirp.o
>>   common-obj-$(CONFIG_VDE) += vde.o
>>   common-obj-$(CONFIG_NETMAP) += netmap.o
>> +common-obj-y += filter.o
>> diff --git a/net/filter.c b/net/filter.c
>> new file mode 100644
>> index 0000000..3b810c8
>> --- /dev/null
>> +++ b/net/filter.c
>> @@ -0,0 +1,138 @@
>> +/*
>> + * Copyright (c) 2015 FUJITSU LIMITED
>> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>> + * later.  See the COPYING file in the top-level directory.
>> + */
>> +
>> +#include "qemu-common.h"
>> +#include "qapi/qmp/qerror.h"
>> +#include "qemu/error-report.h"
>> +
>> +#include "net/filter.h"
>> +#include "net/net.h"
>> +#include "net/vhost_net.h"
>> +#include "qom/object_interfaces.h"
>> +
>> +static char *netfilter_get_netdev_id(Object *obj, Error **errp)
>> +{
>> +    NetFilterState *nf = NETFILTER(obj);
>> +
>> +    return g_strdup(nf->netdev_id);
>> +}
>> +
>> +static void netfilter_set_netdev_id(Object *obj, const char *str, Error **errp)
>> +{
>> +    NetFilterState *nf = NETFILTER(obj);
>> +
>> +    nf->netdev_id = g_strdup(str);
>> +}
>> +
>> +static int netfilter_get_chain(Object *obj, Error **errp G_GNUC_UNUSED)
>> +{
>> +    NetFilterState *nf = NETFILTER(obj);
>> +    return nf->chain;
>> +}
>> +
>> +static void netfilter_set_chain(Object *obj, int chain, Error **errp)
>> +{
>> +    NetFilterState *nf = NETFILTER(obj);
>> +    nf->chain = chain;
>> +}
>> +
>> +static void netfilter_init(Object *obj)
>> +{
>> +    object_property_add_str(obj, "netdev",
>> +                            netfilter_get_netdev_id, netfilter_set_netdev_id,
>> +                            NULL);
>> +    object_property_add_enum(obj, "chain", "NetFilterChain",
>> +                             NetFilterChain_lookup,
>> +                             netfilter_get_chain, netfilter_set_chain,
>> +                             NULL);
>> +}
>> +
>> +static void netfilter_finalize(Object *obj)
>> +{
>> +    NetFilterState *nf = NETFILTER(obj);
>> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(obj);
>> +
>> +    if (nfc->cleanup) {
>
> Is ->cleanup optional?

Yes, it's optional, as well as setup, if a concrete filter do
not have things to setup/cleanup.

>
>> +        nfc->cleanup(nf);
>> +    }
>> +
>> +    if (nf->netdev && !QTAILQ_EMPTY(&nf->netdev->filters)) {
>
> How can nf->netdev be null?
>
> How can nf->netdev->filters be empty?

When object initialize failed. for example, in netfilter_complete
no nf->netdev_id provided, object initialize will fail, and the
netfilter_finalize will be called, at the mean time,
nf->netdev is null, and nf->netdev->filters is empty.
>
>> +        QTAILQ_REMOVE(&nf->netdev->filters, nf, next);
>> +    }
>> +}
>
> Putting netfilter_finalize() after netfilter_complete() would be easier
> to understand, because then destruction follows creation.
>
>> +
>> +static void netfilter_complete(UserCreatable *uc, Error **errp)
>> +{
>> +    NetFilterState *nf = NETFILTER(uc);
>> +    NetClientState *ncs[MAX_QUEUE_NUM];
>> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc);
>> +    int queues;
>> +    Error *local_err = NULL;
>> +
>> +    if (!nf->netdev_id) {
>> +        error_setg(errp, "Parameter 'netdev' is required");
>> +        return;
>> +    }
>> +
>> +    queues = qemu_find_net_clients_except(nf->netdev_id, ncs,
>> +                                          NET_CLIENT_OPTIONS_KIND_NIC,
>> +                                          MAX_QUEUE_NUM);
>> +    if (queues < 1) {
>> +        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "netdev",
>> +                   "a network backend id");
>> +        return;
>> +    } else if (queues > 1) {
>> +        error_setg(errp, "Multi queue is not supported");
>
> We spell this multiqueue elsewhere.
>
> Since you're only interested in a single queue, you could save stack
> space by making ncs[] just one element large.  Not worth it if you
> intend to support multiqueue soon.

We will support multiqueue soon.

>
>> +        return;
>> +    }
>> +
>> +    if (get_vhost_net(ncs[0])) {
>> +        error_setg(errp, "Vhost is not supported");
>> +        return;
>> +    }
>> +
>> +    nf->netdev = ncs[0];
>> +
>> +    if (nfc->setup) {
>
> Is ->setup optional?

Yes, as said earlier.

>
>> +        nfc->setup(nf, &local_err);
>> +        if (local_err) {
>> +            error_propagate(errp, local_err);
>> +            return;
>> +        }
>> +    }
>> +    QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);
>
> I think I'd create netdev functions to add and remove filters, so the
> filter code doesn't have to modify the netdev state.  Right now,
> NetClientState member filters is initialized in net.c, but modified
> here.  But I'm not the net maintainer :)
>
>> +}
>> +
>> +static void netfilter_class_init(ObjectClass *oc, void *data)
>> +{
>> +    UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
>> +
>> +    ucc->complete = netfilter_complete;
>> +}
>> +
>> +static const TypeInfo netfilter_info = {
>> +    .name = TYPE_NETFILTER,
>> +    .parent = TYPE_OBJECT,
>> +    .abstract = true,
>> +    .class_size = sizeof(NetFilterClass),
>> +    .class_init = netfilter_class_init,
>> +    .instance_size = sizeof(NetFilterState),
>> +    .instance_init = netfilter_init,
>> +    .instance_finalize = netfilter_finalize,
>> +    .interfaces = (InterfaceInfo[]) {
>> +        { TYPE_USER_CREATABLE },
>> +        { }
>> +    }
>> +};
>> +
>> +static void register_types(void)
>> +{
>> +    type_register_static(&netfilter_info);
>> +}
>> +
>> +type_init(register_types);
>> diff --git a/net/net.c b/net/net.c
>> index 28a5597..033f4f3 100644
>> --- a/net/net.c
>> +++ b/net/net.c
>> @@ -44,6 +44,7 @@
>>   #include "qapi/opts-visitor.h"
>>   #include "qapi/dealloc-visitor.h"
>>   #include "sysemu/sysemu.h"
>> +#include "net/filter.h"
>>
>>   /* Net bridge is currently not supported for W32. */
>>   #if !defined(_WIN32)
>> @@ -287,6 +288,7 @@ static void qemu_net_client_setup(NetClientState *nc,
>>
>>       nc->incoming_queue = qemu_new_net_queue(nc);
>>       nc->destructor = destructor;
>> +    QTAILQ_INIT(&nc->filters);
>>   }
>>
>>   NetClientState *qemu_new_net_client(NetClientInfo *info,
>> @@ -384,6 +386,7 @@ void qemu_del_net_client(NetClientState *nc)
>>   {
>>       NetClientState *ncs[MAX_QUEUE_NUM];
>>       int queues, i;
>> +    NetFilterState *nf, *next;
>>
>>       assert(nc->info->type != NET_CLIENT_OPTIONS_KIND_NIC);
>>
>> @@ -395,6 +398,10 @@ void qemu_del_net_client(NetClientState *nc)
>>                                             MAX_QUEUE_NUM);
>>       assert(queues != 0);
>>
>> +    QTAILQ_FOREACH_SAFE(nf, &nc->filters, next, next) {
>> +        object_unparent(OBJECT(nf));
>> +    }
>> +
>>       /* If there is a peer NIC, delete and cleanup client, but do not free. */
>>       if (nc->peer && nc->peer->info->type == NET_CLIENT_OPTIONS_KIND_NIC) {
>>           NICState *nic = qemu_get_nic(nc->peer);
>> diff --git a/qapi-schema.json b/qapi-schema.json
>> index 2bada60..546500a 100644
>> --- a/qapi-schema.json
>> +++ b/qapi-schema.json
>> @@ -2551,6 +2551,24 @@
>>       'opts': 'NetClientOptions' } }
>>
>>   ##
>> +# @NetFilterChain
>> +#
>> +# netfilter chain, a netfilter is attached to a netdev, captures the
>> +# network packets of the netdev.
>> +#
>> +# @all: the filter will receive packets both sent to/from the netdev, this
>> +#       is the default chain.
>> +#
>> +# @in: the filter will receive packets sent to the netdev.
>> +#
>> +# @out: the filter will receive packets sent from the netdev.
>
> Uh, inhowfar is this a "chain"?  As far as I can tell, it specifies
> whether a netfilter is attached to the transmit queue, the receive
> queue, or both.

netback's input chain or output chain, queue is mostly like a word that
related to the code implementation? English is not my first language, so
I'm not sure about this.

>
>> +#
>> +# Since 2.5
>> +##
>> +{ 'enum': 'NetFilterChain',
>> +  'data': [ 'all', 'in', 'out' ] }
>> +
>> +##
>>   # @InetSocketAddress
>>   #
>>   # Captures a socket address or address range in the Internet namespace.
>> diff --git a/vl.c b/vl.c
>> index 066a080..ec589e2 100644
>> --- a/vl.c
>> +++ b/vl.c
>> @@ -2794,6 +2794,7 @@ static bool object_create_initial(const char *type)
>>       if (g_str_equal(type, "rng-egd")) {
>>           return false;
>>       }
>> +    /* TODO: return false for concrete netfilters */
>
> This confused me briefly, until I realized the context.  You mean
> something like

Ah, I'm sorry for the confusing...

>
>      /* TODO implement netfilters */
>
>>       return true;
>>   }
>>
>> @@ -4313,12 +4314,6 @@ int main(int argc, char **argv, char **envp)
>>           exit(0);
>>       }
>>
>> -    if (qemu_opts_foreach(qemu_find_opts("object"),
>> -                          object_create,
>> -                          object_create_delayed, NULL)) {
>> -        exit(1);
>> -    }
>> -
>>       machine_opts = qemu_get_machine_opts();
>>       if (qemu_opt_foreach(machine_opts, machine_set_property, current_machine,
>>                            NULL)) {
>> @@ -4424,6 +4419,12 @@ int main(int argc, char **argv, char **envp)
>>           exit(1);
>>       }
>>
>> +    if (qemu_opts_foreach(qemu_find_opts("object"),
>> +                          object_create,
>> +                          object_create_delayed, NULL)) {
>> +        exit(1);
>> +    }
>> +
>>   #ifdef CONFIG_TPM
>>       if (tpm_init() < 0) {
>>           exit(1);
>
> I'd make this code motion a separate patch, to make it more prominent in
> git-log.
>
> The rest of the patch is harmless: it creates a new abstact class.
>
> The code motion isn't: it actually changes behavior.
>
> Overall, patch is neatly done.
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 09/12] netfilter: add a netbuffer filter

[Markus Armbruster]

Yang Hongyang <yanghy@cn.fujitsu.com> writes:

> This filter is to buffer/release packets, this feature can be used
> when using MicroCheckpointing, or other Remus like VM FT solutions, you

What's "Remus"?

> can also use it to simulate the network delay.
> It has an interval option, if supplied, this filter will release
> packets by interval.

Suggest "will delay packets by that time interval."

Is interval really optional?

>
> Usage:
>  -netdev tap,id=bn0
>  -object filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000
>
> NOTE:
>  the scale of interval is microsecond.

Perhaps "interval is in microseconds".

>
> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
> ---
> v11: add a fixme comment from Jason
> v10: use NetQueue flush api to flush packets
>      sent_cb can not be called when we already return size
> v9: adjustment due to the qapi change
> v7: use QTAILQ_FOREACH_SAFE() when flush packets
> v6: move the interval check earlier and some comment adjust
> v5: remove dummy sent_cb
>     change interval type from int64 to uint32
>     check interval!=0 when initialise
>     rename FILTERBUFFERState to FilterBufferState
> v4: remove bh
>     pass the packet to next filter instead of receiver
> v3: check packet's sender and sender->peer when flush it
> ---
>  net/Makefile.objs   |   1 +
>  net/filter-buffer.c | 170 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>  qemu-options.hx     |  18 ++++++
>  vl.c                |   7 ++-
>  4 files changed, 195 insertions(+), 1 deletion(-)
>  create mode 100644 net/filter-buffer.c
>
> diff --git a/net/Makefile.objs b/net/Makefile.objs
> index 914aec0..5fa2f97 100644
> --- a/net/Makefile.objs
> +++ b/net/Makefile.objs
> @@ -14,3 +14,4 @@ common-obj-$(CONFIG_SLIRP) += slirp.o
>  common-obj-$(CONFIG_VDE) += vde.o
>  common-obj-$(CONFIG_NETMAP) += netmap.o
>  common-obj-y += filter.o
> +common-obj-y += filter-buffer.o
> diff --git a/net/filter-buffer.c b/net/filter-buffer.c
> new file mode 100644
> index 0000000..ef94e91
> --- /dev/null
> +++ b/net/filter-buffer.c
> @@ -0,0 +1,170 @@
> +/*
> + * Copyright (c) 2015 FUJITSU LIMITED
> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or
> + * later.  See the COPYING file in the top-level directory.
> + */
> +
> +#include "net/filter.h"
> +#include "net/queue.h"
> +#include "qemu-common.h"
> +#include "qemu/timer.h"
> +#include "qemu/iov.h"
> +#include "qapi/qmp/qerror.h"
> +#include "qapi-visit.h"
> +#include "qom/object.h"
> +
> +#define TYPE_FILTER_BUFFER "filter-buffer"
> +
> +#define FILTER_BUFFER(obj) \
> +    OBJECT_CHECK(FilterBufferState, (obj), TYPE_FILTER_BUFFER)
> +
> +struct FilterBufferState {
> +    NetFilterState parent_obj;
> +
> +    NetQueue *incoming_queue;
> +    uint32_t interval;
> +    QEMUTimer release_timer;
> +};
> +typedef struct FilterBufferState FilterBufferState;

Again, not splitting the declaration is more concise.

> +
> +static void filter_buffer_flush(NetFilterState *nf)
> +{
> +    FilterBufferState *s = FILTER_BUFFER(nf);
> +
> +    if (!qemu_net_queue_flush(s->incoming_queue)) {
> +        /* Unable to empty the queue, purge remaining packets */
> +        qemu_net_queue_purge(s->incoming_queue, nf->netdev);
> +    }
> +}

This either flushes or purges incoming_queue, where "purge" means
dropping packets.  Correct?

> +
> +static void filter_buffer_release_timer(void *opaque)
> +{
> +    NetFilterState *nf = opaque;
> +    FilterBufferState *s = FILTER_BUFFER(nf);

Style nit: blank line between declarations and statements, please.

> +    filter_buffer_flush(nf);

Is purging correct here?

> +    timer_mod(&s->release_timer,
> +              qemu_clock_get_us(QEMU_CLOCK_VIRTUAL) + s->interval);

Timer rearmed to fire again in s->interval microseconds.

> +}
> +
> +/* filter APIs */
> +static ssize_t filter_buffer_receive_iov(NetFilterState *nf,
> +                                         NetClientState *sender,
> +                                         unsigned flags,
> +                                         const struct iovec *iov,
> +                                         int iovcnt,
> +                                         NetPacketSent *sent_cb)
> +{
> +    FilterBufferState *s = FILTER_BUFFER(nf);
> +
> +    /*
> +     * we return size when buffer a packet, the sender will take it as
> +     * a already sent packet, so sent_cb should not be called later

Humor me: when a comment has multiple sentences, start each one with a
capital letter, and end it with punctuation.

> +     * FIXME: even if guest can't receive packet for some reasons. Filter
> +     * can still accept packet until its internal queue is full.
> +     */

I'm not sure I understand the comment.

> +    qemu_net_queue_append_iov(s->incoming_queue, sender, flags,
> +                              iov, iovcnt, NULL);
> +    return iov_size(iov, iovcnt);
> +}
> +
> +static void filter_buffer_cleanup(NetFilterState *nf)
> +{
> +    FilterBufferState *s = FILTER_BUFFER(nf);
> +
> +    if (s->interval) {
> +        timer_del(&s->release_timer);
> +    }
> +
> +    /* flush packets */
> +    if (s->incoming_queue) {
> +        filter_buffer_flush(nf);

I guess purging is correct here.

> +        g_free(s->incoming_queue);
> +    }
> +}
> +
> +static void filter_buffer_setup(NetFilterState *nf, Error **errp)
> +{
> +    FilterBufferState *s = FILTER_BUFFER(nf);
> +
> +    /*
> +     * this check should be dropped when there're VM FT solutions like MC
> +     * or COLO use this filter to release packets on demand.
> +     */

If you end a sentence with a period, you get to start it with a capital
letter :)

"there're" is odd.  Perhaps something like "We may want to accept zero
interval when VM FT solutions like MC or COLO use this filter to release
packets on demand."

> +    if (!s->interval) {
> +        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "interval",
> +                   "a non-zero interval");

How can this happen?  Doesn't filter_buffer_set_interval() catch zero
intervals already?

> +        return;
> +    }
> +
> +    s->incoming_queue = qemu_new_net_queue(qemu_netfilter_pass_to_next, nf);
> +    if (s->interval) {

Condition cannot be false.  Same in filter_buffer_cleanup().

> +        timer_init_us(&s->release_timer, QEMU_CLOCK_VIRTUAL,
> +                      filter_buffer_release_timer, nf);
> +        timer_mod(&s->release_timer,
> +                  qemu_clock_get_us(QEMU_CLOCK_VIRTUAL) + s->interval);

Timer armed to fire in s->interval microseconds.

Unless I misunderstand something, this doesn't actually delay each
packet by s->interval microseconds, it batches packet delivery: all
packets arriving in a given interval are delayed until the end of the
interval.  Correct?

> +    }
> +}
> +
> +static void filter_buffer_class_init(ObjectClass *oc, void *data)
> +{
> +    NetFilterClass *nfc = NETFILTER_CLASS(oc);
> +
> +    nfc->setup = filter_buffer_setup;
> +    nfc->cleanup = filter_buffer_cleanup;
> +    nfc->receive_iov = filter_buffer_receive_iov;
> +}
> +
> +static void filter_buffer_get_interval(Object *obj, Visitor *v, void *opaque,
> +                                       const char *name, Error **errp)
> +{
> +    FilterBufferState *s = FILTER_BUFFER(obj);
> +    uint32_t value = s->interval;
> +
> +    visit_type_uint32(v, &value, name, errp);
> +}
> +
> +static void filter_buffer_set_interval(Object *obj, Visitor *v, void *opaque,
> +                                       const char *name, Error **errp)
> +{
> +    FilterBufferState *s = FILTER_BUFFER(obj);
> +    Error *local_err = NULL;
> +    uint32_t value;
> +
> +    visit_type_uint32(v, &value, name, &local_err);
> +    if (local_err) {
> +        goto out;
> +    }
> +    if (!value) {
> +        error_setg(&local_err, "Property '%s.%s' doesn't take value '%"
> +                   PRIu32 "'", object_get_typename(obj), name, value);

What does it take?  That's what the user wants to know...  Perhaps
"Property '%s.%s' requires a positive value".

> +        goto out;
> +    }
> +    s->interval = value;
> +
> +out:
> +    error_propagate(errp, local_err);
> +}
> +
> +static void filter_buffer_init(Object *obj)
> +{
> +    object_property_add(obj, "interval", "int",
> +                        filter_buffer_get_interval,
> +                        filter_buffer_set_interval, NULL, NULL, NULL);
> +}
> +
> +static const TypeInfo filter_buffer_info = {
> +    .name = TYPE_FILTER_BUFFER,
> +    .parent = TYPE_NETFILTER,
> +    .class_init = filter_buffer_class_init,
> +    .instance_init = filter_buffer_init,
> +    .instance_size = sizeof(FilterBufferState),
> +};
> +
> +static void register_types(void)
> +{
> +    type_register_static(&filter_buffer_info);
> +}
> +
> +type_init(register_types);
> diff --git a/qemu-options.hx b/qemu-options.hx
> index 7e147b8..b09f97f 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -3642,6 +3642,24 @@ in PEM format, in filenames @var{ca-cert.pem}, @var{ca-crl.pem} (optional),
>  @var{server-cert.pem} (only servers), @var{server-key.pem} (only servers),
>  @var{client-cert.pem} (only clients), and @var{client-key.pem} (only clients).
>  
> +@item -object filter-buffer,id=@var{id},netdev=@var{netdevid}[,chain=@var{all|in|out}][,interval=@var{t}]
> +
> +Buffer network packets on netdev @var{netdevid}.
> +If interval @var{t} provided, will release packets by interval.
> +Interval scale: microsecond.
> +
> +If interval @var{t} not provided, you have to make sure the packets can be

Are you sure omitting t works?  filter_buffer_set_interval() rejects
zero...

> +released, either by manually remove this filter or call the release buffer API,
> +otherwise, the packets will be buffered forever. Use with caution.

Please wrap your lines a bit earlier.

> +
> +chain @var{all|in|out} is an option that can be applied to any netfilter, default is @option{all}.
> +
> +@option{all} means this filter will receive packets both sent to/from the netdev
> +
> +@option{in} means this filter will receive packets sent to the netdev
> +
> +@option{out} means this filter will receive packets sent from the netdev
> +

I'd describe this like "filter is inserted in the receive / transmit
queue".

>  @end table
>  
>  ETEXI
> diff --git a/vl.c b/vl.c
> index ec589e2..3cf89d5 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -2794,7 +2794,12 @@ static bool object_create_initial(const char *type)
>      if (g_str_equal(type, "rng-egd")) {
>          return false;
>      }
> -    /* TODO: return false for concrete netfilters */
> +
> +    /* return false for concrete netfilters */

I find this comment useless, please drop it :)

> +    if (g_str_equal(type, "filter-buffer")) {
> +        return false;
> +    }
> +
>      return true;
>  }

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Markus Armbruster]

Yang Hongyang <yanghy@cn.fujitsu.com> writes:

> On 09/24/2015 03:43 PM, Markus Armbruster wrote:
>> This has finally reached the front of my review queue.  I apologize for
>> the loooong delay.
>>
>> Copying Paolo for another pair of eyeballs (he wrote this code).
>>
> [...]
>>> +
>>> +    opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
>>> +    qemu_opts_del(opts);
>>
>> qemu_find_opts_err("object", &error_abort) please, because when it
>> fails, we want to die right away, not when the null pointer it returns
>> gets dereferenced.
>
> Thanks for the review.
> Jason, do you want me to propose a fix on top of this series or simply drop
> this for now because this patch is an independent bug fix and won't affect the
> other filter patch series.
>
>>
>> Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
>> your patch's fault.
>>
>> Elsewhere, we store the QemuOpts in the object just so we can delete it:
>> DeviceState, DriveInfo.  Paolo, what do you think?
>
> I don't get it. Currently, only objects created at the beginning through
> QEMU command line will be stored in the QemuOpts, objects that created
> with object_add won't stored in QemuOpts. Do you mean for DeviceState,
> DriveInfo they store there QemuOpts explicity so that they can delete it?
> Why don't we just delete it from objects directly instead?

Let me elaborate.

We have the same pattern in multiple places: some kind of object gets
configured via QemuOpts, and an object's QemuOpts need to stay around
until the object dies.

Example 1: Block device backends

    DriveInfo has a member opts.

    drive_new() stores the QemuOpts in dinfo->opts.

    drive_info_del() destroys dinfo->opts.

    Note: DriveInfo member opts is always non-null.  But not every
    BlockBackend has a DriveInfo.

Example 2: Device frontends

    DeviceState has a member opts.

    qdev_device_add() stores the QemuOpts in dev->opts.

    device_finalize() destroys dev->opts.

    Note: DeviceState member opts may be null (not every device is
    created by qdev_device_add()).  Fine, because qemu_opts_del(NULL) is
    a no-op.

Example 3: Character device backends

    CharDriverState has a member opts.

    qemu_chr_new_from_opts() stores the QemuOpts in chr->opts.

    qemu_chr_delete() destroys chr->opts.

Example 4: Network device backends

    Two cases

    A. netdev

       qmp_netdev_add() does not store the QemuOpts.

       qmp_netdev_del() still needs to destroy it.  It has to find it
       somehow.  Here's how it does it:

           opts = qemu_opts_find(qemu_find_opts_err("netdev", NULL), id);
           if (!opts) {
               error_setg(errp, "Device '%s' is not a netdev", id);
               return;
           }

       The !opts condition is a non-obvious way to test "not created
       with -netdev", see commit 645c949.  Note that the commit's claim
       that qemu_opts_del(NULL) crashes is no longer true since commit
       4782183.

    B. Legacy net

       hmp_host_net_add() does not store the QemuOpts.

       hmp_host_net_remove() still needs to destroy it.  I can't see
       where that happens, and I'm not sure it does.

Example 5: Generic object

    object_create() does not store the QemuOpts.

    It still needs to be destroyed along with the object.  It isn't, and
    your patch fixes it.

Personally, I find the technique in example 1-3 easier to understand
than the one in example 4-5.

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Yang Hongyang]

On 09/24/2015 05:42 PM, Markus Armbruster wrote:
> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>
>> On 09/24/2015 03:43 PM, Markus Armbruster wrote:
>>> This has finally reached the front of my review queue.  I apologize for
>>> the loooong delay.
>>>
>>> Copying Paolo for another pair of eyeballs (he wrote this code).
>>>
>> [...]
>>>> +
>>>> +    opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
>>>> +    qemu_opts_del(opts);
>>>
>>> qemu_find_opts_err("object", &error_abort) please, because when it
>>> fails, we want to die right away, not when the null pointer it returns
>>> gets dereferenced.
>>
>> Thanks for the review.
>> Jason, do you want me to propose a fix on top of this series or simply drop
>> this for now because this patch is an independent bug fix and won't affect the
>> other filter patch series.
>>
>>>
>>> Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
>>> your patch's fault.
>>>
>>> Elsewhere, we store the QemuOpts in the object just so we can delete it:
>>> DeviceState, DriveInfo.  Paolo, what do you think?
>>
>> I don't get it. Currently, only objects created at the beginning through
>> QEMU command line will be stored in the QemuOpts, objects that created
>> with object_add won't stored in QemuOpts. Do you mean for DeviceState,
>> DriveInfo they store there QemuOpts explicity so that they can delete it?
>> Why don't we just delete it from objects directly instead?
>
> Let me elaborate.

Thanks very much for the elaboration.

>
> We have the same pattern in multiple places: some kind of object gets
> configured via QemuOpts, and an object's QemuOpts need to stay around
> until the object dies.
>
> Example 1: Block device backends
>
>      DriveInfo has a member opts.
>
>      drive_new() stores the QemuOpts in dinfo->opts.
>
>      drive_info_del() destroys dinfo->opts.
>
>      Note: DriveInfo member opts is always non-null.  But not every
>      BlockBackend has a DriveInfo.
>
> Example 2: Device frontends
>
>      DeviceState has a member opts.
>
>      qdev_device_add() stores the QemuOpts in dev->opts.
>
>      device_finalize() destroys dev->opts.
>
>      Note: DeviceState member opts may be null (not every device is
>      created by qdev_device_add()).  Fine, because qemu_opts_del(NULL) is
>      a no-op.
>
> Example 3: Character device backends
>
>      CharDriverState has a member opts.
>
>      qemu_chr_new_from_opts() stores the QemuOpts in chr->opts.
>
>      qemu_chr_delete() destroys chr->opts.

1-3 store there ops in there own state, not in global ops group right?

>
> Example 4: Network device backends
>
>      Two cases
>
>      A. netdev
>
>         qmp_netdev_add() does not store the QemuOpts.
>
>         qmp_netdev_del() still needs to destroy it.  It has to find it
>         somehow.  Here's how it does it:
>
>             opts = qemu_opts_find(qemu_find_opts_err("netdev", NULL), id);
>             if (!opts) {
>                 error_setg(errp, "Device '%s' is not a netdev", id);
>                 return;
>             }
>
>         The !opts condition is a non-obvious way to test "not created
>         with -netdev", see commit 645c949.  Note that the commit's claim
>         that qemu_opts_del(NULL) crashes is no longer true since commit
>         4782183.
>
>      B. Legacy net
>
>         hmp_host_net_add() does not store the QemuOpts.

I'm afraid it does store the QemuOpts, but not in it's own state.
net/net.c:
1088     qemu_opt_set(opts, "type", device, &error_abort);
This will store the QemuOpts, or am I misunderstood it?

>
>         hmp_host_net_remove() still needs to destroy it.  I can't see
>         where that happens, and I'm not sure it does.
>
> Example 5: Generic object
>
>      object_create() does not store the QemuOpts.
>
>      It still needs to be destroyed along with the object.  It isn't, and
>      your patch fixes it.
>
> Personally, I find the technique in example 1-3 easier to understand
> than the one in example 4-5.

I agree that opts should not be used to determine not created something
while there's case when something created but Opts not stored.

> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Yang Hongyang]

On 09/24/2015 05:42 PM, Markus Armbruster wrote:
> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>
>> On 09/24/2015 03:43 PM, Markus Armbruster wrote:
>>> This has finally reached the front of my review queue.  I apologize for
>>> the loooong delay.
>>>
>>> Copying Paolo for another pair of eyeballs (he wrote this code).
>>>
>> [...]
>>>> +
>>>> +    opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
>>>> +    qemu_opts_del(opts);
>>>
>>> qemu_find_opts_err("object", &error_abort) please, because when it
>>> fails, we want to die right away, not when the null pointer it returns
>>> gets dereferenced.
>>
>> Thanks for the review.
>> Jason, do you want me to propose a fix on top of this series or simply drop
>> this for now because this patch is an independent bug fix and won't affect the
>> other filter patch series.
>>
>>>
>>> Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
>>> your patch's fault.
>>>
>>> Elsewhere, we store the QemuOpts in the object just so we can delete it:
>>> DeviceState, DriveInfo.  Paolo, what do you think?
>>
>> I don't get it. Currently, only objects created at the beginning through
>> QEMU command line will be stored in the QemuOpts, objects that created
>> with object_add won't stored in QemuOpts. Do you mean for DeviceState,
>> DriveInfo they store there QemuOpts explicity so that they can delete it?
>> Why don't we just delete it from objects directly instead?
>
> Let me elaborate.
>
> We have the same pattern in multiple places: some kind of object gets
> configured via QemuOpts, and an object's QemuOpts need to stay around
> until the object dies.
>
> Example 1: Block device backends
>
>      DriveInfo has a member opts.
>
>      drive_new() stores the QemuOpts in dinfo->opts.
>
>      drive_info_del() destroys dinfo->opts.
>
>      Note: DriveInfo member opts is always non-null.  But not every
>      BlockBackend has a DriveInfo.
>
> Example 2: Device frontends
>
>      DeviceState has a member opts.
>
>      qdev_device_add() stores the QemuOpts in dev->opts.
>
>      device_finalize() destroys dev->opts.
>
>      Note: DeviceState member opts may be null (not every device is
>      created by qdev_device_add()).  Fine, because qemu_opts_del(NULL) is
>      a no-op.
>
> Example 3: Character device backends
>
>      CharDriverState has a member opts.
>
>      qemu_chr_new_from_opts() stores the QemuOpts in chr->opts.
>
>      qemu_chr_delete() destroys chr->opts.
>
> Example 4: Network device backends
>
>      Two cases
>
>      A. netdev
>
>         qmp_netdev_add() does not store the QemuOpts.

The QemuOpts stored by qmp_netdev_add() and also hmp_netdev_add().
through this function:
net/net.c: qmp_netdev_add()
1134     opts = qemu_opts_from_qdict(opts_list, qdict, &local_err);

hmp.c: hmp_netdev_add()
1579     opts = qemu_opts_from_qdict(qemu_find_opts("netdev"), qdict, &err);

>
>         qmp_netdev_del() still needs to destroy it.  It has to find it
>         somehow.  Here's how it does it:
>
>             opts = qemu_opts_find(qemu_find_opts_err("netdev", NULL), id);
>             if (!opts) {
>                 error_setg(errp, "Device '%s' is not a netdev", id);
>                 return;
>             }
>
>         The !opts condition is a non-obvious way to test "not created
>         with -netdev", see commit 645c949.  Note that the commit's claim
>         that qemu_opts_del(NULL) crashes is no longer true since commit
>         4782183.
>
>      B. Legacy net
>
>         hmp_host_net_add() does not store the QemuOpts.
>
>         hmp_host_net_remove() still needs to destroy it.  I can't see
>         where that happens, and I'm not sure it does.
>
> Example 5: Generic object
>
>      object_create() does not store the QemuOpts.
>
>      It still needs to be destroyed along with the object.  It isn't, and
>      your patch fixes it.
>
> Personally, I find the technique in example 1-3 easier to understand
> than the one in example 4-5.
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Markus Armbruster]

Yang Hongyang <yanghy@cn.fujitsu.com> writes:

> On 09/24/2015 05:42 PM, Markus Armbruster wrote:
>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>
>>> On 09/24/2015 03:43 PM, Markus Armbruster wrote:
>>>> This has finally reached the front of my review queue.  I apologize for
>>>> the loooong delay.
>>>>
>>>> Copying Paolo for another pair of eyeballs (he wrote this code).
>>>>
>>> [...]
>>>>> +
>>>>> +    opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
>>>>> +    qemu_opts_del(opts);
>>>>
>>>> qemu_find_opts_err("object", &error_abort) please, because when it
>>>> fails, we want to die right away, not when the null pointer it returns
>>>> gets dereferenced.
>>>
>>> Thanks for the review.
>>> Jason, do you want me to propose a fix on top of this series or simply drop
>>> this for now because this patch is an independent bug fix and won't
>>> affect the
>>> other filter patch series.
>>>
>>>>
>>>> Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
>>>> your patch's fault.
>>>>
>>>> Elsewhere, we store the QemuOpts in the object just so we can delete it:
>>>> DeviceState, DriveInfo.  Paolo, what do you think?
>>>
>>> I don't get it. Currently, only objects created at the beginning through
>>> QEMU command line will be stored in the QemuOpts, objects that created
>>> with object_add won't stored in QemuOpts. Do you mean for DeviceState,
>>> DriveInfo they store there QemuOpts explicity so that they can delete it?
>>> Why don't we just delete it from objects directly instead?
>>
>> Let me elaborate.
>
> Thanks very much for the elaboration.
>
>>
>> We have the same pattern in multiple places: some kind of object gets
>> configured via QemuOpts, and an object's QemuOpts need to stay around
>> until the object dies.
>>
>> Example 1: Block device backends
>>
>>      DriveInfo has a member opts.
>>
>>      drive_new() stores the QemuOpts in dinfo->opts.
>>
>>      drive_info_del() destroys dinfo->opts.
>>
>>      Note: DriveInfo member opts is always non-null.  But not every
>>      BlockBackend has a DriveInfo.
>>
>> Example 2: Device frontends
>>
>>      DeviceState has a member opts.
>>
>>      qdev_device_add() stores the QemuOpts in dev->opts.
>>
>>      device_finalize() destroys dev->opts.
>>
>>      Note: DeviceState member opts may be null (not every device is
>>      created by qdev_device_add()).  Fine, because qemu_opts_del(NULL) is
>>      a no-op.
>>
>> Example 3: Character device backends
>>
>>      CharDriverState has a member opts.
>>
>>      qemu_chr_new_from_opts() stores the QemuOpts in chr->opts.
>>
>>      qemu_chr_delete() destroys chr->opts.
>
> 1-3 store there ops in there own state, not in global ops group right?

Both!  But keeping a pointer in their own state simplifies calling
qemu_opts_del() on destruction, and also makes it more obvious what is
keeping the QemuOpts alive.

>> Example 4: Network device backends
>>
>>      Two cases
>>
>>      A. netdev
>>
>>         qmp_netdev_add() does not store the QemuOpts.
>>
>>         qmp_netdev_del() still needs to destroy it.  It has to find it
>>         somehow.  Here's how it does it:
>>
>>             opts = qemu_opts_find(qemu_find_opts_err("netdev", NULL), id);
>>             if (!opts) {
>>                 error_setg(errp, "Device '%s' is not a netdev", id);
>>                 return;
>>             }
>>
>>         The !opts condition is a non-obvious way to test "not created
>>         with -netdev", see commit 645c949.  Note that the commit's claim
>>         that qemu_opts_del(NULL) crashes is no longer true since commit
>>         4782183.
>>
>>      B. Legacy net
>>
>>         hmp_host_net_add() does not store the QemuOpts.
>
> I'm afraid it does store the QemuOpts, but not in it's own state.
> net/net.c:
> 1088     qemu_opt_set(opts, "type", device, &error_abort);
> This will store the QemuOpts, or am I misunderstood it?

Doesn't store opts anywhere, actually.  It merely modifies it (adds a
parameter "type")

>>
>>         hmp_host_net_remove() still needs to destroy it.  I can't see
>>         where that happens, and I'm not sure it does.
>>
>> Example 5: Generic object
>>
>>      object_create() does not store the QemuOpts.
>>
>>      It still needs to be destroyed along with the object.  It isn't, and
>>      your patch fixes it.
>>
>> Personally, I find the technique in example 1-3 easier to understand
>> than the one in example 4-5.
>
> I agree that opts should not be used to determine not created something
> while there's case when something created but Opts not stored.

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Markus Armbruster]

Yang Hongyang <yanghy@cn.fujitsu.com> writes:

> On 09/24/2015 05:42 PM, Markus Armbruster wrote:
>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>
>>> On 09/24/2015 03:43 PM, Markus Armbruster wrote:
>>>> This has finally reached the front of my review queue.  I apologize for
>>>> the loooong delay.
>>>>
>>>> Copying Paolo for another pair of eyeballs (he wrote this code).
>>>>
>>> [...]
>>>>> +
>>>>> +    opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
>>>>> +    qemu_opts_del(opts);
>>>>
>>>> qemu_find_opts_err("object", &error_abort) please, because when it
>>>> fails, we want to die right away, not when the null pointer it returns
>>>> gets dereferenced.
>>>
>>> Thanks for the review.
>>> Jason, do you want me to propose a fix on top of this series or simply drop
>>> this for now because this patch is an independent bug fix and won't
>>> affect the
>>> other filter patch series.
>>>
>>>>
>>>> Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
>>>> your patch's fault.
>>>>
>>>> Elsewhere, we store the QemuOpts in the object just so we can delete it:
>>>> DeviceState, DriveInfo.  Paolo, what do you think?
>>>
>>> I don't get it. Currently, only objects created at the beginning through
>>> QEMU command line will be stored in the QemuOpts, objects that created
>>> with object_add won't stored in QemuOpts. Do you mean for DeviceState,
>>> DriveInfo they store there QemuOpts explicity so that they can delete it?
>>> Why don't we just delete it from objects directly instead?
>>
>> Let me elaborate.
>>
>> We have the same pattern in multiple places: some kind of object gets
>> configured via QemuOpts, and an object's QemuOpts need to stay around
>> until the object dies.
>>
>> Example 1: Block device backends
>>
>>      DriveInfo has a member opts.
>>
>>      drive_new() stores the QemuOpts in dinfo->opts.
>>
>>      drive_info_del() destroys dinfo->opts.
>>
>>      Note: DriveInfo member opts is always non-null.  But not every
>>      BlockBackend has a DriveInfo.
>>
>> Example 2: Device frontends
>>
>>      DeviceState has a member opts.
>>
>>      qdev_device_add() stores the QemuOpts in dev->opts.
>>
>>      device_finalize() destroys dev->opts.
>>
>>      Note: DeviceState member opts may be null (not every device is
>>      created by qdev_device_add()).  Fine, because qemu_opts_del(NULL) is
>>      a no-op.
>>
>> Example 3: Character device backends
>>
>>      CharDriverState has a member opts.
>>
>>      qemu_chr_new_from_opts() stores the QemuOpts in chr->opts.
>>
>>      qemu_chr_delete() destroys chr->opts.
>>
>> Example 4: Network device backends
>>
>>      Two cases
>>
>>      A. netdev
>>
>>         qmp_netdev_add() does not store the QemuOpts.
>
> The QemuOpts stored by qmp_netdev_add() and also hmp_netdev_add().
> through this function:
> net/net.c: qmp_netdev_add()
> 1134     opts = qemu_opts_from_qdict(opts_list, qdict, &local_err);
>
> hmp.c: hmp_netdev_add()
> 1579     opts = qemu_opts_from_qdict(qemu_find_opts("netdev"), qdict, &err);

That's where the QemuOpts are created.  By "does not store" I mean "does
not store in its own state, unlike example 1-3".

>>
>>         qmp_netdev_del() still needs to destroy it.  It has to find it
>>         somehow.  Here's how it does it:
>>
>>             opts = qemu_opts_find(qemu_find_opts_err("netdev", NULL), id);
>>             if (!opts) {
>>                 error_setg(errp, "Device '%s' is not a netdev", id);
>>                 return;
>>             }
>>
>>         The !opts condition is a non-obvious way to test "not created
>>         with -netdev", see commit 645c949.  Note that the commit's claim
>>         that qemu_opts_del(NULL) crashes is no longer true since commit
>>         4782183.
>>
>>      B. Legacy net
>>
>>         hmp_host_net_add() does not store the QemuOpts.
>>
>>         hmp_host_net_remove() still needs to destroy it.  I can't see
>>         where that happens, and I'm not sure it does.
>>
>> Example 5: Generic object
>>
>>      object_create() does not store the QemuOpts.
>>
>>      It still needs to be destroyed along with the object.  It isn't, and
>>      your patch fixes it.
>>
>> Personally, I find the technique in example 1-3 easier to understand
>> than the one in example 4-5.
>> .
>>

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Markus Armbruster]

Yang Hongyang <yanghy@cn.fujitsu.com> writes:

> On 09/24/2015 04:41 PM, Markus Armbruster wrote:
>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>
>>> Add a netfilter object based on QOM.
>>>
>>> A netfilter is attached to a netdev, captures all network packets
>>> that pass through the netdev. When we delete the netdev, we also
>>> delete the netfilter object attached to it, because if the netdev is
>>> removed, the filter which attached to it is useless.
>>>
>>> QTAILQ_ENTRY next used by netdev, filter belongs to the specific netdev is
>>> in this queue.
>>
>> I don't get this paragraph.  Not sure it's needed.
>>
>>> Also init delayed object after net_init_clients, because netfilters need
>>> to be initialized after net clients initialized.
>>
>> A paragraph starting with "Also" in a commit message is a pretty good
>> sign the patch should be split :)
>>
>>>
>>> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
>>> ---
>>> v11: no need to free nf->netdev_id, it will be auto freeed while object deleted
>>>       remove global_list net_filters, will add back when needed
>>> v10: use QOM for netfilter
>>> v9: use flat union instead of simple union in QAPI schema
>>> v8: include vhost_net header
>>> v7: add check for vhost
>>>      fix error propagate bug
>>> v6: add multiqueue support (net_filter_init1)
>>> v5: remove model from NetFilterState
>>>      add a sent_cb param to receive_iov API
>>> ---
>>>   include/net/filter.h    |  60 +++++++++++++++++++++
>>>   include/net/net.h       |   1 +
>>>   include/qemu/typedefs.h |   1 +
>>>   net/Makefile.objs       |   1 +
>>>   net/filter.c            | 138 ++++++++++++++++++++++++++++++++++++++++++++++++
>>>   net/net.c               |   7 +++
>>>   qapi-schema.json        |  18 +++++++
>>>   vl.c                    |  13 ++---
>>>   8 files changed, 233 insertions(+), 6 deletions(-)
>>>   create mode 100644 include/net/filter.h
>>>   create mode 100644 net/filter.c
>>>
>>> diff --git a/include/net/filter.h b/include/net/filter.h
>>> new file mode 100644
>>> index 0000000..226f2f7
>>> --- /dev/null
>>> +++ b/include/net/filter.h
>>> @@ -0,0 +1,60 @@
>>> +/*
>>> + * Copyright (c) 2015 FUJITSU LIMITED
>>> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
>>> + *
>>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>>> + * later.  See the COPYING file in the top-level directory.
>>> + */
>>> +
>>> +#ifndef QEMU_NET_FILTER_H
>>> +#define QEMU_NET_FILTER_H
>>> +
>>> +#include "qom/object.h"
>>> +#include "qemu-common.h"
>>> +#include "qemu/typedefs.h"
>>> +#include "net/queue.h"
>>> +
>>> +#define TYPE_NETFILTER "netfilter"
>>> +#define NETFILTER(obj) \
>>> +    OBJECT_CHECK(NetFilterState, (obj), TYPE_NETFILTER)
>>> +#define NETFILTER_GET_CLASS(obj) \
>>> +    OBJECT_GET_CLASS(NetFilterClass, (obj), TYPE_NETFILTER)
>>> +#define NETFILTER_CLASS(klass) \
>>> +    OBJECT_CLASS_CHECK(NetFilterClass, (klass), TYPE_NETFILTER)
>>> +
>>> +typedef void (FilterSetup) (NetFilterState *nf, Error **errp);
>>> +typedef void (FilterCleanup) (NetFilterState *nf);
>>> +/*
>>> + * Return:
>>> + *   0: finished handling the packet, we should continue
>>> + *   size: filter stolen this packet, we stop pass this packet further
>>> + */
>>> +typedef ssize_t (FilterReceiveIOV)(NetFilterState *nc,
>>> +                                   NetClientState *sender,
>>> +                                   unsigned flags,
>>> +                                   const struct iovec *iov,
>>> +                                   int iovcnt,
>>> +                                   NetPacketSent *sent_cb);
>>> +
>>> +struct NetFilterClass {
>>> +    ObjectClass parent_class;
>>> +
>>> +    FilterSetup *setup;
>>> +    FilterCleanup *cleanup;
>>> +    FilterReceiveIOV *receive_iov;
>>> +};
>>> +typedef struct NetFilterClass NetFilterClass;
>>
>> Not splitting the declaration is more concise:
>>
>>      typedef struct {
>>          ObjectClass parent_class;
>>          FilterSetup *setup;
>>          FilterCleanup *cleanup;
>>          FilterReceiveIOV *receive_iov;
>>      } NetFilterClass;
>>
>> Are any of the methods optional?  If yes, please add suitable comments.
>
> Hi Markus, I split it because the checkpatch.pl told me to do so...

Understand.  However, it's a recent change to checkpatch.pl that's going
to be reverted:
Message-ID: <55FAEB33.50809@redhat.com>
http://lists.gnu.org/archive/html/qemu-devel/2015-09/msg04644.html

[...]

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Markus Armbruster]

Yang Hongyang <yanghy@cn.fujitsu.com> writes:

> On 09/24/2015 04:41 PM, Markus Armbruster wrote:
>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>
>>> Add a netfilter object based on QOM.
>>>
>>> A netfilter is attached to a netdev, captures all network packets
>>> that pass through the netdev. When we delete the netdev, we also
>>> delete the netfilter object attached to it, because if the netdev is
>>> removed, the filter which attached to it is useless.
>>>
>>> QTAILQ_ENTRY next used by netdev, filter belongs to the specific netdev is
>>> in this queue.
>>
>> I don't get this paragraph.  Not sure it's needed.
>>
>>> Also init delayed object after net_init_clients, because netfilters need
>>> to be initialized after net clients initialized.
>>
>> A paragraph starting with "Also" in a commit message is a pretty good
>> sign the patch should be split :)
>>
>>>
>>> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
>>> ---
>>> v11: no need to free nf->netdev_id, it will be auto freeed while object deleted
>>>       remove global_list net_filters, will add back when needed
>>> v10: use QOM for netfilter
>>> v9: use flat union instead of simple union in QAPI schema
>>> v8: include vhost_net header
>>> v7: add check for vhost
>>>      fix error propagate bug
>>> v6: add multiqueue support (net_filter_init1)
>>> v5: remove model from NetFilterState
>>>      add a sent_cb param to receive_iov API
>>> ---
>>>   include/net/filter.h    |  60 +++++++++++++++++++++
>>>   include/net/net.h       |   1 +
>>>   include/qemu/typedefs.h |   1 +
>>>   net/Makefile.objs       |   1 +
>>>   net/filter.c            | 138 ++++++++++++++++++++++++++++++++++++++++++++++++
>>>   net/net.c               |   7 +++
>>>   qapi-schema.json        |  18 +++++++
>>>   vl.c                    |  13 ++---
>>>   8 files changed, 233 insertions(+), 6 deletions(-)
>>>   create mode 100644 include/net/filter.h
>>>   create mode 100644 net/filter.c
>>>
>>> diff --git a/include/net/filter.h b/include/net/filter.h
>>> new file mode 100644
>>> index 0000000..226f2f7
>>> --- /dev/null
>>> +++ b/include/net/filter.h
>>> @@ -0,0 +1,60 @@
>>> +/*
>>> + * Copyright (c) 2015 FUJITSU LIMITED
>>> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
>>> + *
>>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>>> + * later.  See the COPYING file in the top-level directory.
>>> + */
>>> +
>>> +#ifndef QEMU_NET_FILTER_H
>>> +#define QEMU_NET_FILTER_H
>>> +
>>> +#include "qom/object.h"
>>> +#include "qemu-common.h"
>>> +#include "qemu/typedefs.h"
>>> +#include "net/queue.h"
>>> +
>>> +#define TYPE_NETFILTER "netfilter"
>>> +#define NETFILTER(obj) \
>>> +    OBJECT_CHECK(NetFilterState, (obj), TYPE_NETFILTER)
>>> +#define NETFILTER_GET_CLASS(obj) \
>>> +    OBJECT_GET_CLASS(NetFilterClass, (obj), TYPE_NETFILTER)
>>> +#define NETFILTER_CLASS(klass) \
>>> +    OBJECT_CLASS_CHECK(NetFilterClass, (klass), TYPE_NETFILTER)
>>> +
>>> +typedef void (FilterSetup) (NetFilterState *nf, Error **errp);
>>> +typedef void (FilterCleanup) (NetFilterState *nf);
>>> +/*
>>> + * Return:
>>> + *   0: finished handling the packet, we should continue
>>> + *   size: filter stolen this packet, we stop pass this packet further
>>> + */
>>> +typedef ssize_t (FilterReceiveIOV)(NetFilterState *nc,
>>> +                                   NetClientState *sender,
>>> +                                   unsigned flags,
>>> +                                   const struct iovec *iov,
>>> +                                   int iovcnt,
>>> +                                   NetPacketSent *sent_cb);
>>> +
>>> +struct NetFilterClass {
>>> +    ObjectClass parent_class;
>>> +
>>> +    FilterSetup *setup;
>>> +    FilterCleanup *cleanup;
>>> +    FilterReceiveIOV *receive_iov;
>>> +};
>>> +typedef struct NetFilterClass NetFilterClass;
>>
>> Not splitting the declaration is more concise:
>>
>>      typedef struct {
>>          ObjectClass parent_class;
>>          FilterSetup *setup;
>>          FilterCleanup *cleanup;
>>          FilterReceiveIOV *receive_iov;
>>      } NetFilterClass;
>>
>> Are any of the methods optional?  If yes, please add suitable comments.
>>
>>> +
>>> +
>>> +struct NetFilterState {
>>> +    /* private */
>>> +    Object parent;
>>> +
>>> +    /* protected */
>>> +    char *netdev_id;
>>> +    NetClientState *netdev;
>>> +    NetFilterChain chain;
>>> +    QTAILQ_ENTRY(NetFilterState) next;
>>> +};
>>> +
>>> +#endif /* QEMU_NET_FILTER_H */
>>> diff --git a/include/net/net.h b/include/net/net.h
>>> index 6a6cbef..36e5fab 100644
>>> --- a/include/net/net.h
>>> +++ b/include/net/net.h
>>> @@ -92,6 +92,7 @@ struct NetClientState {
>>>       NetClientDestructor *destructor;
>>>       unsigned int queue_index;
>>>       unsigned rxfilter_notify_enabled:1;
>>> +    QTAILQ_HEAD(, NetFilterState) filters;
>>>   };
>>>
>>>   typedef struct NICState {
>>> diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
>>> index f8a9dd6..2c0648f 100644
>>> --- a/include/qemu/typedefs.h
>>> +++ b/include/qemu/typedefs.h
>>> @@ -45,6 +45,7 @@ typedef struct Monitor Monitor;
>>>   typedef struct MouseTransformInfo MouseTransformInfo;
>>>   typedef struct MSIMessage MSIMessage;
>>>   typedef struct NetClientState NetClientState;
>>> +typedef struct NetFilterState NetFilterState;
>>>   typedef struct NICInfo NICInfo;
>>>   typedef struct PcGuestInfo PcGuestInfo;
>>>   typedef struct PCIBridge PCIBridge;
>>> diff --git a/net/Makefile.objs b/net/Makefile.objs
>>> index ec19cb3..914aec0 100644
>>> --- a/net/Makefile.objs
>>> +++ b/net/Makefile.objs
>>> @@ -13,3 +13,4 @@ common-obj-$(CONFIG_HAIKU) += tap-haiku.o
>>>   common-obj-$(CONFIG_SLIRP) += slirp.o
>>>   common-obj-$(CONFIG_VDE) += vde.o
>>>   common-obj-$(CONFIG_NETMAP) += netmap.o
>>> +common-obj-y += filter.o
>>> diff --git a/net/filter.c b/net/filter.c
>>> new file mode 100644
>>> index 0000000..3b810c8
>>> --- /dev/null
>>> +++ b/net/filter.c
>>> @@ -0,0 +1,138 @@
>>> +/*
>>> + * Copyright (c) 2015 FUJITSU LIMITED
>>> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
>>> + *
>>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>>> + * later.  See the COPYING file in the top-level directory.
>>> + */
>>> +
>>> +#include "qemu-common.h"
>>> +#include "qapi/qmp/qerror.h"
>>> +#include "qemu/error-report.h"
>>> +
>>> +#include "net/filter.h"
>>> +#include "net/net.h"
>>> +#include "net/vhost_net.h"
>>> +#include "qom/object_interfaces.h"
>>> +
>>> +static char *netfilter_get_netdev_id(Object *obj, Error **errp)
>>> +{
>>> +    NetFilterState *nf = NETFILTER(obj);
>>> +
>>> +    return g_strdup(nf->netdev_id);
>>> +}
>>> +
>>> +static void netfilter_set_netdev_id(Object *obj, const char *str, Error **errp)
>>> +{
>>> +    NetFilterState *nf = NETFILTER(obj);
>>> +
>>> +    nf->netdev_id = g_strdup(str);
>>> +}
>>> +
>>> +static int netfilter_get_chain(Object *obj, Error **errp G_GNUC_UNUSED)
>>> +{
>>> +    NetFilterState *nf = NETFILTER(obj);
>>> +    return nf->chain;
>>> +}
>>> +
>>> +static void netfilter_set_chain(Object *obj, int chain, Error **errp)
>>> +{
>>> +    NetFilterState *nf = NETFILTER(obj);
>>> +    nf->chain = chain;
>>> +}
>>> +
>>> +static void netfilter_init(Object *obj)
>>> +{
>>> +    object_property_add_str(obj, "netdev",
>>> +                            netfilter_get_netdev_id, netfilter_set_netdev_id,
>>> +                            NULL);
>>> +    object_property_add_enum(obj, "chain", "NetFilterChain",
>>> +                             NetFilterChain_lookup,
>>> +                             netfilter_get_chain, netfilter_set_chain,
>>> +                             NULL);
>>> +}
>>> +
>>> +static void netfilter_finalize(Object *obj)
>>> +{
>>> +    NetFilterState *nf = NETFILTER(obj);
>>> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(obj);
>>> +
>>> +    if (nfc->cleanup) {
>>
>> Is ->cleanup optional?
>
> Yes, it's optional, as well as setup, if a concrete filter do
> not have things to setup/cleanup.

Please document that in NetFilterClass.

>>> +        nfc->cleanup(nf);
>>> +    }
>>> +
>>> +    if (nf->netdev && !QTAILQ_EMPTY(&nf->netdev->filters)) {
>>
>> How can nf->netdev be null?
>>
>> How can nf->netdev->filters be empty?
>
> When object initialize failed. for example, in netfilter_complete
> no nf->netdev_id provided, object initialize will fail, and the
> netfilter_finalize will be called, at the mean time,
> nf->netdev is null, and nf->netdev->filters is empty.

Okay.

>>> +        QTAILQ_REMOVE(&nf->netdev->filters, nf, next);
>>> +    }
>>> +}
>>
>> Putting netfilter_finalize() after netfilter_complete() would be easier
>> to understand, because then destruction follows creation.
>>
>>> +
>>> +static void netfilter_complete(UserCreatable *uc, Error **errp)
>>> +{
>>> +    NetFilterState *nf = NETFILTER(uc);
>>> +    NetClientState *ncs[MAX_QUEUE_NUM];
>>> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc);
>>> +    int queues;
>>> +    Error *local_err = NULL;
>>> +
>>> +    if (!nf->netdev_id) {
>>> +        error_setg(errp, "Parameter 'netdev' is required");
>>> +        return;
>>> +    }
>>> +
>>> +    queues = qemu_find_net_clients_except(nf->netdev_id, ncs,
>>> +                                          NET_CLIENT_OPTIONS_KIND_NIC,
>>> +                                          MAX_QUEUE_NUM);
>>> +    if (queues < 1) {
>>> +        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "netdev",
>>> +                   "a network backend id");
>>> +        return;
>>> +    } else if (queues > 1) {
>>> +        error_setg(errp, "Multi queue is not supported");
>>
>> We spell this multiqueue elsewhere.
>>
>> Since you're only interested in a single queue, you could save stack
>> space by making ncs[] just one element large.  Not worth it if you
>> intend to support multiqueue soon.
>
> We will support multiqueue soon.
>
>>
>>> +        return;
>>> +    }
>>> +
>>> +    if (get_vhost_net(ncs[0])) {
>>> +        error_setg(errp, "Vhost is not supported");
>>> +        return;
>>> +    }
>>> +
>>> +    nf->netdev = ncs[0];
>>> +
>>> +    if (nfc->setup) {
>>
>> Is ->setup optional?
>
> Yes, as said earlier.
>
>>
>>> +        nfc->setup(nf, &local_err);
>>> +        if (local_err) {
>>> +            error_propagate(errp, local_err);
>>> +            return;
>>> +        }
>>> +    }
>>> +    QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);
>>
>> I think I'd create netdev functions to add and remove filters, so the
>> filter code doesn't have to modify the netdev state.  Right now,
>> NetClientState member filters is initialized in net.c, but modified
>> here.  But I'm not the net maintainer :)
>>
>>> +}
>>> +
>>> +static void netfilter_class_init(ObjectClass *oc, void *data)
>>> +{
>>> +    UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
>>> +
>>> +    ucc->complete = netfilter_complete;
>>> +}
>>> +
>>> +static const TypeInfo netfilter_info = {
>>> +    .name = TYPE_NETFILTER,
>>> +    .parent = TYPE_OBJECT,
>>> +    .abstract = true,
>>> +    .class_size = sizeof(NetFilterClass),
>>> +    .class_init = netfilter_class_init,
>>> +    .instance_size = sizeof(NetFilterState),
>>> +    .instance_init = netfilter_init,
>>> +    .instance_finalize = netfilter_finalize,
>>> +    .interfaces = (InterfaceInfo[]) {
>>> +        { TYPE_USER_CREATABLE },
>>> +        { }
>>> +    }
>>> +};
>>> +
>>> +static void register_types(void)
>>> +{
>>> +    type_register_static(&netfilter_info);
>>> +}
>>> +
>>> +type_init(register_types);
[...]
>>> diff --git a/qapi-schema.json b/qapi-schema.json
>>> index 2bada60..546500a 100644
>>> --- a/qapi-schema.json
>>> +++ b/qapi-schema.json
>>> @@ -2551,6 +2551,24 @@
>>>       'opts': 'NetClientOptions' } }
>>>
>>>   ##
>>> +# @NetFilterChain
>>> +#
>>> +# netfilter chain, a netfilter is attached to a netdev, captures the
>>> +# network packets of the netdev.
>>> +#
>>> +# @all: the filter will receive packets both sent to/from the netdev, this
>>> +#       is the default chain.
>>> +#
>>> +# @in: the filter will receive packets sent to the netdev.
>>> +#
>>> +# @out: the filter will receive packets sent from the netdev.
>>
>> Uh, inhowfar is this a "chain"?  As far as I can tell, it specifies
>> whether a netfilter is attached to the transmit queue, the receive
>> queue, or both.
>
> netback's input chain or output chain, queue is mostly like a word that
> related to the code implementation? English is not my first language, so
> I'm not sure about this.

Don't worry, we'll get the language polished together :)

In PATCH 9, I learned how this is to be used:

    @item -object filter-buffer,id=@var{id},netdev=@var{netdevid}[,chain=@var{all|in|out}][,interval=@var{t}]

    Buffer network packets on netdev @var{netdevid}.
[...]
    chain @var{all|in|out} is an option that can be applied to any netfilter, default is @option{all}.

    @option{all} means this filter will receive packets both sent to/from the netdev

    @option{in} means this filter will receive packets sent to the netdev

    @option{out} means this filter will receive packets sent from the netdev

In that context, "chain" makes more sense.  I'd still call it "queue".
Precedence: -netdev parameter "queues".  If I understand correctly,
queues=N asks for N rx and tx queues.  A filter would either apply to
all N rx queues, all N tx queues, or all 2*N queues.  Correct?

[...]

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Yang Hongyang]

On 09/24/2015 07:35 PM, Markus Armbruster wrote:
> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>
>> On 09/24/2015 05:42 PM, Markus Armbruster wrote:
>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>>
>>>> On 09/24/2015 03:43 PM, Markus Armbruster wrote:
>>>>> This has finally reached the front of my review queue.  I apologize for
>>>>> the loooong delay.
>>>>>
>>>>> Copying Paolo for another pair of eyeballs (he wrote this code).
>>>>>
>>>> [...]
>>>>>> +
>>>>>> +    opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
>>>>>> +    qemu_opts_del(opts);
>>>>>
>>>>> qemu_find_opts_err("object", &error_abort) please, because when it
>>>>> fails, we want to die right away, not when the null pointer it returns
>>>>> gets dereferenced.
>>>>
>>>> Thanks for the review.
>>>> Jason, do you want me to propose a fix on top of this series or simply drop
>>>> this for now because this patch is an independent bug fix and won't
>>>> affect the
>>>> other filter patch series.
>>>>
>>>>>
>>>>> Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
>>>>> your patch's fault.
>>>>>
>>>>> Elsewhere, we store the QemuOpts in the object just so we can delete it:
>>>>> DeviceState, DriveInfo.  Paolo, what do you think?
>>>>
>>>> I don't get it. Currently, only objects created at the beginning through
>>>> QEMU command line will be stored in the QemuOpts, objects that created
>>>> with object_add won't stored in QemuOpts. Do you mean for DeviceState,
>>>> DriveInfo they store there QemuOpts explicity so that they can delete it?
>>>> Why don't we just delete it from objects directly instead?
>>>
>>> Let me elaborate.
>>
>> Thanks very much for the elaboration.
>>
>>>
>>> We have the same pattern in multiple places: some kind of object gets
>>> configured via QemuOpts, and an object's QemuOpts need to stay around
>>> until the object dies.
>>>
>>> Example 1: Block device backends
>>>
>>>       DriveInfo has a member opts.
>>>
>>>       drive_new() stores the QemuOpts in dinfo->opts.
>>>
>>>       drive_info_del() destroys dinfo->opts.
>>>
>>>       Note: DriveInfo member opts is always non-null.  But not every
>>>       BlockBackend has a DriveInfo.
>>>
>>> Example 2: Device frontends
>>>
>>>       DeviceState has a member opts.
>>>
>>>       qdev_device_add() stores the QemuOpts in dev->opts.
>>>
>>>       device_finalize() destroys dev->opts.
>>>
>>>       Note: DeviceState member opts may be null (not every device is
>>>       created by qdev_device_add()).  Fine, because qemu_opts_del(NULL) is
>>>       a no-op.
>>>
>>> Example 3: Character device backends
>>>
>>>       CharDriverState has a member opts.
>>>
>>>       qemu_chr_new_from_opts() stores the QemuOpts in chr->opts.
>>>
>>>       qemu_chr_delete() destroys chr->opts.
>>
>> 1-3 store there ops in there own state, not in global ops group right?
>
> Both!  But keeping a pointer in their own state simplifies calling
> qemu_opts_del() on destruction, and also makes it more obvious what is
> keeping the QemuOpts alive.

I see. Thanks.

>
>>> Example 4: Network device backends
>>>
>>>       Two cases
>>>
>>>       A. netdev
>>>
>>>          qmp_netdev_add() does not store the QemuOpts.
>>>
>>>          qmp_netdev_del() still needs to destroy it.  It has to find it
>>>          somehow.  Here's how it does it:
>>>
>>>              opts = qemu_opts_find(qemu_find_opts_err("netdev", NULL), id);
>>>              if (!opts) {
>>>                  error_setg(errp, "Device '%s' is not a netdev", id);
>>>                  return;
>>>              }
>>>
>>>          The !opts condition is a non-obvious way to test "not created
>>>          with -netdev", see commit 645c949.  Note that the commit's claim
>>>          that qemu_opts_del(NULL) crashes is no longer true since commit
>>>          4782183.
>>>
>>>       B. Legacy net
>>>
>>>          hmp_host_net_add() does not store the QemuOpts.
>>
>> I'm afraid it does store the QemuOpts, but not in it's own state.
>> net/net.c:
>> 1088     qemu_opt_set(opts, "type", device, &error_abort);
>> This will store the QemuOpts, or am I misunderstood it?
>
> Doesn't store opts anywhere, actually.  It merely modifies it (adds a
> parameter "type")

As you said "store" means store in there own state, then I see...thanks

>
>>>
>>>          hmp_host_net_remove() still needs to destroy it.  I can't see
>>>          where that happens, and I'm not sure it does.
>>>
>>> Example 5: Generic object
>>>
>>>       object_create() does not store the QemuOpts.
>>>
>>>       It still needs to be destroyed along with the object.  It isn't, and
>>>       your patch fixes it.
>>>
>>> Personally, I find the technique in example 1-3 easier to understand
>>> than the one in example 4-5.
>>
>> I agree that opts should not be used to determine not created something
>> while there's case when something created but Opts not stored.
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Yang Hongyang]

On 09/24/2015 07:36 PM, Markus Armbruster wrote:
> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>
>> On 09/24/2015 05:42 PM, Markus Armbruster wrote:
>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>>
>>>> On 09/24/2015 03:43 PM, Markus Armbruster wrote:
>>>>> This has finally reached the front of my review queue.  I apologize for
>>>>> the loooong delay.
>>>>>
>>>>> Copying Paolo for another pair of eyeballs (he wrote this code).
>>>>>
>>>> [...]
>>>>>> +
>>>>>> +    opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
>>>>>> +    qemu_opts_del(opts);
>>>>>
>>>>> qemu_find_opts_err("object", &error_abort) please, because when it
>>>>> fails, we want to die right away, not when the null pointer it returns
>>>>> gets dereferenced.
>>>>
>>>> Thanks for the review.
>>>> Jason, do you want me to propose a fix on top of this series or simply drop
>>>> this for now because this patch is an independent bug fix and won't
>>>> affect the
>>>> other filter patch series.
>>>>
>>>>>
>>>>> Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
>>>>> your patch's fault.
>>>>>
>>>>> Elsewhere, we store the QemuOpts in the object just so we can delete it:
>>>>> DeviceState, DriveInfo.  Paolo, what do you think?
>>>>
>>>> I don't get it. Currently, only objects created at the beginning through
>>>> QEMU command line will be stored in the QemuOpts, objects that created
>>>> with object_add won't stored in QemuOpts. Do you mean for DeviceState,
>>>> DriveInfo they store there QemuOpts explicity so that they can delete it?
>>>> Why don't we just delete it from objects directly instead?
>>>
>>> Let me elaborate.
>>>
>>> We have the same pattern in multiple places: some kind of object gets
>>> configured via QemuOpts, and an object's QemuOpts need to stay around
>>> until the object dies.
>>>
>>> Example 1: Block device backends
>>>
>>>       DriveInfo has a member opts.
>>>
>>>       drive_new() stores the QemuOpts in dinfo->opts.
>>>
>>>       drive_info_del() destroys dinfo->opts.
>>>
>>>       Note: DriveInfo member opts is always non-null.  But not every
>>>       BlockBackend has a DriveInfo.
>>>
>>> Example 2: Device frontends
>>>
>>>       DeviceState has a member opts.
>>>
>>>       qdev_device_add() stores the QemuOpts in dev->opts.
>>>
>>>       device_finalize() destroys dev->opts.
>>>
>>>       Note: DeviceState member opts may be null (not every device is
>>>       created by qdev_device_add()).  Fine, because qemu_opts_del(NULL) is
>>>       a no-op.
>>>
>>> Example 3: Character device backends
>>>
>>>       CharDriverState has a member opts.
>>>
>>>       qemu_chr_new_from_opts() stores the QemuOpts in chr->opts.
>>>
>>>       qemu_chr_delete() destroys chr->opts.
>>>
>>> Example 4: Network device backends
>>>
>>>       Two cases
>>>
>>>       A. netdev
>>>
>>>          qmp_netdev_add() does not store the QemuOpts.
>>
>> The QemuOpts stored by qmp_netdev_add() and also hmp_netdev_add().
>> through this function:
>> net/net.c: qmp_netdev_add()
>> 1134     opts = qemu_opts_from_qdict(opts_list, qdict, &local_err);
>>
>> hmp.c: hmp_netdev_add()
>> 1579     opts = qemu_opts_from_qdict(qemu_find_opts("netdev"), qdict, &err);
>
> That's where the QemuOpts are created.  By "does not store" I mean "does
> not store in its own state, unlike example 1-3".

Understand, thank you.

>
>>>
>>>          qmp_netdev_del() still needs to destroy it.  It has to find it
>>>          somehow.  Here's how it does it:
>>>
>>>              opts = qemu_opts_find(qemu_find_opts_err("netdev", NULL), id);
>>>              if (!opts) {
>>>                  error_setg(errp, "Device '%s' is not a netdev", id);
>>>                  return;
>>>              }
>>>
>>>          The !opts condition is a non-obvious way to test "not created
>>>          with -netdev", see commit 645c949.  Note that the commit's claim
>>>          that qemu_opts_del(NULL) crashes is no longer true since commit
>>>          4782183.
>>>
>>>       B. Legacy net
>>>
>>>          hmp_host_net_add() does not store the QemuOpts.
>>>
>>>          hmp_host_net_remove() still needs to destroy it.  I can't see
>>>          where that happens, and I'm not sure it does.
>>>
>>> Example 5: Generic object
>>>
>>>       object_create() does not store the QemuOpts.
>>>
>>>       It still needs to be destroyed along with the object.  It isn't, and
>>>       your patch fixes it.
>>>
>>> Personally, I find the technique in example 1-3 easier to understand
>>> than the one in example 4-5.
>>> .
>>>
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Yang Hongyang]

On 09/24/2015 07:40 PM, Markus Armbruster wrote:
> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>
>> On 09/24/2015 04:41 PM, Markus Armbruster wrote:
>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>>
>>>> Add a netfilter object based on QOM.
>>>>
>>>> A netfilter is attached to a netdev, captures all network packets
>>>> that pass through the netdev. When we delete the netdev, we also
>>>> delete the netfilter object attached to it, because if the netdev is
>>>> removed, the filter which attached to it is useless.
>>>>
>>>> QTAILQ_ENTRY next used by netdev, filter belongs to the specific netdev is
>>>> in this queue.
>>>
>>> I don't get this paragraph.  Not sure it's needed.
>>>
>>>> Also init delayed object after net_init_clients, because netfilters need
>>>> to be initialized after net clients initialized.
>>>
>>> A paragraph starting with "Also" in a commit message is a pretty good
>>> sign the patch should be split :)
>>>
>>>>
>>>> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
>>>> ---
>>>> v11: no need to free nf->netdev_id, it will be auto freeed while object deleted
>>>>        remove global_list net_filters, will add back when needed
>>>> v10: use QOM for netfilter
>>>> v9: use flat union instead of simple union in QAPI schema
>>>> v8: include vhost_net header
>>>> v7: add check for vhost
>>>>       fix error propagate bug
>>>> v6: add multiqueue support (net_filter_init1)
>>>> v5: remove model from NetFilterState
>>>>       add a sent_cb param to receive_iov API
>>>> ---
>>>>    include/net/filter.h    |  60 +++++++++++++++++++++
>>>>    include/net/net.h       |   1 +
>>>>    include/qemu/typedefs.h |   1 +
>>>>    net/Makefile.objs       |   1 +
>>>>    net/filter.c            | 138 ++++++++++++++++++++++++++++++++++++++++++++++++
>>>>    net/net.c               |   7 +++
>>>>    qapi-schema.json        |  18 +++++++
>>>>    vl.c                    |  13 ++---
>>>>    8 files changed, 233 insertions(+), 6 deletions(-)
>>>>    create mode 100644 include/net/filter.h
>>>>    create mode 100644 net/filter.c
>>>>
>>>> diff --git a/include/net/filter.h b/include/net/filter.h
>>>> new file mode 100644
>>>> index 0000000..226f2f7
>>>> --- /dev/null
>>>> +++ b/include/net/filter.h
>>>> @@ -0,0 +1,60 @@
>>>> +/*
>>>> + * Copyright (c) 2015 FUJITSU LIMITED
>>>> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
>>>> + *
>>>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>>>> + * later.  See the COPYING file in the top-level directory.
>>>> + */
>>>> +
>>>> +#ifndef QEMU_NET_FILTER_H
>>>> +#define QEMU_NET_FILTER_H
>>>> +
>>>> +#include "qom/object.h"
>>>> +#include "qemu-common.h"
>>>> +#include "qemu/typedefs.h"
>>>> +#include "net/queue.h"
>>>> +
>>>> +#define TYPE_NETFILTER "netfilter"
>>>> +#define NETFILTER(obj) \
>>>> +    OBJECT_CHECK(NetFilterState, (obj), TYPE_NETFILTER)
>>>> +#define NETFILTER_GET_CLASS(obj) \
>>>> +    OBJECT_GET_CLASS(NetFilterClass, (obj), TYPE_NETFILTER)
>>>> +#define NETFILTER_CLASS(klass) \
>>>> +    OBJECT_CLASS_CHECK(NetFilterClass, (klass), TYPE_NETFILTER)
>>>> +
>>>> +typedef void (FilterSetup) (NetFilterState *nf, Error **errp);
>>>> +typedef void (FilterCleanup) (NetFilterState *nf);
>>>> +/*
>>>> + * Return:
>>>> + *   0: finished handling the packet, we should continue
>>>> + *   size: filter stolen this packet, we stop pass this packet further
>>>> + */
>>>> +typedef ssize_t (FilterReceiveIOV)(NetFilterState *nc,
>>>> +                                   NetClientState *sender,
>>>> +                                   unsigned flags,
>>>> +                                   const struct iovec *iov,
>>>> +                                   int iovcnt,
>>>> +                                   NetPacketSent *sent_cb);
>>>> +
>>>> +struct NetFilterClass {
>>>> +    ObjectClass parent_class;
>>>> +
>>>> +    FilterSetup *setup;
>>>> +    FilterCleanup *cleanup;
>>>> +    FilterReceiveIOV *receive_iov;
>>>> +};
>>>> +typedef struct NetFilterClass NetFilterClass;
>>>
>>> Not splitting the declaration is more concise:
>>>
>>>       typedef struct {
>>>           ObjectClass parent_class;
>>>           FilterSetup *setup;
>>>           FilterCleanup *cleanup;
>>>           FilterReceiveIOV *receive_iov;
>>>       } NetFilterClass;
>>>
>>> Are any of the methods optional?  If yes, please add suitable comments.
>>
>> Hi Markus, I split it because the checkpatch.pl told me to do so...
>
> Understand.  However, it's a recent change to checkpatch.pl that's going
> to be reverted:
> Message-ID: <55FAEB33.50809@redhat.com>
> http://lists.gnu.org/archive/html/qemu-devel/2015-09/msg04644.html

Thanks for the information.

>
> [...]
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Jason Wang]

On 09/24/2015 04:35 PM, Yang Hongyang wrote:
> On 09/24/2015 03:43 PM, Markus Armbruster wrote:
>> This has finally reached the front of my review queue.  I apologize for
>> the loooong delay.
>>
>> Copying Paolo for another pair of eyeballs (he wrote this code).
>>
> [...]
>>> +
>>> +    opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
>>> +    qemu_opts_del(opts);
>>
>> qemu_find_opts_err("object", &error_abort) please, because when it
>> fails, we want to die right away, not when the null pointer it returns
>> gets dereferenced.
>
> Thanks for the review.
> Jason, do you want me to propose a fix on top of this series or simply
> drop
> this for now because this patch is an independent bug fix and won't
> affect the
> other filter patch series.

Will drop this patch from my tree.

Thanks

>
>>
>> Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
>> your patch's fault.
>>
>> Elsewhere, we store the QemuOpts in the object just so we can delete it:
>> DeviceState, DriveInfo.  Paolo, what do you think?
>
> I don't get it. Currently, only objects created at the beginning through
> QEMU command line will be stored in the QemuOpts, objects that created
> with object_add won't stored in QemuOpts. Do you mean for DeviceState,
> DriveInfo they store there QemuOpts explicity so that they can delete it?
> Why don't we just delete it from objects directly instead?
>
>>
>>>   }
>>>
>>>   MemoryDeviceInfoList *qmp_query_memory_devices(Error **errp)
>> .
>>
>

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Jason Wang]

On 09/24/2015 07:52 PM, Markus Armbruster wrote:
> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>
>> On 09/24/2015 04:41 PM, Markus Armbruster wrote:
>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>>
>>>> Add a netfilter object based on QOM.
>>>>
>>>> A netfilter is attached to a netdev, captures all network packets
>>>> that pass through the netdev. When we delete the netdev, we also
>>>> delete the netfilter object attached to it, because if the netdev is
>>>> removed, the filter which attached to it is useless.
>>>>
>>>> QTAILQ_ENTRY next used by netdev, filter belongs to the specific netdev is
>>>> in this queue.
>>> I don't get this paragraph.  Not sure it's needed.
>>>
>>>> Also init delayed object after net_init_clients, because netfilters need
>>>> to be initialized after net clients initialized.
>>> A paragraph starting with "Also" in a commit message is a pretty good
>>> sign the patch should be split :)
>>>
>>>> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
>>>> ---
>>>> v11: no need to free nf->netdev_id, it will be auto freeed while object deleted
>>>>       remove global_list net_filters, will add back when needed
>>>> v10: use QOM for netfilter
>>>> v9: use flat union instead of simple union in QAPI schema
>>>> v8: include vhost_net header
>>>> v7: add check for vhost
>>>>      fix error propagate bug
>>>> v6: add multiqueue support (net_filter_init1)
>>>> v5: remove model from NetFilterState
>>>>      add a sent_cb param to receive_iov API
>>>> ---
>>>>   include/net/filter.h    |  60 +++++++++++++++++++++
>>>>   include/net/net.h       |   1 +
>>>>   include/qemu/typedefs.h |   1 +
>>>>   net/Makefile.objs       |   1 +
>>>>   net/filter.c            | 138 ++++++++++++++++++++++++++++++++++++++++++++++++
>>>>   net/net.c               |   7 +++
>>>>   qapi-schema.json        |  18 +++++++
>>>>   vl.c                    |  13 ++---
>>>>   8 files changed, 233 insertions(+), 6 deletions(-)
>>>>   create mode 100644 include/net/filter.h
>>>>   create mode 100644 net/filter.c
>>>>
>>>> diff --git a/include/net/filter.h b/include/net/filter.h
>>>> new file mode 100644
>>>> index 0000000..226f2f7
>>>> --- /dev/null
>>>> +++ b/include/net/filter.h
>>>> @@ -0,0 +1,60 @@
>>>> +/*
>>>> + * Copyright (c) 2015 FUJITSU LIMITED
>>>> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
>>>> + *
>>>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>>>> + * later.  See the COPYING file in the top-level directory.
>>>> + */
>>>> +
>>>> +#ifndef QEMU_NET_FILTER_H
>>>> +#define QEMU_NET_FILTER_H
>>>> +
>>>> +#include "qom/object.h"
>>>> +#include "qemu-common.h"
>>>> +#include "qemu/typedefs.h"
>>>> +#include "net/queue.h"
>>>> +
>>>> +#define TYPE_NETFILTER "netfilter"
>>>> +#define NETFILTER(obj) \
>>>> +    OBJECT_CHECK(NetFilterState, (obj), TYPE_NETFILTER)
>>>> +#define NETFILTER_GET_CLASS(obj) \
>>>> +    OBJECT_GET_CLASS(NetFilterClass, (obj), TYPE_NETFILTER)
>>>> +#define NETFILTER_CLASS(klass) \
>>>> +    OBJECT_CLASS_CHECK(NetFilterClass, (klass), TYPE_NETFILTER)
>>>> +
>>>> +typedef void (FilterSetup) (NetFilterState *nf, Error **errp);
>>>> +typedef void (FilterCleanup) (NetFilterState *nf);
>>>> +/*
>>>> + * Return:
>>>> + *   0: finished handling the packet, we should continue
>>>> + *   size: filter stolen this packet, we stop pass this packet further
>>>> + */
>>>> +typedef ssize_t (FilterReceiveIOV)(NetFilterState *nc,
>>>> +                                   NetClientState *sender,
>>>> +                                   unsigned flags,
>>>> +                                   const struct iovec *iov,
>>>> +                                   int iovcnt,
>>>> +                                   NetPacketSent *sent_cb);
>>>> +
>>>> +struct NetFilterClass {
>>>> +    ObjectClass parent_class;
>>>> +
>>>> +    FilterSetup *setup;
>>>> +    FilterCleanup *cleanup;
>>>> +    FilterReceiveIOV *receive_iov;
>>>> +};
>>>> +typedef struct NetFilterClass NetFilterClass;
>>> Not splitting the declaration is more concise:
>>>
>>>      typedef struct {
>>>          ObjectClass parent_class;
>>>          FilterSetup *setup;
>>>          FilterCleanup *cleanup;
>>>          FilterReceiveIOV *receive_iov;
>>>      } NetFilterClass;
>>>
>>> Are any of the methods optional?  If yes, please add suitable comments.
>>>
>>>> +
>>>> +
>>>> +struct NetFilterState {
>>>> +    /* private */
>>>> +    Object parent;
>>>> +
>>>> +    /* protected */
>>>> +    char *netdev_id;
>>>> +    NetClientState *netdev;
>>>> +    NetFilterChain chain;
>>>> +    QTAILQ_ENTRY(NetFilterState) next;
>>>> +};
>>>> +
>>>> +#endif /* QEMU_NET_FILTER_H */
>>>> diff --git a/include/net/net.h b/include/net/net.h
>>>> index 6a6cbef..36e5fab 100644
>>>> --- a/include/net/net.h
>>>> +++ b/include/net/net.h
>>>> @@ -92,6 +92,7 @@ struct NetClientState {
>>>>       NetClientDestructor *destructor;
>>>>       unsigned int queue_index;
>>>>       unsigned rxfilter_notify_enabled:1;
>>>> +    QTAILQ_HEAD(, NetFilterState) filters;
>>>>   };
>>>>
>>>>   typedef struct NICState {
>>>> diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
>>>> index f8a9dd6..2c0648f 100644
>>>> --- a/include/qemu/typedefs.h
>>>> +++ b/include/qemu/typedefs.h
>>>> @@ -45,6 +45,7 @@ typedef struct Monitor Monitor;
>>>>   typedef struct MouseTransformInfo MouseTransformInfo;
>>>>   typedef struct MSIMessage MSIMessage;
>>>>   typedef struct NetClientState NetClientState;
>>>> +typedef struct NetFilterState NetFilterState;
>>>>   typedef struct NICInfo NICInfo;
>>>>   typedef struct PcGuestInfo PcGuestInfo;
>>>>   typedef struct PCIBridge PCIBridge;
>>>> diff --git a/net/Makefile.objs b/net/Makefile.objs
>>>> index ec19cb3..914aec0 100644
>>>> --- a/net/Makefile.objs
>>>> +++ b/net/Makefile.objs
>>>> @@ -13,3 +13,4 @@ common-obj-$(CONFIG_HAIKU) += tap-haiku.o
>>>>   common-obj-$(CONFIG_SLIRP) += slirp.o
>>>>   common-obj-$(CONFIG_VDE) += vde.o
>>>>   common-obj-$(CONFIG_NETMAP) += netmap.o
>>>> +common-obj-y += filter.o
>>>> diff --git a/net/filter.c b/net/filter.c
>>>> new file mode 100644
>>>> index 0000000..3b810c8
>>>> --- /dev/null
>>>> +++ b/net/filter.c
>>>> @@ -0,0 +1,138 @@
>>>> +/*
>>>> + * Copyright (c) 2015 FUJITSU LIMITED
>>>> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
>>>> + *
>>>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>>>> + * later.  See the COPYING file in the top-level directory.
>>>> + */
>>>> +
>>>> +#include "qemu-common.h"
>>>> +#include "qapi/qmp/qerror.h"
>>>> +#include "qemu/error-report.h"
>>>> +
>>>> +#include "net/filter.h"
>>>> +#include "net/net.h"
>>>> +#include "net/vhost_net.h"
>>>> +#include "qom/object_interfaces.h"
>>>> +
>>>> +static char *netfilter_get_netdev_id(Object *obj, Error **errp)
>>>> +{
>>>> +    NetFilterState *nf = NETFILTER(obj);
>>>> +
>>>> +    return g_strdup(nf->netdev_id);
>>>> +}
>>>> +
>>>> +static void netfilter_set_netdev_id(Object *obj, const char *str, Error **errp)
>>>> +{
>>>> +    NetFilterState *nf = NETFILTER(obj);
>>>> +
>>>> +    nf->netdev_id = g_strdup(str);
>>>> +}
>>>> +
>>>> +static int netfilter_get_chain(Object *obj, Error **errp G_GNUC_UNUSED)
>>>> +{
>>>> +    NetFilterState *nf = NETFILTER(obj);
>>>> +    return nf->chain;
>>>> +}
>>>> +
>>>> +static void netfilter_set_chain(Object *obj, int chain, Error **errp)
>>>> +{
>>>> +    NetFilterState *nf = NETFILTER(obj);
>>>> +    nf->chain = chain;
>>>> +}
>>>> +
>>>> +static void netfilter_init(Object *obj)
>>>> +{
>>>> +    object_property_add_str(obj, "netdev",
>>>> +                            netfilter_get_netdev_id, netfilter_set_netdev_id,
>>>> +                            NULL);
>>>> +    object_property_add_enum(obj, "chain", "NetFilterChain",
>>>> +                             NetFilterChain_lookup,
>>>> +                             netfilter_get_chain, netfilter_set_chain,
>>>> +                             NULL);
>>>> +}
>>>> +
>>>> +static void netfilter_finalize(Object *obj)
>>>> +{
>>>> +    NetFilterState *nf = NETFILTER(obj);
>>>> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(obj);
>>>> +
>>>> +    if (nfc->cleanup) {
>>> Is ->cleanup optional?
>> Yes, it's optional, as well as setup, if a concrete filter do
>> not have things to setup/cleanup.
> Please document that in NetFilterClass.
>
>>>> +        nfc->cleanup(nf);
>>>> +    }
>>>> +
>>>> +    if (nf->netdev && !QTAILQ_EMPTY(&nf->netdev->filters)) {
>>> How can nf->netdev be null?
>>>
>>> How can nf->netdev->filters be empty?
>> When object initialize failed. for example, in netfilter_complete
>> no nf->netdev_id provided, object initialize will fail, and the
>> netfilter_finalize will be called, at the mean time,
>> nf->netdev is null, and nf->netdev->filters is empty.
> Okay.
>
>>>> +        QTAILQ_REMOVE(&nf->netdev->filters, nf, next);
>>>> +    }
>>>> +}
>>> Putting netfilter_finalize() after netfilter_complete() would be easier
>>> to understand, because then destruction follows creation.
>>>
>>>> +
>>>> +static void netfilter_complete(UserCreatable *uc, Error **errp)
>>>> +{
>>>> +    NetFilterState *nf = NETFILTER(uc);
>>>> +    NetClientState *ncs[MAX_QUEUE_NUM];
>>>> +    NetFilterClass *nfc = NETFILTER_GET_CLASS(uc);
>>>> +    int queues;
>>>> +    Error *local_err = NULL;
>>>> +
>>>> +    if (!nf->netdev_id) {
>>>> +        error_setg(errp, "Parameter 'netdev' is required");
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    queues = qemu_find_net_clients_except(nf->netdev_id, ncs,
>>>> +                                          NET_CLIENT_OPTIONS_KIND_NIC,
>>>> +                                          MAX_QUEUE_NUM);
>>>> +    if (queues < 1) {
>>>> +        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "netdev",
>>>> +                   "a network backend id");
>>>> +        return;
>>>> +    } else if (queues > 1) {
>>>> +        error_setg(errp, "Multi queue is not supported");
>>> We spell this multiqueue elsewhere.
>>>
>>> Since you're only interested in a single queue, you could save stack
>>> space by making ncs[] just one element large.  Not worth it if you
>>> intend to support multiqueue soon.
>> We will support multiqueue soon.
>>
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    if (get_vhost_net(ncs[0])) {
>>>> +        error_setg(errp, "Vhost is not supported");
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    nf->netdev = ncs[0];
>>>> +
>>>> +    if (nfc->setup) {
>>> Is ->setup optional?
>> Yes, as said earlier.
>>
>>>> +        nfc->setup(nf, &local_err);
>>>> +        if (local_err) {
>>>> +            error_propagate(errp, local_err);
>>>> +            return;
>>>> +        }
>>>> +    }
>>>> +    QTAILQ_INSERT_TAIL(&nf->netdev->filters, nf, next);
>>> I think I'd create netdev functions to add and remove filters, so the
>>> filter code doesn't have to modify the netdev state.  Right now,
>>> NetClientState member filters is initialized in net.c, but modified
>>> here.  But I'm not the net maintainer :)
>>>
>>>> +}
>>>> +
>>>> +static void netfilter_class_init(ObjectClass *oc, void *data)
>>>> +{
>>>> +    UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
>>>> +
>>>> +    ucc->complete = netfilter_complete;
>>>> +}
>>>> +
>>>> +static const TypeInfo netfilter_info = {
>>>> +    .name = TYPE_NETFILTER,
>>>> +    .parent = TYPE_OBJECT,
>>>> +    .abstract = true,
>>>> +    .class_size = sizeof(NetFilterClass),
>>>> +    .class_init = netfilter_class_init,
>>>> +    .instance_size = sizeof(NetFilterState),
>>>> +    .instance_init = netfilter_init,
>>>> +    .instance_finalize = netfilter_finalize,
>>>> +    .interfaces = (InterfaceInfo[]) {
>>>> +        { TYPE_USER_CREATABLE },
>>>> +        { }
>>>> +    }
>>>> +};
>>>> +
>>>> +static void register_types(void)
>>>> +{
>>>> +    type_register_static(&netfilter_info);
>>>> +}
>>>> +
>>>> +type_init(register_types);
> [...]
>>>> diff --git a/qapi-schema.json b/qapi-schema.json
>>>> index 2bada60..546500a 100644
>>>> --- a/qapi-schema.json
>>>> +++ b/qapi-schema.json
>>>> @@ -2551,6 +2551,24 @@
>>>>       'opts': 'NetClientOptions' } }
>>>>
>>>>   ##
>>>> +# @NetFilterChain
>>>> +#
>>>> +# netfilter chain, a netfilter is attached to a netdev, captures the
>>>> +# network packets of the netdev.
>>>> +#
>>>> +# @all: the filter will receive packets both sent to/from the netdev, this
>>>> +#       is the default chain.
>>>> +#
>>>> +# @in: the filter will receive packets sent to the netdev.
>>>> +#
>>>> +# @out: the filter will receive packets sent from the netdev.
>>> Uh, inhowfar is this a "chain"?  As far as I can tell, it specifies
>>> whether a netfilter is attached to the transmit queue, the receive
>>> queue, or both.
>> netback's input chain or output chain, queue is mostly like a word that
>> related to the code implementation? English is not my first language, so
>> I'm not sure about this.
> Don't worry, we'll get the language polished together :)
>
> In PATCH 9, I learned how this is to be used:
>
>     @item -object filter-buffer,id=@var{id},netdev=@var{netdevid}[,chain=@var{all|in|out}][,interval=@var{t}]
>
>     Buffer network packets on netdev @var{netdevid}.
> [...]
>     chain @var{all|in|out} is an option that can be applied to any netfilter, default is @option{all}.
>
>     @option{all} means this filter will receive packets both sent to/from the netdev
>
>     @option{in} means this filter will receive packets sent to the netdev
>
>     @option{out} means this filter will receive packets sent from the netdev
>
> In that context, "chain" makes more sense.  I'd still call it "queue".
> Precedence: -netdev parameter "queues".  If I understand correctly,
> queues=N asks for N rx and tx queues.  A filter would either apply to
> all N rx queues, all N tx queues, or all 2*N queues.  Correct?
>
> [...]

Yes.

Re: [Qemu-devel] [PATCH v11 09/12] netfilter: add a netbuffer filter

[Yang Hongyang]

On 09/24/2015 05:12 PM, Markus Armbruster wrote:
> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>
>> This filter is to buffer/release packets, this feature can be used
>> when using MicroCheckpointing, or other Remus like VM FT solutions, you
>
> What's "Remus"?

Remus is an opensource VM FT solution:
paper: http://http//www.cs.ubc.ca/~andy/papers/remus-nsdi-final.pdf
First implemented on Xen:
http://wiki.xen.org/wiki/Remus

MicroCheckpointing in QEMU is another Remus implementation.

>
>> can also use it to simulate the network delay.
>> It has an interval option, if supplied, this filter will release
>> packets by interval.
>
> Suggest "will delay packets by that time interval."
>
> Is interval really optional?

It supposed to be optional. When the buffer filter has a user:
http://lists.nongnu.org/archive/html/qemu-devel/2015-09/msg00363.html

In this patchset, the zero interval check is removed. packets are
released on demand through filter_buffer_release_all() api call.

>
>>
>> Usage:
>>   -netdev tap,id=bn0
>>   -object filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000
>>
>> NOTE:
>>   the scale of interval is microsecond.
>
> Perhaps "interval is in microseconds".

Better, thanks.

>
>>
>> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
>> ---
>> v11: add a fixme comment from Jason
>> v10: use NetQueue flush api to flush packets
>>       sent_cb can not be called when we already return size
>> v9: adjustment due to the qapi change
>> v7: use QTAILQ_FOREACH_SAFE() when flush packets
>> v6: move the interval check earlier and some comment adjust
>> v5: remove dummy sent_cb
>>      change interval type from int64 to uint32
>>      check interval!=0 when initialise
>>      rename FILTERBUFFERState to FilterBufferState
>> v4: remove bh
>>      pass the packet to next filter instead of receiver
>> v3: check packet's sender and sender->peer when flush it
>> ---
>>   net/Makefile.objs   |   1 +
>>   net/filter-buffer.c | 170 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>>   qemu-options.hx     |  18 ++++++
>>   vl.c                |   7 ++-
>>   4 files changed, 195 insertions(+), 1 deletion(-)
>>   create mode 100644 net/filter-buffer.c
>>
>> diff --git a/net/Makefile.objs b/net/Makefile.objs
>> index 914aec0..5fa2f97 100644
>> --- a/net/Makefile.objs
>> +++ b/net/Makefile.objs
>> @@ -14,3 +14,4 @@ common-obj-$(CONFIG_SLIRP) += slirp.o
>>   common-obj-$(CONFIG_VDE) += vde.o
>>   common-obj-$(CONFIG_NETMAP) += netmap.o
>>   common-obj-y += filter.o
>> +common-obj-y += filter-buffer.o
>> diff --git a/net/filter-buffer.c b/net/filter-buffer.c
>> new file mode 100644
>> index 0000000..ef94e91
>> --- /dev/null
>> +++ b/net/filter-buffer.c
>> @@ -0,0 +1,170 @@
>> +/*
>> + * Copyright (c) 2015 FUJITSU LIMITED
>> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>> + * later.  See the COPYING file in the top-level directory.
>> + */
>> +
>> +#include "net/filter.h"
>> +#include "net/queue.h"
>> +#include "qemu-common.h"
>> +#include "qemu/timer.h"
>> +#include "qemu/iov.h"
>> +#include "qapi/qmp/qerror.h"
>> +#include "qapi-visit.h"
>> +#include "qom/object.h"
>> +
>> +#define TYPE_FILTER_BUFFER "filter-buffer"
>> +
>> +#define FILTER_BUFFER(obj) \
>> +    OBJECT_CHECK(FilterBufferState, (obj), TYPE_FILTER_BUFFER)
>> +
>> +struct FilterBufferState {
>> +    NetFilterState parent_obj;
>> +
>> +    NetQueue *incoming_queue;
>> +    uint32_t interval;
>> +    QEMUTimer release_timer;
>> +};
>> +typedef struct FilterBufferState FilterBufferState;
>
> Again, not splitting the declaration is more concise.
>
>> +
>> +static void filter_buffer_flush(NetFilterState *nf)
>> +{
>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>> +
>> +    if (!qemu_net_queue_flush(s->incoming_queue)) {
>> +        /* Unable to empty the queue, purge remaining packets */
>> +        qemu_net_queue_purge(s->incoming_queue, nf->netdev);
>> +    }
>> +}
>
> This either flushes or purges incoming_queue, where "purge" means
> dropping packets.  Correct?

I think it is correct. Dropping packets is allowed, even on real
hardware, packets may lose.

>
>> +
>> +static void filter_buffer_release_timer(void *opaque)
>> +{
>> +    NetFilterState *nf = opaque;
>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>
> Style nit: blank line between declarations and statements, please.
>
>> +    filter_buffer_flush(nf);
>
> Is purging correct here?
>
>> +    timer_mod(&s->release_timer,
>> +              qemu_clock_get_us(QEMU_CLOCK_VIRTUAL) + s->interval);
>
> Timer rearmed to fire again in s->interval microseconds.

Yes.

>
>> +}
>> +
>> +/* filter APIs */
>> +static ssize_t filter_buffer_receive_iov(NetFilterState *nf,
>> +                                         NetClientState *sender,
>> +                                         unsigned flags,
>> +                                         const struct iovec *iov,
>> +                                         int iovcnt,
>> +                                         NetPacketSent *sent_cb)
>> +{
>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>> +
>> +    /*
>> +     * we return size when buffer a packet, the sender will take it as
>> +     * a already sent packet, so sent_cb should not be called later
>
> Humor me: when a comment has multiple sentences, start each one with a
> capital letter, and end it with punctuation.

Ok.

>
>> +     * FIXME: even if guest can't receive packet for some reasons. Filter
>> +     * can still accept packet until its internal queue is full.
>> +     */
>
> I'm not sure I understand the comment.

This is taken from Jason's comments, may be he can have a better explain?

>
>> +    qemu_net_queue_append_iov(s->incoming_queue, sender, flags,
>> +                              iov, iovcnt, NULL);
>> +    return iov_size(iov, iovcnt);
>> +}
>> +
>> +static void filter_buffer_cleanup(NetFilterState *nf)
>> +{
>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>> +
>> +    if (s->interval) {
>> +        timer_del(&s->release_timer);
>> +    }
>> +
>> +    /* flush packets */
>> +    if (s->incoming_queue) {
>> +        filter_buffer_flush(nf);
>
> I guess purging is correct here.
>
>> +        g_free(s->incoming_queue);
>> +    }
>> +}
>> +
>> +static void filter_buffer_setup(NetFilterState *nf, Error **errp)
>> +{
>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>> +
>> +    /*
>> +     * this check should be dropped when there're VM FT solutions like MC
>> +     * or COLO use this filter to release packets on demand.
>> +     */
>
> If you end a sentence with a period, you get to start it with a capital
> letter :)

Ok, thanks.

>
> "there're" is odd.  Perhaps something like "We may want to accept zero
> interval when VM FT solutions like MC or COLO use this filter to release
> packets on demand."
>
>> +    if (!s->interval) {
>> +        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "interval",
>> +                   "a non-zero interval");
>
> How can this happen?  Doesn't filter_buffer_set_interval() catch zero
> intervals already?

When user do not supply an interval parameter, filter_buffer_set_interval()
won't be called, and the interval is 0.
When we have actual user like I mentioned above, we should allow user to omit
the interval option.

>
>> +        return;
>> +    }
>> +
>> +    s->incoming_queue = qemu_new_net_queue(qemu_netfilter_pass_to_next, nf);
>> +    if (s->interval) {
>
> Condition cannot be false.  Same in filter_buffer_cleanup().
>
>> +        timer_init_us(&s->release_timer, QEMU_CLOCK_VIRTUAL,
>> +                      filter_buffer_release_timer, nf);
>> +        timer_mod(&s->release_timer,
>> +                  qemu_clock_get_us(QEMU_CLOCK_VIRTUAL) + s->interval);
>
> Timer armed to fire in s->interval microseconds.
>
> Unless I misunderstand something, this doesn't actually delay each
> packet by s->interval microseconds, it batches packet delivery: all
> packets arriving in a given interval are delayed until the end of the
> interval.  Correct?

Correct.

>
>> +    }
>> +}
>> +
>> +static void filter_buffer_class_init(ObjectClass *oc, void *data)
>> +{
>> +    NetFilterClass *nfc = NETFILTER_CLASS(oc);
>> +
>> +    nfc->setup = filter_buffer_setup;
>> +    nfc->cleanup = filter_buffer_cleanup;
>> +    nfc->receive_iov = filter_buffer_receive_iov;
>> +}
>> +
>> +static void filter_buffer_get_interval(Object *obj, Visitor *v, void *opaque,
>> +                                       const char *name, Error **errp)
>> +{
>> +    FilterBufferState *s = FILTER_BUFFER(obj);
>> +    uint32_t value = s->interval;
>> +
>> +    visit_type_uint32(v, &value, name, errp);
>> +}
>> +
>> +static void filter_buffer_set_interval(Object *obj, Visitor *v, void *opaque,
>> +                                       const char *name, Error **errp)
>> +{
>> +    FilterBufferState *s = FILTER_BUFFER(obj);
>> +    Error *local_err = NULL;
>> +    uint32_t value;
>> +
>> +    visit_type_uint32(v, &value, name, &local_err);
>> +    if (local_err) {
>> +        goto out;
>> +    }
>> +    if (!value) {
>> +        error_setg(&local_err, "Property '%s.%s' doesn't take value '%"
>> +                   PRIu32 "'", object_get_typename(obj), name, value);
>
> What does it take?  That's what the user wants to know...  Perhaps
> "Property '%s.%s' requires a positive value".

Maybe better, thanks.

>
>> +        goto out;
>> +    }
>> +    s->interval = value;
>> +
>> +out:
>> +    error_propagate(errp, local_err);
>> +}
>> +
>> +static void filter_buffer_init(Object *obj)
>> +{
>> +    object_property_add(obj, "interval", "int",
>> +                        filter_buffer_get_interval,
>> +                        filter_buffer_set_interval, NULL, NULL, NULL);
>> +}
>> +
>> +static const TypeInfo filter_buffer_info = {
>> +    .name = TYPE_FILTER_BUFFER,
>> +    .parent = TYPE_NETFILTER,
>> +    .class_init = filter_buffer_class_init,
>> +    .instance_init = filter_buffer_init,
>> +    .instance_size = sizeof(FilterBufferState),
>> +};
>> +
>> +static void register_types(void)
>> +{
>> +    type_register_static(&filter_buffer_info);
>> +}
>> +
>> +type_init(register_types);
>> diff --git a/qemu-options.hx b/qemu-options.hx
>> index 7e147b8..b09f97f 100644
>> --- a/qemu-options.hx
>> +++ b/qemu-options.hx
>> @@ -3642,6 +3642,24 @@ in PEM format, in filenames @var{ca-cert.pem}, @var{ca-crl.pem} (optional),
>>   @var{server-cert.pem} (only servers), @var{server-key.pem} (only servers),
>>   @var{client-cert.pem} (only clients), and @var{client-key.pem} (only clients).
>>
>> +@item -object filter-buffer,id=@var{id},netdev=@var{netdevid}[,chain=@var{all|in|out}][,interval=@var{t}]
>> +
>> +Buffer network packets on netdev @var{netdevid}.
>> +If interval @var{t} provided, will release packets by interval.
>> +Interval scale: microsecond.
>> +
>> +If interval @var{t} not provided, you have to make sure the packets can be
>
> Are you sure omitting t works?  filter_buffer_set_interval() rejects
> zero...

When we omit t, filter_buffer_set_interval() won't be called.

>
>> +released, either by manually remove this filter or call the release buffer API,
>> +otherwise, the packets will be buffered forever. Use with caution.
>
> Please wrap your lines a bit earlier.
>
>> +
>> +chain @var{all|in|out} is an option that can be applied to any netfilter, default is @option{all}.
>> +
>> +@option{all} means this filter will receive packets both sent to/from the netdev
>> +
>> +@option{in} means this filter will receive packets sent to the netdev
>> +
>> +@option{out} means this filter will receive packets sent from the netdev
>> +
>
> I'd describe this like "filter is inserted in the receive / transmit
> queue".

thanks.

>
>>   @end table
>>
>>   ETEXI
>> diff --git a/vl.c b/vl.c
>> index ec589e2..3cf89d5 100644
>> --- a/vl.c
>> +++ b/vl.c
>> @@ -2794,7 +2794,12 @@ static bool object_create_initial(const char *type)
>>       if (g_str_equal(type, "rng-egd")) {
>>           return false;
>>       }
>> -    /* TODO: return false for concrete netfilters */
>> +
>> +    /* return false for concrete netfilters */
>
> I find this comment useless, please drop it :)

Ok, thanks.

>
>> +    if (g_str_equal(type, "filter-buffer")) {
>> +        return false;
>> +    }
>> +
>>       return true;
>>   }
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 09/12] netfilter: add a netbuffer filter

[Yang Hongyang]

On 09/24/2015 05:12 PM, Markus Armbruster wrote:
> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
[...]
>> diff --git a/vl.c b/vl.c
>> index ec589e2..3cf89d5 100644
>> --- a/vl.c
>> +++ b/vl.c
>> @@ -2794,7 +2794,12 @@ static bool object_create_initial(const char *type)
>>       if (g_str_equal(type, "rng-egd")) {
>>           return false;
>>       }
>> -    /* TODO: return false for concrete netfilters */
>> +
>> +    /* return false for concrete netfilters */
>
> I find this comment useless, please drop it :)

This might be useful for reminding others who wants to implement other filters.

>
>> +    if (g_str_equal(type, "filter-buffer")) {
>> +        return false;
>> +    }
>> +
>>       return true;
>>   }
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 09/12] netfilter: add a netbuffer filter

[Thomas Huth]

On 25/09/15 10:03, Yang Hongyang wrote:
> 
> 
> On 09/24/2015 05:12 PM, Markus Armbruster wrote:
>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
> [...]
>>> diff --git a/vl.c b/vl.c
>>> index ec589e2..3cf89d5 100644
>>> --- a/vl.c
>>> +++ b/vl.c
>>> @@ -2794,7 +2794,12 @@ static bool object_create_initial(const char
>>> *type)
>>>       if (g_str_equal(type, "rng-egd")) {
>>>           return false;
>>>       }
>>> -    /* TODO: return false for concrete netfilters */
>>> +
>>> +    /* return false for concrete netfilters */
>>
>> I find this comment useless, please drop it :)
> 
> This might be useful for reminding others who wants to implement other
> filters.

I think the comment should explain why the code is return false here,
not what the code is doing (which is obvious). So maybe something like:

    /*
     * netfilters require that the corresponding
     * netdevs are already existing
     */

?

 Thomas

Re: [Qemu-devel] [PATCH v11 09/12] netfilter: add a netbuffer filter

[Jason Wang]

On 09/25/2015 03:18 PM, Yang Hongyang wrote:
>
>
> On 09/24/2015 05:12 PM, Markus Armbruster wrote:
>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>
>>> This filter is to buffer/release packets, this feature can be used
>>> when using MicroCheckpointing, or other Remus like VM FT solutions, you
>>
>> What's "Remus"?
>
> Remus is an opensource VM FT solution:
> paper: http://http//www.cs.ubc.ca/~andy/papers/remus-nsdi-final.pdf
> First implemented on Xen:
> http://wiki.xen.org/wiki/Remus
>
> MicroCheckpointing in QEMU is another Remus implementation.
>
>

[...]

>>
>>> +     * FIXME: even if guest can't receive packet for some reasons.
>>> Filter
>>> +     * can still accept packet until its internal queue is full.
>>> +     */
>>
>> I'm not sure I understand the comment.
>
> This is taken from Jason's comments, may be he can have a better explain?

For example. For some reason, receiver could not receive more packets
(.can_receive() returns zero). Without a filter, at most one packet will
be queued in incoming queue and sender's poll will be disabled unit its
sent_cb() was called. With a filter, it will keep receive the packets
without caring about the receiver. This is suboptimal. May need more
thoughts (e.g keeping sent_cb).

[...]

Re: [Qemu-devel] [PATCH v11 09/12] netfilter: add a netbuffer filter

[Yang Hongyang]

On 09/25/2015 04:18 PM, Thomas Huth wrote:
> On 25/09/15 10:03, Yang Hongyang wrote:
>>
>>
>> On 09/24/2015 05:12 PM, Markus Armbruster wrote:
>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>> [...]
>>>> diff --git a/vl.c b/vl.c
>>>> index ec589e2..3cf89d5 100644
>>>> --- a/vl.c
>>>> +++ b/vl.c
>>>> @@ -2794,7 +2794,12 @@ static bool object_create_initial(const char
>>>> *type)
>>>>        if (g_str_equal(type, "rng-egd")) {
>>>>            return false;
>>>>        }
>>>> -    /* TODO: return false for concrete netfilters */
>>>> +
>>>> +    /* return false for concrete netfilters */
>>>
>>> I find this comment useless, please drop it :)
>>
>> This might be useful for reminding others who wants to implement other
>> filters.
>
> I think the comment should explain why the code is return false here,
> not what the code is doing (which is obvious). So maybe something like:
>
>      /*
>       * netfilters require that the corresponding
>       * netdevs are already existing
>       */

Makes more sense, thanks:)

>
> ?
>
>   Thomas
>
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Markus Armbruster]

Jason Wang <jasowang@redhat.com> writes:

> On 09/24/2015 07:52 PM, Markus Armbruster wrote:
>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>
>>> On 09/24/2015 04:41 PM, Markus Armbruster wrote:
>>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
[...]
>>>>> diff --git a/qapi-schema.json b/qapi-schema.json
>>>>> index 2bada60..546500a 100644
>>>>> --- a/qapi-schema.json
>>>>> +++ b/qapi-schema.json
>>>>> @@ -2551,6 +2551,24 @@
>>>>>       'opts': 'NetClientOptions' } }
>>>>>
>>>>>   ##
>>>>> +# @NetFilterChain
>>>>> +#
>>>>> +# netfilter chain, a netfilter is attached to a netdev, captures the
>>>>> +# network packets of the netdev.
>>>>> +#
>>>>> +# @all: the filter will receive packets both sent to/from the netdev, this
>>>>> +#       is the default chain.
>>>>> +#
>>>>> +# @in: the filter will receive packets sent to the netdev.
>>>>> +#
>>>>> +# @out: the filter will receive packets sent from the netdev.
>>>> Uh, inhowfar is this a "chain"?  As far as I can tell, it specifies
>>>> whether a netfilter is attached to the transmit queue, the receive
>>>> queue, or both.
>>> netback's input chain or output chain, queue is mostly like a word that
>>> related to the code implementation? English is not my first language, so
>>> I'm not sure about this.
>> Don't worry, we'll get the language polished together :)
>>
>> In PATCH 9, I learned how this is to be used:
>>
>>     @item -object
>> filter-buffer,id=@var{id},netdev=@var{netdevid}[,chain=@var{all|in|out}][,interval=@var{t}]
>>
>>     Buffer network packets on netdev @var{netdevid}.
>> [...]
>>     chain @var{all|in|out} is an option that can be applied to any
>> netfilter, default is @option{all}.
>>
>>     @option{all} means this filter will receive packets both sent
>> to/from the netdev
>>
>>     @option{in} means this filter will receive packets sent to the netdev
>>
>>     @option{out} means this filter will receive packets sent from the netdev
>>
>> In that context, "chain" makes more sense.  I'd still call it "queue".
>> Precedence: -netdev parameter "queues".  If I understand correctly,
>> queues=N asks for N rx and tx queues.  A filter would either apply to
>> all N rx queues, all N tx queues, or all 2*N queues.  Correct?
>>
>> [...]
>
> Yes.

I think "queue=rx", "queue=tx" and "queue=all" would make the most
sense.  Unless there are more queues than rx and rx; then we'd have to
reconsider "queue=all".

Call the enumeration type NetFilterDirection instead of NetFilterChain.

Re: [Qemu-devel] [PATCH v11 09/12] netfilter: add a netbuffer filter

[Markus Armbruster]

Yang Hongyang <yanghy@cn.fujitsu.com> writes:

> On 09/24/2015 05:12 PM, Markus Armbruster wrote:
>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>
>>> This filter is to buffer/release packets, this feature can be used
>>> when using MicroCheckpointing, or other Remus like VM FT solutions, you
>>
>> What's "Remus"?
>
> Remus is an opensource VM FT solution:
> paper: http://http//www.cs.ubc.ca/~andy/papers/remus-nsdi-final.pdf
> First implemented on Xen:
> http://wiki.xen.org/wiki/Remus
>
> MicroCheckpointing in QEMU is another Remus implementation.
>
>>
>>> can also use it to simulate the network delay.
>>> It has an interval option, if supplied, this filter will release
>>> packets by interval.
>>
>> Suggest "will delay packets by that time interval."
>>
>> Is interval really optional?
>
> It supposed to be optional. When the buffer filter has a user:
> http://lists.nongnu.org/archive/html/qemu-devel/2015-09/msg00363.html
>
> In this patchset, the zero interval check is removed. packets are
> released on demand through filter_buffer_release_all() api call.

Understand.  But is interval optional right at this point in this patch
series?

>>
>>>
>>> Usage:
>>>   -netdev tap,id=bn0
>>>   -object filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000
>>>
>>> NOTE:
>>>   the scale of interval is microsecond.
>>
>> Perhaps "interval is in microseconds".
>
> Better, thanks.
>
>>
>>>
>>> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
>>> ---
>>> v11: add a fixme comment from Jason
>>> v10: use NetQueue flush api to flush packets
>>>       sent_cb can not be called when we already return size
>>> v9: adjustment due to the qapi change
>>> v7: use QTAILQ_FOREACH_SAFE() when flush packets
>>> v6: move the interval check earlier and some comment adjust
>>> v5: remove dummy sent_cb
>>>      change interval type from int64 to uint32
>>>      check interval!=0 when initialise
>>>      rename FILTERBUFFERState to FilterBufferState
>>> v4: remove bh
>>>      pass the packet to next filter instead of receiver
>>> v3: check packet's sender and sender->peer when flush it
>>> ---
>>>   net/Makefile.objs   |   1 +
>>>   net/filter-buffer.c | 170 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>   qemu-options.hx     |  18 ++++++
>>>   vl.c                |   7 ++-
>>>   4 files changed, 195 insertions(+), 1 deletion(-)
>>>   create mode 100644 net/filter-buffer.c
>>>
>>> diff --git a/net/Makefile.objs b/net/Makefile.objs
>>> index 914aec0..5fa2f97 100644
>>> --- a/net/Makefile.objs
>>> +++ b/net/Makefile.objs
>>> @@ -14,3 +14,4 @@ common-obj-$(CONFIG_SLIRP) += slirp.o
>>>   common-obj-$(CONFIG_VDE) += vde.o
>>>   common-obj-$(CONFIG_NETMAP) += netmap.o
>>>   common-obj-y += filter.o
>>> +common-obj-y += filter-buffer.o
>>> diff --git a/net/filter-buffer.c b/net/filter-buffer.c
>>> new file mode 100644
>>> index 0000000..ef94e91
>>> --- /dev/null
>>> +++ b/net/filter-buffer.c
>>> @@ -0,0 +1,170 @@
>>> +/*
>>> + * Copyright (c) 2015 FUJITSU LIMITED
>>> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
>>> + *
>>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>>> + * later.  See the COPYING file in the top-level directory.
>>> + */
>>> +
>>> +#include "net/filter.h"
>>> +#include "net/queue.h"
>>> +#include "qemu-common.h"
>>> +#include "qemu/timer.h"
>>> +#include "qemu/iov.h"
>>> +#include "qapi/qmp/qerror.h"
>>> +#include "qapi-visit.h"
>>> +#include "qom/object.h"
>>> +
>>> +#define TYPE_FILTER_BUFFER "filter-buffer"
>>> +
>>> +#define FILTER_BUFFER(obj) \
>>> +    OBJECT_CHECK(FilterBufferState, (obj), TYPE_FILTER_BUFFER)
>>> +
>>> +struct FilterBufferState {
>>> +    NetFilterState parent_obj;
>>> +
>>> +    NetQueue *incoming_queue;
>>> +    uint32_t interval;
>>> +    QEMUTimer release_timer;
>>> +};
>>> +typedef struct FilterBufferState FilterBufferState;
>>
>> Again, not splitting the declaration is more concise.
>>
>>> +
>>> +static void filter_buffer_flush(NetFilterState *nf)
>>> +{
>>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>>> +
>>> +    if (!qemu_net_queue_flush(s->incoming_queue)) {
>>> +        /* Unable to empty the queue, purge remaining packets */
>>> +        qemu_net_queue_purge(s->incoming_queue, nf->netdev);
>>> +    }
>>> +}
>>
>> This either flushes or purges incoming_queue, where "purge" means
>> dropping packets.  Correct?
>
> I think it is correct. Dropping packets is allowed, even on real
> hardware, packets may lose.
>
>>
>>> +
>>> +static void filter_buffer_release_timer(void *opaque)
>>> +{
>>> +    NetFilterState *nf = opaque;
>>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>>
>> Style nit: blank line between declarations and statements, please.
>>
>>> +    filter_buffer_flush(nf);
>>
>> Is purging correct here?

When the timer expires, we flush as many buffered packets as we can,
then throw away the rest.  Why throw them away?  Shouldn't we leave them
in the buffer, and only throw away packets when the buffer is full?

>>> +    timer_mod(&s->release_timer,
>>> +              qemu_clock_get_us(QEMU_CLOCK_VIRTUAL) + s->interval);
>>
>> Timer rearmed to fire again in s->interval microseconds.
>
> Yes.
>
>>
>>> +}
>>> +
>>> +/* filter APIs */
>>> +static ssize_t filter_buffer_receive_iov(NetFilterState *nf,
>>> +                                         NetClientState *sender,
>>> +                                         unsigned flags,
>>> +                                         const struct iovec *iov,
>>> +                                         int iovcnt,
>>> +                                         NetPacketSent *sent_cb)
>>> +{
>>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>>> +
>>> +    /*
>>> +     * we return size when buffer a packet, the sender will take it as
>>> +     * a already sent packet, so sent_cb should not be called later
>>
>> Humor me: when a comment has multiple sentences, start each one with a
>> capital letter, and end it with punctuation.
>
> Ok.
>
>>
>>> +     * FIXME: even if guest can't receive packet for some reasons. Filter
>>> +     * can still accept packet until its internal queue is full.
>>> +     */
>>
>> I'm not sure I understand the comment.
>
> This is taken from Jason's comments, may be he can have a better explain?
>
>>
>>> +    qemu_net_queue_append_iov(s->incoming_queue, sender, flags,
>>> +                              iov, iovcnt, NULL);
>>> +    return iov_size(iov, iovcnt);
>>> +}
>>> +
>>> +static void filter_buffer_cleanup(NetFilterState *nf)
>>> +{
>>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>>> +
>>> +    if (s->interval) {
>>> +        timer_del(&s->release_timer);
>>> +    }
>>> +
>>> +    /* flush packets */
>>> +    if (s->incoming_queue) {
>>> +        filter_buffer_flush(nf);
>>
>> I guess purging is correct here.
>>
>>> +        g_free(s->incoming_queue);
>>> +    }
>>> +}
>>> +
>>> +static void filter_buffer_setup(NetFilterState *nf, Error **errp)
>>> +{
>>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>>> +
>>> +    /*
>>> +     * this check should be dropped when there're VM FT solutions like MC
>>> +     * or COLO use this filter to release packets on demand.
>>> +     */
>>
>> If you end a sentence with a period, you get to start it with a capital
>> letter :)
>
> Ok, thanks.
>
>>
>> "there're" is odd.  Perhaps something like "We may want to accept zero
>> interval when VM FT solutions like MC or COLO use this filter to release
>> packets on demand."
>>
>>> +    if (!s->interval) {
>>> +        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "interval",
>>> +                   "a non-zero interval");
>>
>> How can this happen?  Doesn't filter_buffer_set_interval() catch zero
>> intervals already?
>
> When user do not supply an interval parameter, filter_buffer_set_interval()
> won't be called, and the interval is 0.
> When we have actual user like I mentioned above, we should allow user to omit
> the interval option.

Let's see whether I understand.

If you specify a non-zero interval, filter_buffer_set_interval() stores
it.

If you specify a zero interval, filter_buffer_set_interval() rejects it.

If you specify no interval, it defaults to zero, and
filter_buffer_setup() fails right here.

Therefore, interval is not optional right now.

Correct?

>>> +        return;
>>> +    }
>>> +
>>> +    s->incoming_queue = qemu_new_net_queue(qemu_netfilter_pass_to_next, nf);
>>> +    if (s->interval) {
>>
>> Condition cannot be false.  Same in filter_buffer_cleanup().
>>
>>> +        timer_init_us(&s->release_timer, QEMU_CLOCK_VIRTUAL,
>>> +                      filter_buffer_release_timer, nf);
>>> +        timer_mod(&s->release_timer,
>>> +                  qemu_clock_get_us(QEMU_CLOCK_VIRTUAL) + s->interval);
>>
>> Timer armed to fire in s->interval microseconds.
>>
>> Unless I misunderstand something, this doesn't actually delay each
>> packet by s->interval microseconds, it batches packet delivery: all
>> packets arriving in a given interval are delayed until the end of the
>> interval.  Correct?
>
> Correct.

Documentation needs to spell that out.

>>> +    }
>>> +}
>>> +
>>> +static void filter_buffer_class_init(ObjectClass *oc, void *data)
>>> +{
>>> +    NetFilterClass *nfc = NETFILTER_CLASS(oc);
>>> +
>>> +    nfc->setup = filter_buffer_setup;
>>> +    nfc->cleanup = filter_buffer_cleanup;
>>> +    nfc->receive_iov = filter_buffer_receive_iov;
>>> +}
>>> +
>>> +static void filter_buffer_get_interval(Object *obj, Visitor *v,
>>> void *opaque,
>>> +                                       const char *name, Error **errp)
>>> +{
>>> +    FilterBufferState *s = FILTER_BUFFER(obj);
>>> +    uint32_t value = s->interval;
>>> +
>>> +    visit_type_uint32(v, &value, name, errp);
>>> +}
>>> +
>>> +static void filter_buffer_set_interval(Object *obj, Visitor *v,
>>> void *opaque,
>>> +                                       const char *name, Error **errp)
>>> +{
>>> +    FilterBufferState *s = FILTER_BUFFER(obj);
>>> +    Error *local_err = NULL;
>>> +    uint32_t value;
>>> +
>>> +    visit_type_uint32(v, &value, name, &local_err);
>>> +    if (local_err) {
>>> +        goto out;
>>> +    }
>>> +    if (!value) {
>>> +        error_setg(&local_err, "Property '%s.%s' doesn't take value '%"
>>> +                   PRIu32 "'", object_get_typename(obj), name, value);
>>
>> What does it take?  That's what the user wants to know...  Perhaps
>> "Property '%s.%s' requires a positive value".
>
> Maybe better, thanks.
>
>>
>>> +        goto out;
>>> +    }
>>> +    s->interval = value;
>>> +
>>> +out:
>>> +    error_propagate(errp, local_err);
>>> +}
>>> +
>>> +static void filter_buffer_init(Object *obj)
>>> +{
>>> +    object_property_add(obj, "interval", "int",
>>> +                        filter_buffer_get_interval,
>>> +                        filter_buffer_set_interval, NULL, NULL, NULL);
>>> +}
>>> +
>>> +static const TypeInfo filter_buffer_info = {
>>> +    .name = TYPE_FILTER_BUFFER,
>>> +    .parent = TYPE_NETFILTER,
>>> +    .class_init = filter_buffer_class_init,
>>> +    .instance_init = filter_buffer_init,
>>> +    .instance_size = sizeof(FilterBufferState),
>>> +};
>>> +
>>> +static void register_types(void)
>>> +{
>>> +    type_register_static(&filter_buffer_info);
>>> +}
>>> +
>>> +type_init(register_types);
>>> diff --git a/qemu-options.hx b/qemu-options.hx
>>> index 7e147b8..b09f97f 100644
>>> --- a/qemu-options.hx
>>> +++ b/qemu-options.hx
>>> @@ -3642,6 +3642,24 @@ in PEM format, in filenames
>>> @var{ca-cert.pem}, @var{ca-crl.pem} (optional),
>>>   @var{server-cert.pem} (only servers), @var{server-key.pem} (only servers),
>>>   @var{client-cert.pem} (only clients), and @var{client-key.pem}
>>> (only clients).
>>>
>>> +@item -object
>>> filter-buffer,id=@var{id},netdev=@var{netdevid}[,chain=@var{all|in|out}][,interval=@var{t}]
>>> +
>>> +Buffer network packets on netdev @var{netdevid}.
>>> +If interval @var{t} provided, will release packets by interval.
>>> +Interval scale: microsecond.
>>> +
>>> +If interval @var{t} not provided, you have to make sure the packets can be
>>
>> Are you sure omitting t works?  filter_buffer_set_interval() rejects
>> zero...
>
> When we omit t, filter_buffer_set_interval() won't be called.

But filter_buffer_setup() will be, and it will fail, won't it?

>>> +released, either by manually remove this filter or call the
>>> release buffer API,
>>> +otherwise, the packets will be buffered forever. Use with caution.
>>
>> Please wrap your lines a bit earlier.
>>
>>> +
>>> +chain @var{all|in|out} is an option that can be applied to any
>>> netfilter, default is @option{all}.
>>> +
>>> +@option{all} means this filter will receive packets both sent
>>> to/from the netdev
>>> +
>>> +@option{in} means this filter will receive packets sent to the netdev
>>> +
>>> +@option{out} means this filter will receive packets sent from the netdev
>>> +
>>
>> I'd describe this like "filter is inserted in the receive / transmit
>> queue".
>
> thanks.
>
>>
>>>   @end table
>>>
>>>   ETEXI
>>> diff --git a/vl.c b/vl.c
>>> index ec589e2..3cf89d5 100644
>>> --- a/vl.c
>>> +++ b/vl.c
>>> @@ -2794,7 +2794,12 @@ static bool object_create_initial(const char *type)
>>>       if (g_str_equal(type, "rng-egd")) {
>>>           return false;
>>>       }
>>> -    /* TODO: return false for concrete netfilters */
>>> +
>>> +    /* return false for concrete netfilters */
>>
>> I find this comment useless, please drop it :)
>
> Ok, thanks.
>
>>
>>> +    if (g_str_equal(type, "filter-buffer")) {
>>> +        return false;
>>> +    }
>>> +
>>>       return true;
>>>   }
>> .
>>

Re: [Qemu-devel] [PATCH v11 09/12] netfilter: add a netbuffer filter

[Markus Armbruster]

Jason Wang <jasowang@redhat.com> writes:

> On 09/25/2015 03:18 PM, Yang Hongyang wrote:
>>
>>
>> On 09/24/2015 05:12 PM, Markus Armbruster wrote:
>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
[...]
>>>> +static ssize_t filter_buffer_receive_iov(NetFilterState *nf,
>>>> +                                         NetClientState *sender,
>>>> +                                         unsigned flags,
>>>> +                                         const struct iovec *iov,
>>>> +                                         int iovcnt,
>>>> +                                         NetPacketSent *sent_cb)
>>>> +{
>>>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>>>> +
>>>> +    /*
>>>> +     * we return size when buffer a packet, the sender will take it as
>>>> +     * a already sent packet, so sent_cb should not be called later
[...]
>>>> +     * FIXME: even if guest can't receive packet for some reasons.
>>>> Filter
>>>> +     * can still accept packet until its internal queue is full.
>>>> +     */
>>>
>>> I'm not sure I understand the comment.
>>
>> This is taken from Jason's comments, may be he can have a better explain?
>
> For example. For some reason, receiver could not receive more packets
> (.can_receive() returns zero). Without a filter, at most one packet will
> be queued in incoming queue and sender's poll will be disabled unit its
> sent_cb() was called. With a filter, it will keep receive the packets
> without caring about the receiver. This is suboptimal. May need more
> thoughts (e.g keeping sent_cb).

Aha.  Perhaps you can work that explanation into the comment.

Re: [Qemu-devel] [PATCH v11 09/12] netfilter: add a netbuffer filter

[Markus Armbruster]

Thomas Huth <thuth@redhat.com> writes:

> On 25/09/15 10:03, Yang Hongyang wrote:
>> 
>> 
>> On 09/24/2015 05:12 PM, Markus Armbruster wrote:
>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>> [...]
>>>> diff --git a/vl.c b/vl.c
>>>> index ec589e2..3cf89d5 100644
>>>> --- a/vl.c
>>>> +++ b/vl.c
>>>> @@ -2794,7 +2794,12 @@ static bool object_create_initial(const char
>>>> *type)
      /*
       * Initial object creation happens before all other
       * QEMU data types are created. The majority of objects
       * can be created at this point. The rng-egd object
       * cannot be created here, as it depends on the chardev
       * already existing.
       */
      static bool object_create_initial(const char *type)
      {
>>>>      if (g_str_equal(type, "rng-egd")) {
>>>>          return false;
>>>>      }
>>>> -    /* TODO: return false for concrete netfilters */
>>>> +
>>>> +    /* return false for concrete netfilters */
     +    if (g_str_equal(type, "filter-buffer")) {
     +        return false;
     +    }
     +
          return true;
      }
>>>
>>> I find this comment useless, please drop it :)
>> 
>> This might be useful for reminding others who wants to implement other
>> filters.

I think as soon as there's one, how to add more is obvious enough.

> I think the comment should explain why the code is return false here,
> not what the code is doing (which is obvious). So maybe something like:
>
>     /*
>      * netfilters require that the corresponding
>      * netdevs are already existing
>      */
>
> ?

That explains *why* netfilters need to be initialized late.  More
useful.

The pre-existing case is explained in the function comment.  Which
becomes misleading at this point.

    /*
     * Can @type objects be created during initial object creation?
     * Initial object creation happens before all other QEMU data types
     * are created.  The majority of objects can be created at this
     * point.  Some can't, because the depend on other things already
     * existing.
     */
    static bool object_create_initial(const char *type)
    {
        /* rng-egd depends on its chardev already existing */
        if (g_str_equal(type, "rng-egd")) {
            return false;
        }

        /* Net filters depend on their netdev */
        if (g_str_equal(type, "filter-buffer")) {
            return false;
        }

        return true;
    }

Having to enumerate alll the net filter names here isn't nice.  An "is
subtype of TYPE_NETFILTER" test would be better.  Can't say whether it's
practical.

Of course, the real solution is to create objects either in topological
or in command line order, but that's not in the cards right now.

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Jason Wang]

On 09/25/2015 10:10 PM, Markus Armbruster wrote:
> Jason Wang <jasowang@redhat.com> writes:
>
>> On 09/24/2015 07:52 PM, Markus Armbruster wrote:
>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>>
>>>> On 09/24/2015 04:41 PM, Markus Armbruster wrote:
>>>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
> [...]
>>>>>> diff --git a/qapi-schema.json b/qapi-schema.json
>>>>>> index 2bada60..546500a 100644
>>>>>> --- a/qapi-schema.json
>>>>>> +++ b/qapi-schema.json
>>>>>> @@ -2551,6 +2551,24 @@
>>>>>>       'opts': 'NetClientOptions' } }
>>>>>>
>>>>>>   ##
>>>>>> +# @NetFilterChain
>>>>>> +#
>>>>>> +# netfilter chain, a netfilter is attached to a netdev, captures the
>>>>>> +# network packets of the netdev.
>>>>>> +#
>>>>>> +# @all: the filter will receive packets both sent to/from the netdev, this
>>>>>> +#       is the default chain.
>>>>>> +#
>>>>>> +# @in: the filter will receive packets sent to the netdev.
>>>>>> +#
>>>>>> +# @out: the filter will receive packets sent from the netdev.
>>>>> Uh, inhowfar is this a "chain"?  As far as I can tell, it specifies
>>>>> whether a netfilter is attached to the transmit queue, the receive
>>>>> queue, or both.
>>>> netback's input chain or output chain, queue is mostly like a word that
>>>> related to the code implementation? English is not my first language, so
>>>> I'm not sure about this.
>>> Don't worry, we'll get the language polished together :)
>>>
>>> In PATCH 9, I learned how this is to be used:
>>>
>>>     @item -object
>>> filter-buffer,id=@var{id},netdev=@var{netdevid}[,chain=@var{all|in|out}][,interval=@var{t}]
>>>
>>>     Buffer network packets on netdev @var{netdevid}.
>>> [...]
>>>     chain @var{all|in|out} is an option that can be applied to any
>>> netfilter, default is @option{all}.
>>>
>>>     @option{all} means this filter will receive packets both sent
>>> to/from the netdev
>>>
>>>     @option{in} means this filter will receive packets sent to the netdev
>>>
>>>     @option{out} means this filter will receive packets sent from the netdev
>>>
>>> In that context, "chain" makes more sense.  I'd still call it "queue".
>>> Precedence: -netdev parameter "queues".  If I understand correctly,
>>> queues=N asks for N rx and tx queues.  A filter would either apply to
>>> all N rx queues, all N tx queues, or all 2*N queues.  Correct?
>>>
>>> [...]
>> Yes.
> I think "queue=rx", "queue=tx" and "queue=all" would make the most
> sense.  Unless there are more queues than rx and rx; then we'd have to
> reconsider "queue=all".
>
> Call the enumeration type NetFilterDirection instead of NetFilterChain.

Ok, this sounds better.

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Yang Hongyang]

On 09/28/2015 01:47 PM, Jason Wang wrote:
>
>
> On 09/25/2015 10:10 PM, Markus Armbruster wrote:
>> Jason Wang <jasowang@redhat.com> writes:
>>
>>> On 09/24/2015 07:52 PM, Markus Armbruster wrote:
>>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>>>
>>>>> On 09/24/2015 04:41 PM, Markus Armbruster wrote:
>>>>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>> [...]
>>>>>>> diff --git a/qapi-schema.json b/qapi-schema.json
>>>>>>> index 2bada60..546500a 100644
>>>>>>> --- a/qapi-schema.json
>>>>>>> +++ b/qapi-schema.json
>>>>>>> @@ -2551,6 +2551,24 @@
>>>>>>>        'opts': 'NetClientOptions' } }
>>>>>>>
>>>>>>>    ##
>>>>>>> +# @NetFilterChain
>>>>>>> +#
>>>>>>> +# netfilter chain, a netfilter is attached to a netdev, captures the
>>>>>>> +# network packets of the netdev.
>>>>>>> +#
>>>>>>> +# @all: the filter will receive packets both sent to/from the netdev, this
>>>>>>> +#       is the default chain.
>>>>>>> +#
>>>>>>> +# @in: the filter will receive packets sent to the netdev.
>>>>>>> +#
>>>>>>> +# @out: the filter will receive packets sent from the netdev.
>>>>>> Uh, inhowfar is this a "chain"?  As far as I can tell, it specifies
>>>>>> whether a netfilter is attached to the transmit queue, the receive
>>>>>> queue, or both.
>>>>> netback's input chain or output chain, queue is mostly like a word that
>>>>> related to the code implementation? English is not my first language, so
>>>>> I'm not sure about this.
>>>> Don't worry, we'll get the language polished together :)
>>>>
>>>> In PATCH 9, I learned how this is to be used:
>>>>
>>>>      @item -object
>>>> filter-buffer,id=@var{id},netdev=@var{netdevid}[,chain=@var{all|in|out}][,interval=@var{t}]
>>>>
>>>>      Buffer network packets on netdev @var{netdevid}.
>>>> [...]
>>>>      chain @var{all|in|out} is an option that can be applied to any
>>>> netfilter, default is @option{all}.
>>>>
>>>>      @option{all} means this filter will receive packets both sent
>>>> to/from the netdev
>>>>
>>>>      @option{in} means this filter will receive packets sent to the netdev
>>>>
>>>>      @option{out} means this filter will receive packets sent from the netdev
>>>>
>>>> In that context, "chain" makes more sense.  I'd still call it "queue".
>>>> Precedence: -netdev parameter "queues".  If I understand correctly,
>>>> queues=N asks for N rx and tx queues.  A filter would either apply to
>>>> all N rx queues, all N tx queues, or all 2*N queues.  Correct?
>>>>
>>>> [...]
>>> Yes.
>> I think "queue=rx", "queue=tx" and "queue=all" would make the most
>> sense.  Unless there are more queues than rx and rx; then we'd have to
>> reconsider "queue=all".
>>
>> Call the enumeration type NetFilterDirection instead of NetFilterChain.
>
> Ok, this sounds better.

Ok, I will respin a v12, thanks.

> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 09/12] netfilter: add a netbuffer filter

[Jason Wang]

On 09/25/2015 11:07 PM, Markus Armbruster wrote:
> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>
>> On 09/24/2015 05:12 PM, Markus Armbruster wrote:
>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>>
>>>> This filter is to buffer/release packets, this feature can be used
>>>> when using MicroCheckpointing, or other Remus like VM FT solutions, you
>>> What's "Remus"?

[...]

>>>
>>>> +
>>>> +static void filter_buffer_release_timer(void *opaque)
>>>> +{
>>>> +    NetFilterState *nf = opaque;
>>>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>>> Style nit: blank line between declarations and statements, please.
>>>
>>>> +    filter_buffer_flush(nf);
>>> Is purging correct here?
> When the timer expires, we flush as many buffered packets as we can,
> then throw away the rest.  Why throw them away?  Shouldn't we leave them
> in the buffer, and only throw away packets when the buffer is full?

May need a "FIXME" or "TODO" here. I think this is for simplicity. We
could queue the packet if the receiver or next filter could not receive
packets. But currently there's no way for the next filter or recivier to
notify us that it can receive more packet. This could be done in the future.

Re: [Qemu-devel] [PATCH v11 09/12] netfilter: add a netbuffer filter

[Yang Hongyang]

On 09/25/2015 11:26 PM, Markus Armbruster wrote:
> Thomas Huth <thuth@redhat.com> writes:
>
>> On 25/09/15 10:03, Yang Hongyang wrote:
>>>
>>>
>>> On 09/24/2015 05:12 PM, Markus Armbruster wrote:
>>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>> [...]
>>>>> diff --git a/vl.c b/vl.c
>>>>> index ec589e2..3cf89d5 100644
>>>>> --- a/vl.c
>>>>> +++ b/vl.c
>>>>> @@ -2794,7 +2794,12 @@ static bool object_create_initial(const char
>>>>> *type)
>        /*
>         * Initial object creation happens before all other
>         * QEMU data types are created. The majority of objects
>         * can be created at this point. The rng-egd object
>         * cannot be created here, as it depends on the chardev
>         * already existing.
>         */
>        static bool object_create_initial(const char *type)
>        {
>>>>>       if (g_str_equal(type, "rng-egd")) {
>>>>>           return false;
>>>>>       }
>>>>> -    /* TODO: return false for concrete netfilters */
>>>>> +
>>>>> +    /* return false for concrete netfilters */
>       +    if (g_str_equal(type, "filter-buffer")) {
>       +        return false;
>       +    }
>       +
>            return true;
>        }
>>>>
>>>> I find this comment useless, please drop it :)
>>>
>>> This might be useful for reminding others who wants to implement other
>>> filters.
>
> I think as soon as there's one, how to add more is obvious enough.
>
>> I think the comment should explain why the code is return false here,
>> not what the code is doing (which is obvious). So maybe something like:
>>
>>      /*
>>       * netfilters require that the corresponding
>>       * netdevs are already existing
>>       */
>>
>> ?
>
> That explains *why* netfilters need to be initialized late.  More
> useful.
>
> The pre-existing case is explained in the function comment.  Which
> becomes misleading at this point.
>
>      /*
>       * Can @type objects be created during initial object creation?
>       * Initial object creation happens before all other QEMU data types
>       * are created.  The majority of objects can be created at this
>       * point.  Some can't, because the depend on other things already
>       * existing.
>       */
>      static bool object_create_initial(const char *type)
>      {
>          /* rng-egd depends on its chardev already existing */
>          if (g_str_equal(type, "rng-egd")) {
>              return false;
>          }
>
>          /* Net filters depend on their netdev */
>          if (g_str_equal(type, "filter-buffer")) {
>              return false;
>          }
>
>          return true;
>      }
>
> Having to enumerate alll the net filter names here isn't nice.  An "is
> subtype of TYPE_NETFILTER" test would be better.  Can't say whether it's
> practical.
>
> Of course, the real solution is to create objects either in topological
> or in command line order, but that's not in the cards right now.

Further work should needed for this, currently, I will remove the comment
as you wish, thanks.

> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 09/12] netfilter: add a netbuffer filter

[Yang Hongyang]

On 09/25/2015 11:07 PM, Markus Armbruster wrote:
> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>
>> On 09/24/2015 05:12 PM, Markus Armbruster wrote:
>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>>
>>>> This filter is to buffer/release packets, this feature can be used
>>>> when using MicroCheckpointing, or other Remus like VM FT solutions, you
>>>
>>> What's "Remus"?
>>
>> Remus is an opensource VM FT solution:
>> paper: http://http//www.cs.ubc.ca/~andy/papers/remus-nsdi-final.pdf
>> First implemented on Xen:
>> http://wiki.xen.org/wiki/Remus
>>
>> MicroCheckpointing in QEMU is another Remus implementation.
>>
>>>
>>>> can also use it to simulate the network delay.
>>>> It has an interval option, if supplied, this filter will release
>>>> packets by interval.
>>>
>>> Suggest "will delay packets by that time interval."
>>>
>>> Is interval really optional?
>>
>> It supposed to be optional. When the buffer filter has a user:
>> http://lists.nongnu.org/archive/html/qemu-devel/2015-09/msg00363.html
>>
>> In this patchset, the zero interval check is removed. packets are
>> released on demand through filter_buffer_release_all() api call.
>
> Understand.  But is interval optional right at this point in this patch
> series?

It's not, will remove this optional description although we
will make it optional soon.

>
>>>
>>>>
>>>> Usage:
>>>>    -netdev tap,id=bn0
>>>>    -object filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000
>>>>
>>>> NOTE:
>>>>    the scale of interval is microsecond.
>>>
>>> Perhaps "interval is in microseconds".
>>
>> Better, thanks.
>>
>>>
>>>>
>>>> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
>>>> ---
>>>> v11: add a fixme comment from Jason
>>>> v10: use NetQueue flush api to flush packets
>>>>        sent_cb can not be called when we already return size
>>>> v9: adjustment due to the qapi change
>>>> v7: use QTAILQ_FOREACH_SAFE() when flush packets
>>>> v6: move the interval check earlier and some comment adjust
>>>> v5: remove dummy sent_cb
>>>>       change interval type from int64 to uint32
>>>>       check interval!=0 when initialise
>>>>       rename FILTERBUFFERState to FilterBufferState
>>>> v4: remove bh
>>>>       pass the packet to next filter instead of receiver
>>>> v3: check packet's sender and sender->peer when flush it
>>>> ---
>>>>    net/Makefile.objs   |   1 +
>>>>    net/filter-buffer.c | 170 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>    qemu-options.hx     |  18 ++++++
>>>>    vl.c                |   7 ++-
>>>>    4 files changed, 195 insertions(+), 1 deletion(-)
>>>>    create mode 100644 net/filter-buffer.c
>>>>
>>>> diff --git a/net/Makefile.objs b/net/Makefile.objs
>>>> index 914aec0..5fa2f97 100644
>>>> --- a/net/Makefile.objs
>>>> +++ b/net/Makefile.objs
>>>> @@ -14,3 +14,4 @@ common-obj-$(CONFIG_SLIRP) += slirp.o
>>>>    common-obj-$(CONFIG_VDE) += vde.o
>>>>    common-obj-$(CONFIG_NETMAP) += netmap.o
>>>>    common-obj-y += filter.o
>>>> +common-obj-y += filter-buffer.o
>>>> diff --git a/net/filter-buffer.c b/net/filter-buffer.c
>>>> new file mode 100644
>>>> index 0000000..ef94e91
>>>> --- /dev/null
>>>> +++ b/net/filter-buffer.c
>>>> @@ -0,0 +1,170 @@
>>>> +/*
>>>> + * Copyright (c) 2015 FUJITSU LIMITED
>>>> + * Author: Yang Hongyang <yanghy@cn.fujitsu.com>
>>>> + *
>>>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>>>> + * later.  See the COPYING file in the top-level directory.
>>>> + */
>>>> +
>>>> +#include "net/filter.h"
>>>> +#include "net/queue.h"
>>>> +#include "qemu-common.h"
>>>> +#include "qemu/timer.h"
>>>> +#include "qemu/iov.h"
>>>> +#include "qapi/qmp/qerror.h"
>>>> +#include "qapi-visit.h"
>>>> +#include "qom/object.h"
>>>> +
>>>> +#define TYPE_FILTER_BUFFER "filter-buffer"
>>>> +
>>>> +#define FILTER_BUFFER(obj) \
>>>> +    OBJECT_CHECK(FilterBufferState, (obj), TYPE_FILTER_BUFFER)
>>>> +
>>>> +struct FilterBufferState {
>>>> +    NetFilterState parent_obj;
>>>> +
>>>> +    NetQueue *incoming_queue;
>>>> +    uint32_t interval;
>>>> +    QEMUTimer release_timer;
>>>> +};
>>>> +typedef struct FilterBufferState FilterBufferState;
>>>
>>> Again, not splitting the declaration is more concise.
>>>
>>>> +
>>>> +static void filter_buffer_flush(NetFilterState *nf)
>>>> +{
>>>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>>>> +
>>>> +    if (!qemu_net_queue_flush(s->incoming_queue)) {
>>>> +        /* Unable to empty the queue, purge remaining packets */
>>>> +        qemu_net_queue_purge(s->incoming_queue, nf->netdev);
>>>> +    }
>>>> +}
>>>
>>> This either flushes or purges incoming_queue, where "purge" means
>>> dropping packets.  Correct?
>>
>> I think it is correct. Dropping packets is allowed, even on real
>> hardware, packets may lose.
>>
>>>
>>>> +
>>>> +static void filter_buffer_release_timer(void *opaque)
>>>> +{
>>>> +    NetFilterState *nf = opaque;
>>>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>>>
>>> Style nit: blank line between declarations and statements, please.
>>>
>>>> +    filter_buffer_flush(nf);
>>>
>>> Is purging correct here?
>
> When the timer expires, we flush as many buffered packets as we can,
> then throw away the rest.  Why throw them away?  Shouldn't we leave them
> in the buffer, and only throw away packets when the buffer is full?
>
>>>> +    timer_mod(&s->release_timer,
>>>> +              qemu_clock_get_us(QEMU_CLOCK_VIRTUAL) + s->interval);
>>>
>>> Timer rearmed to fire again in s->interval microseconds.
>>
>> Yes.
>>
>>>
>>>> +}
>>>> +
>>>> +/* filter APIs */
>>>> +static ssize_t filter_buffer_receive_iov(NetFilterState *nf,
>>>> +                                         NetClientState *sender,
>>>> +                                         unsigned flags,
>>>> +                                         const struct iovec *iov,
>>>> +                                         int iovcnt,
>>>> +                                         NetPacketSent *sent_cb)
>>>> +{
>>>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>>>> +
>>>> +    /*
>>>> +     * we return size when buffer a packet, the sender will take it as
>>>> +     * a already sent packet, so sent_cb should not be called later
>>>
>>> Humor me: when a comment has multiple sentences, start each one with a
>>> capital letter, and end it with punctuation.
>>
>> Ok.
>>
>>>
>>>> +     * FIXME: even if guest can't receive packet for some reasons. Filter
>>>> +     * can still accept packet until its internal queue is full.
>>>> +     */
>>>
>>> I'm not sure I understand the comment.
>>
>> This is taken from Jason's comments, may be he can have a better explain?
>>
>>>
>>>> +    qemu_net_queue_append_iov(s->incoming_queue, sender, flags,
>>>> +                              iov, iovcnt, NULL);
>>>> +    return iov_size(iov, iovcnt);
>>>> +}
>>>> +
>>>> +static void filter_buffer_cleanup(NetFilterState *nf)
>>>> +{
>>>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>>>> +
>>>> +    if (s->interval) {
>>>> +        timer_del(&s->release_timer);
>>>> +    }
>>>> +
>>>> +    /* flush packets */
>>>> +    if (s->incoming_queue) {
>>>> +        filter_buffer_flush(nf);
>>>
>>> I guess purging is correct here.
>>>
>>>> +        g_free(s->incoming_queue);
>>>> +    }
>>>> +}
>>>> +
>>>> +static void filter_buffer_setup(NetFilterState *nf, Error **errp)
>>>> +{
>>>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>>>> +
>>>> +    /*
>>>> +     * this check should be dropped when there're VM FT solutions like MC
>>>> +     * or COLO use this filter to release packets on demand.
>>>> +     */
>>>
>>> If you end a sentence with a period, you get to start it with a capital
>>> letter :)
>>
>> Ok, thanks.
>>
>>>
>>> "there're" is odd.  Perhaps something like "We may want to accept zero
>>> interval when VM FT solutions like MC or COLO use this filter to release
>>> packets on demand."
>>>
>>>> +    if (!s->interval) {
>>>> +        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "interval",
>>>> +                   "a non-zero interval");
>>>
>>> How can this happen?  Doesn't filter_buffer_set_interval() catch zero
>>> intervals already?
>>
>> When user do not supply an interval parameter, filter_buffer_set_interval()
>> won't be called, and the interval is 0.
>> When we have actual user like I mentioned above, we should allow user to omit
>> the interval option.
>
> Let's see whether I understand.
>
> If you specify a non-zero interval, filter_buffer_set_interval() stores
> it.
>
> If you specify a zero interval, filter_buffer_set_interval() rejects it.
>
> If you specify no interval, it defaults to zero, and
> filter_buffer_setup() fails right here.
>
> Therefore, interval is not optional right now.
>
> Correct?
>
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    s->incoming_queue = qemu_new_net_queue(qemu_netfilter_pass_to_next, nf);
>>>> +    if (s->interval) {
>>>
>>> Condition cannot be false.  Same in filter_buffer_cleanup().
>>>
>>>> +        timer_init_us(&s->release_timer, QEMU_CLOCK_VIRTUAL,
>>>> +                      filter_buffer_release_timer, nf);
>>>> +        timer_mod(&s->release_timer,
>>>> +                  qemu_clock_get_us(QEMU_CLOCK_VIRTUAL) + s->interval);
>>>
>>> Timer armed to fire in s->interval microseconds.
>>>
>>> Unless I misunderstand something, this doesn't actually delay each
>>> packet by s->interval microseconds, it batches packet delivery: all
>>> packets arriving in a given interval are delayed until the end of the
>>> interval.  Correct?
>>
>> Correct.
>
> Documentation needs to spell that out.
>
>>>> +    }
>>>> +}
>>>> +
>>>> +static void filter_buffer_class_init(ObjectClass *oc, void *data)
>>>> +{
>>>> +    NetFilterClass *nfc = NETFILTER_CLASS(oc);
>>>> +
>>>> +    nfc->setup = filter_buffer_setup;
>>>> +    nfc->cleanup = filter_buffer_cleanup;
>>>> +    nfc->receive_iov = filter_buffer_receive_iov;
>>>> +}
>>>> +
>>>> +static void filter_buffer_get_interval(Object *obj, Visitor *v,
>>>> void *opaque,
>>>> +                                       const char *name, Error **errp)
>>>> +{
>>>> +    FilterBufferState *s = FILTER_BUFFER(obj);
>>>> +    uint32_t value = s->interval;
>>>> +
>>>> +    visit_type_uint32(v, &value, name, errp);
>>>> +}
>>>> +
>>>> +static void filter_buffer_set_interval(Object *obj, Visitor *v,
>>>> void *opaque,
>>>> +                                       const char *name, Error **errp)
>>>> +{
>>>> +    FilterBufferState *s = FILTER_BUFFER(obj);
>>>> +    Error *local_err = NULL;
>>>> +    uint32_t value;
>>>> +
>>>> +    visit_type_uint32(v, &value, name, &local_err);
>>>> +    if (local_err) {
>>>> +        goto out;
>>>> +    }
>>>> +    if (!value) {
>>>> +        error_setg(&local_err, "Property '%s.%s' doesn't take value '%"
>>>> +                   PRIu32 "'", object_get_typename(obj), name, value);
>>>
>>> What does it take?  That's what the user wants to know...  Perhaps
>>> "Property '%s.%s' requires a positive value".
>>
>> Maybe better, thanks.
>>
>>>
>>>> +        goto out;
>>>> +    }
>>>> +    s->interval = value;
>>>> +
>>>> +out:
>>>> +    error_propagate(errp, local_err);
>>>> +}
>>>> +
>>>> +static void filter_buffer_init(Object *obj)
>>>> +{
>>>> +    object_property_add(obj, "interval", "int",
>>>> +                        filter_buffer_get_interval,
>>>> +                        filter_buffer_set_interval, NULL, NULL, NULL);
>>>> +}
>>>> +
>>>> +static const TypeInfo filter_buffer_info = {
>>>> +    .name = TYPE_FILTER_BUFFER,
>>>> +    .parent = TYPE_NETFILTER,
>>>> +    .class_init = filter_buffer_class_init,
>>>> +    .instance_init = filter_buffer_init,
>>>> +    .instance_size = sizeof(FilterBufferState),
>>>> +};
>>>> +
>>>> +static void register_types(void)
>>>> +{
>>>> +    type_register_static(&filter_buffer_info);
>>>> +}
>>>> +
>>>> +type_init(register_types);
>>>> diff --git a/qemu-options.hx b/qemu-options.hx
>>>> index 7e147b8..b09f97f 100644
>>>> --- a/qemu-options.hx
>>>> +++ b/qemu-options.hx
>>>> @@ -3642,6 +3642,24 @@ in PEM format, in filenames
>>>> @var{ca-cert.pem}, @var{ca-crl.pem} (optional),
>>>>    @var{server-cert.pem} (only servers), @var{server-key.pem} (only servers),
>>>>    @var{client-cert.pem} (only clients), and @var{client-key.pem}
>>>> (only clients).
>>>>
>>>> +@item -object
>>>> filter-buffer,id=@var{id},netdev=@var{netdevid}[,chain=@var{all|in|out}][,interval=@var{t}]
>>>> +
>>>> +Buffer network packets on netdev @var{netdevid}.
>>>> +If interval @var{t} provided, will release packets by interval.
>>>> +Interval scale: microsecond.
>>>> +
>>>> +If interval @var{t} not provided, you have to make sure the packets can be
>>>
>>> Are you sure omitting t works?  filter_buffer_set_interval() rejects
>>> zero...
>>
>> When we omit t, filter_buffer_set_interval() won't be called.
>
> But filter_buffer_setup() will be, and it will fail, won't it?
>
>>>> +released, either by manually remove this filter or call the
>>>> release buffer API,
>>>> +otherwise, the packets will be buffered forever. Use with caution.
>>>
>>> Please wrap your lines a bit earlier.
>>>
>>>> +
>>>> +chain @var{all|in|out} is an option that can be applied to any
>>>> netfilter, default is @option{all}.
>>>> +
>>>> +@option{all} means this filter will receive packets both sent
>>>> to/from the netdev
>>>> +
>>>> +@option{in} means this filter will receive packets sent to the netdev
>>>> +
>>>> +@option{out} means this filter will receive packets sent from the netdev
>>>> +
>>>
>>> I'd describe this like "filter is inserted in the receive / transmit
>>> queue".
>>
>> thanks.
>>
>>>
>>>>    @end table
>>>>
>>>>    ETEXI
>>>> diff --git a/vl.c b/vl.c
>>>> index ec589e2..3cf89d5 100644
>>>> --- a/vl.c
>>>> +++ b/vl.c
>>>> @@ -2794,7 +2794,12 @@ static bool object_create_initial(const char *type)
>>>>        if (g_str_equal(type, "rng-egd")) {
>>>>            return false;
>>>>        }
>>>> -    /* TODO: return false for concrete netfilters */
>>>> +
>>>> +    /* return false for concrete netfilters */
>>>
>>> I find this comment useless, please drop it :)
>>
>> Ok, thanks.
>>
>>>
>>>> +    if (g_str_equal(type, "filter-buffer")) {
>>>> +        return false;
>>>> +    }
>>>> +
>>>>        return true;
>>>>    }
>>> .
>>>
> .
>

-- 
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 09/12] netfilter: add a netbuffer filter

[Markus Armbruster]

Jason Wang <jasowang@redhat.com> writes:

> On 09/25/2015 11:07 PM, Markus Armbruster wrote:
>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>
>>> On 09/24/2015 05:12 PM, Markus Armbruster wrote:
>>>> Yang Hongyang <yanghy@cn.fujitsu.com> writes:
>>>>
>>>>> This filter is to buffer/release packets, this feature can be used
>>>>> when using MicroCheckpointing, or other Remus like VM FT solutions, you
>>>> What's "Remus"?
>
> [...]
>
>>>>
>>>>> +
>>>>> +static void filter_buffer_release_timer(void *opaque)
>>>>> +{
>>>>> +    NetFilterState *nf = opaque;
>>>>> +    FilterBufferState *s = FILTER_BUFFER(nf);
>>>> Style nit: blank line between declarations and statements, please.
>>>>
>>>>> +    filter_buffer_flush(nf);
>>>> Is purging correct here?
>> When the timer expires, we flush as many buffered packets as we can,
>> then throw away the rest.  Why throw them away?  Shouldn't we leave them
>> in the buffer, and only throw away packets when the buffer is full?
>
> May need a "FIXME" or "TODO" here. I think this is for simplicity. We
> could queue the packet if the receiver or next filter could not receive
> packets. But currently there's no way for the next filter or recivier to
> notify us that it can receive more packet. This could be done in the future.

Good enough for me.  Make it FIXME if purging packets is actually wrong,
else TODO.