[Qemu-devel] [PULL V2 0/3] Net patches

[Qemu-devel] [PULL V2 0/3] Net patches

[Jason Wang]

The following changes since commit 83c83f9a5266ff113060f887f106a47920fa6974:

  Merge remote-tracking branch 'bonzini/tags/for-upstream' into staging (2016-11-11 12:51:50 +0000)

are available in the git repository at:

  https://github.com/jasowang/qemu.git tags/net-pull-request

for you to fetch changes up to 73c46860454f8e43d1679d1367552d15a66b7790:

  net: fix sending of data with -net socket, listen backend (2016-11-14 10:26:22 +0800)

----------------------------------------------------------------

Changes from V1:
- no change, V1 misses the list

----------------------------------------------------------------
Daniel P. Berrange (1):
      net: fix sending of data with -net socket, listen backend

Pavel Dovgalyuk (1):
      record/replay: add network support

Yuri Benditovich (1):
      net: skip virtio-net config of deleted nic's peers

 docs/replay.txt          |  14 +++++++
 hw/net/virtio-net.c      |   4 ++
 include/sysemu/replay.h  |  12 ++++++
 net/Makefile.objs        |   1 +
 net/filter-replay.c      |  92 ++++++++++++++++++++++++++++++++++++++++++
 net/net.c                |   5 +--
 net/socket.c             |   1 +
 replay/Makefile.objs     |   1 +
 replay/replay-events.c   |  11 +++++
 replay/replay-internal.h |  10 +++++
 replay/replay-net.c      | 102 +++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay.c          |   2 +-
 vl.c                     |   3 +-
 13 files changed, 253 insertions(+), 5 deletions(-)
 create mode 100644 net/filter-replay.c
 create mode 100644 replay/replay-net.c

[Qemu-devel] [PULL V2 1/3] record/replay: add network support

[Jason Wang]

From: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>

This patch adds support of recording and replaying network packets in
irount rr mode.

Record and replay for network interactions is performed with the network filter.
Each backend must have its own instance of the replay filter as follows:
 -netdev user,id=net1 -device rtl8139,netdev=net1
 -object filter-replay,id=replay,netdev=net1

Replay network filter is used to record and replay network packets. While
recording the virtual machine this filter puts all packets coming from
the outer world into the log. In replay mode packets from the log are
injected into the network device. All interactions with network backend
in replay mode are disabled.

v5 changes:
 - using iov_to_buf function instead of loop

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 docs/replay.txt          |  14 +++++++
 include/sysemu/replay.h  |  12 ++++++
 net/Makefile.objs        |   1 +
 net/filter-replay.c      |  92 ++++++++++++++++++++++++++++++++++++++++++
 replay/Makefile.objs     |   1 +
 replay/replay-events.c   |  11 +++++
 replay/replay-internal.h |  10 +++++
 replay/replay-net.c      | 102 +++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay.c          |   2 +-
 vl.c                     |   3 +-
 10 files changed, 246 insertions(+), 2 deletions(-)
 create mode 100644 net/filter-replay.c
 create mode 100644 replay/replay-net.c

diff --git a/docs/replay.txt b/docs/replay.txt
index 779c6c0..347b2ff 100644
--- a/docs/replay.txt
+++ b/docs/replay.txt
@@ -195,3 +195,17 @@ Queue is flushed at checkpoints and information about processed requests
 is recorded to the log. In replay phase the queue is matched with
 events read from the log. Therefore block devices requests are processed
 deterministically.
+
+Network devices
+---------------
+
+Record and replay for network interactions is performed with the network filter.
+Each backend must have its own instance of the replay filter as follows:
+ -netdev user,id=net1 -device rtl8139,netdev=net1
+ -object filter-replay,id=replay,netdev=net1
+
+Replay network filter is used to record and replay network packets. While
+recording the virtual machine this filter puts all packets coming from
+the outer world into the log. In replay mode packets from the log are
+injected into the network device. All interactions with network backend
+in replay mode are disabled.
diff --git a/include/sysemu/replay.h b/include/sysemu/replay.h
index f80d6d2..abb35ca 100644
--- a/include/sysemu/replay.h
+++ b/include/sysemu/replay.h
@@ -39,6 +39,8 @@ enum ReplayCheckpoint {
 };
 typedef enum ReplayCheckpoint ReplayCheckpoint;
 
+typedef struct ReplayNetState ReplayNetState;
+
 extern ReplayMode replay_mode;
 
 /* Replay process control functions */
@@ -137,4 +139,14 @@ void replay_char_read_all_save_error(int res);
 /*! Writes character read_all execution result into the replay log. */
 void replay_char_read_all_save_buf(uint8_t *buf, int offset);
 
+/* Network */
+
+/*! Registers replay network filter attached to some backend. */
+ReplayNetState *replay_register_net(NetFilterState *nfs);
+/*! Unregisters replay network filter. */
+void replay_unregister_net(ReplayNetState *rns);
+/*! Called to write network packet to the replay log. */
+void replay_net_packet_event(ReplayNetState *rns, unsigned flags,
+                             const struct iovec *iov, int iovcnt);
+
 #endif
diff --git a/net/Makefile.objs b/net/Makefile.objs
index 2a80df5..2e2fd43 100644
--- a/net/Makefile.objs
+++ b/net/Makefile.objs
@@ -19,3 +19,4 @@ common-obj-y += filter-mirror.o
 common-obj-y += colo-compare.o
 common-obj-y += colo.o
 common-obj-y += filter-rewriter.o
+common-obj-y += filter-replay.o
diff --git a/net/filter-replay.c b/net/filter-replay.c
new file mode 100644
index 0000000..cff65f8
--- /dev/null
+++ b/net/filter-replay.c
@@ -0,0 +1,92 @@
+/*
+ * filter-replay.c
+ *
+ * Copyright (c) 2010-2016 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "clients.h"
+#include "qapi/error.h"
+#include "qemu-common.h"
+#include "qemu/error-report.h"
+#include "qemu/iov.h"
+#include "qemu/log.h"
+#include "qemu/timer.h"
+#include "qapi/visitor.h"
+#include "net/filter.h"
+#include "sysemu/replay.h"
+
+#define TYPE_FILTER_REPLAY "filter-replay"
+
+#define FILTER_REPLAY(obj) \
+    OBJECT_CHECK(NetFilterReplayState, (obj), TYPE_FILTER_REPLAY)
+
+struct NetFilterReplayState {
+    NetFilterState nfs;
+    ReplayNetState *rns;
+};
+typedef struct NetFilterReplayState NetFilterReplayState;
+
+static ssize_t filter_replay_receive_iov(NetFilterState *nf,
+                                         NetClientState *sndr,
+                                         unsigned flags,
+                                         const struct iovec *iov,
+                                         int iovcnt, NetPacketSent *sent_cb)
+{
+    NetFilterReplayState *nfrs = FILTER_REPLAY(nf);
+    switch (replay_mode) {
+    case REPLAY_MODE_RECORD:
+        if (nf->netdev == sndr) {
+            replay_net_packet_event(nfrs->rns, flags, iov, iovcnt);
+            return iov_size(iov, iovcnt);
+        }
+        return 0;
+    case REPLAY_MODE_PLAY:
+        /* Drop all packets in replay mode.
+           Packets from the log will be injected by the replay module. */
+        return iov_size(iov, iovcnt);
+    default:
+        /* Pass all the packets. */
+        return 0;
+    }
+}
+
+static void filter_replay_instance_init(Object *obj)
+{
+    NetFilterReplayState *nfrs = FILTER_REPLAY(obj);
+    nfrs->rns = replay_register_net(&nfrs->nfs);
+}
+
+static void filter_replay_instance_finalize(Object *obj)
+{
+    NetFilterReplayState *nfrs = FILTER_REPLAY(obj);
+    replay_unregister_net(nfrs->rns);
+}
+
+static void filter_replay_class_init(ObjectClass *oc, void *data)
+{
+    NetFilterClass *nfc = NETFILTER_CLASS(oc);
+
+    nfc->receive_iov = filter_replay_receive_iov;
+}
+
+static const TypeInfo filter_replay_info = {
+    .name = TYPE_FILTER_REPLAY,
+    .parent = TYPE_NETFILTER,
+    .class_init = filter_replay_class_init,
+    .instance_init = filter_replay_instance_init,
+    .instance_finalize = filter_replay_instance_finalize,
+    .instance_size = sizeof(NetFilterReplayState),
+};
+
+static void filter_replay_register_types(void)
+{
+    type_register_static(&filter_replay_info);
+}
+
+type_init(filter_replay_register_types);
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index c8ad3eb..b2afd40 100644
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -5,3 +5,4 @@ common-obj-y += replay-time.o
 common-obj-y += replay-input.o
 common-obj-y += replay-char.o
 common-obj-y += replay-snapshot.o
+common-obj-y += replay-net.o
diff --git a/replay/replay-events.c b/replay/replay-events.c
index c513913..94a6dcc 100644
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -54,6 +54,9 @@ static void replay_run_event(Event *event)
     case REPLAY_ASYNC_EVENT_BLOCK:
         aio_bh_call(event->opaque);
         break;
+    case REPLAY_ASYNC_EVENT_NET:
+        replay_event_net_run(event->opaque);
+        break;
     default:
         error_report("Replay: invalid async event ID (%d) in the queue",
                     event->event_kind);
@@ -189,6 +192,9 @@ static void replay_save_event(Event *event, int checkpoint)
         case REPLAY_ASYNC_EVENT_BLOCK:
             replay_put_qword(event->id);
             break;
+        case REPLAY_ASYNC_EVENT_NET:
+            replay_event_net_save(event->opaque);
+            break;
         default:
             error_report("Unknown ID %" PRId64 " of replay event", event->id);
             exit(1);
@@ -252,6 +258,11 @@ static Event *replay_read_event(int checkpoint)
             read_id = replay_get_qword();
         }
         break;
+    case REPLAY_ASYNC_EVENT_NET:
+        event = g_malloc0(sizeof(Event));
+        event->event_kind = read_event_kind;
+        event->opaque = replay_event_net_load();
+        return event;
     default:
         error_report("Unknown ID %d of replay event", read_event_kind);
         exit(1);
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 9117e44..c26d079 100644
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -50,6 +50,7 @@ enum ReplayAsyncEventKind {
     REPLAY_ASYNC_EVENT_INPUT_SYNC,
     REPLAY_ASYNC_EVENT_CHAR_READ,
     REPLAY_ASYNC_EVENT_BLOCK,
+    REPLAY_ASYNC_EVENT_NET,
     REPLAY_ASYNC_COUNT
 };
 
@@ -161,6 +162,15 @@ void replay_event_char_read_save(void *opaque);
 /*! Reads char event read from the file. */
 void *replay_event_char_read_load(void);
 
+/* Network devices */
+
+/*! Called to run network event. */
+void replay_event_net_run(void *opaque);
+/*! Writes network event to the file. */
+void replay_event_net_save(void *opaque);
+/*! Reads network from the file. */
+void *replay_event_net_load(void);
+
 /* VMState-related functions */
 
 /* Registers replay VMState.
diff --git a/replay/replay-net.c b/replay/replay-net.c
new file mode 100644
index 0000000..80b7054
--- /dev/null
+++ b/replay/replay-net.c
@@ -0,0 +1,102 @@
+/*
+ * replay-net.c
+ *
+ * Copyright (c) 2010-2016 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "sysemu/replay.h"
+#include "replay-internal.h"
+#include "sysemu/sysemu.h"
+#include "net/net.h"
+#include "net/filter.h"
+#include "qemu/iov.h"
+
+struct ReplayNetState {
+    NetFilterState *nfs;
+    int id;
+};
+
+typedef struct NetEvent {
+    uint8_t id;
+    uint32_t flags;
+    uint8_t *data;
+    size_t size;
+} NetEvent;
+
+static NetFilterState **network_filters;
+static int network_filters_count;
+
+ReplayNetState *replay_register_net(NetFilterState *nfs)
+{
+    ReplayNetState *rns = g_new0(ReplayNetState, 1);
+    rns->nfs = nfs;
+    rns->id = network_filters_count++;
+    network_filters = g_realloc(network_filters,
+                                network_filters_count
+                                    * sizeof(*network_filters));
+    network_filters[network_filters_count - 1] = nfs;
+    return rns;
+}
+
+void replay_unregister_net(ReplayNetState *rns)
+{
+    network_filters[rns->id] = NULL;
+    g_free(rns);
+}
+
+void replay_net_packet_event(ReplayNetState *rns, unsigned flags,
+                             const struct iovec *iov, int iovcnt)
+{
+    NetEvent *event = g_new(NetEvent, 1);
+    event->flags = flags;
+    event->data = g_malloc(iov_size(iov, iovcnt));
+    event->size = iov_size(iov, iovcnt);
+    event->id = rns->id;
+    iov_to_buf(iov, iovcnt, 0, event->data, event->size);
+
+    replay_add_event(REPLAY_ASYNC_EVENT_NET, event, NULL, 0);
+}
+
+void replay_event_net_run(void *opaque)
+{
+    NetEvent *event = opaque;
+    struct iovec iov = {
+        .iov_base = (void *)event->data,
+        .iov_len = event->size
+    };
+
+    assert(event->id < network_filters_count);
+
+    qemu_netfilter_pass_to_next(network_filters[event->id]->netdev,
+        event->flags, &iov, 1, network_filters[event->id]);
+
+    g_free(event->data);
+    g_free(event);
+}
+
+void replay_event_net_save(void *opaque)
+{
+    NetEvent *event = opaque;
+
+    replay_put_byte(event->id);
+    replay_put_dword(event->flags);
+    replay_put_array(event->data, event->size);
+}
+
+void *replay_event_net_load(void)
+{
+    NetEvent *event = g_new(NetEvent, 1);
+
+    event->id = replay_get_byte();
+    event->flags = replay_get_dword();
+    replay_get_array_alloc(&event->data, &event->size);
+
+    return event;
+}
diff --git a/replay/replay.c b/replay/replay.c
index c797aea..7f27cf1 100644
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -21,7 +21,7 @@
 
 /* Current version of the replay mechanism.
    Increase it when file format changes. */
-#define REPLAY_VERSION              0xe02004
+#define REPLAY_VERSION              0xe02005
 /* Size of replay log header */
 #define HEADER_SIZE                 (sizeof(uint32_t) + sizeof(uint64_t))
 
diff --git a/vl.c b/vl.c
index d77dd86..a23de18 100644
--- a/vl.c
+++ b/vl.c
@@ -2859,7 +2859,8 @@ static bool object_create_initial(const char *type)
         g_str_equal(type, "filter-mirror") ||
         g_str_equal(type, "filter-redirector") ||
         g_str_equal(type, "colo-compare") ||
-        g_str_equal(type, "filter-rewriter")) {
+        g_str_equal(type, "filter-rewriter") ||
+        g_str_equal(type, "filter-replay")) {
         return false;
     }
 
-- 
2.7.4

Re: [Qemu-devel] [PULL V2 1/3] record/replay: add network support

[Pavel Dovgalyuk]

Hi, Jason!

Please don't forget this for 2.9.

Pavel Dovgalyuk

> -----Original Message-----
> From: Jason Wang [mailto:jasowang@redhat.com]
> Sent: Monday, November 14, 2016 5:54 AM
> To: peter.maydell@linaro.org
> Cc: qemu-devel@nongnu.org; Pavel Dovgalyuk; Jason Wang
> Subject: [PULL V2 1/3] record/replay: add network support
> 
> From: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> 
> This patch adds support of recording and replaying network packets in
> irount rr mode.
> 
> Record and replay for network interactions is performed with the network filter.
> Each backend must have its own instance of the replay filter as follows:
>  -netdev user,id=net1 -device rtl8139,netdev=net1
>  -object filter-replay,id=replay,netdev=net1
> 
> Replay network filter is used to record and replay network packets. While
> recording the virtual machine this filter puts all packets coming from
> the outer world into the log. In replay mode packets from the log are
> injected into the network device. All interactions with network backend
> in replay mode are disabled.
> 
> v5 changes:
>  - using iov_to_buf function instead of loop
> 
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
>  docs/replay.txt          |  14 +++++++
>  include/sysemu/replay.h  |  12 ++++++
>  net/Makefile.objs        |   1 +
>  net/filter-replay.c      |  92 ++++++++++++++++++++++++++++++++++++++++++
>  replay/Makefile.objs     |   1 +
>  replay/replay-events.c   |  11 +++++
>  replay/replay-internal.h |  10 +++++
>  replay/replay-net.c      | 102 +++++++++++++++++++++++++++++++++++++++++++++++
>  replay/replay.c          |   2 +-
>  vl.c                     |   3 +-
>  10 files changed, 246 insertions(+), 2 deletions(-)
>  create mode 100644 net/filter-replay.c
>  create mode 100644 replay/replay-net.c
> 
> diff --git a/docs/replay.txt b/docs/replay.txt
> index 779c6c0..347b2ff 100644
> --- a/docs/replay.txt
> +++ b/docs/replay.txt
> @@ -195,3 +195,17 @@ Queue is flushed at checkpoints and information about processed requests
>  is recorded to the log. In replay phase the queue is matched with
>  events read from the log. Therefore block devices requests are processed
>  deterministically.
> +
> +Network devices
> +---------------
> +
> +Record and replay for network interactions is performed with the network filter.
> +Each backend must have its own instance of the replay filter as follows:
> + -netdev user,id=net1 -device rtl8139,netdev=net1
> + -object filter-replay,id=replay,netdev=net1
> +
> +Replay network filter is used to record and replay network packets. While
> +recording the virtual machine this filter puts all packets coming from
> +the outer world into the log. In replay mode packets from the log are
> +injected into the network device. All interactions with network backend
> +in replay mode are disabled.
> diff --git a/include/sysemu/replay.h b/include/sysemu/replay.h
> index f80d6d2..abb35ca 100644
> --- a/include/sysemu/replay.h
> +++ b/include/sysemu/replay.h
> @@ -39,6 +39,8 @@ enum ReplayCheckpoint {
>  };
>  typedef enum ReplayCheckpoint ReplayCheckpoint;
> 
> +typedef struct ReplayNetState ReplayNetState;
> +
>  extern ReplayMode replay_mode;
> 
>  /* Replay process control functions */
> @@ -137,4 +139,14 @@ void replay_char_read_all_save_error(int res);
>  /*! Writes character read_all execution result into the replay log. */
>  void replay_char_read_all_save_buf(uint8_t *buf, int offset);
> 
> +/* Network */
> +
> +/*! Registers replay network filter attached to some backend. */
> +ReplayNetState *replay_register_net(NetFilterState *nfs);
> +/*! Unregisters replay network filter. */
> +void replay_unregister_net(ReplayNetState *rns);
> +/*! Called to write network packet to the replay log. */
> +void replay_net_packet_event(ReplayNetState *rns, unsigned flags,
> +                             const struct iovec *iov, int iovcnt);
> +
>  #endif
> diff --git a/net/Makefile.objs b/net/Makefile.objs
> index 2a80df5..2e2fd43 100644
> --- a/net/Makefile.objs
> +++ b/net/Makefile.objs
> @@ -19,3 +19,4 @@ common-obj-y += filter-mirror.o
>  common-obj-y += colo-compare.o
>  common-obj-y += colo.o
>  common-obj-y += filter-rewriter.o
> +common-obj-y += filter-replay.o
> diff --git a/net/filter-replay.c b/net/filter-replay.c
> new file mode 100644
> index 0000000..cff65f8
> --- /dev/null
> +++ b/net/filter-replay.c
> @@ -0,0 +1,92 @@
> +/*
> + * filter-replay.c
> + *
> + * Copyright (c) 2010-2016 Institute for System Programming
> + *                         of the Russian Academy of Sciences.
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
> +
> +#include "qemu/osdep.h"
> +#include "clients.h"
> +#include "qapi/error.h"
> +#include "qemu-common.h"
> +#include "qemu/error-report.h"
> +#include "qemu/iov.h"
> +#include "qemu/log.h"
> +#include "qemu/timer.h"
> +#include "qapi/visitor.h"
> +#include "net/filter.h"
> +#include "sysemu/replay.h"
> +
> +#define TYPE_FILTER_REPLAY "filter-replay"
> +
> +#define FILTER_REPLAY(obj) \
> +    OBJECT_CHECK(NetFilterReplayState, (obj), TYPE_FILTER_REPLAY)
> +
> +struct NetFilterReplayState {
> +    NetFilterState nfs;
> +    ReplayNetState *rns;
> +};
> +typedef struct NetFilterReplayState NetFilterReplayState;
> +
> +static ssize_t filter_replay_receive_iov(NetFilterState *nf,
> +                                         NetClientState *sndr,
> +                                         unsigned flags,
> +                                         const struct iovec *iov,
> +                                         int iovcnt, NetPacketSent *sent_cb)
> +{
> +    NetFilterReplayState *nfrs = FILTER_REPLAY(nf);
> +    switch (replay_mode) {
> +    case REPLAY_MODE_RECORD:
> +        if (nf->netdev == sndr) {
> +            replay_net_packet_event(nfrs->rns, flags, iov, iovcnt);
> +            return iov_size(iov, iovcnt);
> +        }
> +        return 0;
> +    case REPLAY_MODE_PLAY:
> +        /* Drop all packets in replay mode.
> +           Packets from the log will be injected by the replay module. */
> +        return iov_size(iov, iovcnt);
> +    default:
> +        /* Pass all the packets. */
> +        return 0;
> +    }
> +}
> +
> +static void filter_replay_instance_init(Object *obj)
> +{
> +    NetFilterReplayState *nfrs = FILTER_REPLAY(obj);
> +    nfrs->rns = replay_register_net(&nfrs->nfs);
> +}
> +
> +static void filter_replay_instance_finalize(Object *obj)
> +{
> +    NetFilterReplayState *nfrs = FILTER_REPLAY(obj);
> +    replay_unregister_net(nfrs->rns);
> +}
> +
> +static void filter_replay_class_init(ObjectClass *oc, void *data)
> +{
> +    NetFilterClass *nfc = NETFILTER_CLASS(oc);
> +
> +    nfc->receive_iov = filter_replay_receive_iov;
> +}
> +
> +static const TypeInfo filter_replay_info = {
> +    .name = TYPE_FILTER_REPLAY,
> +    .parent = TYPE_NETFILTER,
> +    .class_init = filter_replay_class_init,
> +    .instance_init = filter_replay_instance_init,
> +    .instance_finalize = filter_replay_instance_finalize,
> +    .instance_size = sizeof(NetFilterReplayState),
> +};
> +
> +static void filter_replay_register_types(void)
> +{
> +    type_register_static(&filter_replay_info);
> +}
> +
> +type_init(filter_replay_register_types);
> diff --git a/replay/Makefile.objs b/replay/Makefile.objs
> index c8ad3eb..b2afd40 100644
> --- a/replay/Makefile.objs
> +++ b/replay/Makefile.objs
> @@ -5,3 +5,4 @@ common-obj-y += replay-time.o
>  common-obj-y += replay-input.o
>  common-obj-y += replay-char.o
>  common-obj-y += replay-snapshot.o
> +common-obj-y += replay-net.o
> diff --git a/replay/replay-events.c b/replay/replay-events.c
> index c513913..94a6dcc 100644
> --- a/replay/replay-events.c
> +++ b/replay/replay-events.c
> @@ -54,6 +54,9 @@ static void replay_run_event(Event *event)
>      case REPLAY_ASYNC_EVENT_BLOCK:
>          aio_bh_call(event->opaque);
>          break;
> +    case REPLAY_ASYNC_EVENT_NET:
> +        replay_event_net_run(event->opaque);
> +        break;
>      default:
>          error_report("Replay: invalid async event ID (%d) in the queue",
>                      event->event_kind);
> @@ -189,6 +192,9 @@ static void replay_save_event(Event *event, int checkpoint)
>          case REPLAY_ASYNC_EVENT_BLOCK:
>              replay_put_qword(event->id);
>              break;
> +        case REPLAY_ASYNC_EVENT_NET:
> +            replay_event_net_save(event->opaque);
> +            break;
>          default:
>              error_report("Unknown ID %" PRId64 " of replay event", event->id);
>              exit(1);
> @@ -252,6 +258,11 @@ static Event *replay_read_event(int checkpoint)
>              read_id = replay_get_qword();
>          }
>          break;
> +    case REPLAY_ASYNC_EVENT_NET:
> +        event = g_malloc0(sizeof(Event));
> +        event->event_kind = read_event_kind;
> +        event->opaque = replay_event_net_load();
> +        return event;
>      default:
>          error_report("Unknown ID %d of replay event", read_event_kind);
>          exit(1);
> diff --git a/replay/replay-internal.h b/replay/replay-internal.h
> index 9117e44..c26d079 100644
> --- a/replay/replay-internal.h
> +++ b/replay/replay-internal.h
> @@ -50,6 +50,7 @@ enum ReplayAsyncEventKind {
>      REPLAY_ASYNC_EVENT_INPUT_SYNC,
>      REPLAY_ASYNC_EVENT_CHAR_READ,
>      REPLAY_ASYNC_EVENT_BLOCK,
> +    REPLAY_ASYNC_EVENT_NET,
>      REPLAY_ASYNC_COUNT
>  };
> 
> @@ -161,6 +162,15 @@ void replay_event_char_read_save(void *opaque);
>  /*! Reads char event read from the file. */
>  void *replay_event_char_read_load(void);
> 
> +/* Network devices */
> +
> +/*! Called to run network event. */
> +void replay_event_net_run(void *opaque);
> +/*! Writes network event to the file. */
> +void replay_event_net_save(void *opaque);
> +/*! Reads network from the file. */
> +void *replay_event_net_load(void);
> +
>  /* VMState-related functions */
> 
>  /* Registers replay VMState.
> diff --git a/replay/replay-net.c b/replay/replay-net.c
> new file mode 100644
> index 0000000..80b7054
> --- /dev/null
> +++ b/replay/replay-net.c
> @@ -0,0 +1,102 @@
> +/*
> + * replay-net.c
> + *
> + * Copyright (c) 2010-2016 Institute for System Programming
> + *                         of the Russian Academy of Sciences.
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qemu/error-report.h"
> +#include "sysemu/replay.h"
> +#include "replay-internal.h"
> +#include "sysemu/sysemu.h"
> +#include "net/net.h"
> +#include "net/filter.h"
> +#include "qemu/iov.h"
> +
> +struct ReplayNetState {
> +    NetFilterState *nfs;
> +    int id;
> +};
> +
> +typedef struct NetEvent {
> +    uint8_t id;
> +    uint32_t flags;
> +    uint8_t *data;
> +    size_t size;
> +} NetEvent;
> +
> +static NetFilterState **network_filters;
> +static int network_filters_count;
> +
> +ReplayNetState *replay_register_net(NetFilterState *nfs)
> +{
> +    ReplayNetState *rns = g_new0(ReplayNetState, 1);
> +    rns->nfs = nfs;
> +    rns->id = network_filters_count++;
> +    network_filters = g_realloc(network_filters,
> +                                network_filters_count
> +                                    * sizeof(*network_filters));
> +    network_filters[network_filters_count - 1] = nfs;
> +    return rns;
> +}
> +
> +void replay_unregister_net(ReplayNetState *rns)
> +{
> +    network_filters[rns->id] = NULL;
> +    g_free(rns);
> +}
> +
> +void replay_net_packet_event(ReplayNetState *rns, unsigned flags,
> +                             const struct iovec *iov, int iovcnt)
> +{
> +    NetEvent *event = g_new(NetEvent, 1);
> +    event->flags = flags;
> +    event->data = g_malloc(iov_size(iov, iovcnt));
> +    event->size = iov_size(iov, iovcnt);
> +    event->id = rns->id;
> +    iov_to_buf(iov, iovcnt, 0, event->data, event->size);
> +
> +    replay_add_event(REPLAY_ASYNC_EVENT_NET, event, NULL, 0);
> +}
> +
> +void replay_event_net_run(void *opaque)
> +{
> +    NetEvent *event = opaque;
> +    struct iovec iov = {
> +        .iov_base = (void *)event->data,
> +        .iov_len = event->size
> +    };
> +
> +    assert(event->id < network_filters_count);
> +
> +    qemu_netfilter_pass_to_next(network_filters[event->id]->netdev,
> +        event->flags, &iov, 1, network_filters[event->id]);
> +
> +    g_free(event->data);
> +    g_free(event);
> +}
> +
> +void replay_event_net_save(void *opaque)
> +{
> +    NetEvent *event = opaque;
> +
> +    replay_put_byte(event->id);
> +    replay_put_dword(event->flags);
> +    replay_put_array(event->data, event->size);
> +}
> +
> +void *replay_event_net_load(void)
> +{
> +    NetEvent *event = g_new(NetEvent, 1);
> +
> +    event->id = replay_get_byte();
> +    event->flags = replay_get_dword();
> +    replay_get_array_alloc(&event->data, &event->size);
> +
> +    return event;
> +}
> diff --git a/replay/replay.c b/replay/replay.c
> index c797aea..7f27cf1 100644
> --- a/replay/replay.c
> +++ b/replay/replay.c
> @@ -21,7 +21,7 @@
> 
>  /* Current version of the replay mechanism.
>     Increase it when file format changes. */
> -#define REPLAY_VERSION              0xe02004
> +#define REPLAY_VERSION              0xe02005
>  /* Size of replay log header */
>  #define HEADER_SIZE                 (sizeof(uint32_t) + sizeof(uint64_t))
> 
> diff --git a/vl.c b/vl.c
> index d77dd86..a23de18 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -2859,7 +2859,8 @@ static bool object_create_initial(const char *type)
>          g_str_equal(type, "filter-mirror") ||
>          g_str_equal(type, "filter-redirector") ||
>          g_str_equal(type, "colo-compare") ||
> -        g_str_equal(type, "filter-rewriter")) {
> +        g_str_equal(type, "filter-rewriter") ||
> +        g_str_equal(type, "filter-replay")) {
>          return false;
>      }
> 
> --
> 2.7.4

Re: [Qemu-devel] [PULL V2 1/3] record/replay: add network support

[Jason Wang]

On 2016年12月22日 14:56, Pavel Dovgalyuk wrote:
> Hi, Jason!
>
> Please don't forget this for 2.9.
>
> Pavel Dovgalyuk

Already in my queue so it won't be missed :)

Thanks

[Qemu-devel] [PULL V2 2/3] net: skip virtio-net config of deleted nic's peers

[Jason Wang]

From: Yuri Benditovich <yuri.benditovich@daynix.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1373816
qemu core dump happens during repetitive unpug-plug
with multiple queues and Windows RSS-capable guest.
If back-end delete requested during virtio-net device
initialization, driver still can try configure the device
for multiple queues. The virtio-net device is expected
to be removed as soon as the initialization is done.

Signed-off-by: Yuri Benditovich <yuri.benditovich@daynix.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 hw/net/virtio-net.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 06bfe4b..77a4fae 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -508,6 +508,10 @@ static void virtio_net_set_queues(VirtIONet *n)
     int i;
     int r;
 
+    if (n->nic->peer_deleted) {
+        return;
+    }
+
     for (i = 0; i < n->max_queues; i++) {
         if (i < n->curr_queues) {
             r = peer_attach(n, i);
-- 
2.7.4

[Qemu-devel] [PULL V2 3/3] net: fix sending of data with -net socket, listen backend

[Jason Wang]

From: "Daniel P. Berrange" <berrange@redhat.com>

The use of -net socket,listen was broken in the following
commit

  commit 16a3df403b10c4ac347159e39005fd520b2648bb
  Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
  Date:   Fri May 13 15:35:19 2016 +0800

    net/net: Add SocketReadState for reuse codes

    This function is from net/socket.c, move it to net.c and net.h.
    Add SocketReadState to make others reuse net_fill_rstate().
    suggestion from jason.

This refactored the state out of NetSocketState into a
separate SocketReadState. This refactoring requires
that a callback is provided to be triggered upon
completion of a packet receive from the guest.

The patch only registered this callback in the codepaths
hit by -net socket,connect, not -net socket,listen. So
as a result packets sent by the guest in the latter case
get dropped on the floor.

This bug is hidden because net_fill_rstate() silently
does nothing if the callback is not set.

This patch adds in the middle callback registration
and also adds an assert so that QEMU aborts if there
are any other codepaths hit which are missing the
callback.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 net/net.c    | 5 ++---
 net/socket.c | 1 +
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/net/net.c b/net/net.c
index ec984bf..939fe31 100644
--- a/net/net.c
+++ b/net/net.c
@@ -1653,9 +1653,8 @@ int net_fill_rstate(SocketReadState *rs, const uint8_t *buf, int size)
             if (rs->index >= rs->packet_len) {
                 rs->index = 0;
                 rs->state = 0;
-                if (rs->finalize) {
-                    rs->finalize(rs);
-                }
+                assert(rs->finalize);
+                rs->finalize(rs);
             }
             break;
         }
diff --git a/net/socket.c b/net/socket.c
index 982c8de..fe3547b 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -511,6 +511,7 @@ static int net_socket_listen_init(NetClientState *peer,
     s->fd = -1;
     s->listen_fd = ret;
     s->nc.link_down = true;
+    net_socket_rs_init(&s->rs, net_socket_rs_finalize);
 
     qemu_set_fd_handler(s->listen_fd, net_socket_accept, NULL, s);
     qapi_free_SocketAddress(saddr);
-- 
2.7.4

Re: [Qemu-devel] [PULL V2 0/3] Net patches

[Zhang Chen]

Hi~ Jason~

Can you pick up this patch?

[PATCH] docs: fix COLO architecture diagram


Thanks

Zhang Chen


On 11/14/2016 10:54 AM, Jason Wang wrote:
> The following changes since commit 83c83f9a5266ff113060f887f106a47920fa6974:
>
>    Merge remote-tracking branch 'bonzini/tags/for-upstream' into staging (2016-11-11 12:51:50 +0000)
>
> are available in the git repository at:
>
>    https://github.com/jasowang/qemu.git tags/net-pull-request
>
> for you to fetch changes up to 73c46860454f8e43d1679d1367552d15a66b7790:
>
>    net: fix sending of data with -net socket, listen backend (2016-11-14 10:26:22 +0800)
>
> ----------------------------------------------------------------
>
> Changes from V1:
> - no change, V1 misses the list
>
> ----------------------------------------------------------------
> Daniel P. Berrange (1):
>        net: fix sending of data with -net socket, listen backend
>
> Pavel Dovgalyuk (1):
>        record/replay: add network support
>
> Yuri Benditovich (1):
>        net: skip virtio-net config of deleted nic's peers
>
>   docs/replay.txt          |  14 +++++++
>   hw/net/virtio-net.c      |   4 ++
>   include/sysemu/replay.h  |  12 ++++++
>   net/Makefile.objs        |   1 +
>   net/filter-replay.c      |  92 ++++++++++++++++++++++++++++++++++++++++++
>   net/net.c                |   5 +--
>   net/socket.c             |   1 +
>   replay/Makefile.objs     |   1 +
>   replay/replay-events.c   |  11 +++++
>   replay/replay-internal.h |  10 +++++
>   replay/replay-net.c      | 102 +++++++++++++++++++++++++++++++++++++++++++++++
>   replay/replay.c          |   2 +-
>   vl.c                     |   3 +-
>   13 files changed, 253 insertions(+), 5 deletions(-)
>   create mode 100644 net/filter-replay.c
>   create mode 100644 replay/replay-net.c
>
>
>
>
>

-- 
Thanks
zhangchen

Re: [Qemu-devel] [PULL V2 0/3] Net patches

[Jason Wang]

On 2016年11月14日 14:46, Zhang Chen wrote:
> Hi~ Jason~
>
> Can you pick up this patch?
>
> [PATCH] docs: fix COLO architecture diagram
>
>
> Thanks
>
> Zhang Chen 

Will have a look at this and pick it.

Thanks

Re: [Qemu-devel] [PULL V2 0/3] Net patches

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 449 bytes --]

On Mon, Nov 14, 2016 at 10:54:23AM +0800, Jason Wang wrote:
> Pavel Dovgalyuk (1):
>       record/replay: add network support

QEMU is in soft freeze.  The policy has changed this release and only
bug fixes are being accepted during soft freeze.  Please don't include
new features.

http://qemu-project.org/Planning/SoftFeatureFreeze

Please submit a new version without this patch or explain why it is a
necessary bug fix for -rc0.

Thanks,
Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] [PULL V2 0/3] Net patches

[Jason Wang]

On 2016年11月15日 00:40, Stefan Hajnoczi wrote:
> On Mon, Nov 14, 2016 at 10:54:23AM +0800, Jason Wang wrote:
>> Pavel Dovgalyuk (1):
>>        record/replay: add network support
> QEMU is in soft freeze.  The policy has changed this release and only
> bug fixes are being accepted during soft freeze.  Please don't include
> new features.
>
> http://qemu-project.org/Planning/SoftFeatureFreeze
>
> Please submit a new version without this patch or explain why it is a
> necessary bug fix for -rc0.
>
> Thanks,
> Stefan

Will drop this patch.

Thanks