Re: [Qemu-devel] [PATCH V13 00/12] Introduce COLO-Proxy

[Jason Wang <jasowang@redhat.com>]

On 2016年09月05日 17:37, Zhang Chen wrote:
> COLO-proxy is a part of COLO project. COLO project is
> composed of COLO-frame, COLO-proxy and block-replication.
> It is used to compare the network package to help COLO
> decide whether to do checkpoint. With COLO-proxy's help,
> COLO greatly improves the performance.
>
> The filter-redirector, filter-mirror, colo-compare
> and filter-rewriter compose the COLO-proxy.
>
> COLO-compare
> It is used to compare the network package to help COLO decide
> whether to do checkpoint.
>
> Filter-rewriter
> It will rewrite some of secondary packet to make
> secondary guest's connection established successfully.
> In this module we will rewrite tcp packet's ack to the secondary
> from primary,and rewrite tcp packet's seq to the primary from
> secondary.
>
> The full version in this github:
> https://github.com/zhangckid/qemu/tree/colo-v2.7-proxy-mode-compare-and-rewriter-sep5

Look good, just find one minor issue. And please also:

- Fix the mingw build issue reported by the bot, you can test it through 
docker test
- Move the doc patch to the end of series

Thanks

>
> v13:
>    - add docs/colo-proxy.txt
>    - add MAINTAINERS
>    - remove unnecessary .h
>    - remove QTAILQ_ENTRY(CompareState)
>    - fix some comments
>    - add find_and_check_chardev() to avoid code duplication
>    - remove the "is_unix" in patch3
>    - change error_report() to trace in patch4
>    - use l2hdr_len here instead of ETH_HLEP
>    - fix code style
>    - remove colo_rm_connection()
>    - remove hashtable_size
>    - change g_queue_foreach() to g_queue_find_custom() in patch7
>    - change trace_colo_compare_tcp_miscompare() to fprintf() in patch8
>    - add codes not queue vlan packets
>
> v12:
>    - add qemu-char: Add qemu_chr_add_handlers_full() for GMaincontext
>      to this series as the first patch.
>    - update COLO net ascii figure.
>    - add chardev socket check.
>    - fix some typo.
>    - add some comments.
>    - rename net/colo-base.c to net/colo.c
>    - rename network/transport_layer to network/transport_header.
>    - move the job that clear coon_list when hashtable_size oversize
>      to connection_get.
>    - reuse connection_destroy() do colo_rm_connection().
>    - fix pkt mem leak in colo_compare_connection().
>      (result be released in g_queue_remove(), so it were not leak)
>    - rename thread_name "compare" to "colo-compare".
>    - change icmp compare to memcmp().
>
> v11:
>    - Make patch 5 to a independent patch series.
>      [PATCH V3] qemu-char: Add qemu_chr_add_handlers_full() for GMaincontext
>    - For Jason's comments, merge filter-rewriter to this series.
>      (patch 7,8,9)
>    - Add reverse_connection_key()
>    - remove conn_list in filter-rewriter
>    - remove unprocessed_connections
>    - add some comments
>
> v10:
>    - fix typo
>    - Should we make patch 5 independent with this series?
>      This patch just add a API for qemu-char.
>
> v9:
>   p5:
>    - use chr_update_read_handler_full() replace
>      the chr_update_read_handler()
>    - use io_watch_poll_prepare_full() replace
>      the io_watch_poll_prepare()
>    - use io_watch_poll_funcs_full replace
>      the io_watch_poll_funcs
>    - avoid code duplication
>
> v8:
>   p5:
>    - add new patch:
>      qemu-char: Add qemu_chr_add_handlers_full() for GMaincontext
>
> v7:
>   p5:
>     - add [PATCH]qemu-char: Fix context for g_source_attach()
>       in this patch series.
>
> v6:
>   p6:
>     - add more commit log.
>     - fix icmp comparison to compare all packet.
>
>   p5:
>     - add more cpmments in commit log.
>     - change REGULAR_CHECK_MS to REGULAR_PACKET_CHECK_MS
>     - make check old packet independent to compare thread
>     - remove thread_status
>
>   p4:
>     - change this patch only about
>       Connection and ConnectionKey.
>     - add some comments in commit log.
>     - remove mode in fill_connection_key().
>     - fix some comments and bug.
>     - move colo_conn_state to patch of
>       "work with colo-frame"
>     - remove conn_list_lock.
>     - add MAX_QUEUE_SIZE, if primary_list or
>       secondary_list biger than MAX_QUEUE_SIZE
>       we will drop packet.
>
>   p3:
>     - add new independent kernel jhash patch.
>
>   p2:
>     - add new independent colo-base patch.
>
>   p1:
>     - add a ascii figure and some comments to explain it
>     - move trace.h to p2
>     - move QTAILQ_HEAD(, CompareState) net_compares to
>       patch of "work with colo-frame"
>     - add some comments in qemu-option.hx
>
>
> v5:
>   p3:
>      - comments from Jason
>        we poll and handle chardev in comapre thread,
>        Through this way, there's no need for extra
>        synchronization with main loop
>        this depend on another patch:
>        qemu-char: Fix context for g_source_attach()
>      - remove QemuEvent
>   p2:
>      - remove conn->list_lock
>   p1:
>      - move compare_pri/sec_chr_in to p3
>      - move compare_chr_send to p2
>
> v4:
>   p4:
>      - add some comments
>      - fix some trace-events
>      - fix tcp compare error
>   p3:
>      - add rcu_read_lock().
>      - fix trace name
>      - fix jason's other comments
>      - rebase some Dave's branch function
>   p2:
>      - colo_compare_connection() change g_queue_push_head() to
>      - g_queue_push_tail() match to sorted order.
>      - remove pkt->s
>      - move data structure to colo-base.h
>      - add colo-base.c reuse codes for filter-rewriter
>      - add some filter-rewriter needs struct
>      - depends on previous SocketReadState patch
>   p1:
>      - except move qemu_chr_add_handlers()
>        to colo thread
>      - remove class_finalize
>      - remove secondary arp codes
>      - depends on previous SocketReadState patch
>
> v3:
>    - rebase colo-compare to colo-frame v2.7
>    - fix most of Dave's comments
>      (except RCU)
>    - add TCP,UDP,ICMP and other packet comparison
>    - add trace-event
>    - add some comments
>    - other bug fix
>    - add RFC index
>    - add usage in patch 1/4
>
> v2:
>    - add jhash.h
>
> v1:
>    - initial patch
>
>
> Zhang Chen (12):
>    docs: Add documentation for COLO-proxy
>    qemu-char: Add qemu_chr_add_handlers_full() for GMaincontext
>    colo-compare: introduce colo compare initialization
>    net/colo.c: add colo.c to define and handle packet
>    Jhash: add linux kernel jhashtable in qemu
>    colo-compare: track connection and enqueue packet
>    colo-compare: introduce packet comparison thread
>    colo-compare: add TCP,UDP,ICMP packet comparison
>    filter-rewriter: introduce filter-rewriter initialization
>    filter-rewriter: track connection and parse packet
>    filter-rewriter: rewrite tcp packet to keep secondary connection
>    MAINTAINERS: add maintainer for COLO-proxy
>
>   MAINTAINERS           |   9 +
>   docs/colo-proxy.txt   | 188 ++++++++++++
>   include/qemu/jhash.h  |  59 ++++
>   include/sysemu/char.h |  11 +-
>   net/Makefile.objs     |   3 +
>   net/colo-compare.c    | 776 ++++++++++++++++++++++++++++++++++++++++++++++++++
>   net/colo.c            | 211 ++++++++++++++
>   net/colo.h            |  76 +++++
>   net/filter-rewriter.c | 268 +++++++++++++++++
>   qemu-char.c           |  77 +++--
>   qemu-options.hx       |  52 ++++
>   trace-events          |  16 ++
>   vl.c                  |   4 +-
>   13 files changed, 1724 insertions(+), 26 deletions(-)
>   create mode 100644 docs/colo-proxy.txt
>   create mode 100644 include/qemu/jhash.h
>   create mode 100644 net/colo-compare.c
>   create mode 100644 net/colo.c
>   create mode 100644 net/colo.h
>   create mode 100644 net/filter-rewriter.c
>