From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56651) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aLjTA-0004pG-Vl for qemu-devel@nongnu.org; Tue, 19 Jan 2016 22:30:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aLjT6-0004gD-To for qemu-devel@nongnu.org; Tue, 19 Jan 2016 22:30:04 -0500 Received: from [59.151.112.132] (port=25031 helo=heian.cn.fujitsu.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aLjT6-0004Zi-14 for qemu-devel@nongnu.org; Tue, 19 Jan 2016 22:30:00 -0500 References: <1450780978-19123-1-git-send-email-zhangchen.fnst@cn.fujitsu.com> <568494B8.4080105@redhat.com> <5684E9EB.3070002@cn.fujitsu.com> <568A0527.9040001@redhat.com> <568A2A5F.3090608@cn.fujitsu.com> <568A3F80.8000806@redhat.com> <568A54C2.8050300@cn.fujitsu.com> <568CA327.4020103@redhat.com> <569C8EB7.3060507@cn.fujitsu.com> <569CB08F.4030607@redhat.com> From: Zhang Chen Message-ID: <569EFF25.2020804@cn.fujitsu.com> Date: Wed, 20 Jan 2016 11:29:41 +0800 MIME-Version: 1.0 In-Reply-To: <569CB08F.4030607@redhat.com> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 8bit Subject: Re: [Qemu-devel] [RFC PATCH v2 00/10] Add colo-proxy based on netfilter List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Jason Wang , qemu devel Cc: zhanghailiang , Li Zhijian , Gui jianfeng , "eddie.dong" , "Dr. David Alan Gilbert" , Huang peng , Gong lei , Stefan Hajnoczi , jan.kiszka@siemens.com, Yang Hongyang > Sure. > > Two main comments/suggestions: > > - TCP analysis is missed in current version, maybe you point a git tree > (or another version of RFC) to me for a better understanding of the > design. (Just a skeleton for TCP should be sufficient to discuss). > - I prefer to make the code as reusable as possible. So it's better to > split/decouple the reusable parts from the codes. So a vague idea is: > > 1) Decouple the packet comparing from the netfilter. You've achieved > this 99% since the work has been done in a thread. Just let the thread > poll sockets directly, then the comparing have the possibility to be > reused by other kinds of dataplane. > 2) Implement traffic mirror/redirector as filter. > 3) Implement TCP seq rewriting as a filter. > > Then, in primary node, you need just a traffic mirror, which did: > - mirror ingress traffic to secondary node > - mirror outgress traffic to packet comparing thread > > And in secondadry node, you need two filters: > - A TCP seq rewriter which adjust tcp sequence number. > - A traffic redirector which redirect packet from a socket as ingress > traffic, and redirect outgress traffic to the socket which could be > polled by remote packet comparing thread. > > Thoughts? > > Thanks > >> Thanks >> zhangchen > Hi, Jason. We consider your suggestion to split/decouple the reusable parts from the codes. Due to filter plugin are traversed one by one in order we will split colo-proxy to three filters in each side. But in this plan,primary and secondary both have socket server,startup is a problem. Primary qemu Secondary qemu +----------------------------------------------------------+ +-----------------------------------------------------------+ | +-----------------------------------------------------+ | | +------------------------------------------------------+ | | | | | | | | | | | guest | | | | guest | | | | | | | | | | | +-----------^--------------+--------------------------+ | | +---------------------+--------+-----------------------+ | | | | | | ^ | | | | | | | | | | | +-------------------------------------------------+ | | | | | netfilter | | | | | netfilter | | | | +-----------------------------------------------------+ | | | +------------------------------------------------------+ | | | | | filter excute order | | | | | | | filter excute order | | | | | | +-------------------> | | | | | | | +-------------------> | | | | | | | | | | | | | TCP | | | | +---------+-+ +------v-----+ +----+ +-----+ | | | | | +-----------+ +---+----+---v+rewriter+ +--------+ | | | | | | | | | | | | | | | | | | | | | | | | | | | mirror | | redirect +----> compare | | | +--------> mirror +---> adjust | adjust +-->redirect| | | | | | client | | server | | | | | | | | server | | ack | seq | |client | | | | | | | | | | | | | | | | | | | | | | | | | | +----^------+ +----^-------+ +-----+------+ | | | | +-----------+ +--------+-------------+ +----+---+ | | | | | tx | rx | rx | | | | tx all | rx | | | +-----------------------------------------------------+ | | +------------------------------------------------------+ | | | +-------------------------------------------------------------------------------------------+ | | | | | | | +----------------------------------------------------------+ +-----------------------------------------------------------+ | | |guest receive |guest send | | +--------+------------------------------------v------------+ | | | | | tap | NOTE: filter direction is rx/tx/all | | rx:receive packets sent to the netdev | | tx:receive packets sent by the netdev +----------------------------------------------------------+ guest recv packet route primary tap --> mirror client filter mirror client will send packet to guest,at the same time, copy and forward packet to secondary mirror server. secondary mirror server filter --> TCP rewriter if recv packet is TCP packet,we will adjust ack and update TCP checksum, then send to secondary guest. else directly send to guest. guest send packet route primary guest --> redirect server filter redirect server filter recv primary guest packet but do nothing, just pass to next filter. redirect server filter --> compare filter compare filter recv primary guest packet then waiting scondary redirect packet to compare it. if packet same,send primary packet and clear secondary packet, else send primary packet and do checkpoint. secondary guest --> TCP rewriter filter if the packet is TCP packet,we will adjust seq and update TCP checksum. then send it to redirect client filter. else directly send to redirect client filter. redirect client filter --> redirect server filter forward packet to primary In failover scene（primary is down）, the TCP rewriter will keep servicing for the TCP connection which is established after the last checkpoint。 How about this plan? > . > -- Thanks zhangchen