[Qemu-devel] [PULL 00/14] Net patches

[Qemu-devel] [PULL 00/14] Net patches

[Jason Wang]

The following changes since commit 5fdb4671b08e0d1631447e81348b2b50a6b85bf7:

  Merge remote-tracking branch 'remotes/ehabkost/tags/x86-pull-request' into staging (2015-10-06 13:42:33 +0100)

are available in the git repository at:

  https://github.com/jasowang/qemu.git tags/net-pull-request

for you to fetch changes up to 89b1273742f45c30927df203532fca0d9a3e1af7:

  tests: add test cases for netfilter object (2015-10-12 13:34:32 +0800)

----------------------------------------------------------------

----------------------------------------------------------------
Dana Rubin (1):
      net/vmxnet3: Refine l2 header validation

Jason Wang (1):
      e1000: use alias for default model

Shmulik Ladkani (2):
      vmxnet3: Support reading IMR registers on bar0
      vmxnet3: Add support for VMXNET3_CMD_GET_ADAPTIVE_RING_INFO command

Yang Hongyang (10):
      vl.c: init delayed object after net_init_clients
      init/cleanup of netfilter object
      netfilter: hook packets before net queue send
      net: merge qemu_deliver_packet and qemu_deliver_packet_iov
      net/queue: introduce NetQueueDeliverFunc
      netfilter: add an API to pass the packet to next filter
      netfilter: print filter info associate with the netdev
      net/queue: export qemu_net_queue_append_iov
      netfilter: add a netbuffer filter
      tests: add test cases for netfilter object

 hw/net/e1000.c          |   8 +-
 hw/net/vmxnet3.c        |  19 +++-
 hw/net/vmxnet3.h        |   6 +-
 hw/net/vmxnet_tx_pkt.c  |  19 +++-
 include/net/filter.h    |  77 ++++++++++++++++
 include/net/net.h       |   6 +-
 include/net/queue.h     |  20 ++++-
 include/qemu/typedefs.h |   1 +
 net/Makefile.objs       |   2 +
 net/filter-buffer.c     | 186 ++++++++++++++++++++++++++++++++++++++
 net/filter.c            | 233 ++++++++++++++++++++++++++++++++++++++++++++++++
 net/net.c               | 121 +++++++++++++++++++------
 net/queue.c             |  24 +++--
 qapi-schema.json        |  20 +++++
 qdev-monitor.c          |   1 +
 qemu-options.hx         |  17 ++++
 tests/.gitignore        |   1 +
 tests/Makefile          |   2 +
 tests/test-netfilter.c  | 200 +++++++++++++++++++++++++++++++++++++++++
 vl.c                    |  19 ++--
 20 files changed, 917 insertions(+), 65 deletions(-)
 create mode 100644 include/net/filter.h
 create mode 100644 net/filter-buffer.c
 create mode 100644 net/filter.c
 create mode 100644 tests/test-netfilter.c

-- 
2.1.4

Re: [Qemu-devel] [PULL 00/14] Net patches

[Peter Maydell]

On 12 October 2015 at 09:17, Jason Wang <jasowang@redhat.com> wrote:
> The following changes since commit 5fdb4671b08e0d1631447e81348b2b50a6b85bf7:
>
>   Merge remote-tracking branch 'remotes/ehabkost/tags/x86-pull-request' into staging (2015-10-06 13:42:33 +0100)
>
> are available in the git repository at:
>
>   https://github.com/jasowang/qemu.git tags/net-pull-request
>
> for you to fetch changes up to 89b1273742f45c30927df203532fca0d9a3e1af7:
>
>   tests: add test cases for netfilter object (2015-10-12 13:34:32 +0800)
>
> ----------------------------------------------------------------
>
> ----------------------------------------------------------------
> Dana Rubin (1):
>       net/vmxnet3: Refine l2 header validation
>
> Jason Wang (1):
>       e1000: use alias for default model
>
> Shmulik Ladkani (2):
>       vmxnet3: Support reading IMR registers on bar0
>       vmxnet3: Add support for VMXNET3_CMD_GET_ADAPTIVE_RING_INFO command
>
> Yang Hongyang (10):
>       vl.c: init delayed object after net_init_clients
>       init/cleanup of netfilter object
>       netfilter: hook packets before net queue send
>       net: merge qemu_deliver_packet and qemu_deliver_packet_iov
>       net/queue: introduce NetQueueDeliverFunc
>       netfilter: add an API to pass the packet to next filter
>       netfilter: print filter info associate with the netdev
>       net/queue: export qemu_net_queue_append_iov
>       netfilter: add a netbuffer filter
>       tests: add test cases for netfilter object
>

Applied, thanks.

-- PMM

[Qemu-devel] [PULL 00/14] Net patches

[Jason Wang]

The following changes since commit 1464ad45cd6cdeb0b5c1a54d3d3791396e47e52f:

  Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2016-03-04' into staging (2016-03-06 11:53:27 +0000)

are available in the git repository at:

  https://github.com/jasowang/qemu.git tags/net-pull-request

for you to fetch changes up to a2f2e45c6edbba9e1961056fa77c696208b40c8e:

  net: check packet payload length (2016-03-07 10:15:48 +0800)

----------------------------------------------------------------

- a new netfilter implementation: mirror
- netfilter could be disabled and enabled through qom-set now
- fix netfilter crash when specifiying wrong parameters
- rocker switch now can allow user to specifiy world
- fix OOB access for ne2000

----------------------------------------------------------------
Jason Wang (1):
      net: filter: correctly remove filter from the list during finalization

Jiri Pirko (4):
      rocker: forbid to change world type
      rocker: return -ENOMEM in case of some world alloc fails
      rocker: add name field into WorldOps ale let world specify its name
      rocker: allow user to specify rocker world by property

Paolo Bonzini (1):
      net: simplify net_init_tap_one logic

Prasad J Pandit (2):
      net: ne2000: check ring buffer control registers
      net: check packet payload length

Thomas Huth (1):
      MAINTAINERS: Add entries for include/net/ files

Vincenzo Maffione (1):
      net: netmap: probe netmap interface for virtio-net header

Zhang Chen (2):
      net/filter-mirror:Add filter-mirror
      tests/test-filter-mirror:add filter-mirror unit test

zhanghailiang (2):
      filter: Add 'status' property for filter object
      filter-buffer: Add status_changed callback processing

 MAINTAINERS                   |   2 +
 hw/net/ne2000.c               |   4 +
 hw/net/rocker/rocker.c        |  38 ++++++++-
 hw/net/rocker/rocker_fp.c     |   5 ++
 hw/net/rocker/rocker_fp.h     |   1 +
 hw/net/rocker/rocker_of_dpa.c |   1 +
 hw/net/rocker/rocker_world.c  |   7 +-
 hw/net/rocker/rocker_world.h  |   1 +
 include/net/filter.h          |   4 +
 net/Makefile.objs             |   1 +
 net/checksum.c                |  10 ++-
 net/filter-buffer.c           |  34 ++++++--
 net/filter-mirror.c           | 182 ++++++++++++++++++++++++++++++++++++++++++
 net/filter.c                  |  44 +++++++++-
 net/netmap.c                  |  59 +++++++++-----
 net/tap.c                     |   4 +-
 qemu-options.hx               |   9 ++-
 tests/.gitignore              |   1 +
 tests/Makefile                |   2 +
 tests/test-filter-mirror.c    |  90 +++++++++++++++++++++
 vl.c                          |   3 +-
 21 files changed, 460 insertions(+), 42 deletions(-)
 create mode 100644 net/filter-mirror.c
 create mode 100644 tests/test-filter-mirror.c

Re: [Qemu-devel] [PULL 00/14] Net patches

[Peter Maydell]

On 7 March 2016 at 10:12, Jason Wang <jasowang@redhat.com> wrote:
> The following changes since commit 1464ad45cd6cdeb0b5c1a54d3d3791396e47e52f:
>
>   Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2016-03-04' into staging (2016-03-06 11:53:27 +0000)
>
> are available in the git repository at:
>
>   https://github.com/jasowang/qemu.git tags/net-pull-request
>
> for you to fetch changes up to a2f2e45c6edbba9e1961056fa77c696208b40c8e:
>
>   net: check packet payload length (2016-03-07 10:15:48 +0800)
>
> ----------------------------------------------------------------
>
> - a new netfilter implementation: mirror
> - netfilter could be disabled and enabled through qom-set now
> - fix netfilter crash when specifiying wrong parameters
> - rocker switch now can allow user to specifiy world
> - fix OOB access for ne2000

Hi; I'm afraid this makes "make check" hang for me (Linux, x86-64):

TEST: tests/test-netfilter... (pid=26854)
  /i386/netfilter/addremove_one:                                       OK
  /i386/netfilter/remove_netdev_one:                                   OK
  /i386/netfilter/addremove_multi:                                     OK
  /i386/netfilter/remove_netdev_multi:                                 OK
PASS: tests/test-netfilter
TEST: tests/test-filter-mirror... (pid=26858)
  /i386/netfilter/mirror:

(consistently, every time I run make check, on the same test).

thanks
-- PMM

Re: [Qemu-devel] [PULL 00/14] Net patches

[Jason Wang]

On 03/08/2016 12:51 PM, Peter Maydell wrote:
> On 7 March 2016 at 10:12, Jason Wang <jasowang@redhat.com> wrote:
>> The following changes since commit 1464ad45cd6cdeb0b5c1a54d3d3791396e47e52f:
>>
>>   Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2016-03-04' into staging (2016-03-06 11:53:27 +0000)
>>
>> are available in the git repository at:
>>
>>   https://github.com/jasowang/qemu.git tags/net-pull-request
>>
>> for you to fetch changes up to a2f2e45c6edbba9e1961056fa77c696208b40c8e:
>>
>>   net: check packet payload length (2016-03-07 10:15:48 +0800)
>>
>> ----------------------------------------------------------------
>>
>> - a new netfilter implementation: mirror
>> - netfilter could be disabled and enabled through qom-set now
>> - fix netfilter crash when specifiying wrong parameters
>> - rocker switch now can allow user to specifiy world
>> - fix OOB access for ne2000
> Hi; I'm afraid this makes "make check" hang for me (Linux, x86-64):
>
> TEST: tests/test-netfilter... (pid=26854)
>   /i386/netfilter/addremove_one:                                       OK
>   /i386/netfilter/remove_netdev_one:                                   OK
>   /i386/netfilter/addremove_multi:                                     OK
>   /i386/netfilter/remove_netdev_multi:                                 OK
> PASS: tests/test-netfilter
> TEST: tests/test-filter-mirror... (pid=26858)
>   /i386/netfilter/mirror:
>
> (consistently, every time I run make check, on the same test).
>
> thanks
> -- PMM

Sorry, it manages to pass on my machine before submitting the pull
request. But when I re-try this several times, it fails.

This probably means we have bug in mirror implementation. Chen and
Congyang, please try to fix this bug and resubmit a new version of the
patch.

Will drop mirror from this pull request and submit a V2.

Thanks

Re: [Qemu-devel] [PULL 00/14] Net patches

[Wen Congyang]

On 03/08/2016 03:33 PM, Jason Wang wrote:
> 
> 
> On 03/08/2016 12:51 PM, Peter Maydell wrote:
>> On 7 March 2016 at 10:12, Jason Wang <jasowang@redhat.com> wrote:
>>> The following changes since commit 1464ad45cd6cdeb0b5c1a54d3d3791396e47e52f:
>>>
>>>   Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2016-03-04' into staging (2016-03-06 11:53:27 +0000)
>>>
>>> are available in the git repository at:
>>>
>>>   https://github.com/jasowang/qemu.git tags/net-pull-request
>>>
>>> for you to fetch changes up to a2f2e45c6edbba9e1961056fa77c696208b40c8e:
>>>
>>>   net: check packet payload length (2016-03-07 10:15:48 +0800)
>>>
>>> ----------------------------------------------------------------
>>>
>>> - a new netfilter implementation: mirror
>>> - netfilter could be disabled and enabled through qom-set now
>>> - fix netfilter crash when specifiying wrong parameters
>>> - rocker switch now can allow user to specifiy world
>>> - fix OOB access for ne2000
>> Hi; I'm afraid this makes "make check" hang for me (Linux, x86-64):
>>
>> TEST: tests/test-netfilter... (pid=26854)
>>   /i386/netfilter/addremove_one:                                       OK
>>   /i386/netfilter/remove_netdev_one:                                   OK
>>   /i386/netfilter/addremove_multi:                                     OK
>>   /i386/netfilter/remove_netdev_multi:                                 OK
>> PASS: tests/test-netfilter
>> TEST: tests/test-filter-mirror... (pid=26858)
>>   /i386/netfilter/mirror:
>>
>> (consistently, every time I run make check, on the same test).
>>
>> thanks
>> -- PMM
> 
> Sorry, it manages to pass on my machine before submitting the pull
> request. But when I re-try this several times, it fails.
> 
> This probably means we have bug in mirror implementation. Chen and
> Congyang, please try to fix this bug and resubmit a new version of the
> patch.
> 
> Will drop mirror from this pull request and submit a V2.

OK. what is the version of the kernel that you use?

Thanks
Wen Congyang

> 
> Thanks
> 
> 
> .
> 

Re: [Qemu-devel] [PULL 00/14] Net patches

[Jason Wang]

On 03/08/2016 03:50 PM, Wen Congyang wrote:
> On 03/08/2016 03:33 PM, Jason Wang wrote:
>>
>> On 03/08/2016 12:51 PM, Peter Maydell wrote:
>>> On 7 March 2016 at 10:12, Jason Wang <jasowang@redhat.com> wrote:
>>>> The following changes since commit 1464ad45cd6cdeb0b5c1a54d3d3791396e47e52f:
>>>>
>>>>   Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2016-03-04' into staging (2016-03-06 11:53:27 +0000)
>>>>
>>>> are available in the git repository at:
>>>>
>>>>   https://github.com/jasowang/qemu.git tags/net-pull-request
>>>>
>>>> for you to fetch changes up to a2f2e45c6edbba9e1961056fa77c696208b40c8e:
>>>>
>>>>   net: check packet payload length (2016-03-07 10:15:48 +0800)
>>>>
>>>> ----------------------------------------------------------------
>>>>
>>>> - a new netfilter implementation: mirror
>>>> - netfilter could be disabled and enabled through qom-set now
>>>> - fix netfilter crash when specifiying wrong parameters
>>>> - rocker switch now can allow user to specifiy world
>>>> - fix OOB access for ne2000
>>> Hi; I'm afraid this makes "make check" hang for me (Linux, x86-64):
>>>
>>> TEST: tests/test-netfilter... (pid=26854)
>>>   /i386/netfilter/addremove_one:                                       OK
>>>   /i386/netfilter/remove_netdev_one:                                   OK
>>>   /i386/netfilter/addremove_multi:                                     OK
>>>   /i386/netfilter/remove_netdev_multi:                                 OK
>>> PASS: tests/test-netfilter
>>> TEST: tests/test-filter-mirror... (pid=26858)
>>>   /i386/netfilter/mirror:
>>>
>>> (consistently, every time I run make check, on the same test).
>>>
>>> thanks
>>> -- PMM
>> Sorry, it manages to pass on my machine before submitting the pull
>> request. But when I re-try this several times, it fails.
>>
>> This probably means we have bug in mirror implementation. Chen and
>> Congyang, please try to fix this bug and resubmit a new version of the
>> patch.
>>
>> Will drop mirror from this pull request and submit a V2.
> OK. what is the version of the kernel that you use?

4.2 but probably unrelated. Gdb shows the test wait at recv().

>
> Thanks
> Wen Congyang
>
>> Thanks
>>
>>
>> .
>>
>
>

Re: [Qemu-devel] [PULL 00/14] Net patches

[Zhang Chen]

On 03/08/2016 03:56 PM, Jason Wang wrote:
>
> On 03/08/2016 03:50 PM, Wen Congyang wrote:
>> On 03/08/2016 03:33 PM, Jason Wang wrote:
>>> On 03/08/2016 12:51 PM, Peter Maydell wrote:
>>>> On 7 March 2016 at 10:12, Jason Wang <jasowang@redhat.com> wrote:
>>>>> The following changes since commit 1464ad45cd6cdeb0b5c1a54d3d3791396e47e52f:
>>>>>
>>>>>    Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2016-03-04' into staging (2016-03-06 11:53:27 +0000)
>>>>>
>>>>> are available in the git repository at:
>>>>>
>>>>>    https://github.com/jasowang/qemu.git tags/net-pull-request
>>>>>
>>>>> for you to fetch changes up to a2f2e45c6edbba9e1961056fa77c696208b40c8e:
>>>>>
>>>>>    net: check packet payload length (2016-03-07 10:15:48 +0800)
>>>>>
>>>>> ----------------------------------------------------------------
>>>>>
>>>>> - a new netfilter implementation: mirror
>>>>> - netfilter could be disabled and enabled through qom-set now
>>>>> - fix netfilter crash when specifiying wrong parameters
>>>>> - rocker switch now can allow user to specifiy world
>>>>> - fix OOB access for ne2000
>>>> Hi; I'm afraid this makes "make check" hang for me (Linux, x86-64):
>>>>
>>>> TEST: tests/test-netfilter... (pid=26854)
>>>>    /i386/netfilter/addremove_one:                                       OK
>>>>    /i386/netfilter/remove_netdev_one:                                   OK
>>>>    /i386/netfilter/addremove_multi:                                     OK
>>>>    /i386/netfilter/remove_netdev_multi:                                 OK
>>>> PASS: tests/test-netfilter
>>>> TEST: tests/test-filter-mirror... (pid=26858)
>>>>    /i386/netfilter/mirror:
>>>>
>>>> (consistently, every time I run make check, on the same test).
>>>>
>>>> thanks
>>>> -- PMM
>>> Sorry, it manages to pass on my machine before submitting the pull
>>> request. But when I re-try this several times, it fails.
>>>
>>> This probably means we have bug in mirror implementation. Chen and
>>> Congyang, please try to fix this bug and resubmit a new version of the
>>> patch.
>>>
>>> Will drop mirror from this pull request and submit a V2.
>> OK. what is the version of the kernel that you use?
> 4.2 but probably unrelated. Gdb shows the test wait at recv().

Hi~ Jason.

I found the reason for this problem is that
unix_connect() have not connect to sock_path before iov_send().
It need time to establish connection. so can we fix it with usleep()
like this:

     recv_sock = unix_connect(sock_path, NULL);
     g_assert_cmpint(recv_sock, !=, -1);
+    usleep(1000);

     ret = iov_send(send_sock[0], iov, 2, 0, sizeof(size) + 
sizeof(send_buf));
     g_assert_cmpint(ret, ==, sizeof(send_buf) + sizeof(size));
     close(send_sock[0]);

     ret = qemu_recv(recv_sock, &len, sizeof(len), 0);



>> Thanks
>> Wen Congyang
>>
>>> Thanks
>>>
>>>
>>> .
>>>
>>
>
>
> .
>

-- 
Thanks
zhangchen

Re: [Qemu-devel] [PULL 00/14] Net patches

[Wen Congyang]

On 03/08/2016 05:06 PM, Zhang Chen wrote:
> 
> 
> On 03/08/2016 03:56 PM, Jason Wang wrote:
>>
>> On 03/08/2016 03:50 PM, Wen Congyang wrote:
>>> On 03/08/2016 03:33 PM, Jason Wang wrote:
>>>> On 03/08/2016 12:51 PM, Peter Maydell wrote:
>>>>> On 7 March 2016 at 10:12, Jason Wang <jasowang@redhat.com> wrote:
>>>>>> The following changes since commit 1464ad45cd6cdeb0b5c1a54d3d3791396e47e52f:
>>>>>>
>>>>>>    Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2016-03-04' into staging (2016-03-06 11:53:27 +0000)
>>>>>>
>>>>>> are available in the git repository at:
>>>>>>
>>>>>>    https://github.com/jasowang/qemu.git tags/net-pull-request
>>>>>>
>>>>>> for you to fetch changes up to a2f2e45c6edbba9e1961056fa77c696208b40c8e:
>>>>>>
>>>>>>    net: check packet payload length (2016-03-07 10:15:48 +0800)
>>>>>>
>>>>>> ----------------------------------------------------------------
>>>>>>
>>>>>> - a new netfilter implementation: mirror
>>>>>> - netfilter could be disabled and enabled through qom-set now
>>>>>> - fix netfilter crash when specifiying wrong parameters
>>>>>> - rocker switch now can allow user to specifiy world
>>>>>> - fix OOB access for ne2000
>>>>> Hi; I'm afraid this makes "make check" hang for me (Linux, x86-64):
>>>>>
>>>>> TEST: tests/test-netfilter... (pid=26854)
>>>>>    /i386/netfilter/addremove_one:                                       OK
>>>>>    /i386/netfilter/remove_netdev_one:                                   OK
>>>>>    /i386/netfilter/addremove_multi:                                     OK
>>>>>    /i386/netfilter/remove_netdev_multi:                                 OK
>>>>> PASS: tests/test-netfilter
>>>>> TEST: tests/test-filter-mirror... (pid=26858)
>>>>>    /i386/netfilter/mirror:
>>>>>
>>>>> (consistently, every time I run make check, on the same test).
>>>>>
>>>>> thanks
>>>>> -- PMM
>>>> Sorry, it manages to pass on my machine before submitting the pull
>>>> request. But when I re-try this several times, it fails.
>>>>
>>>> This probably means we have bug in mirror implementation. Chen and
>>>> Congyang, please try to fix this bug and resubmit a new version of the
>>>> patch.
>>>>
>>>> Will drop mirror from this pull request and submit a V2.
>>> OK. what is the version of the kernel that you use?
>> 4.2 but probably unrelated. Gdb shows the test wait at recv().
> 
> Hi~ Jason.
> 
> I found the reason for this problem is that
> unix_connect() have not connect to sock_path before iov_send().

After unix_connect() returns, the connection is established.
qemu char device will call qemu_chr_accept() after the connection
is established. If we send data before qemu_chr_accept() is called,
the data will be dropped by qemu char device:
static int tcp_chr_write(CharDriverState *chr, const uint8_t *buf, int len)         
{                                                                               
    TCPCharDriver *s = chr->opaque;                                             
    if (s->connected) {                                                         
...
        return ret;                                                             
    } else {                                                                    
        /* XXX: indicate an error ? */                                          
        return len;                                                             
    }                                                                           
}                              

We should wait some to let qemu_chr_accept() is called before sending
data.

Thanks
Wen Congyang

> It need time to establish connection. so can we fix it with usleep()
> like this:
> 
>     recv_sock = unix_connect(sock_path, NULL);
>     g_assert_cmpint(recv_sock, !=, -1);
> +    usleep(1000);
> 
>     ret = iov_send(send_sock[0], iov, 2, 0, sizeof(size) + sizeof(send_buf));
>     g_assert_cmpint(ret, ==, sizeof(send_buf) + sizeof(size));
>     close(send_sock[0]);
> 
>     ret = qemu_recv(recv_sock, &len, sizeof(len), 0);
> 
> 
> 
>>> Thanks
>>> Wen Congyang
>>>
>>>> Thanks
>>>>
>>>>
>>>> .
>>>>
>>>
>>
>>
>> .
>>
> 

Re: [Qemu-devel] [PULL 00/14] Net patches

[Peter Maydell]

On 8 March 2016 at 16:06, Zhang Chen <zhangchen.fnst@cn.fujitsu.com> wrote:
> I found the reason for this problem is that
> unix_connect() have not connect to sock_path before iov_send().
> It need time to establish connection. so can we fix it with usleep()
> like this:
>
>     recv_sock = unix_connect(sock_path, NULL);
>     g_assert_cmpint(recv_sock, !=, -1);
> +    usleep(1000);
>
>     ret = iov_send(send_sock[0], iov, 2, 0, sizeof(size) +
> sizeof(send_buf));
>     g_assert_cmpint(ret, ==, sizeof(send_buf) + sizeof(size));
>     close(send_sock[0]);
>
>     ret = qemu_recv(recv_sock, &len, sizeof(len), 0);

I would prefer it if we could find a way to fix this race
reliably rather than just inserting a delay and hoping it
is sufficient. Otherwise the test is likely to be unreliable
if run on a heavily loaded or slow machine.

thanks
-- PMM

Re: [Qemu-devel] [PULL 00/14] Net patches

[Wen Congyang]

On 03/08/2016 05:54 PM, Peter Maydell wrote:
> On 8 March 2016 at 16:06, Zhang Chen <zhangchen.fnst@cn.fujitsu.com> wrote:
>> I found the reason for this problem is that
>> unix_connect() have not connect to sock_path before iov_send().
>> It need time to establish connection. so can we fix it with usleep()
>> like this:
>>
>>     recv_sock = unix_connect(sock_path, NULL);
>>     g_assert_cmpint(recv_sock, !=, -1);
>> +    usleep(1000);
>>
>>     ret = iov_send(send_sock[0], iov, 2, 0, sizeof(size) +
>> sizeof(send_buf));
>>     g_assert_cmpint(ret, ==, sizeof(send_buf) + sizeof(size));
>>     close(send_sock[0]);
>>
>>     ret = qemu_recv(recv_sock, &len, sizeof(len), 0);
> 
> I would prefer it if we could find a way to fix this race
> reliably rather than just inserting a delay and hoping it
> is sufficient. Otherwise the test is likely to be unreliable
> if run on a heavily loaded or slow machine.

Yes, but there is no way to know when tcp_chr_accept() is called. Add a event
to notify it?

Thanks
Wen Congyang

> 
> thanks
> -- PMM
> 
> 
> 

Re: [Qemu-devel] [PULL 00/14] Net patches

[Li Zhijian]

On 03/09/2016 09:36 AM, Wen Congyang wrote:
> On 03/08/2016 05:54 PM, Peter Maydell wrote:
>> On 8 March 2016 at 16:06, Zhang Chen <zhangchen.fnst@cn.fujitsu.com> wrote:
>>> I found the reason for this problem is that
>>> unix_connect() have not connect to sock_path before iov_send().
>>> It need time to establish connection. so can we fix it with usleep()
>>> like this:
>>>
>>>      recv_sock = unix_connect(sock_path, NULL);
>>>      g_assert_cmpint(recv_sock, !=, -1);
>>> +    usleep(1000);
>>>
>>>      ret = iov_send(send_sock[0], iov, 2, 0, sizeof(size) +
>>> sizeof(send_buf));
>>>      g_assert_cmpint(ret, ==, sizeof(send_buf) + sizeof(size));
>>>      close(send_sock[0]);
>>>
>>>      ret = qemu_recv(recv_sock, &len, sizeof(len), 0);
>>
>> I would prefer it if we could find a way to fix this race
>> reliably rather than just inserting a delay and hoping it
>> is sufficient. Otherwise the test is likely to be unreliable
>> if run on a heavily loaded or slow machine.
>
> Yes, but there is no way to know when tcp_chr_accept() is called. Add a event
> to notify it?
>
> Thanks
> Wen Congyang
>

Hi, Jason, PMM
As Congyang said that this is a bug of testcase instead of filter-mirror.
Maybe we should re-wrok the testcase, for example
- using -chardev pipe instead of -chardev socket, because we are
   intend to test the packet mirror fuction instead of -chardev socket

How about that ?


>>
>> thanks
>> -- PMM
>>
>>
>>
>
>
>
>
>

Re: [Qemu-devel] [PULL 00/14] Net patches

[Wen Congyang]

On 03/09/2016 12:26 PM, Li Zhijian wrote:
> 
> 
> On 03/09/2016 09:36 AM, Wen Congyang wrote:
>> On 03/08/2016 05:54 PM, Peter Maydell wrote:
>>> On 8 March 2016 at 16:06, Zhang Chen <zhangchen.fnst@cn.fujitsu.com> wrote:
>>>> I found the reason for this problem is that
>>>> unix_connect() have not connect to sock_path before iov_send().
>>>> It need time to establish connection. so can we fix it with usleep()
>>>> like this:
>>>>
>>>>      recv_sock = unix_connect(sock_path, NULL);
>>>>      g_assert_cmpint(recv_sock, !=, -1);
>>>> +    usleep(1000);
>>>>
>>>>      ret = iov_send(send_sock[0], iov, 2, 0, sizeof(size) +
>>>> sizeof(send_buf));
>>>>      g_assert_cmpint(ret, ==, sizeof(send_buf) + sizeof(size));
>>>>      close(send_sock[0]);
>>>>
>>>>      ret = qemu_recv(recv_sock, &len, sizeof(len), 0);
>>>
>>> I would prefer it if we could find a way to fix this race
>>> reliably rather than just inserting a delay and hoping it
>>> is sufficient. Otherwise the test is likely to be unreliable
>>> if run on a heavily loaded or slow machine.
>>
>> Yes, but there is no way to know when tcp_chr_accept() is called. Add a event
>> to notify it?
>>
>> Thanks
>> Wen Congyang
>>
> 
> Hi, Jason, PMM
> As Congyang said that this is a bug of testcase instead of filter-mirror.
> Maybe we should re-wrok the testcase, for example
> - using -chardev pipe instead of -chardev socket, because we are
>   intend to test the packet mirror fuction instead of -chardev socket

I think it is OK to change it.

Thanks
Wen Congyang

> 
> How about that ?
> 
> 
>>>
>>> thanks
>>> -- PMM
>>>
>>>
>>>
>>
>>
>>
>>
>>
> .
> 

Re: [Qemu-devel] [PULL 00/14] Net patches

[Jason Wang]

On 03/08/2016 05:54 PM, Peter Maydell wrote:
> On 8 March 2016 at 16:06, Zhang Chen <zhangchen.fnst@cn.fujitsu.com> wrote:
>> I found the reason for this problem is that
>> unix_connect() have not connect to sock_path before iov_send().
>> It need time to establish connection. so can we fix it with usleep()
>> like this:
>>
>>     recv_sock = unix_connect(sock_path, NULL);
>>     g_assert_cmpint(recv_sock, !=, -1);
>> +    usleep(1000);
>>
>>     ret = iov_send(send_sock[0], iov, 2, 0, sizeof(size) +
>> sizeof(send_buf));
>>     g_assert_cmpint(ret, ==, sizeof(send_buf) + sizeof(size));
>>     close(send_sock[0]);
>>
>>     ret = qemu_recv(recv_sock, &len, sizeof(len), 0);
> I would prefer it if we could find a way to fix this race
> reliably rather than just inserting a delay and hoping it
> is sufficient. Otherwise the test is likely to be unreliable
> if run on a heavily loaded or slow machine.
>
> thanks
> -- PMM
>

+1

To make sure the connected socket to be proceeded before iov_send(), you
could use some like  qmp("{ 'execute' : 'query-status'}") before
iov_send(). With this we are guaranteed that connected is setting to
true before iov_send().

Re: [Qemu-devel] [PULL 00/14] Net patches

[Li Zhijian]

On 03/10/2016 10:28 AM, Jason Wang wrote:
>
>
> On 03/08/2016 05:54 PM, Peter Maydell wrote:
>> On 8 March 2016 at 16:06, Zhang Chen <zhangchen.fnst@cn.fujitsu.com> wrote:
>>> I found the reason for this problem is that
>>> unix_connect() have not connect to sock_path before iov_send().
>>> It need time to establish connection. so can we fix it with usleep()
>>> like this:
>>>
>>>      recv_sock = unix_connect(sock_path, NULL);
>>>      g_assert_cmpint(recv_sock, !=, -1);
>>> +    usleep(1000);
>>>
>>>      ret = iov_send(send_sock[0], iov, 2, 0, sizeof(size) +
>>> sizeof(send_buf));
>>>      g_assert_cmpint(ret, ==, sizeof(send_buf) + sizeof(size));
>>>      close(send_sock[0]);
>>>
>>>      ret = qemu_recv(recv_sock, &len, sizeof(len), 0);
>> I would prefer it if we could find a way to fix this race
>> reliably rather than just inserting a delay and hoping it
>> is sufficient. Otherwise the test is likely to be unreliable
>> if run on a heavily loaded or slow machine.
>>
>> thanks
>> -- PMM
>>
>
> +1
>
> To make sure the connected socket to be proceeded before iov_send(), you
> could use some like  qmp("{ 'execute' : 'query-status'}") before
> iov_send(). With this we are guaranteed that connected is setting to
> true before iov_send().
>
>

it seem works, but i don't know.
Is this because that both qemu accepting the connection and qmp command are working under *iothread*,
so that when the qemu command returns, we can guaranteed the connection is accepted ?

Thanks
Li Zhijian
>
>
>

Re: [Qemu-devel] [PULL 00/14] Net patches

[Jason Wang]

On 03/10/2016 11:51 AM, Li Zhijian wrote:
>
>
> On 03/10/2016 10:28 AM, Jason Wang wrote:
>>
>>
>> On 03/08/2016 05:54 PM, Peter Maydell wrote:
>>> On 8 March 2016 at 16:06, Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
>>> wrote:
>>>> I found the reason for this problem is that
>>>> unix_connect() have not connect to sock_path before iov_send().
>>>> It need time to establish connection. so can we fix it with usleep()
>>>> like this:
>>>>
>>>>      recv_sock = unix_connect(sock_path, NULL);
>>>>      g_assert_cmpint(recv_sock, !=, -1);
>>>> +    usleep(1000);
>>>>
>>>>      ret = iov_send(send_sock[0], iov, 2, 0, sizeof(size) +
>>>> sizeof(send_buf));
>>>>      g_assert_cmpint(ret, ==, sizeof(send_buf) + sizeof(size));
>>>>      close(send_sock[0]);
>>>>
>>>>      ret = qemu_recv(recv_sock, &len, sizeof(len), 0);
>>> I would prefer it if we could find a way to fix this race
>>> reliably rather than just inserting a delay and hoping it
>>> is sufficient. Otherwise the test is likely to be unreliable
>>> if run on a heavily loaded or slow machine.
>>>
>>> thanks
>>> -- PMM
>>>
>>
>> +1
>>
>> To make sure the connected socket to be proceeded before iov_send(), you
>> could use some like  qmp("{ 'execute' : 'query-status'}") before
>> iov_send(). With this we are guaranteed that connected is setting to
>> true before iov_send().
>>
>>
>
> it seem works, but i don't know.
> Is this because that both qemu accepting the connection and qmp
> command are working under *iothread*,
> so that when the qemu command returns, we can guaranteed the
> connection is accepted ?

I think the problem is the race in main loop between the two handlers.
If the socket netdev read handler was proceed before chardev read
handler, since connected was false the packet will be dropped silently.
After we place whateve a qmp command in the middled, it's guaranteed
that all handlers were proceed after qmp command was executed, so we are
sure the connected is true when doing iov_send().

>
> Thanks
> Li Zhijian
>>
>>
>>
>
>
>

Re: [Qemu-devel] [PULL 00/14] Net patches

[Jason Wang]

On 03/09/2016 12:26 PM, Li Zhijian wrote:
>
>
> On 03/09/2016 09:36 AM, Wen Congyang wrote:
>> On 03/08/2016 05:54 PM, Peter Maydell wrote:
>>> On 8 March 2016 at 16:06, Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
>>> wrote:
>>>> I found the reason for this problem is that
>>>> unix_connect() have not connect to sock_path before iov_send().
>>>> It need time to establish connection. so can we fix it with usleep()
>>>> like this:
>>>>
>>>>      recv_sock = unix_connect(sock_path, NULL);
>>>>      g_assert_cmpint(recv_sock, !=, -1);
>>>> +    usleep(1000);
>>>>
>>>>      ret = iov_send(send_sock[0], iov, 2, 0, sizeof(size) +
>>>> sizeof(send_buf));
>>>>      g_assert_cmpint(ret, ==, sizeof(send_buf) + sizeof(size));
>>>>      close(send_sock[0]);
>>>>
>>>>      ret = qemu_recv(recv_sock, &len, sizeof(len), 0);
>>>
>>> I would prefer it if we could find a way to fix this race
>>> reliably rather than just inserting a delay and hoping it
>>> is sufficient. Otherwise the test is likely to be unreliable
>>> if run on a heavily loaded or slow machine.
>>
>> Yes, but there is no way to know when tcp_chr_accept() is called. Add
>> a event
>> to notify it?
>>
>> Thanks
>> Wen Congyang
>>
>
> Hi, Jason, PMM
> As Congyang said that this is a bug of testcase instead of filter-mirror.
> Maybe we should re-wrok the testcase, for example
> - using -chardev pipe instead of -chardev socket, because we are
>   intend to test the packet mirror fuction instead of -chardev socket
>
> How about that ?

I still prefer to use socket since it will be more similar to its real
usage (e.g COLO).

Btw, I haven't investigated deeply, but at least qmp command trick
should work. But what's better is to passing the pre-connected fd (e.g
the socket created by socketpair()) to chardev, not sure we can do this.

Re: [Qemu-devel] [PULL 00/14] Net patches

[Li Zhijian]

On 03/15/2016 11:07 AM, Jason Wang wrote:
>
>
> On 03/10/2016 11:51 AM, Li Zhijian wrote:
>>
>>
>> On 03/10/2016 10:28 AM, Jason Wang wrote:
>>>
>>>
>>> On 03/08/2016 05:54 PM, Peter Maydell wrote:
>>>> On 8 March 2016 at 16:06, Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
>>>> wrote:
>>>>> I found the reason for this problem is that
>>>>> unix_connect() have not connect to sock_path before iov_send().
>>>>> It need time to establish connection. so can we fix it with usleep()
>>>>> like this:
>>>>>
>>>>>       recv_sock = unix_connect(sock_path, NULL);
>>>>>       g_assert_cmpint(recv_sock, !=, -1);
>>>>> +    usleep(1000);
>>>>>
>>>>>       ret = iov_send(send_sock[0], iov, 2, 0, sizeof(size) +
>>>>> sizeof(send_buf));
>>>>>       g_assert_cmpint(ret, ==, sizeof(send_buf) + sizeof(size));
>>>>>       close(send_sock[0]);
>>>>>
>>>>>       ret = qemu_recv(recv_sock, &len, sizeof(len), 0);
>>>> I would prefer it if we could find a way to fix this race
>>>> reliably rather than just inserting a delay and hoping it
>>>> is sufficient. Otherwise the test is likely to be unreliable
>>>> if run on a heavily loaded or slow machine.
>>>>
>>>> thanks
>>>> -- PMM
>>>>
>>>
>>> +1
>>>
>>> To make sure the connected socket to be proceeded before iov_send(), you
>>> could use some like  qmp("{ 'execute' : 'query-status'}") before
>>> iov_send(). With this we are guaranteed that connected is setting to
>>> true before iov_send().
>>>
>>>
>>
>> it seem works, but i don't know.
>> Is this because that both qemu accepting the connection and qmp
>> command are working under *iothread*,
>> so that when the qemu command returns, we can guaranteed the
>> connection is accepted ?
>
> I think the problem is the race in main loop between the two handlers.
> If the socket netdev read handler was proceed before chardev read
> handler, since connected was false the packet will be dropped silently.
> After we place whateve a qmp command in the middled, it's guaranteed
> that all handlers were proceed after qmp command was executed, so we are
> sure the connected is true when doing iov_send().
>

Got it, thank for your explain.

we will send V9 with '-chardev socket' and add an qmp sync point soon.

Thanks
Li Zhijian

>>
>> Thanks
>> Li Zhijian
>>>
>>>
>>>
>>
>>
>>
>
>
>
> .
>

[Qemu-devel] [PULL 00/14] Net patches

[Jason Wang]

The following changes since commit 6632f6ff96f0537fc34cdc00c760656fc62e23c5:

  Merge remote-tracking branch 'remotes/famz/tags/block-and-testing-pull-request' into staging (2017-07-17 11:46:36 +0100)

are available in the git repository at:

  https://github.com/jasowang/qemu.git tags/net-pull-request

for you to fetch changes up to 189ae6bb5ce1f5a322f8691d00fe942ba43dd601:

  virtio-net: fix offload ctrl endian (2017-07-17 20:13:56 +0800)

----------------------------------------------------------------

- fix virtio-net ctrl offload endian
- vnet header support for variou COLO netfilters and compare thread

----------------------------------------------------------------
Jason Wang (1):
      virtio-net: fix offload ctrl endian

Michal Privoznik (1):
      virtion-net: Prefer is_power_of_2()

Zhang Chen (12):
      net: Add vnet_hdr_len arguments in NetClientState
      net/net.c: Add vnet_hdr support in SocketReadState
      net/filter-mirror.c: Introduce parameter for filter_send()
      net/filter-mirror.c: Make filter mirror support vnet support.
      net/filter-mirror.c: Add new option to enable vnet support for filter-redirector
      net/colo.c: Make vnet_hdr_len as packet property
      net/colo-compare.c: Introduce parameter for compare_chr_send()
      net/colo-compare.c: Make colo-compare support vnet_hdr_len
      net/colo.c: Add vnet packet parse feature in colo-proxy
      net/colo-compare.c: Add vnet packet's tcp/udp/icmp compare
      net/filter-rewriter.c: Make filter-rewriter support vnet_hdr_len
      docs/colo-proxy.txt: Update colo-proxy usage of net driver with vnet_header

 docs/colo-proxy.txt   | 26 ++++++++++++++++
 hw/net/virtio-net.c   |  4 ++-
 include/net/net.h     | 10 ++++--
 net/colo-compare.c    | 84 ++++++++++++++++++++++++++++++++++++++++++---------
 net/colo.c            |  9 +++---
 net/colo.h            |  4 ++-
 net/filter-mirror.c   | 75 +++++++++++++++++++++++++++++++++++++++++----
 net/filter-rewriter.c | 37 ++++++++++++++++++++++-
 net/net.c             | 37 ++++++++++++++++++++---
 net/socket.c          |  8 ++---
 qemu-options.hx       | 19 ++++++------
 11 files changed, 265 insertions(+), 48 deletions(-)

[Qemu-devel] [PULL 01/14] net: Add vnet_hdr_len arguments in NetClientState

[Jason Wang]

From: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>

Add vnet_hdr_len arguments in NetClientState
that make other module get real vnet_hdr_len easily.

Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 include/net/net.h | 1 +
 net/net.c         | 1 +
 2 files changed, 2 insertions(+)

diff --git a/include/net/net.h b/include/net/net.h
index 99b28d5..9a92c70 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -100,6 +100,7 @@ struct NetClientState {
     unsigned int queue_index;
     unsigned rxfilter_notify_enabled:1;
     int vring_enable;
+    int vnet_hdr_len;
     QTAILQ_HEAD(NetFilterHead, NetFilterState) filters;
 };
 
diff --git a/net/net.c b/net/net.c
index 6235aab..b62ce10 100644
--- a/net/net.c
+++ b/net/net.c
@@ -492,6 +492,7 @@ void qemu_set_vnet_hdr_len(NetClientState *nc, int len)
         return;
     }
 
+    nc->vnet_hdr_len = len;
     nc->info->set_vnet_hdr_len(nc, len);
 }
 
-- 
2.7.4

[Qemu-devel] [PULL 02/14] net/net.c: Add vnet_hdr support in SocketReadState

[Jason Wang]

From: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>

We add a flag to decide whether net_fill_rstate() need read
the vnet_hdr_len or not.

Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
Suggested-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 include/net/net.h   |  9 +++++++--
 net/colo-compare.c  |  4 ++--
 net/filter-mirror.c |  2 +-
 net/net.c           | 36 ++++++++++++++++++++++++++++++++----
 net/socket.c        |  8 ++++----
 5 files changed, 46 insertions(+), 13 deletions(-)

diff --git a/include/net/net.h b/include/net/net.h
index 9a92c70..1c55a93 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -112,9 +112,13 @@ typedef struct NICState {
 } NICState;
 
 struct SocketReadState {
-    int state; /* 0 = getting length, 1 = getting data */
+    /* 0 = getting length, 1 = getting vnet header length, 2 = getting data */
+    int state;
+    /* This flag decide whether to read the vnet_hdr_len field */
+    bool vnet_hdr;
     uint32_t index;
     uint32_t packet_len;
+    uint32_t vnet_hdr_len;
     uint8_t buf[NET_BUFSIZE];
     SocketReadStateFinalize *finalize;
 };
@@ -177,7 +181,8 @@ ssize_t qemu_deliver_packet_iov(NetClientState *sender,
 void print_net_client(Monitor *mon, NetClientState *nc);
 void hmp_info_network(Monitor *mon, const QDict *qdict);
 void net_socket_rs_init(SocketReadState *rs,
-                        SocketReadStateFinalize *finalize);
+                        SocketReadStateFinalize *finalize,
+                        bool vnet_hdr);
 
 /* NIC info */
 
diff --git a/net/colo-compare.c b/net/colo-compare.c
index abfc23c..ea9bccc 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -743,8 +743,8 @@ static void colo_compare_complete(UserCreatable *uc, Error **errp)
         return;
     }
 
-    net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize);
-    net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize);
+    net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize, false);
+    net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize, false);
 
     g_queue_init(&s->conn_list);
 
diff --git a/net/filter-mirror.c b/net/filter-mirror.c
index 6043549..1b6211b 100644
--- a/net/filter-mirror.c
+++ b/net/filter-mirror.c
@@ -229,7 +229,7 @@ static void filter_redirector_setup(NetFilterState *nf, Error **errp)
         }
     }
 
-    net_socket_rs_init(&s->rs, redirector_rs_finalize);
+    net_socket_rs_init(&s->rs, redirector_rs_finalize, false);
 
     if (s->indev) {
         chr = qemu_chr_find(s->indev);
diff --git a/net/net.c b/net/net.c
index b62ce10..0e28099 100644
--- a/net/net.c
+++ b/net/net.c
@@ -1616,11 +1616,14 @@ QemuOptsList qemu_net_opts = {
 };
 
 void net_socket_rs_init(SocketReadState *rs,
-                        SocketReadStateFinalize *finalize)
+                        SocketReadStateFinalize *finalize,
+                        bool vnet_hdr)
 {
     rs->state = 0;
+    rs->vnet_hdr = vnet_hdr;
     rs->index = 0;
     rs->packet_len = 0;
+    rs->vnet_hdr_len = 0;
     memset(rs->buf, 0, sizeof(rs->buf));
     rs->finalize = finalize;
 }
@@ -1635,8 +1638,12 @@ int net_fill_rstate(SocketReadState *rs, const uint8_t *buf, int size)
     unsigned int l;
 
     while (size > 0) {
-        /* reassemble a packet from the network */
-        switch (rs->state) { /* 0 = getting length, 1 = getting data */
+        /* Reassemble a packet from the network.
+         * 0 = getting length.
+         * 1 = getting vnet header length.
+         * 2 = getting data.
+         */
+        switch (rs->state) {
         case 0:
             l = 4 - rs->index;
             if (l > size) {
@@ -1650,10 +1657,31 @@ int net_fill_rstate(SocketReadState *rs, const uint8_t *buf, int size)
                 /* got length */
                 rs->packet_len = ntohl(*(uint32_t *)rs->buf);
                 rs->index = 0;
-                rs->state = 1;
+                if (rs->vnet_hdr) {
+                    rs->state = 1;
+                } else {
+                    rs->state = 2;
+                    rs->vnet_hdr_len = 0;
+                }
             }
             break;
         case 1:
+            l = 4 - rs->index;
+            if (l > size) {
+                l = size;
+            }
+            memcpy(rs->buf + rs->index, buf, l);
+            buf += l;
+            size -= l;
+            rs->index += l;
+            if (rs->index == 4) {
+                /* got vnet header length */
+                rs->vnet_hdr_len = ntohl(*(uint32_t *)rs->buf);
+                rs->index = 0;
+                rs->state = 2;
+            }
+            break;
+        case 2:
             l = rs->packet_len - rs->index;
             if (l > size) {
                 l = size;
diff --git a/net/socket.c b/net/socket.c
index dcae1ae..f85ef7d 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -174,7 +174,7 @@ static void net_socket_send(void *opaque)
         closesocket(s->fd);
 
         s->fd = -1;
-        net_socket_rs_init(&s->rs, net_socket_rs_finalize);
+        net_socket_rs_init(&s->rs, net_socket_rs_finalize, false);
         s->nc.link_down = true;
         memset(s->nc.info_str, 0, sizeof(s->nc.info_str));
 
@@ -366,7 +366,7 @@ static NetSocketState *net_socket_fd_init_dgram(NetClientState *peer,
     s->fd = fd;
     s->listen_fd = -1;
     s->send_fn = net_socket_send_dgram;
-    net_socket_rs_init(&s->rs, net_socket_rs_finalize);
+    net_socket_rs_init(&s->rs, net_socket_rs_finalize, false);
     net_socket_read_poll(s, true);
 
     /* mcast: save bound address as dst */
@@ -417,7 +417,7 @@ static NetSocketState *net_socket_fd_init_stream(NetClientState *peer,
 
     s->fd = fd;
     s->listen_fd = -1;
-    net_socket_rs_init(&s->rs, net_socket_rs_finalize);
+    net_socket_rs_init(&s->rs, net_socket_rs_finalize, false);
 
     /* Disable Nagle algorithm on TCP sockets to reduce latency */
     socket_set_nodelay(fd);
@@ -522,7 +522,7 @@ static int net_socket_listen_init(NetClientState *peer,
     s->fd = -1;
     s->listen_fd = fd;
     s->nc.link_down = true;
-    net_socket_rs_init(&s->rs, net_socket_rs_finalize);
+    net_socket_rs_init(&s->rs, net_socket_rs_finalize, false);
 
     qemu_set_fd_handler(s->listen_fd, net_socket_accept, NULL, s);
     return 0;
-- 
2.7.4

[Qemu-devel] [PULL 03/14] net/filter-mirror.c: Introduce parameter for filter_send()

[Jason Wang]

From: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>

This patch change the filter_send() parameter from CharBackend to MirrorState,
we can get more information like vnet_hdr(We use it to support packet with vnet_header).

Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 net/filter-mirror.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/net/filter-mirror.c b/net/filter-mirror.c
index 1b6211b..e8e5b60 100644
--- a/net/filter-mirror.c
+++ b/net/filter-mirror.c
@@ -43,7 +43,7 @@ typedef struct MirrorState {
     SocketReadState rs;
 } MirrorState;
 
-static int filter_send(CharBackend *chr_out,
+static int filter_send(MirrorState *s,
                        const struct iovec *iov,
                        int iovcnt)
 {
@@ -58,14 +58,14 @@ static int filter_send(CharBackend *chr_out,
     }
 
     len = htonl(size);
-    ret = qemu_chr_fe_write_all(chr_out, (uint8_t *)&len, sizeof(len));
+    ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)&len, sizeof(len));
     if (ret != sizeof(len)) {
         goto err;
     }
 
     buf = g_malloc(size);
     iov_to_buf(iov, iovcnt, 0, buf, size);
-    ret = qemu_chr_fe_write_all(chr_out, (uint8_t *)buf, size);
+    ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)buf, size);
     g_free(buf);
     if (ret != size) {
         goto err;
@@ -141,7 +141,7 @@ static ssize_t filter_mirror_receive_iov(NetFilterState *nf,
     MirrorState *s = FILTER_MIRROR(nf);
     int ret;
 
-    ret = filter_send(&s->chr_out, iov, iovcnt);
+    ret = filter_send(s, iov, iovcnt);
     if (ret) {
         error_report("filter mirror send failed(%s)", strerror(-ret));
     }
@@ -164,7 +164,7 @@ static ssize_t filter_redirector_receive_iov(NetFilterState *nf,
     int ret;
 
     if (qemu_chr_fe_backend_connected(&s->chr_out)) {
-        ret = filter_send(&s->chr_out, iov, iovcnt);
+        ret = filter_send(s, iov, iovcnt);
         if (ret) {
             error_report("filter redirector send failed(%s)", strerror(-ret));
         }
-- 
2.7.4

[Qemu-devel] [PULL 04/14] net/filter-mirror.c: Make filter mirror support vnet support.

[Jason Wang]

From: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>

We add the vnet_hdr_support option for filter-mirror, default is disabled.
If you use virtio-net-pci or other driver needs vnet_hdr, please enable it.
You can use it for example:
-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0,vnet_hdr_support

If it has vnet_hdr_support flag, we will change the sending packet format from
struct {int size; const uint8_t buf[];} to {int size; int vnet_hdr_len; const uint8_t buf[];}.
make other module(like colo-compare) know how to parse net packet correctly.

Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 net/filter-mirror.c | 42 +++++++++++++++++++++++++++++++++++++++++-
 qemu-options.hx     |  5 ++---
 2 files changed, 43 insertions(+), 4 deletions(-)

diff --git a/net/filter-mirror.c b/net/filter-mirror.c
index e8e5b60..32ecdb3 100644
--- a/net/filter-mirror.c
+++ b/net/filter-mirror.c
@@ -41,12 +41,14 @@ typedef struct MirrorState {
     CharBackend chr_in;
     CharBackend chr_out;
     SocketReadState rs;
+    bool vnet_hdr;
 } MirrorState;
 
 static int filter_send(MirrorState *s,
                        const struct iovec *iov,
                        int iovcnt)
 {
+    NetFilterState *nf = NETFILTER(s);
     int ret = 0;
     ssize_t size = 0;
     uint32_t len = 0;
@@ -63,6 +65,23 @@ static int filter_send(MirrorState *s,
         goto err;
     }
 
+    if (s->vnet_hdr) {
+        /*
+         * If vnet_hdr = on, we send vnet header len to make other
+         * module(like colo-compare) know how to parse net
+         * packet correctly.
+         */
+        ssize_t vnet_hdr_len;
+
+        vnet_hdr_len = nf->netdev->vnet_hdr_len;
+
+        len = htonl(vnet_hdr_len);
+        ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)&len, sizeof(len));
+        if (ret != sizeof(len)) {
+            goto err;
+        }
+    }
+
     buf = g_malloc(size);
     iov_to_buf(iov, iovcnt, 0, buf, size);
     ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)buf, size);
@@ -229,7 +248,7 @@ static void filter_redirector_setup(NetFilterState *nf, Error **errp)
         }
     }
 
-    net_socket_rs_init(&s->rs, redirector_rs_finalize, false);
+    net_socket_rs_init(&s->rs, redirector_rs_finalize, s->vnet_hdr);
 
     if (s->indev) {
         chr = qemu_chr_find(s->indev);
@@ -318,6 +337,20 @@ static void filter_mirror_set_outdev(Object *obj,
     }
 }
 
+static bool filter_mirror_get_vnet_hdr(Object *obj, Error **errp)
+{
+    MirrorState *s = FILTER_MIRROR(obj);
+
+    return s->vnet_hdr;
+}
+
+static void filter_mirror_set_vnet_hdr(Object *obj, bool value, Error **errp)
+{
+    MirrorState *s = FILTER_MIRROR(obj);
+
+    s->vnet_hdr = value;
+}
+
 static char *filter_redirector_get_outdev(Object *obj, Error **errp)
 {
     MirrorState *s = FILTER_REDIRECTOR(obj);
@@ -337,8 +370,15 @@ static void filter_redirector_set_outdev(Object *obj,
 
 static void filter_mirror_init(Object *obj)
 {
+    MirrorState *s = FILTER_MIRROR(obj);
+
     object_property_add_str(obj, "outdev", filter_mirror_get_outdev,
                             filter_mirror_set_outdev, NULL);
+
+    s->vnet_hdr = false;
+    object_property_add_bool(obj, "vnet_hdr_support",
+                             filter_mirror_get_vnet_hdr,
+                             filter_mirror_set_vnet_hdr, NULL);
 }
 
 static void filter_redirector_init(Object *obj)
diff --git a/qemu-options.hx b/qemu-options.hx
index 2cc70b9..9eee712 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -4249,10 +4249,9 @@ queue @var{all|rx|tx} is an option that can be applied to any netfilter.
 @option{tx}: the filter is attached to the transmit queue of the netdev,
              where it will receive packets sent by the netdev.
 
-@item -object filter-mirror,id=@var{id},netdev=@var{netdevid},outdev=@var{chardevid}[,queue=@var{all|rx|tx}]
+@item -object filter-mirror,id=@var{id},netdev=@var{netdevid},outdev=@var{chardevid},queue=@var{all|rx|tx}[,vnet_hdr_support]
 
-filter-mirror on netdev @var{netdevid},mirror net packet to chardev
-@var{chardevid}
+filter-mirror on netdev @var{netdevid},mirror net packet to chardev@var{chardevid}, if it has the vnet_hdr_support flag, filter-mirror will mirror packet with vnet_hdr_len.
 
 @item -object filter-redirector,id=@var{id},netdev=@var{netdevid},indev=@var{chardevid},
 outdev=@var{chardevid}[,queue=@var{all|rx|tx}]
-- 
2.7.4

[Qemu-devel] [PULL 05/14] net/filter-mirror.c: Add new option to enable vnet support for filter-redirector

[Jason Wang]

From: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>

We add the vnet_hdr_support option for filter-redirector, default is disabled.
If you use virtio-net-pci net driver or other driver needs vnet_hdr, please enable it.
Because colo-compare or other modules needs the vnet_hdr_len to parse
packet, we add this new option send the len to others.
You can use it for example:
-object filter-redirector,id=r0,netdev=hn0,queue=tx,outdev=red0,vnet_hdr_support

Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 net/filter-mirror.c | 23 +++++++++++++++++++++++
 qemu-options.hx     |  6 +++---
 2 files changed, 26 insertions(+), 3 deletions(-)

diff --git a/net/filter-mirror.c b/net/filter-mirror.c
index 32ecdb3..90e2c92 100644
--- a/net/filter-mirror.c
+++ b/net/filter-mirror.c
@@ -368,6 +368,22 @@ static void filter_redirector_set_outdev(Object *obj,
     s->outdev = g_strdup(value);
 }
 
+static bool filter_redirector_get_vnet_hdr(Object *obj, Error **errp)
+{
+    MirrorState *s = FILTER_REDIRECTOR(obj);
+
+    return s->vnet_hdr;
+}
+
+static void filter_redirector_set_vnet_hdr(Object *obj,
+                                           bool value,
+                                           Error **errp)
+{
+    MirrorState *s = FILTER_REDIRECTOR(obj);
+
+    s->vnet_hdr = value;
+}
+
 static void filter_mirror_init(Object *obj)
 {
     MirrorState *s = FILTER_MIRROR(obj);
@@ -383,10 +399,17 @@ static void filter_mirror_init(Object *obj)
 
 static void filter_redirector_init(Object *obj)
 {
+    MirrorState *s = FILTER_REDIRECTOR(obj);
+
     object_property_add_str(obj, "indev", filter_redirector_get_indev,
                             filter_redirector_set_indev, NULL);
     object_property_add_str(obj, "outdev", filter_redirector_get_outdev,
                             filter_redirector_set_outdev, NULL);
+
+    s->vnet_hdr = false;
+    object_property_add_bool(obj, "vnet_hdr_support",
+                             filter_redirector_get_vnet_hdr,
+                             filter_redirector_set_vnet_hdr, NULL);
 }
 
 static void filter_mirror_fini(Object *obj)
diff --git a/qemu-options.hx b/qemu-options.hx
index 9eee712..b86c09e 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -4253,11 +4253,11 @@ queue @var{all|rx|tx} is an option that can be applied to any netfilter.
 
 filter-mirror on netdev @var{netdevid},mirror net packet to chardev@var{chardevid}, if it has the vnet_hdr_support flag, filter-mirror will mirror packet with vnet_hdr_len.
 
-@item -object filter-redirector,id=@var{id},netdev=@var{netdevid},indev=@var{chardevid},
-outdev=@var{chardevid}[,queue=@var{all|rx|tx}]
+@item -object filter-redirector,id=@var{id},netdev=@var{netdevid},indev=@var{chardevid},outdev=@var{chardevid},queue=@var{all|rx|tx}[,vnet_hdr_support]
 
 filter-redirector on netdev @var{netdevid},redirect filter's net packet to chardev
-@var{chardevid},and redirect indev's packet to filter.
+@var{chardevid},and redirect indev's packet to filter.if it has the vnet_hdr_support flag,
+filter-redirector will redirect packet with vnet_hdr_len.
 Create a filter-redirector we need to differ outdev id from indev id, id can not
 be the same. we can just use indev or outdev, but at least one of indev or outdev
 need to be specified.
-- 
2.7.4

[Qemu-devel] [PULL 06/14] net/colo.c: Make vnet_hdr_len as packet property

[Jason Wang]

From: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>

We can use this property flush and send packet with vnet_hdr_len.

Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 net/colo-compare.c    | 8 ++++++--
 net/colo.c            | 3 ++-
 net/colo.h            | 4 +++-
 net/filter-rewriter.c | 2 +-
 4 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/net/colo-compare.c b/net/colo-compare.c
index ea9bccc..9c2b1ea 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -121,9 +121,13 @@ static int packet_enqueue(CompareState *s, int mode)
     Connection *conn;
 
     if (mode == PRIMARY_IN) {
-        pkt = packet_new(s->pri_rs.buf, s->pri_rs.packet_len);
+        pkt = packet_new(s->pri_rs.buf,
+                         s->pri_rs.packet_len,
+                         s->pri_rs.vnet_hdr_len);
     } else {
-        pkt = packet_new(s->sec_rs.buf, s->sec_rs.packet_len);
+        pkt = packet_new(s->sec_rs.buf,
+                         s->sec_rs.packet_len,
+                         s->sec_rs.vnet_hdr_len);
     }
 
     if (parse_packet_early(pkt)) {
diff --git a/net/colo.c b/net/colo.c
index 8cc166b..180eaed 100644
--- a/net/colo.c
+++ b/net/colo.c
@@ -153,13 +153,14 @@ void connection_destroy(void *opaque)
     g_slice_free(Connection, conn);
 }
 
-Packet *packet_new(const void *data, int size)
+Packet *packet_new(const void *data, int size, int vnet_hdr_len)
 {
     Packet *pkt = g_slice_new(Packet);
 
     pkt->data = g_memdup(data, size);
     pkt->size = size;
     pkt->creation_ms = qemu_clock_get_ms(QEMU_CLOCK_HOST);
+    pkt->vnet_hdr_len = vnet_hdr_len;
 
     return pkt;
 }
diff --git a/net/colo.h b/net/colo.h
index 7c524f3..caedb0d 100644
--- a/net/colo.h
+++ b/net/colo.h
@@ -43,6 +43,8 @@ typedef struct Packet {
     int size;
     /* Time of packet creation, in wall clock ms */
     int64_t creation_ms;
+    /* Get vnet_hdr_len from filter */
+    uint32_t vnet_hdr_len;
 } Packet;
 
 typedef struct ConnectionKey {
@@ -82,7 +84,7 @@ Connection *connection_get(GHashTable *connection_track_table,
                            ConnectionKey *key,
                            GQueue *conn_list);
 void connection_hashtable_reset(GHashTable *connection_track_table);
-Packet *packet_new(const void *data, int size);
+Packet *packet_new(const void *data, int size, int vnet_hdr_len);
 void packet_destroy(void *opaque, void *user_data);
 
 #endif /* QEMU_COLO_PROXY_H */
diff --git a/net/filter-rewriter.c b/net/filter-rewriter.c
index afa06e8..63256c7 100644
--- a/net/filter-rewriter.c
+++ b/net/filter-rewriter.c
@@ -158,7 +158,7 @@ static ssize_t colo_rewriter_receive_iov(NetFilterState *nf,
     char *buf = g_malloc0(size);
 
     iov_to_buf(iov, iovcnt, 0, buf, size);
-    pkt = packet_new(buf, size);
+    pkt = packet_new(buf, size, 0);
     g_free(buf);
 
     /*
-- 
2.7.4

[Qemu-devel] [PULL 07/14] net/colo-compare.c: Introduce parameter for compare_chr_send()

[Jason Wang]

From: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>

This patch change the compare_chr_send() parameter from CharBackend to CompareState,
we can get more information like vnet_hdr(We use it to support packet with vnet_header).

Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 net/colo-compare.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/net/colo-compare.c b/net/colo-compare.c
index 9c2b1ea..c5d01da 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -97,7 +97,7 @@ enum {
     SECONDARY_IN,
 };
 
-static int compare_chr_send(CharBackend *out,
+static int compare_chr_send(CompareState *s,
                             const uint8_t *buf,
                             uint32_t size);
 
@@ -483,7 +483,7 @@ static void colo_compare_connection(void *opaque, void *user_data)
         }
 
         if (result) {
-            ret = compare_chr_send(&s->chr_out, pkt->data, pkt->size);
+            ret = compare_chr_send(s, pkt->data, pkt->size);
             if (ret < 0) {
                 error_report("colo_send_primary_packet failed");
             }
@@ -504,7 +504,7 @@ static void colo_compare_connection(void *opaque, void *user_data)
     }
 }
 
-static int compare_chr_send(CharBackend *out,
+static int compare_chr_send(CompareState *s,
                             const uint8_t *buf,
                             uint32_t size)
 {
@@ -515,12 +515,12 @@ static int compare_chr_send(CharBackend *out,
         return 0;
     }
 
-    ret = qemu_chr_fe_write_all(out, (uint8_t *)&len, sizeof(len));
+    ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)&len, sizeof(len));
     if (ret != sizeof(len)) {
         goto err;
     }
 
-    ret = qemu_chr_fe_write_all(out, (uint8_t *)buf, size);
+    ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)buf, size);
     if (ret != size) {
         goto err;
     }
@@ -665,7 +665,7 @@ static void compare_pri_rs_finalize(SocketReadState *pri_rs)
 
     if (packet_enqueue(s, PRIMARY_IN)) {
         trace_colo_compare_main("primary: unsupported packet in");
-        compare_chr_send(&s->chr_out, pri_rs->buf, pri_rs->packet_len);
+        compare_chr_send(s, pri_rs->buf, pri_rs->packet_len);
     } else {
         /* compare connection */
         g_queue_foreach(&s->conn_list, colo_compare_connection, s);
@@ -774,7 +774,7 @@ static void colo_flush_packets(void *opaque, void *user_data)
 
     while (!g_queue_is_empty(&conn->primary_list)) {
         pkt = g_queue_pop_head(&conn->primary_list);
-        compare_chr_send(&s->chr_out, pkt->data, pkt->size);
+        compare_chr_send(s, pkt->data, pkt->size);
         packet_destroy(pkt, NULL);
     }
     while (!g_queue_is_empty(&conn->secondary_list)) {
-- 
2.7.4

[Qemu-devel] [PULL 08/14] net/colo-compare.c: Make colo-compare support vnet_hdr_len

[Jason Wang]

From: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>

We add the vnet_hdr_support option for colo-compare, default is disabled.
If you use virtio-net-pci or other driver needs vnet_hdr, please enable it.
You can use it for example:
-object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0,vnet_hdr_support

COLO-compare can get vnet header length from filter,
Add vnet_hdr_len to struct packet and output packet with
the vnet_hdr_len.

Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 net/colo-compare.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++-------
 qemu-options.hx    |  4 ++--
 2 files changed, 55 insertions(+), 9 deletions(-)

diff --git a/net/colo-compare.c b/net/colo-compare.c
index c5d01da..95911a2 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -73,6 +73,7 @@ typedef struct CompareState {
     CharBackend chr_out;
     SocketReadState pri_rs;
     SocketReadState sec_rs;
+    bool vnet_hdr;
 
     /* connection list: the connections belonged to this NIC could be found
      * in this list.
@@ -99,7 +100,8 @@ enum {
 
 static int compare_chr_send(CompareState *s,
                             const uint8_t *buf,
-                            uint32_t size);
+                            uint32_t size,
+                            uint32_t vnet_hdr_len);
 
 static gint seq_sorter(Packet *a, Packet *b, gpointer data)
 {
@@ -483,7 +485,10 @@ static void colo_compare_connection(void *opaque, void *user_data)
         }
 
         if (result) {
-            ret = compare_chr_send(s, pkt->data, pkt->size);
+            ret = compare_chr_send(s,
+                                   pkt->data,
+                                   pkt->size,
+                                   pkt->vnet_hdr_len);
             if (ret < 0) {
                 error_report("colo_send_primary_packet failed");
             }
@@ -506,7 +511,8 @@ static void colo_compare_connection(void *opaque, void *user_data)
 
 static int compare_chr_send(CompareState *s,
                             const uint8_t *buf,
-                            uint32_t size)
+                            uint32_t size,
+                            uint32_t vnet_hdr_len)
 {
     int ret = 0;
     uint32_t len = htonl(size);
@@ -520,6 +526,18 @@ static int compare_chr_send(CompareState *s,
         goto err;
     }
 
+    if (s->vnet_hdr) {
+        /*
+         * We send vnet header len make other module(like filter-redirector)
+         * know how to parse net packet correctly.
+         */
+        len = htonl(vnet_hdr_len);
+        ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)&len, sizeof(len));
+        if (ret != sizeof(len)) {
+            goto err;
+        }
+    }
+
     ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)buf, size);
     if (ret != size) {
         goto err;
@@ -659,13 +677,32 @@ static void compare_set_outdev(Object *obj, const char *value, Error **errp)
     s->outdev = g_strdup(value);
 }
 
+static bool compare_get_vnet_hdr(Object *obj, Error **errp)
+{
+    CompareState *s = COLO_COMPARE(obj);
+
+    return s->vnet_hdr;
+}
+
+static void compare_set_vnet_hdr(Object *obj,
+                                 bool value,
+                                 Error **errp)
+{
+    CompareState *s = COLO_COMPARE(obj);
+
+    s->vnet_hdr = value;
+}
+
 static void compare_pri_rs_finalize(SocketReadState *pri_rs)
 {
     CompareState *s = container_of(pri_rs, CompareState, pri_rs);
 
     if (packet_enqueue(s, PRIMARY_IN)) {
         trace_colo_compare_main("primary: unsupported packet in");
-        compare_chr_send(s, pri_rs->buf, pri_rs->packet_len);
+        compare_chr_send(s,
+                         pri_rs->buf,
+                         pri_rs->packet_len,
+                         pri_rs->vnet_hdr_len);
     } else {
         /* compare connection */
         g_queue_foreach(&s->conn_list, colo_compare_connection, s);
@@ -747,8 +784,8 @@ static void colo_compare_complete(UserCreatable *uc, Error **errp)
         return;
     }
 
-    net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize, false);
-    net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize, false);
+    net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize, s->vnet_hdr);
+    net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize, s->vnet_hdr);
 
     g_queue_init(&s->conn_list);
 
@@ -774,7 +811,10 @@ static void colo_flush_packets(void *opaque, void *user_data)
 
     while (!g_queue_is_empty(&conn->primary_list)) {
         pkt = g_queue_pop_head(&conn->primary_list);
-        compare_chr_send(s, pkt->data, pkt->size);
+        compare_chr_send(s,
+                         pkt->data,
+                         pkt->size,
+                         pkt->vnet_hdr_len);
         packet_destroy(pkt, NULL);
     }
     while (!g_queue_is_empty(&conn->secondary_list)) {
@@ -792,6 +832,8 @@ static void colo_compare_class_init(ObjectClass *oc, void *data)
 
 static void colo_compare_init(Object *obj)
 {
+    CompareState *s = COLO_COMPARE(obj);
+
     object_property_add_str(obj, "primary_in",
                             compare_get_pri_indev, compare_set_pri_indev,
                             NULL);
@@ -801,6 +843,10 @@ static void colo_compare_init(Object *obj)
     object_property_add_str(obj, "outdev",
                             compare_get_outdev, compare_set_outdev,
                             NULL);
+
+    s->vnet_hdr = false;
+    object_property_add_bool(obj, "vnet_hdr_support", compare_get_vnet_hdr,
+                             compare_set_vnet_hdr, NULL);
 }
 
 static void colo_compare_finalize(Object *obj)
diff --git a/qemu-options.hx b/qemu-options.hx
index b86c09e..91a25ee 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -4282,13 +4282,13 @@ Dump the network traffic on netdev @var{dev} to the file specified by
 The file format is libpcap, so it can be analyzed with tools such as tcpdump
 or Wireshark.
 
-@item -object colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid},
-outdev=@var{chardevid}
+@item -object colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid},outdev=@var{chardevid}[,vnet_hdr_support]
 
 Colo-compare gets packet from primary_in@var{chardevid} and secondary_in@var{chardevid}, than compare primary packet with
 secondary packet. If the packets are same, we will output primary
 packet to outdev@var{chardevid}, else we will notify colo-frame
 do checkpoint and send primary packet to outdev@var{chardevid}.
+if it has the vnet_hdr_support flag, colo compare will send/recv packet with vnet_hdr_len.
 
 we must use it with the help of filter-mirror and filter-redirector.
 
-- 
2.7.4

[Qemu-devel] [PULL 09/14] net/colo.c: Add vnet packet parse feature in colo-proxy

[Jason Wang]

From: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>

Make colo-compare and filter-rewriter can parse vnet packet.

Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 net/colo.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/net/colo.c b/net/colo.c
index 180eaed..28ce7c8 100644
--- a/net/colo.c
+++ b/net/colo.c
@@ -43,11 +43,11 @@ int parse_packet_early(Packet *pkt)
 {
     int network_length;
     static const uint8_t vlan[] = {0x81, 0x00};
-    uint8_t *data = pkt->data;
+    uint8_t *data = pkt->data + pkt->vnet_hdr_len;
     uint16_t l3_proto;
     ssize_t l2hdr_len = eth_get_l2_hdr_length(data);
 
-    if (pkt->size < ETH_HLEN) {
+    if (pkt->size < ETH_HLEN + pkt->vnet_hdr_len) {
         trace_colo_proxy_main("pkt->size < ETH_HLEN");
         return 1;
     }
@@ -73,7 +73,7 @@ int parse_packet_early(Packet *pkt)
     }
 
     network_length = pkt->ip->ip_hl * 4;
-    if (pkt->size < l2hdr_len + network_length) {
+    if (pkt->size < l2hdr_len + network_length + pkt->vnet_hdr_len) {
         trace_colo_proxy_main("pkt->size < network_header + network_length");
         return 1;
     }
-- 
2.7.4

[Qemu-devel] [PULL 10/14] net/colo-compare.c: Add vnet packet's tcp/udp/icmp compare

[Jason Wang]

From: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>

COLO-Proxy just focus on packet payload, so we skip vnet header.

Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 net/colo-compare.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/net/colo-compare.c b/net/colo-compare.c
index 95911a2..ca67c68 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -201,8 +201,11 @@ static int colo_packet_compare_common(Packet *ppkt, Packet *spkt, int offset)
                                    sec_ip_src, sec_ip_dst);
     }
 
+    offset = ppkt->vnet_hdr_len + offset;
+
     if (ppkt->size == spkt->size) {
-        return memcmp(ppkt->data + offset, spkt->data + offset,
+        return memcmp(ppkt->data + offset,
+                      spkt->data + offset,
                       spkt->size - offset);
     } else {
         trace_colo_compare_main("Net packet size are not the same");
@@ -261,8 +264,9 @@ static int colo_packet_compare_tcp(Packet *spkt, Packet *ppkt)
      */
     if (ptcp->th_off > 5) {
         ptrdiff_t tcp_offset;
+
         tcp_offset = ppkt->transport_header - (uint8_t *)ppkt->data
-                     + (ptcp->th_off * 4);
+                     + (ptcp->th_off * 4) - ppkt->vnet_hdr_len;
         res = colo_packet_compare_common(ppkt, spkt, tcp_offset);
     } else if (ptcp->th_sum == stcp->th_sum) {
         res = colo_packet_compare_common(ppkt, spkt, ETH_HLEN);
-- 
2.7.4

[Qemu-devel] [PULL 11/14] net/filter-rewriter.c: Make filter-rewriter support vnet_hdr_len

[Jason Wang]

From: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>

We add the vnet_hdr_support option for filter-rewriter, default is disabled.
If you use virtio-net-pci or other driver needs vnet_hdr, please enable it.
You can use it for example:
-object filter-rewriter,id=rew0,netdev=hn0,queue=all,vnet_hdr_support

We get the vnet_hdr_len from NetClientState that make us
parse net packet correctly.

Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 net/filter-rewriter.c | 37 ++++++++++++++++++++++++++++++++++++-
 qemu-options.hx       |  4 ++--
 2 files changed, 38 insertions(+), 3 deletions(-)

diff --git a/net/filter-rewriter.c b/net/filter-rewriter.c
index 63256c7..55a6cf5 100644
--- a/net/filter-rewriter.c
+++ b/net/filter-rewriter.c
@@ -17,6 +17,7 @@
 #include "qemu-common.h"
 #include "qapi/error.h"
 #include "qapi/qmp/qerror.h"
+#include "qemu/error-report.h"
 #include "qapi-visit.h"
 #include "qom/object.h"
 #include "qemu/main-loop.h"
@@ -33,6 +34,7 @@ typedef struct RewriterState {
     NetQueue *incoming_queue;
     /* hashtable to save connection */
     GHashTable *connection_track_table;
+    bool vnet_hdr;
 } RewriterState;
 
 static void filter_rewriter_flush(NetFilterState *nf)
@@ -155,10 +157,16 @@ static ssize_t colo_rewriter_receive_iov(NetFilterState *nf,
     ConnectionKey key;
     Packet *pkt;
     ssize_t size = iov_size(iov, iovcnt);
+    ssize_t vnet_hdr_len = 0;
     char *buf = g_malloc0(size);
 
     iov_to_buf(iov, iovcnt, 0, buf, size);
-    pkt = packet_new(buf, size, 0);
+
+    if (s->vnet_hdr) {
+        vnet_hdr_len = nf->netdev->vnet_hdr_len;
+    }
+
+    pkt = packet_new(buf, size, vnet_hdr_len);
     g_free(buf);
 
     /*
@@ -237,6 +245,32 @@ static void colo_rewriter_setup(NetFilterState *nf, Error **errp)
     s->incoming_queue = qemu_new_net_queue(qemu_netfilter_pass_to_next, nf);
 }
 
+static bool filter_rewriter_get_vnet_hdr(Object *obj, Error **errp)
+{
+    RewriterState *s = FILTER_COLO_REWRITER(obj);
+
+    return s->vnet_hdr;
+}
+
+static void filter_rewriter_set_vnet_hdr(Object *obj,
+                                         bool value,
+                                         Error **errp)
+{
+    RewriterState *s = FILTER_COLO_REWRITER(obj);
+
+    s->vnet_hdr = value;
+}
+
+static void filter_rewriter_init(Object *obj)
+{
+    RewriterState *s = FILTER_COLO_REWRITER(obj);
+
+    s->vnet_hdr = false;
+    object_property_add_bool(obj, "vnet_hdr_support",
+                             filter_rewriter_get_vnet_hdr,
+                             filter_rewriter_set_vnet_hdr, NULL);
+}
+
 static void colo_rewriter_class_init(ObjectClass *oc, void *data)
 {
     NetFilterClass *nfc = NETFILTER_CLASS(oc);
@@ -250,6 +284,7 @@ static const TypeInfo colo_rewriter_info = {
     .name = TYPE_FILTER_REWRITER,
     .parent = TYPE_NETFILTER,
     .class_init = colo_rewriter_class_init,
+    .instance_init = filter_rewriter_init,
     .instance_size = sizeof(RewriterState),
 };
 
diff --git a/qemu-options.hx b/qemu-options.hx
index 91a25ee..e0e04c8 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -4262,12 +4262,12 @@ Create a filter-redirector we need to differ outdev id from indev id, id can not
 be the same. we can just use indev or outdev, but at least one of indev or outdev
 need to be specified.
 
-@item -object filter-rewriter,id=@var{id},netdev=@var{netdevid}[,queue=@var{all|rx|tx}]
+@item -object filter-rewriter,id=@var{id},netdev=@var{netdevid},queue=@var{all|rx|tx},[vnet_hdr_support]
 
 Filter-rewriter is a part of COLO project.It will rewrite tcp packet to
 secondary from primary to keep secondary tcp connection,and rewrite
 tcp packet to primary from secondary make tcp packet can be handled by
-client.
+client.if it has the vnet_hdr_support flag, we can parse packet with vnet header.
 
 usage:
 colo secondary:
-- 
2.7.4

[Qemu-devel] [PULL 12/14] docs/colo-proxy.txt: Update colo-proxy usage of net driver with vnet_header

[Jason Wang]

From: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>

Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 docs/colo-proxy.txt | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/docs/colo-proxy.txt b/docs/colo-proxy.txt
index c4941de..f6a624f 100644
--- a/docs/colo-proxy.txt
+++ b/docs/colo-proxy.txt
@@ -182,6 +182,32 @@ Secondary(ip:3.3.3.8):
 -chardev socket,id=red1,host=3.3.3.3,port=9004
 -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
 -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
+-object filter-rewriter,id=f3,netdev=hn0,queue=all
+
+If you want to use virtio-net-pci or other driver with vnet_header:
+
+Primary(ip:3.3.3.3):
+-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
+-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
+-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
+-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
+-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
+-chardev socket,id=compare0-0,host=3.3.3.3,port=9001
+-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
+-chardev socket,id=compare_out0,host=3.3.3.3,port=9005
+-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0,vnet_hdr_support
+-object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out,vnet_hdr_support
+-object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0,vnet_hdr_support
+-object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0,vnet_hdr_support
+
+Secondary(ip:3.3.3.8):
+-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown
+-device e1000,netdev=hn0,mac=52:a4:00:12:78:66
+-chardev socket,id=red0,host=3.3.3.3,port=9003
+-chardev socket,id=red1,host=3.3.3.3,port=9004
+-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0,vnet_hdr_support
+-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1,vnet_hdr_support
+-object filter-rewriter,id=f3,netdev=hn0,queue=all,vnet_hdr_support
 
 Note:
   a.COLO-proxy must work with COLO-frame and Block-replication.
-- 
2.7.4

[Qemu-devel] [PULL 13/14] virtion-net: Prefer is_power_of_2()

[Jason Wang]

From: Michal Privoznik <mprivozn@redhat.com>

We have a function that checks if given number is power of two.
We should prefer it instead of expanding the check on our own.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 hw/net/virtio-net.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 5630a9e..657d099 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -1942,7 +1942,7 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
      */
     if (n->net_conf.rx_queue_size < VIRTIO_NET_RX_QUEUE_MIN_SIZE ||
         n->net_conf.rx_queue_size > VIRTQUEUE_MAX_SIZE ||
-        (n->net_conf.rx_queue_size & (n->net_conf.rx_queue_size - 1))) {
+        !is_power_of_2(n->net_conf.rx_queue_size)) {
         error_setg(errp, "Invalid rx_queue_size (= %" PRIu16 "), "
                    "must be a power of 2 between %d and %d.",
                    n->net_conf.rx_queue_size, VIRTIO_NET_RX_QUEUE_MIN_SIZE,
-- 
2.7.4

[Qemu-devel] [PULL 14/14] virtio-net: fix offload ctrl endian

[Jason Wang]

Spec said offloads should be le64, so use virtio_ldq_p() to guarantee
valid endian.

Fixes: 644c98587d4c ("virtio-net: dynamic network offloads configuration")
Cc: qemu-stable@nongnu.org
Cc: Dmitry Fleytman <dfleytma@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 hw/net/virtio-net.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 657d099..148071a 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -758,6 +758,8 @@ static int virtio_net_handle_offloads(VirtIONet *n, uint8_t cmd,
     if (cmd == VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET) {
         uint64_t supported_offloads;
 
+        offloads = virtio_ldq_p(vdev, &offloads);
+
         if (!n->has_vnet_hdr) {
             return VIRTIO_NET_ERR;
         }
-- 
2.7.4

Re: [Qemu-devel] [PULL 00/14] Net patches

[Peter Maydell]

On 17 July 2017 at 13:21, Jason Wang <jasowang@redhat.com> wrote:
> The following changes since commit 6632f6ff96f0537fc34cdc00c760656fc62e23c5:
>
>   Merge remote-tracking branch 'remotes/famz/tags/block-and-testing-pull-request' into staging (2017-07-17 11:46:36 +0100)
>
> are available in the git repository at:
>
>   https://github.com/jasowang/qemu.git tags/net-pull-request
>
> for you to fetch changes up to 189ae6bb5ce1f5a322f8691d00fe942ba43dd601:
>
>   virtio-net: fix offload ctrl endian (2017-07-17 20:13:56 +0800)
>
> ----------------------------------------------------------------
>
> - fix virtio-net ctrl offload endian
> - vnet header support for variou COLO netfilters and compare thread
>
> ----------------------------------------------------------------
> Jason Wang (1):
>       virtio-net: fix offload ctrl endian
>
> Michal Privoznik (1):
>       virtion-net: Prefer is_power_of_2()
>
> Zhang Chen (12):
>       net: Add vnet_hdr_len arguments in NetClientState
>       net/net.c: Add vnet_hdr support in SocketReadState
>       net/filter-mirror.c: Introduce parameter for filter_send()
>       net/filter-mirror.c: Make filter mirror support vnet support.
>       net/filter-mirror.c: Add new option to enable vnet support for filter-redirector
>       net/colo.c: Make vnet_hdr_len as packet property
>       net/colo-compare.c: Introduce parameter for compare_chr_send()
>       net/colo-compare.c: Make colo-compare support vnet_hdr_len
>       net/colo.c: Add vnet packet parse feature in colo-proxy
>       net/colo-compare.c: Add vnet packet's tcp/udp/icmp compare
>       net/filter-rewriter.c: Make filter-rewriter support vnet_hdr_len
>       docs/colo-proxy.txt: Update colo-proxy usage of net driver with vnet_header

Applied, thanks.

-- PMM