From: Lukas Straub <lukasstraub2@web.de>
To: Derek Su <dereksu@qnap.com>
Cc: lizhijian@cn.fujitsu.com, chyang@qnap.com, jasowang@redhat.com,
qemu-devel@nongnu.org, ctcheng@qnap.com, chen.zhang@intel.com,
jwsu1986@gmail.com
Subject: Re: [PATCH v4 2/2] net/colo-compare.c: handling of the full primary or secondary queue
Date: Wed, 8 Apr 2020 21:18:42 +0200 [thread overview]
Message-ID: <20200408211842.2c0f1e4a@luklap> (raw)
In-Reply-To: <20200328124646.7778-3-dereksu@qnap.com>
[-- Attachment #1.1: Type: text/plain, Size: 1170 bytes --]
On Sat, 28 Mar 2020 20:46:46 +0800
Derek Su <dereksu@qnap.com> wrote:
> The pervious handling of the full primary or queue is only dropping
> the packet. If there are lots of clients to the guest VM,
> the "drop" will lead to the lost of the networking connection
> until next checkpoint.
>
> To address the issue, this patch drops the packet firstly.
> Then, do checkpoint and flush packets.
>
> Signed-off-by: Derek Su <dereksu@qnap.com>
Hello,
I had a look at this again and did some benchmarking.
First just qemu 5.0-rc1 with my bugfixes ( https://lists.nongnu.org/archive/html/qemu-devel/2020-04/msg01432.html )
Then qemu 5.0-rc1 with my bugfixes and this patch series.
This commit hurts performance too much:
Client-to-server bandwidth falls from ~45.9 Mbit/s to 22.9 Mbit/s.
Server-to-client bandwidth falls from ~6.3 Mbit/s to just ~674 Kbit/s.
Average latency rises from ~197ms to ~397ms.
Meanwhile the packet loss without this commit is negligible,
only 1-2 ping packets got lost during each test run.
Instead I think we should just turn the error message
into a trace so it doesn't flood the logs.
Regards,
Lukas Straub
[-- Attachment #1.2: qemu-benchmark-summary.txt --]
[-- Type: text/plain, Size: 8864 bytes --]
5.0-rc1 with bugfixes:
*** iperf -c 192.168.178.65 ***
[ 3] local 192.168.178.68 port 33418 connected with 192.168.178.65 port 5001 64 bytes from 192.168.178.65: icmp_seq=53 ttl=64 time=2.57 ms
[ ID] Interval Transfer Bandwidth 64 bytes from 192.168.178.65: icmp_seq=54 ttl=64 time=142 ms
[ 3] 0.0-10.2 sec 60.1 MBytes 49.4 Mbits/sec 64 bytes from 192.168.178.65: icmp_seq=55 ttl=64 time=2.64 ms
[ 3] Sent 892 datagrams rtt min/avg/max/mdev = 1.920/235.514/1810.653/411.394 ms, pipe 4
[ 3] local 192.168.178.68 port 33420 connected with 192.168.178.65 port 5001 64 bytes from 192.168.178.65: icmp_seq=55 ttl=64 time=24.3 ms
[ ID] Interval Transfer Bandwidth 64 bytes from 192.168.178.65: icmp_seq=56 ttl=64 time=505 ms
[ 3] 0.0-10.3 sec 51.0 MBytes 41.7 Mbits/sec 64 bytes from 192.168.178.65: icmp_seq=57 ttl=64 time=11.7 ms
[ 3] Sent 892 datagrams rtt min/avg/max/mdev = 1.962/218.713/1365.510/344.164 ms, pipe 3
[ 3] local 192.168.178.68 port 33422 connected with 192.168.178.65 port 5001 64 bytes from 192.168.178.65: icmp_seq=54 ttl=64 time=49.2 ms
[ ID] Interval Transfer Bandwidth 64 bytes from 192.168.178.65: icmp_seq=55 ttl=64 time=587 ms
[ 3] 0.0-10.8 sec 60.1 MBytes 46.6 Mbits/sec 64 bytes from 192.168.178.65: icmp_seq=56 ttl=64 time=167 ms
[ 3] Sent 892 datagrams rtt min/avg/max/mdev = 1.573/276.375/1384.085/362.027 ms, pipe 3
*** iperf -c 192.168.178.65 -d ***
[ 4] local 192.168.178.68 port 33424 connected with 192.168.178.65 port 5001 64 bytes from 192.168.178.65: icmp_seq=60 ttl=64 time=2.27 ms
[ 5] local 192.168.178.68 port 5001 connected with 192.168.178.65 port 48440 ^C
[ ID] Interval Transfer Bandwidth --- 192.168.178.65 ping statistics ---
[ 4] 0.0-10.1 sec 60.8 MBytes 50.4 Mbits/sec 60 packets transmitted, 59 received, 1.66667% packet loss, time 29595ms
[ 5] 0.0-11.7 sec 11.2 MBytes 8.05 Mbits/sec rtt min/avg/max/mdev = 1.785/254.258/1352.916/395.070 ms, pipe 3
[ 5] local 192.168.178.68 port 33426 connected with 192.168.178.65 port 5001 64 bytes from 192.168.178.65: icmp_seq=59 ttl=64 time=2.98 ms
[ 4] local 192.168.178.68 port 5001 connected with 192.168.178.65 port 48442 ^C
[ ID] Interval Transfer Bandwidth --- 192.168.178.65 ping statistics ---
[ 5] 0.0-10.3 sec 58.6 MBytes 47.6 Mbits/sec 59 packets transmitted, 59 received, 0% packet loss, time 29088ms
[ 4] 0.0-11.9 sec 13.8 MBytes 9.77 Mbits/sec rtt min/avg/max/mdev = 2.026/207.507/1518.013/331.972 ms, pipe 4
[ 5] local 192.168.178.68 port 33432 connected with 192.168.178.65 port 5001 64 bytes from 192.168.178.65: icmp_seq=93 ttl=64 time=12.6 ms
[ 4] local 192.168.178.68 port 5001 connected with 192.168.178.65 port 48446 ^C
[ ID] Interval Transfer Bandwidth --- 192.168.178.65 ping statistics ---
[ 5] 0.0-10.2 sec 59.4 MBytes 49.0 Mbits/sec 93 packets transmitted, 92 received, 1.07527% packet loss, time 46137ms
[ 4] 0.0-31.0 sec 4.41 MBytes 1.19 Mbits/sec rtt min/avg/max/mdev = 2.107/131.212/1405.670/287.854 ms, pipe 3
5.0-rc1 with bugfixes and checkpoint on queue full:
*** iperf -c 192.168.178.65 ***
[ 3] local 192.168.178.68 port 33402 connected with 192.168.178.65 port 5001 64 bytes from 192.168.178.65: icmp_seq=52 ttl=64 time=1329 ms
[ ID] Interval Transfer Bandwidth 64 bytes from 192.168.178.65: icmp_seq=53 ttl=64 time=828 ms
[ 3] 0.0-10.1 sec 35.9 MBytes 29.9 Mbits/sec 64 bytes from 192.168.178.65: icmp_seq=54 ttl=64 time=619 ms
[ 3] Sent 892 datagrams rtt min/avg/max/mdev = 2.183/788.828/3329.045/958.683 ms, pipe 7
[ 3] local 192.168.178.68 port 33404 connected with 192.168.178.65 port 5001 64 bytes from 192.168.178.65: icmp_seq=57 ttl=64 time=2.58 ms
[ ID] Interval Transfer Bandwidth 64 bytes from 192.168.178.65: icmp_seq=58 ttl=64 time=322 ms
[ 3] 0.0-11.5 sec 36.5 MBytes 26.6 Mbits/sec 64 bytes from 192.168.178.65: icmp_seq=59 ttl=64 time=10.2 ms
[ 3] Sent 892 datagrams rtt min/avg/max/mdev = 1.962/610.122/3303.544/907.708 ms, pipe 7
[ 3] local 192.168.178.68 port 33406 connected with 192.168.178.65 port 5001 64 bytes from 192.168.178.65: icmp_seq=55 ttl=64 time=2.60 ms
[ ID] Interval Transfer Bandwidth 64 bytes from 192.168.178.65: icmp_seq=56 ttl=64 time=112 ms
[ 3] 0.0-11.1 sec 16.1 MBytes 12.2 Mbits/sec 64 bytes from 192.168.178.65: icmp_seq=57 ttl=64 time=118 ms
[ 3] Sent 892 datagrams rtt min/avg/max/mdev = 2.088/593.139/2734.770/857.021 ms, pipe 6
*** iperf -c 192.168.178.65 -d ***
[ 5] local 192.168.178.68 port 33408 connected with 192.168.178.65 port 5001 64 bytes from 192.168.178.65: icmp_seq=75 ttl=64 time=2.64 ms
[ 4] local 192.168.178.68 port 5001 connected with 192.168.178.65 port 47492 ^C
[ ID] Interval Transfer Bandwidth --- 192.168.178.65 ping statistics ---
[ 5] 0.0-10.3 sec 51.2 MBytes 41.9 Mbits/sec 75 packets transmitted, 75 received, 0% packet loss, time 37122ms
[ 4] 0.0-25.0 sec 1.00 MBytes 336 Kbits/sec rtt min/avg/max/mdev = 1.830/563.652/1996.992/651.117 ms, pipe 4
[ 4] local 192.168.178.68 port 33410 connected with 192.168.178.65 port 5001 64 bytes from 192.168.178.65: icmp_seq=127 ttl=64 time=2.41 ms
[ 5] local 192.168.178.68 port 5001 connected with 192.168.178.65 port 47494 ^C
[ ID] Interval Transfer Bandwidth --- 192.168.178.65 ping statistics ---
[ 4] 0.0-10.1 sec 39.0 MBytes 32.4 Mbits/sec 127 packets transmitted, 126 received, 0.787402% packet loss, time 63205ms
[ 5] 0.0-50.8 sec 5.01 MBytes 826 Kbits/sec rtt min/avg/max/mdev = 1.791/239.200/2105.867/513.022 ms, pipe 5
[ 4] local 192.168.178.68 port 33414 connected with 192.168.178.65 port 5001 64 bytes from 192.168.178.65: icmp_seq=90 ttl=64 time=69.4 ms
[ 5] local 192.168.178.68 port 5001 connected with 192.168.178.65 port 47496 ^C
[ ID] Interval Transfer Bandwidth --- 192.168.178.65 ping statistics ---
[ 4] 0.0-10.5 sec 46.1 MBytes 37.0 Mbits/sec 90 packets transmitted, 90 received, 0% packet loss, time 44643ms
[ 5] 0.0-31.8 sec 3.26 MBytes 861 Kbits/sec rtt min/avg/max/mdev = 1.909/391.216/1878.484/645.897 ms, pipe 4
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2020-04-08 19:25 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-28 12:46 [PATCH v4 0/2] COLO: handling of the full primary or secondary queue Derek Su
2020-03-28 12:46 ` [PATCH v4 1/2] net/colo-compare.c: Fix memory leak in packet_enqueue() Derek Su
2020-03-31 1:14 ` Zhang, Chen
2020-04-05 22:12 ` Lukas Straub
2020-03-28 12:46 ` [PATCH v4 2/2] net/colo-compare.c: handling of the full primary or secondary queue Derek Su
2020-03-31 1:15 ` Zhang, Chen
2020-04-05 22:11 ` Lukas Straub
2020-04-08 19:18 ` Lukas Straub [this message]
2020-04-09 6:59 ` Zhang, Chen
2020-04-09 7:10 ` Derek Su
2020-04-09 9:02 ` Zhang, Chen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200408211842.2c0f1e4a@luklap \
--to=lukasstraub2@web.de \
--cc=chen.zhang@intel.com \
--cc=chyang@qnap.com \
--cc=ctcheng@qnap.com \
--cc=dereksu@qnap.com \
--cc=jasowang@redhat.com \
--cc=jwsu1986@gmail.com \
--cc=lizhijian@cn.fujitsu.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).