From: yue <kvmluck@163.com>
To: qemu-devel <qemu-devel@nongnu.org>,
stefanha@redhat.com, jasowang@redhat.com
Subject: virtio-net has a sudden high latency
Date: Sun, 29 Dec 2019 21:24:32 +0800 (CST) [thread overview]
Message-ID: <45b54112.86f0.16f51d46a01.Coremail.kvmluck@163.com> (raw)
Hi, all. virtio-net has a sudden high latency when receiving. i think it is irq related, but i have no idea to fix it.
centos 3.10.0-693.21.1.el7.x86_64 ; qemu-kvm-ev-2.6.0-28.el7_3.9.1.x86_64 ;
physical machine has 40 cpus, the two vms are on the same node, cpu load is very low.
1) pic1: inside vm1(192.168.91.4) , ping 192.168.91.5; please notice icmp seq 20
[1577252456.600426] 64 bytes from 192.168.91.5: icmp_seq=12 ttl=64 time=0.692 ms
[1577252457.600514] 64 bytes from 192.168.91.5: icmp_seq=13 ttl=64 time=0.462 ms
[1577252458.600590] 64 bytes from 192.168.91.5: icmp_seq=14 ttl=64 time=0.396 ms
[1577252459.600922] 64 bytes from 192.168.91.5: icmp_seq=15 ttl=64 time=0.888 ms
[1577252460.600445] 64 bytes from 192.168.91.5: icmp_seq=16 ttl=64 time=0.355 ms
[1577252461.601328] 64 bytes from 192.168.91.5: icmp_seq=17 ttl=64 time=0.311 ms
[1577252462.600449] 64 bytes from 192.168.91.5: icmp_seq=18 ttl=64 time=0.424 ms
[1577252463.601725] 64 bytes from 192.168.91.5: icmp_seq=19 ttl=64 time=0.322 ms
[1577252464.606752] 64 bytes from 192.168.91.5: icmp_seq=20 ttl=64 time=5.65 ms
[1577252465.603569] 64 bytes from 192.168.91.5: icmp_seq=21 ttl=64 time=0.438 ms
[1577252466.603679] 64 bytes from 192.168.91.5: icmp_seq=22 ttl=64 time=0.494 ms
[1577252467.603545] 64 bytes from 192.168.91.5: icmp_seq=23 ttl=64 time=0.431 ms
tcpdump inside vm1 ;
13:41:04.601016 IP 192.168.91.4 > 192.168.91.5: ICMP echo request, id 18375, seq 20, length 64
13:41:04.606647 IP 192.168.91.5 > 192.168.91.4: ICMP echo reply, id 18375, seq 20, length 64
2) capture packet on its backend tap device on hypervisor ; tcpdump -nn -i tap5803c575-4a icmp and host 192.168.91.5 and host 192.168.91.4
13:41:04.601166 IP 192.168.91.4 > 192.168.91.5: ICMP echo request, id 18375, seq 20, length 64
13:41:04.601362 IP 192.168.91.5 > 192.168.91.4: ICMP echo reply, id 18375, seq 20, length 64
it shows tap takes no much time between send and receive.
3)
606647 - 601362 = 5.3ms , this is consumed by virtio-net.
similarly, when 192.168.91.5 receive packets from its backend, it also has sudden high latency.
do you all have any ideas?
thanks
reply other threads:[~2019-12-29 13:41 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=45b54112.86f0.16f51d46a01.Coremail.kvmluck@163.com \
--to=kvmluck@163.com \
--cc=jasowang@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).