From: Eric Dumazet <eric.dumazet@gmail.com>
To: sheng qiu <herbert1984106@gmail.com>
Cc: kvm <kvm@vger.kernel.org>,
linux-kernel@vger.kernel.org, netdev <netdev@vger.kernel.org>
Subject: Re: more interrupts (lower performance) in bare-metal compared with running VM
Date: Sat, 28 Jul 2012 06:32:45 +0200 [thread overview]
Message-ID: <1343449965.2626.13121.camel@edumazet-glaptop> (raw)
In-Reply-To: <CAB7xdi=DrE356=U1Jr1Z=ROo2X3XNM5uKcgiZJTKY+EdsTu7gw@mail.gmail.com>
On Fri, 2012-07-27 at 22:09 -0500, sheng qiu wrote:
> Hi all,
>
> i am comparing network throughput performance under bare-metal case
> with that running VM with assigned-device (assigned NIC). i have two
> physical machines (each has a 10Gbit NIC), one is used as remote
> server (run netserver) and the other is used as the target tested one
> (run netperf with different send message size, TCP_STREAM test). the
> remote NIC is connected directly with the tested NIC, both are 10Gbit.
> fore bare-metal case, i enable 1 cpu core, for VM i also configure 1
> vcpu (the memory is sufficient for both bare-metal and VM case). i
> run netperf for 120 seconds and got the following results:
>
> send message interrupts throughput (mbit/s)
> bare-metal 256 10696290 1114.84
> 512 10106786 1391.92
> 1024 10071032 1508.09
> 2048 4560857 3434.65
> 4096 3292200 4762.26
> 8192 3169801 4733.89
> 16384 2780529 4892.6
>
Are these interrupt counts taken on the receiver ?
> VM(assigned NIC) 256 3817904 2249.35
> 512 3599007 4342.81
> 1024 3005601 4134.69
> 2048 2952122 4484
> 4096 2682874 4566.34
> 8192 2786719 4734.39
> 16384 2603835 4540.47
>
> as shown, the interrupts for bare-metal case is much more than the VM
> case for some message size. we also see the throughput for those
> situations is lower than VM case. it's strange that the bare-metal has
> lower performance than the VM case. Does anyone have comments on this?
> i am very confused.
Well, I think you answered to your question. High interrupt rates
are not good for throughput. They might be good for latencies.
Using a VM adds delays and several frames might be delivered per
interrupt.
Using bare metal is faster and only one frame is delivered by NIC per
interrupt.
Try TCP_RR instead of TCP_STREAM for example.
What NIC is it exactly ? It seems it has no coalescing or LRO strategy.
ethtool -k eth0
ethtool -c eth0
What kernel version as used, because 4892 Mbits is not line rate.
parent reply other threads:[~2012-07-28 4:32 UTC|newest]
Thread overview: expand[flat|nested] mbox.gz Atom feed
[parent not found: <CAB7xdi=DrE356=U1Jr1Z=ROo2X3XNM5uKcgiZJTKY+EdsTu7gw@mail.gmail.com>]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1343449965.2626.13121.camel@edumazet-glaptop \
--to=eric.dumazet@gmail.com \
--cc=herbert1984106@gmail.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox