From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
To: "Michael S. Tsirkin" <mst@redhat.com>
Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
virtualization@lists.linux-foundation.org,
linux-arm-kernel@lists.infradead.org
Subject: Re: [RFC] virtio: use mandatory barriers for remote processor vdevs
Date: Mon, 12 Dec 2011 09:27:27 +1100 [thread overview]
Message-ID: <1323642447.19891.8.camel@pasglop> (raw)
In-Reply-To: <20111211122544.GC11504@redhat.com>
On Sun, 2011-12-11 at 14:25 +0200, Michael S. Tsirkin wrote:
> Forwarding some results by Amos, who run multiple netperf streams in
> parallel, from an external box to the guest. TCP_STREAM results were
> noisy. This could be due to buffering done by TCP, where packet size
> varies even as message size is constant.
>
> TCP_RR results were consistent. In this benchmark, after switching
> to mandatory barriers, CPU utilization increased by up to 35% while
> throughput went down by up to 14%. the normalized throughput/cpu
> regressed consistently, between 7 and 35%
>
> The "fix" applied was simply this:
What machine & processor was this ?
Cheers,
Ben.
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 3198f2e..fdccb77 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -23,7 +23,7 @@
>
> /* virtio guest is communicating with a virtual "device" that actually runs on
> * a host processor. Memory barriers are used to control SMP effects. */
> -#ifdef CONFIG_SMP
> +#if 0
> /* Where possible, use SMP barriers which are more lightweight than mandatory
> * barriers, because mandatory barriers control MMIO effects on accesses
> * through relaxed memory I/O windows (which virtio does not use). */
>
>
>
WARNING: multiple messages have this Message-ID (diff)
From: benh@kernel.crashing.org (Benjamin Herrenschmidt)
To: linux-arm-kernel@lists.infradead.org
Subject: [RFC] virtio: use mandatory barriers for remote processor vdevs
Date: Mon, 12 Dec 2011 09:27:27 +1100 [thread overview]
Message-ID: <1323642447.19891.8.camel@pasglop> (raw)
In-Reply-To: <20111211122544.GC11504@redhat.com>
On Sun, 2011-12-11 at 14:25 +0200, Michael S. Tsirkin wrote:
> Forwarding some results by Amos, who run multiple netperf streams in
> parallel, from an external box to the guest. TCP_STREAM results were
> noisy. This could be due to buffering done by TCP, where packet size
> varies even as message size is constant.
>
> TCP_RR results were consistent. In this benchmark, after switching
> to mandatory barriers, CPU utilization increased by up to 35% while
> throughput went down by up to 14%. the normalized throughput/cpu
> regressed consistently, between 7 and 35%
>
> The "fix" applied was simply this:
What machine & processor was this ?
Cheers,
Ben.
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 3198f2e..fdccb77 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -23,7 +23,7 @@
>
> /* virtio guest is communicating with a virtual "device" that actually runs on
> * a host processor. Memory barriers are used to control SMP effects. */
> -#ifdef CONFIG_SMP
> +#if 0
> /* Where possible, use SMP barriers which are more lightweight than mandatory
> * barriers, because mandatory barriers control MMIO effects on accesses
> * through relaxed memory I/O windows (which virtio does not use). */
>
>
>
WARNING: multiple messages have this Message-ID (diff)
From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
To: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Rusty Russell <rusty@rustcorp.com.au>,
akong@redhat.com, Ohad Ben-Cohen <ohad@wizery.com>,
virtualization@lists.linux-foundation.org, kvm@vger.kernel.org,
linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org
Subject: Re: [RFC] virtio: use mandatory barriers for remote processor vdevs
Date: Mon, 12 Dec 2011 09:27:27 +1100 [thread overview]
Message-ID: <1323642447.19891.8.camel@pasglop> (raw)
In-Reply-To: <20111211122544.GC11504@redhat.com>
On Sun, 2011-12-11 at 14:25 +0200, Michael S. Tsirkin wrote:
> Forwarding some results by Amos, who run multiple netperf streams in
> parallel, from an external box to the guest. TCP_STREAM results were
> noisy. This could be due to buffering done by TCP, where packet size
> varies even as message size is constant.
>
> TCP_RR results were consistent. In this benchmark, after switching
> to mandatory barriers, CPU utilization increased by up to 35% while
> throughput went down by up to 14%. the normalized throughput/cpu
> regressed consistently, between 7 and 35%
>
> The "fix" applied was simply this:
What machine & processor was this ?
Cheers,
Ben.
> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index 3198f2e..fdccb77 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -23,7 +23,7 @@
>
> /* virtio guest is communicating with a virtual "device" that actually runs on
> * a host processor. Memory barriers are used to control SMP effects. */
> -#ifdef CONFIG_SMP
> +#if 0
> /* Where possible, use SMP barriers which are more lightweight than mandatory
> * barriers, because mandatory barriers control MMIO effects on accesses
> * through relaxed memory I/O windows (which virtio does not use). */
>
>
>
next prev parent reply other threads:[~2011-12-11 22:27 UTC|newest]
Thread overview: 111+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-11-29 12:31 [RFC] virtio: use mandatory barriers for remote processor vdevs Ohad Ben-Cohen
2011-11-29 12:31 ` Ohad Ben-Cohen
2011-11-29 13:11 ` Michael S. Tsirkin
2011-11-29 13:11 ` Michael S. Tsirkin
2011-11-29 13:11 ` Michael S. Tsirkin
2011-11-29 13:57 ` Ohad Ben-Cohen
2011-11-29 13:57 ` Ohad Ben-Cohen
2011-11-29 13:57 ` Ohad Ben-Cohen
2011-11-29 15:16 ` Michael S. Tsirkin
2011-11-29 15:16 ` Michael S. Tsirkin
2011-11-29 15:16 ` Michael S. Tsirkin
2011-11-30 11:45 ` Ohad Ben-Cohen
2011-11-30 11:45 ` Ohad Ben-Cohen
2011-11-30 11:45 ` Ohad Ben-Cohen
2011-11-30 14:59 ` Michael S. Tsirkin
2011-11-30 14:59 ` Michael S. Tsirkin
2011-11-30 14:59 ` Michael S. Tsirkin
2011-11-30 16:04 ` Ohad Ben-Cohen
2011-11-30 16:04 ` Ohad Ben-Cohen
2011-11-30 16:04 ` Ohad Ben-Cohen
2011-11-30 16:15 ` Michael S. Tsirkin
2011-11-30 16:15 ` Michael S. Tsirkin
2011-11-30 16:15 ` Michael S. Tsirkin
2011-11-30 16:24 ` Ohad Ben-Cohen
2011-11-30 16:24 ` Ohad Ben-Cohen
2011-11-30 16:24 ` Ohad Ben-Cohen
2011-11-30 23:27 ` Ohad Ben-Cohen
2011-11-30 23:27 ` Ohad Ben-Cohen
2011-11-30 23:27 ` Ohad Ben-Cohen
2011-11-30 23:43 ` Michael S. Tsirkin
2011-11-30 23:43 ` Michael S. Tsirkin
2011-11-30 23:43 ` Michael S. Tsirkin
2011-12-01 6:20 ` Ohad Ben-Cohen
2011-12-01 6:20 ` Ohad Ben-Cohen
2011-12-01 6:20 ` Ohad Ben-Cohen
2011-11-29 15:19 ` Michael S. Tsirkin
2011-11-29 15:19 ` Michael S. Tsirkin
2011-11-29 15:19 ` Michael S. Tsirkin
2011-11-30 11:55 ` Ohad Ben-Cohen
2011-11-30 11:55 ` Ohad Ben-Cohen
2011-11-30 11:55 ` Ohad Ben-Cohen
2011-11-30 14:50 ` Michael S. Tsirkin
2011-11-30 14:50 ` Michael S. Tsirkin
2011-11-30 14:50 ` Michael S. Tsirkin
2011-11-30 22:43 ` Ohad Ben-Cohen
2011-11-30 22:43 ` Ohad Ben-Cohen
2011-11-30 22:43 ` Ohad Ben-Cohen
2011-11-30 23:13 ` Michael S. Tsirkin
2011-11-30 23:13 ` Michael S. Tsirkin
2011-11-30 23:13 ` Michael S. Tsirkin
2011-12-01 2:28 ` Rusty Russell
2011-12-01 2:28 ` Rusty Russell
2011-12-01 2:28 ` Rusty Russell
2011-12-01 7:15 ` Ohad Ben-Cohen
2011-12-01 7:15 ` Ohad Ben-Cohen
2011-12-01 7:15 ` Ohad Ben-Cohen
2011-12-01 8:12 ` Michael S. Tsirkin
2011-12-01 8:12 ` Michael S. Tsirkin
2011-12-01 8:12 ` Michael S. Tsirkin
2011-12-02 0:26 ` Rusty Russell
2011-12-02 0:26 ` Rusty Russell
2011-12-02 0:26 ` Rusty Russell
2011-12-01 6:14 ` Ohad Ben-Cohen
2011-12-01 6:14 ` Ohad Ben-Cohen
2011-12-01 6:14 ` Ohad Ben-Cohen
2011-12-01 9:09 ` Michael S. Tsirkin
2011-12-01 9:09 ` Michael S. Tsirkin
2011-12-01 9:09 ` Michael S. Tsirkin
2011-12-02 23:09 ` Benjamin Herrenschmidt
2011-12-02 23:09 ` Benjamin Herrenschmidt
2011-12-02 23:09 ` Benjamin Herrenschmidt
2011-12-03 5:14 ` Rusty Russell
2011-12-03 5:14 ` Rusty Russell
2011-12-03 5:14 ` Rusty Russell
2011-12-11 12:25 ` Michael S. Tsirkin
2011-12-11 12:25 ` Michael S. Tsirkin
2011-12-11 12:25 ` Michael S. Tsirkin
2011-12-11 22:27 ` Benjamin Herrenschmidt [this message]
2011-12-11 22:27 ` Benjamin Herrenschmidt
2011-12-11 22:27 ` Benjamin Herrenschmidt
2011-12-12 3:06 ` Amos Kong
2011-12-12 3:06 ` Amos Kong
2011-12-12 3:06 ` Amos Kong
2011-12-12 5:12 ` Rusty Russell
2011-12-12 5:12 ` Rusty Russell
2011-12-12 5:12 ` Rusty Russell
2011-12-12 23:56 ` Amos Kong
2011-12-12 23:56 ` Amos Kong
2011-12-12 23:56 ` Amos Kong
2011-12-19 2:35 ` Rusty Russell
2011-12-19 2:35 ` Rusty Russell
2011-12-19 2:35 ` Rusty Russell
2011-12-19 2:19 ` Amos Kong
2011-12-19 2:19 ` Amos Kong
2011-12-19 2:19 ` Amos Kong
2011-12-19 2:41 ` Benjamin Herrenschmidt
2011-12-19 2:41 ` Benjamin Herrenschmidt
2011-12-19 2:41 ` Benjamin Herrenschmidt
2011-12-19 7:21 ` Amos Kong
2011-12-19 7:21 ` Amos Kong
2011-12-19 7:21 ` Amos Kong
2011-12-19 2:50 ` Amos Kong
2011-12-19 2:50 ` Amos Kong
2011-12-19 2:50 ` Amos Kong
2011-12-19 8:37 ` Rusty Russell
2011-12-19 8:37 ` Rusty Russell
2011-12-19 8:37 ` Rusty Russell
2011-12-03 6:01 ` Ohad Ben-Cohen
2011-12-03 6:01 ` Ohad Ben-Cohen
2011-12-03 6:01 ` Ohad Ben-Cohen
-- strict thread matches above, loose matches on Subject: below --
2011-11-29 12:31 Ohad Ben-Cohen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1323642447.19891.8.camel@pasglop \
--to=benh@kernel.crashing.org \
--cc=kvm@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mst@redhat.com \
--cc=virtualization@lists.linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.