virtio disk host cpu overhead?

virtio disk host cpu overhead?

[Thomas Fjellstrom]

I've been testing a new samsung 960 evo nvme drive i just got, and running 
tests on it in a guest shows a minimum of double the cpu time on the host as 
it does on the guest. Somehow a guest running a single thread iozone or fio 
test can use 4+ cores on the host side at 50-100%.

I have debian's 4.9.13 kernel installed on the host (4.9.2 on the guest), and 
kvm/qemu 2.8.0. I'm using libvirt and virt-manager to set up the guest, with a 
lvm volume set up on the nvme drive, and exporting a couple volumes to the 
guest which it then either uses lvm on, or testing is done raw on a secondary 
volume. (I've tried both)

Host is a consumer box with an AMD 970 board + FX-8320, and 32GB ram. Guest is 
currently allocated 8 threads, and 1GB ram which it is currently only using 
111MB of while fio is running.

I'm curious to know if this is expected behavior, misconfiguration on my end, 
or a misunderstanding of how things work?

I'd appreciate any insight or help anyone is able and willing to provide.

-- 
Thomas Fjellstrom
thomas@fjellstrom.ca

Re: virtio disk host cpu overhead?

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 587 bytes --]

On Fri, Mar 03, 2017 at 10:05:06AM -0700, Thomas Fjellstrom wrote:
> I've been testing a new samsung 960 evo nvme drive i just got, and running 
> tests on it in a guest shows a minimum of double the cpu time on the host as 
> it does on the guest. Somehow a guest running a single thread iozone or fio 
> test can use 4+ cores on the host side at 50-100%.

Are you using -drive aio=native?

Are you pinning threads on the host?

If the answer to both questions is no, then QEMU uses a thread pool for
disk I/O and those threads could be scheduled on any host CPU.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: virtio disk host cpu overhead?

[Thomas Fjellstrom]

On Friday, March 10, 2017 9:52:07 AM MST Stefan Hajnoczi wrote:
> On Fri, Mar 03, 2017 at 10:05:06AM -0700, Thomas Fjellstrom wrote:
> > I've been testing a new samsung 960 evo nvme drive i just got, and running
> > tests on it in a guest shows a minimum of double the cpu time on the host
> > as it does on the guest. Somehow a guest running a single thread iozone
> > or fio test can use 4+ cores on the host side at 50-100%.
> 
> Are you using -drive aio=native?

Yes.

> Are you pinning threads on the host?

No pinning by default.

> If the answer to both questions is no, then QEMU uses a thread pool for
> disk I/O and those threads could be scheduled on any host CPU.

But should they be pegging 4 or more cores, when the host can do the same work 
with a single core? I don't mind it spreading work out. But using 4x the cpu 
time to do the same work is a bit much.

> Stefan


-- 
Thomas Fjellstrom
thomas@fjellstrom.ca

Re: virtio disk host cpu overhead?

[Paolo Bonzini]

On 11/03/2017 17:33, Thomas Fjellstrom wrote:
>> If the answer to both questions is no, then QEMU uses a thread pool for
>> disk I/O and those threads could be scheduled on any host CPU.
> But should they be pegging 4 or more cores, when the host can do the same work 
> with a single core? I don't mind it spreading work out. But using 4x the cpu 
> time to do the same work is a bit much.

I would expect 2x, but not really more than that.  Which threads are
doing all the work, beyond the VCPU thread and the host I/O thread?

Paolo