Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

[Dongli Zhang <dongli.zhang@oracle.com>]

On 4/16/19 1:34 AM, Wei Li wrote:
> Hi @Paolo Bonzini & @Stefan Hajnoczi,
> 
> Would you please help confirm whether @Paolo Bonzini's multiqueue feature change will benefit virtio-scsi or not? Thanks!
> 
> @Stefan Hajnoczi,
> I also spent some time on exploring the virtio-scsi multi-queue features via num_queues parameter as below, here are what we found:
> 
> 1. Increase number of Queues from one to the same number as CPU will get better IOPS increase.
> 2. Increase number of Queues to the number (e.g. 8) larger than the number of vCPU (e.g. 2) can get even better IOPS increase.

As mentioned in below link, when the number of hw queues is larger than
nr_cpu_ids, the blk-mq layer would limit and only use at most nr_cpu_ids queues
(e.g., /sys/block/sda/mq/).

That is, when the num_queus=4 while vcpus is 2, there should be only 2 queues
available /sys/block/sda/mq/

https://lore.kernel.org/lkml/1553682995-5682-1-git-send-email-dongli.zhang@oracle.com/

I am just curious how increasing the num_queues from 2 to 4 would double the
iops, while there are only 2 vcpus available...

Dongli Zhang

> 
> In addition, It seems Qemu can get better IOPS while the attachment uses more queues than the number of vCPU, how could it possible? Could you please help us better understand the behavior? Thanks a lot!
> 
> 
> Host CPU Configuration:
> CPU(s):                2
> Thread(s) per core:    2
> Core(s) per socket:    1
> Socket(s):             1
> 
> Commands for multi queue Setup:
> (QEMU)  device_add driver=virtio-scsi-pci num_queues=1 id=test1
> (QEMU)  device_add driver=virtio-scsi-pci num_queues=2 id=test2
> (QEMU)  device_add driver=virtio-scsi-pci num_queues=4 id=test4
> (QEMU)  device_add driver=virtio-scsi-pci num_queues=8 id=test8
> 
> 
> Result:
> 	|  8 Queues   |  4 Queues     |      2 Queues    |      Single Queue
> IOPS 	|   +29%         |      27%           |        11%           |      Baseline
> 
> Thanks,
> Wei
> 
> On 4/5/19, 2:09 PM, "Wei Li" <wei.d.li@oracle.com> wrote:
> 
>     Thanks Stefan for your quick response!
>     
>     Hi Paolo,
>     Could you please send us a link related to the multiqueue feature which you are working on so that we could start getting some details about the feature.
>     
>     Thanks again,
>     Wei 
>     
>     On 4/1/19, 3:54 AM, "Stefan Hajnoczi" <stefanha@gmail.com> wrote:
>     
>         On Fri, Mar 29, 2019 at 08:16:36AM -0700, Wei Li wrote:
>         > Thanks Stefan for your reply and guidance!
>         > 
>         > We spent some time on exploring the multiple I/O Threads approach per your feedback. Based on the perf measurement data, we did see some IOPS improvement for multiple volumes, which is great. :)
>         > 
>         > In addition, IOPS for single Volume will still be a bottleneck, it seems like multiqueue block layer feature which Paolo is working on may be able to help improving the IOPS for single volume.
>         > 
>         > @Paolo, @Stefan, 
>         > Would you mind sharing the multiqueue feature code branch with us? So that we could get some rough idea about this feature and maybe start doing some exploration? 
>         
>         Paolo last worked on this code, so he may be able to send you a link.
>         
>         Stefan
>         
>     
> 
> 
>