Re: [PATCH v2] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

[Denis Plotnikov <dplotnikov@virtuozzo.com>]

On 13.02.2020 14:45, Stefan Hajnoczi wrote:
> On Thu, Feb 13, 2020 at 12:28:25PM +0300, Denis Plotnikov wrote:
>>
>> On 13.02.2020 12:08, Stefan Hajnoczi wrote:
>>> On Thu, Feb 13, 2020 at 11:08:35AM +0300, Denis Plotnikov wrote:
>>>> On 12.02.2020 18:43, Stefan Hajnoczi wrote:
>>>>> On Tue, Feb 11, 2020 at 05:14:14PM +0300, Denis Plotnikov wrote:
>>>>>> The goal is to reduce the amount of requests issued by a guest on
>>>>>> 1M reads/writes. This rises the performance up to 4% on that kind of
>>>>>> disk access pattern.
>>>>>>
>>>>>> The maximum chunk size to be used for the guest disk accessing is
>>>>>> limited with seg_max parameter, which represents the max amount of
>>>>>> pices in the scatter-geather list in one guest disk request.
>>>>>>
>>>>>> Since seg_max is virqueue_size dependent, increasing the virtqueue
>>>>>> size increases seg_max, which, in turn, increases the maximum size
>>>>>> of data to be read/write from a guest disk.
>>>>>>
>>>>>> More details in the original problem statment:
>>>>>> https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html
>>>>>>
>>>>>> Suggested-by: Denis V. Lunev <den@openvz.org>
>>>>>> Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
>>>>>> ---
>>>>>>     hw/block/virtio-blk.c | 4 ++--
>>>>>>     hw/core/machine.c     | 2 ++
>>>>>>     hw/scsi/virtio-scsi.c | 4 ++--
>>>>>>     3 files changed, 6 insertions(+), 4 deletions(-)
>>>>>>
>>>>>> diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
>>>>>> index 09f46ed85f..6df3a7a6df 100644
>>>>>> --- a/hw/block/virtio-blk.c
>>>>>> +++ b/hw/block/virtio-blk.c
>>>>>> @@ -914,7 +914,7 @@ static void virtio_blk_update_config(VirtIODevice *vdev, uint8_t *config)
>>>>>>         memset(&blkcfg, 0, sizeof(blkcfg));
>>>>>>         virtio_stq_p(vdev, &blkcfg.capacity, capacity);
>>>>>>         virtio_stl_p(vdev, &blkcfg.seg_max,
>>>>>> -                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 : 128 - 2);
>>>>>> +                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 : 256 - 2);
>>>>> This value must not change on older machine types.
>>>> Yes, that's true, but ..
>>>>> So does this patch
>>>>> need to turn seg-max-adjust *on* in hw_compat_4_2 so that old machine
>>>>> types get 126 instead of 254?
>>>> If we set seg-max-adjust "on" in older machine types, the setups using them
>>>> and having queue_sizes set , for example, 1024 will also set seg_max to 1024
>>>> - 2 which isn't the expected behavior: older mt didn't change seg_max in
>>>> that case and stuck with 128 - 2.
>>>> So, should we, instead, leave the default 128 - 2, for seg_max?
>>> Argh!  Good point :-).
>>>
>>> How about a seg_max_default property that is initialized to 254 for
>>> modern machines and 126 to old machines?
>> Hmm, but we'll achieve the same but with more code changes, don't we?
>> 254 is because the queue-size is 256. We gonna leave 128-2 for older machine
>> types
>> just for not breaking anything. All other seg_max adjustment is provided by
>> seg_max_adjust which is "on" by default in modern machine types.
>>
>> to summarize:
>>
>> modern mt defaults:
>> seg_max_adjust = on
>> queue_size = 256
>>
>> => default seg_max = 254
>> => changing queue-size will change seg_max = queue_size - 2
>>
>> old mt defaults:
>> seg_max_adjust = off
>> queue_size = 128
>>
>> => default seg_max = 126
>> => changing queue-size won't change seg_max, it's always = 126 like it was
>> before
> You're right!  The only strange case is a modern machine type with
> seg_max_adjust=off, where queue_size will be 256 but seg_max will be
> 126.  But no user would want to disable seg_max_adjust, so it's okay.
>
> I agree with you that the line of code can remain unchanged:
>
>    /*
>     * Only old machine types use seg_max_adjust=off and there the default
>     * value of queue_size is 128.
>     */
>    virtio_stl_p(vdev, &blkcfg.seg_max,
>                 s->conf.seg_max_adjust ? s->conf.queue_size - 2 : 128 - 2);
>
> Stefan
Ok, I'll resend the patch sortly
Thanks!

Denis