Re: [Qemu-devel] [PATCH 1/1] virtio-mmio: return the max queue num of virtio-mmio with initial value

[Wei Huang <wei@redhat.com>]

On 07/20/2015 06:07 AM, Peter Maydell wrote:
> On 16 July 2015 at 19:38, Wei Huang <wei@redhat.com> wrote:
>> Recently we found that virtio-console devices consumes lots AArch64 guest
>> memory, roughly 1GB with 8 devices. After debugging, it turns out that lots
>> of factors contribute to this problem: i) guest PAGE_SIZE=64KB, ii)
>> virtio-mmio based devices, and iii) virtio-console device. Here is the
>> detailed analysis:
>>
>> 1. First, during initialization, virtio-mmio driver in guest pokes vq
>>    size by reading VIRTIO_MMIO_QUEUE_NUM_MAX (see virtio_mmio.c file).
>> 2. QEMU returns VIRTQUEUE_MAX_SIZE (1024) to guest VM; And virtio-mmio uses
>>    it as the default vq size.
>> 3. virtio-console driver allocates vring buffers based on this value (see
>>    add_inbuf() function of virtio_console.c file). Because PAGE_SIZE=64KB,
>>    ~64MB is allocated for each virtio-console vq.
>>
>> This patch addresses the problem by returning the iniatlized vring size
>> when VM queries QEMU about VIRTIO_MMIO_QUEUE_NUM_MAX. This is similar to
>> virtio-pci's approach. By doing this, the vq memory consumption is reduced
>> substantially.
> 
> I don't know if this patch is sensible to apply anyway, but from
> this description this really sounds like a guest kernel bug.
> QEMU tells the kernel the maximum queue size it can cope with,
> and if the guest kernel cares about not using insane amounts of
> RAM on queues then it should not blindly use the maximum size
> but restrict it itself...
Yes, this is another way of solving the problem. I think there are three
alternatives:

1. Fix the return value of MMIO_QUEUE_NUM_MAX in QEMU (my patch);
2. In guest VM, virtio-mmio shouldn't query MMIO_QUEUE_NUM_MAX. Instead,
it should behave similarly to virtio-pci driver which queries
VIRTIO_PCI_QUEUE_NUM instead. However this approach requires
modification of VIRTIO Specification as VIRTIO_PCI_QUEUE_NUM is
WRITE-ONLY in virtio-mmio.
3. Fix virtio_console driver in guest VM. This driver currently takes in
info->num and allocates memory based on its value. Apparently we can put
a upper-limit on it.

Your suggestion could fall in to (2) or (3). Any preference?

Thanks,
-Wei



> 
> thanks
> -- PMM
>