Re: virtio-net mq vq initialization (was: [PATCH 0/5] Usual batch of random ARM fixes for kvmtool)

["Michael S. Tsirkin" <mst@redhat.com>]

On Fri, Apr 12, 2013 at 01:41:58PM +0100, Will Deacon wrote:
> Hi Rusty,
> 
> On Fri, Apr 12, 2013 at 12:36:00PM +0100, Rusty Russell wrote:
> > I'm confused.  kvmtool is using too much memory, or the guest?  If
> > kvmtool, the Device Initialization section above applies to the driver,
> > not the device.  If the guest, well, the language says "UP TO N+1".  You
> > want a small guest, don't use them all.  Or any...
> > 
> > What am I missing?
> 
> I don't think you're missing anything. The memory usage is in the guest, not
> kvmtool. If, as you suggest, the spec doesn't mandate greedy initialisation
> of all the virtqueues, then it sounds like a bug in Linux.
> 
> Looking at virtio_mmio.c:
> 
>         /* Allocate pages for the queue - start with a queue as big as
>          * possible (limited by maximum size allowed by device), drop down
>          * to a minimal size, just big enough to fit descriptor table
>          * and two rings (which makes it "alignment_size * 2")
>          */
>         info->num = readl(vm_dev->base + VIRTIO_MMIO_QUEUE_NUM_MAX);
> 
>         /* If the device reports a 0 entry queue, we won't be able to
>          * use it to perform I/O, and vring_new_virtqueue() can't create
>          * empty queues anyway, so don't bother to set up the device.
>          */
>         if (info->num == 0) {
>                 err = -ENOENT;
>                 goto error_alloc_pages;
>         }
> 
>         while (1) {
>                 size = PAGE_ALIGN(vring_size(info->num,
>                                 VIRTIO_MMIO_VRING_ALIGN));
>                 /* Did the last iter shrink the queue below minimum size? */
>                 if (size < VIRTIO_MMIO_VRING_ALIGN * 2) {
>                         err = -ENOMEM;
>                         goto error_alloc_pages;
>                 }
> 
>                 info->queue = alloc_pages_exact(size, GFP_KERNEL | __GFP_ZERO);
>                 if (info->queue)
>                         break;
> 
>                 info->num /= 2;
>         }
> 
> so, although there is some back-off if the allocation fails, it doesn't really
> help userspace when there is enough memory for the maximal buffers, but not
> a lot else.
> 
> Will

Well, that's just a couple of pages per virtqueue. How many of these do
you have?

-- 
MST