From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44138) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZHD9k-0000Rm-M0 for qemu-devel@nongnu.org; Mon, 20 Jul 2015 11:39:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZHD9h-000491-B9 for qemu-devel@nongnu.org; Mon, 20 Jul 2015 11:39:04 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36993) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZHD9h-00048t-6a for qemu-devel@nongnu.org; Mon, 20 Jul 2015 11:39:01 -0400 Message-ID: <55AD1612.9090603@redhat.com> Date: Mon, 20 Jul 2015 10:38:58 -0500 From: Wei Huang MIME-Version: 1.0 References: <1437071893-19457-1-git-send-email-wei@redhat.com> In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH 1/1] virtio-mmio: return the max queue num of virtio-mmio with initial value List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Peter Maydell Cc: Amit Shah , Andrew Jones , QEMU Developers , "Michael S. Tsirkin" On 07/20/2015 06:07 AM, Peter Maydell wrote: > On 16 July 2015 at 19:38, Wei Huang wrote: >> Recently we found that virtio-console devices consumes lots AArch64 guest >> memory, roughly 1GB with 8 devices. After debugging, it turns out that lots >> of factors contribute to this problem: i) guest PAGE_SIZE=64KB, ii) >> virtio-mmio based devices, and iii) virtio-console device. Here is the >> detailed analysis: >> >> 1. First, during initialization, virtio-mmio driver in guest pokes vq >> size by reading VIRTIO_MMIO_QUEUE_NUM_MAX (see virtio_mmio.c file). >> 2. QEMU returns VIRTQUEUE_MAX_SIZE (1024) to guest VM; And virtio-mmio uses >> it as the default vq size. >> 3. virtio-console driver allocates vring buffers based on this value (see >> add_inbuf() function of virtio_console.c file). Because PAGE_SIZE=64KB, >> ~64MB is allocated for each virtio-console vq. >> >> This patch addresses the problem by returning the iniatlized vring size >> when VM queries QEMU about VIRTIO_MMIO_QUEUE_NUM_MAX. This is similar to >> virtio-pci's approach. By doing this, the vq memory consumption is reduced >> substantially. > > I don't know if this patch is sensible to apply anyway, but from > this description this really sounds like a guest kernel bug. > QEMU tells the kernel the maximum queue size it can cope with, > and if the guest kernel cares about not using insane amounts of > RAM on queues then it should not blindly use the maximum size > but restrict it itself... Yes, this is another way of solving the problem. I think there are three alternatives: 1. Fix the return value of MMIO_QUEUE_NUM_MAX in QEMU (my patch); 2. In guest VM, virtio-mmio shouldn't query MMIO_QUEUE_NUM_MAX. Instead, it should behave similarly to virtio-pci driver which queries VIRTIO_PCI_QUEUE_NUM instead. However this approach requires modification of VIRTIO Specification as VIRTIO_PCI_QUEUE_NUM is WRITE-ONLY in virtio-mmio. 3. Fix virtio_console driver in guest VM. This driver currently takes in info->num and allocates memory based on its value. Apparently we can put a upper-limit on it. Your suggestion could fall in to (2) or (3). Any preference? Thanks, -Wei > > thanks > -- PMM >