From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33865) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bhdwn-0000o2-02 for qemu-devel@nongnu.org; Wed, 07 Sep 2016 10:35:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bhdwj-0002uV-PN for qemu-devel@nongnu.org; Wed, 07 Sep 2016 10:35:28 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49298) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bhdwj-0002uK-Ga for qemu-devel@nongnu.org; Wed, 07 Sep 2016 10:35:25 -0400 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AF02BC01BB27 for ; Wed, 7 Sep 2016 14:35:23 +0000 (UTC) Date: Wed, 7 Sep 2016 17:35:22 +0300 From: "Michael S. Tsirkin" Message-ID: <20160907172958-mutt-send-email-mst@kernel.org> References: <1473256389-1185-1-git-send-email-marcel@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1473256389-1185-1-git-send-email-marcel@redhat.com> Subject: Re: [Qemu-devel] [PATCH] virtio-pci: reduce modern_mem_bar size List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Marcel Apfelbaum Cc: qemu-devel@nongnu.org, kraxel@redhat.com On Wed, Sep 07, 2016 at 04:53:09PM +0300, Marcel Apfelbaum wrote: > Currently each VQ Notification Virtio Capability is allocated > on a different page. The idea is to enable split drivers within > guests, however there are no known plans to do that. > The allocation will result in a 8MB BAR, more than various > guest firmwares pre-allocates for PCI Bridges hotplug process. > > Reserve 4 bytes per VQ by default and add a new parameter > "page-per-vq" to be used with split drivers. > > Signed-off-by: Marcel Apfelbaum > --- > Hi, > > To be applied on top of: > [PATCH v5 1/2] pc: Add 2.8 machine (http://patchwork.ozlabs.org/patch/666812/) > > Thanks, > Marcel > > hw/virtio/virtio-pci.c | 18 +++++++++++------- > hw/virtio/virtio-pci.h | 6 ++++++ > include/hw/compat.h | 6 +++++- > 3 files changed, 22 insertions(+), 8 deletions(-) > > diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c > index 755f921..25e7ffa 100644 > --- a/hw/virtio/virtio-pci.c > +++ b/hw/virtio/virtio-pci.c > @@ -307,7 +307,7 @@ static int virtio_pci_ioeventfd_assign(DeviceState *d, EventNotifier *notifier, > MemoryRegion *modern_mr = &proxy->notify.mr; > MemoryRegion *modern_notify_mr = &proxy->notify_pio.mr; > MemoryRegion *legacy_mr = &proxy->bar; > - hwaddr modern_addr = QEMU_VIRTIO_PCI_QUEUE_MEM_MULT * > + hwaddr modern_addr = proxy->queue_mem_mult * > virtio_get_queue_index(vq); > hwaddr legacy_addr = VIRTIO_PCI_QUEUE_NOTIFY; > > @@ -1370,7 +1370,8 @@ static void virtio_pci_notify_write(void *opaque, hwaddr addr, > uint64_t val, unsigned size) > { > VirtIODevice *vdev = opaque; > - unsigned queue = addr / QEMU_VIRTIO_PCI_QUEUE_MEM_MULT; > + VirtIOPCIProxy *proxy = VIRTIO_PCI(DEVICE(vdev)->parent_bus->parent); > + unsigned queue = addr / proxy->queue_mem_mult; > > if (queue < VIRTIO_QUEUE_MAX) { > virtio_queue_notify(vdev, queue); > @@ -1609,7 +1610,7 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp) > struct virtio_pci_notify_cap notify = { > .cap.cap_len = sizeof notify, > .notify_off_multiplier = > - cpu_to_le32(QEMU_VIRTIO_PCI_QUEUE_MEM_MULT), > + cpu_to_le32(proxy->queue_mem_mult), > }; > struct virtio_pci_cfg_cap cfg = { > .cap.cap_len = sizeof cfg, > @@ -1717,6 +1718,9 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) > bool pcie_port = pci_bus_is_express(pci_dev->bus) && > !pci_bus_is_root(pci_dev->bus); > > + proxy->queue_mem_mult = (proxy->flags & VIRTIO_PCI_FLAG_PAGE_PER_VQ) ? > + QEMU_VIRTIO_PCI_QUEUE_MEM_MULT : 4; > + > /* > * virtio pci bar layout used by default. > * subclasses can re-arrange things if needed. > @@ -1744,8 +1748,7 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) > proxy->device.type = VIRTIO_PCI_CAP_DEVICE_CFG; > > proxy->notify.offset = 0x3000; > - proxy->notify.size = > - QEMU_VIRTIO_PCI_QUEUE_MEM_MULT * VIRTIO_QUEUE_MAX; > + proxy->notify.size = proxy->queue_mem_mult * VIRTIO_QUEUE_MAX; > proxy->notify.type = VIRTIO_PCI_CAP_NOTIFY_CFG; > > proxy->notify_pio.offset = 0x0; > @@ -1754,8 +1757,7 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) > > /* subclasses can enforce modern, so do this unconditionally */ > memory_region_init(&proxy->modern_bar, OBJECT(proxy), "virtio-pci", > - 2 * QEMU_VIRTIO_PCI_QUEUE_MEM_MULT * > - VIRTIO_QUEUE_MAX); Maybe add /* PCI BAR regions must be powers of 2 */ > + pow2ceil(proxy->notify.offset + proxy->notify.size)); > > memory_region_init_alias(&proxy->modern_cfg, > OBJECT(proxy), > @@ -1833,6 +1835,8 @@ static Property virtio_pci_properties[] = { > VIRTIO_PCI_FLAG_MODERN_PIO_NOTIFY_BIT, false), > DEFINE_PROP_BIT("x-disable-pcie", VirtIOPCIProxy, flags, > VIRTIO_PCI_FLAG_DISABLE_PCIE_BIT, false), > + DEFINE_PROP_BIT("page-per-vq", VirtIOPCIProxy, flags, > + VIRTIO_PCI_FLAG_PAGE_PER_VQ_BIT, false), > DEFINE_PROP_END_OF_LIST(), > }; > > diff --git a/hw/virtio/virtio-pci.h b/hw/virtio/virtio-pci.h > index 25fbf8a..0fd0d37 100644 > --- a/hw/virtio/virtio-pci.h > +++ b/hw/virtio/virtio-pci.h > @@ -64,6 +64,7 @@ enum { > VIRTIO_PCI_FLAG_MIGRATE_EXTRA_BIT, > VIRTIO_PCI_FLAG_MODERN_PIO_NOTIFY_BIT, > VIRTIO_PCI_FLAG_DISABLE_PCIE_BIT, > + VIRTIO_PCI_FLAG_PAGE_PER_VQ_BIT, > }; > > /* Need to activate work-arounds for buggy guests at vmstate load. */ > @@ -84,6 +85,10 @@ enum { > #define VIRTIO_PCI_FLAG_MODERN_PIO_NOTIFY \ > (1 << VIRTIO_PCI_FLAG_MODERN_PIO_NOTIFY_BIT) > > +/* page per vq flag to be used by split drivers within guests */ > +#define VIRTIO_PCI_FLAG_PAGE_PER_VQ \ > + (1 << VIRTIO_PCI_FLAG_PAGE_PER_VQ_BIT) > + > typedef struct { > MSIMessage msg; > int virq; > @@ -138,6 +143,7 @@ struct VirtIOPCIProxy { > uint32_t msix_bar; > uint32_t modern_io_bar; > uint32_t modern_mem_bar; > + int queue_mem_mult; > int config_cap; > uint32_t flags; > bool disable_modern; I would rather use wrapper functions to calculate it, to avoid duplicating data. E.g. static inline int virtio_pci_queue_mem_mult(struct VirtIOPCIProxy *) { return (proxy->flags & VIRTIO_PCI_FLAG_PAGE_PER_VQ) ? QEMU_VIRTIO_PCI_QUEUE_MEM_MULT : 4; } Otherwise, looks fine. > diff --git a/include/hw/compat.h b/include/hw/compat.h > index 08dd4fb..a1d6694 100644 > --- a/include/hw/compat.h > +++ b/include/hw/compat.h > @@ -2,7 +2,11 @@ > #define HW_COMPAT_H > > #define HW_COMPAT_2_7 \ > - /* empty */ > + {\ > + .driver = "virtio-pci",\ > + .property = "page-per-vq",\ > + .value = "on",\ > + }, > > #define HW_COMPAT_2_6 \ > {\ > -- > 2.5.5