[Qemu-devel] [PULL 00/15] virtio,pc features, fixes

[Qemu-devel] [PULL 00/15] virtio,pc features, fixes

[Michael S. Tsirkin]

I expected to also merge memory allocation refactoring
and vhost user migration support, but that appears to
need more work.

The following changes since commit fa500928ad9da6dd570918e3dfca13c029af07a8:

  Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20150930' into staging (2015-10-01 10:49:38 +0100)

are available in the git repository at:

  git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git tags/for_upstream

for you to fetch changes up to a9faaa0aa97509cc14adb9f7ce2f3bfcd88066c4:

  vhost-user-test: fix predictable filename on tmpfs (2015-10-02 16:40:58 +0300)

----------------------------------------------------------------
virtio,pc features, fixes

New features:
    guest RAM buffer overrun mitigation
    RAM physical address gaps for memory hotplug
    (except refactoring which got some review comments)
    virtio-9f migration (when unmounted)

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

----------------------------------------------------------------
Eduardo Habkost (1):
      pc: Add a comment explaining why pc_compat_2_4() doesn't exist

Greg Kurz (1):
      virtio-9p: migrate virtio subsections

Igor Mammedov (2):
      memhp: extend address auto assignment to support gaps
      pc: memhp: force gaps between DIMM's GPA

Jason Wang (3):
      virtio: introduce virtqueue_unmap_sg()
      virtio: introduce virtqueue_discard()
      virtio-net: correctly drop truncated packets

Michael S. Tsirkin (6):
      oslib: rework anonimous RAM allocation
      oslib: allocate PROT_NONE pages on top of RAM
      exec: allocate PROT_NONE pages on top of RAM
      vhost-user: unit test for new messages
      vhost-user-test: use tmpfs by default
      vhost-user-test: fix predictable filename on tmpfs

Paolo Bonzini (1):
      vhost-user-test: do not reinvent glib-compat.h

Richard Henderson (1):
      virtio: Notice when the system doesn't support MSIx at all

 include/hw/i386/pc.h       |   1 +
 include/hw/mem/pc-dimm.h   |   7 +-
 include/hw/virtio/virtio.h |   2 +
 exec.c                     |  42 ++++++++++-
 hw/9pfs/virtio-9p-device.c |  11 +++
 hw/i386/pc.c               |   5 +-
 hw/i386/pc_piix.c          |   8 +++
 hw/i386/pc_q35.c           |   8 +++
 hw/mem/pc-dimm.c           |  15 ++--
 hw/net/virtio-net.c        |   8 +--
 hw/ppc/spapr.c             |   2 +-
 hw/virtio/virtio-pci.c     |  17 +++--
 hw/virtio/virtio.c         |  21 ++++--
 tests/vhost-user-test.c    | 170 ++++++++++++++++-----------------------------
 util/oslib-posix.c         |  20 ++++--
 15 files changed, 190 insertions(+), 147 deletions(-)

[Qemu-devel] [PULL 01/15] virtio: introduce virtqueue_unmap_sg()

[Michael S. Tsirkin]

From: Jason Wang <jasowang@redhat.com>

Factor out sg unmapping logic. This will be reused by the patch that
can discard descriptor.

Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Andrew James <andrew.james@hpe.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/virtio.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 7504f8b..6f2b96c 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -244,14 +244,12 @@ int virtio_queue_empty(VirtQueue *vq)
     return vring_avail_idx(vq) == vq->last_avail_idx;
 }
 
-void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem,
-                    unsigned int len, unsigned int idx)
+static void virtqueue_unmap_sg(VirtQueue *vq, const VirtQueueElement *elem,
+                               unsigned int len)
 {
     unsigned int offset;
     int i;
 
-    trace_virtqueue_fill(vq, elem, len, idx);
-
     offset = 0;
     for (i = 0; i < elem->in_num; i++) {
         size_t size = MIN(len - offset, elem->in_sg[i].iov_len);
@@ -267,6 +265,14 @@ void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem,
         cpu_physical_memory_unmap(elem->out_sg[i].iov_base,
                                   elem->out_sg[i].iov_len,
                                   0, elem->out_sg[i].iov_len);
+}
+
+void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem,
+                    unsigned int len, unsigned int idx)
+{
+    trace_virtqueue_fill(vq, elem, len, idx);
+
+    virtqueue_unmap_sg(vq, elem, len);
 
     idx = (idx + vring_used_idx(vq)) % vq->vring.num;
 
-- 
MST

[Qemu-devel] [PULL 02/15] virtio: introduce virtqueue_discard()

[Michael S. Tsirkin]

From: Jason Wang <jasowang@redhat.com>

This patch introduces virtqueue_discard() to discard a descriptor and
unmap the sgs. This will be used by the patch that will discard
descriptor when packet is truncated.

Cc: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/virtio/virtio.h | 2 ++
 hw/virtio/virtio.c         | 7 +++++++
 2 files changed, 9 insertions(+)

diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 6201ee8..9d09115 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -146,6 +146,8 @@ void virtio_del_queue(VirtIODevice *vdev, int n);
 void virtqueue_push(VirtQueue *vq, const VirtQueueElement *elem,
                     unsigned int len);
 void virtqueue_flush(VirtQueue *vq, unsigned int count);
+void virtqueue_discard(VirtQueue *vq, const VirtQueueElement *elem,
+                       unsigned int len);
 void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem,
                     unsigned int len, unsigned int idx);
 
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 6f2b96c..d0bc72e 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -267,6 +267,13 @@ static void virtqueue_unmap_sg(VirtQueue *vq, const VirtQueueElement *elem,
                                   0, elem->out_sg[i].iov_len);
 }
 
+void virtqueue_discard(VirtQueue *vq, const VirtQueueElement *elem,
+                       unsigned int len)
+{
+    vq->last_avail_idx--;
+    virtqueue_unmap_sg(vq, elem, len);
+}
+
 void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem,
                     unsigned int len, unsigned int idx)
 {
-- 
MST

[Qemu-devel] [PULL 03/15] virtio-net: correctly drop truncated packets

[Michael S. Tsirkin]

From: Jason Wang <jasowang@redhat.com>

When packet is truncated during receiving, we drop the packets but
neither discard the descriptor nor add and signal used
descriptor. This will lead several issues:

- sg mappings are leaked
- rx will be stalled if a lots of packets were truncated

In order to be consistent with vhost, fix by discarding the descriptor
in this case.

Cc: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/net/virtio-net.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index d388c55..a877614 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -1094,13 +1094,7 @@ static ssize_t virtio_net_receive(NetClientState *nc, const uint8_t *buf, size_t
          * must have consumed the complete packet.
          * Otherwise, drop it. */
         if (!n->mergeable_rx_bufs && offset < size) {
-#if 0
-            error_report("virtio-net truncated non-mergeable packet: "
-                         "i %zd mergeable %d offset %zd, size %zd, "
-                         "guest hdr len %zd, host hdr len %zd",
-                         i, n->mergeable_rx_bufs,
-                         offset, size, n->guest_hdr_len, n->host_hdr_len);
-#endif
+            virtqueue_discard(q->rx_vq, &elem, total);
             return size;
         }
 
-- 
MST

[Qemu-devel] [PULL 04/15] oslib: rework anonimous RAM allocation

[Michael S. Tsirkin]

At the moment we first allocate RAM, sometimes more than necessary for
alignment reasons.  We then free the extra RAM.

Rework this to avoid the temporary allocation: reserve the
range by mapping it with PROT_NONE, then use just the
necessary range with MAP_FIXED.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
---
 util/oslib-posix.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index 3ae4987..27972d4 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -129,9 +129,9 @@ void *qemu_anon_ram_alloc(size_t size, uint64_t *alignment)
 {
     size_t align = QEMU_VMALLOC_ALIGN;
     size_t total = size + align - getpagesize();
-    void *ptr = mmap(0, total, PROT_READ | PROT_WRITE,
-                     MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+    void *ptr = mmap(0, total, PROT_NONE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
     size_t offset = QEMU_ALIGN_UP((uintptr_t)ptr, align) - (uintptr_t)ptr;
+    void *ptr1;
 
     if (ptr == MAP_FAILED) {
         return NULL;
@@ -140,6 +140,14 @@ void *qemu_anon_ram_alloc(size_t size, uint64_t *alignment)
     if (alignment) {
         *alignment = align;
     }
+
+    ptr1 = mmap(ptr + offset, size, PROT_READ | PROT_WRITE,
+                MAP_FIXED | MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+    if (ptr1 == MAP_FAILED) {
+        munmap(ptr, total);
+        return NULL;
+    }
+
     ptr += offset;
     total -= offset;
 
-- 
MST

Re: [Qemu-devel] [PULL 04/15] oslib: rework anonimous RAM allocation

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 722 bytes --]

On 10/02/2015 07:45 AM, Michael S. Tsirkin wrote:

If it's not too late, s/anonimous/anonymous/ in the subject line

> At the moment we first allocate RAM, sometimes more than necessary for
> alignment reasons.  We then free the extra RAM.
> 
> Rework this to avoid the temporary allocation: reserve the
> range by mapping it with PROT_NONE, then use just the
> necessary range with MAP_FIXED.
> 
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
> Acked-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  util/oslib-posix.c | 12 ++++++++++--

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

[Qemu-devel] [PULL 05/15] oslib: allocate PROT_NONE pages on top of RAM

[Michael S. Tsirkin]

This inserts a read and write protected page between RAM and QEMU
memory. This makes it harder to exploit QEMU bugs resulting from buffer
overflows in devices using variants of cpu_physical_memory_map,
dma_memory_map etc.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
---
 util/oslib-posix.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index 27972d4..a0fcdc2 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -128,7 +128,7 @@ void *qemu_memalign(size_t alignment, size_t size)
 void *qemu_anon_ram_alloc(size_t size, uint64_t *alignment)
 {
     size_t align = QEMU_VMALLOC_ALIGN;
-    size_t total = size + align - getpagesize();
+    size_t total = size + align;
     void *ptr = mmap(0, total, PROT_NONE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
     size_t offset = QEMU_ALIGN_UP((uintptr_t)ptr, align) - (uintptr_t)ptr;
     void *ptr1;
@@ -154,8 +154,8 @@ void *qemu_anon_ram_alloc(size_t size, uint64_t *alignment)
     if (offset > 0) {
         munmap(ptr - offset, offset);
     }
-    if (total > size) {
-        munmap(ptr + size, total - size);
+    if (total > size + getpagesize()) {
+        munmap(ptr + size + getpagesize(), total - size - getpagesize());
     }
 
     trace_qemu_anon_ram_alloc(size, ptr);
@@ -172,7 +172,7 @@ void qemu_anon_ram_free(void *ptr, size_t size)
 {
     trace_qemu_anon_ram_free(ptr, size);
     if (ptr) {
-        munmap(ptr, size);
+        munmap(ptr, size + getpagesize());
     }
 }
 
-- 
MST

[Qemu-devel] [PULL 06/15] exec: allocate PROT_NONE pages on top of RAM

[Michael S. Tsirkin]

This inserts a read and write protected page between RAM and QEMU
memory, for file-backend RAM.
This makes it harder to exploit QEMU bugs resulting from buffer
overflows in devices using variants of cpu_physical_memory_map,
dma_memory_map etc.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
---
 exec.c | 42 +++++++++++++++++++++++++++++++++++++++---
 1 file changed, 39 insertions(+), 3 deletions(-)

diff --git a/exec.c b/exec.c
index 47ada31..7d90a52 100644
--- a/exec.c
+++ b/exec.c
@@ -84,6 +84,9 @@ static MemoryRegion io_mem_unassigned;
  */
 #define RAM_RESIZEABLE (1 << 2)
 
+/* An extra page is mapped on top of this RAM.
+ */
+#define RAM_EXTRA (1 << 3)
 #endif
 
 struct CPUTailQ cpus = QTAILQ_HEAD_INITIALIZER(cpus);
@@ -1185,10 +1188,13 @@ static void *file_ram_alloc(RAMBlock *block,
     char *filename;
     char *sanitized_name;
     char *c;
+    void *ptr;
     void *area = NULL;
     int fd;
     uint64_t hpagesize;
+    uint64_t total;
     Error *local_err = NULL;
+    size_t offset;
 
     hpagesize = gethugepagesize(path, &local_err);
     if (local_err) {
@@ -1232,6 +1238,7 @@ static void *file_ram_alloc(RAMBlock *block,
     g_free(filename);
 
     memory = ROUND_UP(memory, hpagesize);
+    total = memory + hpagesize;
 
     /*
      * ftruncate is not supported by hugetlbfs in older
@@ -1243,16 +1250,40 @@ static void *file_ram_alloc(RAMBlock *block,
         perror("ftruncate");
     }
 
-    area = mmap(0, memory, PROT_READ | PROT_WRITE,
-                (block->flags & RAM_SHARED ? MAP_SHARED : MAP_PRIVATE),
+    ptr = mmap(0, total, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS,
+                -1, 0);
+    if (ptr == MAP_FAILED) {
+        error_setg_errno(errp, errno,
+                         "unable to allocate memory range for hugepages");
+        close(fd);
+        goto error;
+    }
+
+    offset = QEMU_ALIGN_UP((uintptr_t)ptr, hpagesize) - (uintptr_t)ptr;
+
+    area = mmap(ptr + offset, memory, PROT_READ | PROT_WRITE,
+                (block->flags & RAM_SHARED ? MAP_SHARED : MAP_PRIVATE) |
+                MAP_FIXED,
                 fd, 0);
     if (area == MAP_FAILED) {
         error_setg_errno(errp, errno,
                          "unable to map backing store for hugepages");
+        munmap(ptr, total);
         close(fd);
         goto error;
     }
 
+    if (offset > 0) {
+        munmap(ptr, offset);
+    }
+    ptr += offset;
+    total -= offset;
+
+    if (total > memory + getpagesize()) {
+        munmap(ptr + memory + getpagesize(),
+               total - memory - getpagesize());
+    }
+
     if (mem_prealloc) {
         os_mem_prealloc(fd, area, memory);
     }
@@ -1570,6 +1601,7 @@ ram_addr_t qemu_ram_alloc_from_file(ram_addr_t size, MemoryRegion *mr,
     new_block->used_length = size;
     new_block->max_length = size;
     new_block->flags = share ? RAM_SHARED : 0;
+    new_block->flags |= RAM_EXTRA;
     new_block->host = file_ram_alloc(new_block, size,
                                      mem_path, errp);
     if (!new_block->host) {
@@ -1671,7 +1703,11 @@ static void reclaim_ramblock(RAMBlock *block)
         xen_invalidate_map_cache_entry(block->host);
 #ifndef _WIN32
     } else if (block->fd >= 0) {
-        munmap(block->host, block->max_length);
+        if (block->flags & RAM_EXTRA) {
+            munmap(block->host, block->max_length + getpagesize());
+        } else {
+            munmap(block->host, block->max_length);
+        }
         close(block->fd);
 #endif
     } else {
-- 
MST

[Qemu-devel] [PULL 07/15] pc: Add a comment explaining why pc_compat_2_4() doesn't exist

[Michael S. Tsirkin]

From: Eduardo Habkost <ehabkost@redhat.com>

pc_compat_2_4() doesn't exist, and we shouldn't create one. Add a
comment explaining why the function doesn't exist and why pc_compat_*()
functions are deprecated.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/pc_piix.c | 7 +++++++
 hw/i386/pc_q35.c  | 7 +++++++
 2 files changed, 14 insertions(+)

diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index 3ffb05f..52fdd55 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -301,6 +301,13 @@ static void pc_init1(MachineState *machine,
     }
 }
 
+/* Looking for a pc_compat_2_4() function? It doesn't exist.
+ * pc_compat_*() functions that run on machine-init time and
+ * change global QEMU state are deprecated. Please don't create
+ * one, and implement any pc-*-2.4 (and newer) compat code in
+ * HW_COMPAT_*, PC_COMPAT_*, or * pc_*_machine_options().
+ */
+
 static void pc_compat_2_3(MachineState *machine)
 {
     PCMachineState *pcms = PC_MACHINE(machine);
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 1b7d3b6..837e430 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -284,6 +284,13 @@ static void pc_q35_init(MachineState *machine)
     }
 }
 
+/* Looking for a pc_compat_2_4() function? It doesn't exist.
+ * pc_compat_*() functions that run on machine-init time and
+ * change global QEMU state are deprecated. Please don't create
+ * one, and implement any pc-*-2.4 (and newer) compat code in
+ * HW_COMPAT_*, PC_COMPAT_*, or * pc_*_machine_options().
+ */
+
 static void pc_compat_2_3(MachineState *machine)
 {
     PCMachineState *pcms = PC_MACHINE(machine);
-- 
MST

[Qemu-devel] [PULL 08/15] virtio: Notice when the system doesn't support MSIx at all

[Michael S. Tsirkin]

From: Richard Henderson <rth@twiddle.net>

And do not issue an error_report in that case.

Signed-off-by: Richard Henderson <rth@twiddle.net>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/virtio-pci.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index eda8205..6703806 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -1491,12 +1491,17 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp)
         pci_set_long(cfg_mask->pci_cfg_data, ~0x0);
     }
 
-    if (proxy->nvectors &&
-        msix_init_exclusive_bar(&proxy->pci_dev, proxy->nvectors,
-                                proxy->msix_bar)) {
-        error_report("unable to init msix vectors to %" PRIu32,
-                     proxy->nvectors);
-        proxy->nvectors = 0;
+    if (proxy->nvectors) {
+        int err = msix_init_exclusive_bar(&proxy->pci_dev, proxy->nvectors,
+                                          proxy->msix_bar);
+        if (err) {
+            /* Notice when a system that supports MSIx can't initialize it.  */
+            if (err != -ENOTSUP) {
+                error_report("unable to init msix vectors to %" PRIu32,
+                             proxy->nvectors);
+            }
+            proxy->nvectors = 0;
+        }
     }
 
     proxy->pci_dev.config_write = virtio_write_config;
-- 
MST

[Qemu-devel] [PULL 09/15] virtio-9p: migrate virtio subsections

[Michael S. Tsirkin]

From: Greg Kurz <gkurz@linux.vnet.ibm.com>

In a cross-endian setup, the virtio-9p device has state in @device_endian. It
must be migrated. This patch just adds the minimal support to live migrate
generic virtio subsections where @device_endian is handled.

Please note that this is unrelated to the fact that we block migration when
the 9p share is mounted in the guest. It fixes the case where we want to
migrate an unactive 9p device (not mounted in the guest) to a QEMU with
different endianness: the migration currently succeeds but leaves the device
in an inconsistent state that causes mount to hang until we reboot the guest.

Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/9pfs/virtio-9p-device.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/hw/9pfs/virtio-9p-device.c b/hw/9pfs/virtio-9p-device.c
index 93a407c..e3abcfa 100644
--- a/hw/9pfs/virtio-9p-device.c
+++ b/hw/9pfs/virtio-9p-device.c
@@ -43,6 +43,16 @@ static void virtio_9p_get_config(VirtIODevice *vdev, uint8_t *config)
     g_free(cfg);
 }
 
+static void virtio_9p_save(QEMUFile *f, void *opaque)
+{
+    virtio_save(VIRTIO_DEVICE(opaque), f);
+}
+
+static int virtio_9p_load(QEMUFile *f, void *opaque, int version_id)
+{
+    return virtio_load(VIRTIO_DEVICE(opaque), f, version_id);
+}
+
 static void virtio_9p_device_realize(DeviceState *dev, Error **errp)
 {
     VirtIODevice *vdev = VIRTIO_DEVICE(dev);
@@ -130,6 +140,7 @@ static void virtio_9p_device_realize(DeviceState *dev, Error **errp)
     }
     v9fs_path_free(&path);
 
+    register_savevm(dev, "virtio-9p", -1, 1, virtio_9p_save, virtio_9p_load, s);
     return;
 out:
     g_free(s->ctx.fs_root);
-- 
MST

Re: [Qemu-devel] [PULL 09/15] virtio-9p: migrate virtio subsections

[Michael S. Tsirkin]

On Fri, Oct 02, 2015 at 04:45:36PM +0300, Michael S. Tsirkin wrote:
> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> 
> In a cross-endian setup, the virtio-9p device has state in @device_endian. It
> must be migrated. This patch just adds the minimal support to live migrate
> generic virtio subsections where @device_endian is handled.
> 
> Please note that this is unrelated to the fact that we block migration when
> the 9p share is mounted in the guest. It fixes the case where we want to
> migrate an unactive 9p device (not mounted in the guest) to a QEMU with
> different endianness: the migration currently succeeds but leaves the device
> in an inconsistent state that causes mount to hang until we reboot the guest.
> 
> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

I rebased and dropped this at Greg's request.

> ---
>  hw/9pfs/virtio-9p-device.c | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/hw/9pfs/virtio-9p-device.c b/hw/9pfs/virtio-9p-device.c
> index 93a407c..e3abcfa 100644
> --- a/hw/9pfs/virtio-9p-device.c
> +++ b/hw/9pfs/virtio-9p-device.c
> @@ -43,6 +43,16 @@ static void virtio_9p_get_config(VirtIODevice *vdev, uint8_t *config)
>      g_free(cfg);
>  }
>  
> +static void virtio_9p_save(QEMUFile *f, void *opaque)
> +{
> +    virtio_save(VIRTIO_DEVICE(opaque), f);
> +}
> +
> +static int virtio_9p_load(QEMUFile *f, void *opaque, int version_id)
> +{
> +    return virtio_load(VIRTIO_DEVICE(opaque), f, version_id);
> +}
> +
>  static void virtio_9p_device_realize(DeviceState *dev, Error **errp)
>  {
>      VirtIODevice *vdev = VIRTIO_DEVICE(dev);
> @@ -130,6 +140,7 @@ static void virtio_9p_device_realize(DeviceState *dev, Error **errp)
>      }
>      v9fs_path_free(&path);
>  
> +    register_savevm(dev, "virtio-9p", -1, 1, virtio_9p_save, virtio_9p_load, s);
>      return;
>  out:
>      g_free(s->ctx.fs_root);
> -- 
> MST
> 

[Qemu-devel] [PULL 10/15] vhost-user-test: do not reinvent glib-compat.h

[Michael S. Tsirkin]

From: Paolo Bonzini <pbonzini@redhat.com>

glib-compat.h has the gunk to support both old-style and new-style
gthread functions.  Use it instead of reinventing it.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Tested-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 tests/vhost-user-test.c | 113 +++++++-----------------------------------------
 1 file changed, 16 insertions(+), 97 deletions(-)

diff --git a/tests/vhost-user-test.c b/tests/vhost-user-test.c
index e301db7..0e04f06 100644
--- a/tests/vhost-user-test.c
+++ b/tests/vhost-user-test.c
@@ -8,7 +8,6 @@
  *
  */
 
-#define QEMU_GLIB_COMPAT_H
 #include <glib.h>
 
 #include "libqtest.h"
@@ -30,12 +29,6 @@
 #define HAVE_MONOTONIC_TIME
 #endif
 
-#if GLIB_CHECK_VERSION(2, 32, 0)
-#define HAVE_MUTEX_INIT
-#define HAVE_COND_INIT
-#define HAVE_THREAD_NEW
-#endif
-
 #define QEMU_CMD_ACCEL  " -machine accel=tcg"
 #define QEMU_CMD_MEM    " -m 512 -object memory-backend-file,id=mem,size=512M,"\
                         "mem-path=%s,share=on -numa node,memdev=mem"
@@ -113,93 +106,21 @@ static VhostUserMsg m __attribute__ ((unused));
 
 int fds_num = 0, fds[VHOST_MEMORY_MAX_NREGIONS];
 static VhostUserMemory memory;
-static GMutex *data_mutex;
-static GCond *data_cond;
-
-static gint64 _get_time(void)
-{
-#ifdef HAVE_MONOTONIC_TIME
-    return g_get_monotonic_time();
-#else
-    GTimeVal time;
-    g_get_current_time(&time);
-
-    return time.tv_sec * G_TIME_SPAN_SECOND + time.tv_usec;
-#endif
-}
-
-static GMutex *_mutex_new(void)
-{
-    GMutex *mutex;
-
-#ifdef HAVE_MUTEX_INIT
-    mutex = g_new(GMutex, 1);
-    g_mutex_init(mutex);
-#else
-    mutex = g_mutex_new();
-#endif
-
-    return mutex;
-}
+static CompatGMutex data_mutex;
+static CompatGCond data_cond;
 
-static void _mutex_free(GMutex *mutex)
-{
-#ifdef HAVE_MUTEX_INIT
-    g_mutex_clear(mutex);
-    g_free(mutex);
-#else
-    g_mutex_free(mutex);
-#endif
-}
-
-static GCond *_cond_new(void)
-{
-    GCond *cond;
-
-#ifdef HAVE_COND_INIT
-    cond = g_new(GCond, 1);
-    g_cond_init(cond);
-#else
-    cond = g_cond_new();
-#endif
-
-    return cond;
-}
-
-static gboolean _cond_wait_until(GCond *cond, GMutex *mutex, gint64 end_time)
+#if !GLIB_CHECK_VERSION(2, 32, 0)
+static gboolean g_cond_wait_until(CompatGCond cond, CompatGMutex mutex,
+                                  gint64 end_time)
 {
     gboolean ret = FALSE;
-#ifdef HAVE_COND_INIT
-    ret = g_cond_wait_until(cond, mutex, end_time);
-#else
+    end_time -= g_get_monotonic_time();
     GTimeVal time = { end_time / G_TIME_SPAN_SECOND,
                       end_time % G_TIME_SPAN_SECOND };
     ret = g_cond_timed_wait(cond, mutex, &time);
-#endif
     return ret;
 }
-
-static void _cond_free(GCond *cond)
-{
-#ifdef HAVE_COND_INIT
-    g_cond_clear(cond);
-    g_free(cond);
-#else
-    g_cond_free(cond);
 #endif
-}
-
-static GThread *_thread_new(const gchar *name, GThreadFunc func, gpointer data)
-{
-    GThread *thread = NULL;
-    GError *error = NULL;
-#ifdef HAVE_THREAD_NEW
-    thread = g_thread_try_new(name, func, data, &error);
-#else
-    thread = g_thread_create(func, data, TRUE, &error);
-#endif
-    return thread;
-}
 
 static void read_guest_mem(void)
 {
@@ -208,11 +129,11 @@ static void read_guest_mem(void)
     int i, j;
     size_t size;
 
-    g_mutex_lock(data_mutex);
+    g_mutex_lock(&data_mutex);
 
-    end_time = _get_time() + 5 * G_TIME_SPAN_SECOND;
+    end_time = g_get_monotonic_time() + 5 * G_TIME_SPAN_SECOND;
     while (!fds_num) {
-        if (!_cond_wait_until(data_cond, data_mutex, end_time)) {
+        if (!g_cond_wait_until(&data_cond, &data_mutex, end_time)) {
             /* timeout has passed */
             g_assert(fds_num);
             break;
@@ -252,7 +173,7 @@ static void read_guest_mem(void)
     }
 
     g_assert_cmpint(1, ==, 1);
-    g_mutex_unlock(data_mutex);
+    g_mutex_unlock(&data_mutex);
 }
 
 static void *thread_function(void *data)
@@ -280,7 +201,7 @@ static void chr_read(void *opaque, const uint8_t *buf, int size)
         return;
     }
 
-    g_mutex_lock(data_mutex);
+    g_mutex_lock(&data_mutex);
     memcpy(p, buf, VHOST_USER_HDR_SIZE);
 
     if (msg.size) {
@@ -313,7 +234,7 @@ static void chr_read(void *opaque, const uint8_t *buf, int size)
         fds_num = qemu_chr_fe_get_msgfds(chr, fds, sizeof(fds) / sizeof(int));
 
         /* signal the test that it can continue */
-        g_cond_signal(data_cond);
+        g_cond_signal(&data_cond);
         break;
 
     case VHOST_USER_SET_VRING_KICK:
@@ -330,7 +251,7 @@ static void chr_read(void *opaque, const uint8_t *buf, int size)
     default:
         break;
     }
-    g_mutex_unlock(data_mutex);
+    g_mutex_unlock(&data_mutex);
 }
 
 static const char *init_hugepagefs(void)
@@ -395,9 +316,9 @@ int main(int argc, char **argv)
     qemu_chr_add_handlers(chr, chr_can_read, chr_read, NULL, chr);
 
     /* run the main loop thread so the chardev may operate */
-    data_mutex = _mutex_new();
-    data_cond = _cond_new();
-    _thread_new(NULL, thread_function, NULL);
+    g_mutex_init(&data_mutex);
+    g_cond_init(&data_cond);
+    g_thread_new(NULL, thread_function, NULL);
 
     qemu_cmd = g_strdup_printf(QEMU_CMD, hugefs, socket_path);
     s = qtest_start(qemu_cmd);
@@ -414,8 +335,6 @@ int main(int argc, char **argv)
     /* cleanup */
     unlink(socket_path);
     g_free(socket_path);
-    _cond_free(data_cond);
-    _mutex_free(data_mutex);
 
     return ret;
 }
-- 
MST

[Qemu-devel] [PULL 11/15] vhost-user: unit test for new messages

[Michael S. Tsirkin]

Data is empty for now, but do make sure master
sets the new feature bit flag.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/vhost-user-test.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/tests/vhost-user-test.c b/tests/vhost-user-test.c
index 0e04f06..87281b9 100644
--- a/tests/vhost-user-test.c
+++ b/tests/vhost-user-test.c
@@ -46,6 +46,8 @@
 
 #define VHOST_MEMORY_MAX_NREGIONS    8
 
+#define VHOST_USER_F_PROTOCOL_FEATURES 30
+
 typedef enum VhostUserRequest {
     VHOST_USER_NONE = 0,
     VHOST_USER_GET_FEATURES = 1,
@@ -62,6 +64,8 @@ typedef enum VhostUserRequest {
     VHOST_USER_SET_VRING_KICK = 12,
     VHOST_USER_SET_VRING_CALL = 13,
     VHOST_USER_SET_VRING_ERR = 14,
+    VHOST_USER_GET_PROTOCOL_FEATURES = 15,
+    VHOST_USER_SET_PROTOCOL_FEATURES = 16,
     VHOST_USER_MAX
 } VhostUserRequest;
 
@@ -214,6 +218,20 @@ static void chr_read(void *opaque, const uint8_t *buf, int size)
         /* send back features to qemu */
         msg.flags |= VHOST_USER_REPLY_MASK;
         msg.size = sizeof(m.u64);
+        msg.u64 = 0x1ULL << VHOST_USER_F_PROTOCOL_FEATURES;
+        p = (uint8_t *) &msg;
+        qemu_chr_fe_write_all(chr, p, VHOST_USER_HDR_SIZE + msg.size);
+        break;
+
+    case VHOST_USER_SET_FEATURES:
+	g_assert_cmpint(msg.u64 & (0x1ULL << VHOST_USER_F_PROTOCOL_FEATURES),
+			!=, 0ULL);
+        break;
+
+    case VHOST_USER_GET_PROTOCOL_FEATURES:
+        /* send back features to qemu */
+        msg.flags |= VHOST_USER_REPLY_MASK;
+        msg.size = sizeof(m.u64);
         msg.u64 = 0;
         p = (uint8_t *) &msg;
         qemu_chr_fe_write_all(chr, p, VHOST_USER_HDR_SIZE + msg.size);
-- 
MST

[Qemu-devel] [PULL 12/15] memhp: extend address auto assignment to support gaps

[Michael S. Tsirkin]

From: Igor Mammedov <imammedo@redhat.com>

setting gap to TRUE will make sparse DIMM
address auto allocation, leaving gaps between
a new DIMM address and preceeding existing DIMM.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/mem/pc-dimm.h |  7 ++++---
 hw/i386/pc.c             |  3 ++-
 hw/mem/pc-dimm.c         | 15 +++++++++------
 hw/ppc/spapr.c           |  2 +-
 4 files changed, 16 insertions(+), 11 deletions(-)

diff --git a/include/hw/mem/pc-dimm.h b/include/hw/mem/pc-dimm.h
index d83bf30..c1ee7b0 100644
--- a/include/hw/mem/pc-dimm.h
+++ b/include/hw/mem/pc-dimm.h
@@ -83,15 +83,16 @@ typedef struct MemoryHotplugState {
 
 uint64_t pc_dimm_get_free_addr(uint64_t address_space_start,
                                uint64_t address_space_size,
-                               uint64_t *hint, uint64_t align, uint64_t size,
-                               Error **errp);
+                               uint64_t *hint, uint64_t align, bool gap,
+                               uint64_t size, Error **errp);
 
 int pc_dimm_get_free_slot(const int *hint, int max_slots, Error **errp);
 
 int qmp_pc_dimm_device_list(Object *obj, void *opaque);
 uint64_t pc_existing_dimms_capacity(Error **errp);
 void pc_dimm_memory_plug(DeviceState *dev, MemoryHotplugState *hpms,
-                         MemoryRegion *mr, uint64_t align, Error **errp);
+                         MemoryRegion *mr, uint64_t align, bool gap,
+                         Error **errp);
 void pc_dimm_memory_unplug(DeviceState *dev, MemoryHotplugState *hpms,
                            MemoryRegion *mr);
 #endif
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 461c128..ef02736 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1644,7 +1644,8 @@ static void pc_dimm_plug(HotplugHandler *hotplug_dev,
         goto out;
     }
 
-    pc_dimm_memory_plug(dev, &pcms->hotplug_memory, mr, align, &local_err);
+    pc_dimm_memory_plug(dev, &pcms->hotplug_memory, mr, align, false,
+                        &local_err);
     if (local_err) {
         goto out;
     }
diff --git a/hw/mem/pc-dimm.c b/hw/mem/pc-dimm.c
index bb04862..6cc6ac3 100644
--- a/hw/mem/pc-dimm.c
+++ b/hw/mem/pc-dimm.c
@@ -32,7 +32,8 @@ typedef struct pc_dimms_capacity {
 } pc_dimms_capacity;
 
 void pc_dimm_memory_plug(DeviceState *dev, MemoryHotplugState *hpms,
-                         MemoryRegion *mr, uint64_t align, Error **errp)
+                         MemoryRegion *mr, uint64_t align, bool gap,
+                         Error **errp)
 {
     int slot;
     MachineState *machine = MACHINE(qdev_get_machine());
@@ -48,7 +49,7 @@ void pc_dimm_memory_plug(DeviceState *dev, MemoryHotplugState *hpms,
 
     addr = pc_dimm_get_free_addr(hpms->base,
                                  memory_region_size(&hpms->mr),
-                                 !addr ? NULL : &addr, align,
+                                 !addr ? NULL : &addr, align, gap,
                                  memory_region_size(mr), &local_err);
     if (local_err) {
         goto out;
@@ -287,8 +288,8 @@ static int pc_dimm_built_list(Object *obj, void *opaque)
 
 uint64_t pc_dimm_get_free_addr(uint64_t address_space_start,
                                uint64_t address_space_size,
-                               uint64_t *hint, uint64_t align, uint64_t size,
-                               Error **errp)
+                               uint64_t *hint, uint64_t align, bool gap,
+                               uint64_t size, Error **errp)
 {
     GSList *list = NULL, *item;
     uint64_t new_addr, ret = 0;
@@ -333,13 +334,15 @@ uint64_t pc_dimm_get_free_addr(uint64_t address_space_start,
             goto out;
         }
 
-        if (ranges_overlap(dimm->addr, dimm_size, new_addr, size)) {
+        if (ranges_overlap(dimm->addr, dimm_size, new_addr,
+                           size + (gap ? 1 : 0))) {
             if (hint) {
                 DeviceState *d = DEVICE(dimm);
                 error_setg(errp, "address range conflicts with '%s'", d->id);
                 goto out;
             }
-            new_addr = QEMU_ALIGN_UP(dimm->addr + dimm_size, align);
+            new_addr = QEMU_ALIGN_UP(dimm->addr + dimm_size + (gap ? 1 : 0),
+                                     align);
         }
     }
     ret = new_addr;
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index a9b5f2a..d1b0e53 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2096,7 +2096,7 @@ static void spapr_memory_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
         goto out;
     }
 
-    pc_dimm_memory_plug(dev, &ms->hotplug_memory, mr, align, &local_err);
+    pc_dimm_memory_plug(dev, &ms->hotplug_memory, mr, align, false, &local_err);
     if (local_err) {
         goto out;
     }
-- 
MST

[Qemu-devel] [PULL 13/15] pc: memhp: force gaps between DIMM's GPA

[Michael S. Tsirkin]

From: Igor Mammedov <imammedo@redhat.com>

mapping DIMMs non contiguously allows to workaround
virtio bug reported earlier:
http://lists.nongnu.org/archive/html/qemu-devel/2015-08/msg00522.html
in this case guest kernel doesn't allocate buffers
that can cross DIMM boundary keeping each buffer
local to a DIMM.

Suggested-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/i386/pc.h | 1 +
 hw/i386/pc.c         | 6 ++++--
 hw/i386/pc_piix.c    | 1 +
 hw/i386/pc_q35.c     | 1 +
 4 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index ab5413f..c13e91d 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -60,6 +60,7 @@ struct PCMachineClass {
 
     /*< public >*/
     bool broken_reserved_end;
+    bool inter_dimm_gap;
     HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
                                            DeviceState *dev);
 };
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index ef02736..efbd41a 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1629,6 +1629,7 @@ static void pc_dimm_plug(HotplugHandler *hotplug_dev,
     HotplugHandlerClass *hhc;
     Error *local_err = NULL;
     PCMachineState *pcms = PC_MACHINE(hotplug_dev);
+    PCMachineClass *pcmc = PC_MACHINE_GET_CLASS(pcms);
     PCDIMMDevice *dimm = PC_DIMM(dev);
     PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm);
     MemoryRegion *mr = ddc->get_memory_region(dimm);
@@ -1644,8 +1645,8 @@ static void pc_dimm_plug(HotplugHandler *hotplug_dev,
         goto out;
     }
 
-    pc_dimm_memory_plug(dev, &pcms->hotplug_memory, mr, align, false,
-                        &local_err);
+    pc_dimm_memory_plug(dev, &pcms->hotplug_memory, mr, align,
+                        pcmc->inter_dimm_gap, &local_err);
     if (local_err) {
         goto out;
     }
@@ -1946,6 +1947,7 @@ static void pc_machine_class_init(ObjectClass *oc, void *data)
     PCMachineClass *pcmc = PC_MACHINE_CLASS(oc);
     HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc);
 
+    pcmc->inter_dimm_gap = true;
     pcmc->get_hotplug_handler = mc->get_hotplug_handler;
     mc->get_hotplug_handler = pc_get_hotpug_handler;
     mc->cpu_index_to_socket_id = pc_cpu_index_to_socket_id;
diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index 52fdd55..4514cd1 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -489,6 +489,7 @@ static void pc_i440fx_2_4_machine_options(MachineClass *m)
     m->alias = NULL;
     m->is_default = 0;
     pcmc->broken_reserved_end = true;
+    pcmc->inter_dimm_gap = false;
     SET_MACHINE_COMPAT(m, PC_COMPAT_2_4);
 }
 
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 837e430..1f100b1 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -392,6 +392,7 @@ static void pc_q35_2_4_machine_options(MachineClass *m)
     pc_q35_2_5_machine_options(m);
     m->alias = NULL;
     pcmc->broken_reserved_end = true;
+    pcmc->inter_dimm_gap = false;
     SET_MACHINE_COMPAT(m, PC_COMPAT_2_4);
 }
 
-- 
MST

[Qemu-devel] [PULL 14/15] vhost-user-test: use tmpfs by default

[Michael S. Tsirkin]

Most people don't run make check by default, so they skip vhost-user
unit tests.  Solve this by using tmpfs instead, unless hugetlbfs is
specified (using an environment variable).

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 tests/vhost-user-test.c | 37 +++++++++++++++++++++++++------------
 1 file changed, 25 insertions(+), 12 deletions(-)

diff --git a/tests/vhost-user-test.c b/tests/vhost-user-test.c
index 87281b9..5e63cbc 100644
--- a/tests/vhost-user-test.c
+++ b/tests/vhost-user-test.c
@@ -272,17 +272,11 @@ static void chr_read(void *opaque, const uint8_t *buf, int size)
     g_mutex_unlock(&data_mutex);
 }
 
-static const char *init_hugepagefs(void)
+static const char *init_hugepagefs(const char *path)
 {
-    const char *path;
     struct statfs fs;
     int ret;
 
-    path = getenv("QTEST_HUGETLBFS_PATH");
-    if (!path) {
-        path = "/hugetlbfs";
-    }
-
     if (access(path, R_OK | W_OK | X_OK)) {
         g_test_message("access on path (%s): %s\n", path, strerror(errno));
         return NULL;
@@ -309,19 +303,31 @@ int main(int argc, char **argv)
 {
     QTestState *s = NULL;
     CharDriverState *chr = NULL;
-    const char *hugefs = 0;
+    const char *hugefs;
     char *socket_path = 0;
     char *qemu_cmd = 0;
     char *chr_path = 0;
     int ret;
+    char template[] = "/tmp/vhost-test-XXXXXX";
+    const char *tmpfs;
+    const char *root;
 
     g_test_init(&argc, &argv, NULL);
 
     module_call_init(MODULE_INIT_QOM);
 
-    hugefs = init_hugepagefs();
-    if (!hugefs) {
-        return 0;
+    tmpfs = mkdtemp(template);
+    if (!tmpfs) {
+          g_test_message("mkdtemp on path (%s): %s\n", template, strerror(errno));
+    }
+    g_assert(tmpfs);
+
+    hugefs = getenv("QTEST_HUGETLBFS_PATH");
+    if (hugefs) {
+        root = init_hugepagefs(hugefs);
+        g_assert(root);
+    } else {
+        root = tmpfs;
     }
 
     socket_path = g_strdup_printf("/tmp/vhost-%d.sock", getpid());
@@ -338,7 +344,7 @@ int main(int argc, char **argv)
     g_cond_init(&data_cond);
     g_thread_new(NULL, thread_function, NULL);
 
-    qemu_cmd = g_strdup_printf(QEMU_CMD, hugefs, socket_path);
+    qemu_cmd = g_strdup_printf(QEMU_CMD, root, socket_path);
     s = qtest_start(qemu_cmd);
     g_free(qemu_cmd);
 
@@ -354,5 +360,12 @@ int main(int argc, char **argv)
     unlink(socket_path);
     g_free(socket_path);
 
+    ret = rmdir(tmpfs);
+    if (ret != 0) {
+        g_test_message("unable to rmdir: path (%s): %s\n",
+                       tmpfs, strerror(errno));
+    }
+    g_assert_cmpint(ret, ==, 0);
+
     return ret;
 }
-- 
MST

Re: [Qemu-devel] [PULL 14/15] vhost-user-test: use tmpfs by default

[Peter Maydell]

On 2 October 2015 at 14:45, Michael S. Tsirkin <mst@redhat.com> wrote:
> Most people don't run make check by default, so they skip vhost-user
> unit tests.  Solve this by using tmpfs instead, unless hugetlbfs is
> specified (using an environment variable).
>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>

Unfortunately I didn't notice before applying the pull, but this
is breaking 'make check' on AArch64 host for me:

TEST: tests/vhost-user-test... (pid=20205)
Warning: path not on HugeTLBFS: /tmp/vhost-test-gRpbwl
qemu-system-i386: -netdev vhost-user,id=net0,chardev=chr0,vhostforce:
vhost-net support is not compiled in
qemu-system-i386: -netdev vhost-user,id=net0,chardev=chr0,vhostforce:
failed to init vhost_net for queue 0

Broken pipe
FAIL: tests/vhost-user-test

Probably reproducible on x86 if you configure with --disable-vhost-net,
though I haven't tried that.

Perhaps tests/vhost-user-test should be set up
in tests/Makefile using
check-qtest-i386-$(CONFIG_VHOST_USER) rather
than CONFIG_LINUX ?

I'd appreciate a quick fix, because this machine is in my set
of systems I test all pullreqs on now...

thanks
-- PMM

Re: [Qemu-devel] [PULL 14/15] vhost-user-test: use tmpfs by default

[Michael S. Tsirkin]

On Mon, Oct 05, 2015 at 11:39:39PM +0100, Peter Maydell wrote:
> On 2 October 2015 at 14:45, Michael S. Tsirkin <mst@redhat.com> wrote:
> > Most people don't run make check by default, so they skip vhost-user
> > unit tests.  Solve this by using tmpfs instead, unless hugetlbfs is
> > specified (using an environment variable).
> >
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> 
> Unfortunately I didn't notice before applying the pull, but this
> is breaking 'make check' on AArch64 host for me:
> 
> TEST: tests/vhost-user-test... (pid=20205)
> Warning: path not on HugeTLBFS: /tmp/vhost-test-gRpbwl
> qemu-system-i386: -netdev vhost-user,id=net0,chardev=chr0,vhostforce:
> vhost-net support is not compiled in
> qemu-system-i386: -netdev vhost-user,id=net0,chardev=chr0,vhostforce:
> failed to init vhost_net for queue 0
> 
> Broken pipe
> FAIL: tests/vhost-user-test
> 
> Probably reproducible on x86 if you configure with --disable-vhost-net,
> though I haven't tried that.
> 
> Perhaps tests/vhost-user-test should be set up
> in tests/Makefile using
> check-qtest-i386-$(CONFIG_VHOST_USER) rather
> than CONFIG_LINUX ?
> 
> I'd appreciate a quick fix, because this machine is in my set
> of systems I test all pullreqs on now...
> 
> thanks
> -- PMM


ok first of all we need this: if I apply it, I see the same
bug as you do. need to fix that too.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>


diff --git a/net/net.c b/net/net.c
index 28a5597..1e6a082 100644
--- a/net/net.c
+++ b/net/net.c
@@ -902,9 +902,7 @@ static int (* const net_client_init_fun[NET_CLIENT_OPTIONS_KIND_MAX])(
         [NET_CLIENT_OPTIONS_KIND_BRIDGE]    = net_init_bridge,
 #endif
         [NET_CLIENT_OPTIONS_KIND_HUBPORT]   = net_init_hubport,
-#ifdef CONFIG_VHOST_NET_USED
         [NET_CLIENT_OPTIONS_KIND_VHOST_USER] = net_init_vhost_user,
-#endif
 #ifdef CONFIG_L2TPV3
         [NET_CLIENT_OPTIONS_KIND_L2TPV3]    = net_init_l2tpv3,
 #endif

Re: [Qemu-devel] [PULL 14/15] vhost-user-test: use tmpfs by default

[Michael S. Tsirkin]

On Mon, Oct 05, 2015 at 11:39:39PM +0100, Peter Maydell wrote:
> On 2 October 2015 at 14:45, Michael S. Tsirkin <mst@redhat.com> wrote:
> > Most people don't run make check by default, so they skip vhost-user
> > unit tests.  Solve this by using tmpfs instead, unless hugetlbfs is
> > specified (using an environment variable).
> >
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> 
> Unfortunately I didn't notice before applying the pull, but this
> is breaking 'make check' on AArch64 host for me:
> 
> TEST: tests/vhost-user-test... (pid=20205)
> Warning: path not on HugeTLBFS: /tmp/vhost-test-gRpbwl
> qemu-system-i386: -netdev vhost-user,id=net0,chardev=chr0,vhostforce:
> vhost-net support is not compiled in
> qemu-system-i386: -netdev vhost-user,id=net0,chardev=chr0,vhostforce:
> failed to init vhost_net for queue 0
> 
> Broken pipe
> FAIL: tests/vhost-user-test
> 
> Probably reproducible on x86 if you configure with --disable-vhost-net,
> though I haven't tried that.
> 
> Perhaps tests/vhost-user-test should be set up
> in tests/Makefile using
> check-qtest-i386-$(CONFIG_VHOST_USER) rather
> than CONFIG_LINUX ?
> 
> I'd appreciate a quick fix, because this machine is in my set
> of systems I test all pullreqs on now...
> 
> thanks
> -- PMM

I think you are right, but just to be on the safe side, let's test
both for now.

If this helps you, pls feel free to apply.

I will look at cleaning this up later.

-->

tests: vhost-user: disable unless CONFIG_VHOST_NET

vhost-user depends on vhost-net. We should probably fix that.
For now, let's disable the test otherwise.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

---

diff --git a/tests/Makefile b/tests/Makefile
index 4063639..e6474ba 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -188,7 +188,9 @@ gcov-files-i386-y += hw/usb/hcd-xhci.c
 check-qtest-i386-y += tests/pc-cpu-test$(EXESUF)
 check-qtest-i386-y += tests/q35-test$(EXESUF)
 gcov-files-i386-y += hw/pci-host/q35.c
+ifeq ($(CONFIG_VHOST_NET),y)
 check-qtest-i386-$(CONFIG_LINUX) += tests/vhost-user-test$(EXESUF)
+endif
 check-qtest-x86_64-y = $(check-qtest-i386-y)
 gcov-files-i386-y += i386-softmmu/hw/timer/mc146818rtc.c
 gcov-files-x86_64-y = $(subst i386-softmmu/,x86_64-softmmu/,$(gcov-files-i386-y))

Re: [Qemu-devel] [PULL 14/15] vhost-user-test: use tmpfs by default

[Peter Maydell]

On 6 October 2015 at 00:17, Michael S. Tsirkin <mst@redhat.com> wrote:
> On Mon, Oct 05, 2015 at 11:39:39PM +0100, Peter Maydell wrote:
>> On 2 October 2015 at 14:45, Michael S. Tsirkin <mst@redhat.com> wrote:
>> > Most people don't run make check by default, so they skip vhost-user
>> > unit tests.  Solve this by using tmpfs instead, unless hugetlbfs is
>> > specified (using an environment variable).
>> >
>> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>> > Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>>
>> Unfortunately I didn't notice before applying the pull, but this
>> is breaking 'make check' on AArch64 host for me:
>>
>> TEST: tests/vhost-user-test... (pid=20205)
>> Warning: path not on HugeTLBFS: /tmp/vhost-test-gRpbwl
>> qemu-system-i386: -netdev vhost-user,id=net0,chardev=chr0,vhostforce:
>> vhost-net support is not compiled in
>> qemu-system-i386: -netdev vhost-user,id=net0,chardev=chr0,vhostforce:
>> failed to init vhost_net for queue 0
>>
>> Broken pipe
>> FAIL: tests/vhost-user-test
>>
>> Probably reproducible on x86 if you configure with --disable-vhost-net,
>> though I haven't tried that.
>>
>> Perhaps tests/vhost-user-test should be set up
>> in tests/Makefile using
>> check-qtest-i386-$(CONFIG_VHOST_USER) rather
>> than CONFIG_LINUX ?
>>
>> I'd appreciate a quick fix, because this machine is in my set
>> of systems I test all pullreqs on now...
>>
>> thanks
>> -- PMM
>
> I think you are right, but just to be on the safe side, let's test
> both for now.
>
> If this helps you, pls feel free to apply.
>
> I will look at cleaning this up later.

Yes, this fixes the test run on aarch64; I'll apply it to
master.

-- PMM

Re: [Qemu-devel] [PULL 14/15] vhost-user-test: use tmpfs by default

[Peter Maydell]

On 6 October 2015 at 09:41, Peter Maydell <peter.maydell@linaro.org> wrote:
> On 6 October 2015 at 00:17, Michael S. Tsirkin <mst@redhat.com> wrote:
>> If this helps you, pls feel free to apply.
>>
>> I will look at cleaning this up later.
>
> Yes, this fixes the test run on aarch64; I'll apply it to
> master.

Now applied; thanks.

-- PMM

Re: [Qemu-devel] [PULL 14/15] vhost-user-test: use tmpfs by default

[Marc-André Lureau]

Hi

On Tue, Oct 6, 2015 at 1:17 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
> On Mon, Oct 05, 2015 at 11:39:39PM +0100, Peter Maydell wrote:
>> On 2 October 2015 at 14:45, Michael S. Tsirkin <mst@redhat.com> wrote:
>> > Most people don't run make check by default, so they skip vhost-user
>> > unit tests.  Solve this by using tmpfs instead, unless hugetlbfs is
>> > specified (using an environment variable).
>> >
>> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>> > Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>>
>> Unfortunately I didn't notice before applying the pull, but this
>> is breaking 'make check' on AArch64 host for me:
>>
>> TEST: tests/vhost-user-test... (pid=20205)
>> Warning: path not on HugeTLBFS: /tmp/vhost-test-gRpbwl
>> qemu-system-i386: -netdev vhost-user,id=net0,chardev=chr0,vhostforce:
>> vhost-net support is not compiled in
>> qemu-system-i386: -netdev vhost-user,id=net0,chardev=chr0,vhostforce:
>> failed to init vhost_net for queue 0
>>
>> Broken pipe
>> FAIL: tests/vhost-user-test
>>
>> Probably reproducible on x86 if you configure with --disable-vhost-net,
>> though I haven't tried that.
>>
>> Perhaps tests/vhost-user-test should be set up
>> in tests/Makefile using
>> check-qtest-i386-$(CONFIG_VHOST_USER) rather
>> than CONFIG_LINUX ?
>>
>> I'd appreciate a quick fix, because this machine is in my set
>> of systems I test all pullreqs on now...
>>
>> thanks
>> -- PMM
>
> I think you are right, but just to be on the safe side, let's test
> both for now.
>
> If this helps you, pls feel free to apply.
>
> I will look at cleaning this up later.
>
> -->
>
> tests: vhost-user: disable unless CONFIG_VHOST_NET
>
> vhost-user depends on vhost-net. We should probably fix that.
> For now, let's disable the test otherwise.
>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>

Unfortunately, this commit disables vhost-user-test all together,
because CONFIG_VHOST_NET is a target config.

> ---
>
> diff --git a/tests/Makefile b/tests/Makefile
> index 4063639..e6474ba 100644
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -188,7 +188,9 @@ gcov-files-i386-y += hw/usb/hcd-xhci.c
>  check-qtest-i386-y += tests/pc-cpu-test$(EXESUF)
>  check-qtest-i386-y += tests/q35-test$(EXESUF)
>  gcov-files-i386-y += hw/pci-host/q35.c
> +ifeq ($(CONFIG_VHOST_NET),y)
>  check-qtest-i386-$(CONFIG_LINUX) += tests/vhost-user-test$(EXESUF)
> +endif
>  check-qtest-x86_64-y = $(check-qtest-i386-y)
>  gcov-files-i386-y += i386-softmmu/hw/timer/mc146818rtc.c
>  gcov-files-x86_64-y = $(subst i386-softmmu/,x86_64-softmmu/,$(gcov-files-i386-y))
>
>



-- 
Marc-André Lureau

[Qemu-devel] [PULL 15/15] vhost-user-test: fix predictable filename on tmpfs

[Michael S. Tsirkin]

vhost-user-test uses getpid to create a unique filename. This name is
predictable, and a security problem.  Instead, use a tmp directory
created by mkdtemp, which is a suggested best practice.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 tests/vhost-user-test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/vhost-user-test.c b/tests/vhost-user-test.c
index 5e63cbc..56df5cc 100644
--- a/tests/vhost-user-test.c
+++ b/tests/vhost-user-test.c
@@ -330,7 +330,7 @@ int main(int argc, char **argv)
         root = tmpfs;
     }
 
-    socket_path = g_strdup_printf("/tmp/vhost-%d.sock", getpid());
+    socket_path = g_strdup_printf("%s/vhost.sock", tmpfs);
 
     /* create char dev and add read handlers */
     qemu_add_opts(&qemu_chardev_opts);
-- 
MST

Re: [Qemu-devel] [PULL 00/15] virtio,pc features, fixes

[Michael S. Tsirkin]

On Fri, Oct 02, 2015 at 04:44:58PM +0300, Michael S. Tsirkin wrote:
> I expected to also merge memory allocation refactoring
> and vhost user migration support, but that appears to
> need more work.
> 
> The following changes since commit fa500928ad9da6dd570918e3dfca13c029af07a8:
> 
>   Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20150930' into staging (2015-10-01 10:49:38 +0100)
> 
> are available in the git repository at:
> 
>   git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git tags/for_upstream
> 
> for you to fetch changes up to a9faaa0aa97509cc14adb9f7ce2f3bfcd88066c4:

I rebased, dropping 1 patch.
new commit 6fdac09370530be0cc6fe9e8d425c0670ba994b1
Sorry about the noise.

>   vhost-user-test: fix predictable filename on tmpfs (2015-10-02 16:40:58 +0300)
> 
> ----------------------------------------------------------------
> virtio,pc features, fixes
> 
> New features:
>     guest RAM buffer overrun mitigation
>     RAM physical address gaps for memory hotplug
>     (except refactoring which got some review comments)
>     virtio-9f migration (when unmounted)
> 
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> 
> ----------------------------------------------------------------
> Eduardo Habkost (1):
>       pc: Add a comment explaining why pc_compat_2_4() doesn't exist
> 
> Greg Kurz (1):
>       virtio-9p: migrate virtio subsections
> 
> Igor Mammedov (2):
>       memhp: extend address auto assignment to support gaps
>       pc: memhp: force gaps between DIMM's GPA
> 
> Jason Wang (3):
>       virtio: introduce virtqueue_unmap_sg()
>       virtio: introduce virtqueue_discard()
>       virtio-net: correctly drop truncated packets
> 
> Michael S. Tsirkin (6):
>       oslib: rework anonimous RAM allocation
>       oslib: allocate PROT_NONE pages on top of RAM
>       exec: allocate PROT_NONE pages on top of RAM
>       vhost-user: unit test for new messages
>       vhost-user-test: use tmpfs by default
>       vhost-user-test: fix predictable filename on tmpfs
> 
> Paolo Bonzini (1):
>       vhost-user-test: do not reinvent glib-compat.h
> 
> Richard Henderson (1):
>       virtio: Notice when the system doesn't support MSIx at all
> 
>  include/hw/i386/pc.h       |   1 +
>  include/hw/mem/pc-dimm.h   |   7 +-
>  include/hw/virtio/virtio.h |   2 +
>  exec.c                     |  42 ++++++++++-
>  hw/9pfs/virtio-9p-device.c |  11 +++
>  hw/i386/pc.c               |   5 +-
>  hw/i386/pc_piix.c          |   8 +++
>  hw/i386/pc_q35.c           |   8 +++
>  hw/mem/pc-dimm.c           |  15 ++--
>  hw/net/virtio-net.c        |   8 +--
>  hw/ppc/spapr.c             |   2 +-
>  hw/virtio/virtio-pci.c     |  17 +++--
>  hw/virtio/virtio.c         |  21 ++++--
>  tests/vhost-user-test.c    | 170 ++++++++++++++++-----------------------------
>  util/oslib-posix.c         |  20 ++++--
>  15 files changed, 190 insertions(+), 147 deletions(-)
> 

Re: [Qemu-devel] [PULL 00/15] virtio,pc features, fixes

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 636 bytes --]

On 10/02/2015 08:06 AM, Michael S. Tsirkin wrote:
> On Fri, Oct 02, 2015 at 04:44:58PM +0300, Michael S. Tsirkin wrote:
>> I expected to also merge memory allocation refactoring
>> and vhost user migration support, but that appears to
>> need more work.
>>

> 
> I rebased, dropping 1 patch.
> new commit 6fdac09370530be0cc6fe9e8d425c0670ba994b1
> Sorry about the noise.
> 

>> Michael S. Tsirkin (6):
>>       oslib: rework anonimous RAM allocation

Do you want to also rebase to fix the subject line typo?

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PULL 00/15] virtio,pc features, fixes

[Peter Maydell]

On 2 October 2015 at 15:06, Michael S. Tsirkin <mst@redhat.com> wrote:
> On Fri, Oct 02, 2015 at 04:44:58PM +0300, Michael S. Tsirkin wrote:
>> I expected to also merge memory allocation refactoring
>> and vhost user migration support, but that appears to
>> need more work.
>>
>> The following changes since commit fa500928ad9da6dd570918e3dfca13c029af07a8:
>>
>>   Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20150930' into staging (2015-10-01 10:49:38 +0100)
>>
>> are available in the git repository at:
>>
>>   git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git tags/for_upstream
>>
>> for you to fetch changes up to a9faaa0aa97509cc14adb9f7ce2f3bfcd88066c4:
>
> I rebased, dropping 1 patch.
> new commit 6fdac09370530be0cc6fe9e8d425c0670ba994b1
> Sorry about the noise.
>
>>   vhost-user-test: fix predictable filename on tmpfs (2015-10-02 16:40:58 +0300)
>>
>> ----------------------------------------------------------------
>> virtio,pc features, fixes
>>
>> New features:
>>     guest RAM buffer overrun mitigation
>>     RAM physical address gaps for memory hotplug
>>     (except refactoring which got some review comments)
>>     virtio-9f migration (when unmounted)
>>
>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>>

Applied, thanks.

-- PMM