[Qemu-devel] [PULL 0/4] Block patches

[Qemu-devel] [PULL 0/4] Block patches

[Stefan Hajnoczi]

The following changes since commit 5a24f20a7208a58fb80d78ca0521bba6f4d7b145:

  Merge remote-tracking branch 'remotes/mjt/tags/pull-trivial-patches-2015-04-04' into staging (2015-04-07 14:33:46 +0100)

are available in the git repository at:

  git://github.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 05b685fbabb7fdcab72cb42b27db916fd74b2265:

  block/iscsi: handle zero events from iscsi_which_events (2015-04-09 10:31:45 +0100)

----------------------------------------------------------------

----------------------------------------------------------------

Kevin Wolf (1):
  qcow2: Fix header update with overridden backing file

Paolo Bonzini (2):
  virtio-blk: correctly dirty guest memory
  aio: strengthen memory barriers for bottom half scheduling

Peter Lieven (1):
  block/iscsi: handle zero events from iscsi_which_events

 async.c                         | 28 ++++++------
 block/iscsi.c                   | 33 +++++++++++---
 block/qcow2.c                   | 29 ++++++++++---
 block/qcow2.h                   |  6 +++
 hw/block/dataplane/virtio-blk.c |  3 +-
 hw/block/virtio-blk.c           | 13 +++++-
 include/hw/virtio/virtio-blk.h  |  1 +
 tests/qemu-iotests/130          | 95 +++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/130.out      | 43 +++++++++++++++++++
 tests/qemu-iotests/group        |  1 +
 10 files changed, 220 insertions(+), 32 deletions(-)
 create mode 100755 tests/qemu-iotests/130
 create mode 100644 tests/qemu-iotests/130.out

-- 
2.1.0

[Qemu-devel] [PULL 1/4] qcow2: Fix header update with overridden backing file

[Stefan Hajnoczi]

From: Kevin Wolf <kwolf@redhat.com>

In recent qemu versions, it is possible to override the backing file
name and format that is stored in the image file with values given at
runtime. In such cases, the temporary override could end up in the
image header if the qcow2 header was updated, while obviously correct
behaviour would be to leave the on-disk backing file path/format
unchanged.

Fix this and add a test case for it.

Reported-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1428411796-2852-1-git-send-email-kwolf@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/qcow2.c              | 29 ++++++++++----
 block/qcow2.h              |  6 +++
 tests/qemu-iotests/130     | 95 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/130.out | 43 +++++++++++++++++++++
 tests/qemu-iotests/group   |  1 +
 5 files changed, 167 insertions(+), 7 deletions(-)
 create mode 100755 tests/qemu-iotests/130
 create mode 100644 tests/qemu-iotests/130.out

diff --git a/block/qcow2.c b/block/qcow2.c
index 32bdf75..316a8db 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -140,6 +140,7 @@ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset,
                 return 3;
             }
             bs->backing_format[ext.len] = '\0';
+            s->image_backing_format = g_strdup(bs->backing_format);
 #ifdef DEBUG_EXT
             printf("Qcow2: Got format extension %s\n", bs->backing_format);
 #endif
@@ -884,6 +885,7 @@ static int qcow2_open(BlockDriverState *bs, QDict *options, int flags,
             goto fail;
         }
         bs->backing_file[len] = '\0';
+        s->image_backing_file = g_strdup(bs->backing_file);
     }
 
     /* Internal snapshots */
@@ -1457,6 +1459,9 @@ static void qcow2_close(BlockDriverState *bs)
     g_free(s->unknown_header_fields);
     cleanup_unknown_header_ext(bs);
 
+    g_free(s->image_backing_file);
+    g_free(s->image_backing_format);
+
     g_free(s->cluster_cache);
     qemu_vfree(s->cluster_data);
     qcow2_refcount_close(bs);
@@ -1622,9 +1627,10 @@ int qcow2_update_header(BlockDriverState *bs)
     }
 
     /* Backing file format header extension */
-    if (*bs->backing_format) {
+    if (s->image_backing_format) {
         ret = header_ext_add(buf, QCOW2_EXT_MAGIC_BACKING_FORMAT,
-                             bs->backing_format, strlen(bs->backing_format),
+                             s->image_backing_format,
+                             strlen(s->image_backing_format),
                              buflen);
         if (ret < 0) {
             goto fail;
@@ -1682,8 +1688,8 @@ int qcow2_update_header(BlockDriverState *bs)
     buflen -= ret;
 
     /* Backing file name */
-    if (*bs->backing_file) {
-        size_t backing_file_len = strlen(bs->backing_file);
+    if (s->image_backing_file) {
+        size_t backing_file_len = strlen(s->image_backing_file);
 
         if (buflen < backing_file_len) {
             ret = -ENOSPC;
@@ -1691,7 +1697,7 @@ int qcow2_update_header(BlockDriverState *bs)
         }
 
         /* Using strncpy is ok here, since buf is not NUL-terminated. */
-        strncpy(buf, bs->backing_file, buflen);
+        strncpy(buf, s->image_backing_file, buflen);
 
         header->backing_file_offset = cpu_to_be64(buf - ((char*) header));
         header->backing_file_size   = cpu_to_be32(backing_file_len);
@@ -1712,9 +1718,17 @@ fail:
 static int qcow2_change_backing_file(BlockDriverState *bs,
     const char *backing_file, const char *backing_fmt)
 {
+    BDRVQcowState *s = bs->opaque;
+
     pstrcpy(bs->backing_file, sizeof(bs->backing_file), backing_file ?: "");
     pstrcpy(bs->backing_format, sizeof(bs->backing_format), backing_fmt ?: "");
 
+    g_free(s->image_backing_file);
+    g_free(s->image_backing_format);
+
+    s->image_backing_file = backing_file ? g_strdup(bs->backing_file) : NULL;
+    s->image_backing_format = backing_fmt ? g_strdup(bs->backing_format) : NULL;
+
     return qcow2_update_header(bs);
 }
 
@@ -2751,8 +2765,9 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
     }
 
     if (backing_file || backing_format) {
-        ret = qcow2_change_backing_file(bs, backing_file ?: bs->backing_file,
-                                        backing_format ?: bs->backing_format);
+        ret = qcow2_change_backing_file(bs,
+                    backing_file ?: s->image_backing_file,
+                    backing_format ?: s->image_backing_format);
         if (ret < 0) {
             return ret;
         }
diff --git a/block/qcow2.h b/block/qcow2.h
index aa6d367..422b825 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -283,6 +283,12 @@ typedef struct BDRVQcowState {
     QLIST_HEAD(, Qcow2UnknownHeaderExtension) unknown_header_ext;
     QTAILQ_HEAD (, Qcow2DiscardRegion) discards;
     bool cache_discards;
+
+    /* Backing file path and format as stored in the image (this is not the
+     * effective path/format, which may be the result of a runtime option
+     * override) */
+    char *image_backing_file;
+    char *image_backing_format;
 } BDRVQcowState;
 
 struct QCowAIOCB;
diff --git a/tests/qemu-iotests/130 b/tests/qemu-iotests/130
new file mode 100755
index 0000000..bc26247
--- /dev/null
+++ b/tests/qemu-iotests/130
@@ -0,0 +1,95 @@
+#!/bin/bash
+#
+# Test that temporary backing file overrides (on the command line or in
+# blockdev-add) don't replace the original path stored in the image during
+# header updates.
+#
+# Copyright (C) 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=kwolf@redhat.com
+
+seq="$(basename $0)"
+echo "QA output created by $seq"
+
+here="$PWD"
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    _cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+. ./common.qemu
+
+_supported_fmt qcow2
+_supported_proto generic
+_supported_os Linux
+
+qemu_comm_method="monitor"
+
+
+TEST_IMG="$TEST_IMG.orig" _make_test_img 64M
+TEST_IMG="$TEST_IMG.base" _make_test_img 64M
+_make_test_img 64M
+_img_info | _filter_img_info
+
+echo
+echo "=== HMP commit ==="
+echo
+# bdrv_make_empty() involves a header update for qcow2
+
+# Test that a backing file isn't written
+_launch_qemu -drive file="$TEST_IMG",backing.file.filename="$TEST_IMG.base"
+_send_qemu_cmd $QEMU_HANDLE "commit ide0-hd0" "(qemu)"
+_send_qemu_cmd $QEMU_HANDLE '' '(qemu)'
+_cleanup_qemu
+_img_info | _filter_img_info
+
+# Make sure that if there was a backing file that was just overridden on the
+# command line, that backing file is retained, with the right format
+_make_test_img -F raw -b "$TEST_IMG.orig" 64M
+_launch_qemu -drive file="$TEST_IMG",backing.file.filename="$TEST_IMG.base",backing.driver=$IMGFMT
+_send_qemu_cmd $QEMU_HANDLE "commit ide0-hd0" "(qemu)"
+_send_qemu_cmd $QEMU_HANDLE '' '(qemu)'
+_cleanup_qemu
+_img_info | _filter_img_info
+
+echo
+echo "=== Marking image dirty (lazy refcounts) ==="
+echo
+
+# Test that a backing file isn't written
+_make_test_img 64M
+$QEMU_IO -c "open -o backing.file.filename=$TEST_IMG.base,lazy-refcounts=on $TEST_IMG" -c "write 0 4k" | _filter_qemu_io
+_img_info | _filter_img_info
+
+# Make sure that if there was a backing file that was just overridden on the
+# command line, that backing file is retained, with the right format
+_make_test_img -F raw -b "$TEST_IMG.orig" 64M
+$QEMU_IO -c "open -o backing.file.filename=$TEST_IMG.base,backing.driver=$IMGFMT,lazy-refcounts=on $TEST_IMG" -c "write 0 4k" | _filter_qemu_io
+_img_info | _filter_img_info
+
+# success, all done
+echo '*** done'
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/130.out b/tests/qemu-iotests/130.out
new file mode 100644
index 0000000..ea68b5d
--- /dev/null
+++ b/tests/qemu-iotests/130.out
@@ -0,0 +1,43 @@
+QA output created by 130
+Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+image: TEST_DIR/t.IMGFMT
+file format: IMGFMT
+virtual size: 64M (67108864 bytes)
+
+=== HMP commit ===
+
+QEMU X.Y.Z monitor - type 'help' for more information
+(qemu) c^[[K^[[Dco^[[K^[[D^[[Dcom^[[K^[[D^[[D^[[Dcomm^[[K^[[D^[[D^[[D^[[Dcommi^[[K^[[D^[[D^[[D^[[D^[[Dcommit^[[K^[[D^[[D^[[D^[[D^[[D^[[Dcommit ^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit i^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit id^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit ide^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit ide0^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit ide0-^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit ide0-h^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit ide0-hd^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit ide0-hd0^[[K
+(qemu) 
+image: TEST_DIR/t.IMGFMT
+file format: IMGFMT
+virtual size: 64M (67108864 bytes)
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file='TEST_DIR/t.IMGFMT.orig' backing_fmt='raw'
+QEMU X.Y.Z monitor - type 'help' for more information
+(qemu) c^[[K^[[Dco^[[K^[[D^[[Dcom^[[K^[[D^[[D^[[Dcomm^[[K^[[D^[[D^[[D^[[Dcommi^[[K^[[D^[[D^[[D^[[D^[[Dcommit^[[K^[[D^[[D^[[D^[[D^[[D^[[Dcommit ^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit i^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit id^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit ide^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit ide0^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit ide0-^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit ide0-h^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit ide0-hd^[[K^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[D^[[Dcommit ide0-hd0^[[K
+(qemu) 
+image: TEST_DIR/t.IMGFMT
+file format: IMGFMT
+virtual size: 64M (67108864 bytes)
+backing file: TEST_DIR/t.IMGFMT.orig
+backing file format: raw
+
+=== Marking image dirty (lazy refcounts) ===
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+wrote 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+image: TEST_DIR/t.IMGFMT
+file format: IMGFMT
+virtual size: 64M (67108864 bytes)
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file='TEST_DIR/t.IMGFMT.orig' backing_fmt='raw'
+wrote 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+image: TEST_DIR/t.IMGFMT
+file format: IMGFMT
+virtual size: 64M (67108864 bytes)
+backing file: TEST_DIR/t.IMGFMT.orig
+backing file format: raw
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 6262127..bcf2578 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -124,3 +124,4 @@
 121 rw auto
 123 rw auto quick
 128 rw auto quick
+130 rw auto quick
-- 
2.1.0

[Qemu-devel] [PULL 2/4] virtio-blk: correctly dirty guest memory

[Stefan Hajnoczi]

From: Paolo Bonzini <pbonzini@redhat.com>

After qemu_iovec_destroy, the QEMUIOVector's size is zeroed and
the zero size ultimately is used to compute virtqueue_push's len
argument.  Therefore, reads from virtio-blk devices did not
migrate their results correctly.  (Writes were okay).

Save the size in virtio_blk_handle_request, and use it when the request
is completed.

Based on a patch by Wen Congyang.

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Tested-by: Li Zhijian <lizhijian@cn.fujitsu.com>
Message-id: 1427997044-392-1-git-send-email-pbonzini@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/block/dataplane/virtio-blk.c |  3 +--
 hw/block/virtio-blk.c           | 13 ++++++++++++-
 include/hw/virtio/virtio-blk.h  |  1 +
 3 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c
index cd41478..3db139b 100644
--- a/hw/block/dataplane/virtio-blk.c
+++ b/hw/block/dataplane/virtio-blk.c
@@ -77,8 +77,7 @@ static void complete_request_vring(VirtIOBlockReq *req, unsigned char status)
     VirtIOBlockDataPlane *s = req->dev->dataplane;
     stb_p(&req->in->status, status);
 
-    vring_push(s->vdev, &req->dev->dataplane->vring, &req->elem,
-               req->qiov.size + sizeof(*req->in));
+    vring_push(s->vdev, &req->dev->dataplane->vring, &req->elem, req->in_len);
 
     /* Suppress notification to guest by BH and its scheduled
      * flag because requests are completed as a batch after io
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index 000c38d..9546fd2 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -33,6 +33,7 @@ VirtIOBlockReq *virtio_blk_alloc_request(VirtIOBlock *s)
     VirtIOBlockReq *req = g_slice_new(VirtIOBlockReq);
     req->dev = s;
     req->qiov.size = 0;
+    req->in_len = 0;
     req->next = NULL;
     req->mr_next = NULL;
     return req;
@@ -54,7 +55,7 @@ static void virtio_blk_complete_request(VirtIOBlockReq *req,
     trace_virtio_blk_req_complete(req, status);
 
     stb_p(&req->in->status, status);
-    virtqueue_push(s->vq, &req->elem, req->qiov.size + sizeof(*req->in));
+    virtqueue_push(s->vq, &req->elem, req->in_len);
     virtio_notify(vdev, s->vq);
 }
 
@@ -102,6 +103,14 @@ static void virtio_blk_rw_complete(void *opaque, int ret)
         if (ret) {
             int p = virtio_ldl_p(VIRTIO_DEVICE(req->dev), &req->out.type);
             bool is_read = !(p & VIRTIO_BLK_T_OUT);
+            /* Note that memory may be dirtied on read failure.  If the
+             * virtio request is not completed here, as is the case for
+             * BLOCK_ERROR_ACTION_STOP, the memory may not be copied
+             * correctly during live migration.  While this is ugly,
+             * it is acceptable because the device is free to write to
+             * the memory until the request is completed (which will
+             * happen on the other side of the migration).
+             */
             if (virtio_blk_handle_rw_error(req, -ret, is_read)) {
                 continue;
             }
@@ -496,6 +505,8 @@ void virtio_blk_handle_request(VirtIOBlockReq *req, MultiReqBuffer *mrb)
         exit(1);
     }
 
+    /* We always touch the last byte, so just see how big in_iov is.  */
+    req->in_len = iov_size(in_iov, in_num);
     req->in = (void *)in_iov[in_num - 1].iov_base
               + in_iov[in_num - 1].iov_len
               - sizeof(struct virtio_blk_inhdr);
diff --git a/include/hw/virtio/virtio-blk.h b/include/hw/virtio/virtio-blk.h
index b3ffcd9..6bf5905 100644
--- a/include/hw/virtio/virtio-blk.h
+++ b/include/hw/virtio/virtio-blk.h
@@ -67,6 +67,7 @@ typedef struct VirtIOBlockReq {
     struct virtio_blk_inhdr *in;
     struct virtio_blk_outhdr out;
     QEMUIOVector qiov;
+    size_t in_len;
     struct VirtIOBlockReq *next;
     struct VirtIOBlockReq *mr_next;
     BlockAcctCookie acct;
-- 
2.1.0

[Qemu-devel] [PULL 3/4] aio: strengthen memory barriers for bottom half scheduling

[Stefan Hajnoczi]

From: Paolo Bonzini <pbonzini@redhat.com>

There are two problems with memory barriers in async.c.  The fix is
to use atomic_xchg in order to achieve sequential consistency between
the scheduling of a bottom half and the corresponding execution.

First, if bh->scheduled is already 1 in qemu_bh_schedule, QEMU does
not execute a memory barrier to order any writes needed by the callback
before the read of bh->scheduled.  If the other side sees req->state as
THREAD_ACTIVE, the callback is not invoked and you get deadlock.

Second, the memory barrier in aio_bh_poll is too weak.  Without this
patch, it is possible that bh->scheduled = 0 is not "published" until
after the callback has returned.  Another thread wants to schedule the
bottom half, but it sees bh->scheduled = 1 and does nothing.  This causes
a lost wakeup.  The memory barrier should have been changed to smp_mb()
in commit 924fe12 (aio: fix qemu_bh_schedule() bh->ctx race condition,
2014-06-03) together with qemu_bh_schedule()'s.  Guess who reviewed
that patch?

Both of these involve a store and a load, so they are reproducible on
x86_64 as well.  It is however much easier on aarch64, where the
libguestfs test suite triggers the bug fairly easily.  Even there the
failure can go away or appear depending on compiler optimization level,
tracing options, or even kernel debugging options.

Paul Leveille however reported how to trigger the problem within 15
minutes on x86_64 as well.  His (untested) recipe, reproduced here
for reference, is the following:

   1) Qcow2 (or 3) is critical – raw files alone seem to avoid the problem.

   2) Use “cache=directsync” rather than the default of
   “cache=none” to make it happen easier.

   3) Use a server with a write-back RAID controller to allow for rapid
   IO rates.

   4) Run a random-access load that (mostly) writes chunks to various
   files on the virtual block device.

      a. I use ‘diskload.exe c:25’, a Microsoft HCT load
         generator, on Windows VMs.

      b. Iometer can probably be configured to generate a similar load.

   5) Run multiple VMs in parallel, against the same storage device,
   to shake the failure out sooner.

   6) IvyBridge and Haswell processors for certain; not sure about others.

A similar patch survived over 12 hours of testing, where an unpatched
QEMU would fail within 15 minutes.

This bug is, most likely, also the cause of failures in the libguestfs
testsuite on AArch64.

Thanks to Laszlo Ersek for initially reporting this bug, to Stefan
Hajnoczi for suggesting closer examination of qemu_bh_schedule, and to
Paul for providing test input and a prototype patch.

Reported-by: Laszlo Ersek <lersek@redhat.com>
Reported-by: Paul Leveille <Paul.Leveille@stratus.com>
Reported-by: John Snow <jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 1428419779-26062-1-git-send-email-pbonzini@redhat.com
Suggested-by: Paul Leveille <Paul.Leveille@stratus.com>
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 async.c | 28 ++++++++++++----------------
 1 file changed, 12 insertions(+), 16 deletions(-)

diff --git a/async.c b/async.c
index 2be88cc..2b51e87 100644
--- a/async.c
+++ b/async.c
@@ -72,12 +72,13 @@ int aio_bh_poll(AioContext *ctx)
         /* Make sure that fetching bh happens before accessing its members */
         smp_read_barrier_depends();
         next = bh->next;
-        if (!bh->deleted && bh->scheduled) {
-            bh->scheduled = 0;
-            /* Paired with write barrier in bh schedule to ensure reading for
-             * idle & callbacks coming after bh's scheduling.
-             */
-            smp_rmb();
+        /* The atomic_xchg is paired with the one in qemu_bh_schedule.  The
+         * implicit memory barrier ensures that the callback sees all writes
+         * done by the scheduling thread.  It also ensures that the scheduling
+         * thread sees the zero before bh->cb has run, and thus will call
+         * aio_notify again if necessary.
+         */
+        if (!bh->deleted && atomic_xchg(&bh->scheduled, 0)) {
             if (!bh->idle)
                 ret = 1;
             bh->idle = 0;
@@ -108,33 +109,28 @@ int aio_bh_poll(AioContext *ctx)
 
 void qemu_bh_schedule_idle(QEMUBH *bh)
 {
-    if (bh->scheduled)
-        return;
     bh->idle = 1;
     /* Make sure that idle & any writes needed by the callback are done
      * before the locations are read in the aio_bh_poll.
      */
-    smp_wmb();
-    bh->scheduled = 1;
+    atomic_mb_set(&bh->scheduled, 1);
 }
 
 void qemu_bh_schedule(QEMUBH *bh)
 {
     AioContext *ctx;
 
-    if (bh->scheduled)
-        return;
     ctx = bh->ctx;
     bh->idle = 0;
-    /* Make sure that:
+    /* The memory barrier implicit in atomic_xchg makes sure that:
      * 1. idle & any writes needed by the callback are done before the
      *    locations are read in the aio_bh_poll.
      * 2. ctx is loaded before scheduled is set and the callback has a chance
      *    to execute.
      */
-    smp_mb();
-    bh->scheduled = 1;
-    aio_notify(ctx);
+    if (atomic_xchg(&bh->scheduled, 1) == 0) {
+        aio_notify(ctx);
+    }
 }
 
 
-- 
2.1.0

[Qemu-devel] [PULL 4/4] block/iscsi: handle zero events from iscsi_which_events

[Stefan Hajnoczi]

From: Peter Lieven <pl@kamp.de>

newer libiscsi versions may return zero events from iscsi_which_events.

In this case iscsi_service will return immediately without any progress.
To avoid busy waiting for iscsi_which_events to change we deregister all
read and write handlers in this case and schedule a timer to periodically
check iscsi_which_events for changed events.

Next libiscsi version will introduce async reconnects and zero events
are returned while libiscsi is waiting for a reconnect retry.

Signed-off-by: Peter Lieven <pl@kamp.de>
Message-id: 1428437295-29577-1-git-send-email-pl@kamp.de
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/iscsi.c | 33 +++++++++++++++++++++++++++------
 1 file changed, 27 insertions(+), 6 deletions(-)

diff --git a/block/iscsi.c b/block/iscsi.c
index 3e34b1f..ba33290 100644
--- a/block/iscsi.c
+++ b/block/iscsi.c
@@ -56,6 +56,7 @@ typedef struct IscsiLun {
     uint64_t num_blocks;
     int events;
     QEMUTimer *nop_timer;
+    QEMUTimer *event_timer;
     uint8_t lbpme;
     uint8_t lbprz;
     uint8_t has_write_same;
@@ -95,6 +96,7 @@ typedef struct IscsiAIOCB {
 #endif
 } IscsiAIOCB;
 
+#define EVENT_INTERVAL 250
 #define NOP_INTERVAL 5000
 #define MAX_NOP_FAILURES 3
 #define ISCSI_CMD_RETRIES ARRAY_SIZE(iscsi_retry_times)
@@ -256,21 +258,30 @@ static void
 iscsi_set_events(IscsiLun *iscsilun)
 {
     struct iscsi_context *iscsi = iscsilun->iscsi;
-    int ev;
+    int ev = iscsi_which_events(iscsi);
 
-    /* We always register a read handler.  */
-    ev = POLLIN;
-    ev |= iscsi_which_events(iscsi);
     if (ev != iscsilun->events) {
         aio_set_fd_handler(iscsilun->aio_context,
                            iscsi_get_fd(iscsi),
-                           iscsi_process_read,
+                           (ev & POLLIN) ? iscsi_process_read : NULL,
                            (ev & POLLOUT) ? iscsi_process_write : NULL,
                            iscsilun);
+        iscsilun->events = ev;
+    }
 
+    /* newer versions of libiscsi may return zero events. In this
+     * case start a timer to ensure we are able to return to service
+     * once this situation changes. */
+    if (!ev) {
+        timer_mod(iscsilun->event_timer,
+                  qemu_clock_get_ms(QEMU_CLOCK_REALTIME) + EVENT_INTERVAL);
     }
+}
 
-    iscsilun->events = ev;
+static void iscsi_timed_set_events(void *opaque)
+{
+    IscsiLun *iscsilun = opaque;
+    iscsi_set_events(iscsilun);
 }
 
 static void
@@ -1214,6 +1225,11 @@ static void iscsi_detach_aio_context(BlockDriverState *bs)
         timer_free(iscsilun->nop_timer);
         iscsilun->nop_timer = NULL;
     }
+    if (iscsilun->event_timer) {
+        timer_del(iscsilun->event_timer);
+        timer_free(iscsilun->event_timer);
+        iscsilun->event_timer = NULL;
+    }
 }
 
 static void iscsi_attach_aio_context(BlockDriverState *bs,
@@ -1230,6 +1246,11 @@ static void iscsi_attach_aio_context(BlockDriverState *bs,
                                         iscsi_nop_timed_event, iscsilun);
     timer_mod(iscsilun->nop_timer,
               qemu_clock_get_ms(QEMU_CLOCK_REALTIME) + NOP_INTERVAL);
+
+    /* Prepare a timer for a delayed call to iscsi_set_events */
+    iscsilun->event_timer = aio_timer_new(iscsilun->aio_context,
+                                          QEMU_CLOCK_REALTIME, SCALE_MS,
+                                          iscsi_timed_set_events, iscsilun);
 }
 
 static bool iscsi_is_write_protected(IscsiLun *iscsilun)
-- 
2.1.0

Re: [Qemu-devel] [PULL 0/4] Block patches

[Peter Maydell]

On 9 April 2015 at 10:55, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> The following changes since commit 5a24f20a7208a58fb80d78ca0521bba6f4d7b145:
>
>   Merge remote-tracking branch 'remotes/mjt/tags/pull-trivial-patches-2015-04-04' into staging (2015-04-07 14:33:46 +0100)
>
> are available in the git repository at:
>
>   git://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 05b685fbabb7fdcab72cb42b27db916fd74b2265:
>
>   block/iscsi: handle zero events from iscsi_which_events (2015-04-09 10:31:45 +0100)

Applied, thanks.

-- PMM

[Qemu-devel] [PULL 0/4] Block patches

[Stefan Hajnoczi]

The following changes since commit 14c7d99333e4a474c65bdae6f99aa8837e8078e6:

  Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20160714' into staging (2016-07-14 17:32:53 +0100)

are available in the git repository at:

  git://github.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 325e00006139a30a4c747cf35489ea5bee8da272:

  linux-aio: prevent submitting more than MAX_EVENTS (2016-07-15 12:45:47 +0100)

----------------------------------------------------------------

----------------------------------------------------------------

Cao jin (1):
  aio_ctx_check: follow CODING_STYLE

Paolo Bonzini (1):
  linux-aio: share one LinuxAioState within an AioContext

Roman Pen (1):
  linux-aio: prevent submitting more than MAX_EVENTS

Vladimir Sementsov-Ogievskiy (1):
  spec/parallels: fix a mistake

 async.c                  |  25 +++++++++-
 block/linux-aio.c        |  36 ++++++++------
 block/raw-aio.h          |  68 ---------------------------
 block/raw-posix.c        | 119 ++++++-----------------------------------------
 block/raw-win32.c        |   2 +-
 docs/specs/parallels.txt |   2 +-
 include/block/aio.h      |  13 ++++++
 include/block/raw-aio.h  |  68 +++++++++++++++++++++++++++
 8 files changed, 143 insertions(+), 190 deletions(-)
 delete mode 100644 block/raw-aio.h
 create mode 100644 include/block/raw-aio.h

-- 
2.7.4

Re: [Qemu-devel] [PULL 0/4] Block patches

[Peter Maydell]

On 15 July 2016 at 18:59, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> The following changes since commit 14c7d99333e4a474c65bdae6f99aa8837e8078e6:
>
>   Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20160714' into staging (2016-07-14 17:32:53 +0100)
>
> are available in the git repository at:
>
>   git://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 325e00006139a30a4c747cf35489ea5bee8da272:
>
>   linux-aio: prevent submitting more than MAX_EVENTS (2016-07-15 12:45:47 +0100)
>
> ----------------------------------------------------------------
>
> ----------------------------------------------------------------

I'm afraid this fails to build for w32:

/home/petmay01/linaro/qemu-for-merges/block/win32-aio.c:30:21: fatal
error: raw-aio.h: No such file or directory
 #include "raw-aio.h"
                     ^
compilation terminated.

Looks like that should be "block/raw-aio.h".

thanks
-- PMM

Re: [Qemu-devel] [PULL 0/4] Block patches

[Paolo Bonzini]

On 18/07/2016 12:23, Peter Maydell wrote:
> I'm afraid this fails to build for w32:
> 
> /home/petmay01/linaro/qemu-for-merges/block/win32-aio.c:30:21: fatal
> error: raw-aio.h: No such file or directory
>  #include "raw-aio.h"
>                      ^
> compilation terminated.
> 
> Looks like that should be "block/raw-aio.h".

But that's in the patch:

diff --git a/block/raw-win32.c b/block/raw-win32.c
index 62edb1a..9b813d9 100644
--- a/block/raw-win32.c
+++ b/block/raw-win32.c
@@ -27,7 +27,7 @@
 #include "qemu/timer.h"
 #include "block/block_int.h"
 #include "qemu/module.h"
-#include "raw-aio.h"
+#include "block/raw-aio.h"
 #include "trace.h"
 #include "block/thread-pool.h"
 #include "qemu/iov.h"

Did you get a conflict, or did git screw up some resolution?

Paolo

Re: [Qemu-devel] [PULL 0/4] Block patches

[Peter Maydell]

On 18 July 2016 at 15:57, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>
> On 18/07/2016 12:23, Peter Maydell wrote:
>> I'm afraid this fails to build for w32:
>>
>> /home/petmay01/linaro/qemu-for-merges/block/win32-aio.c:30:21: fatal
>> error: raw-aio.h: No such file or directory
>>  #include "raw-aio.h"
>>                      ^
>> compilation terminated.
>>
>> Looks like that should be "block/raw-aio.h".
>
> But that's in the patch:
>
> diff --git a/block/raw-win32.c b/block/raw-win32.c

This is not the .c file which is mentioned in the above
compiler error message :-)

thanks
-- PMM

Re: [Qemu-devel] [PULL 0/4] Block patches

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 782 bytes --]

On Mon, Jul 18, 2016 at 03:58:29PM +0100, Peter Maydell wrote:
> On 18 July 2016 at 15:57, Paolo Bonzini <pbonzini@redhat.com> wrote:
> >
> >
> > On 18/07/2016 12:23, Peter Maydell wrote:
> >> I'm afraid this fails to build for w32:
> >>
> >> /home/petmay01/linaro/qemu-for-merges/block/win32-aio.c:30:21: fatal
> >> error: raw-aio.h: No such file or directory
> >>  #include "raw-aio.h"
> >>                      ^
> >> compilation terminated.
> >>
> >> Looks like that should be "block/raw-aio.h".
> >
> > But that's in the patch:
> >
> > diff --git a/block/raw-win32.c b/block/raw-win32.c
> 
> This is not the .c file which is mentioned in the above
> compiler error message :-)

I resolved the issue yesterday and sent a new block pull request.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[Qemu-devel] [PULL 0/4] Block patches

[Max Reitz]

The following changes since commit 5d3217340adcb6c4f0e4af5d2b865331eb2ff63d:

  disas: Fix ATTRIBUTE_UNUSED define clash with ALSA headers (2016-07-19 16:40:39 +0100)

are available in the git repository at:

  git://github.com/XanClic/qemu.git tags/pull-block-2016-07-20

for you to fetch changes up to bafea5b7c26dd14895f7be64685a12645a75f4cf:

  block: export LUKS specific data to qemu-img info (2016-07-20 00:34:03 +0200)

----------------------------------------------------------------
Block patches for master

----------------------------------------------------------------
Cao jin (1):
      AioContext: correct comments

Daniel P. Berrange (2):
      crypto: add support for querying parameters for block encryption
      block: export LUKS specific data to qemu-img info

Vladimir Sementsov-Ogievskiy (1):
      qcow2: do not allocate extra memory

 block/crypto.c         | 49 ++++++++++++++++++++++++++++++++
 block/qcow.c           |  2 +-
 block/qcow2.c          |  2 +-
 crypto/block-luks.c    | 67 ++++++++++++++++++++++++++++++++++++++++++++
 crypto/block.c         | 17 +++++++++++
 crypto/blockpriv.h     |  4 +++
 include/block/aio.h    |  2 +-
 include/crypto/block.h | 16 +++++++++++
 qapi/block-core.json   |  6 +++-
 qapi/crypto.json       | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++
 10 files changed, 237 insertions(+), 4 deletions(-)

-- 
2.9.0

Re: [Qemu-devel] [PULL 0/4] Block patches

[Peter Maydell]

On 19 July 2016 at 23:47, Max Reitz <mreitz@redhat.com> wrote:
> The following changes since commit 5d3217340adcb6c4f0e4af5d2b865331eb2ff63d:
>
>   disas: Fix ATTRIBUTE_UNUSED define clash with ALSA headers (2016-07-19 16:40:39 +0100)
>
> are available in the git repository at:
>
>   git://github.com/XanClic/qemu.git tags/pull-block-2016-07-20
>
> for you to fetch changes up to bafea5b7c26dd14895f7be64685a12645a75f4cf:
>
>   block: export LUKS specific data to qemu-img info (2016-07-20 00:34:03 +0200)
>
> ----------------------------------------------------------------
> Block patches for master
>
> ----------------------------------------------------------------

Fails to build on everything:

  GEN   qapi-visit.h
In file included from /Users/pm215/src/qemu-for-merges/qapi-schema.json:9:
/Users/pm215/src/qemu-for-merges/qapi/crypto.json:299: Union
'QCryptoBlockInfo' data missing 'qcow' branch


thanks
-- PMM

Re: [Qemu-devel] [PULL 0/4] Block patches

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1461 bytes --]

On 07/20/2016 10:05 AM, Peter Maydell wrote:
> On 19 July 2016 at 23:47, Max Reitz <mreitz@redhat.com> wrote:
>> The following changes since commit 5d3217340adcb6c4f0e4af5d2b865331eb2ff63d:
>>
>>   disas: Fix ATTRIBUTE_UNUSED define clash with ALSA headers (2016-07-19 16:40:39 +0100)
>>
>> are available in the git repository at:
>>
>>   git://github.com/XanClic/qemu.git tags/pull-block-2016-07-20
>>
>> for you to fetch changes up to bafea5b7c26dd14895f7be64685a12645a75f4cf:
>>
>>   block: export LUKS specific data to qemu-img info (2016-07-20 00:34:03 +0200)
>>
>> ----------------------------------------------------------------
>> Block patches for master
>>
>> ----------------------------------------------------------------
> 
> Fails to build on everything:
> 
>   GEN   qapi-visit.h
> In file included from /Users/pm215/src/qemu-for-merges/qapi-schema.json:9:
> /Users/pm215/src/qemu-for-merges/qapi/crypto.json:299: Union
> 'QCryptoBlockInfo' data missing 'qcow' branch

Aha. Cause is two branches developed in parallel; commit d0b18239 forces
all branches of a flat union to be listed (to avoid an abort() if the
user passes a branch that was not listed); solution is to expand the
crypto.json addition to cover all branches, even if it means an empty
type for the branches that have no additional data.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PULL 0/4] Block patches

[Peter Maydell]

On 20 July 2016 at 22:16, Eric Blake <eblake@redhat.com> wrote:
> On 07/20/2016 10:05 AM, Peter Maydell wrote:
>> On 19 July 2016 at 23:47, Max Reitz <mreitz@redhat.com> wrote:
>>> The following changes since commit 5d3217340adcb6c4f0e4af5d2b865331eb2ff63d:
>>>
>>>   disas: Fix ATTRIBUTE_UNUSED define clash with ALSA headers (2016-07-19 16:40:39 +0100)
>>>
>>> are available in the git repository at:
>>>
>>>   git://github.com/XanClic/qemu.git tags/pull-block-2016-07-20
>>>
>>> for you to fetch changes up to bafea5b7c26dd14895f7be64685a12645a75f4cf:
>>>
>>>   block: export LUKS specific data to qemu-img info (2016-07-20 00:34:03 +0200)
>>>
>>> ----------------------------------------------------------------
>>> Block patches for master
>>>
>>> ----------------------------------------------------------------
>>
>> Fails to build on everything:
>>
>>   GEN   qapi-visit.h
>> In file included from /Users/pm215/src/qemu-for-merges/qapi-schema.json:9:
>> /Users/pm215/src/qemu-for-merges/qapi/crypto.json:299: Union
>> 'QCryptoBlockInfo' data missing 'qcow' branch
>
> Aha. Cause is two branches developed in parallel; commit d0b18239 forces
> all branches of a flat union to be listed (to avoid an abort() if the
> user passes a branch that was not listed); solution is to expand the
> crypto.json addition to cover all branches, even if it means an empty
> type for the branches that have no additional data.

I'm just processing the last other outstanding pullreq now,
so unless a respin of this arrives by tomorrow lunchtime UK
time it's going to miss rc0, I think.

thanks
-- PMM

Re: [Qemu-devel] [PULL 0/4] Block patches

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 1929 bytes --]

On 21.07.2016 21:14, Peter Maydell wrote:
> On 20 July 2016 at 22:16, Eric Blake <eblake@redhat.com> wrote:
>> On 07/20/2016 10:05 AM, Peter Maydell wrote:
>>> On 19 July 2016 at 23:47, Max Reitz <mreitz@redhat.com> wrote:
>>>> The following changes since commit 5d3217340adcb6c4f0e4af5d2b865331eb2ff63d:
>>>>
>>>>   disas: Fix ATTRIBUTE_UNUSED define clash with ALSA headers (2016-07-19 16:40:39 +0100)
>>>>
>>>> are available in the git repository at:
>>>>
>>>>   git://github.com/XanClic/qemu.git tags/pull-block-2016-07-20
>>>>
>>>> for you to fetch changes up to bafea5b7c26dd14895f7be64685a12645a75f4cf:
>>>>
>>>>   block: export LUKS specific data to qemu-img info (2016-07-20 00:34:03 +0200)
>>>>
>>>> ----------------------------------------------------------------
>>>> Block patches for master
>>>>
>>>> ----------------------------------------------------------------
>>>
>>> Fails to build on everything:
>>>
>>>   GEN   qapi-visit.h
>>> In file included from /Users/pm215/src/qemu-for-merges/qapi-schema.json:9:
>>> /Users/pm215/src/qemu-for-merges/qapi/crypto.json:299: Union
>>> 'QCryptoBlockInfo' data missing 'qcow' branch
>>
>> Aha. Cause is two branches developed in parallel; commit d0b18239 forces
>> all branches of a flat union to be listed (to avoid an abort() if the
>> user passes a branch that was not listed); solution is to expand the
>> crypto.json addition to cover all branches, even if it means an empty
>> type for the branches that have no additional data.
> 
> I'm just processing the last other outstanding pullreq now,
> so unless a respin of this arrives by tomorrow lunchtime UK
> time it's going to miss rc0, I think.

Since there's nothing critical in this pull request, I'll drop the
crypto patches and keep the non-offending patches for the next pull
request (for rc1).

So no need to wait for another pull request from me for rc0.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 498 bytes --]

[Qemu-devel] [PULL 0/4] Block patches

[Stefan Hajnoczi]

The following changes since commit d992f2f1368ceb92e6bfd8efece174110f4236ff:

  Merge remote-tracking branch 'remotes/artyom/tags/pull-sun4v-20170226' into staging (2017-02-26 22:40:23 +0000)

are available in the git repository at:

  git://github.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 1ab17f9f5c63c2798d707aeb22588e4fcc17b2cd:

  tests-aio-multithread: use atomic_read properly (2017-02-27 14:00:53 +0000)

----------------------------------------------------------------

----------------------------------------------------------------

Paolo Bonzini (4):
  curl: do not use aio_context_acquire/release
  nfs: do not use aio_context_acquire/release
  iscsi: do not use aio_context_acquire/release
  tests-aio-multithread: use atomic_read properly

 block/curl.c                 | 24 ++++++++-----
 block/iscsi.c                | 83 ++++++++++++++++++++++++++++++++++----------
 block/nfs.c                  | 23 +++++++++---
 tests/test-aio-multithread.c |  4 +--
 4 files changed, 100 insertions(+), 34 deletions(-)

-- 
2.9.3

Re: [Qemu-devel] [PULL 0/4] Block patches

[Peter Maydell]

On 27 February 2017 at 16:34, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> The following changes since commit d992f2f1368ceb92e6bfd8efece174110f4236ff:
>
>   Merge remote-tracking branch 'remotes/artyom/tags/pull-sun4v-20170226' into staging (2017-02-26 22:40:23 +0000)
>
> are available in the git repository at:
>
>   git://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 1ab17f9f5c63c2798d707aeb22588e4fcc17b2cd:
>
>   tests-aio-multithread: use atomic_read properly (2017-02-27 14:00:53 +0000)
>
> ----------------------------------------------------------------
>
> ----------------------------------------------------------------
>
> Paolo Bonzini (4):
>   curl: do not use aio_context_acquire/release
>   nfs: do not use aio_context_acquire/release
>   iscsi: do not use aio_context_acquire/release
>   tests-aio-multithread: use atomic_read properly

Applied, thanks.

-- PMM

[Qemu-devel] [PULL 0/4] Block patches

[Stefan Hajnoczi]

The following changes since commit 2b483739791b33c46e6084b51edcf62107058ae1:

  Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20170904-2' into staging (2017-09-04 17:21:24 +0100)

are available in the git repository at:

  git://github.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to b461151ff31c7925f271c297e8abed20231ac7d3:

  block: document semantics of bdrv_co_preadv|pwritev (2017-09-05 11:07:02 +0100)

----------------------------------------------------------------

----------------------------------------------------------------

Daniel P. Berrange (1):
  block: document semantics of bdrv_co_preadv|pwritev

Stefan Hajnoczi (3):
  qemu.py: make VM() a context manager
  iotests.py: add FilePath context manager
  qemu-iotests: use context managers for resource cleanup in 194

 include/block/block_int.h     |  31 +++++++++++
 scripts/qemu.py               |  16 +++++-
 tests/qemu-iotests/194        | 117 +++++++++++++++++++++---------------------
 tests/qemu-iotests/iotests.py |  26 ++++++++++
 4 files changed, 130 insertions(+), 60 deletions(-)

-- 
2.13.5

Re: [Qemu-devel] [PULL 0/4] Block patches

[Peter Maydell]

On 5 September 2017 at 14:29, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> The following changes since commit 2b483739791b33c46e6084b51edcf62107058ae1:
>
>   Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20170904-2' into staging (2017-09-04 17:21:24 +0100)
>
> are available in the git repository at:
>
>   git://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to b461151ff31c7925f271c297e8abed20231ac7d3:
>
>   block: document semantics of bdrv_co_preadv|pwritev (2017-09-05 11:07:02 +0100)
>
> ----------------------------------------------------------------
>
> ----------------------------------------------------------------
>
> Daniel P. Berrange (1):
>   block: document semantics of bdrv_co_preadv|pwritev
>
> Stefan Hajnoczi (3):
>   qemu.py: make VM() a context manager
>   iotests.py: add FilePath context manager
>   qemu-iotests: use context managers for resource cleanup in 194
>
>  include/block/block_int.h     |  31 +++++++++++
>  scripts/qemu.py               |  16 +++++-
>  tests/qemu-iotests/194        | 117 +++++++++++++++++++++---------------------
>  tests/qemu-iotests/iotests.py |  26 ++++++++++
>  4 files changed, 130 insertions(+), 60 deletions(-)

Applied, thanks.

-- PMM

[Qemu-devel] [PULL 0/4] Block patches

[Stefan Hajnoczi]

The following changes since commit f58d9620aa4a514b1227074ff56eefd1334a6225:

  Merge remote-tracking branch 'remotes/rth/tags/pull-dt-20180326' into staging (2018-03-27 10:27:34 +0100)

are available in the Git repository at:

  git://github.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to f5a53faad4bfbf1b86012a13055d2a1a774a42b6:

  MAINTAINERS: add include/block/aio-wait.h (2018-03-27 13:05:48 +0100)

----------------------------------------------------------------

----------------------------------------------------------------

Stefan Hajnoczi (4):
  queue: add QSIMPLEQ_PREPEND()
  coroutine: avoid co_queue_wakeup recursion
  coroutine: add test-aio coroutine queue chaining test case
  MAINTAINERS: add include/block/aio-wait.h

 MAINTAINERS                  |   1 +
 include/qemu/coroutine_int.h |   1 -
 include/qemu/queue.h         |   8 ++++
 block/io.c                   |   3 +-
 tests/test-aio.c             |  65 ++++++++++++++++++++-----
 util/qemu-coroutine-lock.c   |  34 -------------
 util/qemu-coroutine.c        | 110 +++++++++++++++++++++++--------------------
 7 files changed, 121 insertions(+), 101 deletions(-)

-- 
2.14.3

Re: [Qemu-devel] [PULL 0/4] Block patches

[Peter Maydell]

On 27 March 2018 at 15:41, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> The following changes since commit f58d9620aa4a514b1227074ff56eefd1334a6225:
>
>   Merge remote-tracking branch 'remotes/rth/tags/pull-dt-20180326' into staging (2018-03-27 10:27:34 +0100)
>
> are available in the Git repository at:
>
>   git://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to f5a53faad4bfbf1b86012a13055d2a1a774a42b6:
>
>   MAINTAINERS: add include/block/aio-wait.h (2018-03-27 13:05:48 +0100)
>
> ----------------------------------------------------------------
>
> ----------------------------------------------------------------
>
> Stefan Hajnoczi (4):
>   queue: add QSIMPLEQ_PREPEND()
>   coroutine: avoid co_queue_wakeup recursion
>   coroutine: add test-aio coroutine queue chaining test case
>   MAINTAINERS: add include/block/aio-wait.h
>
>  MAINTAINERS                  |   1 +
>  include/qemu/coroutine_int.h |   1 -
>  include/qemu/queue.h         |   8 ++++
>  block/io.c                   |   3 +-
>  tests/test-aio.c             |  65 ++++++++++++++++++++-----
>  util/qemu-coroutine-lock.c   |  34 -------------
>  util/qemu-coroutine.c        | 110 +++++++++++++++++++++++--------------------
>  7 files changed, 121 insertions(+), 101 deletions(-)
>

Applied, thanks.

-- PMM

Re: [Qemu-devel] [PULL 0/4] Block patches

[no-reply]

Hi,

This series failed docker-build@min-glib build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

Type: series
Message-id: 20180327144124.32533-1-stefanha@redhat.com
Subject: [Qemu-devel] [PULL 0/4] Block patches

=== TEST SCRIPT BEGIN ===
#!/bin/bash
set -e
git submodule update --init dtc
# Let docker tests dump environment info
export SHOW_ENV=1
export J=8
time make docker-test-build@min-glib
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
58d76f2171 queue: add QSIMPLEQ_PREPEND()

=== OUTPUT BEGIN ===
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-6awfngpj/src/dtc'...
Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42'
  BUILD   min-glib
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-6awfngpj/src'
  GEN     /var/tmp/patchew-tester-tmp-6awfngpj/src/docker-src.2018-03-31-03.11.27.1848/qemu.tar
Cloning into '/var/tmp/patchew-tester-tmp-6awfngpj/src/docker-src.2018-03-31-03.11.27.1848/qemu.tar.vroot'...
done.
Checking out files:  33% (2006/6066)   
Checking out files:  34% (2063/6066)   
Checking out files:  35% (2124/6066)   
Checking out files:  36% (2184/6066)   
Checking out files:  37% (2245/6066)   
Checking out files:  38% (2306/6066)   
Checking out files:  39% (2366/6066)   
Checking out files:  40% (2427/6066)   
Checking out files:  41% (2488/6066)   
Checking out files:  42% (2548/6066)   
Checking out files:  43% (2609/6066)   
Checking out files:  44% (2670/6066)   
Checking out files:  45% (2730/6066)   
Checking out files:  46% (2791/6066)   
Checking out files:  47% (2852/6066)   
Checking out files:  48% (2912/6066)   
Checking out files:  49% (2973/6066)   
Checking out files:  50% (3033/6066)   
Checking out files:  51% (3094/6066)   
Checking out files:  52% (3155/6066)   
Checking out files:  53% (3215/6066)   
Checking out files:  54% (3276/6066)   
Checking out files:  55% (3337/6066)   
Checking out files:  56% (3397/6066)   
Checking out files:  57% (3458/6066)   
Checking out files:  58% (3519/6066)   
Checking out files:  59% (3579/6066)   
Checking out files:  59% (3623/6066)   
Checking out files:  60% (3640/6066)   
Checking out files:  61% (3701/6066)   
Checking out files:  62% (3761/6066)   
Checking out files:  63% (3822/6066)   
Checking out files:  64% (3883/6066)   
Checking out files:  65% (3943/6066)   
Checking out files:  66% (4004/6066)   
Checking out files:  67% (4065/6066)   
Checking out files:  68% (4125/6066)   
Checking out files:  69% (4186/6066)   
Checking out files:  70% (4247/6066)   
Checking out files:  71% (4307/6066)   
Checking out files:  72% (4368/6066)   
Checking out files:  73% (4429/6066)   
Checking out files:  74% (4489/6066)   
Checking out files:  75% (4550/6066)   
Checking out files:  76% (4611/6066)   
Checking out files:  77% (4671/6066)   
Checking out files:  78% (4732/6066)   
Checking out files:  79% (4793/6066)   
Checking out files:  80% (4853/6066)   
Checking out files:  81% (4914/6066)   
Checking out files:  82% (4975/6066)   
Checking out files:  83% (5035/6066)   
Checking out files:  84% (5096/6066)   
Checking out files:  85% (5157/6066)   
Checking out files:  86% (5217/6066)   
Checking out files:  87% (5278/6066)   
Checking out files:  88% (5339/6066)   
Checking out files:  89% (5399/6066)   
Checking out files:  90% (5460/6066)   
Checking out files:  90% (5517/6066)   
Checking out files:  91% (5521/6066)   
Checking out files:  92% (5581/6066)   
Checking out files:  93% (5642/6066)   
Checking out files:  94% (5703/6066)   
Checking out files:  94% (5725/6066)   
Checking out files:  95% (5763/6066)   
Checking out files:  96% (5824/6066)   
Checking out files:  97% (5885/6066)   
Checking out files:  98% (5945/6066)   
Checking out files:  98% (5988/6066)   
Checking out files:  99% (6006/6066)   
Checking out files: 100% (6066/6066)   
Checking out files: 100% (6066/6066), done.
Your branch is up-to-date with 'origin/test'.
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-6awfngpj/src/docker-src.2018-03-31-03.11.27.1848/qemu.tar.vroot/dtc'...
Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42'
Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Cloning into '/var/tmp/patchew-tester-tmp-6awfngpj/src/docker-src.2018-03-31-03.11.27.1848/qemu.tar.vroot/ui/keycodemapdb'...
Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce'
tar: /var/tmp/patchew-tester-tmp-6awfngpj/src/docker-src.2018-03-31-03.11.27.1848/qemu.tar: Wrote only 2048 of 10240 bytes
tar: Error is not recoverable: exiting now
failed to create tar file
  COPY    RUNNER
    RUN test-build in qemu:min-glib 
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
/var/tmp/qemu/run: line 32: prep_fail: command not found
Environment variables:
HOSTNAME=7ec65e6e427d
MAKEFLAGS= -j8
J=8
CCACHE_DIR=/var/tmp/ccache
EXTRA_CONFIGURE_OPTS=
V=
SHOW_ENV=1
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
TARGET_LIST=
SHLVL=1
HOME=/root
TEST_DIR=/tmp/qemu-test
FEATURES= dtc
DEBUG=
_=/usr/bin/env

Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/install

ERROR: DTC (libfdt) version >= 1.4.2 not present.
       Please install the DTC (libfdt) devel package

Traceback (most recent call last):
  File "./tests/docker/docker.py", line 407, in <module>
    sys.exit(main())
  File "./tests/docker/docker.py", line 404, in main
    return args.cmdobj.run(args, argv)
  File "./tests/docker/docker.py", line 261, in run
    return Docker().run(argv, args.keep, quiet=args.quiet)
  File "./tests/docker/docker.py", line 229, in run
    quiet=quiet)
  File "./tests/docker/docker.py", line 147, in _do_check
    return subprocess.check_call(self._command + cmd, **kwargs)
  File "/usr/lib64/python2.7/subprocess.py", line 186, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker', 'run', '--label', 'com.qemu.instance.uuid=c90672f634b211e88e0f52540069c830', '-u', '0', '--security-opt', 'seccomp=unconfined', '--rm', '--net=none', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=8', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/root/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-6awfngpj/src/docker-src.2018-03-31-03.11.27.1848:/var/tmp/qemu:z,ro', 'qemu:min-glib', '/var/tmp/qemu/run', 'test-build']' returned non-zero exit status 1
make[1]: *** [tests/docker/Makefile.include:129: docker-run] Error 1
make[1]: Leaving directory '/var/tmp/patchew-tester-tmp-6awfngpj/src'
make: *** [tests/docker/Makefile.include:163: docker-run-test-build@min-glib] Error 2

real	0m51.126s
user	0m9.121s
sys	0m6.557s
=== OUTPUT END ===

Test command exited with code: 2


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

[Qemu-devel] [PULL 0/4] Block patches

[Jeff Cody]

The following changes since commit d01beac177d44491d7db8747b79d94e1b53d173b:

  Merge remote-tracking branch 'remotes/kraxel/tags/vga-20180507-pull-request' into staging (2018-05-08 14:23:02 +0100)

are available in the git repository at:

  git://github.com/codyprime/qemu-kvm-jtc.git tags/block-pull-request

for you to fetch changes up to a2cb9239b7610ffb00f9ced5cd7640d40b0e1ccf:

  sheepdog: Fix sd_co_create_opts() memory leaks (2018-05-08 10:47:27 -0400)

----------------------------------------------------------------
Block patches
----------------------------------------------------------------

Kevin Wolf (1):
  sheepdog: Fix sd_co_create_opts() memory leaks

Max Reitz (2):
  block/mirror: Make cancel always cancel pre-READY
  iotests: Add test for cancelling a mirror job

Stefan Hajnoczi (1):
  block/mirror: honor ratelimit again

 block/mirror.c             |  14 +++--
 block/sheepdog.c           |   4 +-
 tests/qemu-iotests/185.out |   4 +-
 tests/qemu-iotests/218     | 138 +++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/218.out |  30 ++++++++++
 tests/qemu-iotests/group   |   1 +
 6 files changed, 183 insertions(+), 8 deletions(-)
 create mode 100644 tests/qemu-iotests/218
 create mode 100644 tests/qemu-iotests/218.out

-- 
2.13.6

Re: [Qemu-devel] [PULL 0/4] Block patches

[Peter Maydell]

On 8 May 2018 at 16:19, Jeff Cody <jcody@redhat.com> wrote:
> The following changes since commit d01beac177d44491d7db8747b79d94e1b53d173b:
>
>   Merge remote-tracking branch 'remotes/kraxel/tags/vga-20180507-pull-request' into staging (2018-05-08 14:23:02 +0100)
>
> are available in the git repository at:
>
>   git://github.com/codyprime/qemu-kvm-jtc.git tags/block-pull-request
>
> for you to fetch changes up to a2cb9239b7610ffb00f9ced5cd7640d40b0e1ccf:
>
>   sheepdog: Fix sd_co_create_opts() memory leaks (2018-05-08 10:47:27 -0400)
>
> ----------------------------------------------------------------
> Block patches
> ----------------------------------------------------------------
>

Applied, thanks.

-- PMM

[Qemu-devel] [PULL 0/4] Block patches

[Jeff Cody]

The following changes since commit c416eecea5f3aea863ab8fda5a36a24157b8f704:

  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging (2018-05-15 17:02:00 +0100)

are available in the git repository at:

  git://github.com/codyprime/qemu-kvm-jtc.git tags/block-pull-request

for you to fetch changes up to c82be42cc803b36fd7aed5dceec68312c7056fd5:

  nfs: Remove processed options from QDict (2018-05-16 13:37:47 -0400)

----------------------------------------------------------------
Block patches
----------------------------------------------------------------

Kevin Wolf (2):
  nfs: Fix error path in nfs_options_qdict_to_qapi()
  nfs: Remove processed options from QDict

Stefan Hajnoczi (2):
  qemu-iotests: reduce chance of races in 185
  blockjob: do not cancel timer in resume

 block/nfs.c                |  8 ++++++++
 blockjob.c                 | 22 +++++++++++++++-------
 tests/qemu-iotests/185     | 17 +++++++++++++----
 tests/qemu-iotests/185.out | 12 +++++-------
 4 files changed, 41 insertions(+), 18 deletions(-)

-- 
2.13.6

Re: [Qemu-devel] [PULL 0/4] Block patches

[Peter Maydell]

On 16 May 2018 at 18:42, Jeff Cody <jcody@redhat.com> wrote:
> The following changes since commit c416eecea5f3aea863ab8fda5a36a24157b8f704:
>
>   Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging (2018-05-15 17:02:00 +0100)
>
> are available in the git repository at:
>
>   git://github.com/codyprime/qemu-kvm-jtc.git tags/block-pull-request
>
> for you to fetch changes up to c82be42cc803b36fd7aed5dceec68312c7056fd5:
>
>   nfs: Remove processed options from QDict (2018-05-16 13:37:47 -0400)
>
> ----------------------------------------------------------------
> Block patches
> ----------------------------------------------------------------
>
> Kevin Wolf (2):
>   nfs: Fix error path in nfs_options_qdict_to_qapi()
>   nfs: Remove processed options from QDict
>
> Stefan Hajnoczi (2):
>   qemu-iotests: reduce chance of races in 185
>   blockjob: do not cancel timer in resume

Applied, thanks.

-- PMM

[Qemu-devel] [PULL 0/4] Block patches

[Jeff Cody]

The following changes since commit 19b599f7664b2ebfd0f405fb79c14dd241557452:

  Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2018-08-27-v2' into staging (2018-08-27 16:44:20 +0100)

are available in the Git repository at:

  git://github.com/codyprime/qemu-kvm-jtc.git tags/block-pull-request

for you to fetch changes up to 8af2eb7b43a1b694fd6d1d090025027d6b72caac:

  block/rbd: add deprecation documentation for filename keyvalue pairs (2018-09-12 08:51:45 -0400)

----------------------------------------------------------------
Block patches for RBD
----------------------------------------------------------------

Jeff Cody (4):
  block/rbd: pull out qemu_rbd_convert_options
  block/rbd: Attempt to parse legacy filenames
  block/rbd: add iotest for rbd legacy keyvalue filename parsing
  block/rbd: add deprecation documentation for filename keyvalue pairs

 block/rbd.c                | 89 ++++++++++++++++++++++++++++++++------
 qemu-deprecated.texi       | 15 +++++++
 tests/qemu-iotests/231     | 62 ++++++++++++++++++++++++++
 tests/qemu-iotests/231.out |  9 ++++
 tests/qemu-iotests/group   |  1 +
 5 files changed, 162 insertions(+), 14 deletions(-)
 create mode 100755 tests/qemu-iotests/231
 create mode 100644 tests/qemu-iotests/231.out

-- 
2.17.1

[Qemu-devel] [PULL 0/4] Block patches

[Stefan Hajnoczi]

The following changes since commit 20d6c7312f1b812bb9c750f4087f69ac8485cc90:

  Merge remote-tracking branch 'remotes/palmer/tags/riscv-for-master-3.2-part1' into staging (2019-01-03 13:26:30 +0000)

are available in the Git repository at:

  git://github.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 39a0408e768cd00142f5b57d27ab234282bf4df5:

  dmg: don't skip zero chunk (2019-01-04 11:15:09 +0000)

----------------------------------------------------------------
Pull request

Bug fixes for the .dmg image file format.

----------------------------------------------------------------

Julio Faracco (1):
  dmg: Fixing wrong dmg block type value for block terminator.

yuchenlin (3):
  dmg: fix binary search
  dmg: use enumeration type instead of hard coding number
  dmg: don't skip zero chunk

 block/dmg.c | 31 ++++++++++++++++++++-----------
 1 file changed, 20 insertions(+), 11 deletions(-)

-- 
2.20.1

Re: [Qemu-devel] [PULL 0/4] Block patches

[Peter Maydell]

On Fri, 4 Jan 2019 at 11:21, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> The following changes since commit 20d6c7312f1b812bb9c750f4087f69ac8485cc90:
>
>   Merge remote-tracking branch 'remotes/palmer/tags/riscv-for-master-3.2-part1' into staging (2019-01-03 13:26:30 +0000)
>
> are available in the Git repository at:
>
>   git://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 39a0408e768cd00142f5b57d27ab234282bf4df5:
>
>   dmg: don't skip zero chunk (2019-01-04 11:15:09 +0000)
>
> ----------------------------------------------------------------
> Pull request
>
> Bug fixes for the .dmg image file format.
>
> ----------------------------------------------------------------

Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/4.0
for any user-visible changes.

-- PMM