[Qemu-devel] [PATCH v2 for-4.0 00/13] block: byte-based blocking read/write

[Qemu-devel] [PATCH v2 for-4.0 00/13] block: byte-based blocking read/write

[Eric Blake]

Based-on: <20181114210548.1098207-1-eblake@redhat.com>
[file-posix: Better checks of 64-bit copy_range]
Based-on: <20181101182738.70462-1-vsementsov@virtuozzo.com>
[0/7 qcow2 decompress in threads] - more specifically, on Kevin's block-next branch

Also available at
https://repo.or.cz/qemu/ericb.git/shortlog/refs/tags/block-byte-blocking-v2

This series is a logical followup to other byte-based cleanups I have
done. The only remaining mention of sectors in public block.h APIs after
this series are in bdrv_nb_sectors() (converting those callser to use
bdrv_getlength() not only requires more work, but would be our
opportunity to see if we can quit rounding block sizes up beyond
request_align sizes for protocols that support sub-sector sizes), and
for computing geometries (where sectors actually make sense).

v1 was: https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg04686.html
Since then:

- fold in a straggler patch that dropped from my v8 qcow2 compression
series for 3.1
- improve commit message of 4/13, to show what auditing I performed [Berto]
- based on that audit, add a lot more patches to change bl.max_transfer
to be guaranteed non-zero and allow > 2G values where safe (note that
the block layer still fragments < 2G, so a larger advertisement doesn't
actually cause large read/write transactions)

001/13:[0005] [FC] 'qcow2: Prefer byte-based calls into bs->file'
002/13:[----] [--] 'vdi: Switch to byte-based calls'
003/13:[----] [--] 'vvfat: Switch to byte-based calls'
004/13:[----] [--] 'block: Removed unused sector-based blocking I/O'
005/13:[down] 'block: Switch to 64-bit bl.max_transfer'
006/13:[down] 'blkdebug: Audit for read/write 64-bit cleanness'
007/13:[down] 'blklogwrites: Audit for read/write 64-bit cleanness'
008/13:[down] 'crypto: Audit for read/write 64-bit cleanness'
009/13:[down] 'RFC: crypto: Rely on block layer for fragmentation'
010/13:[down] 'file-posix: Audit for read/write 64-bit cleanness'
011/13:[down] 'qcow2: Audit for read/write 64-bit cleanness'
012/13:[down] 'block: Document need for audit of read/write 64-bit cleanness'
013/13:[down] 'block: Enforce non-zero bl.max_transfer'

Patch 9/13 is marked RFC; if we like it, I'd rather squash 8 and 9
into a single patch; if we don't like it, it can be dropped without
affecting the rest of the series.

Eric Blake (13):
  qcow2: Prefer byte-based calls into bs->file
  vdi: Switch to byte-based calls
  vvfat: Switch to byte-based calls
  block: Removed unused sector-based blocking I/O
  block: Switch to 64-bit bl.max_transfer
  blkdebug: Audit for read/write 64-bit cleanness
  blklogwrites: Audit for read/write 64-bit cleanness
  crypto: Audit for read/write 64-bit cleanness
  RFC: crypto: Rely on block layer for fragmentation
  file-posix: Audit for read/write 64-bit cleanness
  qcow2: Audit for read/write 64-bit cleanness
  block: Document need for audit of read/write 64-bit cleanness
  block: Enforce non-zero bl.max_transfer

 qapi/block-core.json      |  2 +-
 include/block/block.h     |  4 --
 include/block/block_int.h | 10 +++--
 block/io.c                | 68 +++++++++++----------------------
 block/blkdebug.c          | 17 +++------
 block/blklogwrites.c      |  1 +
 block/bochs.c             |  1 +
 block/cloop.c             |  1 +
 block/crypto.c            | 79 +++++++++++++++------------------------
 block/dmg.c               |  1 +
 block/file-posix.c        |  3 ++
 block/file-win32.c        |  2 +
 block/qcow.c              |  1 +
 block/qcow2-refcount.c    |  6 +--
 block/qcow2.c             |  1 +
 block/rbd.c               |  1 +
 block/vdi.c               | 14 +++----
 block/vvfat.c             | 21 ++++++-----
 block/vxhs.c              |  1 +
 19 files changed, 98 insertions(+), 136 deletions(-)

-- 
2.17.2

[Qemu-devel] [PATCH v2 01/13] qcow2: Prefer byte-based calls into bs->file

[Eric Blake]

We had only a few sector-based stragglers left; convert them to use
our preferred byte-based accesses.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>

---
v2: rebased to threaded decompression handling
[moved from a different series]
v5: commit message tweak
v2: indentation fix
---
 block/qcow2-refcount.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
index 1c63ac244ac..3ef53c0b2c6 100644
--- a/block/qcow2-refcount.c
+++ b/block/qcow2-refcount.c
@@ -2374,8 +2374,8 @@ write_refblocks:
         on_disk_refblock = (void *)((char *) *refcount_table +
                                     refblock_index * s->cluster_size);

-        ret = bdrv_write(bs->file, refblock_offset / BDRV_SECTOR_SIZE,
-                         on_disk_refblock, s->cluster_sectors);
+        ret = bdrv_pwrite(bs->file, refblock_offset, on_disk_refblock,
+                          s->cluster_size);
         if (ret < 0) {
             fprintf(stderr, "ERROR writing refblock: %s\n", strerror(-ret));
             goto fail;
@@ -2597,7 +2597,7 @@ fail:
  * - 0 if writing to this offset will not affect the mentioned metadata
  * - a positive QCow2MetadataOverlap value indicating one overlapping section
  * - a negative value (-errno) indicating an error while performing a check,
- *   e.g. when bdrv_read failed on QCOW2_OL_INACTIVE_L2
+ *   e.g. when bdrv_pread failed on QCOW2_OL_INACTIVE_L2
  */
 int qcow2_check_metadata_overlap(BlockDriverState *bs, int ign, int64_t offset,
                                  int64_t size)
-- 
2.17.2

[Qemu-devel] [PATCH v2 02/13] vdi: Switch to byte-based calls

[Eric Blake]

We are gradually moving away from sector-based interfaces, towards
byte-based.  Make the change for the last few sector-based calls
into the block layer from the vdi driver.

Ideally, the vdi driver should switch to doing everything
byte-based, but that's a more invasive change that requires a
bit more auditing.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
---
 block/vdi.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/block/vdi.c b/block/vdi.c
index 2380daa583e..b5d699f62c2 100644
--- a/block/vdi.c
+++ b/block/vdi.c
@@ -377,7 +377,7 @@ static int vdi_open(BlockDriverState *bs, QDict *options, int flags,

     logout("\n");

-    ret = bdrv_read(bs->file, 0, (uint8_t *)&header, 1);
+    ret = bdrv_pread(bs->file, 0, &header, sizeof(header));
     if (ret < 0) {
         goto fail;
     }
@@ -467,15 +467,14 @@ static int vdi_open(BlockDriverState *bs, QDict *options, int flags,
     s->header = header;

     bmap_size = header.blocks_in_image * sizeof(uint32_t);
-    bmap_size = DIV_ROUND_UP(bmap_size, SECTOR_SIZE);
-    s->bmap = qemu_try_blockalign(bs->file->bs, bmap_size * SECTOR_SIZE);
+    s->bmap = qemu_try_blockalign(bs->file->bs, bmap_size);
     if (s->bmap == NULL) {
         ret = -ENOMEM;
         goto fail;
     }

-    ret = bdrv_read(bs->file, s->bmap_sector, (uint8_t *)s->bmap,
-                    bmap_size);
+    ret = bdrv_pread(bs->file, s->bmap_sector * SECTOR_SIZE, s->bmap,
+                     bmap_size);
     if (ret < 0) {
         goto fail_free_bmap;
     }
@@ -694,7 +693,7 @@ nonallocating_write:
         assert(VDI_IS_ALLOCATED(bmap_first));
         *header = s->header;
         vdi_header_to_le(header);
-        ret = bdrv_write(bs->file, 0, block, 1);
+        ret = bdrv_pwrite(bs->file, 0, block, SECTOR_SIZE);
         g_free(block);
         block = NULL;

@@ -712,7 +711,8 @@ nonallocating_write:
         base = ((uint8_t *)&s->bmap[0]) + bmap_first * SECTOR_SIZE;
         logout("will write %u block map sectors starting from entry %u\n",
                n_sectors, bmap_first);
-        ret = bdrv_write(bs->file, offset, base, n_sectors);
+        ret = bdrv_pwrite(bs->file, offset * SECTOR_SIZE, base,
+                          n_sectors * SECTOR_SIZE);
     }

     return ret;
-- 
2.17.2

[Qemu-devel] [PATCH v2 03/13] vvfat: Switch to byte-based calls

[Eric Blake]

We are gradually moving away from sector-based interfaces, towards
byte-based.  Make the change for the last few sector-based calls
into the block layer from the vvfat driver.

Ideally, the vvfat driver should switch to doing everything
byte-based, but that's a more invasive change that requires a
bit more auditing.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
---
 block/vvfat.c | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/block/vvfat.c b/block/vvfat.c
index b7b61ea8b78..915a05ae4f2 100644
--- a/block/vvfat.c
+++ b/block/vvfat.c
@@ -1494,8 +1494,8 @@ static int vvfat_read(BlockDriverState *bs, int64_t sector_num,
                 DLOG(fprintf(stderr, "sectors %" PRId64 "+%" PRId64
                              " allocated\n", sector_num,
                              n >> BDRV_SECTOR_BITS));
-                if (bdrv_read(s->qcow, sector_num, buf + i * 0x200,
-                              n >> BDRV_SECTOR_BITS)) {
+                if (bdrv_pread(s->qcow, sector_num * BDRV_SECTOR_BITS,
+                               buf + i * 0x200, n) < 0) {
                     return -1;
                 }
                 i += (n >> BDRV_SECTOR_BITS) - 1;
@@ -1983,7 +1983,8 @@ static uint32_t get_cluster_count_for_direntry(BDRVVVFATState* s,
                         if (res) {
                             return -1;
                         }
-                        res = bdrv_write(s->qcow, offset, s->cluster_buffer, 1);
+                        res = bdrv_pwrite(s->qcow, offset * BDRV_SECTOR_SIZE,
+                                          s->cluster_buffer, BDRV_SECTOR_SIZE);
                         if (res) {
                             return -2;
                         }
@@ -2146,7 +2147,7 @@ DLOG(checkpoint());
      * - get modified FAT
      * - compare the two FATs (TODO)
      * - get buffer for marking used clusters
-     * - recurse direntries from root (using bs->bdrv_read to make
+     * - recurse direntries from root (using bs->bdrv_pread to make
      *    sure to get the new data)
      *   - check that the FAT agrees with the size
      *   - count the number of clusters occupied by this directory and
@@ -2911,9 +2912,9 @@ static int handle_deletes(BDRVVVFATState* s)
 /*
  * synchronize mapping with new state:
  *
- * - copy FAT (with bdrv_read)
+ * - copy FAT (with bdrv_pread)
  * - mark all filenames corresponding to mappings as deleted
- * - recurse direntries from root (using bs->bdrv_read)
+ * - recurse direntries from root (using bs->bdrv_pread)
  * - delete files corresponding to mappings marked as deleted
  */
 static int do_commit(BDRVVVFATState* s)
@@ -2933,10 +2934,10 @@ static int do_commit(BDRVVVFATState* s)
         return ret;
     }

-    /* copy FAT (with bdrv_read) */
+    /* copy FAT (with bdrv_pread) */
     memcpy(s->fat.pointer, s->fat2, 0x200 * s->sectors_per_fat);

-    /* recurse direntries from root (using bs->bdrv_read) */
+    /* recurse direntries from root (using bs->bdrv_pread) */
     ret = commit_direntries(s, 0, -1);
     if (ret) {
         fprintf(stderr, "Fatal: error while committing (%d)\n", ret);
@@ -3050,7 +3051,8 @@ DLOG(checkpoint());
      * Use qcow backend. Commit later.
      */
 DLOG(fprintf(stderr, "Write to qcow backend: %d + %d\n", (int)sector_num, nb_sectors));
-    ret = bdrv_write(s->qcow, sector_num, buf, nb_sectors);
+    ret = bdrv_pwrite(s->qcow, sector_num * BDRV_SECTOR_SIZE, buf,
+                      nb_sectors * BDRV_SECTOR_SIZE);
     if (ret < 0) {
         fprintf(stderr, "Error writing to qcow backend\n");
         return ret;
-- 
2.17.2

[Qemu-devel] [PATCH v2 04/13] block: Removed unused sector-based blocking I/O

[Eric Blake]

We are gradually moving away from sector-based interfaces, towards
byte-based.  Now that all callers of blocking I/O have been converted
to use our preferred byte-based bdrv_p{read,write}(), we can
delete the unused bdrv_{read,write}().

Note that the old byte-based code checked that callers passed values
for transactions that did not exceed 2G limits (to prevent scaling
the caller's input by the sector size from exceeding a signed 32-bit
length). We can double-check that the block layer does the same thing
for byte-based code, by noting that even though the newer signatures
allow a 64-bit length parameter, the block layer is actually
fragmenting things to honor both bl.max_transfer (0 if the driver
has no inherent limit, or has not been audited for 64-bit cleanness;
non-zero if the driver wants to enforce a particular cap) as well as
some sanity sizing, as follows:

- bdrv_merge_limits(): Uses the smaller cap of either driver (or leaves things
uncapped if both drivers enforce no cap)
- bdrv_co_do_copy_on_readv(): Caps things to 2G if driver did not request cap
- bdrv_aligned_preadv(): Caps things to 2G if driver did not request cap
- bdrv_co_do_pwrite_zeroes(): Caps things to MAX_BOUNCE_BUFFER if driver
did not request cap
- bdrv_aligned_pwritev(): Caps things to 2G if driver did not request cap

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

---
v2: Enhance commit message with audit results
---
 include/block/block.h |  4 ----
 block/io.c            | 48 ++++++++-----------------------------------
 2 files changed, 8 insertions(+), 44 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index 7f5453b45b6..52b18373807 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -307,10 +307,6 @@ int bdrv_reopen_prepare(BDRVReopenState *reopen_state,
                         BlockReopenQueue *queue, Error **errp);
 void bdrv_reopen_commit(BDRVReopenState *reopen_state);
 void bdrv_reopen_abort(BDRVReopenState *reopen_state);
-int bdrv_read(BdrvChild *child, int64_t sector_num,
-              uint8_t *buf, int nb_sectors);
-int bdrv_write(BdrvChild *child, int64_t sector_num,
-               const uint8_t *buf, int nb_sectors);
 int bdrv_pwrite_zeroes(BdrvChild *child, int64_t offset,
                        int bytes, BdrvRequestFlags flags);
 int bdrv_make_zero(BdrvChild *child, BdrvRequestFlags flags);
diff --git a/block/io.c b/block/io.c
index bd9d688f8ba..39d4e7a3ae1 100644
--- a/block/io.c
+++ b/block/io.c
@@ -836,45 +836,6 @@ static int bdrv_prwv_co(BdrvChild *child, int64_t offset,
     return rwco.ret;
 }

-/*
- * Process a synchronous request using coroutines
- */
-static int bdrv_rw_co(BdrvChild *child, int64_t sector_num, uint8_t *buf,
-                      int nb_sectors, bool is_write, BdrvRequestFlags flags)
-{
-    QEMUIOVector qiov;
-    struct iovec iov = {
-        .iov_base = (void *)buf,
-        .iov_len = nb_sectors * BDRV_SECTOR_SIZE,
-    };
-
-    if (nb_sectors < 0 || nb_sectors > BDRV_REQUEST_MAX_SECTORS) {
-        return -EINVAL;
-    }
-
-    qemu_iovec_init_external(&qiov, &iov, 1);
-    return bdrv_prwv_co(child, sector_num << BDRV_SECTOR_BITS,
-                        &qiov, is_write, flags);
-}
-
-/* return < 0 if error. See bdrv_write() for the return codes */
-int bdrv_read(BdrvChild *child, int64_t sector_num,
-              uint8_t *buf, int nb_sectors)
-{
-    return bdrv_rw_co(child, sector_num, buf, nb_sectors, false, 0);
-}
-
-/* Return < 0 if error. Important errors are:
-  -EIO         generic I/O error (may happen for all errors)
-  -ENOMEDIUM   No media inserted.
-  -EINVAL      Invalid sector number or nb_sectors
-  -EACCES      Trying to write a read-only device
-*/
-int bdrv_write(BdrvChild *child, int64_t sector_num,
-               const uint8_t *buf, int nb_sectors)
-{
-    return bdrv_rw_co(child, sector_num, (uint8_t *)buf, nb_sectors, true, 0);
-}

 int bdrv_pwrite_zeroes(BdrvChild *child, int64_t offset,
                        int bytes, BdrvRequestFlags flags)
@@ -897,7 +858,7 @@ int bdrv_pwrite_zeroes(BdrvChild *child, int64_t offset,
  * flags are passed through to bdrv_pwrite_zeroes (e.g. BDRV_REQ_MAY_UNMAP,
  * BDRV_REQ_FUA).
  *
- * Returns < 0 on error, 0 on success. For error codes see bdrv_write().
+ * Returns < 0 on error, 0 on success. For error codes see bdrv_pwrite().
  */
 int bdrv_make_zero(BdrvChild *child, BdrvRequestFlags flags)
 {
@@ -935,6 +896,7 @@ int bdrv_make_zero(BdrvChild *child, BdrvRequestFlags flags)
     }
 }

+/* return < 0 if error. See bdrv_pwrite() for the return codes */
 int bdrv_preadv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov)
 {
     int ret;
@@ -975,6 +937,12 @@ int bdrv_pwritev(BdrvChild *child, int64_t offset, QEMUIOVector *qiov)
     return qiov->size;
 }

+/* Return < 0 if error. Important errors are:
+  -EIO         generic I/O error (may happen for all errors)
+  -ENOMEDIUM   No media inserted.
+  -EINVAL      Invalid sector number or nb_sectors
+  -EACCES      Trying to write a read-only device
+*/
 int bdrv_pwrite(BdrvChild *child, int64_t offset, const void *buf, int bytes)
 {
     QEMUIOVector qiov;
-- 
2.17.2

[Qemu-devel] [PATCH v2 05/13] block: Switch to 64-bit bl.max_transfer

[Eric Blake]

This change has no semantic impact: all drivers either leave the
value at 0 (no inherent 32-bit limit is still translated into
fragmentation below 2G; see the previous patch for that audit), or
set it to a value less than 2G.  However, switching to a larger
type and enforcing the 2G cap at the block layer makes it easier
to audit specific drivers for their robustness to larger sizing,
by letting them specify a value larger than INT_MAX if they have
been audited to be 64-bit clean.

Signed-off-by: Eric Blake <eblake@redhat.com>
---
 include/block/block_int.h | 8 +++++---
 block/io.c                | 7 +++++++
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index f605622216d..b19d94dac5f 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -549,9 +549,11 @@ typedef struct BlockLimits {

     /* Maximal transfer length in bytes.  Need not be power of 2, but
      * must be multiple of opt_transfer and bl.request_alignment, or 0
-     * for no 32-bit limit.  For now, anything larger than INT_MAX is
-     * clamped down. */
-    uint32_t max_transfer;
+     * for no 32-bit limit.  The block layer fragments all actions
+     * below 2G, so setting this value to anything larger than INT_MAX
+     * implies that the driver has been audited for 64-bit
+     * cleanness. */
+    uint64_t max_transfer;

     /* memory alignment, in bytes so that no bounce buffer is needed */
     size_t min_mem_alignment;
diff --git a/block/io.c b/block/io.c
index 39d4e7a3ae1..4911a1123eb 100644
--- a/block/io.c
+++ b/block/io.c
@@ -159,6 +159,13 @@ void bdrv_refresh_limits(BlockDriverState *bs, Error **errp)
     if (drv->bdrv_refresh_limits) {
         drv->bdrv_refresh_limits(bs, errp);
     }
+
+    /* Clamp max_transfer to 2G */
+    if (bs->bl.max_transfer > UINT32_MAX) {
+        bs->bl.max_transfer = QEMU_ALIGN_DOWN(BDRV_REQUEST_MAX_BYTES,
+                                              MAX(bs->bl.opt_transfer,
+                                                  bs->bl.request_alignment));
+    }
 }

 /**
-- 
2.17.2

Re: [Qemu-devel] [PATCH v2 05/13] block: Switch to 64-bit bl.max_transfer

[Kevin Wolf]

Am 15.11.2018 um 03:03 hat Eric Blake geschrieben:
> This change has no semantic impact: all drivers either leave the
> value at 0 (no inherent 32-bit limit is still translated into
> fragmentation below 2G; see the previous patch for that audit), or
> set it to a value less than 2G.  However, switching to a larger
> type and enforcing the 2G cap at the block layer makes it easier
> to audit specific drivers for their robustness to larger sizing,
> by letting them specify a value larger than INT_MAX if they have
> been audited to be 64-bit clean.
> 
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
>  include/block/block_int.h | 8 +++++---
>  block/io.c                | 7 +++++++
>  2 files changed, 12 insertions(+), 3 deletions(-)
> 
> diff --git a/include/block/block_int.h b/include/block/block_int.h
> index f605622216d..b19d94dac5f 100644
> --- a/include/block/block_int.h
> +++ b/include/block/block_int.h
> @@ -549,9 +549,11 @@ typedef struct BlockLimits {
> 
>      /* Maximal transfer length in bytes.  Need not be power of 2, but
>       * must be multiple of opt_transfer and bl.request_alignment, or 0
> -     * for no 32-bit limit.  For now, anything larger than INT_MAX is
> -     * clamped down. */
> -    uint32_t max_transfer;
> +     * for no 32-bit limit.  The block layer fragments all actions
> +     * below 2G, so setting this value to anything larger than INT_MAX
> +     * implies that the driver has been audited for 64-bit
> +     * cleanness. */
> +    uint64_t max_transfer;
> 
>      /* memory alignment, in bytes so that no bounce buffer is needed */
>      size_t min_mem_alignment;
> diff --git a/block/io.c b/block/io.c
> index 39d4e7a3ae1..4911a1123eb 100644
> --- a/block/io.c
> +++ b/block/io.c
> @@ -159,6 +159,13 @@ void bdrv_refresh_limits(BlockDriverState *bs, Error **errp)
>      if (drv->bdrv_refresh_limits) {
>          drv->bdrv_refresh_limits(bs, errp);
>      }
> +
> +    /* Clamp max_transfer to 2G */
> +    if (bs->bl.max_transfer > UINT32_MAX) {

UINT32_MAX is 4G, not 2G.

Would it make more sense to make BDRV_REQUEST_MAX_BYTES the maximum
anyway? Allowing higher (but not too high) explicit values than what we
clamp to feels a bit odd.

BDRV_REQUEST_MAX_BYTES is probably also what drivers really expect
today.

> +        bs->bl.max_transfer = QEMU_ALIGN_DOWN(BDRV_REQUEST_MAX_BYTES,
> +                                              MAX(bs->bl.opt_transfer,
> +                                                  bs->bl.request_alignment));
> +    }
>  }

Kevin

Re: [Qemu-devel] [PATCH v2 05/13] block: Switch to 64-bit bl.max_transfer

[Eric Blake]

On 11/15/18 9:45 AM, Kevin Wolf wrote:
> Am 15.11.2018 um 03:03 hat Eric Blake geschrieben:
>> This change has no semantic impact: all drivers either leave the
>> value at 0 (no inherent 32-bit limit is still translated into
>> fragmentation below 2G; see the previous patch for that audit), or
>> set it to a value less than 2G.  However, switching to a larger
>> type and enforcing the 2G cap at the block layer makes it easier
>> to audit specific drivers for their robustness to larger sizing,
>> by letting them specify a value larger than INT_MAX if they have
>> been audited to be 64-bit clean.
>>

>> +++ b/block/io.c
>> @@ -159,6 +159,13 @@ void bdrv_refresh_limits(BlockDriverState *bs, Error **errp)
>>       if (drv->bdrv_refresh_limits) {
>>           drv->bdrv_refresh_limits(bs, errp);
>>       }
>> +
>> +    /* Clamp max_transfer to 2G */
>> +    if (bs->bl.max_transfer > UINT32_MAX) {
> 
> UINT32_MAX is 4G, not 2G.
> 
> Would it make more sense to make BDRV_REQUEST_MAX_BYTES the maximum
> anyway?

D'oh.  Yes, that's what I intended, possibly by spelling it INT_MAX (the 
fact that the 'if' goes away in patch 13 is not an excuse for sloppy 
coding in the meantime).

> Allowing higher (but not too high) explicit values than what we
> clamp to feels a bit odd.
> 
> BDRV_REQUEST_MAX_BYTES is probably also what drivers really expect
> today.

Correct.  Well, at least the unaudited drivers (the rest of this series 
audits a few drivers that can handle larger byte values).

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH v2 05/13] block: Switch to 64-bit bl.max_transfer

[Kevin Wolf]

Am 15.11.2018 um 17:28 hat Eric Blake geschrieben:
> On 11/15/18 9:45 AM, Kevin Wolf wrote:
> > Am 15.11.2018 um 03:03 hat Eric Blake geschrieben:
> > > This change has no semantic impact: all drivers either leave the
> > > value at 0 (no inherent 32-bit limit is still translated into
> > > fragmentation below 2G; see the previous patch for that audit), or
> > > set it to a value less than 2G.  However, switching to a larger
> > > type and enforcing the 2G cap at the block layer makes it easier
> > > to audit specific drivers for their robustness to larger sizing,
> > > by letting them specify a value larger than INT_MAX if they have
> > > been audited to be 64-bit clean.
> > > 
> 
> > > +++ b/block/io.c
> > > @@ -159,6 +159,13 @@ void bdrv_refresh_limits(BlockDriverState *bs, Error **errp)
> > >       if (drv->bdrv_refresh_limits) {
> > >           drv->bdrv_refresh_limits(bs, errp);
> > >       }
> > > +
> > > +    /* Clamp max_transfer to 2G */
> > > +    if (bs->bl.max_transfer > UINT32_MAX) {
> > 
> > UINT32_MAX is 4G, not 2G.
> > 
> > Would it make more sense to make BDRV_REQUEST_MAX_BYTES the maximum
> > anyway?
> 
> D'oh.  Yes, that's what I intended, possibly by spelling it INT_MAX (the
> fact that the 'if' goes away in patch 13 is not an excuse for sloppy coding
> in the meantime).

INT_MAX is not a different spelling of BDRV_REQUEST_MAX_BYTES. The
latter is slightly lower (0x7ffffe00).

Kevin

Re: [Qemu-devel] [PATCH v2 05/13] block: Switch to 64-bit bl.max_transfer

[Eric Blake]

On 11/16/18 9:32 AM, Kevin Wolf wrote:
> Am 15.11.2018 um 17:28 hat Eric Blake geschrieben:
>> On 11/15/18 9:45 AM, Kevin Wolf wrote:
>>> Am 15.11.2018 um 03:03 hat Eric Blake geschrieben:
>>>> This change has no semantic impact: all drivers either leave the
>>>> value at 0 (no inherent 32-bit limit is still translated into
>>>> fragmentation below 2G; see the previous patch for that audit), or
>>>> set it to a value less than 2G.  However, switching to a larger
>>>> type and enforcing the 2G cap at the block layer makes it easier
>>>> to audit specific drivers for their robustness to larger sizing,
>>>> by letting them specify a value larger than INT_MAX if they have
>>>> been audited to be 64-bit clean.
>>>>

>>>> +
>>>> +    /* Clamp max_transfer to 2G */
>>>> +    if (bs->bl.max_transfer > UINT32_MAX) {
>>>
>>> UINT32_MAX is 4G, not 2G.
>>>
>>> Would it make more sense to make BDRV_REQUEST_MAX_BYTES the maximum
>>> anyway?
>>
>> D'oh.  Yes, that's what I intended, possibly by spelling it INT_MAX (the
>> fact that the 'if' goes away in patch 13 is not an excuse for sloppy coding
>> in the meantime).
> 
> INT_MAX is not a different spelling of BDRV_REQUEST_MAX_BYTES. The
> latter is slightly lower (0x7ffffe00).

Ah, but:

> +    if (bs->bl.max_transfer > UINT32_MAX) {
> +        bs->bl.max_transfer = QEMU_ALIGN_DOWN(BDRV_REQUEST_MAX_BYTES,
> +                                              MAX(bs->bl.opt_transfer,
> +                                                  bs->bl.request_alignment));
> +    }

QEMU_ALIGN_DOWN() will change INT_MAX to BDRV_REQUEST_MAX_BYTES if 
bs->bl.request_alignment is 512 and opt_transfer is 0; and if either 
alignment number is larger than 512, max_transfer is capped even lower 
regardless of whether I was aligning INT_MAX or BDRV_REQUEST_MAX_BYTES 
down to an alignment boundary.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH v2 05/13] block: Switch to 64-bit bl.max_transfer

[Kevin Wolf]

Am 16.11.2018 um 16:54 hat Eric Blake geschrieben:
> On 11/16/18 9:32 AM, Kevin Wolf wrote:
> > Am 15.11.2018 um 17:28 hat Eric Blake geschrieben:
> > > On 11/15/18 9:45 AM, Kevin Wolf wrote:
> > > > Am 15.11.2018 um 03:03 hat Eric Blake geschrieben:
> > > > > This change has no semantic impact: all drivers either leave the
> > > > > value at 0 (no inherent 32-bit limit is still translated into
> > > > > fragmentation below 2G; see the previous patch for that audit), or
> > > > > set it to a value less than 2G.  However, switching to a larger
> > > > > type and enforcing the 2G cap at the block layer makes it easier
> > > > > to audit specific drivers for their robustness to larger sizing,
> > > > > by letting them specify a value larger than INT_MAX if they have
> > > > > been audited to be 64-bit clean.
> > > > > 
> 
> > > > > +
> > > > > +    /* Clamp max_transfer to 2G */
> > > > > +    if (bs->bl.max_transfer > UINT32_MAX) {
> > > > 
> > > > UINT32_MAX is 4G, not 2G.
> > > > 
> > > > Would it make more sense to make BDRV_REQUEST_MAX_BYTES the maximum
> > > > anyway?
> > > 
> > > D'oh.  Yes, that's what I intended, possibly by spelling it INT_MAX (the
> > > fact that the 'if' goes away in patch 13 is not an excuse for sloppy coding
> > > in the meantime).
> > 
> > INT_MAX is not a different spelling of BDRV_REQUEST_MAX_BYTES. The
> > latter is slightly lower (0x7ffffe00).
> 
> Ah, but:
> 
> > +    if (bs->bl.max_transfer > UINT32_MAX) {
> > +        bs->bl.max_transfer = QEMU_ALIGN_DOWN(BDRV_REQUEST_MAX_BYTES,
> > +                                              MAX(bs->bl.opt_transfer,
> > +                                                  bs->bl.request_alignment));
> > +    }
> 
> QEMU_ALIGN_DOWN() will change INT_MAX to BDRV_REQUEST_MAX_BYTES if
> bs->bl.request_alignment is 512 and opt_transfer is 0; and if either
> alignment number is larger than 512, max_transfer is capped even lower
> regardless of whether I was aligning INT_MAX or BDRV_REQUEST_MAX_BYTES down
> to an alignment boundary.

That's true. But why use INT_MAX and argue that it will be rounded to
the right value later when you can just use BDRV_REQUEST_MAX_BYTES,
which is obviously correct without additional correction?

Kevin

[Qemu-devel] [PATCH v2 06/13] blkdebug: Audit for read/write 64-bit cleanness

[Eric Blake]

Since the block layer is never supposed to hand us an offset + bytes
that would exceed off_t, we can assert this in rule_check(). With
that in place, there is nothing else in the pread, pwrite, or
pwrite_zeroes code paths that can't handle inputs larger than 2G
(even if the block layer currently never hands us something
that large); update the refresh_limits callback to document this
fact, when the user doesn't specify an override.

For a user override, we have to change the QAPI type to 'uint64'
instead of 'int'. At the same time, we can also change 'align'
to 'int32' to match the existing checks in blkdebug_open() that
alignment is always smaller than 2G.

Signed-off-by: Eric Blake <eblake@redhat.com>
---
 qapi/block-core.json |  2 +-
 block/blkdebug.c     | 17 +++++------------
 2 files changed, 6 insertions(+), 13 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index d4fe710836e..32f0edd189f 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3122,7 +3122,7 @@
 { 'struct': 'BlockdevOptionsBlkdebug',
   'data': { 'image': 'BlockdevRef',
             '*config': 'str',
-            '*align': 'int', '*max-transfer': 'int32',
+            '*align': 'int32', '*max-transfer': 'uint64',
             '*opt-write-zero': 'int32', '*max-write-zero': 'int32',
             '*opt-discard': 'int32', '*max-discard': 'int32',
             '*inject-error': ['BlkdebugInjectErrorOptions'],
diff --git a/block/blkdebug.c b/block/blkdebug.c
index 0759452925b..be4d65f86a0 100644
--- a/block/blkdebug.c
+++ b/block/blkdebug.c
@@ -415,9 +415,7 @@ static int blkdebug_open(BlockDriverState *bs, QDict *options, int flags,
     align = MAX(s->align, bs->file->bs->bl.request_alignment);

     s->max_transfer = qemu_opt_get_size(opts, "max-transfer", 0);
-    if (s->max_transfer &&
-        (s->max_transfer >= INT_MAX ||
-         !QEMU_IS_ALIGNED(s->max_transfer, align))) {
+    if (s->max_transfer && !QEMU_IS_ALIGNED(s->max_transfer, align)) {
         error_setg(errp, "Cannot meet constraints with max-transfer %" PRIu64,
                    s->max_transfer);
         goto out;
@@ -477,6 +475,7 @@ static int rule_check(BlockDriverState *bs, uint64_t offset, uint64_t bytes)
     int error;
     bool immediately;

+    assert(offset <= INT64_MAX - bytes);
     QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) {
         uint64_t inject_offset = rule->options.inject.offset;

@@ -517,9 +516,7 @@ blkdebug_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
     /* Sanity check block layer guarantees */
     assert(QEMU_IS_ALIGNED(offset, bs->bl.request_alignment));
     assert(QEMU_IS_ALIGNED(bytes, bs->bl.request_alignment));
-    if (bs->bl.max_transfer) {
-        assert(bytes <= bs->bl.max_transfer);
-    }
+    assert(bytes <= bs->bl.max_transfer);

     err = rule_check(bs, offset, bytes);
     if (err) {
@@ -538,9 +535,7 @@ blkdebug_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
     /* Sanity check block layer guarantees */
     assert(QEMU_IS_ALIGNED(offset, bs->bl.request_alignment));
     assert(QEMU_IS_ALIGNED(bytes, bs->bl.request_alignment));
-    if (bs->bl.max_transfer) {
-        assert(bytes <= bs->bl.max_transfer);
-    }
+    assert(bytes <= bs->bl.max_transfer);

     err = rule_check(bs, offset, bytes);
     if (err) {
@@ -865,9 +860,7 @@ static void blkdebug_refresh_limits(BlockDriverState *bs, Error **errp)
     if (s->align) {
         bs->bl.request_alignment = s->align;
     }
-    if (s->max_transfer) {
-        bs->bl.max_transfer = s->max_transfer;
-    }
+    bs->bl.max_transfer = s->max_transfer ?: INT64_MAX;
     if (s->opt_write_zero) {
         bs->bl.pwrite_zeroes_alignment = s->opt_write_zero;
     }
-- 
2.17.2

[Qemu-devel] [PATCH v2 07/13] blklogwrites: Audit for read/write 64-bit cleanness

[Eric Blake]

Nothing in blk_log_writes_co_do_log() is inherently limited by
a 32-bit type; document this by updating the refresh_limits
callback to document that this driver is 64-bit clean.

Signed-off-by: Eric Blake <eblake@redhat.com>
---
 block/blklogwrites.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/block/blklogwrites.c b/block/blklogwrites.c
index ff98cd55333..c945d3d77d7 100644
--- a/block/blklogwrites.c
+++ b/block/blklogwrites.c
@@ -328,6 +328,7 @@ static void blk_log_writes_refresh_limits(BlockDriverState *bs, Error **errp)
 {
     BDRVBlkLogWritesState *s = bs->opaque;
     bs->bl.request_alignment = s->sectorsize;
+    bs->bl.max_transfer = INT64_MAX;
 }

 static int coroutine_fn
-- 
2.17.2

[Qemu-devel] [PATCH v2 08/13] crypto: Audit for read/write 64-bit cleanness

[Eric Blake]

The crypto read/write functions do their own fragmentation (because
everything has to go through a bounce buffer); while we could
advertise BLOCK_CRYPTO_MAX_IO_SIZE as our max_transfer (and let the
block layer do our fragmentation for us), I'm instead choosing to
document that this driver is 64-bit clean.

Signed-off-by: Eric Blake <eblake@redhat.com>
---
 block/crypto.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/block/crypto.c b/block/crypto.c
index 33ee01bebd9..259ef2649e1 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -453,6 +453,7 @@ static void block_crypto_refresh_limits(BlockDriverState *bs, Error **errp)
     BlockCrypto *crypto = bs->opaque;
     uint64_t sector_size = qcrypto_block_get_sector_size(crypto->block);
     bs->bl.request_alignment = sector_size; /* No sub-sector I/O */
+    bs->bl.max_transfer = INT64_MAX;
 }


-- 
2.17.2

[Qemu-devel] [PATCH v2 09/13] RFC: crypto: Rely on block layer for fragmentation

[Eric Blake]

No need to reimplement fragmentation to BLOCK_CRYPTO_MAX_IO_SIZE
ourselves when we can ask the block layer to do it for us.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
Question - is this patch for 'crypto' acceptable, or should we stick
with just the previous one that marks things as 64-bit clean?
---
 block/crypto.c | 80 +++++++++++++++++++-------------------------------
 1 file changed, 30 insertions(+), 50 deletions(-)

diff --git a/block/crypto.c b/block/crypto.c
index 259ef2649e1..b004cef22c2 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -328,8 +328,6 @@ block_crypto_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
                        QEMUIOVector *qiov, int flags)
 {
     BlockCrypto *crypto = bs->opaque;
-    uint64_t cur_bytes; /* number of bytes in current iteration */
-    uint64_t bytes_done = 0;
     uint8_t *cipher_data = NULL;
     QEMUIOVector hd_qiov;
     int ret = 0;
@@ -346,38 +344,30 @@ block_crypto_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
     /* Bounce buffer because we don't wish to expose cipher text
      * in qiov which points to guest memory.
      */
-    cipher_data =
-        qemu_try_blockalign(bs->file->bs, MIN(BLOCK_CRYPTO_MAX_IO_SIZE,
-                                              qiov->size));
+    assert(qiov->size <= BLOCK_CRYPTO_MAX_IO_SIZE);
+    cipher_data = qemu_try_blockalign(bs->file->bs, qiov->size);
     if (cipher_data == NULL) {
         ret = -ENOMEM;
         goto cleanup;
     }

-    while (bytes) {
-        cur_bytes = MIN(bytes, BLOCK_CRYPTO_MAX_IO_SIZE);
+    qemu_iovec_reset(&hd_qiov);
+    qemu_iovec_add(&hd_qiov, cipher_data, bytes);

-        qemu_iovec_reset(&hd_qiov);
-        qemu_iovec_add(&hd_qiov, cipher_data, cur_bytes);
-
-        ret = bdrv_co_preadv(bs->file, payload_offset + offset + bytes_done,
-                             cur_bytes, &hd_qiov, 0);
-        if (ret < 0) {
-            goto cleanup;
-        }
-
-        if (qcrypto_block_decrypt(crypto->block, offset + bytes_done,
-                                  cipher_data, cur_bytes, NULL) < 0) {
-            ret = -EIO;
-            goto cleanup;
-        }
-
-        qemu_iovec_from_buf(qiov, bytes_done, cipher_data, cur_bytes);
+    ret = bdrv_co_preadv(bs->file, payload_offset + offset, bytes,
+                         &hd_qiov, 0);
+    if (ret < 0) {
+        goto cleanup;
+    }

-        bytes -= cur_bytes;
-        bytes_done += cur_bytes;
+    if (qcrypto_block_decrypt(crypto->block, offset,
+                              cipher_data, bytes, NULL) < 0) {
+        ret = -EIO;
+        goto cleanup;
     }

+    qemu_iovec_from_buf(qiov, 0, cipher_data, bytes);
+
  cleanup:
     qemu_iovec_destroy(&hd_qiov);
     qemu_vfree(cipher_data);
@@ -391,8 +381,6 @@ block_crypto_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
                         QEMUIOVector *qiov, int flags)
 {
     BlockCrypto *crypto = bs->opaque;
-    uint64_t cur_bytes; /* number of bytes in current iteration */
-    uint64_t bytes_done = 0;
     uint8_t *cipher_data = NULL;
     QEMUIOVector hd_qiov;
     int ret = 0;
@@ -409,36 +397,28 @@ block_crypto_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
     /* Bounce buffer because we're not permitted to touch
      * contents of qiov - it points to guest memory.
      */
-    cipher_data =
-        qemu_try_blockalign(bs->file->bs, MIN(BLOCK_CRYPTO_MAX_IO_SIZE,
-                                              qiov->size));
+    assert(qiov->size <= BLOCK_CRYPTO_MAX_IO_SIZE);
+    cipher_data = qemu_try_blockalign(bs->file->bs, qiov->size);
     if (cipher_data == NULL) {
         ret = -ENOMEM;
         goto cleanup;
     }

-    while (bytes) {
-        cur_bytes = MIN(bytes, BLOCK_CRYPTO_MAX_IO_SIZE);
+    qemu_iovec_to_buf(qiov, 0, cipher_data, bytes);

-        qemu_iovec_to_buf(qiov, bytes_done, cipher_data, cur_bytes);
+    if (qcrypto_block_encrypt(crypto->block, offset,
+                              cipher_data, bytes, NULL) < 0) {
+        ret = -EIO;
+        goto cleanup;
+    }

-        if (qcrypto_block_encrypt(crypto->block, offset + bytes_done,
-                                  cipher_data, cur_bytes, NULL) < 0) {
-            ret = -EIO;
-            goto cleanup;
-        }
+    qemu_iovec_reset(&hd_qiov);
+    qemu_iovec_add(&hd_qiov, cipher_data, bytes);

-        qemu_iovec_reset(&hd_qiov);
-        qemu_iovec_add(&hd_qiov, cipher_data, cur_bytes);
-
-        ret = bdrv_co_pwritev(bs->file, payload_offset + offset + bytes_done,
-                              cur_bytes, &hd_qiov, flags);
-        if (ret < 0) {
-            goto cleanup;
-        }
-
-        bytes -= cur_bytes;
-        bytes_done += cur_bytes;
+    ret = bdrv_co_pwritev(bs->file, payload_offset + offset,
+                          bytes, &hd_qiov, flags);
+    if (ret < 0) {
+        goto cleanup;
     }

  cleanup:
@@ -453,7 +433,7 @@ static void block_crypto_refresh_limits(BlockDriverState *bs, Error **errp)
     BlockCrypto *crypto = bs->opaque;
     uint64_t sector_size = qcrypto_block_get_sector_size(crypto->block);
     bs->bl.request_alignment = sector_size; /* No sub-sector I/O */
-    bs->bl.max_transfer = INT64_MAX;
+    bs->bl.max_transfer = BLOCK_CRYPTO_MAX_IO_SIZE;
 }


-- 
2.17.2

Re: [Qemu-devel] [PATCH v2 09/13] RFC: crypto: Rely on block layer for fragmentation

[Kevin Wolf]

Am 15.11.2018 um 03:03 hat Eric Blake geschrieben:
> No need to reimplement fragmentation to BLOCK_CRYPTO_MAX_IO_SIZE
> ourselves when we can ask the block layer to do it for us.
> 
> Signed-off-by: Eric Blake <eblake@redhat.com>
> 
> ---
> Question - is this patch for 'crypto' acceptable, or should we stick
> with just the previous one that marks things as 64-bit clean?

I don't know what Dan thinks, but I like it.

Kevin

>  block/crypto.c | 80 +++++++++++++++++++-------------------------------
>  1 file changed, 30 insertions(+), 50 deletions(-)
> 
> diff --git a/block/crypto.c b/block/crypto.c
> index 259ef2649e1..b004cef22c2 100644
> --- a/block/crypto.c
> +++ b/block/crypto.c
> @@ -328,8 +328,6 @@ block_crypto_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
>                         QEMUIOVector *qiov, int flags)
>  {
>      BlockCrypto *crypto = bs->opaque;
> -    uint64_t cur_bytes; /* number of bytes in current iteration */
> -    uint64_t bytes_done = 0;
>      uint8_t *cipher_data = NULL;
>      QEMUIOVector hd_qiov;
>      int ret = 0;
> @@ -346,38 +344,30 @@ block_crypto_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
>      /* Bounce buffer because we don't wish to expose cipher text
>       * in qiov which points to guest memory.
>       */
> -    cipher_data =
> -        qemu_try_blockalign(bs->file->bs, MIN(BLOCK_CRYPTO_MAX_IO_SIZE,
> -                                              qiov->size));
> +    assert(qiov->size <= BLOCK_CRYPTO_MAX_IO_SIZE);
> +    cipher_data = qemu_try_blockalign(bs->file->bs, qiov->size);
>      if (cipher_data == NULL) {
>          ret = -ENOMEM;
>          goto cleanup;
>      }
> 
> -    while (bytes) {
> -        cur_bytes = MIN(bytes, BLOCK_CRYPTO_MAX_IO_SIZE);
> +    qemu_iovec_reset(&hd_qiov);
> +    qemu_iovec_add(&hd_qiov, cipher_data, bytes);
> 
> -        qemu_iovec_reset(&hd_qiov);
> -        qemu_iovec_add(&hd_qiov, cipher_data, cur_bytes);
> -
> -        ret = bdrv_co_preadv(bs->file, payload_offset + offset + bytes_done,
> -                             cur_bytes, &hd_qiov, 0);
> -        if (ret < 0) {
> -            goto cleanup;
> -        }
> -
> -        if (qcrypto_block_decrypt(crypto->block, offset + bytes_done,
> -                                  cipher_data, cur_bytes, NULL) < 0) {
> -            ret = -EIO;
> -            goto cleanup;
> -        }
> -
> -        qemu_iovec_from_buf(qiov, bytes_done, cipher_data, cur_bytes);
> +    ret = bdrv_co_preadv(bs->file, payload_offset + offset, bytes,
> +                         &hd_qiov, 0);
> +    if (ret < 0) {
> +        goto cleanup;
> +    }
> 
> -        bytes -= cur_bytes;
> -        bytes_done += cur_bytes;
> +    if (qcrypto_block_decrypt(crypto->block, offset,
> +                              cipher_data, bytes, NULL) < 0) {
> +        ret = -EIO;
> +        goto cleanup;
>      }
> 
> +    qemu_iovec_from_buf(qiov, 0, cipher_data, bytes);
> +
>   cleanup:
>      qemu_iovec_destroy(&hd_qiov);
>      qemu_vfree(cipher_data);
> @@ -391,8 +381,6 @@ block_crypto_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
>                          QEMUIOVector *qiov, int flags)
>  {
>      BlockCrypto *crypto = bs->opaque;
> -    uint64_t cur_bytes; /* number of bytes in current iteration */
> -    uint64_t bytes_done = 0;
>      uint8_t *cipher_data = NULL;
>      QEMUIOVector hd_qiov;
>      int ret = 0;
> @@ -409,36 +397,28 @@ block_crypto_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
>      /* Bounce buffer because we're not permitted to touch
>       * contents of qiov - it points to guest memory.
>       */
> -    cipher_data =
> -        qemu_try_blockalign(bs->file->bs, MIN(BLOCK_CRYPTO_MAX_IO_SIZE,
> -                                              qiov->size));
> +    assert(qiov->size <= BLOCK_CRYPTO_MAX_IO_SIZE);
> +    cipher_data = qemu_try_blockalign(bs->file->bs, qiov->size);
>      if (cipher_data == NULL) {
>          ret = -ENOMEM;
>          goto cleanup;
>      }
> 
> -    while (bytes) {
> -        cur_bytes = MIN(bytes, BLOCK_CRYPTO_MAX_IO_SIZE);
> +    qemu_iovec_to_buf(qiov, 0, cipher_data, bytes);
> 
> -        qemu_iovec_to_buf(qiov, bytes_done, cipher_data, cur_bytes);
> +    if (qcrypto_block_encrypt(crypto->block, offset,
> +                              cipher_data, bytes, NULL) < 0) {
> +        ret = -EIO;
> +        goto cleanup;
> +    }
> 
> -        if (qcrypto_block_encrypt(crypto->block, offset + bytes_done,
> -                                  cipher_data, cur_bytes, NULL) < 0) {
> -            ret = -EIO;
> -            goto cleanup;
> -        }
> +    qemu_iovec_reset(&hd_qiov);
> +    qemu_iovec_add(&hd_qiov, cipher_data, bytes);
> 
> -        qemu_iovec_reset(&hd_qiov);
> -        qemu_iovec_add(&hd_qiov, cipher_data, cur_bytes);
> -
> -        ret = bdrv_co_pwritev(bs->file, payload_offset + offset + bytes_done,
> -                              cur_bytes, &hd_qiov, flags);
> -        if (ret < 0) {
> -            goto cleanup;
> -        }
> -
> -        bytes -= cur_bytes;
> -        bytes_done += cur_bytes;
> +    ret = bdrv_co_pwritev(bs->file, payload_offset + offset,
> +                          bytes, &hd_qiov, flags);
> +    if (ret < 0) {
> +        goto cleanup;
>      }
> 
>   cleanup:
> @@ -453,7 +433,7 @@ static void block_crypto_refresh_limits(BlockDriverState *bs, Error **errp)
>      BlockCrypto *crypto = bs->opaque;
>      uint64_t sector_size = qcrypto_block_get_sector_size(crypto->block);
>      bs->bl.request_alignment = sector_size; /* No sub-sector I/O */
> -    bs->bl.max_transfer = INT64_MAX;
> +    bs->bl.max_transfer = BLOCK_CRYPTO_MAX_IO_SIZE;
>  }
> 
> 
> -- 
> 2.17.2
> 

Re: [Qemu-devel] [PATCH v2 09/13] RFC: crypto: Rely on block layer for fragmentation

[Daniel P. Berrangé]

On Wed, Nov 14, 2018 at 08:03:30PM -0600, Eric Blake wrote:
> No need to reimplement fragmentation to BLOCK_CRYPTO_MAX_IO_SIZE
> ourselves when we can ask the block layer to do it for us.
> 
> Signed-off-by: Eric Blake <eblake@redhat.com>
> 
> ---
> Question - is this patch for 'crypto' acceptable, or should we stick
> with just the previous one that marks things as 64-bit clean?

Unless I'm missing something, this is functionally equivalent to
the existing code, and is simpler to read, so I don't see any
obvious downside  to this patch.

Assuming that you've run 'qemu-iotests/check -luks' on it to
validate it then ...

   Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>



> ---
>  block/crypto.c | 80 +++++++++++++++++++-------------------------------
>  1 file changed, 30 insertions(+), 50 deletions(-)
> 
> diff --git a/block/crypto.c b/block/crypto.c
> index 259ef2649e1..b004cef22c2 100644
> --- a/block/crypto.c
> +++ b/block/crypto.c
> @@ -328,8 +328,6 @@ block_crypto_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
>                         QEMUIOVector *qiov, int flags)
>  {
>      BlockCrypto *crypto = bs->opaque;
> -    uint64_t cur_bytes; /* number of bytes in current iteration */
> -    uint64_t bytes_done = 0;
>      uint8_t *cipher_data = NULL;
>      QEMUIOVector hd_qiov;
>      int ret = 0;
> @@ -346,38 +344,30 @@ block_crypto_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
>      /* Bounce buffer because we don't wish to expose cipher text
>       * in qiov which points to guest memory.
>       */
> -    cipher_data =
> -        qemu_try_blockalign(bs->file->bs, MIN(BLOCK_CRYPTO_MAX_IO_SIZE,
> -                                              qiov->size));
> +    assert(qiov->size <= BLOCK_CRYPTO_MAX_IO_SIZE);
> +    cipher_data = qemu_try_blockalign(bs->file->bs, qiov->size);
>      if (cipher_data == NULL) {
>          ret = -ENOMEM;
>          goto cleanup;
>      }
> 
> -    while (bytes) {
> -        cur_bytes = MIN(bytes, BLOCK_CRYPTO_MAX_IO_SIZE);
> +    qemu_iovec_reset(&hd_qiov);
> +    qemu_iovec_add(&hd_qiov, cipher_data, bytes);
> 
> -        qemu_iovec_reset(&hd_qiov);
> -        qemu_iovec_add(&hd_qiov, cipher_data, cur_bytes);
> -
> -        ret = bdrv_co_preadv(bs->file, payload_offset + offset + bytes_done,
> -                             cur_bytes, &hd_qiov, 0);
> -        if (ret < 0) {
> -            goto cleanup;
> -        }
> -
> -        if (qcrypto_block_decrypt(crypto->block, offset + bytes_done,
> -                                  cipher_data, cur_bytes, NULL) < 0) {
> -            ret = -EIO;
> -            goto cleanup;
> -        }
> -
> -        qemu_iovec_from_buf(qiov, bytes_done, cipher_data, cur_bytes);
> +    ret = bdrv_co_preadv(bs->file, payload_offset + offset, bytes,
> +                         &hd_qiov, 0);
> +    if (ret < 0) {
> +        goto cleanup;
> +    }
> 
> -        bytes -= cur_bytes;
> -        bytes_done += cur_bytes;
> +    if (qcrypto_block_decrypt(crypto->block, offset,
> +                              cipher_data, bytes, NULL) < 0) {
> +        ret = -EIO;
> +        goto cleanup;
>      }
> 
> +    qemu_iovec_from_buf(qiov, 0, cipher_data, bytes);
> +
>   cleanup:
>      qemu_iovec_destroy(&hd_qiov);
>      qemu_vfree(cipher_data);
> @@ -391,8 +381,6 @@ block_crypto_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
>                          QEMUIOVector *qiov, int flags)
>  {
>      BlockCrypto *crypto = bs->opaque;
> -    uint64_t cur_bytes; /* number of bytes in current iteration */
> -    uint64_t bytes_done = 0;
>      uint8_t *cipher_data = NULL;
>      QEMUIOVector hd_qiov;
>      int ret = 0;
> @@ -409,36 +397,28 @@ block_crypto_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
>      /* Bounce buffer because we're not permitted to touch
>       * contents of qiov - it points to guest memory.
>       */
> -    cipher_data =
> -        qemu_try_blockalign(bs->file->bs, MIN(BLOCK_CRYPTO_MAX_IO_SIZE,
> -                                              qiov->size));
> +    assert(qiov->size <= BLOCK_CRYPTO_MAX_IO_SIZE);
> +    cipher_data = qemu_try_blockalign(bs->file->bs, qiov->size);
>      if (cipher_data == NULL) {
>          ret = -ENOMEM;
>          goto cleanup;
>      }
> 
> -    while (bytes) {
> -        cur_bytes = MIN(bytes, BLOCK_CRYPTO_MAX_IO_SIZE);
> +    qemu_iovec_to_buf(qiov, 0, cipher_data, bytes);
> 
> -        qemu_iovec_to_buf(qiov, bytes_done, cipher_data, cur_bytes);
> +    if (qcrypto_block_encrypt(crypto->block, offset,
> +                              cipher_data, bytes, NULL) < 0) {
> +        ret = -EIO;
> +        goto cleanup;
> +    }
> 
> -        if (qcrypto_block_encrypt(crypto->block, offset + bytes_done,
> -                                  cipher_data, cur_bytes, NULL) < 0) {
> -            ret = -EIO;
> -            goto cleanup;
> -        }
> +    qemu_iovec_reset(&hd_qiov);
> +    qemu_iovec_add(&hd_qiov, cipher_data, bytes);
> 
> -        qemu_iovec_reset(&hd_qiov);
> -        qemu_iovec_add(&hd_qiov, cipher_data, cur_bytes);
> -
> -        ret = bdrv_co_pwritev(bs->file, payload_offset + offset + bytes_done,
> -                              cur_bytes, &hd_qiov, flags);
> -        if (ret < 0) {
> -            goto cleanup;
> -        }
> -
> -        bytes -= cur_bytes;
> -        bytes_done += cur_bytes;
> +    ret = bdrv_co_pwritev(bs->file, payload_offset + offset,
> +                          bytes, &hd_qiov, flags);
> +    if (ret < 0) {
> +        goto cleanup;
>      }
> 
>   cleanup:
> @@ -453,7 +433,7 @@ static void block_crypto_refresh_limits(BlockDriverState *bs, Error **errp)
>      BlockCrypto *crypto = bs->opaque;
>      uint64_t sector_size = qcrypto_block_get_sector_size(crypto->block);
>      bs->bl.request_alignment = sector_size; /* No sub-sector I/O */
> -    bs->bl.max_transfer = INT64_MAX;
> +    bs->bl.max_transfer = BLOCK_CRYPTO_MAX_IO_SIZE;
>  }
> 
> 
> -- 
> 2.17.2
> 
> 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[Qemu-devel] [PATCH v2 10/13] file-posix: Audit for read/write 64-bit cleanness

[Eric Blake]

Any use of aio is inherently limited by size_t aio_nbytes in struct
aiocb.  read() is similarly limited to size_t bytes, although in
practice, the ssize_t return means any read attempt on a 32-bit
platform for more than 2G will likely return a short read (if that
much memory was even available to begin with).  And while preadv()
can technically read more than size_t bytes by use of more than one
iov, the fact that you can only pass a finite number of iov each of
which is limited to size_t bytes is a limiting factor.

While we already attempt other methods at populating a more reasonable
max_transfer limit in the cases where the kernel makes that
information available, it is important that we at least let the block
layer know about our hard limitation of size_t bytes (mainly
applicable to 32-bit compilation).  At the same time, on 64-bit
platforms, that means we are now advertising that we don't have any
other unintended size-botching problems, if the block layer were to
start handing us requests larger than 2G.

Signed-off-by: Eric Blake <eblake@redhat.com>
---
 block/file-posix.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/block/file-posix.c b/block/file-posix.c
index 48ad3bb372a..4b43ff8cb5c 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -1073,6 +1073,9 @@ static void raw_refresh_limits(BlockDriverState *bs, Error **errp)
     raw_probe_alignment(bs, s->fd, errp);
     bs->bl.min_mem_alignment = s->buf_align;
     bs->bl.opt_mem_alignment = MAX(s->buf_align, getpagesize());
+    if (!bs->bl.max_transfer) {
+        bs->bl.max_transfer = SIZE_MAX;
+    }
 }

 static int check_for_dasd(int fd)
-- 
2.17.2

[Qemu-devel] [PATCH v2 11/13] qcow2: Audit for read/write 64-bit cleanness

[Eric Blake]

The qcow2 read/write functions do their own fragmentation (because
of cluster remapping); while we could advertise s->cluster_size
and let the block layer do fragmentation for us, that would NOT
solve the issue of the block layer handing us a length less than
a cluster but at an offset which overlaps a cluster boundary. Thus,
we still have to fragment ourselves, at which point it is easiest
to just document that this driver is 64-bit clean.

Signed-off-by: Eric Blake <eblake@redhat.com>
---
 block/qcow2.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/block/qcow2.c b/block/qcow2.c
index 0b5ad130060..1dd3491f77f 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1687,6 +1687,7 @@ static void qcow2_refresh_limits(BlockDriverState *bs, Error **errp)
     }
     bs->bl.pwrite_zeroes_alignment = s->cluster_size;
     bs->bl.pdiscard_alignment = s->cluster_size;
+    bs->bl.max_transfer = INT64_MAX;
 }

 static int qcow2_reopen_prepare(BDRVReopenState *state,
-- 
2.17.2

[Qemu-devel] [PATCH v2 12/13] block: Document need for audit of read/write 64-bit cleanness

[Eric Blake]

At this time, any block driver that has not been audited for
64-bit cleanness, but which uses byte-based callbacks, should
explicitly document that the driver wants the block layer to
cap things at 2G.  This patch has no semantic change.  And it
shows that the things I'm not interested in auditing are:
bochs, cloop, dmg, file-win32, qcow, rbd, vvfat, vxhs

Signed-off-by: Eric Blake <eblake@redhat.com>
---
 block/bochs.c      | 1 +
 block/cloop.c      | 1 +
 block/dmg.c        | 1 +
 block/file-win32.c | 2 ++
 block/qcow.c       | 1 +
 block/rbd.c        | 1 +
 block/vvfat.c      | 1 +
 block/vxhs.c       | 1 +
 8 files changed, 9 insertions(+)

diff --git a/block/bochs.c b/block/bochs.c
index 22e7d442113..cfa449ffb6f 100644
--- a/block/bochs.c
+++ b/block/bochs.c
@@ -202,6 +202,7 @@ fail:
 static void bochs_refresh_limits(BlockDriverState *bs, Error **errp)
 {
     bs->bl.request_alignment = BDRV_SECTOR_SIZE; /* No sub-sector I/O */
+    bs->bl.max_transfer = BDRV_REQUEST_MAX_BYTES;
 }

 static int64_t seek_to_sector(BlockDriverState *bs, int64_t sector_num)
diff --git a/block/cloop.c b/block/cloop.c
index df2b85f7234..1ac0ed234d8 100644
--- a/block/cloop.c
+++ b/block/cloop.c
@@ -211,6 +211,7 @@ fail:
 static void cloop_refresh_limits(BlockDriverState *bs, Error **errp)
 {
     bs->bl.request_alignment = BDRV_SECTOR_SIZE; /* No sub-sector I/O */
+    bs->bl.max_transfer = BDRV_REQUEST_MAX_BYTES;
 }

 static inline int cloop_read_block(BlockDriverState *bs, int block_num)
diff --git a/block/dmg.c b/block/dmg.c
index 50e91aef6d9..ea5553b6bf2 100644
--- a/block/dmg.c
+++ b/block/dmg.c
@@ -555,6 +555,7 @@ fail:
 static void dmg_refresh_limits(BlockDriverState *bs, Error **errp)
 {
     bs->bl.request_alignment = BDRV_SECTOR_SIZE; /* No sub-sector I/O */
+    bs->bl.max_transfer = BDRV_REQUEST_MAX_BYTES;
 }

 static inline int is_sector_in_chunk(BDRVDMGState* s,
diff --git a/block/file-win32.c b/block/file-win32.c
index f1e2187f3bd..91b2dd9ed88 100644
--- a/block/file-win32.c
+++ b/block/file-win32.c
@@ -256,6 +256,7 @@ static void raw_probe_alignment(BlockDriverState *bs, Error **errp)

     /* XXX Does Windows support AIO on less than 512-byte alignment? */
     bs->bl.request_alignment = 512;
+    bs->bl.max_transfer = BDRV_REQUEST_MAX_BYTES;
 }

 static void raw_parse_flags(int flags, bool use_aio, int *access_flags,
@@ -716,6 +717,7 @@ static void hdev_refresh_limits(BlockDriverState *bs, Error **errp)
 {
     /* XXX Does Windows support AIO on less than 512-byte alignment? */
     bs->bl.request_alignment = 512;
+    bs->bl.max_transfer = BDRV_REQUEST_MAX_BYTES;
 }

 static int hdev_open(BlockDriverState *bs, QDict *options, int flags,
diff --git a/block/qcow.c b/block/qcow.c
index 4518cb4c35e..ae6deacdb1f 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -618,6 +618,7 @@ static void qcow_refresh_limits(BlockDriverState *bs, Error **errp)
      * it's easier to let the block layer handle rounding than to
      * audit this code further. */
     bs->bl.request_alignment = BDRV_SECTOR_SIZE;
+    bs->bl.max_transfer = BDRV_REQUEST_MAX_BYTES;
 }

 static coroutine_fn int qcow_co_preadv(BlockDriverState *bs, uint64_t offset,
diff --git a/block/rbd.c b/block/rbd.c
index 8a1a9f4b6e2..248e635b077 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -236,6 +236,7 @@ static void qemu_rbd_refresh_limits(BlockDriverState *bs, Error **errp)
 {
     /* XXX Does RBD support AIO on less than 512-byte alignment? */
     bs->bl.request_alignment = 512;
+    bs->bl.max_transfer = BDRV_REQUEST_MAX_BYTES;
 }


diff --git a/block/vvfat.c b/block/vvfat.c
index 915a05ae4f2..35f35328baf 100644
--- a/block/vvfat.c
+++ b/block/vvfat.c
@@ -1304,6 +1304,7 @@ fail:
 static void vvfat_refresh_limits(BlockDriverState *bs, Error **errp)
 {
     bs->bl.request_alignment = BDRV_SECTOR_SIZE; /* No sub-sector I/O */
+    bs->bl.max_transfer = BDRV_REQUEST_MAX_BYTES;
 }

 static inline void vvfat_close_current_file(BDRVVVFATState *s)
diff --git a/block/vxhs.c b/block/vxhs.c
index 0cb0a007e90..5828f53432c 100644
--- a/block/vxhs.c
+++ b/block/vxhs.c
@@ -221,6 +221,7 @@ static void vxhs_refresh_limits(BlockDriverState *bs, Error **errp)
 {
     /* XXX Does VXHS support AIO on less than 512-byte alignment? */
     bs->bl.request_alignment = 512;
+    bs->bl.max_transfer = BDRV_REQUEST_MAX_BYTES;
 }

 static int vxhs_init_and_ref(void)
-- 
2.17.2

[Qemu-devel] [PATCH v2 13/13] block: Enforce non-zero bl.max_transfer

[Eric Blake]

The raw format driver and the filter drivers default to picking
up the same limits as what they wrap, and I've audited that they
are otherwise simple enough in their passthrough to be 64-bit
clean; it's not worth changing their .bdrv_refresh_limits to
advertise anything different.  Previous patches changed all
remaining byte-based format and protocol drivers to supply an
explicit max_transfer consistent with an audit (or lack thereof)
of their code.  And for drivers still using sector-based
callbacks (gluster, iscsi, parallels, qed, replication, sheepdog,
ssh, vhdx), we can easily supply a 2G default limit while waiting
for a later per-driver conversion and auditing while moving to
byte-based callbacks.

With that, we can assert that max_transfer is now always set (either
by sector-based default, by merge with a backing layer, or by
explicit settings), and enforce that it be non-zero by the end
of bdrv_refresh_limits().  This in turn lets us simplify portions
of the block layer to use MIN() instead of MIN_NON_ZERO(), while
still fragmenting things to no larger than 2G chunks.  Also, we no
longer need to rewrite the driver's bl.max_transfer to a value below
2G.  There should be no semantic change from this patch, although
it does open the door if a future patch wants to let the block layer
choose a larger value than 2G while still honoring bl.max_transfer
for fragmentation.

Signed-off-by: Eric Blake <eblake@redhat.com>
---

[hmm - in sending this email, I realize that I didn't specifically
check whether quorum always picks up a sane limit from its children,
since it is byte-based but lacks a .bdrv_refresh_limits]

 include/block/block_int.h | 10 +++++-----
 block/io.c                | 25 ++++++++++++-------------
 2 files changed, 17 insertions(+), 18 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index b19d94dac5f..410553bb0cc 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -548,11 +548,11 @@ typedef struct BlockLimits {
     uint32_t opt_transfer;

     /* Maximal transfer length in bytes.  Need not be power of 2, but
-     * must be multiple of opt_transfer and bl.request_alignment, or 0
-     * for no 32-bit limit.  The block layer fragments all actions
-     * below 2G, so setting this value to anything larger than INT_MAX
-     * implies that the driver has been audited for 64-bit
-     * cleanness. */
+     * must be larger than opt_transfer and request_alignment (the
+     * block layer rounds it down as needed).  The block layer
+     * fragments all actions below 2G, so setting this value to
+     * anything larger than INT_MAX implies that the driver has been
+     * audited for 64-bit cleanness. */
     uint64_t max_transfer;

     /* memory alignment, in bytes so that no bounce buffer is needed */
diff --git a/block/io.c b/block/io.c
index 4911a1123eb..0024be0bf31 100644
--- a/block/io.c
+++ b/block/io.c
@@ -129,6 +129,9 @@ void bdrv_refresh_limits(BlockDriverState *bs, Error **errp)
     /* Default alignment based on whether driver has byte interface */
     bs->bl.request_alignment = (drv->bdrv_co_preadv ||
                                 drv->bdrv_aio_preadv) ? 1 : 512;
+    /* Default cap at 2G for drivers without byte interface */
+    if (bs->bl.request_alignment == 1)
+        bs->bl.max_transfer = BDRV_REQUEST_MAX_BYTES;

     /* Take some limits from the children as a default */
     if (bs->file) {
@@ -160,12 +163,11 @@ void bdrv_refresh_limits(BlockDriverState *bs, Error **errp)
         drv->bdrv_refresh_limits(bs, errp);
     }

-    /* Clamp max_transfer to 2G */
-    if (bs->bl.max_transfer > UINT32_MAX) {
-        bs->bl.max_transfer = QEMU_ALIGN_DOWN(BDRV_REQUEST_MAX_BYTES,
-                                              MAX(bs->bl.opt_transfer,
-                                                  bs->bl.request_alignment));
-    }
+    /* Check that we got a maximum cap, and round it down as needed */
+    assert(bs->bl.max_transfer);
+    bs->bl.max_transfer = QEMU_ALIGN_DOWN(bs->bl.max_transfer,
+                                          MAX(bs->bl.opt_transfer,
+                                              bs->bl.request_alignment));
 }

 /**
@@ -1145,8 +1147,7 @@ static int coroutine_fn bdrv_co_do_copy_on_readv(BdrvChild *child,
     int64_t cluster_bytes;
     size_t skip_bytes;
     int ret;
-    int max_transfer = MIN_NON_ZERO(bs->bl.max_transfer,
-                                    BDRV_REQUEST_MAX_BYTES);
+    int max_transfer = MIN(bs->bl.max_transfer, BDRV_REQUEST_MAX_BYTES);
     unsigned int progress = 0;

     if (!drv) {
@@ -1286,8 +1287,7 @@ static int coroutine_fn bdrv_aligned_preadv(BdrvChild *child,
     assert((bytes & (align - 1)) == 0);
     assert(!qiov || bytes == qiov->size);
     assert((bs->open_flags & BDRV_O_NO_IO) == 0);
-    max_transfer = QEMU_ALIGN_DOWN(MIN_NON_ZERO(bs->bl.max_transfer, INT_MAX),
-                                   align);
+    max_transfer = QEMU_ALIGN_DOWN(MIN(bs->bl.max_transfer, INT_MAX), align);

     /* TODO: We would need a per-BDS .supported_read_flags and
      * potential fallback support, if we ever implement any read flags
@@ -1460,7 +1460,7 @@ static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs,
     int max_write_zeroes = MIN_NON_ZERO(bs->bl.max_pwrite_zeroes, INT_MAX);
     int alignment = MAX(bs->bl.pwrite_zeroes_alignment,
                         bs->bl.request_alignment);
-    int max_transfer = MIN_NON_ZERO(bs->bl.max_transfer, MAX_BOUNCE_BUFFER);
+    int max_transfer = MIN(bs->bl.max_transfer, MAX_BOUNCE_BUFFER);

     if (!drv) {
         return -ENOMEDIUM;
@@ -1668,8 +1668,7 @@ static int coroutine_fn bdrv_aligned_pwritev(BdrvChild *child,
     assert((offset & (align - 1)) == 0);
     assert((bytes & (align - 1)) == 0);
     assert(!qiov || bytes == qiov->size);
-    max_transfer = QEMU_ALIGN_DOWN(MIN_NON_ZERO(bs->bl.max_transfer, INT_MAX),
-                                   align);
+    max_transfer = QEMU_ALIGN_DOWN(MIN(bs->bl.max_transfer, INT_MAX), align);

     ret = bdrv_co_write_req_prepare(child, offset, bytes, req, flags);

-- 
2.17.2

Re: [Qemu-devel] [PATCH v2 13/13] block: Enforce non-zero bl.max_transfer

[Kevin Wolf]

Am 15.11.2018 um 03:03 hat Eric Blake geschrieben:
> The raw format driver and the filter drivers default to picking
> up the same limits as what they wrap, and I've audited that they
> are otherwise simple enough in their passthrough to be 64-bit
> clean; it's not worth changing their .bdrv_refresh_limits to
> advertise anything different.  Previous patches changed all
> remaining byte-based format and protocol drivers to supply an
> explicit max_transfer consistent with an audit (or lack thereof)
> of their code.  And for drivers still using sector-based
> callbacks (gluster, iscsi, parallels, qed, replication, sheepdog,
> ssh, vhdx), we can easily supply a 2G default limit while waiting
> for a later per-driver conversion and auditing while moving to
> byte-based callbacks.
> 
> With that, we can assert that max_transfer is now always set (either
> by sector-based default, by merge with a backing layer, or by
> explicit settings), and enforce that it be non-zero by the end
> of bdrv_refresh_limits().  This in turn lets us simplify portions
> of the block layer to use MIN() instead of MIN_NON_ZERO(), while
> still fragmenting things to no larger than 2G chunks.  Also, we no
> longer need to rewrite the driver's bl.max_transfer to a value below
> 2G.  There should be no semantic change from this patch, although
> it does open the door if a future patch wants to let the block layer
> choose a larger value than 2G while still honoring bl.max_transfer
> for fragmentation.
> 
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> 
> [hmm - in sending this email, I realize that I didn't specifically
> check whether quorum always picks up a sane limit from its children,
> since it is byte-based but lacks a .bdrv_refresh_limits]
> 
>  include/block/block_int.h | 10 +++++-----
>  block/io.c                | 25 ++++++++++++-------------
>  2 files changed, 17 insertions(+), 18 deletions(-)
> 
> diff --git a/include/block/block_int.h b/include/block/block_int.h
> index b19d94dac5f..410553bb0cc 100644
> --- a/include/block/block_int.h
> +++ b/include/block/block_int.h
> @@ -548,11 +548,11 @@ typedef struct BlockLimits {
>      uint32_t opt_transfer;
> 
>      /* Maximal transfer length in bytes.  Need not be power of 2, but
> -     * must be multiple of opt_transfer and bl.request_alignment, or 0
> -     * for no 32-bit limit.  The block layer fragments all actions
> -     * below 2G, so setting this value to anything larger than INT_MAX
> -     * implies that the driver has been audited for 64-bit
> -     * cleanness. */
> +     * must be larger than opt_transfer and request_alignment (the
> +     * block layer rounds it down as needed).  The block layer
> +     * fragments all actions below 2G, so setting this value to
> +     * anything larger than INT_MAX implies that the driver has been
> +     * audited for 64-bit cleanness. */
>      uint64_t max_transfer;
> 
>      /* memory alignment, in bytes so that no bounce buffer is needed */
> diff --git a/block/io.c b/block/io.c
> index 4911a1123eb..0024be0bf31 100644
> --- a/block/io.c
> +++ b/block/io.c
> @@ -129,6 +129,9 @@ void bdrv_refresh_limits(BlockDriverState *bs, Error **errp)
>      /* Default alignment based on whether driver has byte interface */
>      bs->bl.request_alignment = (drv->bdrv_co_preadv ||
>                                  drv->bdrv_aio_preadv) ? 1 : 512;
> +    /* Default cap at 2G for drivers without byte interface */
> +    if (bs->bl.request_alignment == 1)
> +        bs->bl.max_transfer = BDRV_REQUEST_MAX_BYTES;

Missing braces, and comment and code don't match (the comment suggests
that the condition should be != 1).

Kevin

Re: [Qemu-devel] [PATCH v2 13/13] block: Enforce non-zero bl.max_transfer

[Eric Blake]

On 11/15/18 10:24 AM, Kevin Wolf wrote:
> Am 15.11.2018 um 03:03 hat Eric Blake geschrieben:
>> The raw format driver and the filter drivers default to picking
>> up the same limits as what they wrap, and I've audited that they
>> are otherwise simple enough in their passthrough to be 64-bit
>> clean; it's not worth changing their .bdrv_refresh_limits to
>> advertise anything different.  Previous patches changed all
>> remaining byte-based format and protocol drivers to supply an
>> explicit max_transfer consistent with an audit (or lack thereof)
>> of their code.  And for drivers still using sector-based
>> callbacks (gluster, iscsi, parallels, qed, replication, sheepdog,
>> ssh, vhdx), we can easily supply a 2G default limit while waiting
>> for a later per-driver conversion and auditing while moving to
>> byte-based callbacks.
>>
>> With that, we can assert that max_transfer is now always set (either
>> by sector-based default, by merge with a backing layer, or by
>> explicit settings), and enforce that it be non-zero by the end
>> of bdrv_refresh_limits().  This in turn lets us simplify portions
>> of the block layer to use MIN() instead of MIN_NON_ZERO(), while
>> still fragmenting things to no larger than 2G chunks.  Also, we no
>> longer need to rewrite the driver's bl.max_transfer to a value below
>> 2G.  There should be no semantic change from this patch, although
>> it does open the door if a future patch wants to let the block layer
>> choose a larger value than 2G while still honoring bl.max_transfer
>> for fragmentation.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>>
>> [hmm - in sending this email, I realize that I didn't specifically
>> check whether quorum always picks up a sane limit from its children,
>> since it is byte-based but lacks a .bdrv_refresh_limits]
>>
>>   include/block/block_int.h | 10 +++++-----
>>   block/io.c                | 25 ++++++++++++-------------
>>   2 files changed, 17 insertions(+), 18 deletions(-)
>>
>> diff --git a/include/block/block_int.h b/include/block/block_int.h
>> index b19d94dac5f..410553bb0cc 100644
>> --- a/include/block/block_int.h
>> +++ b/include/block/block_int.h
>> @@ -548,11 +548,11 @@ typedef struct BlockLimits {
>>       uint32_t opt_transfer;
>>
>>       /* Maximal transfer length in bytes.  Need not be power of 2, but
>> -     * must be multiple of opt_transfer and bl.request_alignment, or 0
>> -     * for no 32-bit limit.  The block layer fragments all actions
>> -     * below 2G, so setting this value to anything larger than INT_MAX
>> -     * implies that the driver has been audited for 64-bit
>> -     * cleanness. */
>> +     * must be larger than opt_transfer and request_alignment (the
>> +     * block layer rounds it down as needed).  The block layer
>> +     * fragments all actions below 2G, so setting this value to
>> +     * anything larger than INT_MAX implies that the driver has been
>> +     * audited for 64-bit cleanness. */
>>       uint64_t max_transfer;
>>
>>       /* memory alignment, in bytes so that no bounce buffer is needed */
>> diff --git a/block/io.c b/block/io.c
>> index 4911a1123eb..0024be0bf31 100644
>> --- a/block/io.c
>> +++ b/block/io.c
>> @@ -129,6 +129,9 @@ void bdrv_refresh_limits(BlockDriverState *bs, Error **errp)
>>       /* Default alignment based on whether driver has byte interface */
>>       bs->bl.request_alignment = (drv->bdrv_co_preadv ||
>>                                   drv->bdrv_aio_preadv) ? 1 : 512;
>> +    /* Default cap at 2G for drivers without byte interface */
>> +    if (bs->bl.request_alignment == 1)
>> +        bs->bl.max_transfer = BDRV_REQUEST_MAX_BYTES;
> 
> Missing braces, and comment and code don't match (the comment suggests
> that the condition should be != 1).

Indeed. I shouldn't send patches late at night ;)  In fact, it proves I 
didn't run iotests on any of the protocol drivers that would be impacted 
by this code (gluster, iscsi, sheepdog, ssh), and thus, would probably 
fail the added assertion when they leave bl.max_transfer at 0 because I 
didn't initialize a default for them.  The format drivers (parallels, 
qed, replication, vhdx) would probably still work because we merge in 
the max_transfer of bs->file before the assert that bl.max_transfer is 
non-zero.

Lesson learned - don't submit v3 of this patch until I've at least done 
better due diligence at running iotests on some of the impacted drivers.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH v2 for-4.0 00/13] block: byte-based blocking read/write

[no-reply]

Hi,

This series failed docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

Message-id: 20181115020334.1189829-1-eblake@redhat.com
Type: series
Subject: [Qemu-devel] [PATCH v2 for-4.0 00/13] block: byte-based blocking read/write

=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-mingw@fedora SHOW_ENV=1 J=8
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
2e66e20 block: Enforce non-zero bl.max_transfer
8b272d1 block: Document need for audit of read/write 64-bit cleanness
261da46 qcow2: Audit for read/write 64-bit cleanness
1ba6c76 file-posix: Audit for read/write 64-bit cleanness
00e8a09 RFC: crypto: Rely on block layer for fragmentation
be37a5a crypto: Audit for read/write 64-bit cleanness
b7af666 blklogwrites: Audit for read/write 64-bit cleanness
222d7a4 blkdebug: Audit for read/write 64-bit cleanness
21175b1 block: Switch to 64-bit bl.max_transfer
b472b9d block: Removed unused sector-based blocking I/O
dfb2b66 vvfat: Switch to byte-based calls
e47df95 vdi: Switch to byte-based calls
d7ab2c2 qcow2: Prefer byte-based calls into bs->file

=== OUTPUT BEGIN ===
  BUILD   fedora
make[1]: Entering directory `/var/tmp/patchew-tester-tmp-hw0lvnzz/src'
  GEN     /var/tmp/patchew-tester-tmp-hw0lvnzz/src/docker-src.2018-11-15-04.01.25.3134/qemu.tar
Cloning into '/var/tmp/patchew-tester-tmp-hw0lvnzz/src/docker-src.2018-11-15-04.01.25.3134/qemu.tar.vroot'...
done.
Checking out files:  46% (3007/6455)   
Checking out files:  47% (3034/6455)   
Checking out files:  48% (3099/6455)   
Checking out files:  49% (3163/6455)   
Checking out files:  50% (3228/6455)   
Checking out files:  51% (3293/6455)   
Checking out files:  52% (3357/6455)   
Checking out files:  53% (3422/6455)   
Checking out files:  54% (3486/6455)   
Checking out files:  55% (3551/6455)   
Checking out files:  56% (3615/6455)   
Checking out files:  57% (3680/6455)   
Checking out files:  58% (3744/6455)   
Checking out files:  59% (3809/6455)   
Checking out files:  60% (3873/6455)   
Checking out files:  61% (3938/6455)   
Checking out files:  62% (4003/6455)   
Checking out files:  63% (4067/6455)   
Checking out files:  64% (4132/6455)   
Checking out files:  65% (4196/6455)   
Checking out files:  66% (4261/6455)   
Checking out files:  67% (4325/6455)   
Checking out files:  68% (4390/6455)   
Checking out files:  69% (4454/6455)   
Checking out files:  70% (4519/6455)   
Checking out files:  71% (4584/6455)   
Checking out files:  72% (4648/6455)   
Checking out files:  73% (4713/6455)   
Checking out files:  74% (4777/6455)   
Checking out files:  75% (4842/6455)   
Checking out files:  76% (4906/6455)   
Checking out files:  77% (4971/6455)   
Checking out files:  78% (5035/6455)   
Checking out files:  79% (5100/6455)   
Checking out files:  80% (5164/6455)   
Checking out files:  81% (5229/6455)   
Checking out files:  82% (5294/6455)   
Checking out files:  83% (5358/6455)   
Checking out files:  84% (5423/6455)   
Checking out files:  85% (5487/6455)   
Checking out files:  86% (5552/6455)   
Checking out files:  87% (5616/6455)   
Checking out files:  88% (5681/6455)   
Checking out files:  89% (5745/6455)   
Checking out files:  90% (5810/6455)   
Checking out files:  91% (5875/6455)   
Checking out files:  92% (5939/6455)   
Checking out files:  93% (6004/6455)   
Checking out files:  94% (6068/6455)   
Checking out files:  95% (6133/6455)   
Checking out files:  96% (6197/6455)   
Checking out files:  97% (6262/6455)   
Checking out files:  98% (6326/6455)   
Checking out files:  99% (6391/6455)   
Checking out files: 100% (6455/6455)   
Checking out files: 100% (6455/6455), done.
Submodule 'dtc' (https://git.qemu.org/git/dtc.git) registered for path 'dtc'
Cloning into 'dtc'...
Submodule path 'dtc': checked out '88f18909db731a627456f26d779445f84e449536'
Submodule 'ui/keycodemapdb' (https://git.qemu.org/git/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Cloning into 'ui/keycodemapdb'...
Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce'
  COPY    RUNNER
    RUN test-mingw in qemu:fedora 
Packages installed:
SDL2-devel-2.0.9-1.fc28.x86_64
bc-1.07.1-5.fc28.x86_64
bison-3.0.4-9.fc28.x86_64
bluez-libs-devel-5.50-1.fc28.x86_64
brlapi-devel-0.6.7-19.fc28.x86_64
bzip2-1.0.6-26.fc28.x86_64
bzip2-devel-1.0.6-26.fc28.x86_64
ccache-3.4.2-2.fc28.x86_64
clang-6.0.1-2.fc28.x86_64
device-mapper-multipath-devel-0.7.4-3.git07e7bd5.fc28.x86_64
findutils-4.6.0-19.fc28.x86_64
flex-2.6.1-7.fc28.x86_64
gcc-8.2.1-5.fc28.x86_64
gcc-c++-8.2.1-5.fc28.x86_64
gettext-0.19.8.1-14.fc28.x86_64
git-2.17.2-1.fc28.x86_64
glib2-devel-2.56.3-2.fc28.x86_64
glusterfs-api-devel-4.1.5-1.fc28.x86_64
gnutls-devel-3.6.4-1.fc28.x86_64
gtk3-devel-3.22.30-1.fc28.x86_64
hostname-3.20-3.fc28.x86_64
libaio-devel-0.3.110-11.fc28.x86_64
libasan-8.2.1-5.fc28.x86_64
libattr-devel-2.4.48-3.fc28.x86_64
libcap-devel-2.25-9.fc28.x86_64
libcap-ng-devel-0.7.9-4.fc28.x86_64
libcurl-devel-7.59.0-8.fc28.x86_64
libfdt-devel-1.4.7-1.fc28.x86_64
libpng-devel-1.6.34-6.fc28.x86_64
librbd-devel-12.2.8-1.fc28.x86_64
libssh2-devel-1.8.0-7.fc28.x86_64
libubsan-8.2.1-5.fc28.x86_64
libusbx-devel-1.0.22-1.fc28.x86_64
libxml2-devel-2.9.8-4.fc28.x86_64
llvm-6.0.1-8.fc28.x86_64
lzo-devel-2.08-12.fc28.x86_64
make-4.2.1-6.fc28.x86_64
mingw32-SDL2-2.0.9-1.fc28.noarch
mingw32-bzip2-1.0.6-9.fc27.noarch
mingw32-curl-7.57.0-1.fc28.noarch
mingw32-glib2-2.56.1-1.fc28.noarch
mingw32-gmp-6.1.2-2.fc27.noarch
mingw32-gnutls-3.6.3-1.fc28.noarch
mingw32-gtk3-3.22.30-1.fc28.noarch
mingw32-libjpeg-turbo-1.5.1-3.fc27.noarch
mingw32-libpng-1.6.29-2.fc27.noarch
mingw32-libssh2-1.8.0-3.fc27.noarch
mingw32-libtasn1-4.13-1.fc28.noarch
mingw32-nettle-3.4-1.fc28.noarch
mingw32-pixman-0.34.0-3.fc27.noarch
mingw32-pkg-config-0.28-9.fc27.x86_64
mingw64-SDL2-2.0.9-1.fc28.noarch
mingw64-bzip2-1.0.6-9.fc27.noarch
mingw64-curl-7.57.0-1.fc28.noarch
mingw64-glib2-2.56.1-1.fc28.noarch
mingw64-gmp-6.1.2-2.fc27.noarch
mingw64-gnutls-3.6.3-1.fc28.noarch
mingw64-gtk3-3.22.30-1.fc28.noarch
mingw64-libjpeg-turbo-1.5.1-3.fc27.noarch
mingw64-libpng-1.6.29-2.fc27.noarch
mingw64-libssh2-1.8.0-3.fc27.noarch
mingw64-libtasn1-4.13-1.fc28.noarch
mingw64-nettle-3.4-1.fc28.noarch
mingw64-pixman-0.34.0-3.fc27.noarch
mingw64-pkg-config-0.28-9.fc27.x86_64
ncurses-devel-6.1-5.20180224.fc28.x86_64
nettle-devel-3.4-2.fc28.x86_64
nss-devel-3.39.0-1.0.fc28.x86_64
numactl-devel-2.0.11-8.fc28.x86_64
package PyYAML is not installed
package libjpeg-devel is not installed
perl-5.26.2-414.fc28.x86_64
pixman-devel-0.34.0-8.fc28.x86_64
python3-3.6.6-1.fc28.x86_64
snappy-devel-1.1.7-5.fc28.x86_64
sparse-0.5.2-1.fc28.x86_64
spice-server-devel-0.14.0-4.fc28.x86_64
systemtap-sdt-devel-4.0-1.fc28.x86_64
tar-1.30-3.fc28.x86_64
usbredir-devel-0.8.0-1.fc28.x86_64
virglrenderer-devel-0.6.0-4.20170210git76b3da97b.fc28.x86_64
vte3-devel-0.36.5-6.fc28.x86_64
which-2.21-8.fc28.x86_64
xen-devel-4.10.2-2.fc28.x86_64
zlib-devel-1.2.11-8.fc28.x86_64

Environment variables:
TARGET_LIST=
PACKAGES=bc     bison     bluez-libs-devel     brlapi-devel     bzip2     bzip2-devel     ccache     clang     device-mapper-multipath-devel     findutils     flex     gcc     gcc-c++     gettext     git     glib2-devel     glusterfs-api-devel     gnutls-devel     gtk3-devel     hostname     libaio-devel     libasan     libattr-devel     libcap-devel     libcap-ng-devel     libcurl-devel     libfdt-devel     libjpeg-devel     libpng-devel     librbd-devel     libssh2-devel     libubsan     libusbx-devel     libxml2-devel     llvm     lzo-devel     make     mingw32-bzip2     mingw32-curl     mingw32-glib2     mingw32-gmp     mingw32-gnutls     mingw32-gtk3     mingw32-libjpeg-turbo     mingw32-libpng     mingw32-libssh2     mingw32-libtasn1     mingw32-nettle     mingw32-pixman     mingw32-pkg-config     mingw32-SDL2     mingw64-bzip2     mingw64-curl     mingw64-glib2     mingw64-gmp     mingw64-gnutls     mingw64-gtk3     mingw64-libjpeg-turbo     mingw64-libpng     mingw64-libssh2     mingw64-libtasn1     mingw64-nettle     mingw64-pixman     mingw64-pkg-config     mingw64-SDL2     ncurses-devel     nettle-devel     nss-devel     numactl-devel     perl     pixman-devel     python3     PyYAML     SDL2-devel     snappy-devel     sparse     spice-server-devel     systemtap-sdt-devel     tar     usbredir-devel     virglrenderer-devel     vte3-devel     which     xen-devel     zlib-devel
J=8
V=
HOSTNAME=ecb9bddba6bc
DEBUG=
SHOW_ENV=1
PWD=/
HOME=/
CCACHE_DIR=/var/tmp/ccache
FBR=f28
DISTTAG=f28container
QEMU_CONFIGURE_OPTS=--python=/usr/bin/python3
FGC=f28
TEST_DIR=/tmp/qemu-test
SHLVL=1
FEATURES=mingw clang pyyaml asan dtc
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
MAKEFLAGS= -j8
EXTRA_CONFIGURE_OPTS=
_=/usr/bin/env

Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/install --python=/usr/bin/python3 --cross-prefix=x86_64-w64-mingw32- --enable-trace-backends=simple --enable-gnutls --enable-nettle --enable-curl --enable-vnc --enable-bzip2 --enable-guest-agent --with-sdlabi=2.0
Install prefix    /tmp/qemu-test/install
BIOS directory    /tmp/qemu-test/install
firmware path     /tmp/qemu-test/install/share/qemu-firmware
binary directory  /tmp/qemu-test/install
library directory /tmp/qemu-test/install/lib
module directory  /tmp/qemu-test/install/lib
libexec directory /tmp/qemu-test/install/libexec
include directory /tmp/qemu-test/install/include
config directory  /tmp/qemu-test/install
local state directory   queried at runtime
Windows SDK       no
Source path       /tmp/qemu-test/src
GIT binary        git
GIT submodules    
C compiler        x86_64-w64-mingw32-gcc
Host C compiler   cc
C++ compiler      x86_64-w64-mingw32-g++
Objective-C compiler clang
ARFLAGS           rv
CFLAGS            -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g 
QEMU_CFLAGS       -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/pixman-1  -I$(SRC_PATH)/dtc/libfdt -Werror -DHAS_LIBSSH2_SFTP_FSYNC -I/usr/x86_64-w64-mingw32/sys-root/mingw/include  -mms-bitfields -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/glib-2.0 -I/usr/x86_64-w64-mingw32/sys-root/mingw/lib/glib-2.0/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include  -m64 -mcx16 -mthreads -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv  -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/p11-kit-1  -I/usr/x86_64-w64-mingw32/sys-root/mingw/include   -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/libpng16 
LDFLAGS           -Wl,--nxcompat -Wl,--no-seh -Wl,--dynamicbase -Wl,--warn-common -m64 -g 
QEMU_LDFLAGS      -L$(BUILD_DIR)/dtc/libfdt 
make              make
install           install
python            /usr/bin/python3 -B
smbd              /usr/sbin/smbd
module support    no
host CPU          x86_64
host big endian   no
target list       x86_64-softmmu aarch64-softmmu
gprof enabled     no
sparse enabled    no
strip binaries    yes
profiler          no
static build      no
SDL support       yes (2.0.9)
GTK support       yes (3.22.30)
GTK GL support    no
VTE support       no 
TLS priority      NORMAL
GNUTLS support    yes
libgcrypt         no
nettle            yes (3.4)
libtasn1          yes
curses support    no
virgl support     no 
curl support      yes
mingw32 support   yes
Audio drivers     dsound
Block whitelist (rw) 
Block whitelist (ro) 
VirtFS support    no
Multipath support no
VNC support       yes
VNC SASL support  no
VNC JPEG support  yes
VNC PNG support   yes
xen support       no
brlapi support    no
bluez  support    no
Documentation     no
PIE               no
vde support       no
netmap support    no
Linux AIO support no
ATTR/XATTR support no
Install blobs     yes
KVM support       no
HAX support       yes
HVF support       no
WHPX support      no
TCG support       yes
TCG debug enabled no
TCG interpreter   no
malloc trim support no
RDMA support      no
PVRDMA support    no
fdt support       git
membarrier        no
preadv support    no
fdatasync         no
madvise           no
posix_madvise     no
posix_memalign    no
libcap-ng support no
vhost-net support no
vhost-crypto support no
vhost-scsi support no
vhost-vsock support no
vhost-user support no
Trace backends    simple
Trace output file trace-<pid>
spice support     no 
rbd support       no
xfsctl support    no
smartcard support no
libusb            no
usb net redir     no
OpenGL support    no
OpenGL dmabufs    no
libiscsi support  no
libnfs support    no
build guest agent yes
QGA VSS support   no
QGA w32 disk info yes
QGA MSI support   no
seccomp support   no
coroutine backend win32
coroutine pool    yes
debug stack usage no
mutex debugging   no
crypto afalg      no
GlusterFS support no
gcov              gcov
gcov enabled      no
TPM support       yes
libssh2 support   yes
TPM passthrough   no
TPM emulator      no
QOM debugging     yes
Live block migration yes
lzo support       no
snappy support    no
bzip2 support     yes
NUMA host support no
libxml2           no
tcmalloc support  no
jemalloc support  no
avx2 optimization yes
replication support yes
VxHS block device no
bochs support     yes
cloop support     yes
dmg support       yes
qcow v1 support   yes
vdi support       yes
vvfat support     yes
qed support       yes
parallels support yes
sheepdog support  yes
capstone          no
docker            no
libpmem support   no
libudev           no

NOTE: cross-compilers enabled:  'x86_64-w64-mingw32-gcc'
  GEN     config-host.h
  GEN     x86_64-softmmu/config-devices.mak.tmp
  GEN     aarch64-softmmu/config-devices.mak.tmp
  GEN     qemu-options.def
  GEN     qapi-gen
  GEN     trace/generated-helpers-wrappers.h
  GEN     trace/generated-tcg-tracers.h
  GEN     trace/generated-helpers.h
  GEN     trace/generated-helpers.c
  GEN     x86_64-softmmu/config-devices.mak
  GEN     aarch64-softmmu/config-devices.mak
  GEN     module_block.h
  GEN     ui/input-keymap-atset1-to-qcode.c
  GEN     ui/input-keymap-linux-to-qcode.c
  GEN     ui/input-keymap-qcode-to-atset1.c
  GEN     ui/input-keymap-qcode-to-atset2.c
  GEN     ui/input-keymap-qcode-to-atset3.c
  GEN     ui/input-keymap-qcode-to-linux.c
  GEN     ui/input-keymap-qcode-to-qnum.c
  GEN     ui/input-keymap-qcode-to-sun.c
  GEN     ui/input-keymap-qnum-to-qcode.c
  GEN     ui/input-keymap-usb-to-qcode.c
  GEN     ui/input-keymap-win32-to-qcode.c
  GEN     ui/input-keymap-x11-to-qcode.c
  GEN     ui/input-keymap-xorgevdev-to-qcode.c
  GEN     ui/input-keymap-xorgkbd-to-qcode.c
  GEN     ui/input-keymap-xorgxquartz-to-qcode.c
  GEN     ui/input-keymap-xorgxwin-to-qcode.c
  GEN     ui/input-keymap-osx-to-qcode.c
  GEN     tests/test-qapi-gen
  GEN     trace-root.h
  GEN     accel/kvm/trace.h
  GEN     accel/tcg/trace.h
  GEN     audio/trace.h
  GEN     block/trace.h
  GEN     chardev/trace.h
  GEN     crypto/trace.h
  GEN     hw/9pfs/trace.h
  GEN     hw/acpi/trace.h
  GEN     hw/alpha/trace.h
  GEN     hw/arm/trace.h
  GEN     hw/audio/trace.h
  GEN     hw/block/trace.h
  GEN     hw/block/dataplane/trace.h
  GEN     hw/char/trace.h
  GEN     hw/display/trace.h
  GEN     hw/dma/trace.h
  GEN     hw/hppa/trace.h
  GEN     hw/i2c/trace.h
  GEN     hw/i386/trace.h
  GEN     hw/i386/xen/trace.h
  GEN     hw/ide/trace.h
  GEN     hw/input/trace.h
  GEN     hw/intc/trace.h
  GEN     hw/isa/trace.h
  GEN     hw/mem/trace.h
  GEN     hw/misc/trace.h
  GEN     hw/misc/macio/trace.h
  GEN     hw/net/trace.h
  GEN     hw/nvram/trace.h
  GEN     hw/pci/trace.h
  GEN     hw/pci-host/trace.h
  GEN     hw/ppc/trace.h
  GEN     hw/rdma/trace.h
  GEN     hw/rdma/vmw/trace.h
  GEN     hw/s390x/trace.h
  GEN     hw/scsi/trace.h
  GEN     hw/sd/trace.h
  GEN     hw/sparc/trace.h
  GEN     hw/sparc64/trace.h
  GEN     hw/timer/trace.h
  GEN     hw/tpm/trace.h
  GEN     hw/usb/trace.h
  GEN     hw/vfio/trace.h
  GEN     hw/virtio/trace.h
  GEN     hw/watchdog/trace.h
  GEN     hw/xen/trace.h
  GEN     io/trace.h
  GEN     linux-user/trace.h
  GEN     migration/trace.h
  GEN     nbd/trace.h
  GEN     net/trace.h
  GEN     qapi/trace.h
  GEN     qom/trace.h
  GEN     scsi/trace.h
  GEN     target/arm/trace.h
  GEN     target/i386/trace.h
  GEN     target/mips/trace.h
  GEN     target/ppc/trace.h
  GEN     target/s390x/trace.h
  GEN     target/sparc/trace.h
  GEN     ui/trace.h
  GEN     util/trace.h
  GEN     trace-root.c
  GEN     accel/kvm/trace.c
  GEN     accel/tcg/trace.c
  GEN     audio/trace.c
  GEN     block/trace.c
  GEN     chardev/trace.c
  GEN     crypto/trace.c
  GEN     hw/9pfs/trace.c
  GEN     hw/acpi/trace.c
  GEN     hw/alpha/trace.c
  GEN     hw/arm/trace.c
  GEN     hw/audio/trace.c
  GEN     hw/block/trace.c
  GEN     hw/block/dataplane/trace.c
  GEN     hw/char/trace.c
  GEN     hw/display/trace.c
  GEN     hw/dma/trace.c
  GEN     hw/hppa/trace.c
  GEN     hw/i2c/trace.c
  GEN     hw/i386/trace.c
  GEN     hw/i386/xen/trace.c
  GEN     hw/ide/trace.c
  GEN     hw/input/trace.c
  GEN     hw/intc/trace.c
  GEN     hw/isa/trace.c
  GEN     hw/mem/trace.c
  GEN     hw/misc/trace.c
  GEN     hw/misc/macio/trace.c
  GEN     hw/net/trace.c
  GEN     hw/nvram/trace.c
  GEN     hw/pci/trace.c
  GEN     hw/pci-host/trace.c
  GEN     hw/ppc/trace.c
  GEN     hw/rdma/trace.c
  GEN     hw/rdma/vmw/trace.c
  GEN     hw/s390x/trace.c
  GEN     hw/scsi/trace.c
  GEN     hw/sd/trace.c
  GEN     hw/sparc/trace.c
  GEN     hw/sparc64/trace.c
  GEN     hw/timer/trace.c
  GEN     hw/tpm/trace.c
  GEN     hw/usb/trace.c
  GEN     hw/vfio/trace.c
  GEN     hw/virtio/trace.c
  GEN     hw/watchdog/trace.c
  GEN     hw/xen/trace.c
  GEN     io/trace.c
  GEN     linux-user/trace.c
  GEN     migration/trace.c
  GEN     nbd/trace.c
  GEN     net/trace.c
  GEN     qapi/trace.c
  GEN     qom/trace.c
  GEN     scsi/trace.c
  GEN     target/arm/trace.c
  GEN     target/i386/trace.c
  GEN     target/mips/trace.c
  GEN     target/ppc/trace.c
  GEN     target/s390x/trace.c
  GEN     target/sparc/trace.c
  GEN     ui/trace.c
  GEN     util/trace.c
  GEN     config-all-devices.mak
	 DEP /tmp/qemu-test/src/dtc/tests/dumptrees.c
	 DEP /tmp/qemu-test/src/dtc/tests/trees.S
	 DEP /tmp/qemu-test/src/dtc/tests/testutils.c
	 DEP /tmp/qemu-test/src/dtc/tests/value-labels.c
	 DEP /tmp/qemu-test/src/dtc/tests/asm_tree_dump.c
	 DEP /tmp/qemu-test/src/dtc/tests/truncated_memrsv.c
	 DEP /tmp/qemu-test/src/dtc/tests/truncated_string.c
	 DEP /tmp/qemu-test/src/dtc/tests/truncated_property.c
	 DEP /tmp/qemu-test/src/dtc/tests/check_full.c
	 DEP /tmp/qemu-test/src/dtc/tests/check_header.c
	 DEP /tmp/qemu-test/src/dtc/tests/check_path.c
	 DEP /tmp/qemu-test/src/dtc/tests/overlay_bad_fixup.c
	 DEP /tmp/qemu-test/src/dtc/tests/overlay.c
	 DEP /tmp/qemu-test/src/dtc/tests/subnode_iterate.c
	 DEP /tmp/qemu-test/src/dtc/tests/property_iterate.c
	 DEP /tmp/qemu-test/src/dtc/tests/integer-expressions.c
	 DEP /tmp/qemu-test/src/dtc/tests/utilfdt_test.c
	 DEP /tmp/qemu-test/src/dtc/tests/path_offset_aliases.c
	 DEP /tmp/qemu-test/src/dtc/tests/add_subnode_with_nops.c
	 DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_unordered.c
	 DEP /tmp/qemu-test/src/dtc/tests/dtb_reverse.c
	 DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_ordered.c
	 DEP /tmp/qemu-test/src/dtc/tests/extra-terminating-null.c
	 DEP /tmp/qemu-test/src/dtc/tests/incbin.c
	 DEP /tmp/qemu-test/src/dtc/tests/boot-cpuid.c
	 DEP /tmp/qemu-test/src/dtc/tests/phandle_format.c
	 DEP /tmp/qemu-test/src/dtc/tests/path-references.c
	 DEP /tmp/qemu-test/src/dtc/tests/references.c
	 DEP /tmp/qemu-test/src/dtc/tests/string_escapes.c
	 DEP /tmp/qemu-test/src/dtc/tests/propname_escapes.c
	 DEP /tmp/qemu-test/src/dtc/tests/appendprop2.c
	 DEP /tmp/qemu-test/src/dtc/tests/del_node.c
	 DEP /tmp/qemu-test/src/dtc/tests/appendprop1.c
	 DEP /tmp/qemu-test/src/dtc/tests/del_property.c
	 DEP /tmp/qemu-test/src/dtc/tests/setprop.c
	 DEP /tmp/qemu-test/src/dtc/tests/set_name.c
	 DEP /tmp/qemu-test/src/dtc/tests/rw_tree1.c
	 DEP /tmp/qemu-test/src/dtc/tests/open_pack.c
	 DEP /tmp/qemu-test/src/dtc/tests/nopulate.c
	 DEP /tmp/qemu-test/src/dtc/tests/mangle-layout.c
	 DEP /tmp/qemu-test/src/dtc/tests/move_and_save.c
	 DEP /tmp/qemu-test/src/dtc/tests/sw_states.c
	 DEP /tmp/qemu-test/src/dtc/tests/sw_tree1.c
	 DEP /tmp/qemu-test/src/dtc/tests/nop_node.c
	 DEP /tmp/qemu-test/src/dtc/tests/nop_property.c
	 DEP /tmp/qemu-test/src/dtc/tests/setprop_inplace.c
	 DEP /tmp/qemu-test/src/dtc/tests/stringlist.c
	 DEP /tmp/qemu-test/src/dtc/tests/addr_size_cells2.c
	 DEP /tmp/qemu-test/src/dtc/tests/addr_size_cells.c
	 DEP /tmp/qemu-test/src/dtc/tests/notfound.c
	 DEP /tmp/qemu-test/src/dtc/tests/sized_cells.c
	 DEP /tmp/qemu-test/src/dtc/tests/char_literal.c
	 DEP /tmp/qemu-test/src/dtc/tests/get_alias.c
	 DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_compatible.c
	 DEP /tmp/qemu-test/src/dtc/tests/node_check_compatible.c
	 DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_phandle.c
	 DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_prop_value.c
	 DEP /tmp/qemu-test/src/dtc/tests/parent_offset.c
	 DEP /tmp/qemu-test/src/dtc/tests/supernode_atdepth_offset.c
	 DEP /tmp/qemu-test/src/dtc/tests/get_path.c
	 DEP /tmp/qemu-test/src/dtc/tests/get_phandle.c
	 DEP /tmp/qemu-test/src/dtc/tests/getprop.c
	 DEP /tmp/qemu-test/src/dtc/tests/get_name.c
	 DEP /tmp/qemu-test/src/dtc/tests/path_offset.c
	 DEP /tmp/qemu-test/src/dtc/tests/subnode_offset.c
	 DEP /tmp/qemu-test/src/dtc/tests/find_property.c
	 DEP /tmp/qemu-test/src/dtc/tests/root_node.c
	 DEP /tmp/qemu-test/src/dtc/tests/get_mem_rsv.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_overlay.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_addresses.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_empty_tree.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_strerror.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_rw.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_sw.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_wip.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt_ro.c
	 DEP /tmp/qemu-test/src/dtc/libfdt/fdt.c
	 DEP /tmp/qemu-test/src/dtc/util.c
	 DEP /tmp/qemu-test/src/dtc/fdtoverlay.c
	 DEP /tmp/qemu-test/src/dtc/fdtput.c
	 DEP /tmp/qemu-test/src/dtc/fdtget.c
	 DEP /tmp/qemu-test/src/dtc/fdtdump.c
	 LEX convert-dtsv0-lexer.lex.c
	 DEP /tmp/qemu-test/src/dtc/srcpos.c
	 BISON dtc-parser.tab.c
	 LEX dtc-lexer.lex.c
	 DEP /tmp/qemu-test/src/dtc/treesource.c
	 DEP /tmp/qemu-test/src/dtc/livetree.c
	 DEP /tmp/qemu-test/src/dtc/fstree.c
	 DEP /tmp/qemu-test/src/dtc/flattree.c
	 DEP /tmp/qemu-test/src/dtc/dtc.c
	 DEP /tmp/qemu-test/src/dtc/data.c
	 DEP /tmp/qemu-test/src/dtc/checks.c
	 DEP convert-dtsv0-lexer.lex.c
	 DEP dtc-parser.tab.c
	 DEP dtc-lexer.lex.c
	CHK version_gen.h
	UPD version_gen.h
	 DEP /tmp/qemu-test/src/dtc/util.c
	 CC libfdt/fdt.o
	 CC libfdt/fdt_ro.o
	 CC libfdt/fdt_wip.o
	 CC libfdt/fdt_sw.o
	 CC libfdt/fdt_rw.o
	 CC libfdt/fdt_strerror.o
	 CC libfdt/fdt_empty_tree.o
	 CC libfdt/fdt_addresses.o
	 CC libfdt/fdt_overlay.o
	 AR libfdt/libfdt.a
x86_64-w64-mingw32-ar: creating libfdt/libfdt.a
a - libfdt/fdt.o
a - libfdt/fdt_ro.o
a - libfdt/fdt_wip.o
a - libfdt/fdt_sw.o
a - libfdt/fdt_rw.o
a - libfdt/fdt_strerror.o
a - libfdt/fdt_empty_tree.o
a - libfdt/fdt_addresses.o
a - libfdt/fdt_overlay.o
  RC      version.o
  GEN     qga/qapi-generated/qapi-gen
  CC      qapi/qapi-types.o
  CC      qapi/qapi-builtin-types.o
  CC      qapi/qapi-types-block-core.o
  CC      qapi/qapi-types-block.o
  CC      qapi/qapi-types-char.o
  CC      qapi/qapi-types-common.o
  CC      qapi/qapi-types-crypto.o
  CC      qapi/qapi-types-introspect.o
  CC      qapi/qapi-types-job.o
  CC      qapi/qapi-types-migration.o
  CC      qapi/qapi-types-misc.o
  CC      qapi/qapi-types-net.o
  CC      qapi/qapi-types-rocker.o
  CC      qapi/qapi-types-run-state.o
  CC      qapi/qapi-types-sockets.o
  CC      qapi/qapi-types-tpm.o
  CC      qapi/qapi-types-trace.o
  CC      qapi/qapi-types-transaction.o
  CC      qapi/qapi-types-ui.o
  CC      qapi/qapi-builtin-visit.o
  CC      qapi/qapi-visit.o
  CC      qapi/qapi-visit-block-core.o
  CC      qapi/qapi-visit-block.o
  CC      qapi/qapi-visit-char.o
  CC      qapi/qapi-visit-common.o
  CC      qapi/qapi-visit-crypto.o
  CC      qapi/qapi-visit-introspect.o
  CC      qapi/qapi-visit-job.o
  CC      qapi/qapi-visit-migration.o
  CC      qapi/qapi-visit-misc.o
  CC      qapi/qapi-visit-net.o
  CC      qapi/qapi-visit-rocker.o
  CC      qapi/qapi-visit-run-state.o
  CC      qapi/qapi-visit-sockets.o
  CC      qapi/qapi-visit-tpm.o
  CC      qapi/qapi-visit-trace.o
  CC      qapi/qapi-visit-transaction.o
  CC      qapi/qapi-visit-ui.o
  CC      qapi/qapi-events.o
  CC      qapi/qapi-events-block-core.o
  CC      qapi/qapi-events-block.o
  CC      qapi/qapi-events-char.o
  CC      qapi/qapi-events-common.o
  CC      qapi/qapi-events-crypto.o
  CC      qapi/qapi-events-introspect.o
  CC      qapi/qapi-events-job.o
  CC      qapi/qapi-events-migration.o
  CC      qapi/qapi-events-misc.o
  CC      qapi/qapi-events-net.o
  CC      qapi/qapi-events-rocker.o
  CC      qapi/qapi-events-run-state.o
  CC      qapi/qapi-events-sockets.o
  CC      qapi/qapi-events-tpm.o
  CC      qapi/qapi-events-trace.o
  CC      qapi/qapi-events-transaction.o
  CC      qapi/qapi-events-ui.o
  CC      qapi/qapi-introspect.o
  CC      qapi/qapi-visit-core.o
  CC      qapi/qapi-dealloc-visitor.o
  CC      qapi/qobject-input-visitor.o
  CC      qapi/qobject-output-visitor.o
  CC      qapi/qmp-registry.o
  CC      qapi/qmp-dispatch.o
  CC      qapi/string-input-visitor.o
  CC      qapi/string-output-visitor.o
  CC      qapi/opts-visitor.o
  CC      qapi/qapi-clone-visitor.o
  CC      qapi/qmp-event.o
  CC      qapi/qapi-util.o
  CC      qobject/qnull.o
  CC      qobject/qnum.o
  CC      qobject/qstring.o
  CC      qobject/qlist.o
  CC      qobject/qdict.o
  CC      qobject/qbool.o
  CC      qobject/qlit.o
  CC      qobject/qjson.o
  CC      qobject/qobject.o
  CC      qobject/json-lexer.o
  CC      qobject/json-streamer.o
  CC      qobject/block-qdict.o
  CC      qobject/json-parser.o
  CC      trace/simple.o
  CC      trace/control.o
  CC      trace/qmp.o
  CC      util/osdep.o
  CC      util/cutils.o
  CC      util/unicode.o
  CC      util/qemu-timer-common.o
  CC      util/bufferiszero.o
  CC      util/lockcnt.o
  CC      util/aiocb.o
  CC      util/async.o
  CC      util/aio-wait.o
  CC      util/thread-pool.o
  CC      util/qemu-timer.o
  CC      util/main-loop.o
  CC      util/iohandler.o
  CC      util/aio-win32.o
  CC      util/event_notifier-win32.o
  CC      util/oslib-win32.o
  CC      util/qemu-thread-win32.o
  CC      util/envlist.o
  CC      util/path.o
  CC      util/module.o
  CC      util/host-utils.o
  CC      util/bitmap.o
  CC      util/bitops.o
  CC      util/hbitmap.o
  CC      util/fifo8.o
  CC      util/acl.o
  CC      util/cacheinfo.o
  CC      util/error.o
  CC      util/qemu-error.o
  CC      util/id.o
  CC      util/iov.o
  CC      util/qemu-config.o
  CC      util/qemu-sockets.o
  CC      util/uri.o
  CC      util/notify.o
  CC      util/qemu-option.o
  CC      util/qemu-progress.o
  CC      util/keyval.o
  CC      util/hexdump.o
  CC      util/crc32c.o
  CC      util/getauxval.o
  CC      util/uuid.o
  CC      util/throttle.o
  CC      util/readline.o
  CC      util/rcu.o
  CC      util/qemu-coroutine.o
  CC      util/qemu-coroutine-lock.o
  CC      util/qemu-coroutine-io.o
  CC      util/qemu-coroutine-sleep.o
  CC      util/coroutine-win32.o
  CC      util/buffer.o
  CC      util/timed-average.o
  CC      util/base64.o
  CC      util/log.o
  CC      util/pagesize.o
  CC      util/qdist.o
  CC      util/qht.o
  CC      util/qsp.o
  CC      util/range.o
  CC      util/stats64.o
  CC      util/systemd.o
  CC      util/iova-tree.o
  CC      trace-root.o
  CC      accel/kvm/trace.o
  CC      accel/tcg/trace.o
  CC      audio/trace.o
  CC      block/trace.o
  CC      chardev/trace.o
  CC      crypto/trace.o
  CC      hw/9pfs/trace.o
  CC      hw/acpi/trace.o
  CC      hw/alpha/trace.o
  CC      hw/arm/trace.o
  CC      hw/audio/trace.o
  CC      hw/block/trace.o
  CC      hw/block/dataplane/trace.o
  CC      hw/display/trace.o
  CC      hw/char/trace.o
  CC      hw/dma/trace.o
  CC      hw/hppa/trace.o
  CC      hw/i2c/trace.o
  CC      hw/i386/trace.o
  CC      hw/i386/xen/trace.o
  CC      hw/ide/trace.o
  CC      hw/input/trace.o
  CC      hw/intc/trace.o
  CC      hw/isa/trace.o
  CC      hw/mem/trace.o
  CC      hw/misc/trace.o
  CC      hw/misc/macio/trace.o
  CC      hw/net/trace.o
  CC      hw/nvram/trace.o
  CC      hw/pci-host/trace.o
  CC      hw/pci/trace.o
  CC      hw/ppc/trace.o
  CC      hw/rdma/trace.o
  CC      hw/rdma/vmw/trace.o
  CC      hw/s390x/trace.o
  CC      hw/scsi/trace.o
  CC      hw/sd/trace.o
  CC      hw/sparc/trace.o
  CC      hw/timer/trace.o
  CC      hw/sparc64/trace.o
  CC      hw/tpm/trace.o
  CC      hw/usb/trace.o
  CC      hw/vfio/trace.o
  CC      hw/virtio/trace.o
  CC      hw/watchdog/trace.o
  CC      hw/xen/trace.o
  CC      io/trace.o
  CC      linux-user/trace.o
  CC      migration/trace.o
  CC      nbd/trace.o
  CC      net/trace.o
  CC      qapi/trace.o
  CC      qom/trace.o
  CC      scsi/trace.o
  CC      target/arm/trace.o
  CC      target/i386/trace.o
  CC      target/mips/trace.o
  CC      target/ppc/trace.o
  CC      target/s390x/trace.o
  CC      target/sparc/trace.o
  CC      ui/trace.o
  CC      util/trace.o
  CC      crypto/pbkdf-stub.o
  CC      stubs/arch-query-cpu-def.o
  CC      stubs/arch-query-cpu-model-expansion.o
  CC      stubs/arch-query-cpu-model-comparison.o
  CC      stubs/arch-query-cpu-model-baseline.o
  CC      stubs/bdrv-next-monitor-owned.o
  CC      stubs/blk-commit-all.o
  CC      stubs/blockdev-close-all-bdrv-states.o
  CC      stubs/clock-warp.o
  CC      stubs/cpu-get-clock.o
  CC      stubs/cpu-get-icount.o
  CC      stubs/dump.o
  CC      stubs/fdset.o
  CC      stubs/error-printf.o
  CC      stubs/gdbstub.o
  CC      stubs/get-vm-name.o
  CC      stubs/iothread.o
  CC      stubs/iothread-lock.o
  CC      stubs/is-daemonized.o
  CC      stubs/machine-init-done.o
  CC      stubs/migr-blocker.o
  CC      stubs/change-state-handler.o
  CC      stubs/monitor.o
  CC      stubs/notify-event.o
  CC      stubs/qtest.o
  CC      stubs/replay.o
  CC      stubs/runstate-check.o
  CC      stubs/set-fd-handler.o
  CC      stubs/sysbus.o
  CC      stubs/slirp.o
  CC      stubs/tpm.o
  CC      stubs/trace-control.o
  CC      stubs/uuid.o
  CC      stubs/vm-stop.o
  CC      stubs/vmstate.o
  CC      stubs/fd-register.o
  CC      stubs/target-monitor-defs.o
  CC      stubs/qmp_memory_device.o
  CC      stubs/target-get-monitor-def.o
  CC      stubs/pc_madt_cpu_entry.o
  CC      stubs/vmgenid.o
  CC      stubs/xen-common.o
  CC      stubs/xen-hvm.o
  CC      stubs/pci-host-piix.o
  CC      stubs/ram-block.o
  CC      stubs/ramfb.o
  CC      block.o
  GEN     qemu-img-cmds.h
  CC      blockjob.o
  CC      job.o
  CC      qemu-io-cmds.o
  CC      replication.o
  CC      block/raw-format.o
  CC      block/vmdk.o
  CC      block/vpc.o
  CC      block/qcow.o
  CC      block/vdi.o
  CC      block/cloop.o
  CC      block/bochs.o
  CC      block/vvfat.o
  CC      block/dmg.o
  CC      block/qcow2.o
  CC      block/qcow2-refcount.o
  CC      block/qcow2-cluster.o
  CC      block/qcow2-snapshot.o
  CC      block/qcow2-cache.o
  CC      block/qcow2-bitmap.o
  CC      block/qed.o
  CC      block/qed-l2-cache.o
  CC      block/qed-table.o
/tmp/qemu-test/src/block/qcow2-cluster.c: In function 'qcow2_decompress_cluster':
/tmp/qemu-test/src/block/qcow2-cluster.c:1630:15: error: implicit declaration of function 'bdrv_read'; did you mean 'bdrv_pread'? [-Werror=implicit-function-declaration]
         ret = bdrv_read(bs->file, coffset >> 9, s->cluster_data,
               ^~~~~~~~~
               bdrv_pread
/tmp/qemu-test/src/block/qcow2-cluster.c:1630:15: error: nested extern declaration of 'bdrv_read' [-Werror=nested-externs]
cc1: all warnings being treated as errors
make: *** [/tmp/qemu-test/src/rules.mak:69: block/qcow2-cluster.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 563, in <module>
    sys.exit(main())
  File "./tests/docker/docker.py", line 560, in main
    return args.cmdobj.run(args, argv)
  File "./tests/docker/docker.py", line 306, in run
    return Docker().run(argv, args.keep, quiet=args.quiet)
  File "./tests/docker/docker.py", line 274, in run
    quiet=quiet)
  File "./tests/docker/docker.py", line 181, in _do_check
    return subprocess.check_call(self._command + cmd, **kwargs)
  File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=143d71fae8b511e88eaf68b59973b7d0', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '--net=none', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=8', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-hw0lvnzz/src/docker-src.2018-11-15-04.01.25.3134:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-hw0lvnzz/src'
make: *** [docker-run-test-mingw@fedora] Error 2

real	1m29.666s
user	0m17.289s
sys	0m3.398s
=== OUTPUT END ===

Test command exited with code: 2


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

Re: [Qemu-devel] [PATCH v2 for-4.0 00/13] block: byte-based blocking read/write

[Eric Blake]

On 11/15/18 3:02 AM, no-reply@patchew.org wrote:
> Hi,
> 
> This series failed docker-mingw@fedora build test. Please find the testing commands and
> their output below. If you have Docker installed, you can probably reproduce it
> locally.
> 
> Message-id: 20181115020334.1189829-1-eblake@redhat.com
> Type: series
> Subject: [Qemu-devel] [PATCH v2 for-4.0 00/13] block: byte-based blocking read/write
> 

> /tmp/qemu-test/src/block/qcow2-cluster.c: In function 'qcow2_decompress_cluster':
> /tmp/qemu-test/src/block/qcow2-cluster.c:1630:15: error: implicit declaration of function 'bdrv_read'; did you mean 'bdrv_pread'? [-Werror=implicit-function-declaration]
>           ret = bdrv_read(bs->file, coffset >> 9, s->cluster_data,
>                 ^~~~~~~~~
>                 bdrv_pread
> /tmp/qemu-test/src/block/qcow2-cluster.c:1630:15: error: nested extern declaration of 'bdrv_read' [-Werror=nested-externs]

False positive - for some reason, patchew didn't properly follow my 
Based-on tags that this depends on Kevin's block-next branch:

> Based-on: <20181114210548.1098207-1-eblake@redhat.com>
> [file-posix: Better checks of 64-bit copy_range]
> Based-on: <20181101182738.70462-1-vsementsov@virtuozzo.com>
> [0/7 qcow2 decompress in threads] - more specifically, on Kevin's block-next branch


-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH v2 for-4.0 00/13] block: byte-based blocking read/write

[no-reply]

Hi,

This series failed docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

Message-id: 20181115020334.1189829-1-eblake@redhat.com
Type: series
Subject: [Qemu-devel] [PATCH v2 for-4.0 00/13] block: byte-based blocking read/write

=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-quick@centos7 SHOW_ENV=1 J=8
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
2e66e20 block: Enforce non-zero bl.max_transfer
8b272d1 block: Document need for audit of read/write 64-bit cleanness
261da46 qcow2: Audit for read/write 64-bit cleanness
1ba6c76 file-posix: Audit for read/write 64-bit cleanness
00e8a09 RFC: crypto: Rely on block layer for fragmentation
be37a5a crypto: Audit for read/write 64-bit cleanness
b7af666 blklogwrites: Audit for read/write 64-bit cleanness
222d7a4 blkdebug: Audit for read/write 64-bit cleanness
21175b1 block: Switch to 64-bit bl.max_transfer
b472b9d block: Removed unused sector-based blocking I/O
dfb2b66 vvfat: Switch to byte-based calls
e47df95 vdi: Switch to byte-based calls
d7ab2c2 qcow2: Prefer byte-based calls into bs->file

=== OUTPUT BEGIN ===
  BUILD   centos7
make[1]: Entering directory `/var/tmp/patchew-tester-tmp-9bah4vqa/src'
  GEN     /var/tmp/patchew-tester-tmp-9bah4vqa/src/docker-src.2018-11-15-04.03.19.7513/qemu.tar
Cloning into '/var/tmp/patchew-tester-tmp-9bah4vqa/src/docker-src.2018-11-15-04.03.19.7513/qemu.tar.vroot'...
done.
Checking out files:  46% (3007/6455)   
Checking out files:  47% (3034/6455)   
Checking out files:  48% (3099/6455)   
Checking out files:  49% (3163/6455)   
Checking out files:  50% (3228/6455)   
Checking out files:  51% (3293/6455)   
Checking out files:  52% (3357/6455)   
Checking out files:  53% (3422/6455)   
Checking out files:  54% (3486/6455)   
Checking out files:  55% (3551/6455)   
Checking out files:  56% (3615/6455)   
Checking out files:  57% (3680/6455)   
Checking out files:  58% (3744/6455)   
Checking out files:  59% (3809/6455)   
Checking out files:  60% (3873/6455)   
Checking out files:  61% (3938/6455)   
Checking out files:  62% (4003/6455)   
Checking out files:  63% (4067/6455)   
Checking out files:  64% (4132/6455)   
Checking out files:  65% (4196/6455)   
Checking out files:  66% (4261/6455)   
Checking out files:  67% (4325/6455)   
Checking out files:  68% (4390/6455)   
Checking out files:  69% (4454/6455)   
Checking out files:  70% (4519/6455)   
Checking out files:  71% (4584/6455)   
Checking out files:  72% (4648/6455)   
Checking out files:  73% (4713/6455)   
Checking out files:  74% (4777/6455)   
Checking out files:  75% (4842/6455)   
Checking out files:  76% (4906/6455)   
Checking out files:  77% (4971/6455)   
Checking out files:  78% (5035/6455)   
Checking out files:  79% (5100/6455)   
Checking out files:  80% (5164/6455)   
Checking out files:  81% (5229/6455)   
Checking out files:  82% (5294/6455)   
Checking out files:  83% (5358/6455)   
Checking out files:  84% (5423/6455)   
Checking out files:  85% (5487/6455)   
Checking out files:  86% (5552/6455)   
Checking out files:  87% (5616/6455)   
Checking out files:  88% (5681/6455)   
Checking out files:  89% (5745/6455)   
Checking out files:  90% (5810/6455)   
Checking out files:  91% (5875/6455)   
Checking out files:  92% (5939/6455)   
Checking out files:  93% (6004/6455)   
Checking out files:  94% (6068/6455)   
Checking out files:  95% (6133/6455)   
Checking out files:  96% (6197/6455)   
Checking out files:  97% (6262/6455)   
Checking out files:  98% (6326/6455)   
Checking out files:  99% (6391/6455)   
Checking out files: 100% (6455/6455)   
Checking out files: 100% (6455/6455), done.
Submodule 'dtc' (https://git.qemu.org/git/dtc.git) registered for path 'dtc'
Cloning into 'dtc'...
Submodule path 'dtc': checked out '88f18909db731a627456f26d779445f84e449536'
Submodule 'ui/keycodemapdb' (https://git.qemu.org/git/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Cloning into 'ui/keycodemapdb'...
Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce'
  COPY    RUNNER
    RUN test-quick in qemu:centos7 
Packages installed:
SDL-devel-1.2.15-14.el7.x86_64
bison-3.0.4-1.el7.x86_64
bzip2-1.0.6-13.el7.x86_64
bzip2-devel-1.0.6-13.el7.x86_64
ccache-3.3.4-1.el7.x86_64
csnappy-devel-0-6.20150729gitd7bc683.el7.x86_64
flex-2.5.37-3.el7.x86_64
gcc-4.8.5-28.el7_5.1.x86_64
gettext-0.19.8.1-2.el7.x86_64
git-1.8.3.1-14.el7_5.x86_64
glib2-devel-2.54.2-2.el7.x86_64
libaio-devel-0.3.109-13.el7.x86_64
libepoxy-devel-1.3.1-2.el7_5.x86_64
libfdt-devel-1.4.6-1.el7.x86_64
lzo-devel-2.06-8.el7.x86_64
make-3.82-23.el7.x86_64
mesa-libEGL-devel-17.2.3-8.20171019.el7.x86_64
mesa-libgbm-devel-17.2.3-8.20171019.el7.x86_64
nettle-devel-2.7.1-8.el7.x86_64
package g++ is not installed
package librdmacm-devel is not installed
pixman-devel-0.34.0-1.el7.x86_64
spice-glib-devel-0.34-3.el7_5.2.x86_64
spice-server-devel-0.14.0-2.el7_5.5.x86_64
tar-1.26-34.el7.x86_64
vte-devel-0.28.2-10.el7.x86_64
xen-devel-4.8.4.43.ge52ec4b787-1.el7.x86_64
zlib-devel-1.2.7-17.el7.x86_64

Environment variables:
PACKAGES=bison     bzip2     bzip2-devel     ccache     csnappy-devel     flex     g++     gcc     gettext     git     glib2-devel     libaio-devel     libepoxy-devel     libfdt-devel     librdmacm-devel     lzo-devel     make     mesa-libEGL-devel     mesa-libgbm-devel     nettle-devel     pixman-devel     SDL-devel     spice-glib-devel     spice-server-devel     tar     vte-devel     xen-devel     zlib-devel
HOSTNAME=953c455c1ccb
MAKEFLAGS= -j8
J=8
CCACHE_DIR=/var/tmp/ccache
EXTRA_CONFIGURE_OPTS=
V=
SHOW_ENV=1
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
TARGET_LIST=
SHLVL=1
HOME=/home/patchew
TEST_DIR=/tmp/qemu-test
FEATURES= dtc
DEBUG=
_=/usr/bin/env

Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/install
No C++ compiler available; disabling C++ specific optional code
Install prefix    /tmp/qemu-test/install
BIOS directory    /tmp/qemu-test/install/share/qemu
firmware path     /tmp/qemu-test/install/share/qemu-firmware
binary directory  /tmp/qemu-test/install/bin
library directory /tmp/qemu-test/install/lib
module directory  /tmp/qemu-test/install/lib/qemu
libexec directory /tmp/qemu-test/install/libexec
include directory /tmp/qemu-test/install/include
config directory  /tmp/qemu-test/install/etc
local state directory   /tmp/qemu-test/install/var
Manual directory  /tmp/qemu-test/install/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path       /tmp/qemu-test/src
GIT binary        git
GIT submodules    
C compiler        cc
Host C compiler   cc
C++ compiler      
Objective-C compiler cc
ARFLAGS           rv
CFLAGS            -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g 
QEMU_CFLAGS       -I/usr/include/pixman-1    -Werror   -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include   -fPIE -DPIE -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv  -Wendif-labels -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -Wno-missing-braces   -I/usr/include/libpng15     -pthread -I/usr/include/spice-server -I/usr/include/cacard -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/nss3 -I/usr/include/nspr4 -I/usr/include/spice-1  
LDFLAGS           -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g 
QEMU_LDFLAGS       
make              make
install           install
python            python -B
smbd              /usr/sbin/smbd
module support    no
host CPU          x86_64
host big endian   no
target list       x86_64-softmmu aarch64-softmmu
gprof enabled     no
sparse enabled    no
strip binaries    yes
profiler          no
static build      no
SDL support       yes (1.2.15)
GTK support       no 
GTK GL support    no
VTE support       no 
TLS priority      NORMAL
GNUTLS support    no
libgcrypt         no
nettle            yes (2.7.1)
libtasn1          no
curses support    yes
virgl support     no 
curl support      no
mingw32 support   no
Audio drivers     oss
Block whitelist (rw) 
Block whitelist (ro) 
VirtFS support    no
Multipath support no
VNC support       yes
VNC SASL support  no
VNC JPEG support  no
VNC PNG support   yes
xen support       yes
xen ctrl version  40800
pv dom build      no
brlapi support    no
bluez  support    no
Documentation     no
PIE               yes
vde support       no
netmap support    no
Linux AIO support yes
ATTR/XATTR support yes
Install blobs     yes
KVM support       yes
HAX support       no
HVF support       no
WHPX support      no
TCG support       yes
TCG debug enabled no
TCG interpreter   no
malloc trim support yes
RDMA support      yes
PVRDMA support    yes
fdt support       system
membarrier        no
preadv support    yes
fdatasync         yes
madvise           yes
posix_madvise     yes
posix_memalign    yes
libcap-ng support no
vhost-net support yes
vhost-crypto support yes
vhost-scsi support yes
vhost-vsock support yes
vhost-user support yes
Trace backends    log
spice support     yes (0.12.13/0.14.0)
rbd support       no
xfsctl support    no
smartcard support yes
libusb            no
usb net redir     no
OpenGL support    yes
OpenGL dmabufs    yes
libiscsi support  no
libnfs support    no
build guest agent yes
QGA VSS support   no
QGA w32 disk info no
QGA MSI support   no
seccomp support   no
coroutine backend ucontext
coroutine pool    yes
debug stack usage no
mutex debugging   no
crypto afalg      no
GlusterFS support no
gcov              gcov
gcov enabled      no
TPM support       yes
libssh2 support   no
TPM passthrough   yes
TPM emulator      yes
QOM debugging     yes
Live block migration yes
lzo support       yes
snappy support    no
bzip2 support     yes
NUMA host support no
libxml2           no
tcmalloc support  no
jemalloc support  no
avx2 optimization yes
replication support yes
VxHS block device no
bochs support     yes
cloop support     yes
dmg support       yes
qcow v1 support   yes
vdi support       yes
vvfat support     yes
qed support       yes
parallels support yes
sheepdog support  yes
capstone          no
docker            no
libpmem support   no
libudev           no

WARNING: Use of SDL 1.2 is deprecated and will be removed in
WARNING: future releases. Please switch to using SDL 2.0

NOTE: cross-compilers enabled:  'cc'
  GEN     x86_64-softmmu/config-devices.mak.tmp
  GEN     aarch64-softmmu/config-devices.mak.tmp
  GEN     config-host.h
  GEN     qemu-options.def
  GEN     qapi-gen
  GEN     trace/generated-tcg-tracers.h
  GEN     trace/generated-helpers-wrappers.h
  GEN     trace/generated-helpers.h
  GEN     aarch64-softmmu/config-devices.mak
  GEN     x86_64-softmmu/config-devices.mak
  GEN     trace/generated-helpers.c
  GEN     module_block.h
  GEN     ui/input-keymap-atset1-to-qcode.c
  GEN     ui/input-keymap-linux-to-qcode.c
  GEN     ui/input-keymap-qcode-to-atset1.c
  GEN     ui/input-keymap-qcode-to-atset2.c
  GEN     ui/input-keymap-qcode-to-atset3.c
  GEN     ui/input-keymap-qcode-to-linux.c
  GEN     ui/input-keymap-qcode-to-qnum.c
  GEN     ui/input-keymap-qcode-to-sun.c
  GEN     ui/input-keymap-qnum-to-qcode.c
  GEN     ui/input-keymap-usb-to-qcode.c
  GEN     ui/input-keymap-win32-to-qcode.c
  GEN     ui/input-keymap-x11-to-qcode.c
  GEN     ui/input-keymap-xorgevdev-to-qcode.c
  GEN     ui/input-keymap-xorgkbd-to-qcode.c
  GEN     ui/input-keymap-xorgxquartz-to-qcode.c
  GEN     ui/input-keymap-xorgxwin-to-qcode.c
  GEN     ui/input-keymap-osx-to-qcode.c
  GEN     tests/test-qapi-gen
  GEN     trace-root.h
  GEN     accel/kvm/trace.h
  GEN     accel/tcg/trace.h
  GEN     audio/trace.h
  GEN     block/trace.h
  GEN     chardev/trace.h
  GEN     crypto/trace.h
  GEN     hw/9pfs/trace.h
  GEN     hw/acpi/trace.h
  GEN     hw/alpha/trace.h
  GEN     hw/arm/trace.h
  GEN     hw/audio/trace.h
  GEN     hw/block/trace.h
  GEN     hw/block/dataplane/trace.h
  GEN     hw/char/trace.h
  GEN     hw/display/trace.h
  GEN     hw/dma/trace.h
  GEN     hw/hppa/trace.h
  GEN     hw/i2c/trace.h
  GEN     hw/i386/trace.h
  GEN     hw/i386/xen/trace.h
  GEN     hw/ide/trace.h
  GEN     hw/input/trace.h
  GEN     hw/intc/trace.h
  GEN     hw/isa/trace.h
  GEN     hw/mem/trace.h
  GEN     hw/misc/trace.h
  GEN     hw/misc/macio/trace.h
  GEN     hw/net/trace.h
  GEN     hw/nvram/trace.h
  GEN     hw/pci/trace.h
  GEN     hw/pci-host/trace.h
  GEN     hw/ppc/trace.h
  GEN     hw/rdma/trace.h
  GEN     hw/rdma/vmw/trace.h
  GEN     hw/s390x/trace.h
  GEN     hw/scsi/trace.h
  GEN     hw/sd/trace.h
  GEN     hw/sparc/trace.h
  GEN     hw/sparc64/trace.h
  GEN     hw/timer/trace.h
  GEN     hw/tpm/trace.h
  GEN     hw/usb/trace.h
  GEN     hw/vfio/trace.h
  GEN     hw/virtio/trace.h
  GEN     hw/watchdog/trace.h
  GEN     hw/xen/trace.h
  GEN     io/trace.h
  GEN     linux-user/trace.h
  GEN     migration/trace.h
  GEN     nbd/trace.h
  GEN     net/trace.h
  GEN     qapi/trace.h
  GEN     qom/trace.h
  GEN     scsi/trace.h
  GEN     target/arm/trace.h
  GEN     target/i386/trace.h
  GEN     target/mips/trace.h
  GEN     target/ppc/trace.h
  GEN     target/s390x/trace.h
  GEN     target/sparc/trace.h
  GEN     ui/trace.h
  GEN     util/trace.h
  GEN     trace-root.c
  GEN     accel/kvm/trace.c
  GEN     accel/tcg/trace.c
  GEN     audio/trace.c
  GEN     block/trace.c
  GEN     chardev/trace.c
  GEN     crypto/trace.c
  GEN     hw/9pfs/trace.c
  GEN     hw/acpi/trace.c
  GEN     hw/alpha/trace.c
  GEN     hw/arm/trace.c
  GEN     hw/audio/trace.c
  GEN     hw/block/trace.c
  GEN     hw/block/dataplane/trace.c
  GEN     hw/char/trace.c
  GEN     hw/display/trace.c
  GEN     hw/dma/trace.c
  GEN     hw/hppa/trace.c
  GEN     hw/i2c/trace.c
  GEN     hw/i386/trace.c
  GEN     hw/i386/xen/trace.c
  GEN     hw/ide/trace.c
  GEN     hw/input/trace.c
  GEN     hw/intc/trace.c
  GEN     hw/isa/trace.c
  GEN     hw/mem/trace.c
  GEN     hw/misc/trace.c
  GEN     hw/misc/macio/trace.c
  GEN     hw/net/trace.c
  GEN     hw/nvram/trace.c
  GEN     hw/pci/trace.c
  GEN     hw/pci-host/trace.c
  GEN     hw/ppc/trace.c
  GEN     hw/rdma/trace.c
  GEN     hw/rdma/vmw/trace.c
  GEN     hw/s390x/trace.c
  GEN     hw/scsi/trace.c
  GEN     hw/sd/trace.c
  GEN     hw/sparc/trace.c
  GEN     hw/sparc64/trace.c
  GEN     hw/timer/trace.c
  GEN     hw/tpm/trace.c
  GEN     hw/usb/trace.c
  GEN     hw/vfio/trace.c
  GEN     hw/virtio/trace.c
  GEN     hw/watchdog/trace.c
  GEN     hw/xen/trace.c
  GEN     io/trace.c
  GEN     linux-user/trace.c
  GEN     migration/trace.c
  GEN     nbd/trace.c
  GEN     net/trace.c
  GEN     qapi/trace.c
  GEN     qom/trace.c
  GEN     scsi/trace.c
  GEN     target/arm/trace.c
  GEN     target/i386/trace.c
  GEN     target/mips/trace.c
  GEN     target/ppc/trace.c
  GEN     target/s390x/trace.c
  GEN     target/sparc/trace.c
  GEN     ui/trace.c
  GEN     util/trace.c
  GEN     config-all-devices.mak
  CC      tests/qemu-iotests/socket_scm_helper.o
  GEN     qga/qapi-generated/qapi-gen
  CC      qapi/qapi-builtin-types.o
  CC      qapi/qapi-types.o
  CC      qapi/qapi-types-block-core.o
  CC      qapi/qapi-types-block.o
  CC      qapi/qapi-types-char.o
  CC      qapi/qapi-types-common.o
  CC      qapi/qapi-types-crypto.o
  CC      qapi/qapi-types-introspect.o
  CC      qapi/qapi-types-job.o
  CC      qapi/qapi-types-migration.o
  CC      qapi/qapi-types-misc.o
  CC      qapi/qapi-types-net.o
  CC      qapi/qapi-types-rocker.o
  CC      qapi/qapi-types-run-state.o
  CC      qapi/qapi-types-sockets.o
  CC      qapi/qapi-types-tpm.o
  CC      qapi/qapi-types-trace.o
  CC      qapi/qapi-types-transaction.o
  CC      qapi/qapi-types-ui.o
  CC      qapi/qapi-builtin-visit.o
  CC      qapi/qapi-visit.o
  CC      qapi/qapi-visit-block-core.o
  CC      qapi/qapi-visit-block.o
  CC      qapi/qapi-visit-char.o
  CC      qapi/qapi-visit-common.o
  CC      qapi/qapi-visit-crypto.o
  CC      qapi/qapi-visit-introspect.o
  CC      qapi/qapi-visit-job.o
  CC      qapi/qapi-visit-migration.o
  CC      qapi/qapi-visit-misc.o
  CC      qapi/qapi-visit-net.o
  CC      qapi/qapi-visit-rocker.o
  CC      qapi/qapi-visit-run-state.o
  CC      qapi/qapi-visit-sockets.o
  CC      qapi/qapi-visit-tpm.o
  CC      qapi/qapi-visit-ui.o
  CC      qapi/qapi-visit-trace.o
  CC      qapi/qapi-visit-transaction.o
  CC      qapi/qapi-events.o
  CC      qapi/qapi-events-block-core.o
  CC      qapi/qapi-events-char.o
  CC      qapi/qapi-events-block.o
  CC      qapi/qapi-events-common.o
  CC      qapi/qapi-events-crypto.o
  CC      qapi/qapi-events-introspect.o
  CC      qapi/qapi-events-job.o
  CC      qapi/qapi-events-migration.o
  CC      qapi/qapi-events-misc.o
  CC      qapi/qapi-events-net.o
  CC      qapi/qapi-events-rocker.o
  CC      qapi/qapi-events-run-state.o
  CC      qapi/qapi-events-sockets.o
  CC      qapi/qapi-events-tpm.o
  CC      qapi/qapi-events-trace.o
  CC      qapi/qapi-events-transaction.o
  CC      qapi/qapi-events-ui.o
  CC      qapi/qapi-introspect.o
  CC      qapi/qapi-visit-core.o
  CC      qapi/qobject-input-visitor.o
  CC      qapi/qapi-dealloc-visitor.o
  CC      qapi/qobject-output-visitor.o
  CC      qapi/qmp-registry.o
  CC      qapi/qmp-dispatch.o
  CC      qapi/string-input-visitor.o
  CC      qapi/string-output-visitor.o
  CC      qapi/opts-visitor.o
  CC      qapi/qapi-clone-visitor.o
  CC      qapi/qapi-util.o
  CC      qapi/qmp-event.o
  CC      qobject/qnull.o
  CC      qobject/qnum.o
  CC      qobject/qstring.o
  CC      qobject/qdict.o
  CC      qobject/qlist.o
  CC      qobject/qbool.o
  CC      qobject/qlit.o
  CC      qobject/qobject.o
  CC      qobject/qjson.o
  CC      qobject/json-lexer.o
  CC      qobject/json-streamer.o
  CC      qobject/json-parser.o
  CC      qobject/block-qdict.o
  CC      trace/control.o
  CC      trace/qmp.o
  CC      util/osdep.o
  CC      util/cutils.o
  CC      util/unicode.o
  CC      util/qemu-timer-common.o
  CC      util/bufferiszero.o
  CC      util/lockcnt.o
  CC      util/aiocb.o
  CC      util/async.o
  CC      util/aio-wait.o
  CC      util/thread-pool.o
  CC      util/qemu-timer.o
  CC      util/main-loop.o
  CC      util/iohandler.o
  CC      util/aio-posix.o
  CC      util/compatfd.o
  CC      util/event_notifier-posix.o
  CC      util/mmap-alloc.o
  CC      util/oslib-posix.o
  CC      util/qemu-openpty.o
  CC      util/qemu-thread-posix.o
  CC      util/memfd.o
  CC      util/envlist.o
  CC      util/path.o
  CC      util/module.o
  CC      util/host-utils.o
  CC      util/bitmap.o
  CC      util/bitops.o
  CC      util/hbitmap.o
  CC      util/fifo8.o
  CC      util/acl.o
  CC      util/cacheinfo.o
  CC      util/error.o
  CC      util/qemu-error.o
  CC      util/id.o
  CC      util/iov.o
  CC      util/qemu-config.o
  CC      util/qemu-sockets.o
  CC      util/uri.o
  CC      util/notify.o
  CC      util/qemu-option.o
  CC      util/qemu-progress.o
  CC      util/keyval.o
  CC      util/hexdump.o
  CC      util/crc32c.o
  CC      util/uuid.o
  CC      util/throttle.o
  CC      util/getauxval.o
  CC      util/readline.o
  CC      util/rcu.o
  CC      util/qemu-coroutine.o
  CC      util/qemu-coroutine-lock.o
  CC      util/qemu-coroutine-io.o
  CC      util/qemu-coroutine-sleep.o
  CC      util/coroutine-ucontext.o
  CC      util/buffer.o
  CC      util/timed-average.o
  CC      util/base64.o
  CC      util/log.o
  CC      util/pagesize.o
  CC      util/qdist.o
  CC      util/qht.o
  CC      util/qsp.o
  CC      util/range.o
  CC      util/stats64.o
  CC      util/systemd.o
  CC      util/iova-tree.o
  CC      util/vfio-helpers.o
  CC      util/drm.o
  CC      trace-root.o
  CC      accel/kvm/trace.o
  CC      accel/tcg/trace.o
  CC      audio/trace.o
  CC      block/trace.o
  CC      chardev/trace.o
  CC      crypto/trace.o
  CC      hw/9pfs/trace.o
  CC      hw/acpi/trace.o
  CC      hw/alpha/trace.o
  CC      hw/arm/trace.o
  CC      hw/audio/trace.o
  CC      hw/block/trace.o
  CC      hw/block/dataplane/trace.o
  CC      hw/char/trace.o
  CC      hw/display/trace.o
  CC      hw/dma/trace.o
  CC      hw/hppa/trace.o
  CC      hw/i2c/trace.o
  CC      hw/i386/trace.o
  CC      hw/i386/xen/trace.o
  CC      hw/ide/trace.o
  CC      hw/input/trace.o
  CC      hw/intc/trace.o
  CC      hw/isa/trace.o
  CC      hw/mem/trace.o
  CC      hw/misc/trace.o
  CC      hw/net/trace.o
  CC      hw/misc/macio/trace.o
  CC      hw/nvram/trace.o
  CC      hw/pci/trace.o
  CC      hw/pci-host/trace.o
  CC      hw/ppc/trace.o
  CC      hw/rdma/trace.o
  CC      hw/rdma/vmw/trace.o
  CC      hw/s390x/trace.o
  CC      hw/scsi/trace.o
  CC      hw/sd/trace.o
  CC      hw/sparc/trace.o
  CC      hw/sparc64/trace.o
  CC      hw/timer/trace.o
  CC      hw/tpm/trace.o
  CC      hw/usb/trace.o
  CC      hw/vfio/trace.o
  CC      hw/virtio/trace.o
  CC      hw/watchdog/trace.o
  CC      hw/xen/trace.o
  CC      io/trace.o
  CC      linux-user/trace.o
  CC      migration/trace.o
  CC      nbd/trace.o
  CC      net/trace.o
  CC      qapi/trace.o
  CC      qom/trace.o
  CC      scsi/trace.o
  CC      target/arm/trace.o
  CC      target/i386/trace.o
  CC      target/mips/trace.o
  CC      target/s390x/trace.o
  CC      target/ppc/trace.o
  CC      target/sparc/trace.o
  CC      ui/trace.o
  CC      util/trace.o
  CC      crypto/pbkdf-stub.o
  CC      stubs/arch-query-cpu-def.o
  CC      stubs/arch-query-cpu-model-expansion.o
  CC      stubs/arch-query-cpu-model-baseline.o
  CC      stubs/arch-query-cpu-model-comparison.o
  CC      stubs/bdrv-next-monitor-owned.o
  CC      stubs/blk-commit-all.o
  CC      stubs/blockdev-close-all-bdrv-states.o
  CC      stubs/clock-warp.o
  CC      stubs/cpu-get-clock.o
  CC      stubs/cpu-get-icount.o
  CC      stubs/dump.o
  CC      stubs/error-printf.o
  CC      stubs/fdset.o
  CC      stubs/gdbstub.o
  CC      stubs/get-vm-name.o
  CC      stubs/iothread.o
  CC      stubs/iothread-lock.o
  CC      stubs/is-daemonized.o
  CC      stubs/linux-aio.o
  CC      stubs/machine-init-done.o
  CC      stubs/migr-blocker.o
  CC      stubs/change-state-handler.o
  CC      stubs/monitor.o
  CC      stubs/notify-event.o
  CC      stubs/qtest.o
  CC      stubs/replay.o
  CC      stubs/runstate-check.o
  CC      stubs/set-fd-handler.o
  CC      stubs/slirp.o
  CC      stubs/sysbus.o
  CC      stubs/tpm.o
  CC      stubs/trace-control.o
  CC      stubs/uuid.o
  CC      stubs/vm-stop.o
  CC      stubs/vmstate.o
  CC      stubs/qmp_memory_device.o
  CC      stubs/target-monitor-defs.o
  CC      stubs/target-get-monitor-def.o
  CC      stubs/pc_madt_cpu_entry.o
  CC      stubs/vmgenid.o
  CC      stubs/xen-common.o
  CC      stubs/xen-hvm.o
  CC      stubs/pci-host-piix.o
  CC      stubs/ram-block.o
  CC      stubs/ramfb.o
  CC      contrib/ivshmem-client/ivshmem-client.o
  CC      contrib/ivshmem-client/main.o
  CC      contrib/ivshmem-server/ivshmem-server.o
  CC      contrib/ivshmem-server/main.o
  CC      qemu-nbd.o
  CC      block.o
  CC      blockjob.o
  CC      job.o
  CC      qemu-io-cmds.o
  CC      replication.o
  CC      block/raw-format.o
  CC      block/vmdk.o
  CC      block/vpc.o
  CC      block/qcow.o
  CC      block/vdi.o
  CC      block/cloop.o
  CC      block/bochs.o
  CC      block/vvfat.o
  CC      block/dmg.o
  CC      block/qcow2.o
  CC      block/qcow2-refcount.o
  CC      block/qcow2-cluster.o
  CC      block/qcow2-snapshot.o
  CC      block/qcow2-cache.o
  CC      block/qcow2-bitmap.o
/tmp/qemu-test/src/block/qcow2-cluster.c: In function 'qcow2_decompress_cluster':
/tmp/qemu-test/src/block/qcow2-cluster.c:1630:9: error: implicit declaration of function 'bdrv_read' [-Werror=implicit-function-declaration]
         ret = bdrv_read(bs->file, coffset >> 9, s->cluster_data,
         ^
/tmp/qemu-test/src/block/qcow2-cluster.c:1630:9: error: nested extern declaration of 'bdrv_read' [-Werror=nested-externs]
cc1: all warnings being treated as errors
make: *** [block/qcow2-cluster.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 563, in <module>
    sys.exit(main())
  File "./tests/docker/docker.py", line 560, in main
    return args.cmdobj.run(args, argv)
  File "./tests/docker/docker.py", line 306, in run
    return Docker().run(argv, args.keep, quiet=args.quiet)
  File "./tests/docker/docker.py", line 274, in run
    quiet=quiet)
  File "./tests/docker/docker.py", line 181, in _do_check
    return subprocess.check_call(self._command + cmd, **kwargs)
  File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=58a1a172e8b511e8a0b368b59973b7d0', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '--net=none', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=8', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-9bah4vqa/src/docker-src.2018-11-15-04.03.19.7513:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-9bah4vqa/src'
make: *** [docker-run-test-quick@centos7] Error 2

real	1m28.535s
user	0m17.660s
sys	0m3.549s
=== OUTPUT END ===

Test command exited with code: 2


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com