qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [Qemu-devel] [PULL 00/10] Block patches
@ 2015-12-22  8:54 Stefan Hajnoczi
  2015-12-22  8:54 ` [Qemu-devel] [PULL 01/10] virtio-blk: trivial code optimization Stefan Hajnoczi
                   ` (10 more replies)
  0 siblings, 11 replies; 12+ messages in thread
From: Stefan Hajnoczi @ 2015-12-22  8:54 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Stefan Hajnoczi

The following changes since commit c688084506cf2cf2eba4ba9df4e91abb6e3dab83:

  Merge remote-tracking branch 'remotes/berrange/tags/pull-qcrypto-secrets-base-2015-12-18-1' into staging (2015-12-18 17:04:15 +0000)

are available in the git repository at:

  git://github.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 723697551a7e926abe7d3c7f2966012b8075143d:

  sdhci: add optional quirk property to disable card insertion/removal interrupts (2015-12-22 16:34:26 +0800)

----------------------------------------------------------------

----------------------------------------------------------------

Andrew Baumann (2):
  sdhci: don't raise a command index error for an unexpected response
  sdhci: add optional quirk property to disable card insertion/removal
    interrupts

Gonglei (1):
  virtio-blk: trivial code optimization

Peter Crosthwaite (1):
  sd: sdhci: Delete over-zealous power check

Stefan Hajnoczi (4):
  block: add BlockLimits.max_iov field
  block-backend: add blk_get_max_iov()
  block: replace IOV_MAX with BlockLimits.max_iov
  block/mirror: replace IOV_MAX with blk_get_max_iov()

Vladimir Sementsov-Ogievskiy (1):
  parallels: add format spec

Yang Wei (1):
  scripts/gdb: Fix a python exception in mtree.py

 MAINTAINERS                    |   1 +
 block/block-backend.c          |   5 +
 block/io.c                     |  10 +-
 block/mirror.c                 |   6 +-
 docs/specs/parallels.txt       | 228 +++++++++++++++++++++++++++++++++++++++++
 hw/block/virtio-blk.c          |  28 ++---
 hw/sd/sdhci.c                  |  10 +-
 include/block/block_int.h      |   3 +
 include/hw/sd/sdhci.h          |   1 +
 include/sysemu/block-backend.h |   1 +
 scripts/qemugdb/mtree.py       |  10 +-
 11 files changed, 272 insertions(+), 31 deletions(-)
 create mode 100644 docs/specs/parallels.txt

-- 
2.5.0

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Qemu-devel] [PULL 01/10] virtio-blk: trivial code optimization
  2015-12-22  8:54 [Qemu-devel] [PULL 00/10] Block patches Stefan Hajnoczi
@ 2015-12-22  8:54 ` Stefan Hajnoczi
  2015-12-22  8:54 ` [Qemu-devel] [PULL 02/10] block: add BlockLimits.max_iov field Stefan Hajnoczi
                   ` (9 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Stefan Hajnoczi @ 2015-12-22  8:54 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Gonglei, Stefan Hajnoczi

From: Gonglei <arei.gonglei@huawei.com>

1. avoid possible superflous checking
2. make code more robustness

["make code more robustness" refers to avoiding integer
underflows/overflows.
--Stefan]

Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Message-id: 1447207166-12612-1-git-send-email-arei.gonglei@huawei.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/block/virtio-blk.c | 28 ++++++++++------------------
 1 file changed, 10 insertions(+), 18 deletions(-)

diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index b88b726..f72d4b6 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -407,24 +407,16 @@ void virtio_blk_submit_multireq(BlockBackend *blk, MultiReqBuffer *mrb)
     for (i = 0; i < mrb->num_reqs; i++) {
         VirtIOBlockReq *req = mrb->reqs[i];
         if (num_reqs > 0) {
-            bool merge = true;
-
-            /* merge would exceed maximum number of IOVs */
-            if (niov + req->qiov.niov > IOV_MAX) {
-                merge = false;
-            }
-
-            /* merge would exceed maximum transfer length of backend device */
-            if (req->qiov.size / BDRV_SECTOR_SIZE + nb_sectors > max_xfer_len) {
-                merge = false;
-            }
-
-            /* requests are not sequential */
-            if (sector_num + nb_sectors != req->sector_num) {
-                merge = false;
-            }
-
-            if (!merge) {
+            /*
+             * NOTE: We cannot merge the requests in below situations:
+             * 1. requests are not sequential
+             * 2. merge would exceed maximum number of IOVs
+             * 3. merge would exceed maximum transfer length of backend device
+             */
+            if (sector_num + nb_sectors != req->sector_num ||
+                niov > IOV_MAX - req->qiov.niov ||
+                req->qiov.size / BDRV_SECTOR_SIZE > max_xfer_len ||
+                nb_sectors > max_xfer_len - req->qiov.size / BDRV_SECTOR_SIZE) {
                 submit_requests(blk, mrb, start, num_reqs, niov);
                 num_reqs = 0;
             }
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [Qemu-devel] [PULL 02/10] block: add BlockLimits.max_iov field
  2015-12-22  8:54 [Qemu-devel] [PULL 00/10] Block patches Stefan Hajnoczi
  2015-12-22  8:54 ` [Qemu-devel] [PULL 01/10] virtio-blk: trivial code optimization Stefan Hajnoczi
@ 2015-12-22  8:54 ` Stefan Hajnoczi
  2015-12-22  8:54 ` [Qemu-devel] [PULL 03/10] block-backend: add blk_get_max_iov() Stefan Hajnoczi
                   ` (8 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Stefan Hajnoczi @ 2015-12-22  8:54 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Peter Lieven, Stefan Hajnoczi

The maximum number of struct iovec elements depends on the
BlockDriverState.  The raw-posix and iSCSI protocols have a maximum of
IOV_MAX but others could have different values.

Cc: Peter Lieven <pl@kamp.de>
Suggested-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/io.c                | 7 +++++++
 include/block/block_int.h | 3 +++
 2 files changed, 10 insertions(+)

diff --git a/block/io.c b/block/io.c
index 841f5b5..42050a0 100644
--- a/block/io.c
+++ b/block/io.c
@@ -166,9 +166,13 @@ void bdrv_refresh_limits(BlockDriverState *bs, Error **errp)
         bs->bl.max_transfer_length = bs->file->bs->bl.max_transfer_length;
         bs->bl.min_mem_alignment = bs->file->bs->bl.min_mem_alignment;
         bs->bl.opt_mem_alignment = bs->file->bs->bl.opt_mem_alignment;
+        bs->bl.max_iov = bs->file->bs->bl.max_iov;
     } else {
         bs->bl.min_mem_alignment = 512;
         bs->bl.opt_mem_alignment = getpagesize();
+
+        /* Safe default since most protocols use readv()/writev()/etc */
+        bs->bl.max_iov = IOV_MAX;
     }
 
     if (bs->backing) {
@@ -189,6 +193,9 @@ void bdrv_refresh_limits(BlockDriverState *bs, Error **errp)
         bs->bl.min_mem_alignment =
             MAX(bs->bl.min_mem_alignment,
                 bs->backing->bs->bl.min_mem_alignment);
+        bs->bl.max_iov =
+            MIN(bs->bl.max_iov,
+                bs->backing->bs->bl.max_iov);
     }
 
     /* Then let the driver override it */
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 9a1c466..256609d 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -330,6 +330,9 @@ typedef struct BlockLimits {
 
     /* memory alignment for bounce buffer */
     size_t opt_mem_alignment;
+
+    /* maximum number of iovec elements */
+    int max_iov;
 } BlockLimits;
 
 typedef struct BdrvOpBlocker BdrvOpBlocker;
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [Qemu-devel] [PULL 03/10] block-backend: add blk_get_max_iov()
  2015-12-22  8:54 [Qemu-devel] [PULL 00/10] Block patches Stefan Hajnoczi
  2015-12-22  8:54 ` [Qemu-devel] [PULL 01/10] virtio-blk: trivial code optimization Stefan Hajnoczi
  2015-12-22  8:54 ` [Qemu-devel] [PULL 02/10] block: add BlockLimits.max_iov field Stefan Hajnoczi
@ 2015-12-22  8:54 ` Stefan Hajnoczi
  2015-12-22  8:54 ` [Qemu-devel] [PULL 04/10] block: replace IOV_MAX with BlockLimits.max_iov Stefan Hajnoczi
                   ` (7 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Stefan Hajnoczi @ 2015-12-22  8:54 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Stefan Hajnoczi

Add a function to query BlockLimits.max_iov.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/block-backend.c          | 5 +++++
 include/sysemu/block-backend.h | 1 +
 2 files changed, 6 insertions(+)

diff --git a/block/block-backend.c b/block/block-backend.c
index 36ccc9e..f41d326 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -1023,6 +1023,11 @@ int blk_get_max_transfer_length(BlockBackend *blk)
     }
 }
 
+int blk_get_max_iov(BlockBackend *blk)
+{
+    return blk->bs->bl.max_iov;
+}
+
 void blk_set_guest_block_size(BlockBackend *blk, int align)
 {
     blk->guest_block_size = align;
diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h
index fb068ea4..dc24476 100644
--- a/include/sysemu/block-backend.h
+++ b/include/sysemu/block-backend.h
@@ -146,6 +146,7 @@ void blk_lock_medium(BlockBackend *blk, bool locked);
 void blk_eject(BlockBackend *blk, bool eject_flag);
 int blk_get_flags(BlockBackend *blk);
 int blk_get_max_transfer_length(BlockBackend *blk);
+int blk_get_max_iov(BlockBackend *blk);
 void blk_set_guest_block_size(BlockBackend *blk, int align);
 void *blk_blockalign(BlockBackend *blk, size_t size);
 bool blk_op_is_blocked(BlockBackend *blk, BlockOpType op, Error **errp);
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [Qemu-devel] [PULL 04/10] block: replace IOV_MAX with BlockLimits.max_iov
  2015-12-22  8:54 [Qemu-devel] [PULL 00/10] Block patches Stefan Hajnoczi
                   ` (2 preceding siblings ...)
  2015-12-22  8:54 ` [Qemu-devel] [PULL 03/10] block-backend: add blk_get_max_iov() Stefan Hajnoczi
@ 2015-12-22  8:54 ` Stefan Hajnoczi
  2015-12-22  8:54 ` [Qemu-devel] [PULL 05/10] block/mirror: replace IOV_MAX with blk_get_max_iov() Stefan Hajnoczi
                   ` (6 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Stefan Hajnoczi @ 2015-12-22  8:54 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Stefan Hajnoczi

Request merging must not result in a huge request that exceeds the
maximum number of iovec elements.  Use BlockLimits.max_iov instead of
hardcoding IOV_MAX.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/io.c            | 3 ++-
 hw/block/virtio-blk.c | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/block/io.c b/block/io.c
index 42050a0..63e3678 100644
--- a/block/io.c
+++ b/block/io.c
@@ -1889,7 +1889,8 @@ static int multiwrite_merge(BlockDriverState *bs, BlockRequest *reqs,
             merge = 1;
         }
 
-        if (reqs[outidx].qiov->niov + reqs[i].qiov->niov + 1 > IOV_MAX) {
+        if (reqs[outidx].qiov->niov + reqs[i].qiov->niov + 1 >
+            bs->bl.max_iov) {
             merge = 0;
         }
 
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index f72d4b6..51f867b 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -414,7 +414,7 @@ void virtio_blk_submit_multireq(BlockBackend *blk, MultiReqBuffer *mrb)
              * 3. merge would exceed maximum transfer length of backend device
              */
             if (sector_num + nb_sectors != req->sector_num ||
-                niov > IOV_MAX - req->qiov.niov ||
+                niov > blk_get_max_iov(blk) - req->qiov.niov ||
                 req->qiov.size / BDRV_SECTOR_SIZE > max_xfer_len ||
                 nb_sectors > max_xfer_len - req->qiov.size / BDRV_SECTOR_SIZE) {
                 submit_requests(blk, mrb, start, num_reqs, niov);
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [Qemu-devel] [PULL 05/10] block/mirror: replace IOV_MAX with blk_get_max_iov()
  2015-12-22  8:54 [Qemu-devel] [PULL 00/10] Block patches Stefan Hajnoczi
                   ` (3 preceding siblings ...)
  2015-12-22  8:54 ` [Qemu-devel] [PULL 04/10] block: replace IOV_MAX with BlockLimits.max_iov Stefan Hajnoczi
@ 2015-12-22  8:54 ` Stefan Hajnoczi
  2015-12-22  8:54 ` [Qemu-devel] [PULL 06/10] parallels: add format spec Stefan Hajnoczi
                   ` (5 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Stefan Hajnoczi @ 2015-12-22  8:54 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Stefan Hajnoczi

Use blk_get_max_iov() instead of hardcoding IOV_MAX, which may not apply
to all BlockDrivers.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/mirror.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/block/mirror.c b/block/mirror.c
index fc34a9c..f201f2b 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -161,13 +161,15 @@ static void mirror_read_complete(void *opaque, int ret)
 static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s)
 {
     BlockDriverState *source = s->common.bs;
-    int nb_sectors, sectors_per_chunk, nb_chunks;
+    int nb_sectors, sectors_per_chunk, nb_chunks, max_iov;
     int64_t end, sector_num, next_chunk, next_sector, hbitmap_next_sector;
     uint64_t delay_ns = 0;
     MirrorOp *op;
     int pnum;
     int64_t ret;
 
+    max_iov = MIN(source->bl.max_iov, s->target->bl.max_iov);
+
     s->sector_num = hbitmap_iter_next(&s->hbi);
     if (s->sector_num < 0) {
         bdrv_dirty_iter_init(s->dirty_bitmap, &s->hbi);
@@ -248,7 +250,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s)
             trace_mirror_break_buf_busy(s, nb_chunks, s->in_flight);
             break;
         }
-        if (IOV_MAX < nb_chunks + added_chunks) {
+        if (max_iov < nb_chunks + added_chunks) {
             trace_mirror_break_iov_max(s, nb_chunks, added_chunks);
             break;
         }
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [Qemu-devel] [PULL 06/10] parallels: add format spec
  2015-12-22  8:54 [Qemu-devel] [PULL 00/10] Block patches Stefan Hajnoczi
                   ` (4 preceding siblings ...)
  2015-12-22  8:54 ` [Qemu-devel] [PULL 05/10] block/mirror: replace IOV_MAX with blk_get_max_iov() Stefan Hajnoczi
@ 2015-12-22  8:54 ` Stefan Hajnoczi
  2015-12-22  8:54 ` [Qemu-devel] [PULL 07/10] scripts/gdb: Fix a python exception in mtree.py Stefan Hajnoczi
                   ` (4 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Stefan Hajnoczi @ 2015-12-22  8:54 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Vladimir Sementsov-Ogievskiy, Stefan Hajnoczi,
	Denis V. Lunev, John Snow

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

This specifies Parallels image format as implemented in Parallels Cloud
Server 6.10

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
Message-id: 1448626806-17591-1-git-send-email-den@openvz.org
CC: Eric Blake <eblake@redhat.com>
CC: John Snow <jsnow@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 MAINTAINERS              |   1 +
 docs/specs/parallels.txt | 228 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 229 insertions(+)
 create mode 100644 docs/specs/parallels.txt

diff --git a/MAINTAINERS b/MAINTAINERS
index 55a0fd8..46bba68 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1490,6 +1490,7 @@ M: Denis V. Lunev <den@openvz.org>
 L: qemu-block@nongnu.org
 S: Supported
 F: block/parallels.c
+F: docs/specs/parallels.txt
 
 qed
 M: Stefan Hajnoczi <stefanha@redhat.com>
diff --git a/docs/specs/parallels.txt b/docs/specs/parallels.txt
new file mode 100644
index 0000000..b4fe229
--- /dev/null
+++ b/docs/specs/parallels.txt
@@ -0,0 +1,228 @@
+= License =
+
+Copyright (c) 2015 Denis Lunev
+Copyright (c) 2015 Vladimir Sementsov-Ogievskiy
+
+This work is licensed under the terms of the GNU GPL, version 2 or later.
+See the COPYING file in the top-level directory.
+
+= Parallels Expandable Image File Format =
+
+A Parallels expandable image file consists of three consecutive parts:
+    * header
+    * BAT
+    * data area
+
+All numbers in a Parallels expandable image are stored in little-endian byte
+order.
+
+
+== Definitions ==
+
+    Sector    A 512-byte data chunk.
+
+    Cluster   A data chunk of the size specified in the image header.
+              Currently, the default size is 1MiB (2048 sectors). In previous
+              versions, cluster sizes of 63 sectors, 256 and 252 kilobytes were
+              used.
+
+    BAT       Block Allocation Table, an entity that contains information for
+              guest-to-host I/O data address translation.
+
+
+== Header ==
+
+The header is placed at the start of an image and contains the following
+fields:
+
+Bytes:
+   0 - 15:    magic
+              Must contain "WithoutFreeSpace" or "WithouFreSpacExt".
+
+  16 - 19:    version
+              Must be 2.
+
+  20 - 23:    heads
+              Disk geometry parameter for guest.
+
+  24 - 27:    cylinders
+              Disk geometry parameter for guest.
+
+  28 - 31:    tracks
+              Cluster size, in sectors.
+
+  32 - 35:    nb_bat_entries
+              Disk size, in clusters (BAT size).
+
+  36 - 43:    nb_sectors
+              Disk size, in sectors.
+
+              For "WithoutFreeSpace" images:
+              Only the lowest 4 bytes are used. The highest 4 bytes must be
+              cleared in this case.
+
+              For "WithouFreSpacExt" images, there are no such
+              restrictions.
+
+  44 - 47:    in_use
+              Set to 0x746F6E59 when the image is opened by software in R/W
+              mode; set to 0x312e3276 when the image is closed.
+
+              A zero in this field means that the image was opened by an old
+              version of the software that doesn't support Format Extension
+              (see below).
+
+              Other values are not allowed.
+
+  48 - 51:    data_off
+              An offset, in sectors, from the start of the file to the start of
+              the data area.
+
+              For "WithoutFreeSpace" images:
+              - If data_off is zero, the offset is calculated as the end of BAT
+                table plus some padding to ensure sector size alignment.
+              - If data_off is non-zero, the offset should be aligned to sector
+                size. However it is recommended to align it to cluster size for
+                newly created images.
+
+              For "WithouFreSpacExt" images:
+              data_off must be non-zero and aligned to cluster size.
+
+  52 - 55:    flags
+              Miscellaneous flags.
+
+              Bit 0: Empty Image bit. If set, the image should be
+                     considered clear.
+
+              Bits 2-31: Unused.
+
+  56 - 63:    ext_off
+              Format Extension offset, an offset, in sectors, from the start of
+              the file to the start of the Format Extension Cluster.
+
+              ext_off must meet the same requirements as cluster offsets
+              defined by BAT entries (see below).
+
+
+== BAT ==
+
+BAT is placed immediately after the image header. In the file, BAT is a
+contiguous array of 32-bit unsigned little-endian integers with
+(bat_entries * 4) bytes size.
+
+Each BAT entry contains an offset from the start of the file to the
+corresponding cluster. The offset set in clusters for "WithouFreSpacExt" images
+and in sectors for "WithoutFreeSpace" images.
+
+If a BAT entry is zero, the corresponding cluster is not allocated and should
+be considered as filled with zeroes.
+
+Cluster offsets specified by BAT entries must meet the following requirements:
+    - the value must not be lower than data offset (provided by header.data_off
+      or calculated as specified above),
+    - the value must be lower than the desired file size,
+    - the value must be unique among all BAT entries,
+    - the result of (cluster offset - data offset) must be aligned to cluster
+      size.
+
+
+== Data Area ==
+
+The data area is an area from the data offset (provided by header.data_off or
+calculated as specified above) to the end of the file. It represents a
+contiguous array of clusters. Most of them are allocated by the BAT, some may
+be allocated by the ext_off field in the header while other may be allocated by
+extensions. All clusters allocated by ext_off and extensions should meet the
+same requirements as clusters specified by BAT entries.
+
+
+== Format Extension ==
+
+The Format Extension is an area 1 cluster in size that provides additional
+format features. This cluster is addressed by the ext_off field in the header.
+The format of the Format Extension area is the following:
+
+   0 -  7:    magic
+              Must be 0xAB234CEF23DCEA87
+
+   8 - 23:    m_CheckSum
+              The MD5 checksum of the entire Header Extension cluster except
+              the first 24 bytes.
+
+    The above are followed by feature sections or "extensions". The last
+    extension must be "End of features" (see below).
+
+Each feature section has the following format:
+
+   0 -  7:    magic
+              The identifier of the feature:
+              0x0000000000000000 - End of features
+              0x20385FAE252CB34A - Dirty bitmap
+
+   8 - 15:    flags
+              External flags for extension:
+
+              Bit 0: NECESSARY
+                     If the software cannot load the extension (due to an
+                     unknown magic number or error), the file should not be
+                     changed. If this flag is unset and there is an error on
+                     loading the extension, said extension should be dropped.
+
+              Bit 1: TRANSIT
+                     If there is an unknown extension with this flag set,
+                     said extension should be left as is.
+
+              If neither NECESSARY nor TRANSIT are set, the extension should be
+              dropped.
+
+  16 - 19:    data_size
+              The size of the following feature data, in bytes.
+
+  20 - 23:    unused32
+              Align header to 8 bytes boundary.
+
+  variable:   data (data_size bytes)
+
+    The above is followed by padding to the next 8 bytes boundary, then the
+    next extension starts.
+
+    The last extension must be "End of features" with all the fields set to 0.
+
+
+=== Dirty bitmaps feature ===
+
+This feature provides a way of storing dirty bitmaps in the image. The fields
+of its data area are:
+
+   0 -  7:    size
+              The bitmap size, should be equal to disk size in sectors.
+
+   8 - 23:    id
+              An identifier for backup consistency checking.
+
+  24 - 27:    granularity
+              Bitmap granularity, in sectors. I.e., the number of sectors
+              corresponding to one bit of the bitmap. Granularity must be
+              a power of 2.
+
+  28 - 31:    l1_size
+              The number of entries in the L1 table of the bitmap.
+
+  variable:   l1 (64 * l1_size bytes)
+              L1 offset table (in bytes)
+
+A dirty bitmap is stored using a one-level structure for the mapping to host
+clusters - an L1 table.
+
+Given an offset in bytes into the bitmap data, the offset in bytes into the
+image file can be obtained as follows:
+
+    offset = l1_table[offset / cluster_size] + (offset % cluster_size)
+
+If an L1 table entry is 0, the corresponding cluster of the bitmap is assumed
+to be zero.
+
+If an L1 table entry is 1, the corresponding cluster of the bitmap is assumed
+to have all bits set.
+
+If an L1 table entry is not 0 or 1, it allocates a cluster from the data area.
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [Qemu-devel] [PULL 07/10] scripts/gdb: Fix a python exception in mtree.py
  2015-12-22  8:54 [Qemu-devel] [PULL 00/10] Block patches Stefan Hajnoczi
                   ` (5 preceding siblings ...)
  2015-12-22  8:54 ` [Qemu-devel] [PULL 06/10] parallels: add format spec Stefan Hajnoczi
@ 2015-12-22  8:54 ` Stefan Hajnoczi
  2015-12-22  8:54 ` [Qemu-devel] [PULL 08/10] sd: sdhci: Delete over-zealous power check Stefan Hajnoczi
                   ` (3 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Stefan Hajnoczi @ 2015-12-22  8:54 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Stefan Hajnoczi, Yang Wei

From: Yang Wei <w90p710@gmail.com>

The following exception is threw:
Python Exception <class 'NameError'> name 'long' is not defined:
Error occurred in Python command: name 'long' is not defined

Python 2.4+, int()/long() have been unified, so replace long
with int.

Signed-off-by: Yang Wei <w90p710@gmail.com>
Message-id: 1449316340-4030-1-git-send-email-w90p710@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 scripts/qemugdb/mtree.py | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/scripts/qemugdb/mtree.py b/scripts/qemugdb/mtree.py
index 06011c3..cc8131c 100644
--- a/scripts/qemugdb/mtree.py
+++ b/scripts/qemugdb/mtree.py
@@ -21,7 +21,7 @@ def isnull(ptr):
     return ptr == gdb.Value(0).cast(ptr.type)
 
 def int128(p):
-    return long(p['lo']) + (long(p['hi']) << 64)
+    return int(p['lo']) + (int(p['hi']) << 64)
 
 class MtreeCommand(gdb.Command):
     '''Display the memory tree hierarchy'''
@@ -40,11 +40,11 @@ class MtreeCommand(gdb.Command):
     def process_queue(self):
         while self.queue:
             ptr = self.queue.pop(0)
-            if long(ptr) in self.seen:
+            if int(ptr) in self.seen:
                 continue
             self.print_item(ptr)
     def print_item(self, ptr, offset = gdb.Value(0), level = 0):
-        self.seen.add(long(ptr))
+        self.seen.add(int(ptr))
         addr = ptr['addr']
         addr += offset
         size = int128(ptr['size'])
@@ -58,8 +58,8 @@ class MtreeCommand(gdb.Command):
             klass = ' (RAM)'
         gdb.write('%s%016x-%016x %s%s (@ %s)\n'
                   % ('  ' * level,
-                     long(addr),
-                     long(addr + (size - 1)),
+                     int(addr),
+                     int(addr + (size - 1)),
                      ptr['name'].string(),
                      klass,
                      ptr,
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [Qemu-devel] [PULL 08/10] sd: sdhci: Delete over-zealous power check
  2015-12-22  8:54 [Qemu-devel] [PULL 00/10] Block patches Stefan Hajnoczi
                   ` (6 preceding siblings ...)
  2015-12-22  8:54 ` [Qemu-devel] [PULL 07/10] scripts/gdb: Fix a python exception in mtree.py Stefan Hajnoczi
@ 2015-12-22  8:54 ` Stefan Hajnoczi
  2015-12-22  8:54 ` [Qemu-devel] [PULL 09/10] sdhci: don't raise a command index error for an unexpected response Stefan Hajnoczi
                   ` (2 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Stefan Hajnoczi @ 2015-12-22  8:54 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Peter Crosthwaite, Stefan Hajnoczi, Andrew Baumann,
	Peter Crosthwaite

From: Peter Crosthwaite <crosthwaitepeter@gmail.com>

This check was conditionalising SD card operation on the card being
powered by the SDHCI host controller. It is however possible
(particularly in embedded systems) for the power control of the SD card
to be managed outside of SDHCI. This can be as trivial as hard-wiring
the SD slot VCC to a constant power-rail.

This means the guest SDHCI can validly opt-out of the SDHCI power
control feature while still using the card. So delete this check to
allow operation of the card with SDHCI power control.

This is needed for at least Xilinx Zynq and Raspberry Pi, and
also makes Freescale i.MX25 work for me. The digilent Zybo board
has a public schematic which shows SD VCC hardwiring:

http://digilentinc.com/Data/Products/ZYBO/ZYBO_sch_VB.3.pdf
bottom of page 3.

Signed-off-by: Peter Crosthwaite <crosthwaite.peter@gmail.com>
Reviewed-by: Sai Pavan Boddu <saipava@xilinx.com>
Signed-off-by: Andrew Baumann <Andrew.Baumann@microsoft.com>
Message-id: 1450738069-18664-2-git-send-email-Andrew.Baumann@microsoft.com
[AB: Add Pi to list of devices fixed in commit message]
Signed-off-by: Andrew Baumann <Andrew.Baumann@microsoft.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/sd/sdhci.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/sd/sdhci.c b/hw/sd/sdhci.c
index 8612760..bc39d48 100644
--- a/hw/sd/sdhci.c
+++ b/hw/sd/sdhci.c
@@ -831,7 +831,7 @@ static void sdhci_data_transfer(void *opaque)
 
 static bool sdhci_can_issue_command(SDHCIState *s)
 {
-    if (!SDHC_CLOCK_IS_ON(s->clkcon) || !(s->pwrcon & SDHC_POWER_ON) ||
+    if (!SDHC_CLOCK_IS_ON(s->clkcon) ||
         (((s->prnsts & SDHC_DATA_INHIBIT) || s->stopped_state) &&
         ((s->cmdreg & SDHC_CMD_DATA_PRESENT) ||
         ((s->cmdreg & SDHC_CMD_RESPONSE) == SDHC_CMD_RSP_WITH_BUSY &&
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [Qemu-devel] [PULL 09/10] sdhci: don't raise a command index error for an unexpected response
  2015-12-22  8:54 [Qemu-devel] [PULL 00/10] Block patches Stefan Hajnoczi
                   ` (7 preceding siblings ...)
  2015-12-22  8:54 ` [Qemu-devel] [PULL 08/10] sd: sdhci: Delete over-zealous power check Stefan Hajnoczi
@ 2015-12-22  8:54 ` Stefan Hajnoczi
  2015-12-22  8:54 ` [Qemu-devel] [PULL 10/10] sdhci: add optional quirk property to disable card insertion/removal interrupts Stefan Hajnoczi
  2015-12-22 14:47 ` [Qemu-devel] [PULL 00/10] Block patches Peter Maydell
  10 siblings, 0 replies; 12+ messages in thread
From: Stefan Hajnoczi @ 2015-12-22  8:54 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Stefan Hajnoczi, Andrew Baumann

From: Andrew Baumann <Andrew.Baumann@microsoft.com>

This deletes a block of code that raised a command index error if a
command returned response data, but the guest did not set the
appropriate bits in the response register to handle such a response. I
cannot find any documentation that suggests the controller should
behave in this way, the error code doesn't make sense (command index
error is defined for the case where the index in a response does not
match that of the issued command), and in at least one case (CMD23
issued by UEFI on Raspberry Pi 2), actual hardware does not do this.

Signed-off-by: Andrew Baumann <Andrew.Baumann@microsoft.com>
Reviewed-by: Peter Crosthwaite <crosthwaite.peter@gmail.com>
Message-id: 1450738069-18664-3-git-send-email-Andrew.Baumann@microsoft.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/sd/sdhci.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/hw/sd/sdhci.c b/hw/sd/sdhci.c
index bc39d48..dd83e89 100644
--- a/hw/sd/sdhci.c
+++ b/hw/sd/sdhci.c
@@ -243,9 +243,6 @@ static void sdhci_send_command(SDHCIState *s)
             (s->cmdreg & SDHC_CMD_RESPONSE) == SDHC_CMD_RSP_WITH_BUSY) {
             s->norintsts |= SDHC_NIS_TRSCMP;
         }
-    } else if (rlen != 0 && (s->errintstsen & SDHC_EISEN_CMDIDX)) {
-        s->errintsts |= SDHC_EIS_CMDIDX;
-        s->norintsts |= SDHC_NIS_ERR;
     }
 
     if (s->norintstsen & SDHC_NISEN_CMDCMP) {
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [Qemu-devel] [PULL 10/10] sdhci: add optional quirk property to disable card insertion/removal interrupts
  2015-12-22  8:54 [Qemu-devel] [PULL 00/10] Block patches Stefan Hajnoczi
                   ` (8 preceding siblings ...)
  2015-12-22  8:54 ` [Qemu-devel] [PULL 09/10] sdhci: don't raise a command index error for an unexpected response Stefan Hajnoczi
@ 2015-12-22  8:54 ` Stefan Hajnoczi
  2015-12-22 14:47 ` [Qemu-devel] [PULL 00/10] Block patches Peter Maydell
  10 siblings, 0 replies; 12+ messages in thread
From: Stefan Hajnoczi @ 2015-12-22  8:54 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Stefan Hajnoczi, Andrew Baumann

From: Andrew Baumann <Andrew.Baumann@microsoft.com>

This is needed for a quirk of the Raspberry Pi (bcm2835/6) MMC
controller, where the card insert bit is documented as unimplemented
(always reads zero, doesn't generate interrupts) but is in fact
observed on hardware as set at power on, but is cleared (and remains
clear) on subsequent controller resets.

Signed-off-by: Andrew Baumann <Andrew.Baumann@microsoft.com>
Reviewed-by: Peter Crosthwaite <crosthwaite.peter@gmail.com>
Message-id: 1450738069-18664-4-git-send-email-Andrew.Baumann@microsoft.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/sd/sdhci.c         | 5 ++++-
 include/hw/sd/sdhci.h | 1 +
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/hw/sd/sdhci.c b/hw/sd/sdhci.c
index dd83e89..7acb4d7 100644
--- a/hw/sd/sdhci.c
+++ b/hw/sd/sdhci.c
@@ -193,7 +193,9 @@ static void sdhci_reset(SDHCIState *s)
      * initialization */
     memset(&s->sdmasysad, 0, (uintptr_t)&s->capareg - (uintptr_t)&s->sdmasysad);
 
-    sd_set_cb(s->card, s->ro_cb, s->eject_cb);
+    if (!s->noeject_quirk) {
+        sd_set_cb(s->card, s->ro_cb, s->eject_cb);
+    }
     s->data_count = 0;
     s->stopped_state = sdhc_not_stopped;
 }
@@ -1276,6 +1278,7 @@ static Property sdhci_sysbus_properties[] = {
     DEFINE_PROP_UINT32("capareg", SDHCIState, capareg,
             SDHC_CAPAB_REG_DEFAULT),
     DEFINE_PROP_UINT32("maxcurr", SDHCIState, maxcurr, 0),
+    DEFINE_PROP_BOOL("noeject-quirk", SDHCIState, noeject_quirk, false),
     DEFINE_PROP_END_OF_LIST(),
 };
 
diff --git a/include/hw/sd/sdhci.h b/include/hw/sd/sdhci.h
index e78d938..ffd1f80 100644
--- a/include/hw/sd/sdhci.h
+++ b/include/hw/sd/sdhci.h
@@ -77,6 +77,7 @@ typedef struct SDHCIState {
     uint32_t buf_maxsz;
     uint16_t data_count;   /* current element in FIFO buffer */
     uint8_t  stopped_state;/* Current SDHC state */
+    bool     noeject_quirk;/* Quirk to disable card insert/remove interrupts */
     /* Buffer Data Port Register - virtual access point to R and W buffers */
     /* Software Reset Register - always reads as 0 */
     /* Force Event Auto CMD12 Error Interrupt Reg - write only */
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: [Qemu-devel] [PULL 00/10] Block patches
  2015-12-22  8:54 [Qemu-devel] [PULL 00/10] Block patches Stefan Hajnoczi
                   ` (9 preceding siblings ...)
  2015-12-22  8:54 ` [Qemu-devel] [PULL 10/10] sdhci: add optional quirk property to disable card insertion/removal interrupts Stefan Hajnoczi
@ 2015-12-22 14:47 ` Peter Maydell
  10 siblings, 0 replies; 12+ messages in thread
From: Peter Maydell @ 2015-12-22 14:47 UTC (permalink / raw)
  To: Stefan Hajnoczi; +Cc: QEMU Developers

On 22 December 2015 at 08:54, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> The following changes since commit c688084506cf2cf2eba4ba9df4e91abb6e3dab83:
>
>   Merge remote-tracking branch 'remotes/berrange/tags/pull-qcrypto-secrets-base-2015-12-18-1' into staging (2015-12-18 17:04:15 +0000)
>
> are available in the git repository at:
>
>   git://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 723697551a7e926abe7d3c7f2966012b8075143d:
>
>   sdhci: add optional quirk property to disable card insertion/removal interrupts (2015-12-22 16:34:26 +0800)
>
> ----------------------------------------------------------------
>
> ----------------------------------------------------------------
>
> Andrew Baumann (2):
>   sdhci: don't raise a command index error for an unexpected response
>   sdhci: add optional quirk property to disable card insertion/removal
>     interrupts
>
> Gonglei (1):
>   virtio-blk: trivial code optimization
>
> Peter Crosthwaite (1):
>   sd: sdhci: Delete over-zealous power check
>
> Stefan Hajnoczi (4):
>   block: add BlockLimits.max_iov field
>   block-backend: add blk_get_max_iov()
>   block: replace IOV_MAX with BlockLimits.max_iov
>   block/mirror: replace IOV_MAX with blk_get_max_iov()
>
> Vladimir Sementsov-Ogievskiy (1):
>   parallels: add format spec
>
> Yang Wei (1):
>   scripts/gdb: Fix a python exception in mtree.py
>
>  MAINTAINERS                    |   1 +
>  block/block-backend.c          |   5 +
>  block/io.c                     |  10 +-
>  block/mirror.c                 |   6 +-
>  docs/specs/parallels.txt       | 228 +++++++++++++++++++++++++++++++++++++++++
>  hw/block/virtio-blk.c          |  28 ++---
>  hw/sd/sdhci.c                  |  10 +-
>  include/block/block_int.h      |   3 +
>  include/hw/sd/sdhci.h          |   1 +
>  include/sysemu/block-backend.h |   1 +
>  scripts/qemugdb/mtree.py       |  10 +-
>  11 files changed, 272 insertions(+), 31 deletions(-)
>  create mode 100644 docs/specs/parallels.txt

Applied, thanks.

-- PMM

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2015-12-22 14:47 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-12-22  8:54 [Qemu-devel] [PULL 00/10] Block patches Stefan Hajnoczi
2015-12-22  8:54 ` [Qemu-devel] [PULL 01/10] virtio-blk: trivial code optimization Stefan Hajnoczi
2015-12-22  8:54 ` [Qemu-devel] [PULL 02/10] block: add BlockLimits.max_iov field Stefan Hajnoczi
2015-12-22  8:54 ` [Qemu-devel] [PULL 03/10] block-backend: add blk_get_max_iov() Stefan Hajnoczi
2015-12-22  8:54 ` [Qemu-devel] [PULL 04/10] block: replace IOV_MAX with BlockLimits.max_iov Stefan Hajnoczi
2015-12-22  8:54 ` [Qemu-devel] [PULL 05/10] block/mirror: replace IOV_MAX with blk_get_max_iov() Stefan Hajnoczi
2015-12-22  8:54 ` [Qemu-devel] [PULL 06/10] parallels: add format spec Stefan Hajnoczi
2015-12-22  8:54 ` [Qemu-devel] [PULL 07/10] scripts/gdb: Fix a python exception in mtree.py Stefan Hajnoczi
2015-12-22  8:54 ` [Qemu-devel] [PULL 08/10] sd: sdhci: Delete over-zealous power check Stefan Hajnoczi
2015-12-22  8:54 ` [Qemu-devel] [PULL 09/10] sdhci: don't raise a command index error for an unexpected response Stefan Hajnoczi
2015-12-22  8:54 ` [Qemu-devel] [PULL 10/10] sdhci: add optional quirk property to disable card insertion/removal interrupts Stefan Hajnoczi
2015-12-22 14:47 ` [Qemu-devel] [PULL 00/10] Block patches Peter Maydell

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).