From: Stefan Hajnoczi <stefanha@redhat.com>
To: qemu-devel@nongnu.org
Cc: Kevin Wolf <kwolf@redhat.com>, Maor Lipchuk <mlipchuk@redhat.com>,
Eric Blake <eblake@redhat.com>, Max Reitz <mreitz@redhat.com>,
Alberto Garcia <berto@igalia.com>, John Snow <jsnow@redhat.com>,
Nir Soffer <nsoffer@redhat.com>,
"Daniel P. Berrange" <berrange@redhat.com>,
Stefan Hajnoczi <stefanha@redhat.com>
Subject: [Qemu-devel] [PATCH v9 4/9] qcow2: make refcount size calculation conservative
Date: Wed, 5 Jul 2017 13:57:33 +0100 [thread overview]
Message-ID: <20170705125738.8777-5-stefanha@redhat.com> (raw)
In-Reply-To: <20170705125738.8777-1-stefanha@redhat.com>
The refcount metadata size calculation is inaccurate and can produce
numbers that are too small. This is bad because we should calculate a
conservative number - one that is guaranteed to be large enough.
This patch switches the approach to a fixed point calculation because
the existing equation is hard to solve when inaccuracies are taken care
of.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
---
block/qcow2.c | 83 ++++++++++++++++++++++++++++++-----------------------------
1 file changed, 42 insertions(+), 41 deletions(-)
diff --git a/block/qcow2.c b/block/qcow2.c
index 058cc56..538f867 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2140,6 +2140,43 @@ static int preallocate(BlockDriverState *bs)
return 0;
}
+/* qcow2_refcount_metadata_size:
+ * @clusters: number of clusters to refcount (including data and L1/L2 tables)
+ * @cluster_size: size of a cluster, in bytes
+ * @refcount_order: refcount bits power-of-2 exponent
+ *
+ * Returns: Number of bytes required for refcount blocks and table metadata.
+ */
+static int64_t qcow2_refcount_metadata_size(int64_t clusters,
+ size_t cluster_size,
+ int refcount_order)
+{
+ /*
+ * Every host cluster is reference-counted, including metadata (even
+ * refcount metadata is recursively included).
+ *
+ * An accurate formula for the size of refcount metadata size is difficult
+ * to derive. An easier method of calculation is finding the fixed point
+ * where no further refcount blocks or table clusters are required to
+ * reference count every cluster.
+ */
+ int64_t blocks_per_table_cluster = cluster_size / sizeof(uint64_t);
+ int64_t refcounts_per_block = cluster_size * 8 / (1 << refcount_order);
+ int64_t table = 0; /* number of refcount table clusters */
+ int64_t blocks = 0; /* number of refcount block clusters */
+ int64_t last;
+ int64_t n = 0;
+
+ do {
+ last = n;
+ blocks = DIV_ROUND_UP(clusters + table + blocks, refcounts_per_block);
+ table = DIV_ROUND_UP(blocks, blocks_per_table_cluster);
+ n = clusters + blocks + table;
+ } while (n != last);
+
+ return (blocks + table) * cluster_size;
+}
+
/**
* qcow2_calc_prealloc_size:
* @total_size: virtual disk size in bytes
@@ -2153,22 +2190,9 @@ static int64_t qcow2_calc_prealloc_size(int64_t total_size,
size_t cluster_size,
int refcount_order)
{
- /* Note: The following calculation does not need to be exact; if it is a
- * bit off, either some bytes will be "leaked" (which is fine) or we
- * will need to increase the file size by some bytes (which is fine,
- * too, as long as the bulk is allocated here). Therefore, using
- * floating point arithmetic is fine. */
int64_t meta_size = 0;
- uint64_t nreftablee, nrefblocke, nl1e, nl2e, refblock_count;
+ uint64_t nl1e, nl2e;
int64_t aligned_total_size = align_offset(total_size, cluster_size);
- int cluster_bits = ctz32(cluster_size);
- int refblock_bits, refblock_size;
- /* refcount entry size in bytes */
- double rces = (1 << refcount_order) / 8.;
-
- /* see qcow2_open() */
- refblock_bits = cluster_bits - (refcount_order - 3);
- refblock_size = 1 << refblock_bits;
/* header: 1 cluster */
meta_size += cluster_size;
@@ -2183,33 +2207,10 @@ static int64_t qcow2_calc_prealloc_size(int64_t total_size,
nl1e = align_offset(nl1e, cluster_size / sizeof(uint64_t));
meta_size += nl1e * sizeof(uint64_t);
- /* total size of refcount blocks
- *
- * note: every host cluster is reference-counted, including metadata
- * (even refcount blocks are recursively included).
- * Let:
- * a = total_size (this is the guest disk size)
- * m = meta size not including refcount blocks and refcount tables
- * c = cluster size
- * y1 = number of refcount blocks entries
- * y2 = meta size including everything
- * rces = refcount entry size in bytes
- * then,
- * y1 = (y2 + a)/c
- * y2 = y1 * rces + y1 * rces * sizeof(u64) / c + m
- * we can get y1:
- * y1 = (a + m) / (c - rces - rces * sizeof(u64) / c)
- */
- nrefblocke = (aligned_total_size + meta_size + cluster_size)
- / (cluster_size - rces - rces * sizeof(uint64_t)
- / cluster_size);
- refblock_count = DIV_ROUND_UP(nrefblocke, refblock_size);
- meta_size += refblock_count * cluster_size;
-
- /* total size of refcount tables */
- nreftablee = align_offset(refblock_count,
- cluster_size / sizeof(uint64_t));
- meta_size += nreftablee * sizeof(uint64_t);
+ /* total size of refcount table and blocks */
+ meta_size += qcow2_refcount_metadata_size(
+ (meta_size + aligned_total_size) / cluster_size,
+ cluster_size, refcount_order);
return meta_size + aligned_total_size;
}
--
2.9.4
next prev parent reply other threads:[~2017-07-05 12:58 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-05 12:57 [Qemu-devel] [PATCH v9 0/9] qemu-img: add measure sub-command Stefan Hajnoczi
2017-07-05 12:57 ` [Qemu-devel] [PATCH v9 1/9] block: add bdrv_measure() API Stefan Hajnoczi
2017-07-05 12:57 ` [Qemu-devel] [PATCH v9 2/9] raw-format: add bdrv_measure() support Stefan Hajnoczi
2017-07-05 12:57 ` [Qemu-devel] [PATCH v9 3/9] qcow2: extract preallocation calculation function Stefan Hajnoczi
2017-07-05 12:57 ` Stefan Hajnoczi [this message]
2017-07-05 12:57 ` [Qemu-devel] [PATCH v9 5/9] qcow2: extract image creation option parsing Stefan Hajnoczi
2017-07-05 12:57 ` [Qemu-devel] [PATCH v9 6/9] qcow2: add bdrv_measure() support Stefan Hajnoczi
2017-07-11 18:59 ` Eric Blake
2017-07-11 19:04 ` Eric Blake
2017-07-05 12:57 ` [Qemu-devel] [PATCH v9 7/9] qemu-img: add measure subcommand Stefan Hajnoczi
2017-07-05 12:57 ` [Qemu-devel] [PATCH v9 8/9] qemu-iotests: support per-format golden output files Stefan Hajnoczi
2017-07-05 12:57 ` [Qemu-devel] [PATCH v9 9/9] iotests: add test 178 for qemu-img measure Stefan Hajnoczi
2017-07-09 19:36 ` [Qemu-devel] [PATCH v9 0/9] qemu-img: add measure sub-command Max Reitz
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170705125738.8777-5-stefanha@redhat.com \
--to=stefanha@redhat.com \
--cc=berrange@redhat.com \
--cc=berto@igalia.com \
--cc=eblake@redhat.com \
--cc=jsnow@redhat.com \
--cc=kwolf@redhat.com \
--cc=mlipchuk@redhat.com \
--cc=mreitz@redhat.com \
--cc=nsoffer@redhat.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).