From: Stefan Hajnoczi <stefanha@redhat.com>
To: qemu-devel@nongnu.org
Cc: Nir Soffer <nsoffer@redhat.com>, Kevin Wolf <kwolf@redhat.com>,
Maor Lipchuk <mlipchuk@redhat.com>,
"Daniel P. Berrange" <berrange@redhat.com>,
Eric Blake <eblake@redhat.com>, Alberto Garcia <berto@igalia.com>,
John Snow <jsnow@redhat.com>,
Stefan Hajnoczi <stefanha@redhat.com>
Subject: [Qemu-devel] [PATCH v6 4/9] qcow2: make refcount size calculation conservative
Date: Mon, 8 May 2017 10:15:31 -0400 [thread overview]
Message-ID: <20170508141536.20690-5-stefanha@redhat.com> (raw)
In-Reply-To: <20170508141536.20690-1-stefanha@redhat.com>
The refcount metadata size calculation is inaccurate and can produce
numbers that are too small. This is bad because we should calculate a
conservative number - one that is guaranteed to be large enough.
This patch switches the approach to a fixed point calculation because
the existing equation is hard to solve when inaccuracies are taken care
of.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
---
block/qcow2.c | 82 ++++++++++++++++++++++++++++++-----------------------------
1 file changed, 42 insertions(+), 40 deletions(-)
diff --git a/block/qcow2.c b/block/qcow2.c
index 5569b63..ff0d825 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2095,6 +2095,43 @@ static int preallocate(BlockDriverState *bs)
return 0;
}
+/* qcow2_refcount_metadata_size:
+ * @clusters: number of clusters to refcount (including data and L1/L2 tables)
+ * @cluster_size: size of a cluster, in bytes
+ * @refcount_order: refcount bits power-of-2 exponent
+ *
+ * Returns: Number of bytes required for refcount blocks and table metadata.
+ */
+static int64_t qcow2_refcount_metadata_size(int64_t clusters,
+ size_t cluster_size,
+ int refcount_order)
+{
+ /*
+ * Every host cluster is reference-counted, including metadata (even
+ * refcount metadata is recursively included).
+ *
+ * An accurate formula for the size of refcount metadata size is difficult
+ * to derive. An easier method of calculation is finding the fixed point
+ * where no further refcount blocks or table clusters are required to
+ * reference count every cluster.
+ */
+ int64_t blocks_per_table_cluster = cluster_size / sizeof(uint64_t);
+ int64_t refcounts_per_block = cluster_size * 8 / (1 << refcount_order);
+ int64_t table = 0; /* number of refcount table clusters */
+ int64_t blocks = 0; /* number of refcount block clusters */
+ int64_t last;
+ int64_t n = 0;
+
+ do {
+ last = n;
+ blocks = DIV_ROUND_UP(clusters + table + blocks, refcounts_per_block);
+ table = DIV_ROUND_UP(blocks, blocks_per_table_cluster);
+ n = clusters + blocks + table;
+ } while (n != last);
+
+ return (blocks + table) * cluster_size;
+}
+
/**
* qcow2_calc_prealloc_size:
* @total_size: virtual disk size in bytes
@@ -2108,22 +2145,9 @@ static int64_t qcow2_calc_prealloc_size(int64_t total_size,
size_t cluster_size,
int refcount_order)
{
- /* Note: The following calculation does not need to be exact; if it is a
- * bit off, either some bytes will be "leaked" (which is fine) or we
- * will need to increase the file size by some bytes (which is fine,
- * too, as long as the bulk is allocated here). Therefore, using
- * floating point arithmetic is fine. */
int64_t meta_size = 0;
- uint64_t nreftablee, nrefblocke, nl1e, nl2e;
+ uint64_t nl1e, nl2e;
int64_t aligned_total_size = align_offset(total_size, cluster_size);
- int cluster_bits = ctz32(cluster_size);
- int refblock_bits, refblock_size;
- /* refcount entry size in bytes */
- double rces = (1 << refcount_order) / 8.;
-
- /* see qcow2_open() */
- refblock_bits = cluster_bits - (refcount_order - 3);
- refblock_size = 1 << refblock_bits;
/* header: 1 cluster */
meta_size += cluster_size;
@@ -2138,32 +2162,10 @@ static int64_t qcow2_calc_prealloc_size(int64_t total_size,
nl1e = align_offset(nl1e, cluster_size / sizeof(uint64_t));
meta_size += nl1e * sizeof(uint64_t);
- /* total size of refcount blocks
- *
- * note: every host cluster is reference-counted, including metadata
- * (even refcount blocks are recursively included).
- * Let:
- * a = total_size (this is the guest disk size)
- * m = meta size not including refcount blocks and refcount tables
- * c = cluster size
- * y1 = number of refcount blocks entries
- * y2 = meta size including everything
- * rces = refcount entry size in bytes
- * then,
- * y1 = (y2 + a)/c
- * y2 = y1 * rces + y1 * rces * sizeof(u64) / c + m
- * we can get y1:
- * y1 = (a + m) / (c - rces - rces * sizeof(u64) / c)
- */
- nrefblocke = (aligned_total_size + meta_size + cluster_size)
- / (cluster_size - rces - rces * sizeof(uint64_t)
- / cluster_size);
- meta_size += DIV_ROUND_UP(nrefblocke, refblock_size) * cluster_size;
-
- /* total size of refcount tables */
- nreftablee = nrefblocke / refblock_size;
- nreftablee = align_offset(nreftablee, cluster_size / sizeof(uint64_t));
- meta_size += nreftablee * sizeof(uint64_t);
+ /* total size of refcount table and blocks */
+ meta_size += qcow2_refcount_metadata_size(
+ (meta_size + aligned_total_size) / cluster_size,
+ cluster_size, refcount_order);
return meta_size + aligned_total_size;
}
--
2.9.3
next prev parent reply other threads:[~2017-05-08 14:16 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-05-08 14:15 [Qemu-devel] [PATCH v6 0/9] qemu-img: add measure sub-command Stefan Hajnoczi
2017-05-08 14:15 ` [Qemu-devel] [PATCH v6 1/9] block: add bdrv_measure() API Stefan Hajnoczi
2017-05-08 14:50 ` Eric Blake
2017-05-09 15:28 ` Stefan Hajnoczi
2017-05-08 14:15 ` [Qemu-devel] [PATCH v6 2/9] raw-format: add bdrv_measure() support Stefan Hajnoczi
2017-05-08 14:52 ` Eric Blake
2017-05-08 14:15 ` [Qemu-devel] [PATCH v6 3/9] qcow2: extract preallocation calculation function Stefan Hajnoczi
2017-05-08 14:15 ` Stefan Hajnoczi [this message]
2017-05-08 15:00 ` [Qemu-devel] [PATCH v6 4/9] qcow2: make refcount size calculation conservative Eric Blake
2017-05-08 19:06 ` Max Reitz
2017-05-08 21:26 ` Max Reitz
2017-05-09 15:32 ` Stefan Hajnoczi
2017-05-08 14:15 ` [Qemu-devel] [PATCH v6 5/9] qcow2: extract image creation option parsing Stefan Hajnoczi
2017-05-08 14:15 ` [Qemu-devel] [PATCH v6 6/9] qcow2: add bdrv_measure() support Stefan Hajnoczi
2017-05-08 14:15 ` [Qemu-devel] [PATCH v6 7/9] qemu-img: add measure subcommand Stefan Hajnoczi
2017-06-12 10:00 ` Alberto Garcia
2017-06-13 9:09 ` Stefan Hajnoczi
2017-05-08 14:15 ` [Qemu-devel] [PATCH v6 8/9] qemu-iotests: support per-format golden output files Stefan Hajnoczi
2017-05-08 14:15 ` [Qemu-devel] [PATCH v6 9/9] iotests: add test 178 for qemu-img measure Stefan Hajnoczi
2017-05-24 12:59 ` [Qemu-devel] [PATCH v6 0/9] qemu-img: add measure sub-command Stefan Hajnoczi
2017-06-12 9:29 ` Stefan Hajnoczi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170508141536.20690-5-stefanha@redhat.com \
--to=stefanha@redhat.com \
--cc=berrange@redhat.com \
--cc=berto@igalia.com \
--cc=eblake@redhat.com \
--cc=jsnow@redhat.com \
--cc=kwolf@redhat.com \
--cc=mlipchuk@redhat.com \
--cc=nsoffer@redhat.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).