* [Qemu-devel] [PATCH] qcow2: Define and use QCOW2_COMPRESSED_SECTOR_SIZE
@ 2019-05-10 16:22 Alberto Garcia
2019-05-13 11:28 ` Stefano Garzarella
2019-05-13 12:58 ` Kevin Wolf
0 siblings, 2 replies; 6+ messages in thread
From: Alberto Garcia @ 2019-05-10 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Alberto Garcia, qemu-block, Max Reitz
When an L2 table entry points to a compressed cluster the space used
by the data is specified in 512-byte sectors. This size is independent
from BDRV_SECTOR_SIZE and is specific to the qcow2 file format.
The QCOW2_COMPRESSED_SECTOR_SIZE constant defined in this patch makes
this explicit.
Signed-off-by: Alberto Garcia <berto@igalia.com>
---
block/qcow2-cluster.c | 5 +++--
block/qcow2-refcount.c | 25 ++++++++++++++-----------
block/qcow2.c | 3 ++-
block/qcow2.h | 4 ++++
4 files changed, 23 insertions(+), 14 deletions(-)
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 974a4e8656..b36f4aa84a 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -796,8 +796,9 @@ int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs,
return cluster_offset;
}
- nb_csectors = ((cluster_offset + compressed_size - 1) >> 9) -
- (cluster_offset >> 9);
+ nb_csectors =
+ (cluster_offset + compressed_size - 1) / QCOW2_COMPRESSED_SECTOR_SIZE -
+ (cluster_offset / QCOW2_COMPRESSED_SECTOR_SIZE);
cluster_offset |= QCOW_OFLAG_COMPRESSED |
((uint64_t)nb_csectors << s->csize_shift);
diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
index fa7ac1f7cb..780bd49a00 100644
--- a/block/qcow2-refcount.c
+++ b/block/qcow2-refcount.c
@@ -1172,12 +1172,11 @@ void qcow2_free_any_clusters(BlockDriverState *bs, uint64_t l2_entry,
switch (ctype) {
case QCOW2_CLUSTER_COMPRESSED:
{
- int nb_csectors;
- nb_csectors = ((l2_entry >> s->csize_shift) &
- s->csize_mask) + 1;
- qcow2_free_clusters(bs,
- (l2_entry & s->cluster_offset_mask) & ~511,
- nb_csectors * 512, type);
+ int64_t offset = (l2_entry & s->cluster_offset_mask)
+ & QCOW2_COMPRESSED_SECTOR_MASK;
+ int size = QCOW2_COMPRESSED_SECTOR_SIZE *
+ (((l2_entry >> s->csize_shift) & s->csize_mask) + 1);
+ qcow2_free_clusters(bs, offset, size, type);
}
break;
case QCOW2_CLUSTER_NORMAL:
@@ -1317,9 +1316,12 @@ int qcow2_update_snapshot_refcount(BlockDriverState *bs,
nb_csectors = ((entry >> s->csize_shift) &
s->csize_mask) + 1;
if (addend != 0) {
+ uint64_t coffset = (entry & s->cluster_offset_mask)
+ & QCOW2_COMPRESSED_SECTOR_MASK;
ret = update_refcount(
- bs, (entry & s->cluster_offset_mask) & ~511,
- nb_csectors * 512, abs(addend), addend < 0,
+ bs, coffset,
+ nb_csectors * QCOW2_COMPRESSED_SECTOR_SIZE,
+ abs(addend), addend < 0,
QCOW2_DISCARD_SNAPSHOT);
if (ret < 0) {
goto fail;
@@ -1635,9 +1637,10 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res,
nb_csectors = ((l2_entry >> s->csize_shift) &
s->csize_mask) + 1;
l2_entry &= s->cluster_offset_mask;
- ret = qcow2_inc_refcounts_imrt(bs, res,
- refcount_table, refcount_table_size,
- l2_entry & ~511, nb_csectors * 512);
+ ret = qcow2_inc_refcounts_imrt(
+ bs, res, refcount_table, refcount_table_size,
+ l2_entry & QCOW2_COMPRESSED_SECTOR_MASK,
+ nb_csectors * QCOW2_COMPRESSED_SECTOR_SIZE);
if (ret < 0) {
goto fail;
}
diff --git a/block/qcow2.c b/block/qcow2.c
index a520d116ef..80679a84d2 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -4188,7 +4188,8 @@ qcow2_co_preadv_compressed(BlockDriverState *bs,
coffset = file_cluster_offset & s->cluster_offset_mask;
nb_csectors = ((file_cluster_offset >> s->csize_shift) & s->csize_mask) + 1;
- csize = nb_csectors * 512 - (coffset & 511);
+ csize = nb_csectors * QCOW2_COMPRESSED_SECTOR_SIZE -
+ (coffset & ~QCOW2_COMPRESSED_SECTOR_MASK);
buf = g_try_malloc(csize);
if (!buf) {
diff --git a/block/qcow2.h b/block/qcow2.h
index fdee297f33..7e796877d6 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -74,6 +74,10 @@
#define MIN_CLUSTER_BITS 9
#define MAX_CLUSTER_BITS 21
+/* Defined in the qcow2 spec (compressed cluster descriptor) */
+#define QCOW2_COMPRESSED_SECTOR_SIZE 512U
+#define QCOW2_COMPRESSED_SECTOR_MASK (~(QCOW2_COMPRESSED_SECTOR_SIZE - 1))
+
/* Must be at least 2 to cover COW */
#define MIN_L2_CACHE_SIZE 2 /* cache entries */
--
2.11.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] [PATCH] qcow2: Define and use QCOW2_COMPRESSED_SECTOR_SIZE
2019-05-10 16:22 [Qemu-devel] [PATCH] qcow2: Define and use QCOW2_COMPRESSED_SECTOR_SIZE Alberto Garcia
@ 2019-05-13 11:28 ` Stefano Garzarella
2019-05-13 11:48 ` Alberto Garcia
2019-05-13 12:58 ` Kevin Wolf
1 sibling, 1 reply; 6+ messages in thread
From: Stefano Garzarella @ 2019-05-13 11:28 UTC (permalink / raw)
To: Alberto Garcia; +Cc: Kevin Wolf, qemu-devel, qemu-block, Max Reitz
The patch LGTM, just a comment below.
On Fri, May 10, 2019 at 07:22:54PM +0300, Alberto Garcia wrote:
> When an L2 table entry points to a compressed cluster the space used
> by the data is specified in 512-byte sectors. This size is independent
> from BDRV_SECTOR_SIZE and is specific to the qcow2 file format.
>
> The QCOW2_COMPRESSED_SECTOR_SIZE constant defined in this patch makes
> this explicit.
>
> Signed-off-by: Alberto Garcia <berto@igalia.com>
> ---
> block/qcow2-cluster.c | 5 +++--
> block/qcow2-refcount.c | 25 ++++++++++++++-----------
> block/qcow2.c | 3 ++-
> block/qcow2.h | 4 ++++
> 4 files changed, 23 insertions(+), 14 deletions(-)
>
> diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
> index 974a4e8656..b36f4aa84a 100644
> --- a/block/qcow2-cluster.c
> +++ b/block/qcow2-cluster.c
> @@ -796,8 +796,9 @@ int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs,
> return cluster_offset;
> }
>
> - nb_csectors = ((cluster_offset + compressed_size - 1) >> 9) -
> - (cluster_offset >> 9);
> + nb_csectors =
> + (cluster_offset + compressed_size - 1) / QCOW2_COMPRESSED_SECTOR_SIZE -
> + (cluster_offset / QCOW2_COMPRESSED_SECTOR_SIZE);
>
> cluster_offset |= QCOW_OFLAG_COMPRESSED |
> ((uint64_t)nb_csectors << s->csize_shift);
> diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
> index fa7ac1f7cb..780bd49a00 100644
> --- a/block/qcow2-refcount.c
> +++ b/block/qcow2-refcount.c
> @@ -1172,12 +1172,11 @@ void qcow2_free_any_clusters(BlockDriverState *bs, uint64_t l2_entry,
> switch (ctype) {
> case QCOW2_CLUSTER_COMPRESSED:
> {
> - int nb_csectors;
> - nb_csectors = ((l2_entry >> s->csize_shift) &
> - s->csize_mask) + 1;
> - qcow2_free_clusters(bs,
> - (l2_entry & s->cluster_offset_mask) & ~511,
> - nb_csectors * 512, type);
> + int64_t offset = (l2_entry & s->cluster_offset_mask)
> + & QCOW2_COMPRESSED_SECTOR_MASK;
> + int size = QCOW2_COMPRESSED_SECTOR_SIZE *
> + (((l2_entry >> s->csize_shift) & s->csize_mask) + 1);
What about using int64_t type for the 'size' variable?
(because the qcow2_free_clusters() 'size' parameter is int64_t)
> + qcow2_free_clusters(bs, offset, size, type);
> }
> break;
> case QCOW2_CLUSTER_NORMAL:
> @@ -1317,9 +1316,12 @@ int qcow2_update_snapshot_refcount(BlockDriverState *bs,
> nb_csectors = ((entry >> s->csize_shift) &
> s->csize_mask) + 1;
> if (addend != 0) {
> + uint64_t coffset = (entry & s->cluster_offset_mask)
> + & QCOW2_COMPRESSED_SECTOR_MASK;
> ret = update_refcount(
> - bs, (entry & s->cluster_offset_mask) & ~511,
> - nb_csectors * 512, abs(addend), addend < 0,
> + bs, coffset,
> + nb_csectors * QCOW2_COMPRESSED_SECTOR_SIZE,
> + abs(addend), addend < 0,
> QCOW2_DISCARD_SNAPSHOT);
> if (ret < 0) {
> goto fail;
> @@ -1635,9 +1637,10 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res,
> nb_csectors = ((l2_entry >> s->csize_shift) &
> s->csize_mask) + 1;
> l2_entry &= s->cluster_offset_mask;
> - ret = qcow2_inc_refcounts_imrt(bs, res,
> - refcount_table, refcount_table_size,
> - l2_entry & ~511, nb_csectors * 512);
> + ret = qcow2_inc_refcounts_imrt(
> + bs, res, refcount_table, refcount_table_size,
> + l2_entry & QCOW2_COMPRESSED_SECTOR_MASK,
> + nb_csectors * QCOW2_COMPRESSED_SECTOR_SIZE);
> if (ret < 0) {
> goto fail;
> }
> diff --git a/block/qcow2.c b/block/qcow2.c
> index a520d116ef..80679a84d2 100644
> --- a/block/qcow2.c
> +++ b/block/qcow2.c
> @@ -4188,7 +4188,8 @@ qcow2_co_preadv_compressed(BlockDriverState *bs,
>
> coffset = file_cluster_offset & s->cluster_offset_mask;
> nb_csectors = ((file_cluster_offset >> s->csize_shift) & s->csize_mask) + 1;
> - csize = nb_csectors * 512 - (coffset & 511);
> + csize = nb_csectors * QCOW2_COMPRESSED_SECTOR_SIZE -
> + (coffset & ~QCOW2_COMPRESSED_SECTOR_MASK);
>
> buf = g_try_malloc(csize);
> if (!buf) {
> diff --git a/block/qcow2.h b/block/qcow2.h
> index fdee297f33..7e796877d6 100644
> --- a/block/qcow2.h
> +++ b/block/qcow2.h
> @@ -74,6 +74,10 @@
> #define MIN_CLUSTER_BITS 9
> #define MAX_CLUSTER_BITS 21
>
> +/* Defined in the qcow2 spec (compressed cluster descriptor) */
> +#define QCOW2_COMPRESSED_SECTOR_SIZE 512U
> +#define QCOW2_COMPRESSED_SECTOR_MASK (~(QCOW2_COMPRESSED_SECTOR_SIZE - 1))
> +
> /* Must be at least 2 to cover COW */
> #define MIN_L2_CACHE_SIZE 2 /* cache entries */
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] [PATCH] qcow2: Define and use QCOW2_COMPRESSED_SECTOR_SIZE
2019-05-13 11:28 ` Stefano Garzarella
@ 2019-05-13 11:48 ` Alberto Garcia
2019-05-13 13:14 ` Stefano Garzarella
0 siblings, 1 reply; 6+ messages in thread
From: Alberto Garcia @ 2019-05-13 11:48 UTC (permalink / raw)
To: Stefano Garzarella; +Cc: Kevin Wolf, qemu-devel, qemu-block, Max Reitz
On Mon 13 May 2019 01:28:46 PM CEST, Stefano Garzarella wrote:
>> + int size = QCOW2_COMPRESSED_SECTOR_SIZE *
>> + (((l2_entry >> s->csize_shift) & s->csize_mask) + 1);
>
> What about using int64_t type for the 'size' variable?
> (because the qcow2_free_clusters() 'size' parameter is int64_t)
The maximum size that can be read from a compressed cluster descriptor
using the formula above is twice the cluster size (more information on
commit abd3622cc03cf41ed542126a540385f30a4c0175 and on the Compressed
Clusters Descriptor spec in docs/interop/qcow2.txt).
Since the maximum allowed cluster size is 2MB, the value of the 'size'
variable can never be larger than 4MB, which fits comfortably on a
32-bit integer. We would need to support 512MB clusters in order to have
problems with this.
Berto
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] [PATCH] qcow2: Define and use QCOW2_COMPRESSED_SECTOR_SIZE
2019-05-10 16:22 [Qemu-devel] [PATCH] qcow2: Define and use QCOW2_COMPRESSED_SECTOR_SIZE Alberto Garcia
2019-05-13 11:28 ` Stefano Garzarella
@ 2019-05-13 12:58 ` Kevin Wolf
1 sibling, 0 replies; 6+ messages in thread
From: Kevin Wolf @ 2019-05-13 12:58 UTC (permalink / raw)
To: Alberto Garcia; +Cc: qemu-devel, qemu-block, Max Reitz
Am 10.05.2019 um 18:22 hat Alberto Garcia geschrieben:
> When an L2 table entry points to a compressed cluster the space used
> by the data is specified in 512-byte sectors. This size is independent
> from BDRV_SECTOR_SIZE and is specific to the qcow2 file format.
>
> The QCOW2_COMPRESSED_SECTOR_SIZE constant defined in this patch makes
> this explicit.
>
> Signed-off-by: Alberto Garcia <berto@igalia.com>
Thanks, applied to the block branch.
Kevin
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] [PATCH] qcow2: Define and use QCOW2_COMPRESSED_SECTOR_SIZE
2019-05-13 11:48 ` Alberto Garcia
@ 2019-05-13 13:14 ` Stefano Garzarella
2019-05-13 13:48 ` Alberto Garcia
0 siblings, 1 reply; 6+ messages in thread
From: Stefano Garzarella @ 2019-05-13 13:14 UTC (permalink / raw)
To: Alberto Garcia; +Cc: Kevin Wolf, qemu-devel, qemu-block, Max Reitz
On Mon, May 13, 2019 at 01:48:27PM +0200, Alberto Garcia wrote:
> On Mon 13 May 2019 01:28:46 PM CEST, Stefano Garzarella wrote:
> >> + int size = QCOW2_COMPRESSED_SECTOR_SIZE *
> >> + (((l2_entry >> s->csize_shift) & s->csize_mask) + 1);
> >
> > What about using int64_t type for the 'size' variable?
> > (because the qcow2_free_clusters() 'size' parameter is int64_t)
>
> The maximum size that can be read from a compressed cluster descriptor
> using the formula above is twice the cluster size (more information on
> commit abd3622cc03cf41ed542126a540385f30a4c0175 and on the Compressed
> Clusters Descriptor spec in docs/interop/qcow2.txt).
>
> Since the maximum allowed cluster size is 2MB, the value of the 'size'
> variable can never be larger than 4MB, which fits comfortably on a
> 32-bit integer. We would need to support 512MB clusters in order to have
> problems with this.
Thanks for the explaination and sorry for that!
Cheers,
Stefano
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] [PATCH] qcow2: Define and use QCOW2_COMPRESSED_SECTOR_SIZE
2019-05-13 13:14 ` Stefano Garzarella
@ 2019-05-13 13:48 ` Alberto Garcia
0 siblings, 0 replies; 6+ messages in thread
From: Alberto Garcia @ 2019-05-13 13:48 UTC (permalink / raw)
To: Stefano Garzarella; +Cc: Kevin Wolf, qemu-devel, qemu-block, Max Reitz
On Mon 13 May 2019 03:14:40 PM CEST, Stefano Garzarella wrote:
>> Since the maximum allowed cluster size is 2MB, the value of the
>> 'size' variable can never be larger than 4MB, which fits comfortably
>> on a 32-bit integer. We would need to support 512MB clusters in order
>> to have problems with this.
>
> Thanks for the explaination and sorry for that!
No need to say sorry, the question was perfectly valid and it could have
been an actual bug, so I appreciate that you wrote the e-mail.
Thanks,
Berto
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-05-13 13:52 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-05-10 16:22 [Qemu-devel] [PATCH] qcow2: Define and use QCOW2_COMPRESSED_SECTOR_SIZE Alberto Garcia
2019-05-13 11:28 ` Stefano Garzarella
2019-05-13 11:48 ` Alberto Garcia
2019-05-13 13:14 ` Stefano Garzarella
2019-05-13 13:48 ` Alberto Garcia
2019-05-13 12:58 ` Kevin Wolf
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).