[Qemu-devel] [PATCH V3 0/3] Show backing file ancestors count in HMP

[Qemu-devel] [PATCH V3 0/3] Show backing file ancestors count in HMP

[benoit.canet]

From: Benoît Canet <benoit@irqsave.net>

In some setups many backing files and snapshot are chained.
This lead to the formation of huge trees of snapshots all depending
on a common ancestor.

Hence if something bad happen to this common ancestor all the snapshot
of the tree will be broken.

This patch add an easy way for the user to monitor backing file ancestors
count and take the good decision (streaming).

in v2:

lcapitulino: -Fix typo in qapi-schema.json
             -rename *file_ancestors_count to
              *backing_file_ancestors_count

in v3:

kwolf: qapi-schema.json backing_file_ancestors_count field not optional
       Add a (since 1.2) comment
       keep display in HMP optional

Benoît Canet (3):
  block: create bdrv_get_backing_file_ancestors_count()
  block: Use bdrv_get_backing_file_ancestors_count()
  hmp: show the backing file ancestors count

 block.c          |   16 ++++++++++++++++
 block.h          |    1 +
 hmp.c            |    2 ++
 qapi-schema.json |    9 ++++++---
 4 files changed, 25 insertions(+), 3 deletions(-)

-- 
1.7.9.5

[Qemu-devel] [PATCH V3 1/3] block: create bdrv_get_backing_file_ancestors_count()

[benoit.canet]

From: Benoît Canet <benoit@irqsave.net>

Create bdrv_get_backing_file_ancestors_count() in order to be
able to show in QMP and HMP how many ancestors backing an image a
block device have.

Signed-off-by: Benoit Canet <benoit@irqsave.net>
---
 block.c |   13 +++++++++++++
 block.h |    1 +
 2 files changed, 14 insertions(+)

diff --git a/block.c b/block.c
index ce7eb8f..03e0860 100644
--- a/block.c
+++ b/block.c
@@ -2754,6 +2754,19 @@ BlockDriverState *bdrv_find_backing_image(BlockDriverState *bs,
     return NULL;
 }
 
+int bdrv_get_backing_file_ancestors_count(BlockDriverState *bs)
+{
+    if (!bs->drv) {
+        return 0;
+    }
+
+    if (!bs->backing_hd) {
+        return 0;
+    }
+
+    return 1 + bdrv_get_backing_file_ancestors_count(bs->backing_hd);
+}
+
 #define NB_SUFFIXES 4
 
 char *get_human_readable_size(char *buf, int buf_size, int64_t size)
diff --git a/block.h b/block.h
index c89590d..5b1ba2b 100644
--- a/block.h
+++ b/block.h
@@ -174,6 +174,7 @@ int coroutine_fn bdrv_co_is_allocated_above(BlockDriverState *top,
                                             int nb_sectors, int *pnum);
 BlockDriverState *bdrv_find_backing_image(BlockDriverState *bs,
     const char *backing_file);
+int bdrv_get_backing_file_ancestors_count(BlockDriverState *bs);
 int bdrv_truncate(BlockDriverState *bs, int64_t offset);
 int64_t bdrv_getlength(BlockDriverState *bs);
 int64_t bdrv_get_allocated_file_size(BlockDriverState *bs);
-- 
1.7.9.5

[Qemu-devel] [PATCH V3 2/3] block: Use bdrv_get_backing_file_ancestors_count()

[benoit.canet]

From: Benoît Canet <benoit@irqsave.net>

Use the dedicated counting function in qmp_query_block in order to
propagate the backing file count to HMP.

Signed-off-by: Benoit Canet <benoit@irqsave.net>
---
 block.c          |    3 +++
 qapi-schema.json |    9 ++++++---
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/block.c b/block.c
index 03e0860..929605d 100644
--- a/block.c
+++ b/block.c
@@ -2450,6 +2450,9 @@ BlockInfoList *qmp_query_block(Error **errp)
                 info->value->inserted->backing_file = g_strdup(bs->backing_file);
             }
 
+            info->value->inserted->backing_file_ancestors_count =
+                bdrv_get_backing_file_ancestors_count(bs);
+
             if (bs->io_limits_enabled) {
                 info->value->inserted->bps =
                                bs->io_limits.bps[BLOCK_IO_LIMIT_TOTAL];
diff --git a/qapi-schema.json b/qapi-schema.json
index a92adb1..21c1c2a 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -398,6 +398,8 @@
 #
 # @backing_file: #optional the name of the backing file (for copy-on-write)
 #
+# @backing_file_ancestors_count: #the count of ancestors backing files (since: 1.2)
+#
 # @encrypted: true if the backing device is encrypted
 #
 # @bps: total throughput limit in bytes per second is specified
@@ -418,9 +420,10 @@
 ##
 { 'type': 'BlockDeviceInfo',
   'data': { 'file': 'str', 'ro': 'bool', 'drv': 'str',
-            '*backing_file': 'str', 'encrypted': 'bool',
-            'bps': 'int', 'bps_rd': 'int', 'bps_wr': 'int',
-            'iops': 'int', 'iops_rd': 'int', 'iops_wr': 'int'} }
+            '*backing_file': 'str', 'backing_file_ancestors_count': 'int',
+            'encrypted': 'bool', 'bps': 'int', 'bps_rd': 'int',
+            'bps_wr': 'int', 'iops': 'int', 'iops_rd': 'int',
+            'iops_wr': 'int'} }
 
 ##
 # @BlockDeviceIoStatus:
-- 
1.7.9.5

[Qemu-devel] [PATCH V3 3/3] hmp: show the backing file ancestors count

[benoit.canet]

From: Benoît Canet <benoit@irqsave.net>

Signed-off-by: Benoit Canet <benoit@irqsave.net>
---
 hmp.c |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/hmp.c b/hmp.c
index 6b72a64..19dcb65 100644
--- a/hmp.c
+++ b/hmp.c
@@ -227,6 +227,8 @@ void hmp_info_block(Monitor *mon)
             if (info->value->inserted->has_backing_file) {
                 monitor_printf(mon, " backing_file=");
                 monitor_print_filename(mon, info->value->inserted->backing_file);
+                monitor_printf(mon, " backing_file_ancestors_count=%" PRId64,
+                    info->value->inserted->backing_file_ancestors_count);
             }
             monitor_printf(mon, " ro=%d drv=%s encrypted=%d",
                            info->value->inserted->ro,
-- 
1.7.9.5

Re: [Qemu-devel] [PATCH V3 1/3] block: create bdrv_get_backing_file_ancestors_count()

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 2029 bytes --]

On 07/25/2012 06:36 AM, benoit.canet@gmail.com wrote:
> From: Benoît Canet <benoit@irqsave.net>
> 
> Create bdrv_get_backing_file_ancestors_count() in order to be
> able to show in QMP and HMP how many ancestors backing an image a
> block device have.
> 
> Signed-off-by: Benoit Canet <benoit@irqsave.net>
> ---
>  block.c |   13 +++++++++++++
>  block.h |    1 +
>  2 files changed, 14 insertions(+)
> 
> diff --git a/block.c b/block.c
> index ce7eb8f..03e0860 100644
> --- a/block.c
> +++ b/block.c
> @@ -2754,6 +2754,19 @@ BlockDriverState *bdrv_find_backing_image(BlockDriverState *bs,
>      return NULL;
>  }
>  
> +int bdrv_get_backing_file_ancestors_count(BlockDriverState *bs)
> +{
> +    if (!bs->drv) {
> +        return 0;
> +    }
> +
> +    if (!bs->backing_hd) {
> +        return 0;
> +    }
> +
> +    return 1 + bdrv_get_backing_file_ancestors_count(bs->backing_hd);

Is there any risk of stack overflow for a hugely nested setup?  Then
again, I suspect you are going to run into other issues if you are
nested that deeply, before this recursion could cause overflow worth
worrying about.

This is an O(n) operation; is it worth storing the nesting depth as part
of a BlockDriverState to turn it into an O(1) operation?  That is, you
already have to do the O(n) traversal once when originally opening the
chain, so why not have opening the chain populate a struct member with
how deep the opening went, so that all future queries can just return
that struct member.  Of course, you then run into the issue that
operations that change the depth (snapshot, block streaming, and the
proposed block commit) would then be O(n) to adjust the counts of every
member of the chain, instead of O(1) because the count is computed on
the fly.  Just food for thought, and not a technical reason requiring
you to rewrite the patch unless you really like the idea.

-- 
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 620 bytes --]

Re: [Qemu-devel] [PATCH V3 2/3] block: Use bdrv_get_backing_file_ancestors_count()

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1093 bytes --]

On 07/25/2012 06:36 AM, benoit.canet@gmail.com wrote:
> From: Benoît Canet <benoit@irqsave.net>
> 
> Use the dedicated counting function in qmp_query_block in order to
> propagate the backing file count to HMP.
> 
> Signed-off-by: Benoit Canet <benoit@irqsave.net>

> +++ b/qapi-schema.json
> @@ -398,6 +398,8 @@
>  #
>  # @backing_file: #optional the name of the backing file (for copy-on-write)
>  #
> +# @backing_file_ancestors_count: #the count of ancestors backing files (since: 1.2)

s/ancestors/ancestor's/ ?

s/#//

You are making it sound like this is the count of ancestors of the
backing file (that is, 'base <- sn1 <- sn2' would have a count of 1,
since there is 1 ancestor of the backing file sn1; and what if I have no
backing file?), rather than a count of ancestors of this file (base has
a count of 0, base <- sn1 has a count of 1).  Maybe a better wording is:

@backing_file_depth: number of files in the backing file chain (since: 1.2)

-- 
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 620 bytes --]

Re: [Qemu-devel] [PATCH V3 2/3] block: Use bdrv_get_backing_file_ancestors_count()

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1220 bytes --]

On 07/25/2012 06:36 AM, benoit.canet@gmail.com wrote:

> +++ b/qapi-schema.json
> @@ -398,6 +398,8 @@
>  #
>  # @backing_file: #optional the name of the backing file (for copy-on-write)
>  #
> +# @backing_file_ancestors_count: #the count of ancestors backing files (since: 1.2)
> +#
>  # @encrypted: true if the backing device is encrypted
>  #
>  # @bps: total throughput limit in bytes per second is specified
> @@ -418,9 +420,10 @@
>  ##
>  { 'type': 'BlockDeviceInfo',
>    'data': { 'file': 'str', 'ro': 'bool', 'drv': 'str',
> -            '*backing_file': 'str', 'encrypted': 'bool',
> -            'bps': 'int', 'bps_rd': 'int', 'bps_wr': 'int',
> -            'iops': 'int', 'iops_rd': 'int', 'iops_wr': 'int'} }
> +            '*backing_file': 'str', 'backing_file_ancestors_count': 'int',

I hit send too soon.  QMP prefers '-' over '_', so the new parameter
should be 'backing-file-ancestors-count' (or with my proposal,
'backing-file-depth').  Yes, this particular struct has some _ for
historical reasons, but that doesn't mean that new elements must share
the misery.

-- 
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 620 bytes --]

Re: [Qemu-devel] [PATCH V3 1/3] block: create bdrv_get_backing_file_ancestors_count()

[Benoît Canet]

Le Wednesday 25 Jul 2012 à 10:27:09 (-0600), Eric Blake a écrit :
> On 07/25/2012 06:36 AM, benoit.canet@gmail.com wrote:
> > From: Benoît Canet <benoit@irqsave.net>
> > 
> > Create bdrv_get_backing_file_ancestors_count() in order to be
> > able to show in QMP and HMP how many ancestors backing an image a
> > block device have.
> > 
> > Signed-off-by: Benoit Canet <benoit@irqsave.net>
> > ---
> >  block.c |   13 +++++++++++++
> >  block.h |    1 +
> >  2 files changed, 14 insertions(+)
> > 
> > diff --git a/block.c b/block.c
> > index ce7eb8f..03e0860 100644
> > --- a/block.c
> > +++ b/block.c
> > @@ -2754,6 +2754,19 @@ BlockDriverState *bdrv_find_backing_image(BlockDriverState *bs,
> >      return NULL;
> >  }
> >  
> > +int bdrv_get_backing_file_ancestors_count(BlockDriverState *bs)
> > +{
> > +    if (!bs->drv) {
> > +        return 0;
> > +    }
> > +
> > +    if (!bs->backing_hd) {
> > +        return 0;
> > +    }
> > +
> > +    return 1 + bdrv_get_backing_file_ancestors_count(bs->backing_hd);
> 
> Is there any risk of stack overflow for a hugely nested setup?  Then
> again, I suspect you are going to run into other issues if you are
> nested that deeply, before this recursion could cause overflow worth
> worrying about.
>
I don't know.

> This is an O(n) operation; is it worth storing the nesting depth as part
> of a BlockDriverState to turn it into an O(1) operation?  That is, you
> already have to do the O(n) traversal once when originally opening the
> chain, so why not have opening the chain populate a struct member with
> how deep the opening went, so that all future queries can just return
> that struct member.  Of course, you then run into the issue that
> operations that change the depth (snapshot, block streaming, and the
> proposed block commit) would then be O(n) to adjust the counts of every
> member of the chain, instead of O(1) because the count is computed on
> the fly.  Just food for thought, and not a technical reason requiring
> you to rewrite the patch unless you really like the idea.

Hello,

I agree that 0(1) would be way better but wouldn't it increase further
the know how required to touch the block layer.
(I think about newcomers like me implementing another feature requiring
to change the depth and not being aware of this obligation)

Benoît

> 
> -- 
> Eric Blake   eblake@redhat.com    +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
> 

Re: [Qemu-devel] [PATCH V3 2/3] block: Use bdrv_get_backing_file_ancestors_count()

[Benoît Canet]

Le Wednesday 25 Jul 2012 à 10:37:53 (-0600), Eric Blake a écrit :
> On 07/25/2012 06:36 AM, benoit.canet@gmail.com wrote:
> > From: Benoît Canet <benoit@irqsave.net>
> > 
> > Use the dedicated counting function in qmp_query_block in order to
> > propagate the backing file count to HMP.
> > 
> > Signed-off-by: Benoit Canet <benoit@irqsave.net>
> 
> > +++ b/qapi-schema.json
> > @@ -398,6 +398,8 @@
> >  #
> >  # @backing_file: #optional the name of the backing file (for copy-on-write)
> >  #
> > +# @backing_file_ancestors_count: #the count of ancestors backing files (since: 1.2)
> 
> s/ancestors/ancestor's/ ?
> 
> s/#//
> 
> You are making it sound like this is the count of ancestors of the
> backing file (that is, 'base <- sn1 <- sn2' would have a count of 1,
> since there is 1 ancestor of the backing file sn1; and what if I have no
> backing file?), rather than a count of ancestors of this file (base has
> a count of 0, base <- sn1 has a count of 1).  Maybe a better wording is:
> 
> @backing_file_depth: number of files in the backing file chain (since: 1.2)

I'll change it

Benoît

> 
> -- 
> Eric Blake   eblake@redhat.com    +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
> 

Re: [Qemu-devel] [PATCH V3 2/3] block: Use bdrv_get_backing_file_ancestors_count()

[Benoît Canet]

Le Wednesday 25 Jul 2012 à 10:39:11 (-0600), Eric Blake a écrit :
> On 07/25/2012 06:36 AM, benoit.canet@gmail.com wrote:
> 
> > +++ b/qapi-schema.json
> > @@ -398,6 +398,8 @@
> >  #
> >  # @backing_file: #optional the name of the backing file (for copy-on-write)
> >  #
> > +# @backing_file_ancestors_count: #the count of ancestors backing files (since: 1.2)
> > +#
> >  # @encrypted: true if the backing device is encrypted
> >  #
> >  # @bps: total throughput limit in bytes per second is specified
> > @@ -418,9 +420,10 @@
> >  ##
> >  { 'type': 'BlockDeviceInfo',
> >    'data': { 'file': 'str', 'ro': 'bool', 'drv': 'str',
> > -            '*backing_file': 'str', 'encrypted': 'bool',
> > -            'bps': 'int', 'bps_rd': 'int', 'bps_wr': 'int',
> > -            'iops': 'int', 'iops_rd': 'int', 'iops_wr': 'int'} }
> > +            '*backing_file': 'str', 'backing_file_ancestors_count': 'int',
> 
> I hit send too soon.  QMP prefers '-' over '_', so the new parameter
> should be 'backing-file-ancestors-count' (or with my proposal,
> 'backing-file-depth').  Yes, this particular struct has some _ for
> historical reasons, but that doesn't mean that new elements must share
> the misery.

I'll change it too.

Benoît
> 
> -- 
> Eric Blake   eblake@redhat.com    +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>