[Qemu-devel] [RFC] Making 'block-stream', and 'block-commit' accept node-name

[Qemu-devel] [RFC] Making 'block-stream', and 'block-commit' accept node-name

[Kashyap Chamarthy]

Observe the following ('qmp-shell', for brevity) invocation of the four
major types (stream, commit, mirror, backup) of live block operations:

    (QEMU) block-stream device=node-D base=a.qcow2 job-id=job-block-stream
    (QEMU) block-commit device=node-D base=a.qcow2 top=b.qcow2 job-id=job-block-commit
    (QEMU) drive-mirror device=node-D target=e.qcow2 sync=full job-id=job-drive-mirror
    (QEMU) drive-backup device=node-D sync=full target=e.qcow2 job-id=job-drive-backup
    (QEMU) blockdev-backup device=node-B target=node-E sync=full job-id=job-blockdev-backup
    (QEMU) blockdev-mirror device=node-D target=node-E sync=full job-id=job-blockdev-mirror

While we have `blockdev-{mirror, backup}` as 'node-name'-alternatives
for `drive-{mirror, backup}`, as the eagle-eyed will, the `block-stream`
and `block-commit` commands still operate on file names for parameters
'base' and 'top'. 

When I brought it up on #qemu, OFTC, where Max gave some context, and
even said: "We just shouldn't use filenames there ['commit', 'stream'],
simple as that".

Does upstream have a plan here for 2.10 / 3.0?  Will there be new
`blockdev-stream`, and `blockdev-commit`, or will `block-{stream,
commit}` will be adjusted to take 'node-name'?

* * *

Then I _vaguely_ remembered reading a thread on the upstream list about
creating a new 'blockdev-stream' (but I didn't find anything about
'blockdev-commit'), even if a bit confusing.  Sure enough, when I went
digging the archives, I found, where Alberto extended the 'device'
parameter to take 'node-name', and there was an ensuing discussion about
a new 'blockdev-stream':

    https://lists.gnu.org/archive/html/qemu-devel/2016-10/msg03236.html
    -- [PATCH v11 00/19] Support streaming to an intermediate layer

And this patch had the said discussion:

    https://lists.gnu.org/archive/html/qemu-block/2016-10/msg00117.html

tl;dr seemed to be: "add a new `blockdev-stream` [and
`blockdev-commit`?] because it is easy to introspect, and is a 'saner'
interface".

To recap from that thread, Kevin wrote[1]:

    "Oh, we still have those filename-based parameters? :-/

    "Should we introduce a new, clean blockdev-stream command that fixes
    this and matches the common name pattern? Of course, block-stream
    vs.  blockdev-stream could be a bit confusing, too..."

And, Eric seemed agreeable to a new command[2], saying, "a new command
is easy to introspect".

Markus chimed in[3] with some background context, and a design
discussion aobut whether it makes sense to extend existing commands vs.
adding new, giving pros / cons of both.

Finally, Kevin concluded[4]:

    "It has never been a sane interface in the first place (identifying
    a backing file node by its filename).
    
    "We ended up having two versions of all block job commands anyway
    (one that creates an image file, and later one that just takes a
    node-name of an existing node), except for image streaming so far.
    So it would be consistent (and enable consistent naming for the
    preferred commands) to have it here, too."

[1] https://lists.gnu.org/archive/html/qemu-block/2016-10/msg00319.html
[2] https://lists.gnu.org/archive/html/qemu-block/2016-10/msg00328.html 
[3] https://lists.gnu.org/archive/html/qemu-block/2016-10/msg00333.html
[4] https://lists.gnu.org/archive/html/qemu-block/2016-10/msg00354.html

-- 
/kashyap

Re: [Qemu-devel] [RFC] Making 'block-stream', and 'block-commit' accept node-name

[Alberto Garcia]

On Mon 29 May 2017 09:03:22 PM CEST, Kashyap Chamarthy wrote:

> Observe the following ('qmp-shell', for brevity) invocation of the
> four major types (stream, commit, mirror, backup) of live block
> operations:
>
>     (QEMU) block-stream device=node-D base=a.qcow2 job-id=job-block-stream
>     (QEMU) block-commit device=node-D base=a.qcow2 top=b.qcow2 job-id=job-block-commit
>     (QEMU) drive-mirror device=node-D target=e.qcow2 sync=full job-id=job-drive-mirror
>     (QEMU) drive-backup device=node-D sync=full target=e.qcow2 job-id=job-drive-backup
>     (QEMU) blockdev-backup device=node-B target=node-E sync=full job-id=job-blockdev-backup
>     (QEMU) blockdev-mirror device=node-D target=node-E sync=full job-id=job-blockdev-mirror
>
> While we have `blockdev-{mirror, backup}` as 'node-name'-alternatives
> for `drive-{mirror, backup}`, as the eagle-eyed will, the
> `block-stream` and `block-commit` commands still operate on file names
> for parameters 'base' and 'top'.

block-stream does accept a node name for the top and base images (see
554b614765090f47d and 312fe09cc8af86c).

block-commit hasn't changed, but it should be possible to follow a
similar approach.

Berto

Re: [Qemu-devel] [RFC] Making 'block-stream', and 'block-commit' accept node-name

[Kashyap Chamarthy]

On Tue, May 30, 2017 at 04:53:23PM +0200, Alberto Garcia wrote:
> On Mon 29 May 2017 09:03:22 PM CEST, Kashyap Chamarthy wrote:
> 
> > Observe the following ('qmp-shell', for brevity) invocation of the
> > four major types (stream, commit, mirror, backup) of live block
> > operations:
> >
> >     (QEMU) block-stream device=node-D base=a.qcow2 job-id=job-block-stream
> >     (QEMU) block-commit device=node-D base=a.qcow2 top=b.qcow2 job-id=job-block-commit
> >     (QEMU) drive-mirror device=node-D target=e.qcow2 sync=full job-id=job-drive-mirror
> >     (QEMU) drive-backup device=node-D sync=full target=e.qcow2 job-id=job-drive-backup
> >     (QEMU) blockdev-backup device=node-B target=node-E sync=full job-id=job-blockdev-backup
> >     (QEMU) blockdev-mirror device=node-D target=node-E sync=full job-id=job-blockdev-mirror
> >
> > While we have `blockdev-{mirror, backup}` as 'node-name'-alternatives
> > for `drive-{mirror, backup}`, as the eagle-eyed will, the
> > `block-stream` and `block-commit` commands still operate on file names
> > for parameters 'base' and 'top'.
> 
> block-stream does accept a node name for the top and base images (see
> 554b614765090f47d and 312fe09cc8af86c).

Ah, I didn't realize `block-stream` accepts 'node-name' for 'base' and
'top'.  Thanks for the pointers.

But still, I wonder, for consistency's sake: will a new
`blockdev-{stream, commit}` ever be created.

> block-commit hasn't changed, but it should be possible to follow a
> similar approach.

Noted.

[...]

-- 
/kashyap

Re: [Qemu-devel] [RFC] Making 'block-stream', and 'block-commit' accept node-name

[Kashyap Chamarthy]

On Wed, May 31, 2017 at 04:57:20PM +0200, Kashyap Chamarthy wrote:
> On Tue, May 30, 2017 at 04:53:23PM +0200, Alberto Garcia wrote:
> > On Mon 29 May 2017 09:03:22 PM CEST, Kashyap Chamarthy wrote:

[...]

> > > While we have `blockdev-{mirror, backup}` as 'node-name'-alternatives
> > > for `drive-{mirror, backup}`, as the eagle-eyed will, the
> > > `block-stream` and `block-commit` commands still operate on file names
> > > for parameters 'base' and 'top'.
> > 
> > block-stream does accept a node name for the top and base images (see
> > 554b614765090f47d and 312fe09cc8af86c).

Indeed it does.  The key here is the 'base-node' parameter.  Thanks!

Test
----

To turn this:

    [A] <-- [B] <-- [C] <-- [D]

Into:

    [A] <-- [C] <-- [D]

#----------------------------------------------------------------------
Check the backing file depth of image [D]:

    $ qemu-img info --backing-chain d.qcow2  | grep "backing file:"
    backing file: c.qcow2
    backing file: b.qcow2
    backing file: ./a.qcow2
#----------------------------------------------------------------------

Perform intermediate streaming:

(QEMU) block-stream device=node-C base-node=node-A job-id=job0                                                                                                                     
{
    "execute": "block-stream", 
    "arguments": {
        "device": "node-D", 
        "job-id": "job0", 
        "base-node": "node-A"
    }
}
{
    "return": {}
}
(QEMU) 
{u'timestamp': {u'seconds': 1496325065, u'microseconds': 808768}, u'data': {u'device': u'job0', u'type': u'stream', u'speed': 0, u'len': 41126400, u'offset': 41126400}, u'event': u'BLOCK_JOB_COMPLETED'}
#----------------------------------------------------------------------

Check the backing file depth, it is now: A <- C <- D:

$ qemu-img info --backing-chain d.qcow2  | grep "backing file:"
backing file: c.qcow2
backing file: ./a.qcow2
#----------------------------------------------------------------------

[...]

-- 
/kashyap