[Qemu-devel] How to specify the full block driver tree on the CLI ?

[Qemu-devel] How to specify the full block driver tree on the CLI ?

[Daniel P. Berrange]

As previously mentioned, I'm working on support for LUKS full disk
encryption in QEMU. I have a simple driver implemented that works
on top of plain files. eg I can launch qemu-io thus:

 $ qemu-io /home/berrange/VirtualMachines/demo.luks-aes-cbc-plain-sha256

and it'll probe the luks format & instantiate my "luks" block driver impl
on top of the "file" driver. IIUC, I should be able to layer this format
driver on top of any of the QEMU block driver backends though. In particular
I want to be able to layer it on top of any of the network drivers (RBD,
iSCSI and glusterfs). I'm struggling to figure out the right syntax to
specify this to QEMU though, using either qemu-io, or the system emulators
with the -drive arg.  Are there any docs somewhere about the way to
structure the command line arguments to build up a stack of block drivers.
I'd like to figure out the following combinations, for qemu-io, qemu-img
and system emulator -drive syntax.

 - luks -> file
 - qcow2 -> luks -> file
 - luks -> rbd
 - luks -> iscsi
 - luks -> glusterfs

Currently the only required QemuOpt for the luks driver is the ID of
a secret to provide the password.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] How to specify the full block driver tree on the CLI ?

[Kevin Wolf]

Am 29.10.2015 um 00:58 hat Daniel P. Berrange geschrieben:
> As previously mentioned, I'm working on support for LUKS full disk
> encryption in QEMU. I have a simple driver implemented that works
> on top of plain files. eg I can launch qemu-io thus:
> 
>  $ qemu-io /home/berrange/VirtualMachines/demo.luks-aes-cbc-plain-sha256
> 
> and it'll probe the luks format & instantiate my "luks" block driver impl
> on top of the "file" driver. IIUC, I should be able to layer this format
> driver on top of any of the QEMU block driver backends though. In particular
> I want to be able to layer it on top of any of the network drivers (RBD,
> iSCSI and glusterfs).

This part should work automatically as well if you just use the right
URL. qemu probes the format even if you're using a non-file protocol.

> I'm struggling to figure out the right syntax to
> specify this to QEMU though, using either qemu-io, or the system emulators
> with the -drive arg.  Are there any docs somewhere about the way to
> structure the command line arguments to build up a stack of block drivers.

You have a two options. The one that is universal (that is, it works in
all places that open an image), but a bit awkward to use manually is the
json: pseudo-protocol. The "filename" then contains a JSON object of the
QAPI BlockdevOptions type. The QAPI schema (qapi/block-core.json) is
probably the best documentation you get. When specifying JSON objects on
the -drive command line option, don't forget to escape commas by
doubling.

For example:

    qemu-system-x86_64 -drive file='json:{"driver":"luks",,\
    "secret":"x",,"file":{"driver":"file",,"filename":"test.luks"}}'

In qemu proper, you can use a dot syntax for -drive instead:

    qemu-system-x86_64 -drive \
    driver=luks,\
    secret=x,\
    file.driver=file,\
    file.filename=test.luks

In qemu-io, you can't use such syntax on the command line, but the open
command supports an -o option that accepts the same dot syntax.

Note that qemu-img can't deal with this stuff yet, so you'll have
trouble creating an image with such a specification. I guess you need to
create it as a local file first and then use non-qemu tools to copy it
somewhere where it's exported by rbd, iscsi or gluster.

> I'd like to figure out the following combinations, for qemu-io, qemu-img
> and system emulator -drive syntax.
> 
>  - luks -> file
>  - qcow2 -> luks -> file

This is the only case that isn't exactly the same as the example above.
I guess eventually we'll want to make qemu probe on top of luks, so that
you can just specify file=foo.qcow2.luks and it works.

For now, you have to be explicit and nest a level deeper:

    qemu-system-x86_64 -drive \
    driver=qcow2,\
    file.driver=luks,\
    file.secret=x,\
    file.file.driver=file,\
    file.file.filename=test.luks

Or the same structure in JSON, of course.

>  - luks -> rbd
>  - luks -> iscsi
>  - luks -> glusterfs
> 
> Currently the only required QemuOpt for the luks driver is the ID of
> a secret to provide the password.

Hope that helps.

Kevin

Re: [Qemu-devel] How to specify the full block driver tree on the CLI ?

[Daniel P. Berrange]

On Thu, Oct 29, 2015 at 09:11:15AM +0100, Kevin Wolf wrote:
> Am 29.10.2015 um 00:58 hat Daniel P. Berrange geschrieben:
> > As previously mentioned, I'm working on support for LUKS full disk
> > encryption in QEMU. I have a simple driver implemented that works
> > on top of plain files. eg I can launch qemu-io thus:
> > 
> >  $ qemu-io /home/berrange/VirtualMachines/demo.luks-aes-cbc-plain-sha256
> > 
> > and it'll probe the luks format & instantiate my "luks" block driver impl
> > on top of the "file" driver. IIUC, I should be able to layer this format
> > driver on top of any of the QEMU block driver backends though. In particular
> > I want to be able to layer it on top of any of the network drivers (RBD,
> > iSCSI and glusterfs).
> 
> This part should work automatically as well if you just use the right
> URL. qemu probes the format even if you're using a non-file protocol.

Ahh, interesting, I guess I should have just tested it rather than
assuming it doesn't work :-)

> > I'm struggling to figure out the right syntax to
> > specify this to QEMU though, using either qemu-io, or the system emulators
> > with the -drive arg.  Are there any docs somewhere about the way to
> > structure the command line arguments to build up a stack of block drivers.
> 
> You have a two options. The one that is universal (that is, it works in
> all places that open an image), but a bit awkward to use manually is the
> json: pseudo-protocol. The "filename" then contains a JSON object of the
> QAPI BlockdevOptions type. The QAPI schema (qapi/block-core.json) is
> probably the best documentation you get. When specifying JSON objects on
> the -drive command line option, don't forget to escape commas by
> doubling.
> 
> For example:
> 
>     qemu-system-x86_64 -drive file='json:{"driver":"luks",,\
>     "secret":"x",,"file":{"driver":"file",,"filename":"test.luks"}}'
> 
> In qemu proper, you can use a dot syntax for -drive instead:
> 
>     qemu-system-x86_64 -drive \
>     driver=luks,\
>     secret=x,\
>     file.driver=file,\
>     file.filename=test.luks
> 
> In qemu-io, you can't use such syntax on the command line, but the open
> command supports an -o option that accepts the same dot syntax.
> 
> Note that qemu-img can't deal with this stuff yet, so you'll have
> trouble creating an image with such a specification. I guess you need to
> create it as a local file first and then use non-qemu tools to copy it
> somewhere where it's exported by rbd, iscsi or gluster.

I wonder if my patches to qemu-io & qemu-img here do the right thing to
make this dot syntax work....

   https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg04382.html
   https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg04375.html

> 
> > I'd like to figure out the following combinations, for qemu-io, qemu-img
> > and system emulator -drive syntax.
> > 
> >  - luks -> file
> >  - qcow2 -> luks -> file
> 
> This is the only case that isn't exactly the same as the example above.
> I guess eventually we'll want to make qemu probe on top of luks, so that
> you can just specify file=foo.qcow2.luks and it works.
> 
> For now, you have to be explicit and nest a level deeper:
> 
>     qemu-system-x86_64 -drive \
>     driver=qcow2,\
>     file.driver=luks,\
>     file.secret=x,\
>     file.file.driver=file,\
>     file.file.filename=test.luks
> 
> Or the same structure in JSON, of course.
> 
> >  - luks -> rbd
> >  - luks -> iscsi
> >  - luks -> glusterfs
> > 
> > Currently the only required QemuOpt for the luks driver is the ID of
> > a secret to provide the password.
> 
> Hope that helps.

Yep, very helpful thanks.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] How to specify the full block driver tree on the CLI ?

[Kevin Wolf]

Am 29.10.2015 um 09:15 hat Daniel P. Berrange geschrieben:
> On Thu, Oct 29, 2015 at 09:11:15AM +0100, Kevin Wolf wrote:
> > In qemu proper, you can use a dot syntax for -drive instead:
> > 
> >     qemu-system-x86_64 -drive \
> >     driver=luks,\
> >     secret=x,\
> >     file.driver=file,\
> >     file.filename=test.luks
> > 
> > In qemu-io, you can't use such syntax on the command line, but the open
> > command supports an -o option that accepts the same dot syntax.
> > 
> > Note that qemu-img can't deal with this stuff yet, so you'll have
> > trouble creating an image with such a specification. I guess you need to
> > create it as a local file first and then use non-qemu tools to copy it
> > somewhere where it's exported by rbd, iscsi or gluster.
> 
> I wonder if my patches to qemu-io & qemu-img here do the right thing to
> make this dot syntax work....
> 
>    https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg04382.html
>    https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg04375.html

I haven't looked at the series in detail yet, but considering that it's
probably harder to prevent it from working than getting it, I assume
that your patches do allow it.

Just passing the options QDict to bdrv_open() is enough, nesting is
represented with the dot syntax in the keys there. In fact, if you use
blockdev-add in QMP, it first converts the options to a flattened QDict
with dot syntax for the keys before it processes it.

Kevin