[Qemu-devel] Adding dmcrypt to QEMU block drivers

[Qemu-devel] Adding dmcrypt to QEMU block drivers

[Hamilton, Peter A.]

Hi qemu-devel,

I am a member of a development team based out of the Johns Hopkins University Applied Physics Laboratory. Over the past year and a half, we've been working with the OpenStack community on several security features for their Compute and Block Storage services that leverage encrypted data storage. One of these features, ephemeral storage encryption for qcow2-based virtual machines, would leverage the encryption functionality built into the qcow2 file format. However, there are significant issues with the security and implementation of the qcow2 encryption feature that preclude us from using it in OpenStack. For example, there is no support for the following security features: rekeying of encrypted images, key stretching, and cipher configurability.

After discussing some of these details with Daniel Berrange, we are interested in working with you to add and improve the encryption support offered by QEMU. In the past, Daniel has advocated the full adaptation of the LUKS file format used by dmcrypt, which we currently use in OpenStack. Our proposal would focus on adding a dmcrypt-style encryption layer above the QEMU block driver layer, which would transparently encrypt and decrypt all data written to or read from the underlying block device. This would provide encryption support for all backends and file formats supported by QEMU that leverage block drivers. Such support in QEMU provides significantly improved security and renders the existing encryption scheme provided by qcow2 obsolete.

My intent at the moment is to get a feel for your thoughts and concerns about this proposal and to determine who is currently working on QEMU security features or would be interested in working with us on this feature. I've found past discussions in the QEMU community addressing these encryption concerns but am unaware at the moment what the status is for those development efforts. I'd be happy to provide additional information about our past and current work on OpenStack security if anyone is interested.

Selected References:
OpenStack contributions
- our blueprint for encrypted block storage - https://blueprints.launchpad.net/nova/+spec/encrypt-cinder-volumes
- our blueprint for encrypted ephemeral storage - https://blueprints.launchpad.net/nova/+spec/encrypt-ephemeral-storage

QEMU security discussions
- proposals for improving QEMU encryption - https://lists.gnu.org/archive/html/qemu-devel/2013-07/msg03904.html
- prior discussion between myself and Daniel Berrange on encryption - http://lists.gnu.org/archive/html/qemu-devel/2013-06/msg04869.html
- patch describing qcow2 encryption issues - https://lists.gnu.org/archive/html/qemu-devel/2014-01/msg02802.html

Thanks for your time,
Peter Hamilton

Re: [Qemu-devel] Adding dmcrypt to QEMU block drivers

[Markus Armbruster]

Cc'ing a few interested parties.

"Hamilton, Peter A." <Peter.Hamilton@jhuapl.edu> writes:

> Hi qemu-devel,
>
> I am a member of a development team based out of the Johns Hopkins
> University Applied Physics Laboratory. Over the past year and a half,
> we've been working with the OpenStack community on several security
> features for their Compute and Block Storage services that leverage
> encrypted data storage. One of these features, ephemeral storage
> encryption for qcow2-based virtual machines, would leverage the
> encryption functionality built into the qcow2 file format. However,
> there are significant issues with the security and implementation of
> the qcow2 encryption feature that preclude us from using it in
> OpenStack. For example, there is no support for the following security
> features: rekeying of encrypted images, key stretching, and cipher
> configurability.

I think most, if not all of us would agree that the existing encryption
support in QEMU is fertilizer.

> After discussing some of these details with Daniel Berrange, we are
> interested in working with you to add and improve the encryption
> support offered by QEMU. In the past, Daniel has advocated the full
> adaptation of the LUKS file format used by dmcrypt, which we currently
> use in OpenStack. Our proposal would focus on adding a dmcrypt-style
> encryption layer above the QEMU block driver layer, which would
> transparently encrypt and decrypt all data written to or read from the
> underlying block device. This would provide encryption support for all
> backends and file formats supported by QEMU that leverage block
> drivers. Such support in QEMU provides significantly improved security
> and renders the existing encryption scheme provided by qcow2 obsolete.
>
> My intent at the moment is to get a feel for your thoughts and
> concerns about this proposal and to determine who is currently working
> on QEMU security features or would be interested in working with us on
> this feature. I've found past discussions in the QEMU community
> addressing these encryption concerns but am unaware at the moment what
> the status is for those development efforts. I'd be happy to provide
> additional information about our past and current work on OpenStack
> security if anyone is interested.

As far as I know, nobody is working on block device encryption in QEMU
at this time.

The block layer is maintained by Kevin and Stefan (both cc'ed).

They, Eric and myself can assist you with interfaces.

[...]

Re: [Qemu-devel] Adding dmcrypt to QEMU block drivers

[Stefan Hajnoczi]

On Mon, Mar 17, 2014 at 08:48:08PM -0400, Hamilton, Peter A. wrote:
> Hi qemu-devel,
> 
> I am a member of a development team based out of the Johns Hopkins University Applied Physics Laboratory. Over the past year and a half, we've been working with the OpenStack community on several security features for their Compute and Block Storage services that leverage encrypted data storage. One of these features, ephemeral storage encryption for qcow2-based virtual machines, would leverage the encryption functionality built into the qcow2 file format. However, there are significant issues with the security and implementation of the qcow2 encryption feature that preclude us from using it in OpenStack. For example, there is no support for the following security features: rekeying of encrypted images, key stretching, and cipher configurability.
> 
> After discussing some of these details with Daniel Berrange, we are interested in working with you to add and improve the encryption support offered by QEMU. In the past, Daniel has advocated the full adaptation of the LUKS file format used by dmcrypt, which we currently use in OpenStack. Our proposal would focus on adding a dmcrypt-style encryption layer above the QEMU block driver layer, which would transparently encrypt and decrypt all data written to or read from the underlying block device. This would provide encryption support for all backends and file formats supported by QEMU that leverage block drivers. Such support in QEMU provides significantly improved security and renders the existing encryption scheme provided by qcow2 obsolete.
> 
> My intent at the moment is to get a feel for your thoughts and concerns about this proposal and to determine who is currently working on QEMU security features or would be interested in working with us on this feature. I've found past discussions in the QEMU community addressing these encryption concerns but am unaware at the moment what the status is for those development efforts. I'd be happy to provide additional information about our past and current work on OpenStack security if anyone is interested.

I'm not aware of anyone actively working on improving encryption.
Kevin, Markus, I, and others in the community can review patches.

I'm not sure what your concrete proposal is:

A LUKS implementation inside QEMU as a block (filter) driver?

I guess the filter would be deployed below the image format:
qcow2 -> luks -> file

Is there a way to verify against dmcrypt for testing purposes (e.g.
reading a file created with QEMU using dmcrypt to check the contents are
identical and vice versa).

Please let the LUKS/dmcrypt maintainers know about this effort.  We are
not crypto experts, maybe they can help.

> Selected References:
> OpenStack contributions
> - our blueprint for encrypted block storage - https://blueprints.launchpad.net/nova/+spec/encrypt-cinder-volumes
> - our blueprint for encrypted ephemeral storage - https://blueprints.launchpad.net/nova/+spec/encrypt-ephemeral-storage
> 
> QEMU security discussions
> - proposals for improving QEMU encryption - https://lists.gnu.org/archive/html/qemu-devel/2013-07/msg03904.html
> - prior discussion between myself and Daniel Berrange on encryption - http://lists.gnu.org/archive/html/qemu-devel/2013-06/msg04869.html
> - patch describing qcow2 encryption issues - https://lists.gnu.org/archive/html/qemu-devel/2014-01/msg02802.html
> 
> Thanks for your time,
> Peter Hamilton
> 

Re: [Qemu-devel] Adding dmcrypt to QEMU block drivers

[Daniel P. Berrange]

On Tue, Mar 18, 2014 at 02:08:19PM +0100, Stefan Hajnoczi wrote:
> On Mon, Mar 17, 2014 at 08:48:08PM -0400, Hamilton, Peter A. wrote:
> > Hi qemu-devel,
> > 
> > I am a member of a development team based out of the Johns Hopkins University Applied Physics Laboratory. Over the past year and a half, we've been working with the OpenStack community on several security features for their Compute and Block Storage services that leverage encrypted data storage. One of these features, ephemeral storage encryption for qcow2-based virtual machines, would leverage the encryption functionality built into the qcow2 file format. However, there are significant issues with the security and implementation of the qcow2 encryption feature that preclude us from using it in OpenStack. For example, there is no support for the following security features: rekeying of encrypted images, key stretching, and cipher configurability.
> > 
> > After discussing some of these details with Daniel Berrange, we are interested in working with you to add and improve the encryption support offered by QEMU. In the past, Daniel has advocated the full adaptation of the LUKS file format used by dmcrypt, which we currently use in OpenStack. Our proposal would focus on adding a dmcrypt-style encryption layer above the QEMU block driver layer, which would transparently encrypt and decrypt all data written to or read from the underlying block device. This would provide encryption support for all backends and file formats supported by QEMU that leverage block drivers. Such support in QEMU provides significantly improved security and renders the existing encryption scheme provided by qcow2 obsolete.
> > 
> > My intent at the moment is to get a feel for your thoughts and concerns about this proposal and to determine who is currently working on QEMU security features or would be interested in working with us on this feature. I've found past discussions in the QEMU community addressing these encryption concerns but am unaware at the moment what the status is for those development efforts. I'd be happy to provide additional information about our past and current work on OpenStack security if anyone is interested.
> 
> I'm not aware of anyone actively working on improving encryption.
> Kevin, Markus, I, and others in the community can review patches.

I'm also up for reviewing any designs or patches, from the POV of
libvirt & openstack reuqirements in particular.
 
> I'm not sure what your concrete proposal is:
> 
> A LUKS implementation inside QEMU as a block (filter) driver?

Yes, that's basically it IIUC.

> I guess the filter would be deployed below the image format:
> qcow2 -> luks -> file

I could see it being either above or below the image format at
mgmt app's choice. Having it above the image format means only
the payload is encrypted, so you can still query the basic
metadata (like logic disk size, backing files) without decrypting,
which is a nice aspect of the way qcow2 encryption historically
worked. I could see though that people might want even the header
encrypted to prevent anyone seeing anything about the image format
without keys.

Also, we shouldn't be focusing on QCow2 here. While we're certainly
aiming to obsolete QCow2's encryption, we should be aiming to cover
any of the drivers. eg people using the built-in rbd/iscsi/gluster/nfs
backends want to be able to use encryption too - we don't want to
force them to abandon the QEMU native block drivers and go to the
kernel for these network protocols just to use encryption.

> Is there a way to verify against dmcrypt for testing purposes (e.g.
> reading a file created with QEMU using dmcrypt to check the contents are
> identical and vice versa).

I would expect that bi-directional conversion should be a core goal
eg you ought to be able to point "qemu-img convert" at an existing
LUKS device and turn it into a qcow2 image without needing to
decrypt/re-encrypt any of the content. Likewise take a qcow2 file
and turn it into a LUKS device without re-encrypting the payload
data. Such a conversion facility would be a good first step to
verifying compatibility of QEMU's impl.

> Please let the LUKS/dmcrypt maintainers know about this effort.  We are
> not crypto experts, maybe they can help.
> 
> > Selected References:
> > OpenStack contributions
> > - our blueprint for encrypted block storage - https://blueprints.launchpad.net/nova/+spec/encrypt-cinder-volumes
> > - our blueprint for encrypted ephemeral storage - https://blueprints.launchpad.net/nova/+spec/encrypt-ephemeral-storage
> > 
> > QEMU security discussions
> > - proposals for improving QEMU encryption - https://lists.gnu.org/archive/html/qemu-devel/2013-07/msg03904.html
> > - prior discussion between myself and Daniel Berrange on encryption - http://lists.gnu.org/archive/html/qemu-devel/2013-06/msg04869.html
> > - patch describing qcow2 encryption issues - https://lists.gnu.org/archive/html/qemu-devel/2014-01/msg02802.html
> > 

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] Adding dmcrypt to QEMU block drivers

[Kevin Wolf]

Am 18.03.2014 um 14:30 hat Daniel P. Berrange geschrieben:
> Also, we shouldn't be focusing on QCow2 here. While we're certainly
> aiming to obsolete QCow2's encryption, we should be aiming to cover
> any of the drivers. eg people using the built-in rbd/iscsi/gluster/nfs
> backends want to be able to use encryption too - we don't want to
> force them to abandon the QEMU native block drivers and go to the
> kernel for these network protocols just to use encryption.

I think the part that the qcow2 block driver should be contributing is
just that it can automatically create an encryption layer if the image
file header contains a flag that this new encryption mechanism is used.
This way a similar interface as before could be provided, where the user
basically just says '-hda encrypted.qcow2' and qemu will ask for the
password.

Kevin

Re: [Qemu-devel] Adding dmcrypt to QEMU block drivers

[Stefan Hajnoczi]

On Tue, Mar 18, 2014 at 01:30:44PM +0000, Daniel P. Berrange wrote:
> On Tue, Mar 18, 2014 at 02:08:19PM +0100, Stefan Hajnoczi wrote:
> > On Mon, Mar 17, 2014 at 08:48:08PM -0400, Hamilton, Peter A. wrote:
> > I guess the filter would be deployed below the image format:
> > qcow2 -> luks -> file
> 
> I could see it being either above or below the image format at
> mgmt app's choice. Having it above the image format means only
> the payload is encrypted, so you can still query the basic
> metadata (like logic disk size, backing files) without decrypting,
> which is a nice aspect of the way qcow2 encryption historically
> worked. I could see though that people might want even the header
> encrypted to prevent anyone seeing anything about the image format
> without keys.

The difference is that if you encrypt just the payload then dmcrypt
compatibility is much less useful since the data is now intermingled
with image format metadata.

But I agree in some cases it may be the right thing to do.

Re: [Qemu-devel] Adding dmcrypt to QEMU block drivers

[Daniel P. Berrange]

On Thu, Mar 20, 2014 at 09:23:14AM +0100, Stefan Hajnoczi wrote:
> On Tue, Mar 18, 2014 at 01:30:44PM +0000, Daniel P. Berrange wrote:
> > On Tue, Mar 18, 2014 at 02:08:19PM +0100, Stefan Hajnoczi wrote:
> > > On Mon, Mar 17, 2014 at 08:48:08PM -0400, Hamilton, Peter A. wrote:
> > > I guess the filter would be deployed below the image format:
> > > qcow2 -> luks -> file
> > 
> > I could see it being either above or below the image format at
> > mgmt app's choice. Having it above the image format means only
> > the payload is encrypted, so you can still query the basic
> > metadata (like logic disk size, backing files) without decrypting,
> > which is a nice aspect of the way qcow2 encryption historically
> > worked. I could see though that people might want even the header
> > encrypted to prevent anyone seeing anything about the image format
> > without keys.
> 
> The difference is that if you encrypt just the payload then dmcrypt
> compatibility is much less useful since the data is now intermingled
> with image format metadata.

You do get some crazy apps which format a block device with qcow2
though - eg oVirt. So conceptually even doing qcow2 inside a
dmcrypt block could make some sense if apps like that setup!

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|