[dm-crypt] [ANNOUNCE] LUKSMeta

[dm-crypt] [ANNOUNCE] LUKSMeta

[Nathaniel McCallum]

https://github.com/latchset/luksmeta

Hi everyone! Several projects that I am working on or related to
require the ability to store some small metadata that is accessable
before the LUKS volume is unlocked. Since this was not possible with
LUKSv1, and we couldn't wait until LUKSv2, we created a small library
called LUKSMeta.

This simple library allows an application developer to store some
metadata in the gap in the LUKSv1 header (between the end of the
keyslots and the start of the payloadOffset). There are up to eight
"slots" of metadata, similar to the eight keyslots of LUKS. Each slot
is typed by a 16-byte UUID, so that applications don't stomp on each
others' data. Both the LUKSMeta header and the data in each slot is
checksummed (CRC32c) to detect data corruption.

There are four simple functions:

* luksmeta_init() - Write the LUKSMeta header to disk
* luksmeta_get() - Read data/uuid from a LUKSMeta slot
* luksmeta_set() - Write data/uuid to a LUKSMeta slot
* luksmeta_del() - Clear (zero) a LUKSMeta slot

More detailed documentation is available in the header:
https://github.com/latchset/luksmeta/blob/master/luksmeta.h

I have not made the first release, but I would like to do so soon. I
welcome your review/feedback. Thanks!

Nathaniel

Re: [dm-crypt] [ANNOUNCE] LUKSMeta

[Arno Wagner]

Interesting idea.

Do you analyze the header to make sure the gap is there and
of expected size and the LUKS version is known to the library? 
What happens if somebody did a non-default configuration?
What happens with a different header than LUKS v1?

Regards,
Arno

On Fri, May 13, 2016 at 18:01:06 CEST, Nathaniel McCallum wrote:
> https://github.com/latchset/luksmeta
> 
> Hi everyone! Several projects that I am working on or related to
> require the ability to store some small metadata that is accessable
> before the LUKS volume is unlocked. Since this was not possible with
> LUKSv1, and we couldn't wait until LUKSv2, we created a small library
> called LUKSMeta.
> 
> This simple library allows an application developer to store some
> metadata in the gap in the LUKSv1 header (between the end of the
> keyslots and the start of the payloadOffset). There are up to eight
> "slots" of metadata, similar to the eight keyslots of LUKS. Each slot
> is typed by a 16-byte UUID, so that applications don't stomp on each
> others' data. Both the LUKSMeta header and the data in each slot is
> checksummed (CRC32c) to detect data corruption.
> 
> There are four simple functions:
> 
> * luksmeta_init() - Write the LUKSMeta header to disk
> * luksmeta_get() - Read data/uuid from a LUKSMeta slot
> * luksmeta_set() - Write data/uuid to a LUKSMeta slot
> * luksmeta_del() - Clear (zero) a LUKSMeta slot
> 
> More detailed documentation is available in the header:
> https://github.com/latchset/luksmeta/blob/master/luksmeta.h
> 
> I have not made the first release, but I would like to do so soon. I
> welcome your review/feedback. Thanks!
> 
> Nathaniel
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] [ANNOUNCE] LUKSMeta

[Nathaniel McCallum]

The library uses libcryptsetup for all parsing and for locating the
gap. We do not parse anything ourselves. We make the following
assumptions on top of what libcryptsetup officially provides:

* crypt_keyslot_area() offset + length indicates the end of the keyslot
* crypt_get_data_offset() indicates the start of ciphertext data
* Sector size is 512 (ex. crypt_get_data_offset())
* The gap between the last keyslot and the ciphertext data is empty
* 4k alignment for the LUKSMeta header and each LUKSMeta slot

If there is insufficient space to accommodate the current operation, we
return -ENOSPC. If crypt_get_type() does not return CRYPT_LUKS1, we
return -ENOTSUP. All of the major error conditions are documented in
the header: https://github.com/latchset/luksmeta/blob/master/luksmeta.h

In the case of a non-default configuration, libcryptsetup will properly
handle the parsing and we will still perform all of our sanity checks
(like ensuring free space). However, I don't think we support the case
of detatched headers. You are welcome to test this (and patches
welcome). Any customization that violates our above assumptions will
likely result in breakage. However, given that all these assumptions
are also hard-coded in LUKSv1, I don't believe this is possible without
building your own version of libcryptsetup (and breaking
compatibility).

I should also mention that LUKSMeta has an extensive test suite with
near-100% code coverage (we don't currently cover things like hardware
error conditions; patches welcome). This test suite is run on every
commit (via TravisCI). We also periodically run coverity scans (also
via TravisCI).

One additional thing that probably deserves mention: the
crypt_header_backup() and crypt_header_restore() functions do not
currently backup/restore the gap. Thus, using these functions could
result in breakage where the LUKS header is restored but the LUKSMeta
data is not. I don't see any reason why these functions could not be
expanded to also backup/restore the data in the gap. This would not
need to break any existing functionality. But this would be an upstream
change in libcryptsetup.

On Fri, 2016-05-13 at 18:23 +0200, Arno Wagner wrote:
> Interesting idea.
> 
> Do you analyze the header to make sure the gap is there and
> of expected size and the LUKS version is known to the library? 
> What happens if somebody did a non-default configuration?
> What happens with a different header than LUKS v1?
> 
> Regards,
> Arno
> 
> On Fri, May 13, 2016 at 18:01:06 CEST, Nathaniel McCallum wrote:
> > https://github.com/latchset/luksmeta
> > 
> > Hi everyone! Several projects that I am working on or related to
> > require the ability to store some small metadata that is accessable
> > before the LUKS volume is unlocked. Since this was not possible
> > with
> > LUKSv1, and we couldn't wait until LUKSv2, we created a small
> > library
> > called LUKSMeta.
> > 
> > This simple library allows an application developer to store some
> > metadata in the gap in the LUKSv1 header (between the end of the
> > keyslots and the start of the payloadOffset). There are up to eight
> > "slots" of metadata, similar to the eight keyslots of LUKS. Each
> > slot
> > is typed by a 16-byte UUID, so that applications don't stomp on
> > each
> > others' data. Both the LUKSMeta header and the data in each slot is
> > checksummed (CRC32c) to detect data corruption.
> > 
> > There are four simple functions:
> > 
> > * luksmeta_init() - Write the LUKSMeta header to disk
> > * luksmeta_get() - Read data/uuid from a LUKSMeta slot
> > * luksmeta_set() - Write data/uuid to a LUKSMeta slot
> > * luksmeta_del() - Clear (zero) a LUKSMeta slot
> > 
> > More detailed documentation is available in the header:
> > https://github.com/latchset/luksmeta/blob/master/luksmeta.h
> > 
> > I have not made the first release, but I would like to do so soon.
> > I
> > welcome your review/feedback. Thanks!
> > 
> > Nathaniel
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> 

Re: [dm-crypt] [ANNOUNCE] LUKSMeta

[Milan Broz]

Hi,

just for the list - there was some discussion how to store per-slot
metadata for Nathaniel's disk unlocking projects.

Using empty space for metadata is just compromise - we do not need
to touch stable LUKSv1 while these projects can work over existing LUKS devices.

Anyway, I am still taking this as an experimental code, there are for sure
situations when the metadata space is going to be destroyed
(old header backup restore etc).

On 05/13/2016 07:53 PM, Nathaniel McCallum wrote:
> The library uses libcryptsetup for all parsing and for locating the
> gap. We do not parse anything ourselves. We make the following
> assumptions on top of what libcryptsetup officially provides:
> 
> * crypt_keyslot_area() offset + length indicates the end of the keyslot
> * crypt_get_data_offset() indicates the start of ciphertext data
> * Sector size is 512 (ex. crypt_get_data_offset())
> * The gap between the last keyslot and the ciphertext data is empty
> * 4k alignment for the LUKSMeta header and each LUKSMeta slot

yes, all these are true, at least for upstream libcryptsetup.

> However, I don't think we support the case
> of detatched headers. You are welcome to test this (and patches
> welcome).

I do not see reason this should not work, just instead of data device
you have to use device(file) with header.
libcryptsetup exports set_data_device() for adding data device reference later
(but there is trick that in this (only) case data offset can be 0)

Anyway, I do not think it is worth to support it now.
 
> One additional thing that probably deserves mention: the
> crypt_header_backup() and crypt_header_restore() functions do not
> currently backup/restore the gap. Thus, using these functions could
> result in breakage where the LUKS header is restored but the LUKSMeta
> data is not. I don't see any reason why these functions could not be
> expanded to also backup/restore the data in the gap. This would not
> need to break any existing functionality. But this would be an upstream
> change in libcryptsetup.

Because cryptsetup should not backup anything it has no idea about,
it can create suspicion that this area could contain some
backdoor data so we are just wiping it :)
(TBH, wipe logic in libcryptsetup was added to wipe possible data signatures
that can conflict with blkid for example (I remeber mkswap and some version of extX fs)
But this usually happened in the first 4k, not in the gap at the end of header.)

I think the project using your library should provide command that
backups both...
 
(Also current testing code for LUKS2 for in-place upgrade wipes
these data - but this case is not fixable, we are basically
expanding header of empty area.)

Milan

> 
> On Fri, 2016-05-13 at 18:23 +0200, Arno Wagner wrote:
>> Interesting idea.
>>
>> Do you analyze the header to make sure the gap is there and
>> of expected size and the LUKS version is known to the library? 
>> What happens if somebody did a non-default configuration?
>> What happens with a different header than LUKS v1?
>>
>> Regards,
>> Arno
>>
>> On Fri, May 13, 2016 at 18:01:06 CEST, Nathaniel McCallum wrote:
>>> https://github.com/latchset/luksmeta
>>>
>>> Hi everyone! Several projects that I am working on or related to
>>> require the ability to store some small metadata that is accessable
>>> before the LUKS volume is unlocked. Since this was not possible
>>> with
>>> LUKSv1, and we couldn't wait until LUKSv2, we created a small
>>> library
>>> called LUKSMeta.
>>>
>>> This simple library allows an application developer to store some
>>> metadata in the gap in the LUKSv1 header (between the end of the
>>> keyslots and the start of the payloadOffset). There are up to eight
>>> "slots" of metadata, similar to the eight keyslots of LUKS. Each
>>> slot
>>> is typed by a 16-byte UUID, so that applications don't stomp on
>>> each
>>> others' data. Both the LUKSMeta header and the data in each slot is
>>> checksummed (CRC32c) to detect data corruption.
>>>
>>> There are four simple functions:
>>>
>>> * luksmeta_init() - Write the LUKSMeta header to disk
>>> * luksmeta_get() - Read data/uuid from a LUKSMeta slot
>>> * luksmeta_set() - Write data/uuid to a LUKSMeta slot
>>> * luksmeta_del() - Clear (zero) a LUKSMeta slot
>>>
>>> More detailed documentation is available in the header:
>>> https://github.com/latchset/luksmeta/blob/master/luksmeta.h
>>>
>>> I have not made the first release, but I would like to do so soon.
>>> I
>>> welcome your review/feedback. Thanks!
>>>
>>> Nathaniel