Re: [Qemu-devel] blobstore disk format (was Re: Design of the blobstore)

[Stefan Berger <stefanb@linux.vnet.ibm.com>]

On 09/17/2011 03:28 PM, Michael S. Tsirkin wrote:
> On Fri, Sep 16, 2011 at 12:46:40PM -0400, Stefan Berger wrote:
>> On 09/16/2011 10:44 AM, Michael S. Tsirkin wrote:
>>> On Thu, Sep 15, 2011 at 10:33:13AM -0400, Stefan Berger wrote:
>>>> On 09/15/2011 08:28 AM, Michael S. Tsirkin wrote:
>>>>> So the below is a proposal for a directory scheme
>>>>> for storing (optionally multiple) nvram images,
>>>>> along with any metadata.
>>>>> Data is encoded using BER:
>>>>> http://en.wikipedia.org/wiki/Basic_Encoding_Rules
>>>>> Specifically, we mostly use the subsets.
>>>>>
>>>> Would it change anything if we were to think of the NVRAM image as
>>>> another piece of metadata?
>>> Yes, we can do that, sure. I had the feeling that it will help to lay
>>> out the image at the end, to make directory listing
>>> more efficient - the rest of metadata is usually small,
>>> image might be somewhat large.
>>>
>> Why not let a convenience library handle the metadata on the device
>> level, having it create the blob that the NVRAM layer ends up
>> writing and parsing before the device uses it? Otherwise I should
>> maybe rename the nvram to meatdata_store :-/
> Maybe we are talking about different things. All I agrue for
> is using a common standard format for storing metadata,
> instead of having each device roll its own.
That's fine. The TPM code inside libtpms serializes all internal data 
structures for later resumption. It doesn't use ASN.1 but in effect 
endianess-normalizes them and stores them in a packed format that can 
later be resumed along with a version tag prepended where need . Are you 
suggesting to change that ? I hope not...



>>>> I am also wondering whether each device shouldn't just handle the
>>>> metadata itself,
>>> It could be that just means we will have custom code with
>>> different bugs in each device.
>>> Note that from experience with formats, the problem with
>>> time becomes less trivial than it seems as we
>>> need to provide forward and backward compatibility
>>> guarantees.
>>>
>> Is that guaranteed just by using ASN.1 ?
> At least for BER, yes. We can always skip an optional field
> that we don't recognize without knowing anything about
> its internal format.

>> Do we need to add a
>> revision to the metadata?
> IMO, no. Instead we add optional attributes as long as we can
> preserve backwards compatibility, and madatory attributes
> if we can't.
>
Are devices doing this right now or are these future changes to devices' 
code?

>>>> encryption, integrity value (crc32 or sha1) and so on. What
>>>> metadata should there be that really need to be handled on the NVRAM
>>>> API and below level rather than on the device-specific code level?
>>> So checksum  (checksum value and type) 'and so on' are what I call
>>> metadata :) Doing it at device level seems wrong.
>>>
>> You mean doing it at the NVRAM level seems wrong. Of course, again
>> something a device could write into a header prepended to the actual
>> blob. Maybe every device that needs it should do that so that if we
>> were to support encryption of blobs and the key for decryption was
>> wrong one could detect it early without feeding badly decrypted /
>> corrupted state into the device and see what happens.
> Do what? Checksum the data? Well, error detection is nice,
> but it could be that people actually care about not losing
> all of the data on nvram if qemu is killed.  I also wonder whether
> invalidating all data because of a single bit corruption is a bug or a
> feature.
>
The checksuming I think makes sense if encryption is being added so 
decryption and testing for proper key material remains an NVRAM 
operation rather than a device operation.
>>>>> We use a directory as a SET in a CER format.
>>>>> This allows generating directory online without scanning
>>>>> the entries beforehand.
>>>>>
>>>> I guess it is the 'unknown' for me... but what is the advantage of
>>>> using ASN1 for this rather than just writing out packed and
>>>> endianess-normalized data structures (with revision value),
>>> If you want an example of where this 'custom formats are easy
>>> so let us write one' leads to in the end,
>>> look no further than live migration code.
>>> It's a mess of hacks that does not even work across
>>> upstream qemu versions, leave alone across
>>> downstreams (different linux distros).
>>>
>> So is ASN1 the answer or does one still need to add a revision tag
>> to each blob putting in custom code for parsing the different
>> revisions of data structures (I guess) that may be extended/changed
>> over time?
>>
>>     Stefan
> We don't need revisions. We can always parse a new structure
> skipping optional attributes we don't recognize. In case we want to
> break old qemu versions intentially, we can add
> a mandatory attribute.
So you said you had some code for the handling of ASN.1. Can sketch how 
the interaction of devices would work with mandatory and optional 
attributes along with an API? I'd prefer to NOT have the attributes and 
values be a part of the NVRAM API itself but let a (mandatory) library 
handle the serialization and deserialization of these metadata when a 
device wants to write or read state respectively. But maybe I just want 
to keep the NVRAM API 'too simple'.

    Stefan

>>>> having
>>>> them crc32-protected to have some sanity checking in place?
>>>>
>>>>      Stefan
>>> I'm not sure why we want crc specifically in TPM.
>>> If it is 'just because we can' then it probably
>>> applies to other non-volatile storage?
>>> Storage generally?
>>>
>>>>> The rest of the encoding uses a DER format.
>>>>> This makes for fast parsing as entries are easy to skip.
>>>>>
>>>>> Each entry is encoded in DER format.
>>>>> Each entry is a SEQUENCE with two objects:
>>>>> 1. nvram
>>>>> 2. optional name - a UTF8String
>>>>>
>>>>> Binary data is stored as OCTET-STRING values on disk.
>>>>> Any RW metadata is stored as OCTET-STRING value as well.
>>>>> Any RO metadata is stored in appropriate universal encoding,
>>>>> by type.
>>>>>
>>>>> On the context below, an attribute is either a IA5String or a SEQUENCE.
>>>>> If IA5String, this is the attribute name, and it has no value.
>>>>> If SEQUENCE, the first entry in the sequence is an
>>>>> IA5String, it is the attribute name. The rest of the entries
>>>>> represent the attribute value.
>>>>>
>>>>> Mandatory/optional attributes: depends on type.
>>>>> tpm will have realsize as RW mandatory attribute.
>>>>>
>>>>> Each nvram is built as a SEQUENCE including 4 objects
>>>>> 1. type - an IA5String. downstreams can use other types such as
>>>>>                       UUIDs instead to ensure no conflicts with upstream
>>>>> 2. SET of mandatory attributes
>>>>> 3. SET of optional attributes
>>>>> 4. data - a RW OCTET-STRING
>>>>>
>>>>> It is envisioned that attributes won't be too large,
>>>>> so they can easily be kept in memory.
>>>>>
>>>>>