[Qemu-devel] WIP: Migration format: ASN.1/BER schema

[Qemu-devel] WIP: Migration format: ASN.1/BER schema

[Dr. David Alan Gilbert]

Hi,
  I've been looking at reviving the migration-as-ber work that Michael and
Stefan looked at a while ago, and have stuff starting to work, but not
ready yet, so I thought I'd start by posting my current view
of an ASN.1 schema.

Note this is my 1st attempt at ASN.1 so feel free to pick holes!

This schema successfully passes asn1c and libtasn1's asn1Parser.

I've currently got a visitor modified Qemu that generated apparently
valid BER output (and compatible old-format) - unber and openssl asn1parse
dump it apparently happily, but I've not been able to find a tool to verify
it against the schema - suggestions welcome.
(* libtasn1's asn1Decoder OOMs my laptop/and or spits out a DER error, not
that I'm trying to encode as DER)

Some notes:
  * I'm using the universal type codes for most things except the larger
    scale structures/sequences where I've given them our own types.
  * ASN type codes are disgustingly decimal, and even more disgustingly encoded
    in BER, so having things readable in a hex dump is difficult; although the
    values I've used end up with the last byte as an appropriate ASCII letter
    and the other bytes would be if the top bit wasn't set.
  * The 'shape' of it corresponds pretty closely to the shape of the existing
    migration format; although I intend to clean it up a bit 
    - RAMSecEntry has the same flags that currently go into our binary format
      but those can become more of an enum, and sima).
    - I think SecFull/SecMin can probably merge together.
  * Most things are just sequences, including most of VMState; I don't really
    want to go around every VMState structure in the whole of QEMU allocating
    a type ID, however I'm thinking about making it an optional VMState field
    which I'd use if it was set.
  * The only thing that worries me in terms of efficiency so far is the
    'page is all 0' compression encoding which is rather verbose, for everything
    else it seems that although it's chatty it's not that much of an overhead.
  * I've not looked at block migration or spapr yet, (the only other iterative
    migrate users)
  * anything with a .get/.put or using the old style migration registration
    gets bundled up into a blob as an octet-string - this means I don't have
    to fix everything before it can start working.

As soon as I get the corresponding BER input visitor working and sweep the
sawdust out of my code I'll post it; probably a week or two.

Thoughts welcome,

Dave
   

--------------------------------------------------------------------------------

Qemu {}
DEFINITIONS IMPLICIT TAGS ::=
BEGIN

-- Some basic types used in multiple places --
QemuString ::= UTF8String (SIZE (1..255))

-- TODO: 4096 is actually page size whatever that is
FullPage ::= OCTET STRING (SIZE (4096))

RAMBlockID ::= SEQUENCE {
  name   QemuString,
  len    INTEGER
}

RAMSecEntry ::= [ APPLICATION 8914 ] SEQUENCE {
  addr         INTEGER,   -- Address or offset or size
  flags        INTEGER,   -- maybe more explicit type?
  name         QemuString OPTIONAL,

  body         CHOICE {
    bl         SEQUENCE OF RAMBlockID,
    compr      INTEGER (0..255), -- Page filled with this value
    page       FullPage
    -- TODO xbzrle --
  }
}

RAMSecList ::= [ APPLICATION 9810 ] SEQUENCE OF RAMSecEntry

SubSection ::= [ APPLICATION 10707 ] SEQUENCE {
  name         QemuString,
  versionid    INTEGER,

  contents     SEQUENCE OF VMStateEntries
}

SubSecList ::= [ APPLICATION 10700 ] SEQUENCE OF SubSection

VMStateEntries ::= CHOICE {
  -- Hmm need to think more --
  dummy    INTEGER,
  subsecl  SubSecList,
  oldblob  OCTET STRING
}

VMState ::= SEQUENCE OF VMStateEntries

-- Restrict to unsigned?
SectionID ::= INTEGER

SecFull ::= [ APPLICATION 2003 ] SEQUENCE {
  name         QemuString,
  sectionid    SectionID,
  instanceid   INTEGER,
  versionid    INTEGER,

  contents     CHOICE {
    ramsec     RAMSecList,
    -- TODO other iterator initial stuff --
    vmstate    VMState
  }
}

SecMin ::= [ APPLICATION 211 ] SEQUENCE {
  sectionid   SectionID,

  contents     CHOICE {
    ramsec     SEQUENCE OF RAMSecEntry
    -- TODO other iterator general/end stuff --
  }
}

Sections ::= CHOICE {
  full    SecFull,
  min     SecMin
}

-- The whole file --
-- Application tag used to get first 32bits of file
-- to come out as 7f cd c5 51  - the 51 is Q
-- the c5 and cd being E,M but with the top bit set
-- which BER requires
QemuFile ::= [ APPLICATION 1270481 ] SEQUENCE {
  version INTEGER (3),
  top     SEQUENCE OF Sections
}

END

--------------------------------------------------------------------------------
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] WIP: Migration format: ASN.1/BER schema

[Michael S. Tsirkin]

On Fri, Mar 07, 2014 at 04:55:03PM +0000, Dr. David Alan Gilbert wrote:
> Hi,
>   I've been looking at reviving the migration-as-ber work that Michael and
> Stefan looked at a while ago, and have stuff starting to work, but not
> ready yet, so I thought I'd start by posting my current view
> of an ASN.1 schema.
> 
> Note this is my 1st attempt at ASN.1 so feel free to pick holes!
> 
> This schema successfully passes asn1c and libtasn1's asn1Parser.
> 
> I've currently got a visitor modified Qemu that generated apparently
> valid BER output (and compatible old-format) - unber and openssl asn1parse
> dump it apparently happily, but I've not been able to find a tool to verify
> it against the schema - suggestions welcome.
> (* libtasn1's asn1Decoder OOMs my laptop/and or spits out a DER error, not
> that I'm trying to encode as DER)
> 
> Some notes:
>   * I'm using the universal type codes for most things except the larger
>     scale structures/sequences where I've given them our own types.
>   * ASN type codes are disgustingly decimal, and even more disgustingly encoded
>     in BER, so having things readable in a hex dump is difficult; although the
>     values I've used end up with the last byte as an appropriate ASCII letter
>     and the other bytes would be if the top bit wasn't set.
>   * The 'shape' of it corresponds pretty closely to the shape of the existing
>     migration format; although I intend to clean it up a bit 
>     - RAMSecEntry has the same flags that currently go into our binary format
>       but those can become more of an enum, and sima).
>     - I think SecFull/SecMin can probably merge together.
>   * Most things are just sequences, including most of VMState; I don't really
>     want to go around every VMState structure in the whole of QEMU allocating
>     a type ID, however I'm thinking about making it an optional VMState field
>     which I'd use if it was set.
>   * The only thing that worries me in terms of efficiency so far is the
>     'page is all 0' compression encoding which is rather verbose, for everything
>     else it seems that although it's chatty it's not that much of an overhead.
>   * I've not looked at block migration or spapr yet, (the only other iterative
>     migrate users)
>   * anything with a .get/.put or using the old style migration registration
>     gets bundled up into a blob as an octet-string - this means I don't have
>     to fix everything before it can start working.
> 
> As soon as I get the corresponding BER input visitor working and sweep the
> sawdust out of my code I'll post it; probably a week or two.
> 
> Thoughts welcome,
> 
> Dave


I think this is already useful.
For example this will make input parser
more robust against unexpected input
as octet string blobs are bounded, so if we fail
to parse one we at least will not
interpret it as beginning of the next one.


I think it's a good idea to merge this gradually,
even if this means changing format several times:
as long as we can stay compatible with old machine
types.

> 
> --------------------------------------------------------------------------------
> 
> Qemu {}
> DEFINITIONS IMPLICIT TAGS ::=
> BEGIN
> 
> -- Some basic types used in multiple places --
> QemuString ::= UTF8String (SIZE (1..255))
> 
> -- TODO: 4096 is actually page size whatever that is
> FullPage ::= OCTET STRING (SIZE (4096))
> 
> RAMBlockID ::= SEQUENCE {
>   name   QemuString,
>   len    INTEGER

is len needed here?

> }
> 
> RAMSecEntry ::= [ APPLICATION 8914 ] SEQUENCE {
>   addr         INTEGER,   -- Address or offset or size

hmm so which one?

>   flags        INTEGER,   -- maybe more explicit type?
>   name         QemuString OPTIONAL,
> 
>   body         CHOICE {
>     bl         SEQUENCE OF RAMBlockID,
>     compr      INTEGER (0..255), -- Page filled with this value
>     page       FullPage
>     -- TODO xbzrle --

this one confused me.
I expected RAMBlockID followed by a sequence of pages

>   }
> }
> 
> RAMSecList ::= [ APPLICATION 9810 ] SEQUENCE OF RAMSecEntry
> 
> SubSection ::= [ APPLICATION 10707 ] SEQUENCE {
>   name         QemuString,
>   versionid    INTEGER,
> 
>   contents     SEQUENCE OF VMStateEntries
> }
> 
> SubSecList ::= [ APPLICATION 10700 ] SEQUENCE OF SubSection
> 
> VMStateEntries ::= CHOICE {
>   -- Hmm need to think more --

I'm guessing individual devices will have
tagged entries here?

>   dummy    INTEGER,

why is this here?

>   subsecl  SubSecList,
>   oldblob  OCTET STRING
> }
> 
> VMState ::= SEQUENCE OF VMStateEntries
> 
> -- Restrict to unsigned?
> SectionID ::= INTEGER
> 
> SecFull ::= [ APPLICATION 2003 ] SEQUENCE {
>   name         QemuString,
>   sectionid    SectionID,

what is this id exactly?

>   instanceid   INTEGER,
>   versionid    INTEGER,
> 
>   contents     CHOICE {
>     ramsec     RAMSecList,
>     -- TODO other iterator initial stuff --
>     vmstate    VMState
>   }
> }
> 
> SecMin ::= [ APPLICATION 211 ] SEQUENCE {
>   sectionid   SectionID,
> 
>   contents     CHOICE {
>     ramsec     SEQUENCE OF RAMSecEntry
>     -- TODO other iterator general/end stuff --
>   }
> }
> 
> Sections ::= CHOICE {
>   full    SecFull,
>   min     SecMin
> }
> 
> -- The whole file --
> -- Application tag used to get first 32bits of file
> -- to come out as 7f cd c5 51  - the 51 is Q
> -- the c5 and cd being E,M but with the top bit set
> -- which BER requires
> QemuFile ::= [ APPLICATION 1270481 ] SEQUENCE {
>   version INTEGER (3),
>   top     SEQUENCE OF Sections
> }
> 
> END
> 
> --------------------------------------------------------------------------------
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] WIP: Migration format: ASN.1/BER schema

[Dr. David Alan Gilbert]

* Michael S. Tsirkin (mst@redhat.com) wrote:
> On Fri, Mar 07, 2014 at 04:55:03PM +0000, Dr. David Alan Gilbert wrote:

<snip>

> I think this is already useful.
> For example this will make input parser
> more robust against unexpected input
> as octet string blobs are bounded, so if we fail
> to parse one we at least will not
> interpret it as beginning of the next one.

I've also managed (with a slightly improved schema
at the end) to validate my output sequence against the schema
using asn1c (once I'd persuaded it to build stuff and turn
it's debug on).

> I think it's a good idea to merge this gradually,
> even if this means changing format several times:
> as long as we can stay compatible with old machine
> types.

Yes, that would make the merge easier.

> > --------------------------------------------------------------------------------
> > 
> > Qemu {}
> > DEFINITIONS IMPLICIT TAGS ::=
> > BEGIN
> > 
> > -- Some basic types used in multiple places --
> > QemuString ::= UTF8String (SIZE (1..255))
> > 
> > -- TODO: 4096 is actually page size whatever that is
> > FullPage ::= OCTET STRING (SIZE (4096))
> > 
> > RAMBlockID ::= SEQUENCE {
> >   name   QemuString,
> >   len    INTEGER
> 
> is len needed here?
> 
> > }
> > 
> > RAMSecEntry ::= [ APPLICATION 8914 ] SEQUENCE {
> >   addr         INTEGER,   -- Address or offset or size
> 
> hmm so which one?
> 
> >   flags        INTEGER,   -- maybe more explicit type?
> >   name         QemuString OPTIONAL,
> > 
> >   body         CHOICE {
> >     bl         SEQUENCE OF RAMBlockID,
> >     compr      INTEGER (0..255), -- Page filled with this value
> >     page       FullPage
> >     -- TODO xbzrle --
> 
> this one confused me.
> I expected RAMBlockID followed by a sequence of pages

I've kept with the structure of the way the migrate works at the moment;
it transmits the 'shape' of RAM as a list of name/length pairs, and only
once it's validated that it has all the RAM blocks and their lengths match
does it start shifting pages across.

This a RAM section is either the initialisation data (the sequence of RAMBlockID's)
or a stream of pages.
I don't think it's a bad idea to have that RAM list at the beginning,
it catches a lot of silly mismatches - if you've ever hit a complaint
about a mismatched BIOS length or a missing ROM, it's probably come
from the check of this incoming list.

The existing binary format is a stream of things separated by a
64bit int that is an address/offset/or size depending on some bits OR'd
into the bottom of the address; I'd currently split the flags out
of that 64bit int, but hadn't split that further yet.
I'm thinking of splitting this down further, especially so I can
reduce the size of 'all 0' pages.

> > 
> > RAMSecList ::= [ APPLICATION 9810 ] SEQUENCE OF RAMSecEntry
> > 
> > SubSection ::= [ APPLICATION 10707 ] SEQUENCE {
> >   name         QemuString,
> >   versionid    INTEGER,
> > 
> >   contents     SEQUENCE OF VMStateEntries
> > }
> > 
> > SubSecList ::= [ APPLICATION 10700 ] SEQUENCE OF SubSection
> > 
> > VMStateEntries ::= CHOICE {
> >   -- Hmm need to think more --
> 
> I'm guessing individual devices will have
> tagged entries here?

I'm thinking of *allowing* individual devices to have
tagged entries here, but I don't think I'd require it  - since
to require it would mean I'd have to change every existing
VMState to give it a tag.
So I was going to allow VMStateEntries to be a generic set
using universal tags, which we could add specific entries
where we'd given common devices full tags.

> >   dummy    INTEGER,
> 
> why is this here?

It's one of the universal entries; my current full version (that
verifies) is:

VMStateEntries ::= CHOICE {
  -- Hmm need to think more --
  array       SEQUENCE OF VMStateEntries,
  bool        BOOLEAN,
  int         INTEGER,
  oldblob     OCTET STRING,
  subsecl     SubSecList
}

> 
> >   subsecl  SubSecList,
> >   oldblob  OCTET STRING
> > }
> > 
> > VMState ::= SEQUENCE OF VMStateEntries
> > 
> > -- Restrict to unsigned?
> > SectionID ::= INTEGER
> > 
> > SecFull ::= [ APPLICATION 2003 ] SEQUENCE {
> >   name         QemuString,
> >   sectionid    SectionID,
> 
> what is this id exactly?

It's again a hold over from the old format; Sections
come in two varieties 'Full' and 'Min'; 'Full'
sections are typically used for devices that just get
sent once, and thus have a name - the ID doesn't
really do much for them; however devices that send
data iteratively send a Full followed by a series of
Min's - each Min just has the SectionID matching the Full,
so it saves it having to send the name each time,
and it also means that it just has an index rather
than having to string compare the name for every
section.

Note that iterative devices can be interleaved so
you might have some RAM, followed by some disk blocks
and then some more RAM etc, which is why you can't
just have a sequence of pages.

Dave
P.S. Here's the current verison of my schema which
validates against my ber data.

Qemu {}
DEFINITIONS IMPLICIT TAGS ::=
BEGIN

-- Some basic types used in multiple places --
QemuString ::= UTF8String (SIZE (1..255))

-- TODO: 4096 is actually page size whatever that is
FullPage ::= OCTET STRING (SIZE (4096))

RAMBlockID ::= SEQUENCE {
  name   QemuString,
  len    INTEGER
}

RAMSecEntry ::= [ APPLICATION 8914 ] SEQUENCE {
  addr         INTEGER,   -- Address or offset or size
  flags        INTEGER,   -- maybe more explicit type?
  name         QemuString OPTIONAL,

  body         CHOICE {
    bl         SEQUENCE OF RAMBlockID,
    compr      INTEGER (0..255), -- Page filled with this value
    page       FullPage
    -- TODO xbzrle --
  }
}

RAMSecList ::= [ APPLICATION 9810 ] SEQUENCE OF RAMSecEntry

SubSection ::= [ APPLICATION 10707 ] SEQUENCE {
  name         QemuString,
  versionid    INTEGER,

  contents     SEQUENCE OF VMStateEntries
}

SubSecList ::= [ APPLICATION 10700 ] SEQUENCE OF SubSection

VMStateEntries ::= CHOICE {
  -- Hmm need to think more --
  array       SEQUENCE OF VMStateEntries,
  bool        BOOLEAN,
  int         INTEGER,
  oldblob     OCTET STRING,
  subsecl     SubSecList
}

VMState ::= SEQUENCE OF VMStateEntries

-- Restrict to unsigned?
SectionID ::= INTEGER

SecFull ::= [ APPLICATION 2003 ] SEQUENCE {
  name         QemuString,
  sectionid    SectionID,
  instanceid   INTEGER,
  versionid    INTEGER,

  contents     CHOICE {
    ramsec     RAMSecList,
    -- TODO other iterator initial stuff --
    vmstate    VMState,
    oldblob  OCTET STRING
  }
}

SecMin ::= [ APPLICATION 211 ] SEQUENCE {
  sectionid   SectionID,

  contents     CHOICE {
    ramsec     RAMSecList
    -- TODO other iterator general/end stuff --
  }
}

Sections ::= CHOICE {
  full    SecFull,
  min     SecMin
}

-- The whole file --
-- Application tag used to get first 32bits of file
-- to come out as 7f cd c5 51  - the 51 is Q
-- the c5 and cd being E,M but with the top bit set
-- which BER requires
QemuFile ::= [ APPLICATION 1270481 ] SEQUENCE {
  version INTEGER (3),
  top     SEQUENCE OF Sections
}

END
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] WIP: Migration format: ASN.1/BER schema

[Michael S. Tsirkin]

On Wed, Mar 12, 2014 at 01:17:15PM +0000, Dr. David Alan Gilbert wrote:
> * Michael S. Tsirkin (mst@redhat.com) wrote:
> > On Fri, Mar 07, 2014 at 04:55:03PM +0000, Dr. David Alan Gilbert wrote:
> 
> <snip>
> 
> > I think this is already useful.
> > For example this will make input parser
> > more robust against unexpected input
> > as octet string blobs are bounded, so if we fail
> > to parse one we at least will not
> > interpret it as beginning of the next one.
> 
> I've also managed (with a slightly improved schema
> at the end) to validate my output sequence against the schema
> using asn1c (once I'd persuaded it to build stuff and turn
> it's debug on).
> 
> > I think it's a good idea to merge this gradually,
> > even if this means changing format several times:
> > as long as we can stay compatible with old machine
> > types.
> 
> Yes, that would make the merge easier.
> 
> > > --------------------------------------------------------------------------------
> > > 
> > > Qemu {}
> > > DEFINITIONS IMPLICIT TAGS ::=
> > > BEGIN
> > > 
> > > -- Some basic types used in multiple places --
> > > QemuString ::= UTF8String (SIZE (1..255))
> > > 
> > > -- TODO: 4096 is actually page size whatever that is
> > > FullPage ::= OCTET STRING (SIZE (4096))
> > > 
> > > RAMBlockID ::= SEQUENCE {
> > >   name   QemuString,
> > >   len    INTEGER
> > 
> > is len needed here?
> > 
> > > }
> > > 
> > > RAMSecEntry ::= [ APPLICATION 8914 ] SEQUENCE {
> > >   addr         INTEGER,   -- Address or offset or size
> > 
> > hmm so which one?
> > 
> > >   flags        INTEGER,   -- maybe more explicit type?
> > >   name         QemuString OPTIONAL,
> > > 
> > >   body         CHOICE {
> > >     bl         SEQUENCE OF RAMBlockID,
> > >     compr      INTEGER (0..255), -- Page filled with this value
> > >     page       FullPage
> > >     -- TODO xbzrle --
> > 
> > this one confused me.
> > I expected RAMBlockID followed by a sequence of pages
> 
> I've kept with the structure of the way the migrate works at the moment;
> it transmits the 'shape' of RAM as a list of name/length pairs, and only
> once it's validated that it has all the RAM blocks and their lengths match
> does it start shifting pages across.
> This a RAM section is either the initialisation data (the sequence of RAMBlockID's)
> or a stream of pages.
> I don't think it's a bad idea to have that RAM list at the beginning,

okay so why isn't the schema saying this?

RAMSecEntry ::= [ APPLICATION 8914 ] SEQUENCE {
  addr         INTEGER,   -- Address or offset or size
  flags        INTEGER,   -- maybe more explicit type?
  name         QemuString OPTIONAL,

  body         CHOICE {
    bl         SEQUENCE OF RAMBlockID,
    compr      INTEGER (0..255), -- Page filled with this value
    page       FullPage
    -- TODO xbzrle --
  }
}


What you say seems to match what I expected:

    SEQUENCE OF RAMBlockID
  body         CHOICE {
    compr      INTEGER (0..255), -- Page filled with this value
    page       FullPage
    -- TODO xbzrle --
  }


> it catches a lot of silly mismatches - if you've ever hit a complaint
> about a mismatched BIOS length or a missing ROM, it's probably come
> from the check of this incoming list.

well it might become a problem with memory hotplug but it's a
separate question.


> 
> The existing binary format is a stream of things separated by a
> 64bit int that is an address/offset/or size depending on some bits OR'd
> into the bottom of the address; I'd currently split the flags out
> of that 64bit int, but hadn't split that further yet.
> I'm thinking of splitting this down further, especially so I can
> reduce the size of 'all 0' pages.

if flag just tell us the type then just make it a tag.

> > > 
> > > RAMSecList ::= [ APPLICATION 9810 ] SEQUENCE OF RAMSecEntry
> > > 
> > > SubSection ::= [ APPLICATION 10707 ] SEQUENCE {
> > >   name         QemuString,
> > >   versionid    INTEGER,
> > > 
> > >   contents     SEQUENCE OF VMStateEntries
> > > }
> > > 
> > > SubSecList ::= [ APPLICATION 10700 ] SEQUENCE OF SubSection
> > > 
> > > VMStateEntries ::= CHOICE {
> > >   -- Hmm need to think more --
> > 
> > I'm guessing individual devices will have
> > tagged entries here?
> 
> I'm thinking of *allowing* individual devices to have
> tagged entries here,

e.g. context-specific ones? might be good for
sub-sections.

> but I don't think I'd require it  - since
> to require it would mean I'd have to change every existing
> VMState to give it a tag.
> So I was going to allow VMStateEntries to be a generic set
> using universal tags, which we could add specific entries
> where we'd given common devices full tags.
> > >   dummy    INTEGER,
> > 
> > why is this here?
> 
> It's one of the universal entries; my current full version (that
> verifies) is:
> 
> VMStateEntries ::= CHOICE {
>   -- Hmm need to think more --
>   array       SEQUENCE OF VMStateEntries,
>   bool        BOOLEAN,
>   int         INTEGER,
>   oldblob     OCTET STRING,
>   subsecl     SubSecList
> }

Ah I see. It does not do much though.
Looks like you really can put ANY here just as well.

If we want to make it possible to specify devices
fully in the schema, we'll need a per-device tag.


> > 
> > >   subsecl  SubSecList,
> > >   oldblob  OCTET STRING
> > > }
> > > 
> > > VMState ::= SEQUENCE OF VMStateEntries
> > > 
> > > -- Restrict to unsigned?
> > > SectionID ::= INTEGER
> > > 
> > > SecFull ::= [ APPLICATION 2003 ] SEQUENCE {
> > >   name         QemuString,
> > >   sectionid    SectionID,
> > 
> > what is this id exactly?
> 
> It's again a hold over from the old format; Sections
> come in two varieties 'Full' and 'Min'; 'Full'
> sections are typically used for devices that just get
> sent once, and thus have a name - the ID doesn't
> really do much for them; however devices that send
> data iteratively send a Full followed by a series of
> Min's - each Min just has the SectionID matching the Full,
> so it saves it having to send the name each time,
> and it also means that it just has an index rather
> than having to string compare the name for every
> section.
> 
> Note that iterative devices can be interleaved so
> you might have some RAM, followed by some disk blocks
> and then some more RAM etc, which is why you can't
> just have a sequence of pages.

Hmm looks like a premature optimization?
Also, shouldn't name and ID be optional then?

> Dave
> P.S. Here's the current verison of my schema which
> validates against my ber data.
> 
> Qemu {}
> DEFINITIONS IMPLICIT TAGS ::=
> BEGIN
> 
> -- Some basic types used in multiple places --
> QemuString ::= UTF8String (SIZE (1..255))
> 
> -- TODO: 4096 is actually page size whatever that is
> FullPage ::= OCTET STRING (SIZE (4096))
> 
> RAMBlockID ::= SEQUENCE {
>   name   QemuString,
>   len    INTEGER
> }
> 
> RAMSecEntry ::= [ APPLICATION 8914 ] SEQUENCE {
>   addr         INTEGER,   -- Address or offset or size
>   flags        INTEGER,   -- maybe more explicit type?
>   name         QemuString OPTIONAL,
> 
>   body         CHOICE {
>     bl         SEQUENCE OF RAMBlockID,
>     compr      INTEGER (0..255), -- Page filled with this value
>     page       FullPage
>     -- TODO xbzrle --
>   }
> }
> 
> RAMSecList ::= [ APPLICATION 9810 ] SEQUENCE OF RAMSecEntry
> 
> SubSection ::= [ APPLICATION 10707 ] SEQUENCE {
>   name         QemuString,
>   versionid    INTEGER,
> 
>   contents     SEQUENCE OF VMStateEntries
> }
> 
> SubSecList ::= [ APPLICATION 10700 ] SEQUENCE OF SubSection
> 
> VMStateEntries ::= CHOICE {
>   -- Hmm need to think more --
>   array       SEQUENCE OF VMStateEntries,
>   bool        BOOLEAN,
>   int         INTEGER,
>   oldblob     OCTET STRING,
>   subsecl     SubSecList
> }
> 
> VMState ::= SEQUENCE OF VMStateEntries
> 
> -- Restrict to unsigned?
> SectionID ::= INTEGER
> 
> SecFull ::= [ APPLICATION 2003 ] SEQUENCE {
>   name         QemuString,
>   sectionid    SectionID,
>   instanceid   INTEGER,
>   versionid    INTEGER,
> 
>   contents     CHOICE {
>     ramsec     RAMSecList,
>     -- TODO other iterator initial stuff --
>     vmstate    VMState,
>     oldblob  OCTET STRING
>   }
> }
> 
> SecMin ::= [ APPLICATION 211 ] SEQUENCE {
>   sectionid   SectionID,
> 
>   contents     CHOICE {
>     ramsec     RAMSecList
>     -- TODO other iterator general/end stuff --
>   }
> }
> 
> Sections ::= CHOICE {
>   full    SecFull,
>   min     SecMin
> }
> 
> -- The whole file --
> -- Application tag used to get first 32bits of file
> -- to come out as 7f cd c5 51  - the 51 is Q
> -- the c5 and cd being E,M but with the top bit set
> -- which BER requires
> QemuFile ::= [ APPLICATION 1270481 ] SEQUENCE {
>   version INTEGER (3),
>   top     SEQUENCE OF Sections
> }
> 
> END
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK