Proposal for a fixed ram migration stream format for file-based migrations

Proposal for a fixed ram migration stream format for file-based migrations

[Nikolay Borisov]

Hello,

Based on several discussions I've had in the past 2 days and time spent
looking at the migration stream code I came up with the following
proposal for changes to the stream format. Let me recap what we have
right now:


<migration header><ramsize><ramblock header #1><ramblock header #2>...<page#1 offset>(<block name>)<actual page#1 data><device state>

Where <block name> is put only when the current page we are writing
to belongs to a different block than the last written page. My proposal is to
change this format to:

<migration header><ramsize><ramblock #1 header><all dirtied pages belonging to ramblock #1><ramblock #2 header>...<device state>

Each page belonging to a ramblock would be at a fixed offset in the file. Which is going to be
<ramblock hdr offset> + page_offset. The size of the region for a particular ramblock would be
ramblock::used_length but not the whole range is going to be written, only those pages which
have been dirtied. If we assume a 1tb ramblock then we'd make the the region 1 tb but it will
be sparsely populated. This means that we can have the following layout for the memory range:


|---p---ppp----pp---p|
0		     1tb

Each 'p' signifies an allocated page and '-' is a page which hasn't been dirtied ergo its
index in the stream is not touched. This of course would result in having a lot of holes, so when
the incoming migration starts parsing the stream it might end up in a situation
where page at offset 0 is read, but then it has to jump some at arbitrary
location for the next page. To avoid resorting to calling into the filesystem and dealing with
fiemap's format I also intend to extend the ramblock header format by adding a dirty bitmap in
which every bit would signify whether a page has been written or not, the bit's position would be
used to index into the allocated space for pages. Simple maths shows that for 4k pages, 1tb can be
indexed with just 32kbytes of memory. The way the indexing would work is

page = ramblock->offset (this is the offset in the file, not the in-memory ::ofset field) +
bit's position * target_page_size

That way we can still handle sparsely dirtied memory and have direct mapping in the file, this
dirty bitmap would be populated during the course of ram migration and once the final page is written,
those in-memory bitmaps would be written into their respective positions in the file, needless to say
this would be supported only on seekable channels (for now only file-based channels).

Another important thing worth mentioning is I don't intend on incrementing the
format version but rather introducing a new migration capability.

Any thoughts/comments/suggestions, would be highly appreciated.