Dynamic Loading of Domain Memory Image

Dynamic Loading of Domain Memory Image

[Joe Laws]

I am working with Carnegie Mellon University and Intel on their Internet
Suspend and Resume project to decrease the delay of restarting virtual
machines transfered over the Internet.  After reading the Xen research
papers, and looking at the xc_linux_restore() and xc_linux_save()
methods, it appears that Xen needs to load in the entire memory image
file before it can begin executing the domain.  What I will be working
on is the elimination of the need to have the entire memory image
present and accounted for before the domain can execute.

To accomplish this I am going to first break the memory image into a
series of smaller files, one file for the PTE,PDE, and other important
information, and many individual files for the actual pages.  The main
goal will be to have Xen tell the domain that its page is loaded even if
it is not yet present.  In the case that it has not actual been read in,
Xen will trap to the hypervisor and load the page from the appropriate
file.  This allows the memory pages to arrive after the domain is
started.  This technique should allow the domain to run, albeit with a
slower response time, while paging in missing chunks of memory.

Although this project is geared toward decreasing the resume time when
the memory image is sent over the internet, it should also decrease
resume time when loading from the local disk.  There is also the
possibility that it will decrease migration resume time since all dirty
pages can be marked as "not present/trap to hypervisor" at which point
they can be dynamically loaded in a similar fashion.

I welcome any comments, questions, tips, and/or suggestions from other
developers working on Xen. 

-Joe

Re: Dynamic Loading of Domain Memory Image

[Keir Fraser]

On 27/3/07 17:09, "Joe Laws" <jlaws@andrew.cmu.edu> wrote:

> Although this project is geared toward decreasing the resume time when
> the memory image is sent over the internet, it should also decrease
> resume time when loading from the local disk.  There is also the
> possibility that it will decrease migration resume time since all dirty
> pages can be marked as "not present/trap to hypervisor" at which point
> they can be dynamically loaded in a similar fashion.

The problem with pull-based resume is that unless you have pushed the
working set ahead of time (or are in the process of pushing those pages
first) then performance will suck pretty bad as you'll spend all your time
demand-fetching pages with a wide-area RTT latency per page. Is this better
than running the machine in its original location, pushing the pages, and
connecting to the machine remotely until the memory image is fully built up
at the destination? Perhaps you need to build it to find out...

 -- Keir