Re: Re: how to enable shadow page table? Do I have to run HVM guest systems for shadow paging mode?

Re: Re: how to enable shadow page table? Do I have to run HVM guest systems for shadow paging mode?

[Jeremy Fitzhardinge]

Long Wang wrote:
> I looked into the live migration code in Xen. I need more than the mechanism
> provided in live migration, as described below:
>
> 1. scan the states of all the memory pages of a domain, set dirty pages as
> read-only, and clear all dirty bits for all the memory pages (so read-only
> and dirty are two different bits). This procedure is done periodically.
>
> 2. When a readonly page is to be updated by the domain, the update is
> trapped into the hypervisor, when I copy the page content to another place,
> and set the page as writable (this page is not a page table page), makes the
> update.
>
> I find that the mechanism used in live migration cannot handle these. How
> can I enable shadow paging in PV? If not, do I have to find a machine with
> vmx-featured processor and run HVM for this machine?
>   

Well, that's an option.  But if you're going to be hacking Xen anyway, 
then it sounds like log-dirty is close to what you want.  But Tim is 
probably the person to ask about that.

    J

RE: Re: how to enable shadow page table? Do I have to run HVM guest systems for shadow paging mode?

[Dan Magenheimer]

For the record, Transcendent Memory is not a copy-on-write
mechanism, doesn't deal with shadow tables or PTE read/write
bits, and is really not exotic at all.  It's a simple (though
paravirtualized) mechanism for optimizing the usage of
unutilized or underutilized physical memory while ensuring
(most of) that memory is synchronously reclaimable for
unexpectedly urgent needs.  :-)

Dan

> -----Original Message-----
> From: Tim Deegan [mailto:Tim.Deegan@citrix.com]
> Sent: Monday, March 16, 2009 11:41 AM
> To: Jeremy Fitzhardinge
> Cc: Xen-devel; Long Wang
> Subject: [Xen-devel] Re: how to enable shadow page table? Do I have to
> run HVM guest systems for shadow paging mode?
> 
> 
> At 16:30 +0000 on 16 Mar (1237221012), Jeremy Fitzhardinge wrote:
> > Long Wang wrote:
> > > I am working on a research project based on Xen 3.3.1. I 
> want to use the
> > > shadow page table for setting guest system memory pages 
> as read-only and/or
> > > dirty, and perform a copy-on-write mechanism when these 
> memory pages are
> > > updated.
> 
> Several people are already working on copy-on-write memory in Xen, or
> variations on that theme (Mike Sun, Patrick Colp, John Byrne, and
> others) as well as more exotic things like Difference Engine and
> Transcendent Memory.  It would probably be better to talk to those
> people and try to help their efforts than start from scratch.
> 
> Patrick Colp's slides from the most recent Xen Summit give an idea of
> one method of implementing it:
> http://www.xen.org/files/xensummit_oracle09/VMSnapshots.pdf
> 
> Cheers,
> 
> Tim.
> 
> -- 
> Tim Deegan <Tim.Deegan@citrix.com>
> Principal Software Engineer, Citrix Systems (R&D) Ltd.
> [Company #02300071, SL9 0DZ, UK.]
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

Re: Re: how to enable shadow page table? Do I have to run HVM guest systems for shadow paging mode?

[Tim Deegan]

At 17:57 +0000 on 16 Mar (1237226223), Dan Magenheimer wrote:
> For the record, Transcendent Memory is not a copy-on-write
> mechanism, doesn't deal with shadow tables or PTE read/write
> bits, and is really not exotic at all.

Indeed.  I was just including it in the current bundle of "fun things
to do with memory management". :)  And since it helps with systems under
memory pressure it might have solved whatever the underlying problem was.
Almost nobody wants to write a copy-on-write mechanism just for the
beauty of the thing.

Cheers,

Tim.

>  It's a simple (though
> paravirtualized) mechanism for optimizing the usage of
> unutilized or underutilized physical memory while ensuring
> (most of) that memory is synchronously reclaimable for
> unexpectedly urgent needs.  :-)
> 
> Dan
> 
> > -----Original Message-----
> > From: Tim Deegan [mailto:Tim.Deegan@citrix.com]
> > Sent: Monday, March 16, 2009 11:41 AM
> > To: Jeremy Fitzhardinge
> > Cc: Xen-devel; Long Wang
> > Subject: [Xen-devel] Re: how to enable shadow page table? Do I have to
> > run HVM guest systems for shadow paging mode?
> > 
> > 
> > At 16:30 +0000 on 16 Mar (1237221012), Jeremy Fitzhardinge wrote:
> > > Long Wang wrote:
> > > > I am working on a research project based on Xen 3.3.1. I 
> > want to use the
> > > > shadow page table for setting guest system memory pages 
> > as read-only and/or
> > > > dirty, and perform a copy-on-write mechanism when these 
> > memory pages are
> > > > updated.
> > 
> > Several people are already working on copy-on-write memory in Xen, or
> > variations on that theme (Mike Sun, Patrick Colp, John Byrne, and
> > others) as well as more exotic things like Difference Engine and
> > Transcendent Memory.  It would probably be better to talk to those
> > people and try to help their efforts than start from scratch.
> > 
> > Patrick Colp's slides from the most recent Xen Summit give an idea of
> > one method of implementing it:
> > http://www.xen.org/files/xensummit_oracle09/VMSnapshots.pdf
> > 
> > Cheers,
> > 
> > Tim.
> > 
> > -- 
> > Tim Deegan <Tim.Deegan@citrix.com>
> > Principal Software Engineer, Citrix Systems (R&D) Ltd.
> > [Company #02300071, SL9 0DZ, UK.]
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> >

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Citrix Systems (R&D) Ltd.
[Company #02300071, SL9 0DZ, UK.]

Re: Re: how to enable shadow page table? Do I have to run HVM guest systems for shadow paging mode?

[Tim Deegan]

At 18:40 +0000 on 16 Mar (1237228854), longwang@crhc.illinois.edu wrote:
> 2) When I call paging_mode_log_dirty(pdom) in hypervisor, the hypervisor
> crashes. I thought paging_mode_log_dirty() enables the shadow paging as
> well as dirty logging after I had read live migration code. But I was
> wrong. paging_mode_log_dirty() crashes because shadow paging is not
> enabled yet.
> 
> Now I need to enable shadow paging mode in my testbed (PV). 

You wrote all the code before figuring out how to turn on shadow pagetables?

> I tried the
> auto_translated_physmap in the domU config file. But "xm create" failed
> with :
> Error: (1, 'Internal error', 'xc_dom_boot_domU_map: failed to mmap domU
> pages 0x100+0x2da [mmap, errno=14 (Bad address)]\n')

As Jeremy says, auto_translated_physmap doesn't work on recent Xens or
recent Linuxes.  

> So how can I enable shadow paging? Do I have to run HVM for this to work?

If you want the shadow code to hide a layer of address translations
(i.e. have gfn != mfn) then yes, you need HVM.  

Otherwise, you can just copy what the live-migration tools do to turn on
shadow pagetables for PV guests.  It's just one hypercall.  For example,
in python:

 #!/usr/bin/env python
 import sys
 import xen.lowlevel.xc
 xen.lowlevel.xc.xc().shadow_control(dom=int(sys.argv[1]), op=1)

To turn on log-dirty mode, use op=2.  To turn off shadow mode, op=0.

Cheers,

Tim.

> thanks,
> long
> 
> 
> > At 16:30 +0000 on 16 Mar (1237221012), Jeremy Fitzhardinge wrote:
> >> Long Wang wrote:
> >> > I am working on a research project based on Xen 3.3.1. I want to use
> >> the
> >> > shadow page table for setting guest system memory pages as read-only
> >> and/or
> >> > dirty, and perform a copy-on-write mechanism when these memory pages
> >> are
> >> > updated.
> >
> > Several people are already working on copy-on-write memory in Xen, or
> > variations on that theme (Mike Sun, Patrick Colp, John Byrne, and
> > others) as well as more exotic things like Difference Engine and
> > Transcendent Memory.  It would probably be better to talk to those
> > people and try to help their efforts than start from scratch.
> >
> > Patrick Colp's slides from the most recent Xen Summit give an idea of
> > one method of implementing it:
> > http://www.xen.org/files/xensummit_oracle09/VMSnapshots.pdf
> >
> > Cheers,
> >
> > Tim.
> >
> > --
> > Tim Deegan <Tim.Deegan@citrix.com>
> > Principal Software Engineer, Citrix Systems (R&D) Ltd.
> > [Company #02300071, SL9 0DZ, UK.]
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> >
> 
> 

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Citrix Systems (R&D) Ltd.
[Company #02300071, SL9 0DZ, UK.]