Re: Re: how to enable shadow page table? Do I have to run HVM guest systems for shadow paging mode?

Re: Re: how to enable shadow page table? Do I have to run HVM guest systems for shadow paging mode?

[Jeremy Fitzhardinge]

Long Wang wrote:
> I looked into the live migration code in Xen. I need more than the mechanism
> provided in live migration, as described below:
>
> 1. scan the states of all the memory pages of a domain, set dirty pages as
> read-only, and clear all dirty bits for all the memory pages (so read-only
> and dirty are two different bits). This procedure is done periodically.
>
> 2. When a readonly page is to be updated by the domain, the update is
> trapped into the hypervisor, when I copy the page content to another place,
> and set the page as writable (this page is not a page table page), makes the
> update.
>
> I find that the mechanism used in live migration cannot handle these. How
> can I enable shadow paging in PV? If not, do I have to find a machine with
> vmx-featured processor and run HVM for this machine?
>   

Well, that's an option.  But if you're going to be hacking Xen anyway, 
then it sounds like log-dirty is close to what you want.  But Tim is 
probably the person to ask about that.

    J

Re: how to enable shadow page table? Do I have to run HVM guest systems for shadow paging mode?

[Jeremy Fitzhardinge]

Long Wang wrote:
> I am working on a research project based on Xen 3.3.1. I want to use the
> shadow page table for setting guest system memory pages as read-only and/or
> dirty, and perform a copy-on-write mechanism when these memory pages are
> updated. I heard that auto_translated_physmap is not supported any more. 
>   

Yes, auto_translated_physmap is not supported by Xen or by the pvops 
guest kernels.

> Then how can I enable shadow page table? Do I have to run HVM guest systems
> to have shadow paging mode? Can I enable shadow paging mode in PV?
>   

Well, Xen supports log-dirty mode for doing live migration of PV guests, 
which looks to be what you're describing above.  You might want to look 
into the existing live migration machinery to see if it meets your needs.

    J

Re: how to enable shadow page table? Do I have to run HVM guest systems for shadow paging mode?

[Tim Deegan]

At 16:30 +0000 on 16 Mar (1237221012), Jeremy Fitzhardinge wrote:
> Long Wang wrote:
> > I am working on a research project based on Xen 3.3.1. I want to use the
> > shadow page table for setting guest system memory pages as read-only and/or
> > dirty, and perform a copy-on-write mechanism when these memory pages are
> > updated.

Several people are already working on copy-on-write memory in Xen, or
variations on that theme (Mike Sun, Patrick Colp, John Byrne, and
others) as well as more exotic things like Difference Engine and
Transcendent Memory.  It would probably be better to talk to those
people and try to help their efforts than start from scratch.

Patrick Colp's slides from the most recent Xen Summit give an idea of
one method of implementing it:
http://www.xen.org/files/xensummit_oracle09/VMSnapshots.pdf

Cheers,

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Citrix Systems (R&D) Ltd.
[Company #02300071, SL9 0DZ, UK.]

RE: Re: how to enable shadow page table? Do I have to run HVM guest systems for shadow paging mode?

[Dan Magenheimer]

For the record, Transcendent Memory is not a copy-on-write
mechanism, doesn't deal with shadow tables or PTE read/write
bits, and is really not exotic at all.  It's a simple (though
paravirtualized) mechanism for optimizing the usage of
unutilized or underutilized physical memory while ensuring
(most of) that memory is synchronously reclaimable for
unexpectedly urgent needs.  :-)

Dan

> -----Original Message-----
> From: Tim Deegan [mailto:Tim.Deegan@citrix.com]
> Sent: Monday, March 16, 2009 11:41 AM
> To: Jeremy Fitzhardinge
> Cc: Xen-devel; Long Wang
> Subject: [Xen-devel] Re: how to enable shadow page table? Do I have to
> run HVM guest systems for shadow paging mode?
> 
> 
> At 16:30 +0000 on 16 Mar (1237221012), Jeremy Fitzhardinge wrote:
> > Long Wang wrote:
> > > I am working on a research project based on Xen 3.3.1. I 
> want to use the
> > > shadow page table for setting guest system memory pages 
> as read-only and/or
> > > dirty, and perform a copy-on-write mechanism when these 
> memory pages are
> > > updated.
> 
> Several people are already working on copy-on-write memory in Xen, or
> variations on that theme (Mike Sun, Patrick Colp, John Byrne, and
> others) as well as more exotic things like Difference Engine and
> Transcendent Memory.  It would probably be better to talk to those
> people and try to help their efforts than start from scratch.
> 
> Patrick Colp's slides from the most recent Xen Summit give an idea of
> one method of implementing it:
> http://www.xen.org/files/xensummit_oracle09/VMSnapshots.pdf
> 
> Cheers,
> 
> Tim.
> 
> -- 
> Tim Deegan <Tim.Deegan@citrix.com>
> Principal Software Engineer, Citrix Systems (R&D) Ltd.
> [Company #02300071, SL9 0DZ, UK.]
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

Re: Re: how to enable shadow page table? Do I have to run HVM guest systems for shadow paging mode?

[Tim Deegan]

At 17:57 +0000 on 16 Mar (1237226223), Dan Magenheimer wrote:
> For the record, Transcendent Memory is not a copy-on-write
> mechanism, doesn't deal with shadow tables or PTE read/write
> bits, and is really not exotic at all.

Indeed.  I was just including it in the current bundle of "fun things
to do with memory management". :)  And since it helps with systems under
memory pressure it might have solved whatever the underlying problem was.
Almost nobody wants to write a copy-on-write mechanism just for the
beauty of the thing.

Cheers,

Tim.

>  It's a simple (though
> paravirtualized) mechanism for optimizing the usage of
> unutilized or underutilized physical memory while ensuring
> (most of) that memory is synchronously reclaimable for
> unexpectedly urgent needs.  :-)
> 
> Dan
> 
> > -----Original Message-----
> > From: Tim Deegan [mailto:Tim.Deegan@citrix.com]
> > Sent: Monday, March 16, 2009 11:41 AM
> > To: Jeremy Fitzhardinge
> > Cc: Xen-devel; Long Wang
> > Subject: [Xen-devel] Re: how to enable shadow page table? Do I have to
> > run HVM guest systems for shadow paging mode?
> > 
> > 
> > At 16:30 +0000 on 16 Mar (1237221012), Jeremy Fitzhardinge wrote:
> > > Long Wang wrote:
> > > > I am working on a research project based on Xen 3.3.1. I 
> > want to use the
> > > > shadow page table for setting guest system memory pages 
> > as read-only and/or
> > > > dirty, and perform a copy-on-write mechanism when these 
> > memory pages are
> > > > updated.
> > 
> > Several people are already working on copy-on-write memory in Xen, or
> > variations on that theme (Mike Sun, Patrick Colp, John Byrne, and
> > others) as well as more exotic things like Difference Engine and
> > Transcendent Memory.  It would probably be better to talk to those
> > people and try to help their efforts than start from scratch.
> > 
> > Patrick Colp's slides from the most recent Xen Summit give an idea of
> > one method of implementing it:
> > http://www.xen.org/files/xensummit_oracle09/VMSnapshots.pdf
> > 
> > Cheers,
> > 
> > Tim.
> > 
> > -- 
> > Tim Deegan <Tim.Deegan@citrix.com>
> > Principal Software Engineer, Citrix Systems (R&D) Ltd.
> > [Company #02300071, SL9 0DZ, UK.]
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> >

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Citrix Systems (R&D) Ltd.
[Company #02300071, SL9 0DZ, UK.]

Re: Re: how to enable shadow page table? Do I have to run HVM guest systems for shadow paging mode?

[Tim Deegan]

At 18:40 +0000 on 16 Mar (1237228854), longwang@crhc.illinois.edu wrote:
> 2) When I call paging_mode_log_dirty(pdom) in hypervisor, the hypervisor
> crashes. I thought paging_mode_log_dirty() enables the shadow paging as
> well as dirty logging after I had read live migration code. But I was
> wrong. paging_mode_log_dirty() crashes because shadow paging is not
> enabled yet.
> 
> Now I need to enable shadow paging mode in my testbed (PV). 

You wrote all the code before figuring out how to turn on shadow pagetables?

> I tried the
> auto_translated_physmap in the domU config file. But "xm create" failed
> with :
> Error: (1, 'Internal error', 'xc_dom_boot_domU_map: failed to mmap domU
> pages 0x100+0x2da [mmap, errno=14 (Bad address)]\n')

As Jeremy says, auto_translated_physmap doesn't work on recent Xens or
recent Linuxes.  

> So how can I enable shadow paging? Do I have to run HVM for this to work?

If you want the shadow code to hide a layer of address translations
(i.e. have gfn != mfn) then yes, you need HVM.  

Otherwise, you can just copy what the live-migration tools do to turn on
shadow pagetables for PV guests.  It's just one hypercall.  For example,
in python:

 #!/usr/bin/env python
 import sys
 import xen.lowlevel.xc
 xen.lowlevel.xc.xc().shadow_control(dom=int(sys.argv[1]), op=1)

To turn on log-dirty mode, use op=2.  To turn off shadow mode, op=0.

Cheers,

Tim.

> thanks,
> long
> 
> 
> > At 16:30 +0000 on 16 Mar (1237221012), Jeremy Fitzhardinge wrote:
> >> Long Wang wrote:
> >> > I am working on a research project based on Xen 3.3.1. I want to use
> >> the
> >> > shadow page table for setting guest system memory pages as read-only
> >> and/or
> >> > dirty, and perform a copy-on-write mechanism when these memory pages
> >> are
> >> > updated.
> >
> > Several people are already working on copy-on-write memory in Xen, or
> > variations on that theme (Mike Sun, Patrick Colp, John Byrne, and
> > others) as well as more exotic things like Difference Engine and
> > Transcendent Memory.  It would probably be better to talk to those
> > people and try to help their efforts than start from scratch.
> >
> > Patrick Colp's slides from the most recent Xen Summit give an idea of
> > one method of implementing it:
> > http://www.xen.org/files/xensummit_oracle09/VMSnapshots.pdf
> >
> > Cheers,
> >
> > Tim.
> >
> > --
> > Tim Deegan <Tim.Deegan@citrix.com>
> > Principal Software Engineer, Citrix Systems (R&D) Ltd.
> > [Company #02300071, SL9 0DZ, UK.]
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> >
> 
> 

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Citrix Systems (R&D) Ltd.
[Company #02300071, SL9 0DZ, UK.]