[RFC] XI Shadow Page Table Mechanism

[RFC] XI Shadow Page Table Mechanism

[Ben Thomas]

[-- Attachment #1: Type: text/plain, Size: 1545 bytes --]

This post contains the design document for what is currently known
as the "XI Shadow Mechanism".  This is a design for shadow
page table code for fully virtualized HVM domains running on a 64-bit
Xen hypervisor.

This work was undertaken to address a number of goals.  These are
enumerated in the document and include:

- ability to run fully virtualized 32, 32PAE and 64 bit guest
   domains concurrently on a 64-bit hypervisor

- support live migration of fully virtualized domains

- provide good performance and robustness

This design has been implemented and is currently being tested.
It has been supporting the variety of memory models as noted above,
and using widely used Windows and Linux distributions (SuSe,
RedHat and others). At a point in the near future, a patch
will be available.

This design center is the x86-64 architecture. It is not our
intent to completely replace all shadow page management, and
we've attempted to limit the scope of change.

A preliminary version of this design concept has undergone
brief review with some members of the Xen community. We hope
that this is of value to the Xen community and welcome your
feedback and comments.

Thanks,
-b

-- 
------------------------------------------------------------------------
Ben Thomas                                         Virtual Iron Software
bthomas@virtualiron.com                            Tower 1, Floor 2
978-849-1214                                       900 Chelmsford Street
                                                    Lowell, MA 01851

[-- Attachment #2: XI_Design.pdf --]
[-- Type: application/pdf, Size: 66919 bytes --]

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [RFC] XI Shadow Page Table Mechanism

[Anthony Liguori]

Where can we look at the code?

Regards,

Anthony Liguori

Ben Thomas wrote:
> This post contains the design document for what is currently known
> as the "XI Shadow Mechanism".  This is a design for shadow
> page table code for fully virtualized HVM domains running on a 64-bit
> Xen hypervisor.
>
> This work was undertaken to address a number of goals.  These are
> enumerated in the document and include:
>
> - ability to run fully virtualized 32, 32PAE and 64 bit guest
>   domains concurrently on a 64-bit hypervisor
>
> - support live migration of fully virtualized domains
>
> - provide good performance and robustness
>
> This design has been implemented and is currently being tested.
> It has been supporting the variety of memory models as noted above,
> and using widely used Windows and Linux distributions (SuSe,
> RedHat and others). At a point in the near future, a patch
> will be available.
>
> This design center is the x86-64 architecture. It is not our
> intent to completely replace all shadow page management, and
> we've attempted to limit the scope of change.
>
> A preliminary version of this design concept has undergone
> brief review with some members of the Xen community. We hope
> that this is of value to the Xen community and welcome your
> feedback and comments.
>
> Thanks,
> -b
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: [RFC] XI Shadow Page Table Mechanism

[Ben Thomas]

Anthony Liguori wrote:
> Where can we look at the code?
> 
> Regards,
> 
> Anthony Liguori

Hi Anthony,

As I mentioned in the original post, a patch is forthcoming.
It's functional and works in the majority of the cases, and
we're testing it a bit more before releasing it.

It's a pretty sensitive area and the testing matrix is large.
The previous post was a request for comments on the design.


Thanks,
-b

-- 
------------------------------------------------------------------------
Ben Thomas                                         Virtual Iron Software
bthomas@virtualiron.com                            Tower 1, Floor 2
978-849-1214                                       900 Chelmsford Street
                                                    Lowell, MA 01851

Re: [RFC] XI Shadow Page Table Mechanism

[Anthony Liguori]

Ben Thomas wrote:
> This post contains the design document for what is currently known
> as the "XI Shadow Mechanism".  This is a design for shadow
> page table code for fully virtualized HVM domains running on a 64-bit
> Xen hypervisor.
>
> This work was undertaken to address a number of goals.  These are
> enumerated in the document and include:
>
> - ability to run fully virtualized 32, 32PAE and 64 bit guest
>   domains concurrently on a 64-bit hypervisor

This isn't supported currently?  Since an HVM must go through 16 bit, 32 
bit, and 64 bit mode to boot up, how can we start more than one guest at 
a time currently if this doesn't already work?

> - support live migration of fully virtualized domains

Sweet.  What about the current shadow page code limited this?

> - provide good performance and robustness
>
> This design has been implemented and is currently being tested.
> It has been supporting the variety of memory models as noted above,
> and using widely used Windows and Linux distributions (SuSe,
> RedHat and others). At a point in the near future, a patch
> will be available.
>
> This design center is the x86-64 architecture. It is not our
> intent to completely replace all shadow page management, and
> we've attempted to limit the scope of change.
>
> A preliminary version of this design concept has undergone
> brief review with some members of the Xen community. We hope
> that this is of value to the Xen community and welcome your
> feedback and comments.

Can't really comment as it's not quotable.  Some questions that 
immediately come to mind are:

 - how do you deal with large pages within the hypervisor?  do you 
coalesce or just hope there is contiguous pages available?
 - what is the performance benefit in saving the shadow pages for each 
domain?  there's clearly a memory trade-off here so understanding the 
performance gain seems important.
 - OOM can be dealt with in the existing code by just invalidating 
existing mappings to free up pages.  what advantages do your approach 
have to this?  (i realize we don't do this today but in theory, we could).

Interesting stuff.  I'm eager to see the code.

Regards,

Anthony Liguori

> Thanks,
> -b
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel