RE: Pre-virtualization, was Re: linux/arch/xen/i386 or linux/arch/i386/xen

RE: Pre-virtualization, was Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Magenheimer, Dan (HP Labs Fort Collins)]

Excellent!  How is performance relative to the manually
paravirtualized xenlinux?

> -----Original Message-----
> From: Joshua LeVasseur [mailto:jtl@ira.uka.de] 
> Sent: Friday, May 20, 2005 10:38 AM
> To: Magenheimer, Dan (HP Labs Fort Collins)
> Cc: Vincent Hanquez; Chris Wright; 
> xen-devel@lists.xensource.com; Mark Williamson
> Subject: Pre-virtualization, was Re: linux/arch/xen/i386 or 
> linux/arch/i386/xen
> 
> 
> On May 18, 2005, at 17:09, Magenheimer, Dan (HP Labs Fort Collins)  
> wrote:
> > There have been various discussions on this list about
> > "transparent paravirtualization", i.e. the ability for
> > a paravirtualized kernel to run both as a guest of Xen
> > and on bare metal.  This is definitely an objective of
> > Xen/ia64.  Nobody has tried it for Xen/x86, but if it
> > can be done, I'm sure commercial companies and distros
> > would be eager to utilize it (one less set of bits to
> > support).
> 
> 
> Thanks for the lead-in Dan.  As mentioned before on this list, we  
> have an automated, pre-virtualization solution that permits a single  
> binary to execute on bare x86 hardware and on various hypervisors,  
> with good performance.  See the original message:
> http://lists.xensource.com/archives/html/xen-devel/2005-04/msg
00163.html
> 
> We have now released our source code.  For our project web page,  
> source code (BSD license), and a script to build everything, see:
> http://l4ka.org/projects/virtualization/afterburn/
> We tried to minimize the overhead for getting started, but we can't  
> automate the parts that are dependent on the final hardware, 
> and thus  
> some tenacious debug skills may be necessary.  Also see the user's  
> manual.
> 
> Note that our project does use some concepts of transparent para- 
> virtualization, primarily to deal with higher-level OS concepts.   
> Capturing higher-level OS concepts is particularly useful when  
> mapping guest OS concepts to hypervisor concepts, as is common on  
> more traditional kernels, such as executing at user-level on Linux,  
> Windows NT, and our L4 microkernel.  Transparent 
> virtualization isn't  
> really used on our internal Xen infrastructure (although in our  
> public CVS, it is used a little).
> 
> 
> > In many ways, a "xen" subdirectory is much more like
> > a "pci" or "math-emu" subdirectory, than a subarch.
> > For example, mach-es7000 and xen may need to co-exist
> > in the same kernel.
> >
> > So, mach-xen may be a poor choice.  A subtle distinction
> > perhaps but when dealing with Linux kernel developers,
> > purity of thinking may avoid future patch submission
> > arguments.
> 
> With pre-virtualization, the modifications to the guest OS are very  
> minor.  The whole point is to automate the para-virtualization.  So  
> for example, a single binary can execute on the Xen 
> hypervisor, or as  
> a user-level Linux application, without using any of the user-mode  
> Linux support currently in Linux, and without requiring the proposed  
> additions to Linux for Xen.
> 
> -Josh
> 
> 
> 
> 

Re: Pre-virtualization, was Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Joshua LeVasseur]

So far, performance seems comparable.

-Josh


On May 20, 2005, at 18:47, Magenheimer, Dan (HP Labs Fort Collins)  
wrote:

> Excellent!  How is performance relative to the manually
> paravirtualized xenlinux?
>
>
>> -----Original Message-----
>> From: Joshua LeVasseur [mailto:jtl@ira.uka.de]
>> Sent: Friday, May 20, 2005 10:38 AM
>> To: Magenheimer, Dan (HP Labs Fort Collins)
>> Cc: Vincent Hanquez; Chris Wright;
>> xen-devel@lists.xensource.com; Mark Williamson
>> Subject: Pre-virtualization, was Re: linux/arch/xen/i386 or
>> linux/arch/i386/xen
>>
>>
>> On May 18, 2005, at 17:09, Magenheimer, Dan (HP Labs Fort Collins)
>> wrote:
>>
>>> There have been various discussions on this list about
>>> "transparent paravirtualization", i.e. the ability for
>>> a paravirtualized kernel to run both as a guest of Xen
>>> and on bare metal.  This is definitely an objective of
>>> Xen/ia64.  Nobody has tried it for Xen/x86, but if it
>>> can be done, I'm sure commercial companies and distros
>>> would be eager to utilize it (one less set of bits to
>>> support).
>>>
>>
>>
>> Thanks for the lead-in Dan.  As mentioned before on this list, we
>> have an automated, pre-virtualization solution that permits a single
>> binary to execute on bare x86 hardware and on various hypervisors,
>> with good performance.  See the original message:
>> http://lists.xensource.com/archives/html/xen-devel/2005-04/msg
>>
> 00163.html
>
>>
>> We have now released our source code.  For our project web page,
>> source code (BSD license), and a script to build everything, see:
>> http://l4ka.org/projects/virtualization/afterburn/
>> We tried to minimize the overhead for getting started, but we can't
>> automate the parts that are dependent on the final hardware,
>> and thus
>> some tenacious debug skills may be necessary.  Also see the user's
>> manual.
>>
>> Note that our project does use some concepts of transparent para-
>> virtualization, primarily to deal with higher-level OS concepts.
>> Capturing higher-level OS concepts is particularly useful when
>> mapping guest OS concepts to hypervisor concepts, as is common on
>> more traditional kernels, such as executing at user-level on Linux,
>> Windows NT, and our L4 microkernel.  Transparent
>> virtualization isn't
>> really used on our internal Xen infrastructure (although in our
>> public CVS, it is used a little).
>>
>>
>>
>>> In many ways, a "xen" subdirectory is much more like
>>> a "pci" or "math-emu" subdirectory, than a subarch.
>>> For example, mach-es7000 and xen may need to co-exist
>>> in the same kernel.
>>>
>>> So, mach-xen may be a poor choice.  A subtle distinction
>>> perhaps but when dealing with Linux kernel developers,
>>> purity of thinking may avoid future patch submission
>>> arguments.
>>>
>>
>> With pre-virtualization, the modifications to the guest OS are very
>> minor.  The whole point is to automate the para-virtualization.  So
>> for example, a single binary can execute on the Xen
>> hypervisor, or as
>> a user-level Linux application, without using any of the user-mode
>> Linux support currently in Linux, and without requiring the proposed
>> additions to Linux for Xen.
>>
>> -Josh
>>
>>
>>
>>
>

Re: Re: Pre-virtualization, was Re: linux/arch/xen/i386 or linux/arch/i386/xen

[aq]

On 5/21/05, Joshua LeVasseur <jtl@ira.uka.de> wrote:
> 
> So far, performance seems comparable.
> 

Joshua, as I understand, this project would be a competitor of Xen?

is there any published paper about this project?

thank you,
aq

Re: Re: Pre-virtualization, was Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Ronald G. Minnich]

On Sat, 21 May 2005, aq wrote:
> Joshua, as I understand, this project would be a competitor of Xen?
http://l4hq.org/cvsweb/cvsweb/~checkout~/afterburner/afterburn-wedge/doc/userman.pdf?content-type=application/pdf

for the manual.

You still have to modify the kernel, it seems. 

Are there fewer mods? What is the advantage of this over Xen? I think I'm 
missing a key point here. 

ron

Re: Re: Pre-virtualization, was Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Anthony Liguori]

Ronald G. Minnich wrote:

>On Sat, 21 May 2005, aq wrote:
>  
>
>>Joshua, as I understand, this project would be a competitor of Xen?
>>    
>>
>http://l4hq.org/cvsweb/cvsweb/~checkout~/afterburner/afterburn-wedge/doc/userman.pdf?content-type=application/pdf
>
>for the manual.
>
>You still have to modify the kernel, it seems. 
>  
>
Strictly speaking you should be able to achieve "full virtualization" 
with toolchain modifications.  Think of the comparision to VMX.  VMX 
provides exits to call into the hypervisor when certain instructions are 
executed that are not virtualization friendly.  With pre-virtualization, 
you have the compiler pad virtualization unfriendly instructions with 
nop's and then at load time, replace all of those unfriendly sequences 
with calls to the hypervisor.  This way, the same kernel can run on bare 
metal or within a hypervisor.

Of course, just as is likely with VMX, you're probably gonna want to do 
some hand-tuning of Linux for performance reasons.  It seems like 
afterburner incorporates these types of optimizations.

I'd really like to see a "pure" form of pre-virtualization that required 
no modifications at all to the underlying source tree.  Besides being 
interesting from an academic standpoint I think it would be highly 
useful for support legacy Open Source operating systems.

I'm very excited about this technology.  I imagine that you can get all 
the benefits of binary-rewriting with less complexity and better 
performance (with the only limitation being that you have the source 
code for the OS which is fine by me).

Regards,
Anthony Liguori

>Are there fewer mods? What is the advantage of this over Xen? I think I'm 
>missing a key point here. 
>
>ron
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel
>
>  
>

Re: Re: Pre-virtualization, was Re: linux/arch/xen/i386 or linux/arch/i386/xen

[aq]

On 5/21/05, Ronald G. Minnich <rminnich@lanl.gov> wrote:
> 
> 
> On Sat, 21 May 2005, aq wrote:
> > Joshua, as I understand, this project would be a competitor of Xen?
> http://l4hq.org/cvsweb/cvsweb/~checkout~/afterburner/afterburn-wedge/doc/userman.pdf?content-type=application/pdf
> 
> for the manual.

yes, but that is not a (scientific) paper, which would let us
understand more about this technology.

one good point about this project is that it modifies the guest kernel
very little (according to information found at the homepage, around 80
lines of code)

would like to try this new-kid-on-the-block soon :-)

regards,
aq

Re: Re: Pre-virtualization, was Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Ronald G. Minnich]

On Fri, 20 May 2005, Anthony Liguori wrote:

> I'd really like to see a "pure" form of pre-virtualization that required
> no modifications at all to the underlying source tree.  Besides being
> interesting from an academic standpoint I think it would be highly
> useful for support legacy Open Source operating systems.

from my point of view, linux and bsd are the same OS; plan 9 is the hard 
one. I need to modify plan 9 for all of these. At the same time, if I 
understood how to hack the compiler in the manner you describe, it would 
probably not take long at all. 

> I'm very excited about this technology.  I imagine that you can get all
> the benefits of binary-rewriting with less complexity and better
> performance (with the only limitation being that you have the source
> code for the OS which is fine by me).

likewise. It's a big effort to track Xen 3.0 nowadays, with the current
rate of change, and the effort is highly magnified by the fact that Plan 9
does not use gcc.

ron

Re: Re: Pre-virtualization, was Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Ronald G. Minnich]

On Sat, 21 May 2005, aq wrote:

> yes, but that is not a (scientific) paper, which would let us
> understand more about this technology.

agree, but it told me enough to know I can't just boot Plan 9 under it :-)

ron