Xen and Linux

Xen and Linux

[Dimitrie O. Paun]

Hi folks,
And congrats for the release of 2.0 -- this is really cool stuff.

While looking through Xen's features, I kept wondering if it's
possible to have a common kernel binary that would work with/without
Xen. In other words, to dynamically determine if it's running
under Xen, and behave appropriately.

And hence my questions:
  -- is this possible to begin with?
  -- has anyone looked into it already?
  -- what would be the implications?
  -- any significant performance problems?

Well, thank you for your time, and keep up the excellent work
on this amazing project!

-- 
Dimi.



-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Xen and Linux

[Ian Pratt]

> Hi folks,
> And congrats for the release of 2.0 -- this is really cool stuff.
> 
> While looking through Xen's features, I kept wondering if it's
> possible to have a common kernel binary that would work with/without
> Xen. In other words, to dynamically determine if it's running
> under Xen, and behave appropriately.

This has been dubbed "transparent virtualization". 

> And hence my questions:
>   -- is this possible to begin with?

yes, we believe so. It might be a bit gory though.

>   -- has anyone looked into it already?

We've thought about it, and even done a bit of reconnaissance.
Nobody is actively working on it, though.

>   -- what would be the implications?

It would be very cool ;-) We might even get distros to ship it on
their install CD...

>   -- any significant performance problems?

It depends on what granularity the switching is done, but I
believe it could be made to work fine. There needn't be much
run-time memory bloat as the unused functions could be thrown
away. 

There's one simple src change in arch Xen that has some quite far
reaching consequences: we change FIXADDR_TOP which effectively
gives us a 64MB whole at the top of the guest's VM space where
Xen lives. This constant gets compiled into a bunch of different
functions (though I believe it doesn't make it into modules
--phew!).

I suspect it would be unpopular to make FIXADDR_TOP a variable,
and it turns out to be a tricky thing to runtime patch. I think
we just have to have arch-xen specific versions of all of the
functions that use it. 

The simplest way to do this would be to have a tool that builds
an x86 and xen kernel then merges the two together. Sounds a bit
gross, but I think it would work quite well.

The alternative would be to make the changes at source level, but
this is actually more likely to marginally slow down normal
operation. 

2.6 has the beginnings of a boot-time switch mechanism to enable
support for different x86 instruction families. It would need to
be made quite a bit more extensive to cope with arch Xen.

Probably the cleanest solution of all would be to make all the
other architectures adopt Xen's nice clean interfaces and then
have stub routines for talking to the grotty realty of real
hardware. Might be a hard one to sell to Linus et al though ;-)

> Well, thank you for your time, and keep up the excellent work
> on this amazing project!

Cheers,
Ian


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Xen and Linux

[Dimitrie O. Paun]

On Fri, Nov 05, 2004 at 11:28:25PM +0000, Ian Pratt wrote:
> It would be very cool ;-) We might even get distros to ship it on
> their install CD...

Exactly where I was going ... :)

> There's one simple src change in arch Xen that has some quite far
> reaching consequences: we change FIXADDR_TOP which effectively
> gives us a 64MB whole at the top of the guest's VM space where
> Xen lives. This constant gets compiled into a bunch of different
> functions (though I believe it doesn't make it into modules
> --phew!).

Oh. Is there any way that the kernel can reuse that space
if it figures that it's not running under Xen. I haven't
looked too closely at the kernel's memory management, but
I seem to remember the zone allocators...

> I suspect it would be unpopular to make FIXADDR_TOP a variable,
> and it turns out to be a tricky thing to runtime patch. I think
> we just have to have arch-xen specific versions of all of the
> functions that use it.

Yes, in which case we could maybe turn it *there* into a variable.

> The simplest way to do this would be to have a tool that builds
> an x86 and xen kernel then merges the two together. Sounds a bit
> gross, but I think it would work quite well.

Maybe, but it would be bigger, and this maybe a problem for small
devices. Anyhow, not very satisfying :)

> Probably the cleanest solution of all would be to make all the
> other architectures adopt Xen's nice clean interfaces and then
> have stub routines for talking to the grotty realty of real
> hardware. Might be a hard one to sell to Linus et al though ;-)

Yeah, I'd think this would be the preferable route. How large
would such a patch be?. It may be worth it to float it on LKML 
to see the reaction :)

-- 
Dimi.


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Xen and Openbsd

[Dave Feustel]

Openbsd and Netbsd are kissing cousins. Is there any chance of getting 
Openbsd 3.6 (just released) or 3.7 (next release, 6months away) ported to Xen?

Thanks,
Dave Feustel
1-260-422-5330



-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Xen and Openbsd

[Janne Johansson]

Dave Feustel wrote:

>Openbsd and Netbsd are kissing cousins. Is there any chance of getting 
>Openbsd 3.6 (just released) or 3.7 (next release, 6months away) ported to Xen?
>  
>
As far as I can tell, no OS ports itself to Xen, so OpenBSD will run 
whenever someone actually
ports it to xen. Until then, the few xen coders will probably fix bugs 
in xen itself more, and port
less.

It's not a bad idea, though.



-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Xen and Openbsd

[Dave Feustel]

I would be willing to work on this if there were some Xen notes
specifying what needs to be changed in the OS. I don't think
I saw that info at the Xen website, although it's quite possible
I didn't recognize it when I saw it. 

On Sunday 07 November 2004 03:11 pm, Janne Johansson wrote:
> Dave Feustel wrote:
> >Openbsd and Netbsd are kissing cousins. Is there any chance of getting
> >Openbsd 3.6 (just released) or 3.7 (next release, 6months away) ported to
> > Xen?
>
> As far as I can tell, no OS ports itself to Xen, so OpenBSD will run
> whenever someone actually
> ports it to xen. Until then, the few xen coders will probably fix bugs
> in xen itself more, and port
> less.
>
> It's not a bad idea, though.

-- 
Dave Feustel
1-260-422-5330



-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Xen and Openbsd

[Mark A. Williamson]

A good place to start would be the Xen 2.0 Interface Manual (it's in the 
source tree and possibly on the website).  There's also a howto somewhere in 
the source tree about how the virtual device drivers work.

The *frontend* device drivers in the Linux and NetBSD trees are BSD-licensed 
and have been used as templates in other ports, so you can use those for your 
work.  There's various other stuff in the Linux port that's BSD licensed, tho 
I can't remember exactly what...

The xen-devel mailing list gives support to OS porters whenever required.

HTH,
Mark

On Sunday 07 Nov 2004 21:25, Dave Feustel wrote:
> I would be willing to work on this if there were some Xen notes
> specifying what needs to be changed in the OS. I don't think
> I saw that info at the Xen website, although it's quite possible
> I didn't recognize it when I saw it.
>
> On Sunday 07 November 2004 03:11 pm, Janne Johansson wrote:
> > Dave Feustel wrote:
> > >Openbsd and Netbsd are kissing cousins. Is there any chance of getting
> > >Openbsd 3.6 (just released) or 3.7 (next release, 6months away) ported
> > > to Xen?
> >
> > As far as I can tell, no OS ports itself to Xen, so OpenBSD will run
> > whenever someone actually
> > ports it to xen. Until then, the few xen coders will probably fix bugs
> > in xen itself more, and port
> > less.
> >
> > It's not a bad idea, though.


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Xen and Openbsd

[Janne Johansson]

Mark A. Williamson wrote:

> The xen-devel mailing list gives support to OS porters whenever required.
> 

I didn't mean to imply that you wouldn't give support, I tried to shortly
state that it's usually not enough to say "Hey, this OS would be cool to
have under xen, will you port it for me?".

I can see how it came out in a way that could be interpreted as if I said
you wouldn't help porters. Sorry for that.



-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Xen and Openbsd

[Keir Fraser]

> > The xen-devel mailing list gives support to OS porters whenever required.
> 
> I didn't mean to imply that you wouldn't give support, I tried to shortly
> state that it's usually not enough to say "Hey, this OS would be cool to
> have under xen, will you port it for me?".

That's how most of us read your original post - no need to
apologise. :-)

 -- Keir

> I can see how it came out in a way that could be interpreted as if I said
> you wouldn't help porters. Sorry for that.



-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Xen and Openbsd

[Dave Feustel]

On Monday 08 November 2004 06:27 am, Janne Johansson wrote:
> Mark A. Williamson wrote:
> > The xen-devel mailing list gives support to OS porters whenever required.
>
> I didn't mean to imply that you wouldn't give support, I tried to shortly
> state that it's usually not enough to say "Hey, this OS would be cool to
> have under xen, will you port it for me?".
>
> I can see how it came out in a way that could be interpreted as if I said
> you wouldn't help porters. Sorry for that.

I understood what your point was and I didn't take any offense.
I would use Xen ALL the time if it ran with Openbsd, and I will help
in any way that I can to make that happen. I have been a low-level programmer
(ie mostly (8086) assembly level for my entire programming career (~30 years)
but now I am getting old and out-of-touch with current programming paradigms, so
I am not sure just how much I could realistically contribute to the project.
But the spirit is willing! :-) I *have* just started a project that has first priority
until it is finished (I am getting paid for it), but, as I said, I am really interested in
Xen since it is so close to my core interests. Mike Adler and I wrote a simulator for the
CDC 6600 peripheral processor cluster back in 1969 when we worked for CDC 
at Arden Hills, Minnesota.

Dave Feustel



-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Xen and Openbsd

[Keir Fraser]

> I understood what your point was and I didn't take any offense.
> I would use Xen ALL the time if it ran with Openbsd, and I will help
> in any way that I can to make that happen. I have been a low-level programmer
> (ie mostly (8086) assembly level for my entire programming career (~30 years)
> but now I am getting old and out-of-touch with current programming paradigms, so
> I am not sure just how much I could realistically contribute to the project.

Unless you think that even the C language is new-fangled, being
out-of-touch with current paradigms is probably an advantage. :-)

 -- Keir


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Xen and Openbsd

[Dave Feustel]

On Monday 08 November 2004 08:39 am, Keir Fraser wrote:

> Unless you think that even the C language is new-fangled, being
> out-of-touch with current paradigms is probably an advantage. :-)

I figure 80% of my career to date has been spent programming assembly language 
on many machines and microcode(HP2100). I program passably in C, but essentially 
not at all in C++, although, I do have good C++ docs should I need them.

-- 
Dave Feustel
1-260-422-5330



-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Xen and Openbsd

[Mark A. Williamson]

> I can see how it came out in a way that could be interpreted as if I said
> you wouldn't help porters. Sorry for that.

No need to apologise, I knew what you meant!  Errr, guess I apologise now if 
my post gave the wrong impression!

I was just noting that for the previous OS ports, folks on the mailing list 
have always been eager to help, so it's a very valuable resource for anyone 
in that situation (especially when the documentation runs out and you're not 
sure what Xen is doing ;-).

Mark


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Xen and Openbsd

[Keir Fraser]

> I would be willing to work on this if there were some Xen notes
> specifying what needs to be changed in the OS. I don't think
> I saw that info at the Xen website, although it's quite possible
> I didn't recognize it when I saw it. 

If netbsd and openbsd really are that close, the netbsd-2.0 patch in
the Xen repository is likely to be your best guide. Christian Limpach
did that port and may be able to offer help.

 -- Keir


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Xen and Linux

[Ronald G. Minnich]

On Fri, 5 Nov 2004, Ian Pratt wrote:

> > possible to have a common kernel binary that would work with/without
> > Xen. In other words, to dynamically determine if it's running
> > under Xen, and behave appropriately.
> 
> This has been dubbed "transparent virtualization". 

note that IBM did it with CMS under VM, see Melinda Varian's excellent
article on all this.

ron


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Xen and Linux

[David Hopwood]

Ronald G. Minnich wrote:
> On Fri, 5 Nov 2004, Ian Pratt wrote:
> 
>>>possible to have a common kernel binary that would work with/without
>>>Xen. In other words, to dynamically determine if it's running
>>>under Xen, and behave appropriately.
>>
>>This has been dubbed "transparent virtualization". 
> 
> note that IBM did it with CMS under VM, see Melinda Varian's excellent
> article on all this.

<http://pucc.princeton.edu/~melinda/>
<http://pucc.princeton.edu/~melinda/25paper.pdf>

-- 
David Hopwood <david.nospam.hopwood@blueyonder.co.uk>



-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click