Xen - Mosix cluster

Xen - Mosix cluster

[Tom Cranbrook]

This may seems a bit of a strange question, but has any thought been given
to merging the ideas of Xen with Mosix?  A client asked me this, and I
can't come up with a clear answer.  I understand that both provide
clustering, but at a different granularity.  A Xen cluster can migrate an
entire server OS environment, while a Mosix cluster migrates specific
processes.  

I presume there is a basic assumption to Xen usage that the hardware
resource pool, represented by the resources of any single physical computer
system, smp or not, is always mayfold greater than the demands of any guest
domain.  Many OS images running concurently on the same machine is the
idea.  As a side effect, the resource usage of any on OS environment is
also limited at a maximum by the limites on any single node machine. (It's
also limited by configuration, I know.)  

Would building Xen on top of Mosix allow that resource pool to expand to
that of the entire Mosix cluster?   This would, if feasible, allow not only
more domains to be managed by a single Xen kernel, but also allow dynamic
expansion of single domain images beyond the limits of single hardware
nodes.

Just a thought.





-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl

Re: Xen - Mosix cluster

[Mark A. Williamson]

> This may seems a bit of a strange question, but has any thought been given
> to merging the ideas of Xen with Mosix?  A client asked me this, and I
> can't come up with a clear answer.  I understand that both provide
> clustering, but at a different granularity.  A Xen cluster can migrate an
> entire server OS environment, while a Mosix cluster migrates specific
> processes.
>
> I presume there is a basic assumption to Xen usage that the hardware
> resource pool, represented by the resources of any single physical computer
> system, smp or not, is always mayfold greater than the demands of any guest
> domain.  Many OS images running concurently on the same machine is the
> idea.  As a side effect, the resource usage of any on OS environment is
> also limited at a maximum by the limites on any single node machine. (It's
> also limited by configuration, I know.)

All true.  If a node is overloaded, you might migrate some domains off it.

> Would building Xen on top of Mosix allow that resource pool to expand to
> that of the entire Mosix cluster? 

Through Xen migration, you effectively get the resource pool of your entire 
Xen cluster but you can only run processes on their domain's current home 
CPU.

If you were going to use Mosix, it would be *on top* of Xen (not vice-versa) 
and would allow processes to migrate to other domains on other nodes.  Xen 
wouldn't know / care that this was going on, however - it just cares about 
scheduling domains on it's local CPU and doesn't really know about the 
processes within them.

> This would, if feasible, allow not only 
> more domains to be managed by a single Xen kernel, but also allow dynamic
> expansion of single domain images beyond the limits of single hardware
> nodes.

You could theoretically run Mosix in your Xen domains, allowing Mosix to 
migrate processes to other nodes in the cluster.  Xen wouldn't be involved in 
this though.

So, you can run (theoretically - the OpenMosix patches are currently against 
an older Linux kernel) Mosix in XenLinux on top of Xen itself, without 
requiring changes to Xen.  But you can't do this transparently to the guest 
OS.

Btw, somebody was talking about using OpenSSI on top of Xen some time last 
year but I don't know haw they got on.

HTH,
Mark


-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl

Re: Xen - Mosix cluster

[Tom Cranbrook]

>> idea.  As a side effect, the resource usage of any on OS environment is
>> also limited at a maximum by the limites on any single node machine.
(It's
>> also limited by configuration, I know.)
>
>All true.  If a node is overloaded, you might migrate some domains off it.
>

True, but the limit still remains.

Perhaps I am trying to have my cake and eat it too.  The advantages of Xen
are strong isolation of OS environments and more efficent use of resources
on large capacity machines.  The advantage of Mosix and OpenSSI is to
combine multiple machines to run an OS environment greater than a single
machine, using two different approaches.  In a sense, these systems and Xen
are at opposite purposes.  It would seem to me that putting OpenSSI 'on
top' of Zen would cancel out the benifits of Xen, and would not enhance the
benifits of SSI. (other than creating a test system for SSI.  That makes a
lot of sense.)  Mosix 'on top' of Xen is probably not useful either since
the advantage of Mosix is to move a demanding process to another machine,
not to another OS environemnt on the same machine.

If, however, Xen were on top of the stack, we may be able to achive  both
purposes.  The cluster hardware can be consolidated to support a single
base OS environment that can be distributed at the process level of
granularity.  Then Xen builds on this resource pool to provide the guest OS
environment isolation.  From Xens point of view, it would just have a way
big honker of a machine to work with.  This would also allow a high demand
OS environment to grow past the single machine limit.  This might also help
with issues of bringing nodes on and off line.

This would still leave us with a single machine speed limit for any one
proccess, plus overhead,  but ... hey  ...  one thing at a time.





-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl

Re: Xen - Mosix cluster

[Mark A. Williamson]

One can imagine some uses of Mosix in XenLinux atop Xen.  Xen still gives 
isolated driver domains and the ability to migrate domains around when you 
want to take a machine down for maintenance.  These features might be useful 
even if you only have one Mosix VM per node.

I agree that most people will probably just have a straight Mosix cluster or 
use Xen just as a test environment.

> If, however, Xen were on top of the stack, we may be able to achive  both
> purposes.

This is the real problem - Xen is specifically designed to live under the 
operating system kernel and provide a hardware-like abstraction.  It's not 
really aware that processes exist - they're a guest OS concept.  Xen is not 
really aware of the abstractions within a domain...

I see where you're coming from, though - it'd be neat to pool resources more 
flexibly this way.

> From Xens point of view, it would just have a way
> big honker of a machine to work with.

Since Xen lives at the bottom of the stack, this isn't really possible - Xen 
itself is really intimately tied to the specific machine its working on.  In 
contrast, Mosix can provide users with the illusion of a big machine but it's 
intimitely tied into the Linux kernel.

I don't know so much about OpenSSI, how does that abstract things to the user?

> This would also allow a high demand 
> OS environment to grow past the single machine limit.  This might also help
> with issues of bringing nodes on and off line.

This leads to an interesting thought though - Xen does accurate resource 
accounting on what domains have used.  That's one of its strengths.  A cool 
idea (although not one that'd necessarily get done) that'd partially address 
your problem would be to plug Xend into Mosix's process migration mechanisms.

e.g. only allow a process migration to another node if the domain owner has 
paid enough and then keep track of the resource usage on the remote node *as 
well*, so that the total resource usage is known.  One could imagine creating 
XenLinux/Mosix domains on other nodes on-demand when the user's virtual 
machine wants to migrate a compute-intensive process.  (Domains running in a 
ramdisk only take a few hundred milliseconds to start, so this is quite 
feasible).

This way one could get some of the advantages you mention and retain the 
strong isolation, resource accounting, etc.

I'll have a think about that, since it does sound kinda cool ;-)

Mark


-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl

Re: Xen - Mosix cluster

[Tom Cranbrook]

>
>Since Xen lives at the bottom of the stack, this isn't really possible -
Xen 
>itself is really intimately tied to the specific machine its working on. 
In 
>contrast, Mosix can provide users with the illusion of a big machine but
it's 
>intimitely tied into the Linux kernel.
>
>I don't know so much about OpenSSI, how does that abstract things to the
user?
>
SSI presents a single instance OS environment, one file structure, one name
space, one set of services, one proccess list in /proc , that reflects the
conbined resources of the nodes.  Each node machine is totally assemilated
into the one image. I'm not sure of this, but I believe a node can not even
function as a terminal to the SSI.  It's a headless boot off the network.

In Mosix, each node is running an indenpendent OS environment and its own
set of services.  A remote node only comes into play if any one node maxes
out and migrates one of its processes elsewhere to lighten the load.  Each
node can be a fully functional workstation.



>> This would also allow a high demand 
>> OS environment to grow past the single machine limit.  This might also
help
>> with issues of bringing nodes on and off line.
>
>This leads to an interesting thought though - Xen does accurate resource 
>accounting on what domains have used.  That's one of its strengths.  A
cool 
>idea (although not one that'd necessarily get done) that'd partially
address 
>your problem would be to plug Xend into Mosix's process migration
mechanisms.
>
>e.g. only allow a process migration to another node if the domain owner
has 
>paid enough and then keep track of the resource usage on the remote node
*as 
>well*, so that the total resource usage is known.  One could imagine
creating 
>XenLinux/Mosix domains on other nodes on-demand when the user's virtual 
>machine wants to migrate a compute-intensive process.  (Domains running in
a 
>ramdisk only take a few hundred milliseconds to start, so this is quite 
>feasible).
>
>This way one could get some of the advantages you mention and retain the 
>strong isolation, resource accounting, etc.
>
>I'll have a think about that, since it does sound kinda cool ;-)
>
>Mark
>





-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl