[Qemu-devel] QEMU redesigned for MPI (Message Passing Interface)

[Qemu-devel] QEMU redesigned for MPI (Message Passing Interface)

[Victor Vasilchenko]

[-- Attachment #1: Type: text/plain, Size: 1348 bytes --]

Good day,

I've got an idea in my head and can't get rid of it - neither can I find any
solution on web.

The idea is to use the Virtualization Software with the purpose OPPOSITE to
it's natural way.

The practical example below will explain it completely:

1) we take 4 common modern computers - CoreQuad + 8 GB Memory.
2) we assemble a standard Linux cluster with 16 cores and 32G memory.
3) and now - we run the only one virtual guest system, but give it ALL
available resources.

The result:
A Virtual PC running on 15 cores and 31Gb of memory (-1 for host OS and VM
software).

Any modern OS installed inside this Virtual PC will utilize all resources in
normal way :
1) the video encoding software will use 15 threads for transcoding
2) the 3D studios (Maya/Max) will use 15 threads and all 31GB memory
3) make -j 15
4) etc.

There is no need to redesign (with MPI MPICH/LAM support) a standard,
currently used programs - they will just work as one would run them on
standard CoreQuad system,
but almost 4 times faster (15cores).

Looks great so far, doesn't it?

The only BIG question - what virtualization software can support MPI to
effectively work on cluster ?

I am asking this question to QEMU team cause I believe QEMU can be the one.
If you have any ideas or experiments or information - please share with me.

Kind Regards,
Victor.

[-- Attachment #2: Type: text/html, Size: 1479 bytes --]

Re: [Qemu-devel] QEMU redesigned for MPI (Message Passing Interface)

[Mulyadi Santosa]

Hi Victor...

On 11/13/09, Victor Vasilchenko <vvasilchenko@gmail.com> wrote:
> The only BIG question - what virtualization software can support MPI to
> effectively work on cluster ?
>
> I am asking this question to QEMU team cause I believe QEMU can be the one.
> If you have any ideas or experiments or information - please share with me.
>
> Kind Regards,
> Victor.
>

If you check through Qemu list archieve, you shall see that there were
more or less similar question: parallelization of Qemu. Most of the
answers are: it can be hardly done. AFAIK, this is due to how dynamic
translation is done... you need (most of the part I guess) almost 100%
serialization.

This doesn't mean it is entirely impossible, I just said it's quite
hard. Even though the job is mostly code copy (e.g x86 to x86), you
still need to be careful when you hit i.e interrupt disabled code
region. That's all I can remember though...


-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Re: [Qemu-devel] QEMU redesigned for MPI (Message Passing Interface)

[Anthony Liguori]

Victor Vasilchenko wrote:
> Good day,
>
> I've got an idea in my head and can't get rid of it - neither can I 
> find any solution on web.
>
> The idea is to use the Virtualization Software with the purpose 
> OPPOSITE to it's natural way.
>
> The practical example below will explain it completely:
>
> 1) we take 4 common modern computers - CoreQuad + 8 GB Memory.
> 2) we assemble a standard Linux cluster with 16 cores and 32G memory.
> 3) and now - we run the only one virtual guest system, but give it ALL 
> available resources.

What you're describing is commonly referred to as a Single System 
Image.  It's been around for a while and can be found in software-only 
verses (pre-Xen VirtualIron, ScaleMP) and hardware-assisted (IBM, 3leaf).

Regards,

Anthony Liguori

Re: [Qemu-devel] QEMU redesigned for MPI (Message Passing Interface)

[Jamie Lokier]

Mulyadi Santosa wrote:
> Hi Victor...
> 
> On 11/13/09, Victor Vasilchenko <vvasilchenko@gmail.com> wrote:
> > The only BIG question - what virtualization software can support MPI to
> > effectively work on cluster ?
> >
> > I am asking this question to QEMU team cause I believe QEMU can be the one.
> > If you have any ideas or experiments or information - please share with me.
> >
> > Kind Regards,
> > Victor.
> >
> 
> If you check through Qemu list archieve, you shall see that there were
> more or less similar question: parallelization of Qemu.

> Most of the answers are: it can be hardly done. AFAIK, this is due
> to how dynamic translation is done... you need (most of the part I
> guess) almost 100% serialization.

I completely disagree.

Dynamic translation doesn't need 100% serialisation, but to relax that
it does require atomic ops and memory barriers (including implied
ones) in the guest to be translated to equivalents in the host.
Unfortunately x86 instructions imply a lot of barriers
(program-ordered writes), which is great for programming x86 but not
great when translating to non-x86 host code.

Right now, though, qemu's translator does not do that, even for
x86->x86, so it has to be serialised when running multi-CPU guest code.

KVM, on the other hand, does not do dynamic translation and can run
multiple threads in parallel because the atomic/barrier semantics are
preserved exactly.  KVM would be a fine candidate for running multiple
threads on different host machines.

You'd need to serialise memory using page faults, but such techniques
are already implemented with cluster filesystems and writable-mmap'd
files, so it might be possible to use KVM over that.

-- Jamie

Re: [Qemu-devel] QEMU redesigned for MPI (Message Passing Interface)

[Paul Brook]

> > The practical example below will explain it completely:
> >
> > 1) we take 4 common modern computers - CoreQuad + 8 GB Memory.
> > 2) we assemble a standard Linux cluster with 16 cores and 32G memory.
> > 3) and now - we run the only one virtual guest system, but give it ALL
> > available resources.

If the guest isn't aware of this discontinuity then performance will really 
suck. Generally speaking you have to split jobs anyway, the same as you would 
on a regular cluster, the SSI just makes migration and programming a little 
easier.

If you don't believe me then talk to anyone who's used large SSI systems (e.g. 
SGI Altix) - these systems have dedicated hardware assist and interconnect 
designed for SSI operation and you still have to be fairly selective about how 
you use them.

> What you're describing is commonly referred to as a Single System
> Image.  It's been around for a while and can be found in software-only
> verses (pre-Xen VirtualIron, ScaleMP) and hardware-assisted (IBM, 3leaf).

Or better still do it at the OS level (e.g. OpenSSI).

Paul

Re: [Qemu-devel] QEMU redesigned for MPI (Message Passing Interface)

[Avi Kivity]

On 11/17/2009 02:20 PM, Paul Brook wrote:
>> What you're describing is commonly referred to as a Single System
>> Image.  It's been around for a while and can be found in software-only
>> verses (pre-Xen VirtualIron, ScaleMP) and hardware-assisted (IBM, 3leaf).
>>      
> Or better still do it at the OS level (e.g. OpenSSI).
>    

This is at best incredibly difficult since you have to cluster a huge 
program (Linux).  If the OS is closed it's impossible even to start.

If you do this at the virtualization level all you need is distributed 
shared memory for guest RAM and clustering at the qemu level.  If each 
node gets its own PCI bus it shouldn't be too difficult.

-- 
error compiling committee.c: too many arguments to function