[Qemu-devel] RAM overcomittment

[Qemu-devel] RAM overcomittment

[The MoonSeeker]

Hello,

I'd like to know if qemu have a tool that let a VM to use more RAM  
than it have (when initialised).

Another question is  : Can we run run many VM using more RAM than the  
RAM installed (on the workstation)? For example I'd like to create 5  
VM's who have 256MB (can use 256MB but only for a slice of time) but  
normally one VM use only 50MB. On the workstation I have only 512MB  
RAM. Is it possible to create these VM's or have we to have 5 X 256  
RAM + HOST RAM = 1.5 GB installed on the workstation?

I think we can save ressource by using RAM overcomittment. What do  
you think? Is this implemented on qemu? Or will be implemented?

Thank you!

Re: [Qemu-devel] RAM overcomittment

[Paul Brook]

On Wednesday 27 September 2006 22:31, The MoonSeeker wrote:
> Hello,
>
> I'd like to know if qemu have a tool that let a VM to use more RAM
> than it have (when initialised).
>
> Another question is  : Can we run run many VM using more RAM than the
> RAM installed (on the workstation)? For example I'd like to create 5
> VM's who have 256MB (can use 256MB but only for a slice of time) but
> normally one VM use only 50MB. On the workstation I have only 512MB
> RAM. Is it possible to create these VM's or have we to have 5 X 256
> RAM + HOST RAM = 1.5 GB installed on the workstation?
>
> I think we can save ressource by using RAM overcomittment. What do
> you think? Is this implemented on qemu? Or will be implemented?

qemu is just like any other application. It is only limited by how much 
virtual memory your OS can provide. ie. if you have sufficient swap you can 
have as many qemu instances using as much memory as you want.

qemu is currently limits each guest to 2Gb ram. This is independent of how 
much physical memory the host has.

Note that modern OS (everything except DOS) generally use all available ram.  
Telling qemu to use more memory than you have physical ram is liable to cause 
heavy swapping.

Paul

Re: [Qemu-devel] RAM overcomittment

[The MoonSeeker]

[-- Attachment #1: Type: text/plain, Size: 1463 bytes --]

Le 27 sept. 06 à 23:41, Paul Brook a écrit :

> qemu is just like any other application. It is only limited by how  
> much
> virtual memory your OS can provide. ie. if you have sufficient swap  
> you can
> have as many qemu instances using as much memory as you want.
>
> qemu is currently limits each guest to 2Gb ram. This is independent  
> of how
> much physical memory the host has.
>
> Note that modern OS (everything except DOS) generally use all  
> available ram.
> Telling qemu to use more memory than you have physical ram is  
> liable to cause
> heavy swapping.

Ok but some virtual solution like openVZ allow you run more VM than  
the memory installed. By example, with openVZ I can create 10 Virtual  
Machine who have a limite fixe to 200 MB but have guaranteed RAM of  
20MB. With qemu I need to have 10 X 200MB for VM's + 128 MB host of  
RAM installed on the work station...

I think we can Save ressource because in the most case, the VM's will  
never use the 200MB. I think it will be a nice if qemu implemented a  
tool that let use exceed this limitation.

I can show you an application domain :

For my diploma project, I have to create a network simulator but with  
these limitation I can't use qemu. Because if would like to simulate  
20 workstions I need 20 X 128 MB = 2560 MB of RAM... + host RAM!!!  
But in the simulation, the VM's never will use all of ressource..

What do you?

Thank you

[-- Attachment #2: Type: text/html, Size: 3227 bytes --]

Re: [Qemu-devel] RAM overcomittment

[Paul Brook]

> Ok but some virtual solution like openVZ allow you run more VM than
> the memory installed. By example, with openVZ I can create 10 Virtual
> Machine who have a limite fixe to 200 MB but have guaranteed RAM of
> 20MB. With qemu I need to have 10 X 200MB for VM's + 128 MB host of
> RAM installed on the work station...

Nonsense.

> I think we can Save ressource because in the most case, the VM's will  
> never use the 200MB. I think it will be a nice if qemu implemented a  
> tool that let use exceed this limitation.

Like I said, qemu is just like any other application. It's up to your host OS 
to handle management of physical RAM. There's no need for additional tools.

> For my diploma project, I have to create a network simulator but with  
> these limitation I can't use qemu. Because if would like to simulate  
> 20 workstions I need 20 X 128 MB = 2560 MB of RAM... + host RAM!!!  
> But in the simulation, the VM's never will use all of ressource..

Just add a few Gb swap. Even the oldest of machines shouldn't have any problem 
adding say 4Gb swap. Depending on your OS settings you may not even need to 
do that.

Besides which modern entry-level PCs should be able to take 2Gb ram without 
significant problems. Workstations with 8Gb aren't that hard to come by.

Paul

Re: [Qemu-devel] RAM overcomittment

[The MoonSeeker]

[-- Attachment #1: Type: text/plain, Size: 1004 bytes --]


Le 28 sept. 06 à 00:19, Paul Brook a écrit :

>> For my diploma project, I have to create a network simulator but with
>> these limitation I can't use qemu. Because if would like to simulate
>> 20 workstions I need 20 X 128 MB = 2560 MB of RAM... + host RAM!!!
>> But in the simulation, the VM's never will use all of ressource..
>
> Just add a few Gb swap. Even the oldest of machines shouldn't have  
> any problem
> adding say 4Gb swap. Depending on your OS settings you may not even  
> need to
> do that.
>
> Besides which modern entry-level PCs should be able to take 2Gb ram  
> without
> significant problems. Workstations with 8Gb aren't that hard to  
> come by.

If we have enough swap memory, You are saying that is possible to run  
20 MV's with option -M equal 128MB?

For example the VM's (20) normally use 20MB and I have 512Mb of RAM  
on my workstation, what's about the performance?

RAM used = 20X20MB + 112MB for host = 512MB nothing is swap???

Thank you

[-- Attachment #2: Type: text/html, Size: 2758 bytes --]

Re: [Qemu-devel] RAM overcomittment

[Paul Brook]

On Wednesday 27 September 2006 23:53, The MoonSeeker wrote:
> Le 28 sept. 06 à 00:19, Paul Brook a écrit :
> >> For my diploma project, I have to create a network simulator but with
> >> these limitation I can't use qemu. Because if would like to simulate
> >> 20 workstions I need 20 X 128 MB = 2560 MB of RAM... + host RAM!!!
> >> But in the simulation, the VM's never will use all of ressource..
> >
> > Just add a few Gb swap. Even the oldest of machines shouldn't have
> > any problem
> > adding say 4Gb swap. Depending on your OS settings you may not even
> > need to
> > do that.
>
> If we have enough swap memory, You are saying that is possible to run
> 20 MV's with option -M equal 128MB?

Yes, exactly.

Paul

[Qemu-devel] PowerPC Decrementer Clock Rate

[Ely Soto]

[-- Attachment #1: Type: text/plain, Size: 1348 bytes --]

Hi,

Does the time-base setting below signify that the PowerPC Decrementer is 
at 100MHz or is it the CPU?

This isn't completely obvious because sometimes you have something like 4 
bus clocks per decrementer increment.

...
ppc_prep.c
    /* Set time-base frequency to 100 Mhz */
    cpu_ppc_tb_init(env, 100UL * 1000UL * 1000UL);
...

Thanks,

Ely Soto



-----------------------------------------
Notice:  This e-mail is intended solely for use of the individual
or entity to which it is addressed and may contain information that
is proprietary, privileged and exempt from disclosure under
applicable law.  If the reader is not the intended recipient or
agent responsible for delivering the message to the intended
recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly
prohibited.  This communication may also contain data subject to
U.S. export laws.  If so, that data subject to the International
Traffic in Arms Regulation cannot be disseminated, distributed or
copied to foreign nationals, residing in the U.S. or abroad, absent
the express prior approval of the U.S. Department of State.   If
you have received this communication in error, please notify the
sender by reply e-mail and destroy the e-mail message and any
physical copies made of the communication.  Thank you.

[-- Attachment #2: Type: text/html, Size: 1967 bytes --]

Re: [Qemu-devel] RAM overcomittment

[andrzej zaborowski]

Hi,

On 27/09/06, The MoonSeeker <themoonseeker@gmail.com> wrote:
> Ok but some virtual solution like openVZ allow you run more VM than the
> memory installed. By example, with openVZ I can create 10 Virtual Machine
> who have a limite fixe to 200 MB but have guaranteed RAM of 20MB. With qemu
> I need to have 10 X 200MB for VM's + 128 MB host of RAM installed on the
> work station...
>
> I think we can Save ressource because in the most case, the VM's will never
> use the 200MB. I think it will be a nice if qemu implemented a tool that let
> use exceed this limitation.

How would you implement that?

>
> I can show you an application domain :
>
> For my diploma project, I have to create a network simulator but with these
> limitation I can't use qemu. Because if would like to simulate 20 workstions
> I need 20 X 128 MB = 2560 MB of RAM... + host RAM!!! But in the simulation,
> the VM's never will use all of ressource..

Then give the VMs only as much as they will use (note that VMs can
also have swap) or use host swap.

Regards,
-- 
balrog 2oo6

Re: [Qemu-devel] RAM overcomittment

[James Oakley]

On Wednesday 27 September 2006 6:59 pm, The MoonSeeker wrote:
> Le 27 sept. 06 à 23:41, Paul Brook a écrit :
> > qemu is just like any other application. It is only limited by how
> > much
> > virtual memory your OS can provide. ie. if you have sufficient swap
> > you can
> > have as many qemu instances using as much memory as you want.
> >
> > qemu is currently limits each guest to 2Gb ram. This is independent
> > of how
> > much physical memory the host has.
> >
> > Note that modern OS (everything except DOS) generally use all
> > available ram.
> > Telling qemu to use more memory than you have physical ram is
> > liable to cause
> > heavy swapping.
>
> Ok but some virtual solution like openVZ allow you run more VM than
> the memory installed. By example, with openVZ I can create 10 Virtual
> Machine who have a limite fixe to 200 MB but have guaranteed RAM of
> 20MB. With qemu I need to have 10 X 200MB for VM's + 128 MB host of
> RAM installed on the work station...
>
> I think we can Save ressource because in the most case, the VM's will
> never use the 200MB. I think it will be a nice if qemu implemented a
> tool that let use exceed this limitation.

Only Windows guests. The Virtuozzo guys don't advertise the fact that Linux 
*uses all available RAM*.

Overselling RAM for Linux guests as a terrible idea.

-- 
James Oakley
Engineering - SolutionInc Ltd.
joakley@solutioninc.com
http://www.solutioninc.com

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
This e-mail is CONFIDENTIAL and contains information intended only for the
person(s) named. Any other distribution, copying or disclosure is strictly
prohibited. If you have received this e-mail in error, please notify me
immediately at 902 420 0077 or reply by e-mail to the sender and destroy
the original communication.
Thank You.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Re: [Qemu-devel] RAM overcomittment

[Bill C. Riemers]

[-- Attachment #1: Type: text/plain, Size: 2761 bytes --]

I would suggest just the opposite, under  commit your ram.  If your base
operating system is Linux, you can create your swap partitions under a host
linux tmpfs directory.  You can then safely over commit the amount of tmpfs
swap space.  The guest linux system will expect he swap to be slow, so it
won't use it for buffering disk IO or such.  However, until you actually
exhaust the amount of physical ram available the tmpfs swap will actually be
very fast.

Bill


On 9/28/06, James Oakley <joakley@solutioninc.com> wrote:
>
> On Wednesday 27 September 2006 6:59 pm, The MoonSeeker wrote:
> > Le 27 sept. 06 à 23:41, Paul Brook a écrit :
> > > qemu is just like any other application. It is only limited by how
> > > much
> > > virtual memory your OS can provide. ie. if you have sufficient swap
> > > you can
> > > have as many qemu instances using as much memory as you want.
> > >
> > > qemu is currently limits each guest to 2Gb ram. This is independent
> > > of how
> > > much physical memory the host has.
> > >
> > > Note that modern OS (everything except DOS) generally use all
> > > available ram.
> > > Telling qemu to use more memory than you have physical ram is
> > > liable to cause
> > > heavy swapping.
> >
> > Ok but some virtual solution like openVZ allow you run more VM than
> > the memory installed. By example, with openVZ I can create 10 Virtual
> > Machine who have a limite fixe to 200 MB but have guaranteed RAM of
> > 20MB. With qemu I need to have 10 X 200MB for VM's + 128 MB host of
> > RAM installed on the work station...
> >
> > I think we can Save ressource because in the most case, the VM's will
> > never use the 200MB. I think it will be a nice if qemu implemented a
> > tool that let use exceed this limitation.
>
> Only Windows guests. The Virtuozzo guys don't advertise the fact that
> Linux
> *uses all available RAM*.
>
> Overselling RAM for Linux guests as a terrible idea.
>
> --
> James Oakley
> Engineering - SolutionInc Ltd.
> joakley@solutioninc.com
> http://www.solutioninc.com
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> This e-mail is CONFIDENTIAL and contains information intended only for the
> person(s) named. Any other distribution, copying or disclosure is strictly
> prohibited. If you have received this e-mail in error, please notify me
> immediately at 902 420 0077 or reply by e-mail to the sender and destroy
> the original communication.
> Thank You.
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel
>

[-- Attachment #2: Type: text/html, Size: 3402 bytes --]

Re: [Qemu-devel] RAM overcomittment

[James Olsen]

Hello,

TM> Another question is  : Can we run run many VM using more RAM than the
TM> RAM installed (on the workstation)? For example I'd like to create 5  
TM> VM's who have 256MB (can use 256MB but only for a slice of time) but  
TM> normally one VM use only 50MB. On the workstation I have only 512MB  
TM> RAM. Is it possible to create these VM's or have we to have 5 X 256  
TM> RAM + HOST RAM = 1.5 GB installed on the workstation?

TM> I think we can save ressource by using RAM overcomittment. What do  
TM> you think? Is this implemented on qemu? Or will be implemented?

It seems to me that no change to QEMU is needed for this; it should
already be supported by your host OS. Simply allocate the memory that
you want (256mb, for example) for each virtual machine. Depending on
how many virtual machines you have open at once, as well as the memory
used by the host OS and any other applications you're running, the
host OS will fit the as much as it can in the physical ram, and
automatically swap out the excess allocated memory to disk.

However, swapping (using virtual memory) will be a huge performance
killer. It will affect the performance of your host OS and all
applications, as well as for your QEMU instances.

-- 
James
Funny quotes: "There are 10 types of people in the world. 
Those who understand binary, and those who don't." -- Unknown
"A computer once beat me at chess, but it was no match for me at
kick boxing." -- Emo Philips

Re: [Qemu-devel] RAM overcomittment

[The MoonSeeker]

[-- Attachment #1: Type: text/plain, Size: 1550 bytes --]

Le 27 sept. 06 à 23:57, James Olsen a écrit :

> It seems to me that no change to QEMU is needed for this; it should
> already be supported by your host OS. Simply allocate the memory that
> you want (256mb, for example) for each virtual machine. Depending on
> how many virtual machines you have open at once, as well as the memory
> used by the host OS and any other applications you're running, the
> host OS will fit the as much as it can in the physical ram, and
> automatically swap out the excess allocated memory to disk.
>
> However, swapping (using virtual memory) will be a huge performance
> killer. It will affect the performance of your host OS and all
> applications, as well as for your QEMU instances.

The problem is the swaping even the VM don't need all of ressource  
assigned... We lost RAM...

By example you run 4 VM who have 256 MB (assigned with -M option) but  
just use 56MB in most of time..

with qemu we lost 4 X (256-56MB) = 800MB of RAM!

Somme other virtualization solution like OpenVz and VServer let this  
RAM overcommitment. On a VM when you make the command free you can  
show all RAM installed on the workstation, and with file  
configuration you set the guaranteed RAM (in this case 56MB) and the  
limite (in this case 256MB). The limit can be reach only for a cup of  
time (when VM need ressource..)!!!

Do you understant what I mean?

I'd like to create an open network simulator based on qemu but this  
restriction can't let me use qemu...

What do you think?


  

[-- Attachment #2: Type: text/html, Size: 3410 bytes --]

Re: [Qemu-devel] RAM overcomittment

[Paul Brook]

> > However, swapping (using virtual memory) will be a huge performance
> > killer. It will affect the performance of your host OS and all
> > applications, as well as for your QEMU instances.
>
> The problem is the swaping even the VM don't need all of ressource
> assigned... We lost RAM...

You need to fix your host OS then. If the guest isn't using the memory it 
shouldn't be swapped in.

Paul