[Qemu-devel] What is the minimal linux setup for running Qemu ?

[Qemu-devel] What is the minimal linux setup for running Qemu ?

[Khan, Mohammad]

[-- Attachment #1: Type: text/plain, Size: 949 bytes --]

I'm wondering about what the minimum requirements are for Linux host to
be able to run Qemu?  The idea is to have a less resource hungry host so
that the guests would have more resources. Is an X server required? Can
a frame buffer device be used for guest display instead? What parts of
kernel or libraries can be removed. How about such a minimalist distro
which only runs Qemu and related untilities?
Any comments.
Mohammad
 

********************************************
This message is intended only for the use of the Addressee and
may contain information that is PRIVILEGED and CONFIDENTIAL.

If you are not the intended recipient, you are hereby notified
that any dissemination of this communication is strictly prohibited.

If you have received this communication in error, please erase
all copies of the message and its attachments and notify us
immediately.

Thank you.
********************************************

[-- Attachment #2: Type: text/html, Size: 4850 bytes --]

Re: [Qemu-devel] What is the minimal linux setup for running Qemu ?

[Richard Neill]

Dear Mohammad,

My understanding of this is that you are trying to maximise the emulated 
performance. In general, QEmu imposes a performance penalty of (roughly) 
a factor of 4, so if you want good performance on the guest, you need 
Pentium-4 class hardware for the host, with at least 512 MB RAM.

On a system of this level, wasted disk space is probably not too much of 
an issue, so trying to pare down your distro to the bare minimum may 
well not help much. Eg having unncessary libraries on the system won't 
matter at all, if they just take up a 100 MB of disk space.

Likewise, having unnecessary kernel modules compiled on your system 
won't matter, if they aren't actually loaded at the time. That said, if 
you compile the kernel yourself with all the optimisations for your 
system, you'll do better than a distro-supplied kernel.

Running a regular X-server is actually more efficient than the 
frame-buffer device. The FB is simpler and uses less disk space, but 
performance isn't quite so good.

What will probably help most is killing off unneeded processes, stopping 
unwanted daemons (to free up RAM), and running a simple window manager 
(eg icewm; fvwm2) rather than KDE.

Another way to obtain a (fairly) minimal system easily would be to 
install the base distro of, say Mandrake, and let the packaging tool 
resolve the dependencies of qemu. [You'd then need to upgrade to the 
latest qemu version].

Lastly, have you checked out Metropipe's product? This (free) package 
from:  http://www.metropipe.net/ProductsPVPM.shtml
contains:
	All that is needed to run Qemu under Linux
	All that is needed to run Qemu under Windows
	A Linux Guest system.
And it fits on a USB key.


I hope that's useful.

Regards

Richard




Khan, Mohammad wrote:
> I'm wondering about what the minimum requirements are for Linux host to
> be able to run Qemu?  The idea is to have a less resource hungry host so
> that the guests would have more resources. Is an X server required? Can
> a frame buffer device be used for guest display instead? What parts of
> kernel or libraries can be removed. How about such a minimalist distro
> which only runs Qemu and related untilities?
> Any comments.
> Mohammad
>  
> 
> ********************************************
> This message is intended only for the use of the Addressee and
> may contain information that is PRIVILEGED and CONFIDENTIAL.
> 
> If you are not the intended recipient, you are hereby notified
> that any dissemination of this communication is strictly prohibited.
> 
> If you have received this communication in error, please erase
> all copies of the message and its attachments and notify us
> immediately.
> 
> Thank you.
> ********************************************
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel

-- 
rn214@hermes.cam.ac.uk  **  http://www.richardneill.org
Richard Neill, Trinity College, Cambridge, CB21TQ, U.K.

Re: [Qemu-devel] What is the minimal linux setup for running Qemu ?

[jeebs]

From: "Richard Neill" <rn214@hermes.cam.ac.uk>
> Lastly, have you checked out Metropipe's product? This (free) package 
> from:  http://www.metropipe.net/ProductsPVPM.shtml
> contains:
> All that is needed to run Qemu under Linux
> All that is needed to run Qemu under Windows
> A Linux Guest system.
> And it fits on a USB key.

Thanks for telling us about this...  Nothing inovative, but nice to know.

However, I'm not so sure I'd want to run it on a flash memory device!

Any virtual disk is going to do quite a few writes, and flash memory can only do so many before it starts to fail.

It's worse with an OS that does a swap file (such as BartPE (WinXP) or Win9x on a flash device), but even for a non booting drive, the directory structure could still go through a lot of writes.  (Especially if the OS modifies the 'file accessed' time for a file that gets accessed often.)

Sure, flash memory can do many writes, but it only takes a single bit to fail to cause problems and possibly make it impossible to get important stuff off that virtual disk.  And directory structures are prime candidates for failing first.

It would have been better if they had done something to set up a small ram disk to hold things, and then copy it back to the device when you get done.

Re: [Qemu-devel] What is the minimal linux setup for running Qemu ?

[Richard Neill]

jeebs wrote:
> From: "Richard Neill" <rn214@hermes.cam.ac.uk>
> 
>>Lastly, have you checked out Metropipe's product? This (free) package 
>>from:  http://www.metropipe.net/ProductsPVPM.shtml
>>contains:
>>All that is needed to run Qemu under Linux
>>All that is needed to run Qemu under Windows
>>A Linux Guest system.
>>And it fits on a USB key.
> 
> 
> Thanks for telling us about this...  Nothing inovative, but nice to know.
> 
> However, I'm not so sure I'd want to run it on a flash memory device!

Actually, the metropipe key doesn't have this problem. The Guest OS is 
Damnn Small Linux, which is knoppix-based. This is simply an iso CD-ROM 
(image) which is read-only. So the only writes are to your 
home-directory. I haven't tested it, but I'd assume that the memory key 
home directory is mounted sync,noatime.

Metropipe's product is actually quite interesting, since they aim to 
store all your files on their server, and the memory key provides 
firefox,ssh etc and a virtual private network.

> 
> Any virtual disk is going to do quite a few writes, and flash memory can only do so many before it starts to fail.

I thought this number was > 100,000   - so it's still not really an 
issue except for atime and swap.

> 
> It's worse with an OS that does a swap file (such as BartPE (WinXP) or Win9x on a flash device), but even for a non booting drive, the directory structure could still go through a lot of writes.  (Especially if the OS modifies the 'file accessed' time for a file that gets accessed often.)
> 
> Sure, flash memory can do many writes, but it only takes a single bit to fail to cause problems and possibly make it impossible to get important stuff off that virtual disk.  And directory structures are prime candidates for failing first.


Not relevant here, but doesn't JFFS deal with this problem?

> It would have been better if they had done something to set up a small ram disk to hold things, and then copy it back to the device when you get done.



Regards

Richard

Re: [Qemu-devel] What is the minimal linux setup for running Qemu ?

[jeebs]

From: "Richard Neill" <rn214@hermes.cam.ac.uk>

> Actually, the metropipe key doesn't have this problem. The Guest OS is 
> Damnn Small Linux, which is knoppix-based. This is simply an iso CD-ROM 
> (image) which is read-only. So the only writes are to your 

Right.

> home-directory. I haven't tested it, but I'd assume that the memory key 
> home directory is mounted sync,noatime.

I don't happen to have a memory stick to test it with.  (To be honest, I've never gotten around to buying one.)
 
> Metropipe's product is actually quite interesting, since they aim to 
> store all your files on their server, and the memory key provides 
> firefox,ssh etc and a virtual private network.

There's nothing on the web site about that.

Their VPM page talks about storing things 'in your pocket' etc., so it sounds like just a typical local virtual drive right on the memory device itself.  That's what prompted to me make my comment in here.

They do provide a demo of their tunneler, but I don't see anything about the product requiring its use etc. etc.
 
>> Any virtual disk is going to do quite a few writes, and flash memory can only do so many before it starts to fail.
> 
> I thought this number was > 100,000   - so it's still not really an 
> issue except for atime and swap.

I really don't know what the cycle count is.

Certainly on the higher quality flash chips, it's over a million bit write cycles.

But with so many ultra cheap memory sticks with memory from no telling where, it's not something I'd want to depend on for more than a few tens of thousand writes per bit.

And unless there is proof to the contrary about the OS and the apps, you kind of have to assume that anytime you actually run any programs, it's going to update a directory or a config file, or whatever.

Maybe I'm wrong, but at this point, I find it hard to believe that it's not going to be doing lots of writes to the virtual disk when you actually use stuff, like MetroPipe is trying to suggest.   Simply booting it for a demonstration or the ocasional emergency, etc. is no problem.

But MetroPipe is talking about actually using it for real, and I'm real hesitant about the life span of the flash area where the directories are at.  Unless MetroPipe has taken explicit steps to reduce this and I don't know about it.


There isn't enough information of the web site to say for sure how it's set up (I haven't tried it yet), but I'd guess it's nothing more than a typical virtual disk for storage of your stuff, and an ISO for the linux.

Just a simple, no frills setup involving qemu, an empty virtual disk, and a standard DSL iso image.  That's why I said it was interesting, but not inovative.  (If I'm wrong, then great!)



> Not relevant here, but doesn't JFFS deal with this problem?

Don't know.
 

Re: [Qemu-devel] What is the minimal linux setup for running Qemu ?

[Richard Neill]

jeebs wrote:

>> Metropipe's product is actually quite interesting, since they aim
>> to store all your files on their server, and the memory key
>> provides firefox,ssh etc and a virtual private network.
> 
> 
> There's nothing on the web site about that.
> 
> Their VPM page talks about storing things 'in your pocket' etc., so
> it sounds like just a typical local virtual drive right on the memory
> device itself.  That's what prompted to me make my comment in here.
> 
> They do provide a demo of their tunneler, but I don't see anything
> about the product requiring its use etc. etc.
> 
> 

I downloaded their demo, but replaced their metropipe iso with the
"regular" Damn Small Linux iso. It does work well.


>>> Any virtual disk is going to do quite a few writes, and flash
>>> memory can only do so many before it starts to fail.
>> 
>> I thought this number was > 100,000   - so it's still not really an
>>  issue except for atime and swap.
> 
> 
> I really don't know what the cycle count is. Certainly on the higher
> quality flash chips, it's over a million bit write cycles. But with
> so many ultra cheap memory sticks with memory from no telling where,
> it's not something I'd want to depend on for more than a few tens of
> thousand writes per bit. And unless there is proof to the contrary
> about the OS and the apps, you kind of have to assume that anytime
> you actually run any programs, it's going to update a directory or a
> config file, or whatever.
> 

Running the OS cannot do any writing, since the ISO expects to be a
CDROM, and is read-only. Furthermore, Knoppix normally expects /home
(and /tmp) to be a RAMdisk.  This means that the apps won't save their
config files by default.


> Maybe I'm wrong, but at this point, I find it hard to believe that
> it's not going to be doing lots of writes to the virtual disk when
> you actually use stuff, like MetroPipe is trying to suggest.   Simply
> booting it for a demonstration or the ocasional emergency, etc. is no
> problem. But MetroPipe is talking about actually using it for real,
> and I'm real hesitant about the life span of the flash area where the
> directories are at.  Unless MetroPipe has taken explicit
 > steps to reduce this and I don't know about it.
> There isn't enough information of the web site to say for sure how
> it's set up (I haven't tried it yet), but I'd guess it's nothing more
> than a typical virtual disk for storage of your stuff, and an ISO for
> the linux. Just a simple, no frills setup involving qemu, an empty
> virtual disk, and a standard DSL iso image.  That's why I said it was
> interesting, but not inovative.  (If I'm wrong, then great!)
> 
> 

Exactly. And as you say, the problem is that writing to files in a
virtual disk means that any slight corruption to the real physical file
may completely destroy the filesystem on the virtual disk. A crash of
the host would also be less damaging.

What *would* be cute would be to use the virtual vfat driver. Thus files
on the guest map to real files on the memory key. If you do this, the
number of writes will be exactly the same as if you used the memory key
for transport of files in the usual way.


My ideal set up would be the following:


USB-memory-key/
		/qemu/mac   <-  mac binaries for qemu and bios
		/qemu/linux <- linux binaries for qemu and bios
		/qemu/win   <-  windows binaries for qemu and bios

		/dsl/iso    <-  dsl iso itself
		/dsl/extras <-  extra apps for dsl if desired.

		/myfiles   <- an ordinary directory, not a disk image.
			      appearing to the guest as a drive using
                               virtual vfat

		/putty	   <-  ssh binaries for win,linux,mac
		/vnc       <-  vnc viewer binaries for win,linux,mac

		win-startup.bat	 <- start up scripts for qemu/dsl
		mac-startup.sh
		linux-startup.sh


The only problems that remain are:
	-  vfat doesn't support Linux file attributes.
             (there is a solution, but I forget what).

	- you need a script to manually save those files within /home
           that you want to keep. Most can vanish with the ramdisk on
          reboot, but things like ~/.ssh  and ~/.mozilla are useful!
	[DSL and Knoppix both solve this problem somehow ]


In addition, with the falling prices of memory keys, an entire knoppix
iso could be used, if desired. Furthermore, since we know exactly what
the emulated hardware will be, knoppix could be compiled with the
appropriate optimisations.


Regards

Richard

Re: [Qemu-devel] What is the minimal linux setup for running Qemu ?

[Jim C. Brown]

On Thu, Dec 23, 2004 at 01:40:57AM +0000, Richard Neill wrote:
> 
> 
> What *would* be cute would be to use the virtual vfat driver. Thus files
> on the guest map to real files on the memory key. If you do this, the
> number of writes will be exactly the same as if you used the memory key
> for transport of files in the usual way.
> 
> 

Or you could use a loopback on a RAM disk, and then merely copy the loopback
to the flash when you are done. This actually reduces the writes (since you then
only need to write one file out per session) but has its cost in reliability
(if something happens and the session ends w/o the loopback being copied over
(say power goes out) then you loose all the work you have done for that session).

> 
> The only problems that remain are:
> 	-  vfat doesn't support Linux file attributes.
>             (there is a solution, but I forget what).

There are 2 that I know of, but they both have costs:

Simplest is to use a loopback file system on the vfat. This would be quite
badly inefficent tho, and may incur a significant cost in extra writes.

The other solution is to use either umsdos or uvfat. These require an extra
"hidden" file per directory, and the file gets written every time you update
unix file attributes and such, so this incurs an addition cost.

> 
> 	- you need a script to manually save those files within /home
>           that you want to keep. Most can vanish with the ramdisk on
>          reboot, but things like ~/.ssh  and ~/.mozilla are useful!
> 	[DSL and Knoppix both solve this problem somehow ]

I think, by coping those files manually to a directory on the DOS partition of
the hdd.

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.

RE: [Qemu-devel] What is the minimal linux setup for running Qemu ?

[Andreas Bollhalder]

> Or you could use a loopback on a RAM disk, and then merely 
> copy the loopback
> to the flash when you are done. This actually reduces the 
> writes (since you then
> only need to write one file out per session) but has its cost 
> in reliability
> (if something happens and the session ends w/o the loopback 
> being copied over
> (say power goes out) then you loose all the work you have 
> done for that session).

Why not using a temporary disk image on the lokal host drive(s) ?

Andreas

Re: [Qemu-devel] What is the minimal linux setup for running Qemu ?

[Jim C. Brown]

On Thu, Dec 23, 2004 at 07:36:13PM +0100, Andreas Bollhalder wrote:
> > Or you could use a loopback on a RAM disk, and then merely 
> > copy the loopback
> > to the flash when you are done. This actually reduces the 
> > writes (since you then
> > only need to write one file out per session) but has its cost 
> > in reliability
> > (if something happens and the session ends w/o the loopback 
> > being copied over
> > (say power goes out) then you loose all the work you have 
> > done for that session).
> 
> Why not using a temporary disk image on the lokal host drive(s) ?
> 
> Andreas
> 
> 

That works too, it would just be more difficult to set up automaticlly.

> 
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel
> 

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.

[Qemu-devel] Re: What is the minimal linux setup for running Qemu ?

[Ronald]

Le Thu, 23 Dec 2004 01:40:57 +0000, Richard Neill a écrit :


> Exactly. And as you say, the problem is that writing to files in a virtual
> disk means that any slight corruption to the real physical file may
> completely destroy the filesystem on the virtual disk. A crash of the host
> would also be less damaging.
> 
> What *would* be cute would be to use the virtual vfat driver. Thus files
> on the guest map to real files on the memory key. If you do this, the
> number of writes will be exactly the same as if you used the memory key
> for transport of files in the usual way.
> 
> 
> My ideal set up would be the following:
> 
> 
> USB-memory-key/
> 		/qemu/mac   <-  mac binaries for qemu and bios /qemu/linux <- linux
> 		binaries for qemu and bios /qemu/win   <-  windows binaries for qemu and
> 		bios
> 
> 		/dsl/iso    <-  dsl iso itself
> 		/dsl/extras <-  extra apps for dsl if desired.
> 
> 		/myfiles   <- an ordinary directory, not a disk image.
> 			      appearing to the guest as a drive using
>                                virtual vfat
> 
> 		/putty	   <-  ssh binaries for win,linux,mac /vnc       <-  vnc viewer
> 		binaries for win,linux,mac
> 
> 		win-startup.bat	 <- start up scripts for qemu/dsl mac-startup.sh
> 		linux-startup.sh
> 
> 
> The only problems that remain are:
> 	-  vfat doesn't support Linux file attributes.
>              (there is a solution, but I forget what).
> 
> 	- you need a script to manually save those files within /home
>            that you want to keep. Most can vanish with the ramdisk on
>           reboot, but things like ~/.ssh  and ~/.mozilla are useful!
> 	[DSL and Knoppix both solve this problem somehow ]
> 
> 
> In addition, with the falling prices of memory keys, an entire knoppix iso
> could be used, if desired. Furthermore, since we know exactly what the
> emulated hardware will be, knoppix could be compiled with the appropriate
> optimisations.
> 

I find this set up a bit complicated (or I have not understood the goal
maybe), I think you can do something simpler by using qemu's features like
snapshot and compressed qcow image, this way you only write to the usb
stick if you have done some real changes by commiting to the image.