[Qemu-devel] Making qemu images executable (and store command line arguments in them =P)

[Qemu-devel] Making qemu images executable (and store command line arguments in them =P)

[Jorge Lucángeli Obes]

I've been giving some thought to Anthony's idea:

http://kvm.qumranet.com/kvmwiki/Specs/StoringCommandLineInImage

However, maybe I'm just too much on vacations, but I don't seem to
come up with a nice way of doing this. Everything keeps coming back to
creating a new 'container' image format and then implementing block
layer functions that only add the number of sectors occupied by the
command-line to the read and write calls made by QEMU, and then just
relay those calls to the image-specific functions. That doesn't sound
very efficient.

The '#!' trick works nice with scripts, but I don't see it playing
very well with images. ¿Comments? ¿Pointers?

Cheers,
Jorge

Re: [Qemu-devel] Making qemu images executable (and store command line arguments in them =P)

[Mark Williamson]

> I've been giving some thought to Anthony's idea:
>
> http://kvm.qumranet.com/kvmwiki/Specs/StoringCommandLineInImage
>
> However, maybe I'm just too much on vacations, but I don't seem to
> come up with a nice way of doing this. Everything keeps coming back to
> creating a new 'container' image format and then implementing block
> layer functions that only add the number of sectors occupied by the
> command-line to the read and write calls made by QEMU, and then just
> relay those calls to the image-specific functions. That doesn't sound
> very efficient.

It's not necessarily that pretty, but I wouldn't have thought that adding a 
simple offset to block operations will have a measurable performance impact 
(given the latencies involved in block accesses anyhow, and the amount of 
data transferred each time).

> The '#!' trick works nice with scripts, but I don't see it playing
> very well with images. ¿Comments? ¿Pointers?

Well, it's not really necessary, but it would be darn cool :-)  Another cool 
(but admittedly twisted - get the brain soap ready!) thing to do would be to 
statically link a qemu, and then include a virtual machine config and disks 
in a section of the elf file (inspired by glick: 
http://blogs.gnome.org/alexl/2007/08/07/experiments-with-runtime-less-app-bundles/).  
So then you'd have an "executable" VM image which doesn't need a Qemu runtime 
to be available.  There are various variations you could do on this basic 
premise in order to make the file you carry around less terrifyingly huge!

Anyhow, enough of my random ideas...  I was thinking about container formats.

I've missed some of the discussion, but wouldn't tar be an obvious choice?  It 
can expand easily out to a directory hierarchy containing config file and 
multiple virtual disk files, there are standard tools that can manipulate it 
and standard libraries that can be used by Qemu in order to get at the 
contents.  Only problem I see with this approach is that sparse file handling 
might get a bit strange (using real sparse files vs using tar's 
represesntation of sparse files vs compatibility with tars that don't support 
them!).

Cheers,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

Re: [kvm-devel] [Qemu-devel] Making qemu images executable (and store command line arguments in them =P)

[Avi Kivity]

Mark Williamson wrote:
>> I've been giving some thought to Anthony's idea:
>>
>> http://kvm.qumranet.com/kvmwiki/Specs/StoringCommandLineInImage
>>
>> However, maybe I'm just too much on vacations, but I don't seem to
>> come up with a nice way of doing this. Everything keeps coming back to
>> creating a new 'container' image format and then implementing block
>> layer functions that only add the number of sectors occupied by the
>> command-line to the read and write calls made by QEMU, and then just
>> relay those calls to the image-specific functions. That doesn't sound
>> very efficient.
>>     
>
> It's not necessarily that pretty, but I wouldn't have thought that adding a 
> simple offset to block operations will have a measurable performance impact 
> (given the latencies involved in block accesses anyhow, and the amount of 
> data transferred each time).
>
>   

Right.

>> The '#!' trick works nice with scripts, but I don't see it playing
>> very well with images. ¿Comments? ¿Pointers?
>>     
>
> Well, it's not really necessary, but it would be darn cool :-)  Another cool 
> (but admittedly twisted - get the brain soap ready!) thing to do would be to 
> statically link a qemu, and then include a virtual machine config and disks 
> in a section of the elf file (inspired by glick: 
> http://blogs.gnome.org/alexl/2007/08/07/experiments-with-runtime-less-app-bundles/).  
> So then you'd have an "executable" VM image which doesn't need a Qemu runtime 
> to be available.  There are various variations you could do on this basic 
> premise in order to make the file you carry around less terrifyingly huge!
>   

That would make the VM not transportable (think moving from an x86_64
host to an i386 host) and would tie it to a particular version of kvm
userspace.

> Anyhow, enough of my random ideas...  I was thinking about container formats.
>
> I've missed some of the discussion, but wouldn't tar be an obvious choice?  It 
> can expand easily out to a directory hierarchy containing config file and 
> multiple virtual disk files, there are standard tools that can manipulate it 
> and standard libraries that can be used by Qemu in order to get at the 
> contents.  Only problem I see with this approach is that sparse file handling 
> might get a bit strange (using real sparse files vs using tar's 
> represesntation of sparse files vs compatibility with tars that don't support 
> them!).
>   

Also it can't be used in-place, like Anthony's header or the
metadata-in-snapshot idea.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Re: [Qemu-devel] Making qemu images executable (and store command line arguments in them =P)

[Andreas Färber]

Am 16.08.2007 um 01:52 schrieb Mark Williamson:

> I've missed some of the discussion, but wouldn't tar be an obvious  
> choice?  It
> can expand easily out to a directory hierarchy containing config  
> file and
> multiple virtual disk files, there are standard tools that can  
> manipulate it
> and standard libraries that can be used by Qemu in order to get at the
> contents.  Only problem I see with this approach is that sparse  
> file handling
> might get a bit strange (using real sparse files vs using tar's
> represesntation of sparse files vs compatibility with tars that  
> don't support
> them!).

Q uses the extension .qvm on an ordinary directory containing an XML  
configuration file and the image(s), so that double-clicking the  
directory opens the guest with Q. Can GNOME/KDE only do that for  
files so that an archive is necessary?

Andreas

[Qemu-devel] Re: [kvm-devel] Making qemu images executable (and store command line arguments in them =P)

[H. Peter Anvin]

Jorge Lucángeli Obes wrote:
> 
> The '#!' trick works nice with scripts, but I don't see it playing
> very well with images. ¿Comments? ¿Pointers?
> 

Well, you can make it work with a header (you just have to pad it out to
a fixed length or use a variable-offset format), but binfmt_misc might
be a better choice.

	-hpa

Re: [Qemu-devel] Re: [kvm-devel] Making qemu images executable (and store command line arguments in them =P)

[Avi Kivity]

H. Peter Anvin wrote:
> Jorge Lucángeli Obes wrote:
>   
>> The '#!' trick works nice with scripts, but I don't see it playing
>> very well with images. ¿Comments? ¿Pointers?
>>
>>     
>
> Well, you can make it work with a header (you just have to pad it out to
> a fixed length or use a variable-offset format), but binfmt_misc might
> be a better choice.
>
>   

binfmt_misc is Linux specific.  And while kvm is a Linux thing now, qemu
isn't.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Re: [Qemu-devel] Re: [kvm-devel] Making qemu images executable (and store command line arguments in them =P)

[Luke -Jr]

On Thursday 16 August 2007, H. Peter Anvin wrote:
> Jorge Lucángeli Obes wrote:
> > The '#!' trick works nice with scripts, but I don't see it playing
> > very well with images. ¿Comments? ¿Pointers?
>
> Well, you can make it work with a header (you just have to pad it out to
> a fixed length or use a variable-offset format), but binfmt_misc might
> be a better choice.

Can't hurt to support both, can it? ;)

Re: [Qemu-devel] Making qemu images executable (and store command line arguments in them =P)

[Ben Taylor]

---- "Jorge Lucángeli Obes" <t4m5yn@gmail.com> wrote: 
> I've been giving some thought to Anthony's idea:
> 
> http://kvm.qumranet.com/kvmwiki/Specs/StoringCommandLineInImage
> 
> However, maybe I'm just too much on vacations, but I don't seem to
> come up with a nice way of doing this. Everything keeps coming back to
> creating a new 'container' image format and then implementing block
> layer functions that only add the number of sectors occupied by the
> command-line to the read and write calls made by QEMU, and then just
> relay those calls to the image-specific functions. That doesn't sound
> very efficient.

No, and it fundamentally breaks using a real disk with QEMU.

> The '#!' trick works nice with scripts, but I don't see it playing
> very well with images. ¿Comments? ¿Pointers?

Personally, I'm not sure why we wouldn't just write out the command line
data to a file tied to the primary image file, with some kind of time stamp
to correlate the data from the command line and the last updated time
of the primary image file.  It's intuitive, and doesn't require a bucket of
programming to make work.  The down side is if qemu crashes, the
time stamp between the parameter file and the image file may indicate
the potential for "difference", but this can just be a notice (just as snapshots
used to do with the image files in 0.7.x)

The only hard part  here is when a real physical disk is used with
QEMU since it's harder to make sure the name is valid.

Thoughts?

Ben

Re: [Qemu-devel] Making qemu images executable (and store command line arguments in them =P)

[Dan Shearer]

On Thu, Aug 16, 2007 at 01:05:41PM -0400, Ben Taylor wrote:

> Personally, I'm not sure why we wouldn't just write out the command line
> data to a file tied to the primary image file

What about a compatible change to the file format: the last 1024 bytes
are either undefined, or, if a certain marker is present, an executable
commandline? (I was going to say one standard sector, then I thought of
how quickly i18n filenames add up.)

This is metadata, I have a feeling that if we think about it there will
be quite a lot of other metadata we'd like to store with the image.

> with some kind of time stamp to correlate the data from the command
> line and the last updated time of the primary image file.
 
If there have to be separate files I'd prefer a weak hash to timestamps
to avoid the common problems with unexpected drift and no way to
calculate which changed and why. 

-- 
Dan Shearer
dan@shearer.org